An enterprise knowledge base may change embedding models to improve technical-term retrieval, support another language or alter inference costs. The most dangerous failure may not raise an exception: a new model returns the same number of dimensions, the database accepts the query, and evidence quality quietly deteriorates. Migration must establish compatible query and document representations, acceptable retrieval quality and complete handling of source changes.
Matching dimensions establish shape, not semantic compatibility
An embedding model maps text to vectors, and retrieval compares their relationships. Sentence Transformers describes queries and corpus entries as occupying the same vector space and distinguishes query and document encoding.[1] Two models producing 768 dimensions need not assign the same meanings to their coordinates. Unless an explicit cross-encoder compatibility mechanism has been verified, use a matched query and document encoder pair.
A version manifest should record more than a model name: model revision, query and document prompts, preprocessing, truncation length, dimensions, normalization, distance metric, chunking rules and index settings. Changes affecting representation warrant a new retrieval configuration. When a hosted service cannot pin revisions, record its version commitments and regression requirements rather than promising complete reproducibility.
For the first comparison, hold source text, chunks, access filtering, candidate count and reranking constant while changing the embedding path. Simultaneously changing chunking prevents attributing gains to the model. Later experiments may compare optimized system bundles, but should be described accordingly. Apply the selected model's query and document instructions separately; a generic encoding call is not universally appropriate.
Choose isolation and preserve a recoverable configuration
Qdrant's migration documentation describes parallel collections and named vectors, with deployment prerequisites and restrictions for updates and deletes.[2] This article uses parallel collections to make the isolation boundary explicit. That choice is not mandatory for every database. Named-vector support must be checked against the deployed version, not assumed from current documentation.
Keep the serving collection A and build B by encoding the same authoritative source text again. Preserve document and chunk identifiers, source revisions, access metadata and processing receipts. Do not infer new embeddings from old vectors or overwrite an undifferentiated field incrementally while users search mixed spaces. If original text is unrecoverable, resolve provenance and recovery first.
Define an immutable application retrieval bundle containing the query encoder, target collection or vector field, distance function, thresholds, reranking settings and cache namespace. Cut over the bundle. A database alias change does not update the application's encoder, and a model-setting change does not re-embed stored documents. Bind each request to one bundle at entry rather than rereading independently mutable components during execution.
Parallel collections increase storage, backfill and synchronization work. Named vectors may share metadata but require precise field and update management. If a maintenance window is acceptable, pausing writes and rebuilding from a consistent baseline can be simpler than prolonged dual writing. Choose according to source-change rate, allowed interruption and recovery needs, not a zero-downtime label.
Backfill and live changes need one authoritative order
Establish a source snapshot and its corresponding change position, then backfill historical chunks while retaining subsequent changes. The new collection's consumer must apply reliable source revisions and prevent older backfill from overwriting newer content. If the destination cannot compare revisions atomically, coordinate through a single writing path or authoritative state store. A separate read followed by a write does not eliminate races.
Synchronize deletion, access revocation and text updates as well as inserts. Suppose snapshot revision 4 of document D9 remains in the backfill queue when revision 5 deletes it. A late revision 4 must not resurrect the document. Partial updates also need complete encoding input consistent with the revision, avoiding new metadata paired with vectors from old text. Validate write completion and retrieval visibility separately.
Our discussion of RAG incremental updates and deletion consistency covers document revisions and revocation gates. Model migration adds a second axis: representation version. Reconcile both sourceRevision and embeddingRevision. Equal vector counts do not establish completion, because one missing chunk and one obsolete duplicate can cancel out. Compare expected identifiers, source fingerprints and destination representation versions.
After backfill, catch up to a defined checkpoint and reconcile insert, update and deletion records before quality testing. Under continuous traffic, define acceptable synchronization lag and monitor the oldest unprocessed change. Without a provable source order or deletion handling, use a write pause or suspend migration. A background job ending is not proof of completeness.
Compare quality on the same evidence and recalibrate thresholds
Build a query set separate from training and tuning material, covering technical identifiers, abbreviations, Chinese and English, short questions, multi-chunk evidence, unanswerable requests and access boundaries. Record permitted relevant documents or chunks per query. Hide system identity during evidence review so that more fluent answer wording does not bias relevance judgments.
Sentence Transformers' retrieval evaluator accepts queries, a corpus and relevant-document mappings, and reports metrics including Recall@k, MRR and NDCG.[3] Select metrics for the task: Recall@k measures recovery of labeled relevant evidence, while MRR emphasizes the first relevant result's position. A multi-evidence answer is not complete merely because its first result is correct. Incomplete relevance judgments also limit interpretation.
Compare retrieval at the same candidate count first, then hold the generation model fixed to evaluate answers and citations. Report language, source and critical business slices separately so common queries do not conceal costly regressions. Evaluate false evidence selection and incorrect answers on unanswerable queries separately; do not force them into a recall denominator with no relevant documents.
Do not automatically reuse A's similarity threshold. Score distributions can change, and similarity is not a calibrated probability of business correctness. Select abstention thresholds on development data, freeze them, and compare false acceptance, missed useful evidence and abstention on independent tests. Once failed final tests guide further tuning, they have become development material and fresh independent acceptance is needed.
Separate representation quality from approximate-index error. On a manageable subset, compare exact retrieval with the production approximate path. If exact retrieval finds the evidence and approximate search misses it, investigate index and search settings before rejecting the model. Latency measurements should include query encoding, retrieval, reranking and cache misses across the full request path.
Constructed capacity and mixed-version examples
The following figures are illustrative, not measured performance. For one million chunks moving from 768 to 1,024 dimensions, float32 raw vectors occupy approximately 3.072 GB and 4.096 GB respectively, or 7.168 GB together using decimal gigabytes. This excludes graph indexes, text metadata, replicas, logs, temporary files and runtime memory. It is not a machine-sizing recommendation.
At a sustained 100 completed chunks per second, one million chunks have an ideal processing lower bound of 10,000 seconds, or about 2.78 hours. That ignores retries, rate limits, source reads and competition with live changes, so it is not a delivery promise. Measure sustainable throughput and reserve time for replay and reconciliation. If processing cannot keep up with incoming changes, finishing the initial backfill does not produce stable convergence.
Now suppose routing selects collection B while one old application instance still uses encoder A. Different dimensions may trigger an error; equal dimensions can produce silent degradation. Validate the request's bundle version against its destination. Reject a mismatch or restart under one complete bundle. Receiving ten results does not establish compatibility.
Caches can hide the problem. Keys should distinguish retrieval configuration, query and access scope, with evidence-revision dependencies retained. Identify requests served from old cached results during the trial. Testing only warm caches may measure the previous system's quality and latency. Revocation remains a separate mandatory check that a configuration snapshot must not override.
Give cutover, observation and rollback separate release gates
Before release, require complete reconciliation, acceptable critical-slice quality, validated thresholds, agreed synchronization lag and a tested recovery path. Shadow queries support comparison without executing additional business actions for users. Then shift a limited portion of real queries and observe errors, abstention, latency and evidence distributions rather than API success alone.
During the rollback window, retain A's encoder, dependencies and bundle, and continue applying required updates and revocations to its collection. A stale rollback target may restore invalid evidence. Recheck content validity and access controls before returning traffic; catch up or interrupt service if necessary. The old collection's existence alone is not a rollback guarantee.
Exercise interrupted backfill, late records following deletion, encoder–collection mismatch, regression in one language, cache version mixing and a failed synchronization consumer after cutover. Record each detection signal, blocking behavior and recovery evidence. Remove the old representation only after the observation window and stable new operation, following the retention policy. The deliverable is a configuration bundle, source reconciliation, quality comparison and cutover receipts—not merely a count of written vectors.