Google has outlined a way to upgrade vector embedding models in AlloyDB AI without taking production systems offline. The approach is aimed at teams running semantic search and other vector-based workloads on live databases.
It describes a migration path from one embedding model to another while continuing to serve user queries, using in-database model access, automated embedding generation, PostgreSQL functions and concurrent index creation.
This addresses a common production problem in vector search. When a new embedding model becomes available, existing data usually has to be re-embedded, indexes rebuilt and query logic switched in line with the stored vectors, because embeddings from different models cannot be compared directly.
According to the guidance, the process begins by adding a new vector column sized to match the output dimensions of the replacement model. In the example, a production table of product reviews moves from text-embedding-005, which produces 768-dimensional vectors, to gemini-embedding-001, which produces 3,072-dimensional vectors.
Auto-embeddings are then enabled on the new column so the database can backfill vectors in the background. AlloyDB AI can generate embeddings at up to 3,000 per second and track changes to source data as rows are inserted or updated.
Two refresh modes
The system supports two ways to keep vectors current. In transactional mode, embeddings are updated as soon as source data changes. This keeps vectors aligned with the underlying records, but adds write latency because inference runs inside each write transaction.
In manual mode, the database marks changed rows and lets operators refresh stale embeddings later in scheduled batches. This reduces pressure on write performance and is the mode used in the migration example.
Google also outlined operational trade-offs during backfill. Lowering the maximum number of parallel requests reduces the time concurrent writers may wait for a micro-batch to commit, but lengthens the overall backfill window.
In one benchmark on a 10,000-row table, reducing max_parallel_requests from 400 to 20 cut the longest concurrent writer wait from 5.1 seconds to 0.39 seconds, while increasing total backfill time from 5.2 seconds to 7.6 seconds.
Once the initial backfill is complete, changed rows can be caught up through a refresh procedure. The database also provides a progress view showing job state, rows processed and estimated time remaining.
Quality checks
Before switching traffic, Google recommends checking for missing embeddings and validating the new vectors against fresh model invocations. Teams should also test application behaviour, because retrieval patterns can change sharply when moving to a different model.
In Google's internal tests, the top-10 nearest-neighbour results from text-embedding-005 and gemini-embedding-001 overlapped by only 31%. That means teams may need to review assumptions built into ranking, filtering or evaluation pipelines before routing live production traffic to the new model.
The migration process also includes building a new ScaNN vector index on the replacement column. This is done concurrently to avoid blocking writes.
Google said its ScaNN index builds were up to 10 times faster than HNSW on open-source PostgreSQL. In tests on 100,000 rows, a 3,072-dimensional ScaNN index was built in about 50 seconds, while an HNSW index using halfvec took between seven and 13 minutes.
Cutover options
For the final switch, Google outlined two main patterns. One uses an atomic column rename and a query function, so applications do not need code changes when the live embedding column is swapped. The other uses PostgreSQL session variables and a set-returning function, allowing canary releases and traffic splitting between old and new models.
The first option supports blue-green deployment, but not per-query canary testing. The second lets teams direct a percentage of sessions to the new model and revert quickly by changing a session parameter.
There are limits. The atomic rename pattern does not work with transactional auto-embeddings because managed triggers refer to fixed column names. Transactional mode also brings a marked increase in write latency while both models are active.
Google's figures for single-row update operations showed a median write latency of about 37.5 milliseconds with text-embedding-005 alone, rising to roughly 191.0 milliseconds during migration when both models were active, before settling at about 172.0 milliseconds after cutover to gemini-embedding-001.
Storage pressure
Google also warned that storage use rises materially during migration because old and new vectors coexist. In its tests, text-embedding-005 used about 4.5KB per row, while gemini-embedding-001 used about 13.3KB per row.
On that basis, a table with 10 million rows would need about 133GB for the new vector column and about 6GB for the new ScaNN index, before allowing extra headroom for write-ahead logging and page updates.
After cutover, teams can remove the old embedding configuration, drop the legacy index and clear the retired vector column in batches before vacuuming and dropping it. In Google's 100,000-row tests, nulling out vectors and vacuuming before dropping the column reclaimed 381MB of TOAST space for reuse in later model upgrades.