Operating Vector Databases: HA, Reindexing & Backups
Running pgvector, Milvus, Qdrant, Weaviate and OpenSearch in production: shards and replicas, memory sizing with headroom, index builds and online reindexing behind aliases, node loss, snapshots and restore drills, upgrades, recall as an SLO, multi-tenancy and cost.
Overview
The job
Retrieval sits on the request path of most LLM products, so the vector database behind it is a tier-one stateful system. It has every database problem (replicas, backups, upgrades, capacity) plus three of its own: indexes live in RAM and take hours to build, quality can drop while every request still returns 200, and a new embedding model means rebuilding everything.
A concrete case. A support-software company answers customer questions from 100 million help-center chunks. Search runs on a 3-node Qdrant cluster on Kubernetes, each collection with a replication factor of 2 across three zones; a smaller product catalog lives in pgvector on the company's Postgres. This quarter the ML team ships a new embedding model, so the platform team must reindex everything without downtime, keep a rollback path, and prove that last night's snapshot actually restores.
How embeddings, HNSW parameters and quantization work is in Embeddings & Vector Search; the ingestion and retrieval pipeline is in RAG in Production: Ingestion to Answer. This topic is the operator's view.
Interview questions
- Plan the production cluster for 200 million 768-dimension vectors at 2,000 queries per second with high availability. How many nodes, shards and replicas?
- Switch production search to a new embedding model with no downtime and a rollback path. Walk through the operation.
- Design backup and restore for the vector tier. What RPO and RTO do you promise, and do you back up vectors at all?
- Search results got worse but every dashboard is green. How do you make retrieval quality an SLO and catch this next time?
- Design multi-tenant vector search for 10,000 customers, from tiny to very large. How do you isolate them and handle deletion requests?
Sign up free to read the full topic
More MLOps & LLMOps topics
- Model Evaluation — Offline and Online
- Model Packaging and Formats
- Model Registry
- Model Serving
- Model Supply Chain: Signing, Provenance & Safe Formats
- Online and Continual Learning
- Ray on Kubernetes & Dynamic Resource Allocation for GPUs
- Real-World MLOps Case Studies
- Reproducible ML Pipelines
- Responsible AI and Explainability
- Terraform for GPU Clusters on EKS, AKS & GKE
Start free on Cariara · Coding problems · System design studio