Benchmarks measure the one thing that rarely fails
Public ANN benchmarks push a fixed corpus of uniform vectors through unfiltered top-k queries and report recall against queries per second. That is genuinely useful if you write index code. It is close to useless for picking what to run, because unfiltered recall at 500K records is the part that already works. The mainstream engines sit within a couple of points of each other there, and a reranker in front of the model erases whatever difference is left.
What you do notice is everything the benchmark left out. A filter that matches two percent of the corpus comes back with four results instead of twenty. A part number goes missing, because dense vectors do not do exact tokens. The p99 climbs to several times the median the moment three people query at once, and then one Tuesday a re-embedding run takes the index down for a night nobody put in the calendar.
One separation is worth making before any of this. Answer quality is mostly upstream of the database, in chunking, ranking and grounding, and we took that apart in what actually breaks in a RAG knowledge base at 500K records. This article is about the layer underneath: which store the vectors live in, and what that choice does to you in month six.
Filtered search is where a stack quietly loses results
Every real query carries a filter. Tenant, department, document type, effective date, language, permission scope. In a demo the filter is an afterthought, and in production it is the largest single source of two complaints that sound unrelated and are not: it returned nothing, and it returned another team's document.
Both naive implementations fail, in opposite directions. Post-filtering runs the vector search first and drops whatever does not match, which is why Weaviate's docs on filtering point out that you cannot predict how many elements the search will contain, and a restrictive filter can leave you with an empty result. Pre-filtering looks like the obvious fix until you see what it does to the graph: Qdrant's guide to filtering in vector search states it plainly, that pre-filtering should not be used over large datasets because it breaks too many links in the HNSW graph and accuracy drops.
So every engine built something in between, and those somethings are where the real differences live.
| Engine | How filtered search works | What that means in practice |
|---|---|---|
| Qdrant | A filterable index adds extra links between points that match a filter; below a full scan threshold, 10 KB by default, it drops to payload-index search instead | Narrow filters stay accurate, but the fields you filter on need payload indexes, ideally created before ingest |
| Weaviate | An inverted index builds an allow-list for the HNSW search; ACORN is the default filter strategy since v1.34, and a flat-search cut-off at roughly 15% of the dataset switches to brute force | Good defaults out of the box, and the cut-off is a knob you will eventually have an opinion about |
| Pinecone | Filtering happens inside the managed search; 40 KB of filterable metadata per record, and $in / $nin capped at 10,000 values | Fine until a permission filter needs more than 10,000 ids, at which point you redesign the schema around namespaces |
| pgvector | With an approximate index, filtering is applied after the index is scanned; iterative index scans since 0.8.0 keep scanning until enough matching rows are found | Filters, joins and vectors in one SQL query against real data, in exchange for tuning scan limits yourself |
| Chroma | where on metadata plus where_document with $contains on the text | Enough for straightforward filters; the memory ceiling further down usually bites first |
Whatever the engine does internally, the cheapest accuracy is still narrowing the problem before the search runs. In the 500K-record base we operate, retrieval resolves the entity in the question against real tables first, then searches semantically with those canonical IDs applied as metadata constraints. The index gets a far smaller problem, and the filter is exact because it came out of a database lookup rather than a similarity score. That trick is portable to any engine in the table, which is part of why we rate it above the engine choice itself.
Dense vectors cannot find a part number
Ask a pure embedding index for contract SLA-2024-118 and it will hand you three contracts that read like it. The failure is structural rather than a tuning problem: identifiers carry meaning in their characters, not in their semantics, and that is what lexical search is for. Any base where people search by SKU, error code, invoice number or policy ID needs BM25 or sparse vectors sitting next to the embeddings, and the engines are not equal here.
Weaviate treats it as a first-class query type. Its hybrid search exposes an alpha parameter where 0 is pure keyword and 1 is pure vector, with Relative Score Fusion as the default fusion method since v1.24. That is one call and one number to tune, which is about as low-friction as this gets.
Qdrant assembles it from parts, and the parts are good. Hybrid queries combine sparse and dense vectors through a prefetch stage, fused with Reciprocal Rank Fusion or, since v1.11, Distribution-Based Score Fusion, which normalises the score distributions before combining rather than working off positions. Prefetches nest, so a multi-stage retrieve-then-rerank pipeline is expressible in a single request.
Pinecone supports sparse vectors too, with a documented ceiling of 2,048 non-zero values per sparse vector in its database limits. pgvector has no lexical search of its own, and does not need one, because it lives in Postgres: tsvector full-text search is already in the same database, and you fuse the two rankings in SQL. That is more code than Weaviate's alpha, and it is also the version you can debug with EXPLAIN. Chroma's where_document substring match is a filter, not a ranking, so it is not a substitute.
For what it is worth, the base we run does dense and BM25 together, then reranks the merged set. The reranker is the expensive part of that sentence, and it is still the cheapest fix we know for results that are on topic and useless.
The invoice is mostly a RAM invoice
HNSW is a graph you walk, and walking it from disk is not a thing you want to do. Every self-hosted engine therefore has the same underlying economics, and Chroma documents them the most bluntly of the five. Its single-node performance guide says the index must reside in system RAM, that anything under 2 GB is not recommended, and that once a collection exceeds available memory the operating system starts swapping and the system quickly becomes unusable. It even gives you the sizing rule: maximum collection size in millions equals available RAM in gigabytes times 0.245. Queries parallelise up to the number of vCPUs and then queue, which is where the p99 goes.
Weaviate states the same relationship as a planning rule in its resource planning docs: memory usage is roughly two times the memory footprint of all vectors. That doubling is the part teams forget when they size a box off the raw embedding volume.
Compression is the lever that moves this, and Qdrant documents the trade honestly in its quantization guide: scalar quantization gives 4x with an error usually under 1%, binary quantization reaches up to 32x and up to a 40x speedup but wants high-dimensional centred data plus a rescoring pass, and product quantization goes to 64x while being slower, because the distance maths loses its SIMD path. Weaviate points at product quantization for the same reason. None of this is free accuracy, and all of it is cheaper than another 32 GB of RAM.
Managed changes the shape of the cost rather than the amount. On Pinecone's published pricing the Standard plan starts at a $50 monthly minimum, storage runs $0.33 per GB per month, writes are $4 to $4.50 per million write units and reads are $16 to $18 per million read units depending on cloud and region, with egress at $0.10 per GB. You never provision memory. You do get a bill that grows with query volume, which is the opposite exposure from a self-hosted box: one scales with how much you store, the other with how much you ask. A support assistant answering 50,000 questions a month and an internal base answering 500 have very different answers here, and it is the same calculation we walk through for what an AI system costs to build and run.
Plan for the day you change the embedding model
Every corpus gets re-embedded eventually. A better model ships, the dimension count changes, chunking gets fixed, a language is added. That is not a migration you run with an ALTER, it is a full rebuild of the index, and the engines differ more on that day than on any recall chart.
pgvector is the most transparent about the pain, because the pain is yours. Its README is direct that HNSW gives better query performance than IVFFlat but has slower build times and uses more memory, recommends setting maintenance_work_mem to something like 8 GB, and warns with an explicit notice when the graph no longer fits, wording it as no longer fitting into maintenance_work_mem after 100,000 tuples. Dimension caps matter here too: the vector type indexes up to 2,000 dimensions and halfvec up to 4,000, so a 3,072-dimension model forces a decision about half precision before you start rather than after.
Managed services move the work but bill it. Rebuilding on Pinecone means re-upserting the corpus, which is write units at $4 to $4.50 per million on Standard, plus $0.25 per GB if you go through import. That is cheap for one rebuild, and worth knowing before you have done four.
The pattern that survives contact with production is the same everywhere. Build the new index beside the old one under a different collection or namespace, dual-write during the overlap, run your evaluation set against both, then flip an alias. What differs between engines is how cheap that overlap is: for a few hours you are paying for two copies of everything, which is exactly the moment the RAM arithmetic above stops being theoretical.
What we would actually run: Pinecone, Weaviate, Qdrant, Chroma or pgvector
There is no winner, which is the unsatisfying but correct answer. There are workloads, and each of these engines is the right call for one of them. The column that matters most in the table below is the last one, because the trade you accept is the thing you will live with.
| Workload | What we would run | Why | What you are accepting |
|---|---|---|---|
| Under a few million chunks, data already in Postgres | pgvector | One database, one backup, one access model; filters and joins live in the same query as the vectors | Tuning scan limits, and assembling lexical search yourself |
| Heavy metadata filtering, self-hosted, data staying in your own infrastructure | Qdrant | The filterable index and the quantization options are built for narrow filters over large corpora | You operate it: payload indexes, snapshots, upgrades |
| Hybrid search wanted without building it | Weaviate | Hybrid with alpha and score fusion is a first-class query, and ACORN handles filters by default | More concepts to learn, and memory planning still lands on you |
| Small team, no ops budget, spiky traffic | Pinecone | Nothing to provision or keep alive; capacity is somebody else's problem | Per-read costs that grow with usage, and limits you cannot change |
| Prototype, laptop, single machine | Chroma | The shortest path from notebook to working retrieval | A memory ceiling you will meet, and a migration afterwards |
Our bias is on the record and worth weighing: we default to pgvector, and the 500K-record system runs on PostgreSQL with pgvector alongside Python, LangChain and FastAPI. The database has not been the constraint at that size. Retrieval quality has, every time. That is the general lesson rather than a recommendation to copy our stack: a second service is real operational weight, so it should be bought with a specific constraint, not with a feeling that the serious option must be the specialised one. The same argument, at the level of whole systems rather than indexes, is the build versus buy question.
Five tests to run before you commit to anything
None of this needs a vendor call or a proof of concept quarter. Each test below targets one of the failure modes above, and a bad result points at a fix instead of a mood. Run them on your own corpus, not on a sample, because the sample is what makes every engine look identical.
Load ten times what you have. Duplicate your corpus with perturbed vectors if you have to. You are not measuring recall, you are watching what memory does and where build time goes non-linear. If the index no longer fits the box at 10x, you just learned your real planning horizon.
Query with your narrowest real filter. Pick the one that matches under one percent of the corpus, a single tenant on a single document type in a date range, then check how many results come back and whether the right one is among them. Empty or short result sets here are the single most common production surprise.
Search for something by its identifier. Take a part number, contract reference or policy ID that appears exactly once and ask for it. Dense-only setups will return topical neighbours with high confidence scores, which is how this failure hides for months.
Measure p95 and p99 with concurrent load, never the median. Chroma's own docs describe queries parallelising up to the vCPU count and then queueing into linear latency growth, and every engine has a version of that curve. The median under one client tells you nothing about the Monday morning when everyone opens the assistant at the same time.
Rehearse the reindex. Rebuild the index from scratch with a different embedding model and time it end to end, including the double-storage window. Do this while nothing depends on it, because the alternative is doing it for the first time under pressure with a deprecation deadline.
Two afternoons of that will tell you more than any comparison article, including this one, because it runs against your filters and your identifiers. If the results point at retrieval quality rather than the store, the write-up on chunking and ranking at scale is the more useful next read, and if the answers need live system state rather than documents, that is a different problem again: connecting an agent to your data through tools instead of an index.
We build and operate these systems, so weigh that accordingly. The five tests cost you nothing and no vendor. If you would rather have a second pair of eyes on which constraint is going to bite a specific base first, a short audit is a low-key way to get one.
FAQ
Which is the best vector database in 2026?
There is no single answer, and any article that gives you one is selling something. The engines are close enough on unfiltered recall that the decision is made elsewhere: how filtered search behaves under narrow filters, whether lexical search is built in or assembled, whether cost grows with what you store or with how often you query, and how much operational work you can absorb. Pick against your workload, and start from what you already run rather than from a feature matrix.
At what point does pgvector stop being enough?
Later than most people assume. The constraints that eventually force a move are specific rather than general: sustained high query concurrency where Postgres connection handling becomes the bottleneck, filters so narrow that post-scan filtering keeps missing results even with iterative index scans, embedding dimensions above what the index types support, or a corpus large enough that build time and memory make routine reindexing impractical. If none of those are true today, a second database is infrastructure you operate for no answer-quality gain.
Is Chroma good enough for production RAG?
It depends entirely on corpus size, and Chroma (ChromaDB) is unusually honest about the limit. Its own single-node performance documentation states the HNSW index must live in system RAM, that under 2 GB of RAM is not recommended, and that once a collection exceeds available memory the system starts swapping and becomes unusable rather than merely slow. Their sizing rule puts maximum collection size in millions at available RAM in gigabytes times 0.245. Below that ceiling it works; the risk is that growth crosses it without warning.
Does switching vector database mean re-embedding everything?
Not necessarily. Embeddings are just float arrays, so if you keep the source chunks and the vectors, most migrations are an export and a bulk load. What actually gets re-done is the index build, plus any engine-specific structures like payload indexes or sparse representations. The migration that does force full re-embedding is a change of embedding model, which is a different event and worth keeping separate in your planning, since the two are often bundled into one scary project when they do not have to be.
- 10 August 2026Published.