Most teams spend weeks evaluating ChromaDB, Pinecone, and pgvector. They benchmark latency. They model cost curves at scale. They read migration guides and build proof-of-concepts and argue in architecture documents that nobody will look at in six months.
Then they pick one, deploy it, and discover the same problem they would have had with any of them.
The vector database itself is almost never the bottleneck.
---
What a Vector Store Actually Does
A vector database stores numerical representations of content, then finds which stored items are mathematically closest to a query. That is the entire job. In a retrieval augmented generation system, the database is one component in a longer pipeline. The distance calculation is fast. The infrastructure is straightforward. The hard part is the layers on either side of it.
Before the database: someone has to turn raw text, code, or documents into those numerical representations. The quality of those embeddings, the chunking strategy, the preprocessing decisions, the context window each chunk carries: these determine what the database has to work with. Give it garbage, it finds the nearest garbage.
After the database: someone has to decide what to do with the results. How many results to retrieve, within the token budget set for each request. Whether to apply reranking. How to merge them with the user's actual query before sending to a language model. Whether the retrieval result is even relevant to what the user meant. Irrelevant chunks inflate the token cost of every inference call, and in longer prompts the lost in the middle effect reduces how reliably models attend to content that is not near the start or end of the context.
The database sits in the middle, doing math. It does the math quite well regardless of which product you chose.
---
Where the Real Differences Land
That said, the three dominant options have genuine tradeoffs. They are just not the tradeoffs that show up in benchmark blog posts.
ChromaDB runs locally. No API keys, no network latency, no per-query cost. For a code index MCP or any local developer tooling, this matters. If you are building something that runs on a developer's machine, or on a node in a self-hosted cluster, spinning up a managed cloud service adds dependencies that will eventually cause problems. ChromaDB is also where most experimental tooling starts. A developer downloads a context engineering workflow locally, tests retrieval behavior against an actual codebase, and only then decides whether the complexity of managed infrastructure is worth adding.
Pinecone is managed infrastructure that someone else operates. At high query volume, managed infrastructure stops being a luxury and becomes the only practical option. But the cost structure at moderate scale is punishing, and the vendor relationship introduces a dependency that is easy to underestimate until you need to migrate.
pgvector is a PostgreSQL extension. If the rest of the application data lives in Postgres, adding vectors to the same database is often the right call. No new operational surface, no new backup strategy, no new monitoring setup. The query performance at very high scale is not as strong as purpose-built options, but for most applications, very high scale is not the problem being solved today.
One area where the choice of database has a concrete effect: hybrid search. Combining vector similarity with BM25 keyword matching improves recall on code queries where exact symbol names, function signatures, or error strings matter. Pure semantic search misses them. Not every vector database supports hybrid search natively, and the implementations that exist vary enough that switching databases mid-project requires retesting retrieval behavior end to end.
---
The Real Cost Lives in the Layer Above
A code router that decides which context to retrieve for a given developer query depends on retrieval quality, not retrieval speed. The vector database returns results in single-digit milliseconds across all three options at typical workloads. What takes time, and what determines whether the system is actually useful, is whether the retrieved chunks are the right ones.
That problem is a chunking problem. An embedding model problem. A query understanding problem.
Teams building ref.tools MCP integrations or working with ref MCP tooling learn this quickly. The infrastructure question resolves fast. The harder question is what to put in the index and how to slice it. A ripple code review surface that retrieves the wrong file, or retrieves the right file but the wrong section of it, produces suggestions that range from unhelpful to actively misleading. The vector database did its job. The indexing layer did not.
Strands MCP client users running semantic code search hit the same wall. The search returns results. The results are the wrong results. The latency is fine. The relevance is not. Switching from ChromaDB to Pinecone does not fix relevance.
---
What Vendors Will Not Tell You
Every vector database vendor will show you a benchmark that makes their product look optimal for your use case. They will show you the scenario where their architecture shines. They will not show you the scenario where a simple in-process library with a flat file would have worked equally well at one-tenth the operational complexity.
The honest version: most retrieval problems at developer-tool scale would run adequately on any of the three options. The applications that genuinely need Pinecone's architecture are not the ones debating whether to use Pinecone. They are already past that conversation.
For the majority of code analysis MCP implementations, local development tooling, and single-tenant retrieval systems, the operational cost of a managed vector database exceeds the technical benefit. The in-process option is worth taking seriously for longer than most teams do.
---
Where the Industry Is Actually Going
The vector database conversation is collapsing into the embedding model conversation, which is where it belongs.
The retrieval layer is commoditizing. Several well-maintained open-source options now offer comparable performance at zero marginal cost per query. Differentiation is moving to the model producing the embeddings, the quality of the indexing pipeline, and the intelligence of the query layer that decides what to retrieve.
For codebases specifically, the tooling is moving toward domain-aware indexing. Generic embeddings treat function names like prose. Code has structure: call graphs, import chains, symbol tables, comment-to-implementation relationships that carry meaning the text alone does not. A code-aware embedding model that understands these relationships will outperform a generic model on the same vector database by a wider margin than switching databases would produce.
The teams getting retrieval right are spending their time there. The vector database question has an answer. Most of the time, that answer is whichever one you can operate without adding a new system to your stack.
---
The Honest Assessment
Retrieval quality determines whether AI-assisted developer tools are useful or theatrical. It is not a database problem. It is a representation problem.
The teams that figure this out stop benchmarking databases and start evaluating whether their chunks carry enough context to be actionable, whether their embedding model understands the domain it is indexing, and whether their retrieval logic reflects how developers actually search. Which is rarely how documentation is written.
The vector database gets the last few milliseconds right every time. The question is whether everything before it earned those milliseconds.
Most teams never ask it.