When a developer searches for "authentication handler" in a codebase that calls it svc_auth_gate_mkv3, nothing comes back. The search isn't wrong. The model just doesn't know what the team calls things.
This is the quiet failure mode that generic embeddings produce at scale.
What Embeddings Are Actually Doing
An embedding is a compressed representation of meaning. Text goes in, a vector comes out. That vector sits somewhere in a high-dimensional space, positioned near other vectors with similar meaning. Search becomes a question of proximity: find what's closest to this query.
The quality of that representation depends entirely on what the model learned during training.
General-purpose models, the kind trained on Wikipedia articles, public GitHub repositories, and web text, learn a broad understanding of language and code. They know that "authentication" and "login" are related. They know that "middleware" and "interceptor" occupy similar conceptual territory. That's useful for a lot of things.
What they don't know is that in a particular fintech codebase, plaid_handler_v2 is the authentication entry point, or that evt_user_svc means something specific about event sourcing for user services. The model has never seen those tokens in a meaningful context. It treats them as noise.
This is not a bug. It's a consequence of how general representation learning works. The model approximates meaning for the broadest possible input distribution. Codebases are narrow distributions with high internal structure. The approximation breaks down at exactly the point where precision matters most.
Where Code Analysis Tools Break Down
A developer is six months into a mid-sized codebase. They need to find where rate limiting is implemented. They search "rate limiting" in whatever code analysis tool is in the stack, including MCP-based code analysis servers that have seen rapid adoption this year. The results come back with generic middleware classes and comments that mention rate limits conceptually. Nothing points to the module called req_throttle_svc, which is what actually enforces limits in production.
The developer finds it eventually, through grep or through asking someone who's been on the project longer. They file it away mentally and move on.
The chunking strategy behind the index makes this worse. Code indexed in fixed-size blocks splits related concepts across chunk boundaries. A function's name lands in one chunk. Its implementation lands in another. The search returns the name but not the behavior, or the behavior with no clear owner.
This happens a dozen times a day, across every developer on the team. The accumulated cost is invisible in any individual instance. Across a quarter, it's thousands of hours of context-switching overhead that nobody accounts for because there's no line item for "time spent not finding the right thing."
Tools built for code routing and query dispatch face the same ceiling. A code router is only as accurate as the representations it routes against. If the index doesn't understand that throttle and rate limit are synonyms in this codebase, the router sends the query somewhere else, and developer time fills the gap the model left.
Why Generic Embeddings Cap Out
The failure isn't about intelligence. It's about training distribution.
A model trained on public code learns that auth typically precedes middleware or service in identifiers. It learns that functions named validate usually handle input checking. Common patterns appear frequently in training data, so the model learns common patterns well.
Private codebases invert this. The things that matter most are the things the team invented: internal naming conventions, domain-specific abbreviations, proprietary abstractions, and patterns that appear nowhere in public training data. These tokens appear in the training corpus at random or not at all, so the model assigns them weak or arbitrary representations.
The vector for mkv3_session_pool sits nowhere useful in the embedding space. It's close to whatever the model guessed about mk, v3, session, and pool independently, which is approximately correct and usually not correct enough.
Retrieval augmented generation (RAG) systems built on these representations inherit the problem. The index is only as good as the embeddings. Code review tools that depend on accurate context retrieval, including ripple-style review systems that surface related context automatically, run into false negatives not because the code is missing but because the representation didn't learn the connection.
Reranking improves the ordering of retrieved results. It doesn't help when the relevant code never appeared in the first-stage results to begin with. The ceiling is set before reranking has anything to work with.
MCP clients and reference tools in the ref.tools ecosystem and strands-style MCP clients that aggregate context from multiple sources face the same constraint. The context engineering layer can be excellent. If the underlying search can't find the right code, the rest of the pipeline works on the wrong material.
Domain-Specific Representation
The solution is not a bigger general model. A larger model trained on more public code learns more public patterns. The private vocabulary problem remains.
What changes the result is training the embedding model on the actual codebase. Fine-tuned or purpose-trained models can learn that svc_auth_gate_mkv3 and "authentication handler" are semantically equivalent in this context, because they've seen them used interchangeably in comments, docstrings, commit messages, and code review notes. The model learns the internal language of the team.
This is representation learning at the domain level rather than the population level. The embedding space stops being a general approximation and starts being a precise map of a specific codebase.
Hybrid search, which pairs keyword matching with semantic retrieval, benefits directly from this. The keyword leg finds exact strings. The semantic leg finds conceptual neighbors. When the model doesn't understand the internal vocabulary, the semantic leg can't contribute much. Both legs need to work for hybrid search to outperform a basic grep.
Token efficiency follows from retrieval precision. When the first attempt surfaces the right code, the context window fills with relevant material. When it doesn't, the typical response is to widen the query, pull in more files, and hope the right one is in there somewhere. Tokens burn on near-misses. The context window fills with noise instead of signal.
The tradeoffs are real. Training requires data. A small codebase with minimal documentation gives the model less signal to learn from. Early in a project's life, the representations are thinner. They improve as the codebase grows and as the team generates more context through normal work.
There's also the question of staleness. Generic models are static snapshots. A codebase-specific model trained last quarter doesn't know about the refactor that renamed half the service layer. Continuous re-indexing addresses staleness in vector databases, but representation quality doesn't update without retraining. This is an active problem in the field, not a solved one.
Where This Is Going
Context engineering for developers has evolved from a curiosity into its own discipline. The frameworks exist now. Router coding and dispatch patterns have matured enough to be standard infrastructure in serious AI-assisted development setups. The bottleneck has shifted from orchestration to representation quality.
The next phase of this work is representation that adapts. Models that update incrementally as a codebase changes, without requiring full retraining. Methods for learning from implicit signals like which search results developers click, which files they open after a query, where they spend time after a failed search. The model watching the team work and updating its understanding accordingly.
The honest assessment of where the field sits today: orchestration is further ahead than representation. The tooling to route, retrieve, and present context is genuinely good. The tooling to ensure the underlying representations are accurate at the domain level is still catching up.
Long context models offer a partial workaround. Pull in more code, let the model sort through it. The lost in the middle problem limits how far that scales. Accuracy degrades for material positioned away from the prompt boundaries, which is exactly where a retrieval miss tends to land. Better representation is a cleaner fix than longer contexts.
Most teams using semantic search over code are using general embeddings and not measuring how often they miss. The misses don't produce errors. They produce silence, which is harder to notice.
That's worth understanding before deciding the retrieval layer is working.