Building a RepoMap for code retrieval

When an agent works on a real repository, the naive approaches both fail. Sending every file blows the context window and the budget. Sending the few files the user named misses the functions elsewhere that the change actually depends on. What the model needs is situational awareness: what exists, where, and how it connects, at a fraction of the tokens of the source itself. That is what a RepoMap provides: a ranked outline of the most relevant symbols in the codebase, rendered with just enough detail to reason about, sized to fit a token budget.

What goes in the map

A RepoMap is not source code. It is a list of definitions, functions, classes, methods, exported constants, each shown with its signature and location, and often a single line of body or docstring. A function that is 40 lines long appears in the map as one or two lines. That compression is the whole point: you can fit hundreds of relevant symbols in the space five full files would take.

Step 1: Parse the repo into symbols

Use a parser that understands the language rather than regexes. tree-sitter is the standard choice: it has grammars for every common language and produces a concrete syntax tree you can query. From each file, extract two things: the definitions, this file defines parseConfig, and the references, this file calls parseConfig. Tree-sitter's tags queries give you exactly these two categories out of the box.

Step 2: Build a graph

Model the repo as a directed graph. Files are nodes. A reference is an edge: if server.py calls a symbol defined in config.py, draw an edge from server.py to config.py. Weight edges by how many times the symbol is referenced. What you end up with is a dependency graph where heavily-used, foundational code is highly connected and leaf utilities sit at the edges.

Step 3: Rank with personalized PageRank

Now rank the nodes. Plain PageRank surfaces the most-referenced code, which is useful but generic: it will always float the same core utilities regardless of the task. The improvement is personalized PageRank: bias the random-walk restart distribution toward the files that matter for this task, the ones the user mentioned, the files currently open, the files recently edited, and any symbols named in the chat. The result is a ranking that says given what we are working on, here is the code most likely to be relevant, which is exactly the question retrieval should answer.

Plain PageRank tells you what the codebase thinks is important. Personalized PageRank tells you what this task makes important. Only the second one is retrieval.

Step 4: Render to a token budget

Walk the ranked symbols from the top, rendering each, signature, location, and optionally a line of body, and adding it to the map until you hit your budget, say 1,000 to 2,000 tokens for the map. Because you are spending tokens in priority order, a tighter budget simply drops the least relevant symbols first. Tokenize as you go rather than estimating; the difference between fits and truncated mid-symbol matters.

Tuning and pitfalls

Refresh on edits. A stale map that describes code the agent already changed is worse than no map. Recompute after file writes, or at minimum invalidate the changed files.
Down-weight tests and generated code. Test files reference everything, so they can dominate a naive graph. Either exclude them or weight their edges down unless the task is about tests.
Cache the parse. Parsing a large repo on every turn is slow. Cache the syntax trees and only re-parse changed files; the ranking is cheap by comparison.
The map is a pointer, not the answer. The map helps the agent decide which files to open in full. Pair it with a tool that fetches complete symbols on demand.