RAG for Codebases: Retrieval Quality Comes First

Retrieval-Augmented Generation (RAG) is frequently demonstrated on prose documents and Markdown documentation, where fixed-length character or token windows (e.g., 500 tokens with 50-token overlap) yield coherent semantic chunks. When applied to complex software repositories, this naive approach fails immediately. A rigid token boundary cleaves a TypeScript class between its decorator and constructor, isolates a method from its enclosing interface, and strips import declarations required to deduce symbol types.

In building gitloom, my codebase analysis SaaS, I shifted from token-based splitting to Abstract Syntax Tree (AST) parsing using Tree-sitter. Source code is a structured semantic graph, and chunk boundaries must honor that syntax tree:

def chunk_code_file(source_code: str, language: str) -> list[CodeChunk]:
    tree = parser.parse(bytes(source_code, "utf8"))
    chunks = []
    
    for node in tree.root_node.children:
        if node.type in ("class_declaration", "function_declaration", "method_definition"):
            chunk_text = source_code[node.start_byte : node.end_byte]
            context_header = extract_file_context(source_code, node)
            
            chunks.append(CodeChunk(
                content=f"{context_header}\n{chunk_text}",
                symbol_name=node.child_by_field_name("name").text.decode(),
                line_range=(node.start_point[0], node.end_point[0]),
                node_type=node.type
            ))
            
    return chunks

By isolating complete semantic declarations (classes, functions, interface definitions) and prepending parent namespace headers and module imports, each chunk remains independently executable and intelligible to both the vector embedding model and the LLM reader.

Raw code snippets alone frequently lack contextual anchors. A vector embedding of a simple authorization check like 'return user.id == token.userId;' matches dozens of unrelated functions across an entire monorepo. To ground semantic retrieval, gitloom enriches every vector record with static metadata:

• File Path Hierarchy: Embedding the exact repository path (e.g., services/auth/jwt_validator.ts) ensures directory context is factored into embedding similarity.
• Exported vs. Private Scope: Differentiating public API signatures from internal helper functions allows query filters to prioritize interface definitions.
• Incoming and Outbound Call References: Tracking callers and callees provides dependency graph traversal during multi-hop reasoning.

Dense vector embeddings capture conceptual intent (such as 'where is user session revocation handled?'), but perform poorly on exact identifier lookups (such as TokenExpiredException or MAX_RETRY_COUNT). To address this, gitloom pairs pgvector dense cosine search with PostgreSQL full-text search (tsvector) using Reciprocal Rank Fusion (RRF):

WITH semantic_search AS (
  SELECT id, RANK() OVER (ORDER BY embedding <=> :query_vec) AS dense_rank
  FROM code_embeddings
  WHERE repo_id = :repo_id
  LIMIT 30
),
keyword_search AS (
  SELECT id, RANK() OVER (ORDER BY ts_rank_cd(text_search, plainto_tsquery('english', :query_text)) DESC) AS sparse_rank
  FROM code_embeddings
  WHERE repo_id = :repo_id AND text_search @@ plainto_tsquery('english', :query_text)
  LIMIT 30
)
SELECT COALESCE(s.id, k.id) AS chunk_id,
       (COALESCE(1.0 / (60 + s.dense_rank), 0.0) +
        COALESCE(1.0 / (60 + k.sparse_rank), 0.0)) AS rrf_score
FROM semantic_search s
FULL OUTER JOIN keyword_search k ON s.id = k.id
ORDER BY rrf_score DESC
LIMIT 6;

Dumping thirty chunks into an LLM context window causes the well-documented 'lost in the middle' degradation, where models ignore critical facts buried in voluminous prompts. gitloom takes the top 6 RRF candidates, merges contiguous line spans from the same source file, and formats the prompt with explicit line numbers and file paths. High precision retrieval consistently outperforms massive context dumping.