In any retrieval system that chunks documents, there’s a quiet failure mode that doesn’t show up in tests until it hits a real vault. When a document is long enough — a reference book, a design spec, a lengthy technical guide — adaptive chunking splits it into many pieces. Each chunk gets indexed independently. When a query matches multiple chunks of that same document, the retrieval engine has no reason to prefer one chunk over another. So it returns them all. And if your result list has five slots, a single long document can occupy four of them.
That’s exactly what happened in markdown-vault-mcp, the MCP server that handles structured markdown collections with FTS5 full-text search and semantic vector search. The bug was reported by a user working with a large SABSA (Sherwood Applied Business Security Architecture) reference document — a ~150-chunk vault where four of the top five get_similar results were sections from the same book, drowning out genuinely different relevant files.
The root cause wasn’t a typo or a missing index. It was an architectural gap: the search system had no diversity enforcement across chunks of the same document.
Adaptive Chunking: The Feature That Created the Problem
markdown-vault-mcp uses adaptive heading-level chunking — long sections recursively split at deeper heading levels (H1 → H6) until each chunk fits a configurable word budget (default 400 words). This is genuinely useful: it means retrieval targets are contextually coherent rather than arbitrarily cut mid-paragraph.
But it means a 150-chunk document can generate 150 indexed entries. When a query matches the book at multiple points, the ranking function (BM25 for FTS, cosine similarity for vector search) scores each chunk independently. Without any cross-chunk deduplication, the book naturally dominates result lists simply by having more matching entries than any single other document.
This is a well-known problem in information retrieval. Elasticsearch calls it collapse. Qdrant calls it query_points_groups. Vespa calls it grouping. The academic literature (PARADE, MultiAspect) has studied it under the heading of result diversity. The pattern is always the same: at some layer of your retrieval pipeline, you need to group results by parent document and enforce a per-document cap.
The Fix: MaxP Aggregation with Field Collapsing
PR #471 introduced field collapsing across search, get_similar, and get_context.similar in markdown-vault-mcp. The implementation uses MaxP aggregation — for each document, only the top-N highest-scoring chunks are returned, and the document’s score is the maximum score among its sections.
This is the same aggregation strategy used in PARADE (CIKM 2020) and is the default in Elasticsearch’s collapse feature. The logic: if a document has multiple relevant passages, the most relevant passage is what determines how useful the document is. Returning weaker secondary passages at the expense of different documents doesn’t improve the answer.
The chunks_per_doc parameter (renamed chunks_per_file for clarity) controls how many sections per document can appear in a result. Setting it to 2 means: for each document, return its two best-matching chunks. The result list now means “N distinct files” rather than “N chunks.”
The rename from chunks_per_doc to chunks_per_file also clarified the semantics: limit now bounds the number of files returned, not the number of chunks. This is the intuitive behavior — if you ask for 10 results, you expect 10 different documents, not 10 chunks from one dominant document.
The Compound Bug: Length Downweighting
PR #471 fixed the dominance problem, but a second bug emerged from the interaction between two features. PR #433 had introduced length downweighting (alpha=0.25) — a bias that penalizes results from longer documents, based on the intuition that shorter focused documents are often more relevant to specific queries.
This makes sense for search queries against a broad index. But for get_similar — which finds documents similar to a given source document — length downweighting was actively harmful. The SABSA reference book (~150 chunks, raw cosine similarity 0.90 to its own white-paper summary) fell off the top 30 entirely after #471. The compound penalty of field collapsing plus length downweighting crushed the score to ~0.40, while shorter bibliographies at 0.55-0.64 dominated.
The fix in PR #473 was surgical: get_similar and get_context.similar set alpha=0.0 to disable length downweighting entirely. The rationale is that similarity queries have a reference document as the query — that document’s length is a feature, not a bias. The SABSA book should score high when asking for documents similar to the SABSA book. Search queries (query string → documents) are the different use case where length downweighting still helps.
What This Means for RAG Pipelines
The markdown-vault-mcp experience captures a pattern that every production RAG system eventually encounters: retrieval diversity is not automatic, and naive chunk indexing makes it worse.
The symptoms:
- Long documents dominate result lists when queries match them at multiple points
- Setting
top_khigher doesn’t help — you get more chunks from the same dominant documents - The retrieved context has low diversity even when the corpus has relevant alternatives
The solutions map to the layers in the markdown-vault-mcp fix:
-
Query-time grouping — enforce per-document result caps at retrieval time (MaxP or some other aggregation). This is the fastest to implement and works with any existing index.
-
Re-ranking with diversity — a second-stage reranker (Cross-Encoder, Colbert, or a learned model) can explicitly score for result diversity alongside relevance.
-
Embedding strategy — use document-level embeddings or passage-level embeddings depending on use case. Passage-level enables fine-grained matching but requires diversity handling. Document-level avoids the problem but loses passage granularity.
-
Chunking strategy — adaptive heading-level chunking (like markdown-vault-mcp uses) produces more coherent chunks than fixed-size chunking, but generates variable chunk counts per document, which is exactly what makes diversity enforcement necessary.
The SABSA Test Case
The real-vault verification in PR #473 is worth dwelling on. The test used the actual SABSA reference document — a real 150-chunk security architecture book — as the source query for get_similar. Before the fix (post-#471), the book didn’t appear in the top 30. After the fix, it was #1 with a cosine score of 0.9003, surfacing the exact chapter (“Chapter 7: Using This Book as a Practical Guide”) most relevant to its own summary document.
That kind of result — where the fix makes a specific, verifiable improvement on real data, not synthetic test queries — is what separates a working retrieval system from one that passes benchmarks but fails production queries.
For anyone building RAG pipelines: field collapsing is not an optional optimization. It’s the layer that converts “top-k chunks” into “top-k relevant documents.” Without it, you’re not measuring what you think you’re measuring.