Results

Benchmark leaderboard

Retrieval metrics across 35 gold-bearing tasks (v1, 45-task set). Every comparative claim below is bootstrap-tested — the significance badge is load-bearing, not decorative.

Retrieval metrics by retriever

Retriever	Recall@5	Recall@10	MRR	nDCG@10	Success@5
BM25	0.405	0.643	0.498	0.455	0.743
Dense	0.526	0.769	0.472	0.510	0.771
Hybrid	0.510	0.733	0.515	0.509	0.800

n = 35 tasks with gold evidence, v1 corpus (832 docs, 35 diseases).

Reranking effect

Cross-encoder reranking (ms-marco-MiniLM-L-6-v2) applied on top of hybrid first-stage retrieval. The first-stage recall ceiling is the hard limit reranking cannot exceed — evidence the first stage never surfaces in its top-20 pool can’t be recovered downstream.

Recall@5

0.636

Recall@10

0.848

MRR

0.613

nDCG@10

0.621

First-stage recall ceiling @20

0.890

n = 35 tasks. ~11% of gold evidence is never retrieved by hybrid at all within the top 20 — no reranker can recover it.

Bootstrap significance tests

Paired bootstrap over the 35-task set, 95% confidence intervals. A CI that includes zero means the observed difference is not distinguishable from sampling noise at this scale — it is reported honestly as inconclusive, not as a win for either side.

Comparison	Metric	Mean diff	95% CI	n	Result
hybrid vs dense	Recall@10	-0.036	[-0.114, 0.038]	35	Not distinguishable from noise
hybrid vs bm25	Recall@10	+0.090	[0.007, 0.186]	35	95% CI excludes zero
hybrid vs dense	nDCG@10	-0.001	[-0.050, 0.048]	35	Not distinguishable from noise
hybrid vs bm25	nDCG@10	+0.054	[-0.008, 0.115]	35	Not distinguishable from noise
hybrid vs dense	MRR	+0.043	[-0.036, 0.130]	35	Not distinguishable from noise
hybrid vs bm25	MRR	+0.017	[-0.080, 0.113]	35	Not distinguishable from noise
reranked vs pre-rerank hybrid	Recall@10	+0.114	[0.048, 0.190]	35	95% CI excludes zero

2 of 7 comparisons exclude zero at 95% CI. Per claims_registry.md C-019: this supersedes the confidence level of the earlier v0-scale numbers, which were all null (C-012) at n=18.

Figure

Retrieval comparison across BM25, dense, and hybrid retrievers — Retrieval comparison — BM25 vs. dense vs. hybrid, by metric. See /retrieval for the reranking figure and a deeper per-retriever analysis.