Benchmark trust leaderboard

Every benchmark in the catalog ranked by a composite trust score derived from the integrity classifications already mined in benchmark integrity. Peer-reviewed mentions raise the score, independently-verified mentions raise it a little more, disputed mentions push it down, and an all-vendor-claimed benchmark with ≥5 citations takes an orphan penalty on top. This is derived data — not an opinion — and the per-benchmark drilldown shows you exactly which systems contributed to each bucket.

25 benchmarks ranked · formula in $lib/analyses/benchmark-trust.ts.

How the score is computed

score = pr% · 1.0  +  iv% · 1.2  −  disputed% · 1.5  −  vendor_orphan_penalty
        clamped to [0, 100]

Where pr% is the share of peer-reviewed mentions and iv% is the share of independently-verified mentions out of the four evaluable buckets (the "unverifiable" bucket is excluded from the denominator — missing citations shouldn't punish or rescue a benchmark). The vendor-orphan penalty fires when 100% of mentions are vendor-claimed and there are ≥5 mentions — i.e. the benchmark exists only in vendor blog posts.

High ≥ 70 — cite without caveat
Medium 40 ≤ s < 70 — show the integrity breakdown alongside
Low 20 ≤ s < 40 — vendor-heavy or contested
Compromised < 20 — needs independent corroboration

Most trusted (top 5)

highest composite scores

1 ALFWorld 100.0 High 9 peer-reviewed · 9 total
2 AppWorld 100.0 High 2 peer-reviewed · 2 total
3 BABILong 100.0 High 5 peer-reviewed · 5 total
4 BrowseComp 100.0 High 3 peer-reviewed · 3 total
5 HotpotQA 100.0 High 10 peer-reviewed · 10 total

Most gamed (bottom 5)

≥5 vendor-claimed mentions only

1 SWE-bench 0.0 Compromised 5 vendor-claimed · 5 disputed
2 MMLU 17.6 Compromised 14 vendor-claimed · 0 disputed

Full leaderboard

Click any column header to sort. Click a row to expand its list of citing systems with each system's integrity classification.


ALFWorld mentions-only	100.0	9	—	—	—	9	High
AppWorld mentions-only	100.0	2	—	—	—	2	High
BABILong 2y 5mo	100.0	5	—	—	—	5	High
BrowseComp mentions-only	100.0	3	—	—	—	3	High
HotpotQA mentions-only	100.0	10	—	—	—	10	High
ImplicitMemBench 3mo	100.0	1	—	—	—	1	High
LongBench 3y	100.0	3	—	—	—	3	High
Mind2Web mentions-only	100.0	1	—	—	—	1	High
MT-Bench mentions-only	100.0	3	—	—	—	3	High
NaturalQuestions mentions-only	100.0	4	—	—	—	4	High
NIAH 2y 8mo	100.0	7	—	—	—	7	High
PersonaBench mentions-only	100.0	1	—	—	—	1	High
RULER 2y 3mo	100.0	3	—	—	—	3	High
ScienceWorld mentions-only	100.0	2	—	—	—	2	High
TriviaQA mentions-only	100.0	2	—	—	—	2	High
GAIA mentions-only	76.9	10	—	3	—	13	High
LoCoMo 1y 6mo	50.6	22	1	3	5	31	Medium
MemoryAgentBench 1y 1mo	50.0	1	—	1	—	2	Medium
WebArena 3y	30.0	3	—	1	1	5	Low
MMLU mentions-only	17.6	3	—	14	—	17	Compromised
OSWorld 2y 3mo	12.5	2	—	1	1	4	Compromised
AIME mentions-only	0.0	2	—	—	6	8	Compromised
ConvoMem mentions-only	0.0	—	—	1	1	2	Compromised
LongMemEval 1y 9mo	0.0	10	1	—	8	19	Compromised
SWE-bench mentions-only	0.0	2	—	5	5	12	Compromised