Product × benchmark coverage
The companion-and-pivot to benchmark coverage. That view is benchmark-centric — rows are benchmarks, the lens is "is this benchmark adopted?". This view flips the matrix to product-centric: each row is a product, each column is a benchmark, and the cell colour is the integrity tier of the strongest available citation. The flip surfaces the angle no comparable catalog publishes: which products refuse to put a peer-reviewed score on the record at all.
122 products · 25 benchmarks shown · 172 total cells filled
of catalogued products report zero peer-reviewed benchmark scores
Of 1002 records, only 83 have any benchmark mention backed by a peer-reviewed citation (arXiv, OpenReview, ACL, IEEE, ACM, PMLR). 919 products do not — either because they don't benchmark at all, or because the only citations are vendor-self-published. The toggle below narrows the matrix to the subset that does mention benchmarks but without peer-reviewed backing.
What each integrity tier means
- Peer-reviewed
- Cited at a peer-reviewed venue (arXiv, OpenReview, ACL, IEEE, ACM, PMLR).
- Independently verified
- Cited at a non-vendor third party — neutral leaderboard or independent platform.
- Vendor-claimed
- Citation host matches the vendor’s own domain; vendor is making the claim.
- Disputed
- Another catalog row contradicts this number — in-cell dispute signal or >7-pt score divergence on the same benchmark.
- Unverifiable
- No citation that resolves to a host, sentinel value, or depth-floor cell.
- No mention
- Product makes no mention of this benchmark anywhere in the catalogued cells.