Speculative RAG
https://arxiv.org/abs/2407.08223
Smaller distilled specialist generates multiple RAG drafts in parallel from distinct document subsets. Larger generalist verifies in a single pass. Subsets formed by content-similarity clustering to minimise redundancy.
Active · Verified 2026-10-22 · fresh as of 2026-10-22 · 24 cited facts, 1 with a third-party source · Suggest a correction
At a glance
- Type
- Small-model draft, large-model verify vendor fix
- Tier
- T4
- Section
- Retrieval-as-memory hybrids
- Latest release
- not applicable — not OSS fix
- License
- not applicable — not OSS fix
- GitHub
- not applicable — no GitHub repo fix
- Pricing
- not applicable — not commercial fix
- Funding
- not applicable — not commercial fix
- Freshness
- 2026-10-22 fix
- Layer
- agent-infra repo fix
- Licensing
- commercial vendor fix
Memory taxonomy
- storage
- vector
- retrieval
- similarity
- persistence
- long-term
- update
- read-only
- unit
- chunk
- governance
- inspectable
- conflict
- n/a
When to use
Optimised for: retrieval quality / accuracy on long-context QA
Anti-fit: most are research papers; production fitness varies by maintainer
Pros & cons
Pros
Generates candidate answers in parallel from retrieval, picks best.
Cons
Preprint; parallel-generation cost.
Claims & capabilities
+12.97% accuracy on PubHealth; 51% latency reduction vs conventional RAG. SOTA on TriviaQA, MuSiQue, ARC-Challenge.
Technical surface
- API surface
- not applicable — research paper
- Backend storage
- not applicable — research paper
- Deployment
- not applicable — not a deployable product
- Embedding model
- not applicable — research paper
- Multi-tenancy
- not applicable — research paper
- MCP
- not applicable — research paper, no deployed product
- A2A
- not applicable — research paper, no deployed product
- OpenTelemetry
- not applicable — research paper, no deployed product
Alternatives to Speculative RAG → live options ranked by shared design, licence, hosting and MCP support
Similar systems
Other retrieval-as-memory hybrids in the catalog, ranked by inbound references.
- GraphRAG (Microsoft) T1
Modular graph-based RAG. Leiden community detection over knowledge graph; community summaries at multiple levels. DRIFT Search hybrid (Oct 2024). The GraphRAG repo is now largely in maintenance mode: no new features or PRs, only bug fixes and dependency/CVE updates.
- RETRO T3
DeepMind. Retrieval-Enhanced Transformer with chunked cross-attention over a 2T-token retrieval database. Frozen BERT retriever + differentiable encoder.
- Atlas T3
Meta AI. Jointly fine-tunes retriever + LM for few-shot tasks. The facebookresearch/atlas repo is no longer maintained; the code is provided as is.
- RAPTOR T3
Stanford recursive-summarisation tree over the corpus — clusters and summarises documents into a hierarchical index; retrieves at multiple levels. ICLR 2024.
- Self-RAG T4
Trains an LLM end-to-end to emit reflection tokens that decide whether to retrieve, assess passage relevance, and critique output quality — all within a single model.
- HippoRAG / HippoRAG2 T3
Hippocampus-inspired RAG. LLMs + knowledge graphs + Personalized PageRank to mimic neocortex/hippocampus roles. NeurIPS 2024.
Sources (23 cited facts, 1 third-party)
Each fact links to the page it came from; the badge says who published it (how to read sources).
- What it is Small-model draft, large-model verify vendor
- What it does & what's distinct Smaller distilled specialist generates multiple RAG drafts in parallel from distinct doc… vendor
- Claims / benefits +12.97% accuracy on PubHealth; 51% latency reduction vs conventional RAG. SOTA on Trivia… vendor
- Modalities text vendor
- Created 2024-07 vendor
- Citations 97 cites 48/yr paper
- Founders / pedigree Google DeepMind; arxiv 2407.08223 vendor
- Memory volume / scale no scale claim — research vendor
- Performance +12.97% PubHealth SOTA (sc TriviaQA vendor
- Reproducibility No reproductions found vendor
- Code/weights release Paper only — ICLR 2025 (arXiv v2 2025-02-27); Google Research blog links no official code third-party
- Versioning none vendor
- Schema evolution schema with migration vendor
- Validated verticals research, RAG developers (horizontal); some have production deployments vendor
- Anti-fit most are research papers; production fitness varies by maintainer vendor
- Optimised for retrieval quality / accuracy on long-context QA vendor
- Adjacent infrastructure BYO LLM + vector store; OSS reference implementations exist vendor
- Pros Generates candidate answers in parallel from retrieval, picks best. vendor
- Cons Preprint; parallel-generation cost. vendor
- Project & sources arxiv 2407.08223 vendor
- citation-trajectory [] paper
- licensing-bucket commercial vendor
- taxonomy-three-layer-rollup agent-infra repo