RETRO
https://arxiv.org/abs/2112.04426
DeepMind. Retrieval-Enhanced Transformer with chunked cross-attention over a 2T-token retrieval database. Frozen BERT retriever + differentiable encoder.
Research code · Verified 2026-10-23 · fresh as of 2026-10-23 · 26 cited facts, 2 with a third-party source · 3 recorded changes · Suggest a correction
At a glance
- Type
- Trillion-token retrieval transformer vendor fix
- Tier
- T3
- Section
- Retrieval-as-memory hybrids
- Latest release
- v0.3.9 (2023-10-30) registry fix
- License
- not applicable — no official DeepMind code (unofficial lucidrains/RETRO-pytorch is Apache-2.0) repo fix
- Pricing
- searched not found fix
- Funding
- not applicable — not commercial fix
- Freshness
- 2026-10-23 fix
- Layer
- agent-infra repo fix
- Licensing
- open-source repo fix
- Developer activity
- OpenRank 0.39 (12-mo avg 0.71) · 1 participants — 2024-07 third-party fix
Memory taxonomy
- storage
- vector
- retrieval
- similarity
- persistence
- long-term
- update
- read-only
- unit
- chunk
- governance
- opaque
- conflict
- n/a
When to use
Optimised for: retrieval quality / accuracy on long-context QA
Anti-fit: most are research papers; production fitness varies by maintainer
Pros & cons
Pros
DeepMind retrieval augmentation — model trained with retrieval as a first-class component.
Cons
Closed weights; not reproducible; superseded by simpler RAG approaches in practice.
Claims & capabilities
7.5B-param RETRO outperforms 175B Jurassic-1 on 10/16 datasets and 280B Gopher on 9/16 — equivalent quality at 25× fewer parameters. Quality continues improving up to 2T retrieval tokens.
Technical surface
- API surface
- not applicable — research paper
- Backend storage
- not applicable — research paper
- Deployment
- searched not found
- Embedding model
- not applicable — research paper
- Multi-tenancy
- not applicable — research paper
- MCP
- not applicable — research paper, no deployed product
- A2A
- not applicable — research paper, no deployed product
- OpenTelemetry
- not applicable — research paper, no deployed product
Alternatives to RETRO → live options ranked by shared design, licence, hosting and MCP support
Similar systems
Other retrieval-as-memory hybrids in the catalog, ranked by inbound references.
- GraphRAG (Microsoft) T1
Modular graph-based RAG. Leiden community detection over knowledge graph; community summaries at multiple levels. DRIFT Search hybrid (Oct 2024). The GraphRAG repo is now largely in maintenance mode: no new features or PRs, only bug fixes and dependency/CVE updates.
- Atlas T3
Meta AI. Jointly fine-tunes retriever + LM for few-shot tasks. The facebookresearch/atlas repo is no longer maintained; the code is provided as is.
- RAPTOR T3
Stanford recursive-summarisation tree over the corpus — clusters and summarises documents into a hierarchical index; retrieves at multiple levels. ICLR 2024.
- Self-RAG T4
Trains an LLM end-to-end to emit reflection tokens that decide whether to retrieve, assess passage relevance, and critique output quality — all within a single model.
- HippoRAG / HippoRAG2 T3
Hippocampus-inspired RAG. LLMs + knowledge graphs + Personalized PageRank to mimic neocortex/hippocampus roles. NeurIPS 2024.
- MemoRAG T4
RAG variant with a long-context global memory model that drafts answer clues used to guide passage retrieval — improves recall in noisy multi-hop QA.
Sources (25 cited facts, 2 third-party)
Each fact links to the page it came from; the badge says who published it (how to read sources).
- What it is Trillion-token retrieval transformer vendor
- What it does & what's distinct DeepMind. Retrieval-Enhanced Transformer with chunked cross-attention over a 2T-token re… vendor
- Claims / benefits 7.5B-param RETRO outperforms 175B Jurassic-1 on 10/16 datasets and 280B Gopher on 9/16 —… vendor
- Created 2022-01 vendor
- Latest release v0.3.9 (2023-10-30) registry
- License not applicable — no official DeepMind code (unofficial lucidrains/RETRO-pytorch is Apach… repo
- GitHub lucidrains/RETRO-pytorch (unofficial third-party impl.) — ~878 stars repo
- Citations 1.6k cites 324/yr paper
- Founders / pedigree DeepMind (London); 28 authors including Sebastian Borgeaud, Arthur Mensch, Jordan Hoffma… vendor
- Reproducibility Original team only third-party
- Code/weights release Paper only — no official DeepMind code; unofficial PyTorch reimplementation lucidrains/R… repo
- Versioning none vendor
- Schema evolution schema with migration vendor
- Validated verticals research, RAG developers (horizontal); some have production deployments vendor
- Anti-fit most are research papers; production fitness varies by maintainer vendor
- Optimised for retrieval quality / accuracy on long-context QA vendor
- Adjacent infrastructure BYO LLM + vector store; OSS reference implementations exist vendor
- Pros DeepMind retrieval augmentation — model trained with retrieval as a first-class component. vendor
- Cons Closed weights; not reproducible; superseded by simpler RAG approaches in practice. vendor
- Project & sources arxiv 2112.04426 DeepMind blog PyTorch impl vendor
- commit-trajectory [{"ym":"2022-01","cum":78},{"ym":"2022-02","cum":96},{"ym":"2022-03","cum":108},{"ym":"2… repo
- citation-trajectory [{"year":2022,"cum":2,"infl":1},{"year":2023,"cum":7,"infl":1},{"year":2024,"cum":18,"in… paper
- licensing-bucket open-source repo
- taxonomy-three-layer-rollup agent-infra repo
- developer-activity-opendigger {"activity":0.72,"as_of":"2024-07","openrank_12mo":0.71,"openrank_current":0.39,"partici… third-party
What changed
Changes to this record's lifecycle, licence, release, pricing and hosting facts, from the catalog's git history. Subscribe (Atom)