OpenAI Gym Retro
https://github.com/openai/retro
OpenAI's Gym Retro — wraps thousands of classic arcade games (Sega Genesis, NES, SNES, Atari 2600, Game Boy) as Gym RL environments via libretro emulators. Released 2018; powered the OpenAI Retro Contest where agents had to generalise from train-set games to held-out levels of Sonic. Largely supplanted by Atari 100k / Procgen / etc. but still the most-popular path for adding arbitrary retro games to RL pipelines.
Archived · Verified 2026-10-28 · fresh as of 2026-10-28 · 31 cited facts, 1 with a third-party source · 1 recorded change · Suggest a correction
At a glance
- Tier
- T3
- Section
- Memory benchmarks & evaluation
- Created
- 2018-05 (Nichol et al. arXiv 1804.03720) repo fix
- Latest release
- not applicable — not OSS fix
- GitHub
- 3.5k★ Python/C repo fix
- Pricing
- not applicable — open benchmark; free to use repo fix
- Funding
- not applicable — not commercial fix
- Freshness
- 2026-10-28 fix
- Layer
- agent-infra repo fix
- Licensing
- open-source repo fix
- Developer activity
- OpenRank 0.15 (12-mo avg 0.42) · 1 participants — 2026-01 third-party fix
Memory taxonomy
- storage
- n/a
- retrieval
- n/a
- persistence
- n/a
- update
- read-only
- unit
- n/a
- governance
- n/a
- conflict
- n/a
When to use
Optimised for: not applicable - eval dataset, not a system
Anti-fit: not applicable - eval dataset, not a system
Pros & cons
Pros
Largest catalog of retro games as RL environments (1000+); libretro-backed → broad emulator coverage; canonical for arcade-RL.
Cons
No longer actively maintained by OpenAI (2020 deprecation); ROM legalities limit redistribution; outclassed by Atari 100k / Procgen for generalisation eval.
Claims & capabilities
OpenAI 2018; powered Retro Contest 2018 (Sonic generalisation); 1000+ supported games; canonical entry-point for libretro-emulator RL
Technical surface
- API surface
- not applicable — eval dataset, not a system
- Backend storage
- not applicable — eval dataset, not a system
- Deployment
- not applicable — run locally
- Embedding model
- not applicable — eval dataset, not a system
- Multi-tenancy
- not applicable — eval dataset, not a system
- MCP
- not applicable — benchmark / evaluation harness
- A2A
- not applicable — benchmark / evaluation harness
- OpenTelemetry
- not applicable — benchmark / evaluation harness
Similar systems
Other memory benchmarks & evaluation in the catalog, ranked by inbound references.
- LoCoMo T3
Conversations spanning up to 35 sessions / 300 turns / ~9K tokens, built from LLM-agent personas grounded on temporal event graphs and verified by human annotators. Tests QA (single-hop, multi-hop, temporal, commonsense, adversarial), event-graph summarisation, multimodal dialogue.
- GAIA T3
Benchmark for general AI assistants made of real-world questions that require reasoning, multimodality, web browsing and tool use, with a public leaderboard on Hugging Face. Traction: ~917 likes on the Hugging Face dataset
- LongBench T3
Tsinghua THUDM. 21 datasets across 6 categories (single-doc QA, multi-doc QA, summarisation, few-shot ICL, synthetic, code completion) in English + Chinese. ~6.7K words English / 13.4K chars Chinese average. v2 at ACL 2025 with harder, longer instances.
- LongMemEval T3
5 core memory abilities — info extraction, multi-session reasoning, temporal reasoning, knowledge updates, abstention — via 500 curated questions in synthetic chat histories. Two scales: LongMemEval-S (115K tokens), LongMemEval-M (up to 1.5M).
- RULER T3
NVIDIA. Extends NIAH with multi-needle retrieval, multi-hop entity tracing, aggregation, QA — configurable for arbitrary length / difficulty. Evaluated 17 long-context models on 13 tasks.
- InfiniteBench (∞Bench) T3
Tsinghua / OpenBMB. First benchmark with average input >100K tokens. 12 tasks across English + Chinese: long novel QA, code debugging, math in long context, summarisation, retrieval. Realistic documents (novels, code) rather than synthetic filler.
Sources (30 cited facts, 1 third-party)
Each fact links to the page it came from; the badge says who published it (how to read sources).
- What it is Retro arcade RL benchmark — Sega / NES / SNES game integration repo
- What it does & what's distinct OpenAI's Gym Retro — wraps thousands of classic arcade games (Sega Genesis, NES, SNES, A… repo
- Claims / benefits OpenAI 2018; powered Retro Contest 2018 (Sonic generalisation); 1000+ supported games; c… repo
- Modalities visual (retro arcade frames) repo
- Created 2018-05 (Nichol et al. arXiv 1804.03720) repo
- License MIT repo
- GitHub 3.5k★ Python/C repo
- Mindshare OpenAI Retro Contest was a high-profile community competition; deprecated in 2020 but st… repo
- Citations 200+ cites — Nichol et al. arxiv repo
- Pricing not applicable — open benchmark; free to use repo
- Deployment not applicable — run locally repo
- Founders / pedigree Alex Nichol, Vicki Pfau, Christopher Hesse, Oleg Klimov, John Schulman (OpenAI) repo
- Reproducibility Independently reproduced — Retro Contest 2018 had public leaderboard repo
- Code/weights release Code public + game ROMs require user-supply repo
- Versioning not applicable — static benchmark repo
- Tombstoning not applicable — static benchmark repo
- Schema evolution not applicable — fixed evaluation dataset repo
- Forgetting policy not applicable — static benchmark repo
- Pros Largest catalog of retro games as RL environments (1000+); libretro-backed → broad emula… repo
- Cons No longer actively maintained by OpenAI (2020 deprecation); ROM legalities limit redistr… repo
- Project & sources github.com/openai/retro Retro Contest 2018 arxiv 1804.03720 repo
- commit-trajectory [{"ym":"2018-03","cum":41},{"ym":"2018-04","cum":79},{"ym":"2018-05","cum":96},{"ym":"20… repo
- cost-input-usd-per-mtok 0 repo
- cost-output-usd-per-mtok 0 repo
- cost-tier free repo
- cost-pricing-model free-tier repo
- cost-last-verified 2026-06-29 repo
- licensing-bucket open-source repo
- taxonomy-three-layer-rollup agent-infra repo
- developer-activity-opendigger {"activity":0.72,"as_of":"2026-01","openrank_12mo":0.42,"openrank_current":0.15,"partici… third-party
What changed
Changes to this record's lifecycle, licence, release, pricing and hosting facts, from the catalog's git history. Subscribe (Atom)
- 2026-10-22
- Lifecycle first recorded as “archived”