OSRS Bench (Old School RuneScape agent benchmark) alternatives

Community benchmark suite for LLM agents playing Old School RuneScape, a sandbox MMORPG with deep partial-observability, skill-grinding loops, and economy interaction. Full OSRS Bench (Old School RuneScape agent benchmark) record →

  1. 1 MineRL Diamond Challenge Memory benchmarks & evaluation

    also in Memory benchmarks & evaluation · also open source · also self-hostable

    OSS ✓ Self-host ✓ Managed — MCP ? ★ 700 MIT Compare with OSRS Bench (Old School RuneScape agent benchmark) →

  2. 2 Berkeley Function Calling Leaderboard (BFCL) Memory benchmarks & evaluation

    also in Memory benchmarks & evaluation · also open source · also self-hostable

    OSS ✓ Self-host ✓ Managed ? MCP ? ★ 13k released 2026-03-23 Apache-2.0 Compare with OSRS Bench (Old School RuneScape agent benchmark) →

  3. 3 SWE-bench Memory benchmarks & evaluation

    also in Memory benchmarks & evaluation · also open source · also self-hostable

    OSS ✓ Self-host ✓ Managed ? MCP ? ★ 6.1k released 2026-09-02 MIT Compare with OSRS Bench (Old School RuneScape agent benchmark) →

  4. 4 BrowseComp Memory benchmarks & evaluation

    also in Memory benchmarks & evaluation · also open source · also self-hostable

    OSS ✓ Self-host ✓ Managed ? MCP ? ★ 4.7k released 2026-04-22 MIT Compare with OSRS Bench (Old School RuneScape agent benchmark) →

  5. 5 Terminal-Bench Memory benchmarks & evaluation

    also in Memory benchmarks & evaluation · also open source · also self-hostable

    OSS ✓ Self-host ✓ Managed ? MCP ? ★ 2.6k released 2026-07-11 Apache-2.0 Compare with OSRS Bench (Old School RuneScape agent benchmark) →

  6. 6 Habitat 3.0 (social agents) Memory benchmarks & evaluation

    also in Memory benchmarks & evaluation · also open source · also self-hostable

    OSS ✓ Self-host ✓ Managed — MCP ? ★ 2.6k MIT (Habitat-Lab); CC-BY-NC (3D scenes) Compare with OSRS Bench (Old School RuneScape agent benchmark) →

  7. 7 MineDojo Memory benchmarks & evaluation

    also in Memory benchmarks & evaluation · also open source · also self-hostable

    OSS ✓ Self-host ✓ Managed — MCP ? ★ 2.4k MIT (env wrapper); Mojang EULA on Minecraft Compare with OSRS Bench (Old School RuneScape agent benchmark) →

  8. 8 τ-bench (tau-bench / τ²-bench) Memory benchmarks & evaluation

    also in Memory benchmarks & evaluation · also open source · also self-hostable

    OSS ✓ Self-host ✓ Managed ? MCP ? ★ 2.4k released 2026-10-29 MIT Compare with OSRS Bench (Old School RuneScape agent benchmark) →

Ranked by shared memory design (storage, retrieval, conflict handling, memory unit), same section and shared must-haves (open source, self-hosting, MCP), with adoption as a tie-breaker. ✓ confirmed · — no · ? not in the catalog yet. Want picks for your own requirements? Use the chooser. Checking a whole project? Check your stack.