MemConflict
https://arxiv.org/abs/2605.20926
Diagnostic benchmark covering dynamic, static and conditional memory conflicts, with black-box answer and white-box retrieval/ranking metrics; it evaluates A-Mem, LangMem, Letta, MemOS, Mem0 and Memobase. Traction: Head-to-head evaluation of six named memory systems; Renmin Univ + MemTensor
Active · Verified 2026-10-23 · fresh as of 2026-10-23 · 11 cited facts, 0 with a third-party source · 1 recorded change · Suggest a correction
At a glance
- Tier
- T3
- Section
- Memory benchmarks & evaluation
- Created
- 2026-05-16 repo fix
- Latest release
- last commit 2026-06-27 repo fix
- License
- unspecified vendor fix
- GitHub
- https://github.com/TaoZhen1110/MemConflict (13★) repo fix
- Pricing
- searched not found fix
- Funding
- searched not found fix
- Freshness
- 2026-10-23 fix
- Layer
- agent-infra repo fix
Memory taxonomy
- storage
- none-trivial
- retrieval
- none
- persistence
- cross-session
- update
- read-only
- unit
- turn
- governance
- n/a
- conflict
- out-of-scope
When to use
Optimised for: searched not found
Anti-fit: searched not found
Pros & cons
Pros
searched not found
Cons
searched not found
Claims & capabilities
Press Enter to search · Advanced search
Technical surface
- API surface
- searched not found
- Backend storage
- searched not found
- Deployment
- searched not found
- Embedding model
- searched not found
- Multi-tenancy
- searched not found
- MCP
- searched not found
- A2A
- searched not found
- OpenTelemetry
- searched not found
Similar systems
Other memory benchmarks & evaluation in the catalog, ranked by inbound references.
- LoCoMo T3
Conversations spanning up to 35 sessions / 300 turns / ~9K tokens, built from LLM-agent personas grounded on temporal event graphs and verified by human annotators. Tests QA (single-hop, multi-hop, temporal, commonsense, adversarial), event-graph summarisation, multimodal dialogue.
- GAIA T3
Benchmark for general AI assistants made of real-world questions that require reasoning, multimodality, web browsing and tool use, with a public leaderboard on Hugging Face. Traction: ~917 likes on the Hugging Face dataset
- LongBench T3
Tsinghua THUDM. 21 datasets across 6 categories (single-doc QA, multi-doc QA, summarisation, few-shot ICL, synthetic, code completion) in English + Chinese. ~6.7K words English / 13.4K chars Chinese average. v2 at ACL 2025 with harder, longer instances.
- LongMemEval T3
5 core memory abilities — info extraction, multi-session reasoning, temporal reasoning, knowledge updates, abstention — via 500 curated questions in synthetic chat histories. Two scales: LongMemEval-S (115K tokens), LongMemEval-M (up to 1.5M).
- RULER T3
NVIDIA. Extends NIAH with multi-needle retrieval, multi-hop entity tracing, aggregation, QA — configurable for arbitrary length / difficulty. Evaluated 17 long-context models on 13 tasks.
- InfiniteBench (∞Bench) T3
Tsinghua / OpenBMB. First benchmark with average input >100K tokens. 12 tasks across English + Chinese: long novel QA, code debugging, math in long context, summarisation, retrieval. Realistic documents (novels, code) rather than synthetic filler.
Sources (10 cited facts, 0 third-party)
Each fact links to the page it came from; the badge says who published it (how to read sources).
- What it is Benchmark vendor
- What it does & what's distinct Diagnostic benchmark covering dynamic, static and conditional memory conflicts, with bla… vendor
- Claims / benefits Press Enter to search · Advanced search vendor
- Created 2026-05-16 repo
- Latest release last commit 2026-06-27 repo
- License unspecified vendor
- GitHub https://github.com/TaoZhen1110/MemConflict (13★) repo
- Mindshare 13 GitHub stars repo
- Project & sources https://arxiv.org/abs/2605.20926 ; https://github.com/TaoZhen1110/MemConflict vendor
- taxonomy-three-layer-rollup agent-infra repo
What changed
Changes to this record's lifecycle, licence, release, pricing and hosting facts, from the catalog's git history. Subscribe (Atom)
- 2026-10-23 Added to the catalog #222