Recent method papers — theorized, no distinct product

DeepSeek + PKU. Introduces "conditional memory" alongside MoE's conditional compute. 27B-param N-gram lookup store offloads static knowledge from the transformer; O(1) lookup; prefetched from host RAM at near-zero runtime overhead.

Interleaves sparse MoE layers with Mamba-2 layers; Mamba layers maintain a fixed-size recurrent state (constant memory during generation) rather than a growing KV cache. Attention layers use GQA with only 2 KV heads. Two parallel memory …

Replaces MLP layers with sparse cross-layer transcoders to produce "replacement models" where activations correspond to interpretable features; traces attribution graphs showing which features causally influenced which outputs. Includes …

Meta-tokens are learned vocabulary entries that act as placeholders for recurring token subsequences — the model is fine-tuned to understand compressed input where meta-tokens stand in for their corresponding subsequences without any inf…

AI21's hybrid Mamba + Transformer MoE model — 256k context window; open-weights. First production model with SSM-Transformer hybrid architecture.

Selective state-space model — input-dependent SSM with linear scaling; competitive with transformers at smaller scales. CMU.

Augments a transformer with a fixed-size 1B-parameter memory pool distributed across all layers; new knowledge is injected by updating the pool parameters during inference, not via context extension. Distinct from M+ (already in landscap…

Meta-programming for a multi-agent collaborative framework.

Two complementary NAVER Labs Europe papers (Jan 2025). Provence fine-tunes DeBERTa as a sequence labeler to dynamically detect needed pruning amount per query and mask irrelevant context spans before generation, unifying pruning with rer…

Continuous Thought Machines. Each neuron maintains its own history of incoming activations via private per-neuron weight parameters; the model operates along a self-generated "thought step" timeline rather than the input sequence timelin…

Encodes new experiences directly into the LLM's existing parameters by minimizing KL divergence between a teacher model (with context access) and a student model (without). Generates diverse QA pairs from new knowledge as the distillatio…

During prefill, value caches are offloaded to CPU and only low-rank pre-RoPE keys plus compressed landmarks and outliers are kept on GPU. During decoding, landmarks select chunk indices for key reconstruction and value fetching from CPU.…

Workflow-based memory framework component.

Extracts, stores, adapts, and reuses structured plan templates from planning stages of agent applications. NeurIPS 2025 poster.

Global Workspace Layer combining low-rank explicit memory + bottleneck attention + Hopfield-style associative memory. Decouples memory from the Hopfield network via learnable explicit memory. Outperforms sparse-Transformer baselines on S…

Null-space-constrained knowledge editing for language models.

Episodic observations as timestamped vertices; LLM extracts relational triplets into a semantic knowledge graph; episodic edges connect semantic nodes — structural equivalent of hippocampal index binding cortical representations. Dual-la…

Thought-augmented reasoning with LLMs. NeurIPS 2024.

Context-Dependent Memory framework. Three-stage encoding (expert / short-term / long-term) + graph-structured context-dependent indexing. NAACL 2025 Industry.

Comprehensive survey. ACM Computing Surveys 2025. Anchor reference for the continual-learning-of-LLMs subfield.

Maintains recent states in full resolution while compressing older memories with learned compression functions. DeepMind.

Tool creation for disentangling abstract and concrete reasoning. EMNLP Findings.

DIffusion As a Model Of eNvironment Dreams. Trains an RL agent inside a diffusion-based world model — denoising diffusion generates high-fidelity environment rollouts rather than discrete latents. Addresses visual-detail loss in prior di…

Encodes entire document corpora directly into Transformer parameters; the model itself becomes the index.

Hafner et al. General world-model agent. Symlog observation, KL balance, RMSNorm, Block GRU.

System 1: fast retrieval path that pulls compressed, relevant memories and conditions generation. System 2: slow reflection pass that evaluates outcomes and writes curated updates back through a conservator gate. No model-weight changes.

Enhances lifelong model editing with mixture-of-LoRA. AAAI 2025.

Efficient Memory-Augmented Transformer. Encodes millions of QA pairs into key-value memory; MIPS search returns hits in sub-millisecond latency. EMNLP 2022.

Multi-agent RL with semantically coherent episodic buffer; selectively promotes desirable transitions to avoid local convergence.

Kirkpatrick et al. Soft quadratic constraint pulls each weight back toward old values, scaled by importance for prior tasks. PNAS 2017.

LLM agents are experiential learners. AAAI 2024.

Park et al. — landmark agent-simulation paper. Reflection + memory stream + retrieval enable believable agent behavior.

Stanford's iconic memory-stream architecture for believable simulacra — observations + reflections + plans + retrieval by recency/importance/relevance. UIST 2023.

DeepMind. Foundation world model trained unsupervised on internet gaming video. Generates frame-by-frame controllable virtual worlds from text / image / sketch prompts. Spatiotemporal video tokenizer + autoregressive dynamics + latent ac…

Heavy-hitter oracle for efficient generative inference. NeurIPS 2023.

Heavy-Hitter eviction policy for KV cache — retains tokens with cumulative high attention scores; up to 5× throughput.

Symbolic indexing + hierarchical memory refinement + structured episodic retrieval, inspired by hippocampal mechanisms. Nature Scientific Reports 2025.

Re-implements ACT-R's retrieval probability model — temporal decay, prior-access frequency, stochastic noise — as scoring on top of a vector store. Spreading activation over semantic similarity adds associative recall. Forgetting becomes…

Subquadratic alternative to attention — implicit long convolutions + data-controlled gating. Stanford. Ancestor of StripedHyena.

Meta. Self-supervised image learning by predicting target-block representations from a context block in latent space — no hand-crafted augmentations or pixel reconstruction. Foundation of LeCun's hierarchical-JEPA programme. CVPR 2023.

Open-world multi-task agents with memory-augmented multimodal LLMs. TPAMI 2024 . (Distinct from any other JARVIS.)

Neuro-symbolic architecture for manufacturing decision-making. Extracts ACT-R's procedural / declarative / goal buffers as latent neural representations, then injects them into trainable LLM adapter layers via knowledge transfer.

Shift-short attention + LoRA fine-tuning that extends LLaMA context length cheaply — 100k tokens on a single GPU. ICLR 2024.

Structured State-space Duality — Mamba-2 architecture that unifies SSMs and attention; 2-8× faster than Mamba-1. ICML 2024.

External key-value stores with similarity-based cache management; trained with Memory Replay Backpropagation.

Already in catalog as the foundational MemGPT paper. Note: Letta is the productionised successor (cross-listed).

Autonomously augments semantic representations of historical interactions to improve retrieval quality without manual curation. Validated on conversational recommendation, QA, and event summarisation. EMNLP 2025 Main.

Reifies the reuse of user-AI conversational memories. CHI/UIST.

Architecture with external retrieval memory in the transformer.

Concise interface for real-time memory augmentation. CHI.

Three-tier memory model: implicit (weights), working (context), and explicit (compressed KV stored externally). 2.4B model competitive with 7B+ using explicit memory.

Enhances LLMs with long-term memory. Early influential paper.

Tsinghua THUIR. First benchmark covering both declarative and procedural memory for LLM systems. User-feedback simulation framework tests adaptive improvement. Distinct from Renmin's MemBench.

IBM Research. Memory-augmented LLM architecture for multi-hop reasoning. Iteratively re-reads a temporal latent memory with bidirectional GRUs until readout stabilises. NeurIPS 2024.

Before test time, LLM reasons over unlabeled data and stores high-confidence intermediate thoughts as external memory. At test time, retrieves relevant stored thoughts to augment reasoning. No annotated data, no parameter updates.

Boosts LLM agents with meta plan optimisation. EMNLP 2025.

Pritzel et al. Agent stores past state-action-value tuples in a differentiable dictionary; recall via approximate nearest neighbour. ICML 2017.

Efficient KV-cache compression through retrieval heads.

Bulatov / Kuratov / Burtsev. Adds memory tokens passed between segments — no architectural change to the Transformer.

Language agents with verbal reinforcement learning.

Uses tensor decomposition to exploit low intrinsic dimensionality and pixel correlation of stored exemplar images, achieving high compression while preserving discriminative information for class-incremental learning. Compresses the repl…

Combines forward-looking Prospective Reflection (multi-granularity summarisation into personalised memory bank) with backward-looking Retrospective Reflection (RL-driven retrieval refinement guided by cited evidence). Operates at utteran…

RWKV recurrent-LM family — linear-time, constant-memory inference; v7 ('Goose') is the SotA recurrent LM. Community-driven OSS, Linux Foundation hosted.

Self-evolving agents with reflective and memory-augmented abilities. Neurocomputing 2025 .

Memory construction and retrieval for personalised conversational agents.

Iterative self-feedback loop — LLM critiques its own output and revises; in-context experiential memory of the current task.

LLM knows what you're seeking before generation. NeurIPS 2024.

Soft chain-of-thought for efficient reasoning with LLMs. ACL 2025.

Attention-sink discovery — keep the first few tokens always cached; arbitrary-length inference with no fine-tuning. ICLR 2024.

Stanford's Test-Time Training layers — RNN-like hidden state is itself an inner ML model that updates at test time on the current input.

Language models teach themselves to use tools.

Facilitates LLMs mastering 16,000+ real-world APIs.

Extends context through segment-level recurrence + caching of hidden states from prior segments. Foundational long-context architecture.

NVIDIA lifelong-learning agent for Minecraft — automatic curriculum + skill library + iterative prompting. Foundational procedural-memory paper.

Rethinks knowledge memory for lifelong model editing. NeurIPS 2024.

Kruijne et al. RL-learned gating policies open/close multiple working-memory slots independently. Handles hierarchical 12-AX task with multiple concurrent context levels; all-or-nothing slot updates.

RoPE-extension method for context-length scaling — combines NTK-aware interpolation with attention scaling, requires only short fine-tuning. Widely adopted in OSS long-context models.

Optimises context compression for long-horizon LLM agents.

Leverages cross-domain experience for agentic problem solving.

Open agentic framework that uses computers like a human.

Agent that maintains a skill library and reconfigures its toolset per task — procedural memory of successful sub-task solutions.

Towards an efficient self-evolving agent system.

Long-horizon web agents with proactive context management.

Learning unified long-term and short-term memory management.

Life-long personalisation of LLMs.

Generalist agent enabling scalable agentic reasoning.

Self-evolving generative agent for agent generation.

Associative Recurrent Memory Transformer. Stores tokens in Hopfield-style energy basins for constant-time pattern completion.

Long-term memory module that learns to memorise the context at test time. Omega rule (sliding-window update over all past tokens), polynomial feature mappings, Muon optimiser. Generalises to "DeepTransformers" family. May 2025.

Builds web agents with human-inspired web-browsing actions.

Constructivist view of agentic memory for reading comprehension.

Scalable multi-agent framework for Windows UI task automation.

Cognitive-inspired memory-organised RAG for stateful narrative reasoning.

Enhances dialogue consistency via dynamic structured memory and reasoning.

Open-ended evolution of self-improving agents.

General reasoning agent with scalable toolsets.

Hafner / Yan. Solves control tasks by training agents inside a fast, accurate world model.

Test-time learning with adaptive memory.

Uses Bayesian surprise detection to partition the cache into meaningful episodes for coherent retrieval.

Pink et al. — position paper. Uses CLS theory to identify five structural properties of episodic memory absent from current LLM agents: single-shot acquisition, temporal context, source binding, flexible recombination, selective consolid…

Cross-task generalisation by collecting and abstracting successful trajectories into insights for future tasks.

Synthesised LLM multi-agent system with conceptual verbal reinforcement.

Continuous agent evolution via forward learning from experience.

Replay scheduler inspired by the Ebbinghaus forgetting curve, plus intensity-aware regularisation to control replay strength. Mitigates catastrophic forgetting in LLM continual learning.

Tracing hierarchical memory for multi-agent systems.

Separates offline memory (lightweight Memorizer highlighting key facts) from online runtime (Researcher agent that queries the full raw page-store at inference). Inverts usual approach — keep full fidelity offline, pay complexity at quer…

Hierarchical hindsight reflection for multi-task LLM agents.

Co-activation strengthens associative edges (Hebbian Distillation); separate consolidation pass periodically extracts semantic summaries from the episodic graph — directly modelling slow-wave hippocampal-to-cortical consolidation. Dual-p…

Inducing programmatic skills for agentic task enhancement.

Google. "Leave No Context Behind." Compressive memory inside the attention block + masked local + long-term linear attention.

Rethinks long-horizon agents via Markovian state reconstruction.

Real-time LLM personalisation based on human feedback.

IBM Research. Attaches a fast-learning distributed episodic memory (hippocampus-inspired) to a frozen LLM, enabling one-shot knowledge updates without retraining. Supports selective forgetting + information-leakage prevention.

Few-shot mobile GUI agent with unified demonstration benchmark.

Modular procedural memory for multi-agent LLM systems.

Lightweight, efficient memory-augmented generation.

Large memory models — explicit external memory module within transformer.

Trainable SideNet module selectively retrieves relevant cached key-value pairs from memory banks during decoding.

Generative agents for low-cost real-time social interactions.

Extends MemoryLLM with scalable long-term memory.

Multi-graph agentic memory architecture for AI agents.

Memory-Augmented Transformer Using Heterogeneous Knowledge Sources. Unifies unstructured paragraphs + semi-structured QA pairs into fixed-length type-agnostic neural memories. Jun 2024.

Learning memory construction via reinforcement learning.

Learns to synergise memory and reasoning for efficient agents.

Reshapes long-context LLM with multi-conv RL-based memory agent.

Preserves memory states across dialogue turns for coherent long-term interactions. (Cited in survey but the paper's exact arxiv/venue identifier was not recoverable in a quick search — likely a less-prominent dialogue-systems paper.)

Fine-tunes LLM-agent behavior without fine-tuning the underlying model.

Meta-evolution of agent memory systems. Includes EvolveLab — a unified self-evolving memory codebase distilling 12 representative memory systems.

Treats past optimisation gradients as memory registers for procedural computation. Distinct from the 2010.06891 "Memformer" above. (Survey citation — distinct paper identifier not recovered.)

Weaves generative latent memory for self-evolving agents.

Intent-driven memory selection for goal-oriented multi-session agents.

Retrieval Causal Attention actively prunes less important cached entries.

Distils expert adapters for on-device memory systems.

Stores raw conversational episodes without LLM-driven extraction, indexing at sentence level; retrieval expands nucleus matches into neighboring-context clusters to address the embedding-dissimilarity problem in dialogue data. A Retrieva…

Co-pilot alongside reasoning agent. Builds dependency graph over reasoning steps; prunes invalid steps, folds completed sub-trajectories, preserves compact high-salience backbone under fixed context budget — without blocking execution.

Tunes LLMs to use memos for consistent long-range conversation.

Resolves fateful forgetting through human-inspired memory architecture. Distinct from MatrixOrigin's commercial Memoria.

Autonomous context curation for long-horizon agentic tasks.

Pretrained, plug-and-play memory for LLMs.

Meta. Replaces dense feed-forward blocks with trainable key-value layers using product-key lookup.

Language modeling with explicit memory. Landmark.

Explores agent procedural memory.

Closed-loop retrieval controller replacing single-shot retrieve-then-answer. Iterative router chooses between retrieve, reflect, answer; global evidence-gap tracker makes retrieval reasoning explicit and auditable.

Self-evolving agents via runtime reinforcement learning on episodic memory.

Trains LLMs to reason, search, and manage memory via RL.

Optimises short-term memory management for dynamic tool calling.

Builds an LLM-driven hierarchical tree of facts; queries traverse the tree from root to leaves, balancing recall coverage with depth.

Multimodal memory system for lifelong learning agents.

Embedding-space memory consolidation — merges semantically similar memories without LLM arbitration to reduce storage cost.

Manager stores compressed non-parametric search trajectories; Planner is a parametric memory agent producing search plans; Executor acts on plans. Planner evolves via test-time learning (on-the-fly weight updates during inference), creat…

Multi-agent memory system for LLM-based agents.

Language modeling with retriever-pretrained external memory.

Mixed memory-augmented generation for LLM applications.

Sakana AI. "An Evolved Universal Transformer Memory." Neural Attention Memory Models trained via evolution (overcomes non-differentiability of token-keep/discard).

Self-organising agent memory inspired by cognitive science.

First end-to-end pipeline translating natural language into executable Soar rules. Imports the full Soar production-rule cycle: working memory, operator proposal/application, chunking. LLM generator proposes new productions; critic valid…

Open agent social interaction simulations with one million agents.

Episodic simulation and memory for vision-language navigation.

Trains long-context models by simulating long positions with skip-wise positional encodings — avoids high training cost of true long sequences.

Planning- and retrieval-integrated memory for enhanced reasoning.

Synthetic strategy memory for proactive dialogue agents.

Pre-trains a large multi-task world model on offline data across many continuous-control tasks, then extracts task-specific policies by differentiating through the model with first-order gradient optimisation. World model serves as compr…

Memory-enhanced model-based RL agent built on DreamerV3. Modifies S4 to handle long temporal dependencies. Keeps DreamerV3's generality.

Bridging memory retention and retrieval via reversible compression.

Google Research / DeepMind. Prompts LLM to segment long documents into "pages," compresses each to a short gist stored as episodic memory, re-reads original pages on demand at answer time. Pure prompting, no fine-tuning.

Scales agent self-evolution with reasoning memory.

Interactive generation of arbitrarily long text.

Autonomous LLM-based agent for program repair.

Unlocks long-horizon search intelligence via context summarisation.

General read-write memory for large language models.

Retrospective LLM agents with policy-gradient optimisation.

Renormalization-group-based memory evolution for user profiles.

Local-first MCP architecture for RL coding agents that treats memory selection as a logged contextual decision process. Three MCP tools: issue_match (ranks candidates + records telemetry), issue_feedback (maps raw labels to bounded rewar…

Social-network simulation system with LLM-empowered agents.

Enhances LLMs with self-controlled memory framework.

Empowers LLMs with cognitive agency via active context management.

Novel activation approach for memory.

Self-evolving computer-use agent with autonomous learning.

Dynamic regulariser driven by Wasserstein distance between task distributions; automatically relaxes or strengthens constraints in proportion to task similarity.

Sentence-graph memory for long-term conversational agents.

Web agents self-improve by discovering and honing skills.

Memory is a dynamic graph; retrieval fires spreading activation across edges rather than vector similarity. Lateral inhibition suppresses competing activations; temporal decay modulates edge weights. Episodic and semantic stores maintain…

Two-tier: fast compressed summary index (Tier 1) + immutable raw-log store (Tier 2), with explicit provenance pointers linking summary units to source pages. Runtime sufficiency router escalates to raw log only when summary inadequate.

Unified tool retrieval and calling via generation.

Enhances multimodal agents with learnable tool capability memory.

Encodes procedural expertise using SVD decomposition with dynamic expert-vector blending.

Google. Feedback attention loop enables the model to attend to its own latent representations — emerges as working memory. No extra weights; integrates with pre-trained models.

Desktop AgentOS with memory-driven task automation.

Meta AI. Predicts masked spatio-temporal video patches in latent representation space rather than pixel space. Implicit world model without generative decoding. Released for self-supervised video representation learning from VideoMix2M.

Meta. Self-supervised video model. Predicts in latent space (no pixel decoder); supports understanding, prediction, and planning downstream. June 2025.

Structures web-scale evidence with dynamic outlines for research agents.