Catalog Systems Recent method papers — theorized, no distinct product Transformer-XL

Transformer-XL

https://arxiv.org/abs/1901.02860

Extends context through segment-level recurrence + caching of hidden states from prior segments. Foundational long-context architecture.

Research code · Verified 2026-10-23 · fresh as of 2026-10-23 · 29 cited facts, 1 with a third-party source · 1 recorded change · Suggest a correction

At a glance

Type
Segment-level recurrence + cached states vendor fix
Tier
T3
Section
Recent method papers — theorized, no distinct product
Created
2019-01 vendor fix
Latest release
no-release no-release vendor fix
License
Apache-2.0 repo fix
GitHub
3.7k★ +60/mo Python repo fix
Pricing
not applicable — research paper fix
Funding
not applicable — not commercial fix
Freshness
2026-10-23 fix
Layer
agent-infra repo fix
Licensing
open-source repo fix
Developer activity
OpenRank 0.27 (12-mo avg 0.45) · 1 participants — 2025-01 third-party fix
Maintainer health
1 issues in 90 days · 0 releases in 12 months · bus factor 1 — as of 2026-11-01 repo

Memory taxonomy

storage
kv-cache
retrieval
attention
persistence
session
update
evict-oldest
unit
kv-token
governance
inspectable
conflict
n/a

When to use

Optimised for: not applicable - research paper

Anti-fit: not applicable - research paper

Pros & cons

Pros

Defined segment-level recurrence — foundational; cited by every subsequent long-context paper.

Cons

2019 — pre-LLM era; modern architectures (rotary positional embedding, sparse attention) supersede in practice.

Claims & capabilities

Segment-level recurrence mechanism with novel positional encoding overcoming fixed-length context limitations; learns dependencies 80% longer than RNNs and 450% longer than vanilla Transformers; up to 1,800x faster evaluation; SOTA on enwik8 (0.99 bpc), text8 (1.08), WikiText-103 (18.3); generates coherent text articles thousands of tokens long; ACL 2019

Technical surface

API surface
not applicable — research paper
Backend storage
not applicable — research paper
Deployment
not applicable — research paper
Embedding model
not applicable — research paper
Multi-tenancy
not applicable — research paper
MCP
not applicable — research paper, no deployed product
A2A
not applicable — research paper, no deployed product
OpenTelemetry
not applicable — research paper, no deployed product

Similar systems

Other recent method papers — theorized, no distinct product in the catalog, ranked by inbound references.

  • Compressive Transformer T3

    Maintains recent states in full resolution while compressing older memories with learned compression functions. DeepMind.

  • MemGPT v2 / agent-tools T3

    Already in catalog as the foundational MemGPT paper. Note: Letta is the productionised successor (cross-listed).

  • Generative Agents T3

    Park et al. — landmark agent-simulation paper. Reflection + memory stream + retrieval enable believable agent behavior.

  • MemoryBank T3

    Enhances LLMs with long-term memory. Early influential paper.

  • Reflexion T3

    Language agents with verbal reinforcement learning.

  • DreamerV3 T3

    Hafner et al. General world-model agent. Symlog observation, KL balance, RMSNorm, Block GRU.

Sources (28 cited facts, 1 third-party)

Each fact links to the page it came from; the badge says who published it (how to read sources).

What changed

Changes to this record's lifecycle, licence, release, pricing and hosting facts, from the catalog's git history. Subscribe (Atom)

  1. 2026-07-02
    • Lifecycle first recorded as “research-complete”
    #123

Related systems

Referenced by (8)