Catalog Systems Recent method papers — theorized, no distinct product H2O (Heavy-Hitter Oracle)

H2O (Heavy-Hitter Oracle)

https://arxiv.org/abs/2306.14048

Heavy-Hitter eviction policy for KV cache — retains tokens with cumulative high attention scores; up to 5× throughput.

Active · Verified 2026-10-28 · fresh as of 2026-10-28 · 7 cited facts, 0 with a third-party source · Suggest a correction

At a glance

Type
KV-cache eviction policy vendor fix
Tier
T3
Section
Recent method papers — theorized, no distinct product
Created
2023-06 vendor fix
Latest release
searched not found fix
License
searched not found fix
GitHub
searched not found vendor fix
Pricing
not applicable — research paper fix
Funding
not applicable — research paper fix
Freshness
2026-10-28 fix
Layer
agent-infra repo fix

Memory taxonomy

storage
kv
retrieval
similarity
persistence
session
update
replacement
unit
token
governance
opaque
conflict
n/a

When to use

Optimised for: searched not found

Anti-fit: searched not found

Pros & cons

Pros

searched not found

Cons

searched not found

Claims & capabilities

NeurIPS 2023. Heavy-Hitter Oracle (H2O) prunes the KV-cache to <20% of its size with negligible accuracy loss, delivering up to 29x higher throughput than FlexGen and 1.9x lower latency on Llama-7B/13B/30B.

Technical surface

API surface
not applicable — research paper
Backend storage
not applicable — research paper
Deployment
not applicable — research paper
Embedding model
not applicable — research paper
Multi-tenancy
not applicable — research paper
MCP
not applicable — research paper
A2A
not applicable — research paper
OpenTelemetry
not applicable — research paper

Similar systems

Other recent method papers — theorized, no distinct product in the catalog, ranked by inbound references.

  • Compressive Transformer T3

    Maintains recent states in full resolution while compressing older memories with learned compression functions. DeepMind.

  • MemGPT v2 / agent-tools T3

    Already in catalog as the foundational MemGPT paper. Note: Letta is the productionised successor (cross-listed).

  • Transformer-XL T3

    Extends context through segment-level recurrence + caching of hidden states from prior segments. Foundational long-context architecture.

  • Generative Agents T3

    Park et al. — landmark agent-simulation paper. Reflection + memory stream + retrieval enable believable agent behavior.

  • MemoryBank T3

    Enhances LLMs with long-term memory. Early influential paper.

  • Reflexion T3

    Language agents with verbal reinforcement learning.

Sources (6 cited facts, 0 third-party)

Each fact links to the page it came from; the badge says who published it (how to read sources).