NVIDIA ReMEmbR
https://arxiv.org/abs/2409.13682
Builds long-horizon memory by captioning video segments with VILA, storing captions with timestamps + 3D position coordinates in MilvusDB. At query time, LLM iterates retrieval across text, time, and position modalities. Deployed on Nova Carter robot (Jetson Orin).
At a glance
- Type
- Spatio-temporal episodic store (vector DB + coordinates)
- Tier
- T3
- Created
- 2024-08
- Latest release
- no releases
- License
- Custom
- GitHub
- 336★ Python
- Pricing
- searched not found
- Funding
- searched not found
Taxonomy
- storage
- vector
- retrieval
- similarity
- persistence
- long-term
- update
- append-only
- unit
- episode
- governance
- inspectable
- conflict
- append
When to use
Optimised for: real-time perception + spatial reasoning + multi-modal sensor fusion
Anti-fit: not for non-embodied / non-physical use cases
Pros & cons
Pros
Episodic-memory robot reasoning framework — explicit memory of past attempts in a task.
Cons
NVIDIA-stack-tilted; research-stage.
Claims & capabilities
Outperforms LLM/VLM baselines on NaVQA across spatial, temporal, descriptive question types.
Technical surface
- API surface
- not applicable — research paper
- Backend storage
- not applicable — research paper
- Deployment
- Open-source (Custom NVIDIA license)
- Embedding model
- not applicable — research paper
- Multi-tenancy
- not applicable — research paper
- MCP
- not applicable — research paper, no deployed product
- A2A
- not applicable — research paper, no deployed product
- OpenTelemetry
- not applicable — research paper, no deployed product
Similar systems
Other vertical / domain-specific ai memory in the catalog, ranked by inbound references.
- Abridge T1
Clinician-assist ambient documentation. Source mapping: every AI-generated summary element traced back to the source utterance. Audit-and-trust layer over episodic memory. Built on proprietary 1.5M+ medical-encounter dataset.
- ASAPP GenerativeAgent T1
Treats memory as first-class architecture. Captures the digital footprint of every interaction; retrieves preference and history at engagement time. Public example: airline knowing a frequent flyer wants aisle seats with her son — preference-aware, not just history-lookup.
- BenevolentAI T1
Target identification / drug repurposing / mechanism tracing. 85+ data sources, petabyte-scale, rebuilt every few weeks. Wet-lab results re-enter the graph and shift downstream predictions — institutional experimental memory closing a feedback loop.
- Causaly T1
Drug discovery / target identification / causal mechanism tracing. The graph is the memory: 7 years of curated biomedical cause-effect relationships compounding with each new ingestion. Scientific RAG retrieves from a versioned causal substrate.
- Character.ai T1
Chat Memories (user-defined facts), auto-memories for c.ai+ subscribers, pinned memories, in-context retention. PipSqueak 2 model (April 2026) reduces in-conversation drift. Memory Visualization meter forthcoming.
- Charisma.ai T1
No-code interactive narrative platform. Characters track what they know, when they were lied to, what they misremember; story state changes downstream behaviour. Characters can deliberately lie / misremember and track when players have lied.
Related systems
References (1)
- Milvus builds on — storing captions with timestamps + 3D position coordinates in MilvusDB
Referenced by (1)
- Meta-Memory cites — S2 isInfluential citation