Weixian Waylon Li, Jiaxin Zhang, Xianan Jim Yang, Tiejun Ma, Yiwen Guo
10 min
This paper introduces ROMEM, a revolutionary drop-in module for temporal knowledge graphs (KGs) that treats time as a continuous geometric operator rather than a discrete timestamp label. Traditional systems struggle with the static-dynamic dilemma: they can't distinguish permanent facts (e.g., "Obama born in Hawaii") from evolving ones (e.g., "president of USA"). Solutions like overwriting history, LLM arbitration at every step, or recency sorting either lose context, scale poorly, or bury timeless knowledge.
ROMEM solves this elegantly in vector space: facts are embedded as complex vectors, and time rotates them. Dynamic facts spin out of alignment ("shadowed") over time, while static ones stay locked. No deletions, no updates—just pure geometry. It's append-only, scalable for agentic memory, and crushes benchmarks like ICEWS temporal KG completion (72.6 MRR SOTA).
Imagine facts as points on the complex plane. Each relation has a volatility score α_r ∈ (0,1) from the Semantic Speed Gate (more below). At time τ, a fact's embedding rotates by θ(τ) = α_r · ω · τ, where ω is a learnable angular speed.
Retrieval ranks by geometric proximity in the rotated space. Obsolete facts are naturally "shadowed" without touching the database. This enables zero-shot temporal interpolation: query any unseen date τ, and rotations bridge gaps continuously—no discrete timestamps needed.
A pretrained MLP maps relation text embeddings (e.g., BERT for "president of") to α_r. It learns zero-shot that roles like "CEO of" rotate fast (α_r high), while "capital of" stays static (α_r low).
Trained in two-phase: (1) Pretrain gate on semantic volatility from data; (2) Fine-tune rotation spectrum to dataset timestamps. This per-relation scalar decouples what changes (semantics) from when it changes (timestamps), fixing uniform timestamp pitfalls.
Discrete time treats all relations the same:
| Approach | Pros | Cons | |----------|------|------| | Overwrite | Simple | Loses history | | LLM per-ingest | Adaptive | OOM at scale | | Recency sort | Fast | Buries static facts |
ROMEM's functional rotation misaligns invalids via phase without magnitude penalties. Agents get clean contexts via proximity search—no LLM needed for conflicts. Append-only arch scales to millions of facts.
As a drop-in for graph-based agents, ROMEM boosts temporal reasoning 2-3× MRR on MultiTQ, dominates LoCoMo hybrid benchmark, zero-degrades static recall (DMR-MSC), and zero-shots financial QA (FinTMMBench). Think long-term autonomous agents remembering when without forgetting what.
Intuition: Time isn't a label—it's a rotation dial. Turn it, and reality reorients itself geometrically.
Structured memory representations such as knowledge graphs are central to autonomous agents and other long-lived systems. However, most existing approaches model time as discrete metadata, either sorting by recency (burying old-yet-permanent knowledge), simply overwriting outdated facts, or requiring an expensive LLM call at every ingestion step, leaving them unable to distinguish persistent facts from evolving ones. To address this, we introduce RoMem, a drop-in temporal knowledge graph module for structured memory systems, applicable to agentic memory and beyond. A pretrained Semantic Speed Gate maps each relation's text embedding to a volatility score, learning from data that evolving relations (e.g., "president of") should rotate fast while persistent ones (e.g., "born in") should remain stable. Combined with continuous phase rotation, this enables geometric shadowing: obsolete facts are rotated out of phase in complex vector space, so temporally correct facts naturally outrank contradictions without deletion. On temporal knowledge graph completion, RoMem achieves state-of-the-art results on ICEWS05-15 (72.6 MRR). Applied to agentic memory, it delivers 2-3x MRR and answer accuracy on temporal reasoning (MultiTQ), dominates hybrid benchmark (LoCoMo), preserves static memory with zero degradation (DMR-MSC), and generalises zero-shot to unseen financial domains (FinTMMBench).
Alex: Mean reciprocal rank—that's basically how close to the top the correct answer lands on average, right? A higher number means fewer wrong guesses before the good one.
Sam: Exactly. They call these setups temporal knowledge graph embeddings, or TKGE for short—think of it as mapping facts like "person A met leader B at time T" into a spinning 3D space where time twists the connections so only the matching era's facts line up straight. Obsolete ones get rotated away, like clock hands pointing wrong; you score based on how well they align to your question's time. The paper dubs this geometric shadowing: old facts naturally fall behind current ones without any erasure.
Alex: Oh—so for "who's president now," the latest fact snaps into view while older ones blur out geometrically. That sounds clean.
Sam: It is. In agent tasks like MultiTQ, heavy on time conflicts, it roughly doubles retrieval accuracy over baselines. Even on non-time chats like DMR-MSC, it holds steady or edges up slightly, proving the twist doesn't hurt everyday recall. For unseen finance queries on FinTMMBench, the speed gate spots volatile links zero-shot, keeping rotations right without retraining.
Alex: So the rotation acts like a clutch—locking static stuff steady while spinning the changey bits. That's a solid fix for the dilemma.
Alex: But to make it work on real conversations or news, how do they pull the facts out of raw text into that spinning graph?
Sam: They break it into steps using language models. First, the system scans passages for key real-world names—people, organizations, places, products, or events—listing them without grabbing dates or numbers. This spots the main players, like pulling character names from a storybook before connecting the plot; researchers call it named entity recognition, or NER.
Alex: Got it—so nodes first, no times yet. Then what connects them?
Sam: Next comes linking those names into simple statements, like "head does relation to tail," plus timing details from the text—such as a date or when it was observed. It resolves fuzzy times, like "last year" based on context, and handles durations if a start date is clear, ensuring each fact has a time stamp without using time as a player itself. They do this with prompts guiding the model to output structured triples, prioritizing full coverage of facts.
Alex: Okay, structured facts with times attached. But at question time, how does it grab the right starting names and time from something like "Who met whom last week?"
Sam: At query time, it extracts entities from the question the same way, then pulls out any time constraint or ordering hint—like earliest, latest, or none—normalizing to dates and noting if the answer needs a time. This initializes a search in the graph, feeding the right slice into retrieval where rotations align the facts.
Alex: So extraction upfront and at query keeps it precise without guessing.
Sam: Yes. The paper compares against three baselines: Mem0, which stores memories as vectors from whole documents and searches by similarity; Zep, a temporal graph using a database called Neo4j for entities; and HippoRAG, a retrieval system that augments searches with graphs and personalized ranking. Their method, ROMEM, builds directly on HippoRAG's graph-building steps but adds the rotation for time handling. It outperforms them notably on time-mixed tasks, surfacing correct facts higher without slowing down queries.
Alex: Building on something established makes sense for practicality. But computationally, twisting all those facts at query time—doesn't that get expensive with tons of data?
Sam: That's a key concern the paper addresses with a clever math trick. Instead of rotating every stored fact for each question—which would be slow—they shift the rotation entirely to the question side. Imagine the facts sitting still in a fixed grid; you twist just your query vector to match the time, then do a standard similarity search on the unchanged grid. This keeps it fast and works with off-the-shelf tools like FAISS for quick lookups; the paper proves it mathematically in what's called Proposition 1.
Alex: Oh—so the grid stays static, no rebuilding. That preserves speed while handling continuous time.
Sam: Exactly. They explain this works because rotations are orthogonal matrices—special grids that preserve lengths and angles when flipped. In complex space, which treats vectors as paired real-and-imaginary parts like 2D coordinates, element-wise rotations capture graph patterns like symmetries cheaply, unlike full matrix math.
Alex: And for times between known facts, like guessing mid-period—how does it avoid wild swings?
Sam: The paper proves smooth transitions via Proposition 2 on pairs of facts. Picture two competing facts, say Obama consulting Blair in 2007-08 then Xi in 2013-15; their scores start as waves peaking at observation times. With speeds bounded below half a cycle over the gap, one wave falls monotonically as the other rises, crossing once at a natural switch point—not the midpoint, but based on fact strengths. This enables reliable interpolation between anchors zero-shot.
Alex: So no erratic flips—just a steady handoff geometrically. That's a strong foundation.
Sam: In a qualitative trace on ICEWS data, Blair's score peaks early, fades as time advances, and Xi shadows it post-2009 crossover—both facts intact, no deletions. The evidence points to meaningful gains from this structure, especially versus discrete time methods that can't interpolate smoothly.
Alex: So the gate really nails the volatility split—even spotting static ones like "citizen of" without seeing them before. That zero-shot carryover to new domains feels like a solid win for practicality.
Sam: Yes, the pretrained gate draws from event patterns in ICEWS data to gauge change-proneness via text meanings alone. Table values confirm it: dynamic links like "consult" hit 0.87 for fast spin, static ones like "species" stay low at 0.22—working zero-shot on unseen text embeddings.
Alex: Right, so it learns the pattern of change from politics but applies to finance or biology talks. But are there spots where this rotation setup might wobble?
Sam: The paper notes a reliance on the gate's pretraining quality from ICEWS, which shapes speeds for event-heavy data. Higher-frequency wiggles—short-period spins beyond the half-cycle bound—can cause minor local ups and downs in retrieval, especially with sparse facts or shared entity patterns. Smoothing like a rolling average reveals the main trend, but raw scores show these residuals, suggesting bigger datasets could steady it further.
Alex: A grounded way to keep history alive amid updates. No overwrites, just smart phasing. The geometric approach resolves static-dynamic tensions meaningfully, preserving append-only storage while prioritizing timely facts. It's a clear step for agentic systems blending past and present. Thanks for listening to ResearchPod.