Gyeonghun Kim, Donghoon Kim, Minsoo Choi, Suyun Lee
7 min
Abstract
Recent advances in Vision-Language-Action (VLA) models have enabled fine manipulation capabilities, yet performing tasks that require working memory remains a key challenge. Most existing VLA architectures operate under a Markovian assumption, mapping only the current observation to an action, which leads to failure in non-Markovian environments. Inspired by the human 'inner monologue', we propose Hierarchical Explicit Language Memory (HeLM), a hierarchical VLA framework that maintains task-relevant history in explicit language through factorized event detection and event-triggered memory updates. We evaluate HeLM across 10 non-Markovian real-world tasks with 56 variants, including five newly formulated inter-episode tasks that require persistent context across task boundaries. HeLM improves intra-episode success by 12.8 percentage points over the strongest baseline and achieves 78% subtask success on compositional generalization tasks, while using up to 4.5x less memory and running 2.5x faster than the keyframe baseline at long horizons.
Alex: A high-level vision-language model manages memory and chooses an action command. A separate low-level model turns that command into physical movement. The written memory holds abstract progress; the current camera view still guides where the robot moves.
Sam: Does the high-level model rewrite its account continuously? That sounds like another opportunity to introduce mistakes.
Alex: No. At each boundary between chunks of movement, it first checks whether a relevant event occurred. If nothing happened, memory and command stay unchanged. Only a detected event, or a changed instruction, triggers the update stage.
Sam: So detecting completion and deciding what completion means for the plan are separate jobs. How does that work when an instruction ends?
Alex: There are separate memory fields. Working memory tracks progress within the active instruction. Episodic context is a summary retained across instructions. When the task changes, working memory resets while relevant earlier information remains available.
Sam: Give me a case where that retained summary is essential, rather than merely convenient.
Alex: A human demonstrates a sequence of colored button presses. The robot records the observed order. Later, a new instruction asks it to reproduce that order. The buttons' current appearance cannot recover the demonstration, but the retained language summary can.
Sam: Where does the system learn which events deserve an entry? That choice seems to carry much of the task knowledge.
Alex: The authors define a schema called TaskSpec. It specifies allowed commands, relevant events, and corresponding memory updates. They use it to pair short recorded subtasks with synthetic memory states, rather than collecting every full-length task variation.
Sam: Is that schema effectively handing the robot the solution during evaluation?
Alex: At inference, it receives task-specific event and action vocabularies, not the full schema or correct memory transitions. It must detect events and update memory itself. But the event vocabulary is predefined, so this is not open-ended discovery of what matters.
Sam: Now to the evidence. How broad was the physical evaluation, and which comparison best isolates the memory choice?
Alex: The suite contains ten real-world tasks. Across them, the authors test fifty-six variants, covering counting, ordered progress, and recalling objects after concealment or movement. The cleanest comparison is Keyframe-HLP, which stores event images instead of text.
Sam: For that comparison to mean much, the robot's movement skills and high-level model should be matched. Were they?
Alex: They share the action backbone, high-level backbone, and event supervision. Across tasks within a single instruction, HeLM achieves about eighty-six percent average subtask success. The keyframe version achieves about seventy-three percent.
Sam: Subtask success can reward pieces of a failed procedure. If the robot repeats a press or never stops, does that headline score catch it?
Alex: Not directly; order and stopping errors are recorded separately. Under strict scoring, requiring every subtask in order and proper stopping, HeLM achieves about eighty-one percent. The keyframe baseline drops to about twenty-nine percent.
Sam: That makes the case more about reliable completion than partial progress. Does the advantage extend across instructions?
Alex: HeLM also substantially outperforms the keyframe version there. But the other baselines were excluded because they lack support for retaining context across episode boundaries. The authors acknowledge that developing dedicated cross-episode baselines remains open.
Sam: What about unfamiliar combinations? A system could remember progress well and still depend on rehearsed procedures.
Alex: On held-out configurations of learned subtasks, HeLM reaches seventy-eight percent average subtask success without additional high-level fine-tuning. Tests include longer orders, unseen counts, and recombined procedures. That's compositional generalization, not evidence of unrestricted new skill learning.
Sam: And does writing memory actually reduce computation, or just replace one expensive operation with another?
Alex: Initially, generating text makes HeLM slower. At longer tested histories, it becomes two-and-a-half times faster than the keyframe baseline as image tokens accumulate. It also uses less memory, though the high-level model adds overhead compared with a plain action model.
Sam: The memory can be readable and still be wrong. What happens if the robot records an event that never happened?
Alex: That's the central failure risk. Detection errors dominate HeLM's failures, especially false positives such as treating an unopened drawer as opened. Those errors produce incorrect memory updates that the framework cannot self-correct. Physical execution errors are the other major source.
Sam: So who should read the full document, and where should they start?
Alex: Researchers building robot memory or hierarchical policies should read the framework, then the failure analysis and strict task-success evaluation. The TaskSpec examples reveal how much task structure is supplied. Keep the scope in view: predefined discrete events, not continuous state tracking.
Sam: And for someone outside robotic manipulation, what's the finding to carry away?
Alex: Within this task suite, explicit language memory makes progress easier to track and inspect. But a readable memory is not necessarily a reliable one: it still depends on detecting what actually happened.
Sam: Thanks for listening.