How can we bridge the gap between high-level language reasoning and the continuous, uncertain nature of physical robot execution? The authors address the challenge of creating a unified system that allows robots to perform long-horizon tasks—such as mobile manipulation and cross-robot coordination—by moving beyond loosely coupled modules toward a closed-loop, observable, and verifiable execution framework.
Approach
HoloAgent-0 introduces a three-layered architecture centered on an Embodied AgentOS.
Embodied AgentOS: Acts as the central runtime that parses natural language into a sequence of skill nodes, schedules these tasks, monitors execution, and triggers re-planning if a skill fails.
Memory Layer: Maintains persistent 3D spatial grounding (using semantic maps and scene graphs) and temporal history, allowing the agent to remember past outcomes and object locations.
Skill Layer: Exposes robot capabilities (navigation, manipulation, whole-body motion) as typed, monitorable interfaces. Instead of treating skills as black boxes, this layer provides structured feedback (e.g., success, failure, confidence) to the AgentOS, enabling the system to handle real-world uncertainty.
Findings and Significance
The framework was deployed on real hardware to evaluate its performance across several complex scenarios, including object search, long-horizon laundry folding, and cross-robot coordination. By using a ROS2-based command/status bus, HoloAgent-0 allows for modular development where different backends (like HoloNavi for navigation or HoloBrain for manipulation) can be swapped or upgraded without changing the high-level planning logic. This design demonstrates that treating physical execution as a closed-loop system—rather than a one-shot planning task—is essential for robust, real-world robot deployment.
The core runtime layer that manages the lifecycle of a task, from parsing language instructions into skill graphs to monitoring execution and handling failures.
Typed Skill Interface
A standardized communication protocol that defines robot actions with specific parameters, preconditions, and expected status feedback, allowing the AgentOS to interact with heterogeneous robot hardware.
3D Spatial Memory
A persistent data structure that stores geometric and semantic information about the environment, including object locations and room layouts, to ground robot planning in physical reality.
HoloNavi
The specific navigation backend that uses hierarchical semantic matching and active exploration to find targets in complex, multi-room environments.
HoloBrain
A vision-language-action (VLA) model used for manipulation tasks, providing the low-level policy inference required for object interaction.
AI-generated third-party summary by ResearchPod. Not official content or an endorsement by the paper authors or affiliated organizations.