Haoyu Wang, Guoqing Ma, Zeyu Zhang, Yandong Guo, Boxin Shi, Hao Tang
5 min
Abstract
Generalist vision-language-action systems need object-centric 3D evidence and reusable manipulation experience to plan reliable robot trajectories. GeneralVLA provides a hierarchical interface for converting language and RGB-D observations into 3D end-effector paths, but two bottlenecks remain. First, monocular SAM3D-style object reconstruction can hallucinate pose and unseen geometry, while manipulation benefits from stable object shape when calibrated multi-view observations are available. Second, the original KnowledgeBank mainly retrieves semantically similar snippets and appends new knowledge, which makes it difficult to control memory quality, conflicts, confidence, and geometric relevance. To address the first challenge, we introduce GeoFuse-MV3D, a geometry-prior-guided MV-SAM3D reconstruction branch that verifies external geometry cues with input-view masks, applies soft visual-hull support, performs axis-wise refinement, and fuses only geometry while preserving appearance. To address the second challenge, we upgrade KnowledgeBank into a governed long-term memory system with explicit quality, confidence, lifecycle, verifier, and conflict metadata, together with precision-oriented retrieval. Finally, we evaluate the reconstruction branch on GSO-30 and the memory module on Terminal-Bench 2.0 and SWE-Bench Verified; GeoFuse-MV3D improves over the MV-SAM3D baseline by reducing CD and LPIPS by 2.20% and 2.02% while increasing PSNR and SSIM by 2.36% and 1.03%, and KnowledgeBank improves over ReasoningBank by 4.53% on Terminal-Bench SR and 3.73% on SWE-Bench resolve rate, while reducing AS by 4.95% and 5.65%, respectively. Code: https://github.com/AIGeeksGroup/GeneralVLA-2. Website: https://aigeeksgroup.github.io/GeneralVLA-2.
Sam: Precisely. Before the robot can use a past lesson, a built-in verifier checks whether that memory is actually relevant to the current situation. This stops the robot from applying a strategy that worked for one object to a completely different one—which is a surprisingly common failure mode in earlier systems.
Alex: It sounds like the system is being quite deliberate about what it trusts.
Sam: That's the core of the improvement. By making both the visual input and the retrieved memories more reliable, the robot spends less time correcting for bad assumptions and more time executing tasks based on solid evidence.
Alex: What are the actual limitations of this approach?
Sam: That's an important question. The system relies heavily on the quality of its initial inputs—particularly camera calibration. If the robot's cameras are even slightly misaligned, those errors carry through the entire process. Because the system is conservative and avoids guessing, it's actually quite sensitive to that kind of input error.
Alex: So it's a "garbage in, garbage out" situation—if the cameras aren't well-calibrated, the whole 3D model is compromised?
Sam: That's a fair way to put it. There are other constraints too. The system hasn't been tested on long, complex sequences where a robot needs to move around a room, and it doesn't yet handle deformable objects like cloth. It also has no mechanism for a human to step in and correct the robot when it gets confused.
Alex: So it works well within controlled, shorter tasks—but it's not yet a fully autonomous system for open-ended environments.
Sam: Correct. It's a meaningful step forward in how robots process geometry and past experience, but it's not a complete solution. Future work will likely focus on integrating human feedback—giving the robot a way to flag uncertainty and ask for clarification when it encounters a conflict it can't resolve on its own.
Alex: What strikes me is how focused the approach is. Rather than trying to redesign the whole system, the authors concentrated on making the inputs to the planner more trustworthy.
Sam: That's the right way to frame it. For systems where a central planner is directing the robot's actions, the quality of what that planner receives matters enormously. By improving the geometry reconstruction and the memory retrieval, GeneralVLA-2 achieves better task performance without retraining the entire model from scratch. It's a reminder that in complex systems, reliability often comes from cleaning up the foundations, not rebuilding the roof.
Alex: That's a clear and useful place to land. Thanks for walking us through it, Sam—and thanks to everyone listening to ResearchPod.