ARTOO-DARTU: Studying AR-HRC With AR Obstruction Mitigation During a Warehouse Task

Research Context and Motivation

In human-robot collaboration (HRC) within warehouse environments, augmented reality (AR) is increasingly used to provide 'situated analytics'—real-time visual feedback about a robot's status, intent, and path. While these overlays improve coordination, they often create a significant safety and usability risk: the virtual content can obstruct the user's view of critical real-world information, such as inventory labels, shelf markers, or safety signs. As the number of robots in a workspace increases, the frequency of these detrimental obstructions rises, necessitating a system that can manage AR content dynamically.

The ARTOO-DARTU System

The authors introduce ARTOO-DARTU, an AR system designed for the Microsoft HoloLens 2 that provides robot-coupled situated analytics while preserving real-world visibility. The core innovation is the Obstruction Detection and Mitigation (ODM) pipeline. Unlike previous methods that require predefined environment maps or cloud-based APIs, the ODM operates in near real-time on a local edge server. It continuously monitors the user's field of view and automatically adjusts the transparency of AR indicators—such as 'thought bubbles' or location markers—whenever they overlap with identified real-world objects or text. This ensures that the virtual analytics remain spatially anchored to the robot without obscuring the physical environment.

Evaluation and Findings

To test the system, the researchers developed 'Pocket MonstARs,' a gamified inventory-picking task. In this setup, participants collaborated with a TurtleBot 4 to identify and reach targets represented by virtual monsters, while simultaneously needing to read physical labels on boxes. In a study with 34 participants, the researchers compared performance with and without the ODM. The results were clear: while AR analytics generally improved performance, the benefits were only fully realized when the ODM was active. Participants using the ODM were 46% more efficient on the overall task and 61% faster on subtasks that required reading real-world labels. The findings suggest that effective AR-HRC requires not just the presence of information, but active management of the user's visual field.