Identifying the Unknown: Prompt-Free Open Vocabulary Anomaly Recognition for Robot-Object Interaction

As robots move toward open-world autonomy, they must identify previously unseen objects in real-time. Current state-of-the-art open-vocabulary detectors are often 'prompted,' meaning they require a predefined list of candidate object classes at inference time. This constraint limits their practical utility and prevents true discovery of novel objects. The authors seek to develop a 'prompt-free' detection system that can localize and classify arbitrary objects without prior knowledge of the scene's contents.

The authors introduce AnomNOVIC, a two-stage pipeline designed for tabletop robot-object interaction. In the first stage, a Masked Autoencoder (MAE) is trained on images of the robot's empty workspace. By learning to reconstruct only the 'normal' workspace, the MAE naturally flags any object placed on the table as an anomaly. The model outputs both a reconstruction error map and an explicit anomaly mask, which are combined to generate class-agnostic bounding boxes. In the second stage, these bounding boxes are passed to NOVIC, a prompt-free image classifier, which assigns fine-grained, free-form labels to the detected objects.

AnomNOVIC significantly outperforms existing open-vocabulary baselines (such as YOLO-World-v2, OWLv2, and YOLOE) in prompt-free recognition tasks. In a tabletop environment with the NICOL humanoid robot, the system achieved 47.1% AP (Average Precision) for prompt-free recognition, rising to 59.0% AP when class candidates were provided. Furthermore, the model demonstrated strong generalization on an 'in-the-wild' dataset containing 48 unique objects, reaching up to 82.6% detection and classification accuracy. The results confirm that decoupling object localization (via anomaly detection) from classification (via prompt-free labeling) is a highly effective strategy for open-world perception.