FenceXR: AR Movement Replay for Error-Detection Training and Spatially Grounded Feedback

How can augmented reality (AR) be used to help novice fencers develop the perceptual skill of error detection and enable coaches to provide more effective, spatially grounded feedback on complex movements like the lunge?

The authors developed FenceXR, an AR system that reconstructs 3D movement from standard smartphone video. The system features two modules: a Trainee module, which uses 'Ghost Avatar' (superimposed reference) or 'Deviation Overlay' (highlighted joint errors) visualizations to train novices to identify common lunge errors, and a Reviewer module, which allows coaches to attach text or voice annotations directly to specific joints and moments in a 3D replay. The authors validated the reconstruction pipeline and conducted a 18-participant study to measure error-detection accuracy before and after training, alongside a qualitative study with four fencing experts.

Novice fencers showed a significant improvement in error-detection accuracy, rising from near-chance levels (37.5%) to 64.1% after a single training session. Participants reported a shift from holistic, instinct-driven visual scanning to a more structured inspection of specific joints and inter-joint timing. Both visualization modalities were effective, though they encouraged different strategies: Ghost Avatar promoted comparative analysis, while Deviation Overlay facilitated label-guided inspection. Fencing experts viewed the system as a valuable complement to existing tools, particularly for communicating feedback that is difficult to convey through standard 2D video.

This research demonstrates that error detection is a trainable perceptual skill that can be developed through structured AR observation, independent of physical practice. By decoupling feedback from the limitations of 2D video and scarce coach availability, FenceXR provides a scalable way for athletes to refine their mental models of correct technique, offering a promising template for future motor-learning systems in fast-paced sports.