Veronica Ruozzi, Sasan Matinfar, Pasquale Vergara, Alessandro Albanesi, Serena Dell’Aversana, Stefano Carugo, Gianluigi Buccoliero, Nassir Navab, Alberto Redaelli, Emiliano Votta
8 min
Abstract
Percutaneous epicardial access (PEA), performed on a beating heart under fluoroscopy, enables arrhythmia treatment. However, advancing a needle toward the thin and moving pericardium remains highly challenging and risky. To address this problem, we present a physics-driven sonification method for Extended Reality (XR)-based multisensory navigation to enhance user perception during the critical needle landing phase in PEA. Dynamic cardiac anatomy from 4D CTA was reconstructed and registered to a real-world coordinate system. Real-time needle tracking provided the position of the needle tip relative to moving cardiac structures and drove an audio-visual feedback module. The visual display presented navigational cues and dynamic anatomy, while the auditory display encoded physiological cardiac states using a multilayer physical membrane model. A phantom study was conducted with twelve cardiologists performing needle insertions under visual-only and multisensory feedback. The multisensory method significantly improved navigation safety ($χ^2 = 11.30$, $p < 0.01$), reducing myocardial contact (3.64% vs. 7.27%) and increasing correct access (90.91% vs. 52.73%). Needle placement accuracy improved, with closer membrane proximity (Cliff delta = 0.19) and reduced variability ($p < 0.05$). Execution time was comparable, while time-accuracy correlations differed significantly between modalities ($p < 0.01$). NASA-TLX indicated lower cognitive load with multisensory guidance ($p < 0.01$). These results demonstrate the feasibility of physics-driven sonification for improving spatiotemporal awareness and supporting user-centered surgical navigation.
Sam: The system tracks two key distances in real time: how far the needle tip is from the outer pericardial sac, and from the inner heart muscle. These gaps change as the heart beats, so the sound shifts to warn the doctor. It divides the space into four zones along the needle path—far outside the sac, right before piercing it, safely inside, and too close to the heart wall. As the needle enters each zone, the sound changes to match, like different drum taps guiding you closer. Think of it like traffic lights, but with sound instead of colors.
Alex: What makes the sound change in a way that's easy to follow?
Sam: They model it after a real drumhead—a flat, vibrating circle you tap to make noise, like stretching a balloon skin tight. Far away, it's a steady, high-pitched drum beat, calm like tapping the center. As you near the sac, the pitch drops lower and the beats speed up with quicker fade-out, mimicking tapping toward the edge where vibrations tense up. Piercing safely makes a clear, resonant thump; hitting the danger zone screeches harshly and abruptly stops. This physics-based setup uses rules for pitch, decay speed, and rhythm tied to those distances, so the ear picks up the risk naturally without learning codes.
Alex: Like the drum getting tighter and wilder as you edge closer. Does that actually help doctors in practice, or is it just theory?
Sam: A study tested it on heart models with twelve cardiologists using fake patients. With visuals plus this sound, they reached the safe spot about twice as often as with visuals alone, with far fewer heart muscle touches and less mental strain. The paper suggests this combo gives a clear edge in precision under motion.
Alex: That's a meaningful improvement—sound filling the gaps eyes miss. But with the heart moving, how do they sync the models so the distances stay accurate?
Sam: They start with pre-op scans capturing the heart's full beat cycle, then animate those frames in a loop on a see-through headset display. A virtual needle overlays the real one via camera tracking, plus static guides like a lined path. Everything updates together in a closed loop, keeping distances spot-on despite the beat.
Alex: That closed loop for tracking and visuals seems reliable. How does it turn those distances into specific drum changes?
Sam: They simulate a drumhead—a thin sheet that vibrates when hit, producing sound based on where and how hard you tap. Changing the sheet's size lowers the note's pitch, like a bigger drum booming deeper; tweaking how fast vibrations die out makes the tone short or lingering; and adjusting hit timing speeds up or slows the rhythm. Researchers link distances to those traits for natural shifts—like getting closer stretches the drum tighter, dropping the pitch and speeding the beats.
Alex: Got it—like tuning a real drum by stretching or damping it. They tested this on fake hearts with real doctors—what did that show?
Sam: Yes, twelve cardiologists—mostly experts and some novices—did punctures on a chest model mimicking ribs and sternum, tracked by cameras aligning virtual heart from CT scans. Multisensory beat visual-only about twice as often for safe sac entry without heart contact, plus lower mental load per standard surveys. Novices improved most, going from about fifty-four percent safe entries with visuals alone to over ninety percent with sound added. The paper notes this suggests better control in dynamic tasks.
Alex: Twice as reliable for novices—that's a clear step for precision and training. What exactly counted as success—stopping right at the sac without overshooting?
Sam: Success meant reaching the pericardial surface without touching the heart muscle during the advance, checked by tracking the needle tip against the model's moving parts. They used a method like shooting invisible rays from the tip to detect hits on the sac or muscle in real time across the heart's beat cycle. For those who succeeded, the final tip sat closer to the sac under multisensory—though stats showed only a small edge.
Alex: Did doctors report it feeling less demanding mentally?
Sam: Yes, after each run, they filled out a standard survey rating mental effort, frustration, and focus demands—like a scorecard for task strain. Multisensory scored notably lower overall, pointing to reduced cognitive load amid the motion. This fits the idea that sound offloads visual overload, letting focus stay on key cues. Experts noted relying fully on sound when visuals glitched from movement or headset wobble.
Alex: So safer navigation, tighter stops, easier on the brain—sound complements without replacing the X-ray guide. The paper cautious on some stats though—like accuracy not fully proven for clinics?
Sam: Yes, it highlights safety gains like fewer misses and muscle risks as the main win, with consistency edges but no huge accuracy leaps needing bigger tests. Key hurdles include accurately outlining the pericardium from scans, which stays tricky for thin structures. Operating room noise could drown the sounds, headsets might shift with alignment errors, and it hasn't faced true heart motion yet.
Alex: Sound stepping up when visuals glitch does seem like a smart backup for real operating rooms. Pulling it all together, this multisensory setup makes needle guidance more reliable and less taxing—especially for less experienced doctors, though clinic trials are needed next.
Sam: That's a fair summary. The study points to meaningful gains in safety and control on these heart phantoms, with sound handling the dynamic cues visuals struggle with. Thanks for listening to ResearchPod.