Percutaneous epicardial access (PEA), performed on a beating heart under fluoroscopy, enables arrhythmia treatment. However, advancing a needle toward the thin and moving pericardium remains highly challenging and risky. To address this problem, we present a physics-driven sonification method for Extended Reality (XR)-based multisensory navigation to enhance user perception during the critical needle landing phase in PEA. Dynamic cardiac anatomy from 4D CTA was reconstructed and registered to a real-world coordinate system. Real-time needle tracking provided the position of the needle tip relative to moving cardiac structures and drove an audio-visual feedback module. The visual display presented navigational cues and dynamic anatomy, while the auditory display encoded physiological cardiac states using a multilayer physical membrane model. A phantom study was conducted with twelve cardiologists performing needle insertions under visual-only and multisensory feedback. The multisensory method significantly improved navigation safety ($χ^2 = 11.30$, $p < 0.01$), reducing myocardial contact (3.64% vs. 7.27%) and increasing correct access (90.91% vs. 52.73%). Needle placement accuracy improved, with closer membrane proximity (Cliff delta = 0.19) and reduced variability ($p < 0.05$). Execution time was comparable, while time-accuracy correlations differed significantly between modalities ($p < 0.01$). NASA-TLX indicated lower cognitive load with multisensory guidance ($p < 0.01$). These results demonstrate the feasibility of physics-driven sonification for improving spatiotemporal awareness and supporting user-centered surgical navigation.
Alex: Welcome to another episode of ResearchPod. Sam, I've been reading about a procedure where doctors guide needles toward a beating heart—what's this paper about?
Sam: This preprint tackles a tough challenge in heart surgery. It proposes using sound combined with visuals to help doctors safely poke a needle into a thin fluid sac around the heart during a procedure called percutaneous epicardial access, or PEA. The goal is to treat irregular heartbeats by getting tools onto the heart's outer surface without puncturing the heart muscle itself.
Alex: So the core problem here is advancing a rigid needle toward this super-thin sac on a moving heart, right? And regular X-ray images aren't cutting it?
Sam: Yes, exactly. In PEA, the doctor inserts the needle near the breastbone and pushes it toward the pericardial sac—a tiny, fluid-filled bag just millimeters thick that wraps the heart. The heart beats constantly, so the sac moves, and standard fluoroscopy gives only a flat, two-dimensional view that hides depth and nearby tissues like the heart muscle. This makes it hard to stop precisely at the sac without going too far and perforating the myocardium, the heart's inner wall. Studies report complication rates around twelve point five percent, often from those perforations.
Alex: That sounds risky—relying on feel and fuzzy shadows while everything's in motion. How do two-D images fall short for something so precise?
Sam: Fluoroscopy shows the needle as a shadow on a screen, but it flattens the three-dimensional space, so doctors can't easily judge distances to the moving sac or heart wall. Contrast dye highlights the sac briefly, yet motion blurs it, and there's no clear cue for the safe zone between the outer sac wall and inner heart layer. Visual augmented reality overlays, like those on head-mounted displays, add models of the heart but get blocked by the patient's body or suffer alignment errors, limiting their help in this dynamic setup.
Alex: So even fancy three-D heart models overlaid in real time don't solve the occlusion or depth issues? What gap does this leave for guiding the needle right to that millimeter-thick target?
Sam: Precisely—visual cues alone overload the eyes and miss subtle proximity warnings amid the motion. The paper suggests pairing them with sound that reflects the needle's changing distance to these structures, turning abstract gaps into audible cues doctors can feel intuitively. This multisensory approach aims to boost awareness without replacing the X-rays.
Alex: Okay, so sound turns those tricky distances into something a surgeon can sense without staring. But how exactly does it work—walk me through the sounds for getting close without poking the heart muscle.
Sam: The system tracks two key distances in real time: how far the needle tip is from the outer pericardial sac, and from the inner heart muscle. These gaps change as the heart beats, so the sound shifts to warn the doctor. It divides the space into four zones along the needle path—far outside the sac, right before piercing it, safely inside, and too close to the heart wall. As the needle enters each zone, the sound changes to match, like different drum taps guiding you closer. Think of it like traffic lights, but with sound instead of colors.
Alex: What makes the sound change in a way that's easy to follow?
Sam: They model it after a real drumhead—a flat, vibrating circle you tap to make noise, like stretching a balloon skin tight. Far away, it's a steady, high-pitched drum beat, calm like tapping the center. As you near the sac, the pitch drops lower and the beats speed up with quicker fade-out, mimicking tapping toward the edge where vibrations tense up. Piercing safely makes a clear, resonant thump; hitting the danger zone screeches harshly and abruptly stops. This physics-based setup uses rules for pitch, decay speed, and rhythm tied to those distances, so the ear picks up the risk naturally without learning codes.
Alex: Like the drum getting tighter and wilder as you edge closer. Does that actually help doctors in practice, or is it just theory?
Sam: A study tested it on heart models with twelve cardiologists using fake patients. With visuals plus this sound, they reached the safe spot about twice as often as with visuals alone, with far fewer heart muscle touches and less mental strain. The paper suggests this combo gives a clear edge in precision under motion.
Alex: That's a meaningful improvement—sound filling the gaps eyes miss. But with the heart moving, how do they sync the models so the distances stay accurate?
Sam: They start with pre-op scans capturing the heart's full beat cycle, then animate those frames in a loop on a see-through headset display. A virtual needle overlays the real one via camera tracking, plus static guides like a lined path. Everything updates together in a closed loop, keeping distances spot-on despite the beat.
Alex: That closed loop for tracking and visuals seems reliable. How does it turn those distances into specific drum changes?
Sam: They simulate a drumhead—a thin sheet that vibrates when hit, producing sound based on where and how hard you tap. Changing the sheet's size lowers the note's pitch, like a bigger drum booming deeper; tweaking how fast vibrations die out makes the tone short or lingering; and adjusting hit timing speeds up or slows the rhythm. Researchers link distances to those traits for natural shifts—like getting closer stretches the drum tighter, dropping the pitch and speeding the beats.
Alex: Got it—like tuning a real drum by stretching or damping it. They tested this on fake hearts with real doctors—what did that show?
Sam: Yes, twelve cardiologists—mostly experts and some novices—did punctures on a chest model mimicking ribs and sternum, tracked by cameras aligning virtual heart from CT scans. Multisensory beat visual-only about twice as often for safe sac entry without heart contact, plus lower mental load per standard surveys. Novices improved most, going from about fifty-four percent safe entries with visuals alone to over ninety percent with sound added. The paper notes this suggests better control in dynamic tasks.
Alex: Twice as reliable for novices—that's a clear step for precision and training. What exactly counted as success—stopping right at the sac without overshooting?
Sam: Success meant reaching the pericardial surface without touching the heart muscle during the advance, checked by tracking the needle tip against the model's moving parts. They used a method like shooting invisible rays from the tip to detect hits on the sac or muscle in real time across the heart's beat cycle. For those who succeeded, the final tip sat closer to the sac under multisensory—though stats showed only a small edge.
Alex: Did doctors report it feeling less demanding mentally?
Sam: Yes, after each run, they filled out a standard survey rating mental effort, frustration, and focus demands—like a scorecard for task strain. Multisensory scored notably lower overall, pointing to reduced cognitive load amid the motion. This fits the idea that sound offloads visual overload, letting focus stay on key cues. Experts noted relying fully on sound when visuals glitched from movement or headset wobble.
Alex: So safer navigation, tighter stops, easier on the brain—sound complements without replacing the X-ray guide. The paper cautious on some stats though—like accuracy not fully proven for clinics?
Sam: Yes, it highlights safety gains like fewer misses and muscle risks as the main win, with consistency edges but no huge accuracy leaps needing bigger tests. Key hurdles include accurately outlining the pericardium from scans, which stays tricky for thin structures. Operating room noise could drown the sounds, headsets might shift with alignment errors, and it hasn't faced true heart motion yet.
Alex: Sound stepping up when visuals glitch does seem like a smart backup for real operating rooms. Pulling it all together, this multisensory setup makes needle guidance more reliable and less taxing—especially for less experienced doctors, though clinic trials are needed next.
Sam: That's a fair summary. The study points to meaningful gains in safety and control on these heart phantoms, with sound handling the dynamic cues visuals struggle with. Thanks for listening to ResearchPod.