Aysan Ghayouri Pirsoltan, Shima Babakordi, Mohammad Reza Mohammadi
4 min
Abstract
Accurate breast cancer classification from mammography requires effective integration of complementary information from craniocaudal (CC) and mediolateral oblique (MLO) views, which provide a more complete characterization of breast abnormalities. However, existing multi-view learning approaches typically rely on feature-level aggregation or single-stage cross-attention, which can entangle view-specific and shared representations and restrict interaction to limited network depths. To address these limitations, we propose a token-centric dual-view learning framework that unifies prompt-based adaptation and cross-view fusion within a frozen vision transformer backbone. The framework reformulates inter-view interaction as structured token-level communication, where dedicated fusion tokens explicitly encode bidirectional information exchange between CC and MLO views via cross-attention, serving as intermediate carriers of cross-view dependencies rather than relying on direct feature fusion. Unlike conventional methods that apply fusion at a single layer, fusion modules are inserted at multiple transformer depths, enabling progressive and repeated interaction across the encoder hierarchy. Fusion tokens are reintegrated into the token sequence and refined by subsequent transformer layers, facilitating hierarchical propagation of complementary information while preserving view-specific structure. Experiments on VinDr-Mammo and CMMD datasets demonstrate consistent improvements over linear probing, prompt-only adaptation, and conventional fusion baselines. On the VinDr-Mammo BI-RADS classification task, the framework achieves 50.40% F1-score and 0.8090 AUC, including a 0.10 AUC improvement over a dual-view fusion baseline in the binary setting. Ablation studies further validate the effectiveness of token-based fusion and multi-depth interaction design.
Alex: You said "at several points." Is that important?
Sam: Very. Most systems that combine multiple views do it once, at the very end—after each model has already finished processing its image independently. This paper does it throughout, at multiple intermediate stages. Think of it like reading a complex book: instead of waiting until the final page to compare notes with a friend, you stop after every chapter, share what you've each understood, and carry that updated understanding into the next chapter. Each round of note-passing makes the next round more informed.
Alex: So the AI is building a shared understanding gradually, rather than just comparing final answers.
Sam: Correct. And there's another practical advantage worth mentioning. The underlying vision transformer—the core model that already knows how to read images—is kept completely unchanged. The researchers don't retrain it from scratch. They only train the fusion tokens themselves. It's like hiring an experienced doctor and teaching them a new filing system, rather than putting them through medical school again. That makes the whole approach far more efficient to develop.
Alex: And the results?
Sam: The study reports a meaningful improvement in breast cancer classification compared to standard methods that combine the two views more crudely. The paper suggests that the structured, back-and-forth dialogue between views—rather than a one-time merge—is what drives that gain.
Alex: So the contribution here isn't a bigger or more powerful model. It's a smarter way of organizing how the model thinks about information it already has.
Sam: That's a fair summary. The underlying insight is that in medical imaging, how you combine information matters as much as how much information you have. Keeping the sources distinct, letting them talk to each other gradually, and doing that at multiple stages—those design choices, taken together, appear to produce a more reliable result.
Alex: It's a good reminder that in AI for healthcare, careful architecture can matter just as much as raw computing power. Thanks for listening to ResearchPod.