TCAM-Diff: Triplane-Aware Cross-Attention Medical Diffusion Model

Generating high-resolution 3D medical volumes (such as CT or MRI) is computationally expensive due to the massive memory footprint of 3D data. Existing generative models often struggle with this, either by losing critical anatomical detail during compression or by requiring prohibitive hardware resources. This paper asks: can we improve the efficiency and quality of 3D medical image generation by using a more compact, triplane-based representation and a specialized diffusion architecture?

The authors introduce TCAM-Diff, a two-stage generative framework. First, they employ a decoder-only autoencoder to compress 3D volumes into a triplane representation—projecting 3D data onto three orthogonal 2D planes (XY, YZ, XZ). This approach avoids the memory-heavy encoder typical of standard VAEs. Second, they use a triplane-aware cross-attention diffusion model. Unlike previous methods that simply concatenate these planes, TCAM-Diff processes each plane independently and uses cross-attention layers to integrate shared information across the planes, preserving 3D spatial relationships.

TCAM-Diff demonstrates significant improvements over baseline models like VAE-GAN and VQ-GAN. By utilizing a decoder-only architecture, the model reduces memory usage during training, allowing it to handle higher-resolution volumes (up to 512x512x512) that are difficult for traditional models to process. Quantitative results using MSE, SSIM, and the Wasserstein distance (W-GAN critic) show that TCAM-Diff produces higher-fidelity reconstructions and more realistic synthetic samples than existing encoder-decoder methods with similar latent space sizes.