IViT: A Novel Interpretable Visual Transformer for Skin Disease Detection

Clinical diagnosis of skin diseases is often hindered by inter-class similarity and the subjective nature of human assessment. While deep learning models like Vision Transformers (ViT) offer high accuracy, they function as black boxes, lacking the transparency required for clinical trust and regulatory compliance. This paper addresses the challenge of creating an interpretable ViT (IViT) that maintains high diagnostic performance in few-shot scenarios where annotated data is limited.

The authors introduce a framework that combines transfer learning with a Quadratic Programming (QP) constrained feature selection module. By pre-training the ViT backbone on large-scale datasets and fine-tuning it on smaller, specialized skin disease datasets, the model adapts to few-shot conditions. The core innovation is a discrete QP optimization process that screens for discriminative features while minimizing redundancy. A multi-objective loss function is then used to optimize the model's activation distribution, ensuring that the features the model prioritizes align with the specific lesion areas dermatologists focus on during physical examinations.

Experimental results across six standard skin disease datasets demonstrate that IViT achieves an accuracy of 93.80%, which is only 0.21% lower than the standard black-box ViT baseline. This minimal trade-off allows for a significant improvement in model transparency. The feature selection module successfully reduced feature redundancy by 29.5%. Crucially, the model's activation maps—visualizations of which image regions drive the classification—show high consistency with the clinical diagnostic logic used by dermatologists, providing a traceable rationale for each diagnosis.