TabPFN beyond Tabular Data: Calibration and Accuracy on Multimodal Embeddings

This study evaluates TabPFN—a Prior-Fitted Network originally designed for tabular data—as a zero-gradient, in-context learning (ICL) classification head for frozen multimodal encoders. The authors test this approach across 14 datasets, 11 encoders, and three modalities (image, text, and audio), totaling 22,820 evaluation episodes. They compare TabPFN against eight standard classification heads, including k-nearest neighbors, logistic regression, and linear SVMs, focusing on both predictive accuracy and calibration metrics like Negative Log-Likelihood (NLL) and Expected Calibration Error (ECE).

TabPFN demonstrates a significant and consistent advantage in calibration. Across the tested episodes, it reduces NLL by 48–62% and ECE by 2.1–5.3x compared to the average of the baseline methods. While its accuracy is generally competitive with or superior to these baselines, this performance is highly dependent on the experimental regime. The accuracy gains are most pronounced when the number of shots ($k$) is at least 50 and feature dimensions ($d$) are kept between 8 and 32. In scenarios with very limited data or extremely high-dimensional features, the performance gap narrows.

A major contribution of this work is the application of TabPFN to fine-tuned backbones. Fine-tuning often leads to overconfident, poorly calibrated models. The authors demonstrate that by discarding the trained linear head and replacing it with a TabPFN head—without any further gradient updates—researchers can achieve significantly better calibration while preserving the accuracy gains obtained through fine-tuning. This provides a practical, training-free method to improve the reliability of multimodal classifiers in safety-critical applications.

Few-shot multimodal classification is widely used, but the standard practice of using simple linear heads often results in miscalibrated confidence scores, which can be dangerous in high-stakes fields like medical imaging or clinical decision support. This paper provides empirical evidence that TabPFN can serve as a drop-in replacement that fixes these calibration issues without the computational overhead of post-hoc calibration or additional training.