On the Faithfulness of Post-Hoc Concept Bottleneck Models

Post-hoc Concept Bottleneck Models (post-hoc CBMs) aim to make deep learning models interpretable by projecting latent features onto human-understandable concepts. However, researchers often rely on downstream classification accuracy to validate these models. This paper investigates whether such accuracy is a valid proxy for the semantic faithfulness of the learned concepts. The authors analyze two common training paradigms: using auxiliary concept datasets and using surrogate labels generated by vision-language models (VLMs). They formalize the conditions for faithfulness and propose new metrics to detect when concept projections fail to align with ground-truth semantics.

The study demonstrates that high downstream accuracy is insufficient to guarantee concept faithfulness. Using the manifold version of the Johnson-Lindenstrauss lemma, the authors show that even random, semantically meaningless concept projections can preserve enough information to allow a downstream classifier to achieve competitive performance.

Furthermore, the authors identify two critical sources of unfaithfulness:

1. Covariate Shift: When projections are learned on auxiliary datasets, differences between the auxiliary and target distributions can invalidate the learned concepts. The authors provide an upper bound on the resulting generalization error and a practical metric based on domain discrimination to quantify this shift.
2. Systematic Surrogate Noise: When using VLMs to generate surrogate labels, the authors show that unfaithfulness is not caused by the magnitude of noise, but by its systematicity. They derive an orthogonality condition: a model remains faithful if surrogate errors are uncorrelated with backbone activations. They introduce a correlation-based metric to detect when VLM-generated labels are systematically biased.

This work challenges the common practice of using task accuracy as a primary evaluation metric for interpretability. By providing formal definitions and practical metrics for faithfulness, the authors offer researchers a way to verify whether their concept-based models are truly transparent or merely masking opaque decision-making processes. This is essential for building trust in AI systems used in high-stakes domains.