Continual Test-Time Adaptation in Computer Vision: Methods, Benchmarks, and Future Directions

Overview of Continual Test-Time Adaptation

Deep neural networks typically assume that training and test data are drawn from the same distribution. In real-world deployments—such as autonomous driving or medical imaging—this assumption often fails due to environmental changes, sensor drift, or population shifts. Continual Test-Time Adaptation (CTTA) addresses this by allowing a pre-trained model to update its parameters on-the-fly as it encounters a stream of unlabeled, non-stationary target data.

The Challenges of Online Adaptation

Unlike standard domain adaptation, which often assumes access to source data, or continual learning, which typically has access to labels and multiple training epochs, CTTA operates under strict constraints. The model must adapt using only unlabeled test samples, often limited to a single forward-backward pass per batch. This creates two primary failure modes:

• Catastrophic Forgetting: The model may lose its original source knowledge as it continuously updates to fit new, shifting target distributions.
• Error Accumulation: Because CTTA relies on self-generated signals like pseudo-labels or entropy minimization, errors in early adaptation steps can compound, leading to model degradation over time.

Taxonomy of Methods

The authors categorize existing CTTA research into three distinct families:

1. Optimization-based Strategies: These methods focus on the objective function, utilizing techniques like entropy minimization, pseudo-labeling, and parameter restoration to guide the adaptation process.
2. Parameter-efficient Methods: These approaches minimize the computational footprint by adapting only specific parts of the model, such as normalization layers or selected adaptive parameters.
3. Architecture-based Approaches: These leverage structural modifications, including teacher-student frameworks, adapters, visual prompting, and masked modeling, to maintain stability while enabling plasticity.