AdaCount: Training-Free Similarity-Guided Spatial and Feature Adaptation for Zero-Shot Object Counting

Zero-shot object counting (ZOC) aims to count objects based solely on text prompts, eliminating the need for expensive, task-specific training data. While foundation models like SAM3 have advanced this field, they often struggle in dense scenes with many small objects, where limited resolution and lack of focus on target-relevant regions lead to significant undercounting. AdaCount addresses this by introducing a training-free adaptation pipeline that enhances the model's ability to resolve and count objects in crowded environments.

AdaCount operates in a two-pass inference pipeline. In the first pass, it uses SAM3 to discover high-confidence target instances, which are then used to construct image-specific prototypes in the feature space. These prototypes are correlated with the encoder feature map to generate a similarity map that identifies target-relevant regions. This map guides two key adaptations:

1. Similarity-Guided Spatial Warping: The model redistributes image resolution, effectively 'zooming in' on target-relevant regions while compressing background areas, all within a fixed-resolution canvas.
2. Feature Modulation: Using a residual gating strategy, the model amplifies encoder feature responses in regions identified as target-relevant, ensuring the decoder receives more discriminative information.

These adaptations allow the model to allocate more representational capacity to relevant objects while preserving global context, without requiring any model retraining or fine-tuning.

AdaCount establishes a new state-of-the-art for training-free ZOC across six diverse benchmarks. Unlike concurrent methods that rely on heuristic-based adaptive tiling—which can be computationally expensive and inconsistent—AdaCount provides a predictable, efficient, and robust solution. By demonstrating that inference-time spatial and feature adaptation can significantly boost the performance of existing foundation models, this work offers a scalable path for deploying object counting in real-world applications where training data is scarce.