REDI: Corpus Aware Patch Ranking for DINOv3 Token Reduction

Vision Transformers (ViTs) represent images as sequences of patch tokens, where sequence length directly dictates computational cost. While many methods exist to prune or merge tokens to improve efficiency, it remains unclear how to optimally allocate a fixed token budget across an image. The authors introduce REDI (Relevance for DINOv3 Token Reduction), a framework that treats patch ranking as a supervised retrieval problem. By quantizing DINOv3 patch features into a visual vocabulary, the authors construct a class-conditioned TF-IDF table. This corpus-aware score is combined with image-specific attention maps to rank patches, which are then processed by a fixed operator that keeps, merges, or compresses tokens to reduce the sequence length from 201 to 107 tokens.

Under a controlled evaluation protocol using a frozen DINOv3 backbone and a fixed linear classifier, the REDI-ranked reduced sequence achieves 84.706% Top-1 accuracy on ImageNet-1K. This result surpasses the dense baseline (83.514%) and significantly outperforms models using only attention-based ranking (82.634%) or only corpus-based ranking (81.796%). The authors demonstrate that this performance gain is consistent across four different attention formulations, suggesting that class-specific corpus statistics and image-specific attention provide complementary signals for identifying informative patches.

This study provides a rigorous, controlled framework for evaluating token reduction strategies. By isolating the scoring signal from the reduction mechanism, the authors demonstrate that incorporating external corpus-level knowledge can improve classification accuracy even when the total number of tokens is reduced by nearly 47%. This highlights the potential for future research to develop deployable approximations that capture these complementary signals without requiring ground truth labels or dense pre-computation.