ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction

Neural information retrieval (IR) models generally fall into two categories: single-vector models, which are efficient but often lack nuance, and late-interaction models (like the original ColBERT), which provide superior retrieval quality by comparing token-level representations but suffer from a massive memory footprint. This paper asks whether it is possible to combine the high performance of late-interaction models with the storage efficiency of single-vector models.

The authors introduce ColBERTv2, which optimizes late-interaction retrieval through two primary innovations. First, they implement a denoised supervision strategy that uses distillation from a cross-encoder and hard-negative mining to improve the model's ability to distinguish relevant from irrelevant passages. Second, they apply residual compression to the token-level embeddings. By clustering these embeddings into centroids and storing only the quantized residual (the difference between the original vector and its nearest centroid), they significantly reduce the space required to store the index without sacrificing retrieval accuracy.

ColBERTv2 establishes state-of-the-art retrieval quality across both in-domain and out-of-domain benchmarks. The authors demonstrate that their residual compression technique allows for a 6–10x reduction in the space footprint of late-interaction models, making them competitive with single-vector models in terms of storage. Furthermore, the authors introduce LoTTE, a new benchmark for evaluating retrieval models on long-tail, domain-specific topics, where ColBERTv2 consistently outperforms existing methods.

This work bridges the gap between high-performance, resource-heavy retrieval models and efficient, scalable systems. By demonstrating that late-interaction models can be both accurate and lightweight, the authors provide a practical path for deploying sophisticated neural search in real-world applications where storage and memory are constrained.