GPUSparse: GPU-Accelerated Learned Sparse Retrieval with Parallel Inverted Indices

Learned sparse retrieval models like SPLADE provide high-quality, interpretable search results but are bottlenecked by CPU-bound inverted index traversal algorithms (such as WAND or Block-Max WAND). These algorithms are inherently sequential and difficult to parallelize on GPUs. This paper asks whether it is possible to achieve GPU-accelerated exact sparse retrieval that matches the performance and throughput of dense vector search without sacrificing the precision of sparse models.

The author introduces GPUSparse, a system designed to move sparse retrieval scoring from the CPU to the GPU. The system replaces sequential index traversal with a batched scatter-add scoring algorithm. Key technical contributions include:

• A GPU-parallel inverted index that uses block-aligned, warp-coalesced posting lists to optimize memory access.
• A custom fused Triton kernel that combines posting list traversal, score accumulation, and atomic scatter-add operations into a single GPU launch, minimizing memory overhead.
• A detailed analysis of the tradeoff between work-efficiency (processing only relevant terms) and bandwidth-efficiency (utilizing GPU memory throughput).

GPUSparse achieves exact scoring on the MS MARCO passage ranking dataset (8.8M passages) with a Recall@1000 of ≥0.999 compared to CPU ground truth. It delivers a 235x speedup over Pyserini CPU implementations, achieving 1.3ms per query at 787 queries per second (QPS). The study demonstrates that while a document-parallel kernel can achieve higher HBM bandwidth utilization (62.6% of H100 peak), the scatter-add approach is significantly faster in practice because it avoids the massive redundant computation of scanning every document in the collection.

This work bridges the performance gap between sparse and dense retrieval on modern hardware. By enabling GPU-native sparse retrieval, it allows developers to leverage the interpretability and exact-match benefits of sparse models without the latency penalties that have historically made them difficult to deploy in high-throughput, real-time production environments.