TabRank: Chain-of-Thought Distillation for Table Re-Rankers

As multi-stage retrieval systems become the standard for structured data, the effectiveness of the reranker—the component that refines candidate lists—is critical. While Large Reasoning Models (LRMs) have improved passage retrieval through chain-of-thought (CoT) reasoning, applying these techniques to tabular data is challenging. Tables require reasoning over structural properties like schemas and row alignments, and existing CoT distillation methods often suffer from overfitting to specific reasoning trajectories, which degrades cross-domain generalization.

The authors introduce TabRank, a framework for training reasoning-aware rerankers. Instead of the standard approach (CoTGen), where a student model is trained to autoregressively reproduce a teacher's reasoning trace, TabRank uses a conditional distillation strategy (CoTCond). In this method, teacher-generated reasoning traces from DeepSeek-R1 are prepended to the input prompt as context. The student model is then trained only to predict the final ranking, with no loss computed over the reasoning tokens themselves. This decouples the benefits of reasoning guidance from the overhead and potential overfitting of imitating specific reasoning sequences.

TabRank significantly outperforms both standard supervised fine-tuning and traditional CoT distillation across diverse out-of-distribution benchmarks. Specifically, it achieved substantial improvements in Accuracy@10: 30.5% on HybridQA, 15.2% on SQA, 52.9% on TabFact, and 13.1% on TATQA. The authors demonstrate that conditioning the model on reasoning context is consistently superior to forcing the model to generate that reasoning. Furthermore, TabRank generalizes effectively to multi-table retrieval scenarios despite being trained exclusively on single-table data, and it does so with lower inference latency than CoTGen-style models.