A Comparative Study of Bayesian Contextual Bandits for Real-Time Warehouse Sorter Optimization

Warehouse sorters rely on cost functions with static weights to manage item diversion, which often leads to sub-optimal performance in dynamic environments characterized by fluctuating congestion and equipment status. This study investigates whether machine learning frameworks can automatically optimize these weights in real-time to improve operational efficiency.

The authors compared three frameworks: Linear Regression with Gradient Descent (LR+GDO), XGBoost with Bayesian Optimization (XGB+BO), and Bayesian Contextual Bandits (BCB). To overcome the lack of historical data (the cold-start problem), the team used a high-fidelity physics-aware emulator to generate 5,000 training samples. The models were evaluated on their ability to predict rewards based on system context and their capacity to recommend optimal weight configurations that maximize throughput and minimize congestion.

While tree-based models (XGB+BO) showed slightly higher predictive accuracy for reward estimation, the BCB framework emerged as the most effective overall solution. BCB achieved a 2.03% reward uplift over the heuristic baseline and provided several operational advantages: it supports continuous online learning, maintains a strategic balance between exploring new weight configurations and exploiting known high-performing ones, and offers significantly lower inference latency, which is critical for real-time control.

This research demonstrates that Bayesian Contextual Bandits are well-suited for high-frequency industrial control tasks where system dynamics evolve rapidly. By moving away from static, manually tuned cost functions toward a closed-loop, adaptive system, warehouses can better handle stochastic operational conditions, reduce recirculation, and improve overall material handling efficiency.