Haoran Ling, Yuecheng Li, Zeyu Song, Jing Yao, Shuwen Kang, Chi Lu, Wenjin Wu, Peng Jiang
5 min
Abstract
Optimizing modern recommender models still depends heavily on engineers manually iterating over architectural, objective, and training-strategy changes. While LLM-based agents can automate this trial-and-error process, allowing the LLM to both select modification directions and generate concrete hypotheses often leads to unstable search under limited experiment budgets. Inspired by the above challenge, we propose RecHarness, a Bandit-Routed Agentic Harness for automated recommender model optimization. RecHarness separates the optimization process into two steps: a bandit router selects the next modification direction according to historical validation feedback, while the LLM generates a concrete optimization hypothesis and executable code edit within the selected direction. To sustain long-horizon exploration, RecHarness uses a jump-basin mechanism to activate a structural-jump arm when local edits stagnate. Across multiple recommendation tasks, datasets, and model backbones, RecHarness achieves more stable performance improvements and uses limited trial budgets more effectively than LLM-reasoning search. During a 7-day online A/B test on a large-scale short-video advertising platform, the selected candidate improves ADVV by 2.084%, Revenue by 0.534%, and Exposure by 0.559%. Code is available at https://github.com/6lyc/RecHarness.
Alex: What happens if the system gets stuck in a rut, though? If it keeps making small tweaks and stops seeing improvements, does it just spin its wheels indefinitely?
Sam: That's a well-known failure mode in this kind of search process. To prevent it, the researchers built in what they call a "jump-basin" mechanism. Picture the AI exploring a valley in a landscape of possible model improvements—it can wander around the valley floor, making small refinements. But if it recognizes it has explored that valley thoroughly, it activates a "structural-jump" arm. That forces the system to make a much larger, more disruptive change—essentially leaping out of that valley and landing somewhere completely different to explore.
Alex: So it's a deliberate escape hatch. When small progress stalls, the system is designed to take a bigger risk and try something structurally different.
Sam: That's the core of it. By splitting the search into two modes—careful local refinements and bold structural jumps—the system avoids the trap of over-polishing one small area while missing better solutions elsewhere. The paper suggests this is considerably more efficient than letting an AI wander freely through the entire space of possible changes.
Alex: Did they test this in a real production environment, or was this confined to controlled simulations?
Sam: They ran a seven-day live A/B test on a large-scale short-video advertising platform—meaning real users, real traffic, real consequences. The system improved the value delivered to advertisers by over two percent, while also lifting revenue and content exposure. That's a meaningful result in a domain where even small gains at scale translate to significant real-world impact.
Alex: So the key insight wasn't just giving the AI more raw capability—it was giving it a smarter structure for deciding where to spend its effort.
Sam: Precisely. The contribution here isn't that an LLM can write code—we already knew that. It's the framework around it: the routing logic, the memory of past experiments, the escape mechanism when progress stalls. Together, those pieces turn what would otherwise be a chaotic trial-and-error process into something much more disciplined.
Alex: Are there meaningful limitations to flag?
Sam: Yes, and the researchers are upfront about this. The system operates within a predefined set of "arms"—the categories of changes it can consider—and those categories are specified by human experts at the outset. RecHarness can optimize brilliantly within that space, but it cannot autonomously invent entirely new categories of improvement that weren't anticipated by the humans who designed it.
Alex: So it's a capable partner, but the blueprint still starts with human judgment.
Sam: That's the right framing. It automates the iteration loop and removes a lot of the expensive guesswork, but the strategic vision of what's worth exploring still comes from people. That balance—human expertise setting the boundaries, AI doing the disciplined search within them—seems to be where this kind of system works best. Thanks for listening to ResearchPod.