Optimizing modern recommender models still depends heavily on engineers manually iterating over architectural, objective, and training-strategy changes. While LLM-based agents can automate this trial-and-error process, allowing the LLM to both select modification directions and generate concrete hypotheses often leads to unstable search under limited experiment budgets. Inspired by the above challenge, we propose RecHarness, a Bandit-Routed Agentic Harness for automated recommender model optimization. RecHarness separates the optimization process into two steps: a bandit router selects the next modification direction according to historical validation feedback, while the LLM generates a concrete optimization hypothesis and executable code edit within the selected direction. To sustain long-horizon exploration, RecHarness uses a jump-basin mechanism to activate a structural-jump arm when local edits stagnate. Across multiple recommendation tasks, datasets, and model backbones, RecHarness achieves more stable performance improvements and uses limited trial budgets more effectively than LLM-reasoning search. During a 7-day online A/B test on a large-scale short-video advertising platform, the selected candidate improves ADVV by 2.084%, Revenue by 0.534%, and Exposure by 0.559%. Code is available at https://github.com/6lyc/RecHarness.
Alex: Welcome to another episode of ResearchPod. Today, we're looking at how artificial intelligence is changing the way engineers build complex systems. Sam, what exactly are we talking about?
Sam: We're discussing a project called RecHarness. It's an automated system designed to help engineers improve recommender systems—the algorithms that decide what videos or products you see next on your feed. The central puzzle is how to let a computer iterate on its own code without burning through enormous amounts of time and money on experiments that lead nowhere.
Alex: So this paper is asking how we can turn an AI into a reliable engineering partner—one that doesn't just guess blindly?
Sam: That's right. In the past, companies relied on human engineers to manually tweak models for weeks, guessing which changes would actually help. If you let an AI guess randomly, it burns through expensive computing power on bad ideas. RecHarness fixes this by separating two very different jobs: deciding where to look, and figuring out how to write the code.
Alex: That sounds like a deliberate division of labor. How does it actually manage that split?
Sam: Think of it like a professional kitchen. You have a Head Chef—what the researchers call the "Bandit Router"—who decides which area to experiment with based on what's worked well in the past. Then you have a Sous Chef—a large language model, or LLM, which is a type of AI trained to read and write text and code—who takes that direction and actually writes the recipe and executes the change.
Alex: So the "Bandit" part is purely about choosing where to focus? Like deciding whether to change how the model is structured versus how it measures success?
Sam: Exactly. The word "bandit" comes from the classic "slot machine" problem in mathematics—imagine you have a row of slot machines, each with a different payout rate, and you need to figure out which ones are worth playing without wasting all your coins on bad ones. The system uses a technique called Thompson Sampling to strike that balance. If one type of code edit has been producing better results, the system naturally puts more of its limited budget into that direction going forward.
Alex: And the LLM—the Sous Chef—is the one actually writing the code once the direction is chosen?
Sam: Yes. The LLM receives the chosen direction along with a running summary of everything that's been tried before—the researchers call this the "Experiment Skill." It looks at the current best-performing model, which they call the "incumbent," and generates a specific code change to try and beat it. It's not throwing darts randomly; it's building deliberately on top of the best version found so far.
Alex: What happens if the system gets stuck in a rut, though? If it keeps making small tweaks and stops seeing improvements, does it just spin its wheels indefinitely?
Sam: That's a well-known failure mode in this kind of search process. To prevent it, the researchers built in what they call a "jump-basin" mechanism. Picture the AI exploring a valley in a landscape of possible model improvements—it can wander around the valley floor, making small refinements. But if it recognizes it has explored that valley thoroughly, it activates a "structural-jump" arm. That forces the system to make a much larger, more disruptive change—essentially leaping out of that valley and landing somewhere completely different to explore.
Alex: So it's a deliberate escape hatch. When small progress stalls, the system is designed to take a bigger risk and try something structurally different.
Sam: That's the core of it. By splitting the search into two modes—careful local refinements and bold structural jumps—the system avoids the trap of over-polishing one small area while missing better solutions elsewhere. The paper suggests this is considerably more efficient than letting an AI wander freely through the entire space of possible changes.
Alex: Did they test this in a real production environment, or was this confined to controlled simulations?
Sam: They ran a seven-day live A/B test on a large-scale short-video advertising platform—meaning real users, real traffic, real consequences. The system improved the value delivered to advertisers by over two percent, while also lifting revenue and content exposure. That's a meaningful result in a domain where even small gains at scale translate to significant real-world impact.
Alex: So the key insight wasn't just giving the AI more raw capability—it was giving it a smarter structure for deciding where to spend its effort.
Sam: Precisely. The contribution here isn't that an LLM can write code—we already knew that. It's the framework around it: the routing logic, the memory of past experiments, the escape mechanism when progress stalls. Together, those pieces turn what would otherwise be a chaotic trial-and-error process into something much more disciplined.
Alex: Are there meaningful limitations to flag?
Sam: Yes, and the researchers are upfront about this. The system operates within a predefined set of "arms"—the categories of changes it can consider—and those categories are specified by human experts at the outset. RecHarness can optimize brilliantly within that space, but it cannot autonomously invent entirely new categories of improvement that weren't anticipated by the humans who designed it.
Alex: So it's a capable partner, but the blueprint still starts with human judgment.
Sam: That's the right framing. It automates the iteration loop and removes a lot of the expensive guesswork, but the strategic vision of what's worth exploring still comes from people. That balance—human expertise setting the boundaries, AI doing the disciplined search within them—seems to be where this kind of system works best. Thanks for listening to ResearchPod.