Corrado Rainone, Davide Belli, Bence Major, Arash Behboodi
5 min
Abstract
The design space of agentic AI inference spans two extremes: frontier large language models (LLMs), typically hosted in the cloud and offering strong performance across a wide range of tasks at substantially high cost, and more cost-efficient small language models (SLMs), which are amenable to on-device inference. Hybrid multi-agent systems (MASs) combining on-device and cloud models offer a promising middle ground, but they also introduce a complex and poorly understood design space in which task accuracy, monetary cost, and edge energy consumption are tightly coupled; in the absence of general design principles, hybrid components, although not the most prevalent choice, are typically introduced through ad hoc decisions tailored to specific domains. In this work, we examine this design space more systematically. We adapt two representative MAS architectures to support hybrid inference and study how individual design choices shift the operating point along the Pareto frontier of power, cost, and performance. Our findings paint a nuanced picture of hybrid MAS design: while SLMs can effectively benefit from LLM assistance, the optimal architecture is highly task-dependent, and greater frontier-level compute does not consistently translate to better performance.
Alex: Are there different ways to set this up?
Sam: The paper describes two main approaches. In the first, called PEVR, the cloud gives the agent a strict, step-by-step plan upfront and checks whether the agent is following it. If the agent drifts, the supervisor resets the memory and issues a corrected plan. In the second approach, called EVA, the agent works much more independently. The cloud only provides a brief summary and a light nudge when it spots the agent struggling—less like a project manager with a checklist, more like a mentor who trusts you to figure most of it out yourself.
Alex: So one approach is tight control, the other is guided autonomy. Which one works better?
Sam: The research found that neither is always better—it depends on the task. That's actually one of the more significant findings. It means there isn't a single "correct" architecture. The right choice depends on how structured the task is and how much the local agent can be trusted to handle it.
Alex: So the practical takeaway is that these systems need to be matched to the job, not just deployed universally.
Sam: Right. And looking ahead, the researchers suggest the natural next step is systems that can switch between these approaches dynamically—reading the complexity of the task in real time and choosing the appropriate level of cloud involvement on the fly.
Alex: That would be a meaningful improvement. Rather than committing to one management style, the AI decides which kind of oversight it needs at that exact moment.
Sam: Precisely. And that flexibility is what would make these systems genuinely practical for everyday use—capable enough to handle complex tasks, but efficient enough not to drain your battery or compromise your privacy.
Alex: Before we wrap up—what are the honest limitations here? Why aren't systems like this already everywhere?
Sam: The researchers are candid about this. The study covers a specific set of models and scenarios, so we shouldn't assume these results transfer automatically to every situation. Running these simulations is also computationally expensive, which meant the team couldn't conduct as many repeated trials as they would have liked. So it's a solid proof of concept, but not yet a stress-tested universal solution.
Alex: A meaningful step, then, rather than a finished answer.
Sam: That's a fair way to put it. The research shows that decoupling high-level reasoning from the moment-to-moment execution is a viable path forward. The details of how to do that well—across more tasks, more devices, more real-world conditions—that's the work still ahead.
Alex: It's been a grounded look at how we might actually make advanced AI work within the limits of the devices we carry around every day. Thanks for walking me through it, Sam.
Sam: My pleasure. Thanks for listening to ResearchPod.