Rethinking Inference-Time Scaling in Local Computer-Use Agents: Failure Modes and Compute Tradeoffs

This study investigates the effectiveness of inference-time scaling for local computer-use agents (CUAs). While scaling techniques like increasing history length, step budgets, or parallel plan generation have proven effective for large, proprietary models, their utility for resource-constrained local models remains unclear. The authors systematically evaluate these scaling dimensions across four local models on the OSWorld benchmark to determine if additional computation translates into higher task success or simply increases operational costs.

The researchers categorize scaling into four dimensions: contextual (history length), temporal (maximum decoding steps), structural (single-agent vs. two-stage), and parallel (multiple plan generation).

• Contextual Scaling: Providing minimal history is essential for stability, with significant performance gains observed when moving from zero to one screenshot. However, gains saturate at four screenshots; beyond this, additional context increases token costs without improving accuracy.
• Temporal Scaling: Increasing the maximum step budget fails to improve success rates. Instead, it allows agents to persist in erroneous trajectories longer, shifting failures from max-step stalls to premature false successes.
• Structural and Parallel Scaling: Two-stage agents, which decouple planning from grounding, introduce significant overhead. They often fail due to formatting errors or poor plan alignment. While parallel scaling (generating multiple plans) can partially mitigate these issues, it does so at a high computational cost that rarely justifies the marginal gain in success.

As the demand for private, cost-effective, and local AI agents grows, developers must move away from the assumption that "more compute equals better performance." This paper demonstrates that for local models, efficiency is not just about hardware—it is about architectural design. The findings suggest that developers should prioritize moderate history lengths, avoid overly complex two-stage frameworks, and implement failure-aware control mechanisms that can detect when an agent is stuck or has prematurely terminated a task.