Venus Team
4 min
Abstract
Multimodal GUI agents have emerged as a promising paradigm for digital task automation, yet transitioning from benchmark-oriented models to dependable real-world applications remains challenging due to limited environment coverage, brittle task construction, and unreliable reward verification. In this work, we present UI-Venus-2, a general-purpose foundation GUI agent designed to operate across mobile, web, and desktop environments through a unified closed-loop reasoning-action framework. To bridge the gap toward practical deployment, we jointly scale three critical dimensions: (1) Environments, expanding coverage to more than 170 multilingual mobile apps and native desktop operating systems; (2) Tasks, employing a deep-research pipeline for function-grounded instruction generation; and (3) Verification, adopting trace-level and sample-level evaluators with visual keypoints and multi-model voting to ensure reliable RL signals for training. Furthermore, we integrate safety-aware mechanisms to ensure controlled execution of consequential actions. By offering a capable, efficient, and open-source foundation, UI-Venus-2 advances the field toward more generalizable, verifiable, and self-reflective agents for real-world applications.
Alex: Which helps with coverage. But coverage alone doesn't give you a reliable reward signal.
Sam: Correct, and this is the part I'd call the load-bearing mechanism. They use trace-level verification with visual keypoints — essentially checking whether specific visual landmarks on the screen match what a completed step should look like, rather than just checking the final state of the task. That distinction matters a lot for reinforcement learning. A final-state check is easy to fool: the agent can reach a state that looks like success without having actually completed the intermediate steps correctly. Keypoint verification at the trace level makes the reward signal much harder to game.
Alex: So the verification granularity is what keeps the RL from collapsing into reward hacking.
Sam: That's the claim, and it's a reasonable one. The limitation the authors acknowledge is that this whole framework requires a programmatically verifiable environment. If you can't instrument the environment to check visual keypoints at each step, you lose the primary source of reliable supervision. That's a real constraint on deployment scope — it works well in structured digital environments, but it doesn't trivially extend to arbitrary or uncontrolled interfaces.
Alex: And that's worth sitting with. The paper's contribution is a coherent integration of three components that are usually treated separately — but the coherence only holds when all three can be implemented cleanly. Pull out the verification layer and you're back to a much noisier training signal.
Sam: Which is probably why so many prior approaches hit a ceiling. They optimized one component well and left the others underspecified. What UI-Venus-2 offers is a framework where those design decisions are made jointly, and where the interaction between them — particularly between task grounding and step-level verification — is treated as the central engineering problem rather than an afterthought.
Alex: A useful reframe for anyone building in this space. Thanks for walking through it.
Sam: Thanks for listening to ResearchPod.