UI-Venus-2 Technical Report

What they did

1. The agent works through one unified closed-loop reasoning-action framework across mobile, web, and desktop environments: observe the screen, reason about task state, act, then adapt to feedback.
2. Three dimensions were scaled jointly: environments (170+ multilingual mobile apps plus desktop OS), tasks (deep-research-based instruction generation), and verification (trace-level and sample-level evaluators).
3. Training follows three stages: large-scale trajectory-based mid-training, domain-specific offline reinforcement learning, and multi-teacher on-policy distillation (MOPD) to merge domain experts into one model.
4. In MOPD, reasoning text and executable actions are treated differently, concentrating the distillation signal on the action tokens that actually change the environment state.
5. The system also integrates CAPTCHA-solving capability for login/registration gates and safety-aware mechanisms to control potentially consequential actions.

Findings

• Figure 1 and Tables 1-5 report evaluations of UI-Venus-2-27B and 9B against selected strong baselines on benchmarks including MobileWorld, WebVoyager, Odysseys, VenusBench-CAPTCHA, and VenusBench-GD.
• VenusBench-CAPTCHA was evaluated on 219 examples across eight interaction types (OCR text entry, ordered click, rotation, drag, slider puzzle, etc.) using Pass@1.
• Table 6 reports safety evaluation on the OSHARM and OS-BLIND benchmarks using Attack Success Rate (ASR), where lower is better.
• The paper reports that the action-type-conditioned distillation design in the MOPD stage yields more robust fusion results across diverse GUI domains.

Where it can be used

• Experimenting with automating repetitive mobile/web tasks such as logins and form-filling via screen-based interaction rather than APIs
• Building automation pipelines that need to handle CAPTCHA gates as part of the workflow
• Reference for designing general-purpose agents that also cover desktop OS tasks like file management or office software
• Reference for designing verification mechanisms (trace-level and sample-level, multi-model voting) for reinforcement learning reward signals

Limits and open work

• The paper itself notes that baseline comparisons may use different action scaffolds or evaluation protocols reported by their original sources, so comparisons are not fully apples-to-apples.
• Some benchmarks, such as OSWorld-Verified, are evaluated on a different task count (108 tasks) than the original benchmark's official setting (361 tasks), limiting direct comparability.
• Safety mechanisms were only verified on the OSHARM and OS-BLIND benchmarks; broader real-world risk scenarios are not addressed.
• Multilingual mobile coverage is limited to about 100+ Chinese and 70+ English apps, so performance on other languages/regions is unverified.
• The paper explicitly notes that live-web benchmark results may vary depending on the evaluation date because website states change.

Why it matters

GUI agents that perform well only on narrow benchmarks but break down on real apps, websites, and desktops are a major obstacle to practical automation. This work is notable for jointly scaling environment coverage, task generation, and verification reliability as a way to close that gap.

Terms in this paper

• GUI agent · An AI that operates software by performing human-like actions such as clicking and typing based on what it sees on screen
• Offline reinforcement learning · Reinforcement learning that trains on already-collected interaction data rather than live exploration
• Multi-teacher on-policy distillation (MOPD) · A training method where multiple teacher models score actions sampled by the student model itself to train the student
• Trajectory · The full sequence of screens and actions recorded while an agent performs one task
• VLM as a Judge · Using a vision-language model to evaluate and score the outputs of another model

Latest papers

• GUI-CC: Benchmarking Contextual Consistency of GUI World Models as Agent EnvironmentsGUI world models can fake a plausible single screen, but chain several steps together and they often lose track of which app they're even inarXiv:2609.000482026-09-02
• SHAPE of Chain-of-Thought in Math ReasoningA framework that tags each step of an AI's math 'thinking' with what mathematical move it's making and what interpretation it's stuck in, revealing why some reasoning succeeds and why RL training narrows a model's strategiesarXiv:2608.286002026-09-01
• CoVA-SFT: A Large-Scale Dataset for Chain of Visual AbstractionsA new 52K-example dataset teaches multimodal AI to 'draw while thinking' instead of writing everything in wordsarXiv:2608.289582026-09-01
• CNeo-Bench: Diagnosing Large Language Models on Chinese NeologismsLLMs can often explain what Chinese internet slang like 886, yyds, or 彳亍 means, but many fail to reconstruct the original form behind itarXiv:2608.280532026-08-31
• FIRSTPASS: A Multi-Domain, Multi-Round Peer Review Dataset Grounded in Real Editorial OutcomesA dataset that finally teaches AI what biology, chemistry, and physics peer reviewers actually argue about, not just CS reviewersarXiv:2608.261292026-08-28
• JIT-Agent: Scaling Harness Intelligence via Just-in-Time Harness EvolutionAn AI system that writes a custom 'operating scaffold' for other AI agents on the spot, for every new taskarXiv:2608.255932026-08-27
• The Dialect Tax: Dialectal Biases Persist throughout the Language Modeling PipelineAI language models still charge a hidden 'dialect tax' on AAVE and other non-standard English at every stage, not just tokenizationarXiv:2608.249522026-08-27
• Sycophantic Chatbots Cause Delusional Spiraling, Even in Ideal BayesiansA math model shows that even a perfectly rational person can be talked into delusion by a chatbot that keeps agreeing with themarXiv:2602.191412026-08-26

Latest from METAL LAB

• AI Models Skip Words to Talk Faster — A Fields Medalist Builds the Bridge Between Them
• ATV Big Air Tour cuts inventory work from three days to three hours with ChatGPT Work
• Inworld AI Launches Realtime TTS-2, Reigniting Race for Top Voice Synthesis Spot
• Google opens cyber defense AI to governments and enterprises through Fairwind program
• Perplexity Cited 215,000 Pages From Three AI-Targeted Sites