MobileWeave: Program-First Synthesis of Dependency-Linked Tasks for Training Mobile GUI Agents

Ziyi He1*, Xian Wu1*, Qiyuan Chen1*, Haitao Li2†, Cehao Yang3, Yinghao Zhu1, Xichen Zhang4, Xiao Chen5, Yushi Feng1, Lequan Yu1†
1The University of Hong Kong 2LIGHTSPEED 3The Hong Kong University of Science and Technology (Guangzhou) 4The Hong Kong University of Science and Technology 5The Hong Kong Polytechnic University
*Equal contribution    †Corresponding authors
MobileGym SR
43.2
+26.7 vs. Qwen3-VL-8B
AndroidLab SR
64.5
+21.3 vs. Qwen3-VL-8B
MobileWorld SR
27.9
+17.6 vs. Qwen3-VL-8B
Composition-gap split SR
60.1
from 9.2 with atomic-only training

MobileWeave-8B success rates (%). AndroidLab and MobileWorld use apps that never appear in training.

Abstract

Native mobile GUI agents require large-scale training trajectories covering both individual app functions and multi-goal workflows within and across apps, which makes automatic task synthesis a promising way to scale training data. Existing synthesis pipelines, however, produce each task as free-form text, leaving implicit how its subgoals depend on one another. This matters because composition is a skill in its own right: a policy may complete each subtask of a workflow on its own, yet fail to chain these subtasks into the complete workflow.

We present MobileWeave, which instead synthesizes every task as an executable program derived from the app implementations of a controllable simulator. MobileWeave abstracts each app into a typed capability graph with semantic types shared across apps, composes atomic programs into single- and cross-app composite programs through explicit value and object dependencies, and verifies each program by native execution, deriving both its instruction and its completion check from the same program.

Trained on teacher trajectories collected for these tasks and on recovery supervision at the policy's own stalls, MobileWeave-8B improves its base model by 26.7 points on MobileGym and by 8.6, 21.3, and 17.6 points on AndroidWorld, AndroidLab, and MobileWorld, with comparable gains at 4B. Beyond these gains, we find that composition is learnable from task structure: explicit dependencies between subgoals act as a training signal that teaches a policy to compose skills it already masters.

Method

Overview of MobileWeave: capability graph, program weaving with execution and review, and teacher and recovery supervision.
1

Typed capability graph

For each app, a language model extracts entities, semantic types, and operators from the app's state schema, UI code, and native functions. Semantic types are shared across apps, so a meeting's remarks can feed a note's body in another app.

2

Weaving programs

Templates turn the graph into atomic programs, then weave two to four of them into composite programs. Value edges pass a query's output to a later input; object edges point a later step to an object an earlier step selected or created.

3

Verified tasks

Each program is executed natively, without the GUI, and reviewed for a plausible user goal. The same program yields the instruction and the completion check: 822 atomic and 1,376 composite tasks.

4

Trajectories & recovery

A teacher executes the tasks through the GUI, and 1,332 successful trajectories form the SFT set. Loop-targeted recovery then adds one-step corrections at the states where the trained policy itself stalls.

Main Results

Success rate (%) on one Task-OOD benchmark that shares the training simulator and three Env-OOD benchmarks that run on the Android emulator with entirely different apps. All added supervision comes from the MobileGym simulator.

Model Base Task OOD Env OOD
MobileGym AndroidWorld AndroidLab MobileWorld
SRPR
Frontier models
GPT-6.1-Sol–76.285.390.575.476.9
DeepSeek-V4-Flash–47.759.757.850.058.1
Qwen3.8-Max–62.971.785.371.077.8
Gemini-3.1-pro-preview–55.567.870.772.558.1
Qwen3.8-27B–58.673.881.968.170.1
Open-weight models
Qwen3-VL-4BQwen3-VL9.420.144.040.39.1
Qwen3-VL-8BQwen3-VL16.521.747.443.210.3
Step-GUI-4BQwen3-VL9.018.763.947.816.2
MAI-UI-8BQwen3-VL16.828.370.753.627.4
UI-Venus-1.5-8BQwen3-VL15.628.373.755.117.1
GUI-Owl-1.5-8BQwen3-VL17.231.369.035.538.5
Open-data models
ScaleCUA-7BQwen2.5-VL6.615.548.331.27.7
ForgeQwen3-8BQwen3-VL14.822.149.854.310.3
OpenMobile-8BQwen3-VL13.731.564.751.517.7
GUI-Libra-8BQwen3-VL15.625.343.644.914.5
MobileGym-4BQwen3-VL22.334.447.437.717.1
MobileWeave-4BQwen3-VL40.052.152.954.622.8
Gain vs. Qwen3-VL-4B+30.6+32.0+8.9+14.3+13.7
MobileWeave-8BQwen3-VL43.255.456.064.527.9
Gain vs. Qwen3-VL-8B+26.7+33.7+8.6+21.3+17.6

SR: success rate; PR: progress rate. MobileWorld uses the GUI-only setting. Within open-data models, bold and underline mark the best and second-best result in each column. Qwen3-VL and MobileWeave numbers are averaged over eight runs on MobileGym and three runs elsewhere.

Why Program-First?

MobileGym progress rate against the number of training trajectories. MobileWeave: 21.7, 34.5, 40.8, 47.1, 49.3, 50.2 at 0 to 1,000 trajectories. Direct generation: 21.7, 30.2, 35.9, 40.9, 43.5, 45.6.
MobileGym progress rate of Qwen3-VL-8B fine-tuned on MobileWeave vs. direct-generation trajectories.

We compare MobileWeave with a baseline in which a language model reads the same app code and writes complete tasks directly, without a capability graph. Under matched trajectory budgets:

  • With 600 trajectories, MobileWeave tasks reach 38.0 SR on MobileGym, vs. 29.2 for direct generation.
  • MobileWeave stays ahead in PR at every data budget from 200 to 1,000 trajectories.
  • With 600 trajectories, MobileWeave already surpasses the baseline trained on 1,000 (47.1 vs. 45.6 PR).

Composition Is Learnable from Task Structure

A policy trained only on atomic tasks may complete every constituent of a composite task on its own, yet fail on their combination. On 42 such tasks, the composition-gap split, dependency-linked composite training closes most of the gap.

Training dataSplit (42)Other (211)
Atomic only9.283.7
+ 200 independent40.888.9
+ 200 dependency-linked60.192.1

SR (%) on 253 composite tasks whose constituent atomic tasks the atomic-only model completes reliably.

Failed runs per failure type on the composition-gap split. Atomic only / + independent / + dependency-linked: subgoal omission 60/12/13, object-binding error 75/60/30, value-transfer error 38/33/20, premature termination 6/8/2, transition failure 126/86/69.
Failure types on the split. FM 1: subgoal omission; FM 2: object-binding error; FM 3: value-transfer error; FM 4: premature termination; FM 5: transition failure.

Practicing several goals in one episode is enough for the policy to attempt each one: subgoal omissions fall from 60 to 13. The errors between subgoals fall mainly with dependencies. Object-binding errors drop from 75 to 30, value-transfer errors from 38 to 20, and transition failures from 126 to 69.

Case: object binding across apps

Cross-app case: the atomic-only model writes the event's name into the alarm label; the dependency-linked model reads the alarm label and sets it as the calendar event title.
The atomic-only model finds the alarm but writes the event's name into the alarm label and never opens Calendar; the dependency-linked model reads the label and sets it as the event title.

Case: transition between subgoals

Single-app case: the atomic-only model opens the contact's Moments page; the dependency-linked model opens the post editor and publishes the signature.
The atomic-only model reads the signature but then opens the contact's own Moments page; the dependency-linked model returns to Discover, opens the text-post editor, and publishes the signature.

Recovering from Action Loops

Successful trajectories show how to finish a task, but not how to resume once execution stalls. Loop-targeted recovery SFT adds one-step teacher corrections at the policy's own loops.

BenchmarkSR (%) ↑Trajectories with loop ≥ 3 ↓
A + C+ RecoveryA + C+ Recovery
MobileGym40.844.3 +3.595.375.0 −21%
AndroidWorld53.456.0 +2.612.06.7 −44%
AndroidLab59.264.5 +5.315.010.3 −31%

Qwen3-VL-8B. A + C: trained on all 1,332 atomic and composite trajectories; + Recovery: the same model after recovery SFT. Averaged over three runs.

Case: leaving a loop without a teacher

Two Railway 12306 episodes in which MobileWeave-8B repeats an ineffective action, recognizes the cause, and completes the task.
MobileWeave-8B on two Railway 12306 tasks. Top: after five ineffective presses on Submit order, the policy notices that no passenger is selected, selects one, and submits. Bottom: after scrolling five times for the submit button, it dismisses the keyboard that hides it and completes the order. Red dashed frames mark loops; blue frames mark the step that leaves the loop.

What transfers to unseen apps

Failed MobileWorld episodes per category, Qwen3-VL-8B vs. MobileWeave-8B: visual grounding 77 to 56, reasoning and planning 77 to 70, function navigation 32 to 23, attribute omission error 19 to 16, subtask omission 5 to 1.

On MobileWorld, whose apps never appear in training, MobileWeave-8B fails less often than its base model in every error category of the MobileBench-OL taxonomy. The largest drops are in visual grounding (77 → 56) and function navigation (32 → 23), on interfaces the policy has never seen.

BibTeX

@article{he2026mobileweave,
  title   = {MobileWeave: Program-First Synthesis of Dependency-Linked Tasks for Training Mobile GUI Agents},
  author  = {He, Ziyi and Wu, Xian and Chen, Qiyuan and Li, Haitao and Yang, Cehao and Zhu, Yinghao and Zhang, Xichen and Chen, Xiao and Feng, Yushi and Yu, Lequan},
  journal = {arXiv preprint arXiv:2611.23860},
  year    = {2026}
}