AnTrap Exposes the Runtime Fragility of Android GUI Agents

Introduction

A mobile GUI agent may appear competent when it can identify a screen and perform a prescribed sequence of taps, swipes, or text inputs. Real Android devices, however, rarely preserve such a clean execution path. A pop-up can interrupt the interface, an action can fail silently, and an earlier mistake can leave the application in a state that is inconsistent with the agent’s internal assumptions. The paper “Are Android GUI Agents Robust Against Runtime Anomalies?” introduces AnTrap to examine this gap systematically.

Key points

• The benchmark evaluates trajectories, not just outcomes. AnTrap injects runtime perturbations while an agent is carrying out a task. The construction process is designed to keep the task solvable, so failure is more likely to reflect the agent’s inability to detect or recover from an anomaly than an impossible environment.
• Anomalies are organized across four layers. State, Thinking, Action, and Round provide a structured taxonomy with ten fine-grained subcategories. The framework therefore covers more than visual changes: it also captures flawed interpretation, inappropriate operations, and errors that accumulate over multiple rounds.
• Vulnerability is widespread. Evaluations of 16 leading GUI models show that dynamic anomalies cause substantial degradation across the model set. Stronger agents are not immune; they may continue reasoning from an outdated screen state or repeat an action that is no longer valid.
• Training does not solve every class of failure. The authors use GRPO in both original and adversarial environments to separate environment-learnable problems from reasoning bottlenecks. Some one-step traps at the state and action layers can be improved with adversarial reinforcement learning. Deeper contextual traps, including state deadlock, expose limitations that cannot be removed simply by adding more trap-filled training episodes.

Why it matters

AnTrap broadens the meaning of reliability for mobile agents. Completing a task in a static benchmark is not equivalent to maintaining a correct execution loop on a live device. A dependable agent must repeatedly verify the current state, check whether an action actually took effect, recognize when its plan no longer matches the interface, and recover without losing the task objective.

The benchmark also offers a useful way to interpret reinforcement-learning results. Improvement after adversarial training does not necessarily mean that an agent has acquired general reasoning. It may have learned how to respond to recurring, local disruptions. By contrast, a deadlocked state or a failure that depends on a long chain of prior events requires more than a memorized reaction. It calls for state tracking, causal judgment, and a recovery policy that can revise the plan.

For researchers, AnTrap provides a pressure test for moving GUI agents beyond polished demonstrations. For developers, it suggests that anomaly detection and post-action verification should be built into the execution loop rather than treated as optional fallback features. The paper’s central message is straightforward: robustness must be measured under changing conditions, and some of the hardest failures are not merely environmental mistakes but evidence of unresolved reasoning limits.

Source: Hugging Face Daily Papers