SeerGuard Brings Pre-Execution Safety to Mobile GUI Agents

Mobile GUI agents are becoming more capable, but their safety problem is also more severe than in many other AI settings. A single mistaken tap, swipe, or confirmation can trigger irreversible consequences such as deleting data, sending money, or submitting the wrong operation. SeerGuard is designed around that reality: instead of reacting after damage is possible, it tries to stop risky behavior before execution.

What SeerGuard does

The framework introduces two layers of protection:

• Instruction-level screening checks whether a user request is malicious, unauthorized, or clearly unsafe before the task starts.
• Action-level risk assessment evaluates each candidate action in the current GUI state and predicts likely consequences before the action is executed.

This is an important shift in safety design. Rather than only judging the surface form of an action, SeerGuard asks what the action is likely to lead to in the current interface context.

The model behind it

To support these safeguards, the authors build a Safety-Augmented World Model (SAWM) with multi-task learning. The model combines two capabilities:

1. Semantic next-state prediction for estimating how the GUI will change after an action.
2. Safety risk assessment for identifying whether that future state implies danger.

By training these tasks together, the system can connect interface dynamics with safety reasoning. That matters because GUI safety is often about context, not just the action text itself.

What the results suggest

The paper reports that SeerGuard generalizes across different mobile GUI agents. On Qwen3-VL-8B-Instruct, the safety-utility score rises from 0.191 to 0.596 at ω=0.8, while the risk-cost score drops from 0.347 to 0.130 at α=0.8. The authors also analyze SAWM to show that instruction filtering, action risk assessment, and next-state prediction all contribute to the final behavior.

Why this matters

SeerGuard is a reminder that agent safety cannot rely only on post-hoc filtering or policy enforcement after an action is already underway. For mobile agents that operate real apps, safety needs to become predictive. A framework that can anticipate consequences before execution is more practical for high-stakes workflows, especially when the wrong action cannot be undone.

In that sense, SeerGuard points to a broader direction for agent safety: world models are not only for planning and reasoning, but can also serve as the basis for proactive risk control.

Source: Hugging Face Daily Papers