VibeLifeBench Tests Whether Life Agents Can Stay Proactive for Weeks

Introduction

Many AI assistants are being positioned as personal agents, but most benchmarks still ask them to solve short, self-contained tasks in static sandboxes. VibeLifeBench targets a harder question: can an agent manage an everyday-life task over several weeks, while the environment changes in the background and important rules are never explicitly spelled out?

Key ideas

• A task is a timed world, not just a prompt. VibeLifeBench contains 200 long-horizon tasks across ten everyday-life domains. Each task is a scripted multi-week timeline, with a median duration of 29 days, running over 22 mock services and 288 tools.
• Important events can happen silently. The benchmark includes 1,483 silent mutations, such as a seat being voided, a flight becoming delayed, or a phishing email arriving in an inbox. These events do not arrive as direct notifications; an agent has to revisit the world on its own to discover them.
• Implicit constraints matter. Real-life assistance often depends on rules that are not stated in the user’s request: whether a passport satisfies a six-month validity requirement, whether insulin needs a customs letter, or whether choices remain under a hard budget cap. VibeLifeBench grades these constraints even when they are not explicitly given.
• Evaluation looks at what the agent leaves behind. The benchmark uses 12,261 weighted checks that inspect the agent’s observable outputs and environment state, not hidden reasoning. The checks cover final outcomes, timing, and whether implicit requirements were respected.

Why it matters

The reported results are sobering. Across seven frontier models, scores remain low. The best reported model, Claude Opus 5, reaches an avg@3 score of 32.5, and every model drops by 10 to 15 points from the beginning of a timeline to the end. In the flagship 20-day Japan trip task, not a single run refused the phishing email.

This suggests that fluent single-turn tool use is not enough for dependable life assistance. A useful personal agent must decide when to act, when to ask for clarification, and when to stay quiet. It also has to notice unannounced changes and keep one plan coherent from the first day to the last. By open-sourcing the tasks, environments, and evaluation framework, VibeLifeBench could help shift agent research from short task completion toward persistent, proactive, real-world delegation.

Source: Hugging Face Daily Papers