EvoWeb Tests Whether Coding Agents Can Evolve Web Apps Reliably

Introduction

Generating a web page from a single specification is useful, but it captures only a narrow slice of software development. Real products evolve through a stream of follow-up requests. Each change must fit an existing codebase, preserve earlier behavior, and remain usable after several rounds of modification. EvoWeb is designed to measure that longer and more realistic process.

What the benchmark changes

• Cumulative rather than isolated tasks. EvoWeb includes 32 web applications and 320 requests. Later requests build on earlier implementations, turning the benchmark into a sequence of dependent engineering decisions rather than a collection of independent generations.
• Functional dependency graphs. The requests are organized around dependencies between features. This makes it possible to study how performance changes when later work relies on a deeper or larger set of previous capabilities.
• Browser-based, multimodal evaluation. A multimodal web agent tests every intermediate application in a live browser. It checks whether the newly requested behavior works, whether it integrates with existing functionality, and whether previously completed features still operate.
• Trajectory-level reliability. Instead of choosing one universal stopping point, EvoWeb integrates performance across user-tolerance thresholds. The design is intended to describe reliability throughout development, not just at a final checkpoint.

What the experiments show

Experiments with open- and closed-source frontier models reveal meaningful differences between systems. At the same time, the central difficulty is shared: agents often struggle to make a new request work correctly with everything that came before it. The challenge is therefore not limited to producing a new component or interaction. It includes understanding the current state of the application, changing the right parts, and avoiding damage to established behavior.

The paper also reports that later requests with deeper or more numerous dependencies are solved less accurately. Runtime errors become more common as the sequence advances. These findings expose a weakness in one-shot evaluation: a system may produce a convincing first version while gradually accumulating incompatibilities and failures during maintenance.

Why it matters

EvoWeb shifts the question from “Can an agent generate a working web app?” to “Can an agent keep a web app working while it evolves?” That distinction matters for coding assistants intended for real product teams, where requirements arrive incrementally and every change carries maintenance costs.

For model developers, the benchmark can help isolate weaknesses in planning, context management, regression testing, and recovery from runtime failures. For users, it highlights the need for workflows that verify each change instead of trusting a single generated result. Its emphasis on persistent codebases and live-browser checks also offers a more grounded way to compare agents across a development trajectory.

As coding agents take on longer tasks, cumulative evaluation may become as important as measuring their ability to solve isolated tickets. EvoWeb provides one concrete framework for studying that transition and for making cross-request consistency a first-class target.

Source: Hugging Face Daily Papers