Editable Visual Design: Turning AI Images into Structured, Editable Work

Introduction

Generative image models can now produce visually rich posters, infographics, and promotional graphics. Yet a visually convincing output is not necessarily a production-ready design file. End-to-end diffusion generation usually produces a flattened bitmap, making individual elements difficult to move or replace. Text can also be unreliable. Code-based generation offers the opposite advantage: HTML and CSS provide explicit structure and layout control, but coding agents may struggle with complex visual assets and lack a strong sense of overall aesthetics.

Editable Visual Design proposes a workflow intended to combine these strengths. Instead of asking one model to create a finished image in a single pass, it uses a coding agent to coordinate visual reasoning, asset generation, and structured composition.

Key ideas

• Separate creative responsibilities. The vision-language model acts as the creative brain. It interprets requirements, plans the task, and evaluates the visual result. The image generation model serves as an on-demand visual world simulator for producing standalone illustrations and decorative assets.
• Imagine before acting. The agent first determines which visual elements are needed, then writes native HTML/CSS to combine those assets with real text and layout rules.
• Use rendering as feedback. After the page is rendered, the agent examines the result and iteratively adjusts the code, composition, and asset arrangement. This creates a closed loop rather than a one-shot generation process.
• Deliver an editable artifact. Text remains actual text, while visual elements are kept in decoupled layers. Users can drag objects and adjust layouts through a graphical interface instead of regenerating the entire image.
• Replay the design trajectory. Agent Design Replay is intended to reproduce a creative and reasoning path resembling the workflow of a professional designer, not merely preserve the final frame.

Why it matters

The most important shift is from generating a picture to constructing a design document that can continue to be worked on. In posters and infographics, accurate text, independent elements, and revision flexibility are often as important as initial visual appeal. HTML/CSS provides a structured container, the image model supplies details that are difficult to express through code, and the vision-language model coordinates the two.

The approach also illustrates a broader direction for multimodal agents. Rather than producing a single answer, an agent can call specialized tools, inspect the environment, and revise its output in stages. At the same time, the available material describes the framework and its validation on posters, infographics, and related scenarios, but does not provide specific performance figures. It should therefore be viewed as a design workflow proposal, not evidence that conventional design software has already been replaced.

If developed further, this paradigm could lead to generation tools that combine the aesthetic expressiveness of image models with the structural editability of web code and the planning ability of agents. For designers, AI may become less of a one-click image maker and more of a collaborative production partner that can generate, organize, inspect, and revise visual work.

Source: Hugging Face Daily Papers