Teaching Agents to Remember: Self-Direction Turns Mistakes into Reusable Skills

Introduction

An agent that can recover once is not necessarily an agent that learns. Many current systems can inspect a failed trajectory, revise their plan, and try again, but the lesson often disappears when the session ends. Full-parameter fine-tuning offers a more durable alternative, yet it is expensive and can interfere with capabilities the base model already has. The paper Don't Forget What You Just Learned presents Self-Direction as a middle ground between temporary reflection and permanent model updates.

How the mechanism works

When an agent fails, Self-Direction asks it to examine its own trajectory and extract a concrete lesson from the failure. That lesson is then distilled into a lightweight low-rank adapter rather than written directly into the base model. The adapter is stored in a persistent library of lessons.

When a later task resembles the earlier one, the system automatically routes the relevant adapter to the agent. The original model weights remain frozen, limiting the risk that learning one correction will overwrite existing behavior. According to the paper, each adapter introduces trainable parameters equivalent to roughly 0.02% of the base model’s parameter count, making the approach substantially lighter than full fine-tuning.

What the experiments show

The authors evaluate Qwen-family open-weight agents on three settings: open-ended research with GAIA, tool-use workflows with Claw-Eval, and terminal tasks with Terminal-Bench 2.1. The reported overall success-rate gains range from 8.9 to 17.0 percentage points, and the resulting systems perform competitively with frontier systems.

The paper highlights two further findings. First, agents repeat fewer mistakes when they encounter related tasks again, with the reliability of corrections nearly doubling. Second, a small number of learned adapters can generalize to dozens of new tasks without additional training. This suggests that the stored lessons may capture reusable procedures rather than merely memorizing individual examples.

Why it matters

Self-Direction reframes reflection as a persistent operational asset. For research assistants, tool-using agents, and terminal automation, a library of targeted adapters could reduce recurring failures without requiring the whole model to be retrained after every incident. It also provides a modular path to personalization: different lessons can be added, selected, or potentially removed without changing the base model.

The available material does not establish how the library behaves at large scale, how unreliable lessons are detected, or how conflicting adapters should be handled. Those questions will matter in deployment, since a persistent memory system can preserve useful corrections but may also preserve bad ones. Self-Direction is therefore best understood as a promising architecture for durable agent learning, not a complete solution to continual learning.

Source: arXiv