AMD Uses Hierarchical Memory to Boost Small Agent Models

Small language models often struggle as agents not because they cannot reason at all, but because they lack enough successful tool-use experience to lean on. AMD addresses that gap by extracting reusable knowledge from a strong teacher and packaging it as hierarchical memory, rather than relying on another round of training.

What AMD adds

The framework, called Agent Memory Distillation (AMD), turns successful teacher trajectories into three complementary memory types:

• Workflow memory captures task-level strategies and overall execution order
• Subtask memory provides concrete behavioral examples at an intermediate level
• Function memory records per-function calling conventions, common mistakes, and pitfalls

These memories are injected in two ways. Workflow and subtask memories are given proactively at the start of a task, helping the student form a plan. Function memory is retrieved reactively when tool-calling errors happen, so the model can recover from failures more directly.

Why this matters

Tool-using agents need more than generic language ability. They must plan, choose tools, follow APIs, and adapt when calls go wrong. For small models, this is difficult because they cannot easily produce enough high-quality trajectories on their own. AMD’s design splits the problem into manageable layers, making prior experience easier to reuse at inference time.

What the experiments show

The paper evaluates AMD on three tool-use benchmarks: AppWorld, BFCL V3, and ToolSandbox. Four student models in the 4B to 8B range are tested, with GPT-5-mini as the teacher. AMD outperforms existing memory-based baselines and delivers notable average accuracy gains across the benchmarks. The strongest contribution comes from subtask memory. The authors also find that teacher quality is not enough by itself: the teacher must also fit the student well, and 4B-sized students benefit the most.

Takeaway

AMD shifts distillation from weights to memory. That is a useful idea for agent systems, where execution quality often depends on having the right procedural hints at the right time. If memory can be structured, selected, and injected well, smaller agents may become much more capable without expensive retraining.

Source: Hugging Face Daily Papers