MaxKernel Uses Multi-Agent Search to Build Faster TPU Kernels

Writing a fast accelerator kernel is very different from producing ordinary application code. Developers must reason about the mathematical operation, memory movement, parallel execution, compiler constraints, and the behavior of a particular chip. MaxKernel addresses this challenge by placing large language models inside a feedback-driven development loop. Instead of generating code once, the system proposes an implementation, compiles it, tests it, profiles it, and uses the observed results to guide the next attempt.

Three ways to search for a kernel

MaxKernel is built as a multi-agent system. A shared pool of specialized agents handles planning, implementation, self-debugging, correctness testing, and hardware profiling. On top of that shared infrastructure, the paper describes three development paradigms:

• Human-in-the-loop: The model and a developer work through the design step by step. Human input can steer important choices and makes the mode suitable for collaborative development.
• Autonomous optimization: The agent runs an automated loop driven by performance metrics and hardware traces, reducing the amount of manual trial and error.
• Graph-based autonomous search: Multiple candidate implementations and optimization paths are organized as a search graph. This allows the system to explore alternatives more broadly instead of relying on a single sequence of edits.

The three modes reflect different practical needs. Engineers may want direct control when the design is still being understood, while repetitive benchmarking and tuning are natural candidates for automation. Graph search extends the latter idea by treating kernel development as exploration of a design space rather than a linear editing task.

Compiler feedback becomes part of the reasoning loop

Static knowledge alone is rarely enough to optimize TPU kernels. Two programs with identical mathematical behavior can perform differently because of layout decisions, tiling, memory access patterns, or compiler transformations. MaxKernel therefore treats compilation results, test outcomes, and profiling data as evidence that the agents can use to revise their plans.

The system is evaluated on JaxBench, a suite of 50 diverse TPU kernel tasks, and on complex workloads drawn from advanced open-source models. The paper reports that MaxKernel consistently produces highly optimized implementations and can match expert hand-tuned baselines across the benchmark. The provided material does not include detailed speedup figures, so the result is best read as evidence for the overall workflow rather than as a claim about one specific record-setting kernel.

Why it matters

MaxKernel illustrates a division of labor in which agents organize experiments, compilers provide immediate technical feedback, and the hardware serves as the final performance judge. Such a workflow could lower the barrier to specialized kernel development and offer a template for automatic tuning across different AI accelerators.

There are also open questions. Search can be expensive, profiling may expose only part of the hardware behavior, and correctness tests may not cover every edge case. The ability to transfer the approach to new architectures, unfamiliar operator combinations, and production workloads requires further evidence. Since MaxKernel is open source, its agent roles, search procedures, and evaluation setup can be examined and extended by the community.

Source: Hugging Face Daily Papers