Route Anything: Teaching General-Purpose AI to Choose the Right Model

Introduction

Building general-purpose AI does not necessarily require one giant model to handle every capability. An alternative is to assemble a system from specialized models, tools, and execution policies. Such a system could send a language question, an image request, a coding task, or a robotic manipulation command to different candidates instead of forcing one model to do everything.

The paper Route Anything examines the decision layer behind this vision. Its central question is how a router should select and coordinate candidates while trading off task quality against computational cost. The goal is not simply to identify the strongest model overall, but to determine which candidate is best for a particular input and under the information available to the router.

Key findings

• A common framework for models and policies. The study treats models, tools, and policies as candidate configurations and develops a theoretical formulation for routing under quality–cost trade-offs. It also analyzes how the information exposed to a router limits or expands its attainable performance.
• A broad omni-modal evaluation matrix. The researchers assemble trajectories from 25 benchmarks spanning language understanding, image, video and speech generation, coding and workplace agents, and robotic manipulation. The matrix contains 374,401 scored runs across 452 candidate configurations.
• Oracle routing reveals substantial complementarity. When the best candidate can be selected separately for each input, routing outperforms the strongest fixed candidate by 5.4 points. This result indicates that different candidates have complementary strengths rather than a single universally optimal model.
• Learned routers show measurable gains. Across 16 benchmarks, learned routing improves on the best fixed candidate by 0.3 points. The gains reach 2.1 points for video generation and 1.1 points for robot manipulation, while policy time falls by 24%.
• Efficiency may be the larger opportunity. The paper reports an average 8.82× relative efficiency gain across four settings. That number should be read in the context of the study’s reference model, cost definition, and evaluation setup rather than as a universal deployment guarantee.

Why it matters

Route Anything frames routing as a first-class systems problem instead of an ad hoc serving trick. In a heterogeneous AI ecosystem, a routing layer could send easy requests to cheaper candidates, reserve stronger systems for difficult cases, and choose different policies when a task crosses modalities or requires tool use. This may lower average inference cost and make better use of a diverse model portfolio.

The approach is particularly relevant to multimodal assistants and embodied agents. Their workloads are uneven: a text query, a video generation request, and a manipulation action have very different capability and latency requirements. A single fixed model is unlikely to be optimal across all of them, while a routing layer can exploit these differences.

Important questions remain open. The supplied summary does not specify every router input, training procedure, or serving overhead. In practice, routing errors, shifting candidate quality, and transfer between benchmarks may reduce the reported benefits. The router itself must also be fast and reliable enough not to erase the savings it creates.

The broader lesson is not that large general models are becoming unnecessary. Rather, as the ecosystem of specialized models grows, deciding how to select and combine them may become as important as scaling any individual model. Routing could become a foundational layer in the inference stack for general-purpose AI.

Source: Hugging Face Daily Papers