PISA-E Makes LLM Ensembles Adaptive to Prompts and Cost

Introduction

Serving large language models is not only a question of maximizing answer quality. Web applications receive heterogeneous requests, and each request can justify a different balance between capability, latency, and inference expense. Always using the strongest available model may waste resources, while routing every prompt to a single model can leave quality gains from collaboration unused.

Most routing systems therefore ask which model should handle a prompt. Ensemble routing adds a second layer of decisions: which models should be combined, and how many calls are worth making? A new study introduces PISA for prompt-adaptive single-model routing and PISA-E for adaptive ensemble routing.

How the approach works

• Prompt-specific capability estimates: PISA combines soft cluster memberships with model-specific predictors. Rather than relying on a fixed global ranking, it estimates the likely usefulness of each candidate model for the current prompt.
• Joint ensemble selection: PISA-E uses those individual predictions to choose both ensemble composition and cardinality. A runtime quality–cost preference can change the operating point without retraining the router.
• No ensemble-specific training: The method relies on a tractable surrogate for ensemble success and an efficient candidate-selection procedure. This allows it to construct ensembles from individual-model estimates instead of learning a separate predictor for every possible combination.
• Distribution-shift evaluation: The authors test the methods on four LLM routing benchmarks under both in-distribution (ID) and out-of-distribution (OOD) conditions.

Results and implications

For single-model routing, PISA achieves the highest macro-average cost–performance hypervolume among the evaluated routers in both ID and OOD settings. The result indicates that its advantage is not limited to one high-quality operating point; it also reflects performance across a range of cost choices.

For ensemble routing, PISA-E records the best macro-average task performance among the evaluated ensemble approaches when quality is prioritized. It exceeds the strongest competitors by 2.09 percentage points in ID evaluation and 2.87 points in OOD evaluation. At selected lower-cost operating points, it also surpasses the strongest competing ensembles while reducing average inference cost by approximately 36% in ID and 25% in OOD settings.

The broader contribution is a shift from fixed ensembles to request-dependent collaboration. Easy prompts may be handled by a small, inexpensive set of models, while more demanding prompts can justify a larger ensemble. This could give multi-model serving platforms a practical control layer for managing quality and spending. The supplied material does not detail the benchmark task mix, model roster, or latency measurements, so the reported gains should be read as evidence for the routing formulation rather than a universal deployment guarantee.

Source: Hugging Face Daily Papers