Quant-Arena Tests Whether LLMs Can Complete the Quant Research Loop

Introduction

A language model that can produce plausible stock-selection code is not necessarily a capable quantitative researcher. In practice, research is an iterative process: a researcher proposes a market hypothesis, turns it into features and executable signals, tests the idea, and decides whether to refine or discard it. Quant-Arena is designed to evaluate that entire loop rather than one isolated skill.

Key points

• The benchmark targets end-to-end work. A participating model must construct features and stock-scoring code, not merely explain a formula or generate a code fragment.
• Research feedback is separated from the final test. Models can use feedback from 2022–2023 while iterating, but the final strategy is frozen before evaluation on the sealed 2024 period.
• The simulator includes trading frictions. The cash-account evaluator applies trading constraints and costs, with market impact estimated through a simulated order book.
• The score has several components. Realized excess return dominates, while signal quality and feature diversity provide additional information.
• The leaderboard result is sobering. Nine frontier models were tested. GPT-6 Astra ranked highest, yet no model surpassed the equal-weight market in 2024.

Why it matters

Quant-Arena addresses a central weakness in many AI evaluations: success on a local task can conceal failure in the full workflow. A model may write syntactically correct code or discover a pattern in historical data, but those abilities do not show that the resulting strategy will survive a new market regime. By withholding the final evaluation year, the benchmark creates a clearer separation between research-time adaptation and out-of-sample performance.

Its trading environment is also important. Backtests that ignore turnover, execution constraints, or market impact can make fragile ideas look investable. A cash-account setup with costs and simulated order-book impact creates pressure for models to produce strategies that are not only statistically attractive but also operationally plausible. The inclusion of signal quality and feature diversity further discourages judging every submission by a single return number.

The findings should nevertheless be interpreted cautiously. One sealed year cannot establish long-term investment skill, and a leaderboard score is not the same as a deployable product. More market periods, assets, and research conditions would be needed to assess robustness. The current result does show a meaningful gap between generating a strategy and producing one that generalizes. Quant-Arena is open to submissions from different models and agent harnesses, giving the field a public setting in which that gap can be measured over time.

Source: Hugging Face Daily Papers