Back to articles
Evaluation & Benchmarks

TurtLES: A Large-Scale Benchmark for 3D Turbulent Neural PDE Surrogates

3 min read

Why this benchmark matters

Machine-learning surrogates are increasingly considered as faster alternatives to repeated computational fluid dynamics simulations. Yet many existing datasets simplify the problem through regular grids, short sequences, lower-dimensional fields, or relatively mild flow conditions. Those settings are useful for prototyping, but they do not fully test whether a model can preserve the dynamics of realistic three-dimensional turbulence over time.

TurtLES is designed to address that gap. It is a 14 TB benchmark built from high-fidelity implicit large-eddy simulations of turbulent cylinder wakes in the shear-layer transition regime. The dataset is intended as a demanding testbed for neural PDE surrogates, particularly models that must autoregressively advance a flow state for many steps.

What is included

  • 400 trajectories: Each trajectory contains 400 time steps, providing a long temporal horizon for rollout evaluation.
  • Large irregular fields: Individual frames contain roughly 3 to 11 million points and are represented on dense irregular meshes rather than simple Cartesian grids.
  • Physical and geometric variation: The cases span Reynolds numbers from 1000 to 3900, four cross-section shapes, and angles of attack between 0 and 45 degrees.
  • Turbulent structure: The simulations retain physically realistic turbulence and an active energy cascade driven by vortex stretching.
  • Smaller entry points: A mini-TurtLES collection derived from about 0.8 TB of data, together with a 5 GB debug subset, gives researchers more manageable ways to begin experimentation.

What the evaluations reveal

The benchmark does more than report a single accuracy score. It examines whether neural operators can continue producing a plausible flow during autoregressive rollouts. That distinction is important: a model may predict the next frame reasonably well while still accumulating errors that eventually destroy the trajectory.

The reported failure modes fall into three broad categories. First, rollouts can become temporally unstable as prediction errors compound over time. Second, models may lose energy at small spatial scales, producing fields that are overly smooth even when their large structures look plausible. Third, learning on very large irregular domains remains difficult, exposing limitations in architectures and training strategies developed mainly for regular or smaller inputs.

These observations challenge the common practice of judging a surrogate primarily by one-step error or an average field metric. Long-horizon consistency, spectral or scale-dependent behavior, and the ability to operate on irregular geometry are separate capabilities that need to be measured explicitly.

Broader significance

TurtLES is valuable because it brings physical realism, geometric complexity, and long-term dynamics into one evaluation setting. It can support more meaningful comparisons among neural operators, graph-based solvers, and geometry-aware architectures. It also provides a concrete stress test for claims that a surrogate is ready to replace expensive CFD simulations.

The benchmark does come with practical costs. The full 14 TB release requires substantial storage and data movement, and results on the smaller subsets should not automatically be treated as equivalent to results at full scale. Even so, the mini and debug versions can lower the barrier for implementation and debugging.

The central message is straightforward: a useful turbulent-flow surrogate must do more than fit snapshots. It must remain stable across long rollouts, preserve important scales of motion, and handle the irregular, high-resolution domains found in serious three-dimensional simulations.

Source: Hugging Face Daily Papers

Comments

Checking sign-in status...

Loading comments...

Related articles

CCTest · Blog
Why Do Multimodal Models Change Their Answers When a Spectrum Is Rewritten?
Evaluation & Benchmarks
cctest.ai

Why Do Multimodal Models Change Their Answers When a Spectrum Is Rewritten?

A study of optical-spectrum decision tasks finds that changing only the representation can shift one model’s accuracy by as much as 41 percentage points. It also shows that models often answer despite missing evidence, while fine-tuning can create a shortcut in which every truncated input triggers abstention.

Read more