JEPA-Anything: A Factorized Route to World Models Across Domains

Introduction

A world model is meant to help an intelligent system anticipate what will happen after an action, rather than merely describe the current observation. That goal becomes difficult when the underlying worlds are very different. Images contain visual structure, clinical records encode patient trajectories, molecular systems follow physical interactions, and weather or field data evolve continuously over time.

Most predictive models are still designed around one domain and one type of signal. JEPA-Anything, listed by Hugging Face Daily Papers, asks whether a shared learning principle can support world modeling across such radically different systems. Its answer is a framework built around Orthogonal Predictive Factorization, or OPF, within a joint-embedding predictive architecture.

What the method changes

JEPA-style systems predict representations in a latent space instead of reconstructing every detail of a future observation. JEPA-Anything adds a structural assumption: a future latent target may contain several complementary factors that should be modeled separately rather than mixed into one undifferentiated prediction.

• Factorized targets: OPF decomposes latent targets into complementary components.
• Dedicated prediction paths: Each component is learned through its own pathway, encouraging the model to isolate distinct predictive patterns.
• Shared recombination: The predicted factors are recombined inside a common predictive design, preserving an overall view of the future state.
• Multiple objectives: The framework is tested not only for representation learning, but also for intervention prediction, out-of-distribution generalization, and long-horizon dynamics.

This design is intended to sit between a fully domain-specific model and a single monolithic representation. It provides room for specialization while retaining a common training and prediction principle.

Reported results

The paper evaluates JEPA-Anything in seven settings: vision, biology, clinical trajectories, control, molecular dynamics, physical fields, and weather. The abstract describes ten matched dynamics tasks and reports improved metrics on all ten relative to matched JEPA baselines. On Interventional Pong, the method reduces single-intervention prediction error by 34.8%.

Molecular dynamics provides a particularly direct test of long-horizon stability. Across four molecular systems, the paper reports the lowest one-step and 100-step rollout errors among the compared methods. The evaluation also includes forecasting more than 1,000 clinical events, although the supplied material does not provide the full breakdown of datasets, baselines, or statistical analysis.

The work goes beyond forecasting. According to the abstract, a biological intervention nominated through the learned factors received experimental support in cell co-cultures, patient-derived organoids, tumor fragments, and mice. The reported latent orbital modes also recover the Keplerian scaling exponent. These claims are notable, but should be interpreted alongside the complete paper, since the source material provided here is truncated.

Why it matters

The broader idea is to make world models more structured. If a system can identify relatively distinct causes or modes of change before recombining them, its predictions may become easier to inspect, intervene on, and transfer across environments. That could be useful in scientific settings where a good forecast is valuable, but understanding which factor drives the forecast matters just as much.

There are also important open questions. A factorization is not automatically a causal explanation, and factors learned in one domain may not retain the same meaning in another. It remains important to test identifiability, transfer to unseen systems, robustness under distribution shift, and error accumulation over very long rollouts. JEPA-Anything should therefore be viewed as a cross-domain modeling framework and a research direction, not yet as evidence that one universal world model has been solved.

Source: Hugging Face Daily Papers