CS-JEPA: A Decentralized Way for Swarm Robots to Predict a Shared Future

Lead

A central challenge in swarm robotics is not simply helping one robot understand its surroundings. The harder problem is getting many robots, each with only local observations and limited communication, to reason about the same future of the whole group. The paper “One Future, Every Robot” addresses this with Collective-State JEPA, or CS-JEPA: a decentralized predictive architecture designed so that every robot outputs a representation of one shared future collective state.

Key points

• A deployment-oriented formulation: The authors define the task as decentralized shared-state prediction. At deployment, each robot uses a 16-frame local history and sends one 64-float recurrent message per directed edge. The setup avoids global pooling, a target encoder, an episode clock, and recorded future actions.
• JEPA instead of raw reconstruction: CS-JEPA follows the joint-embedding predictive architecture idea. Rather than reconstructing raw future observations, it learns predictive embeddings that represent a common future token field. This shifts the objective toward useful abstraction instead of direct future reconstruction.
• Low-label evaluation: After pretraining without downstream collective labels, the learned representations are frozen. Ridge probes are then fitted using only 6, 12, or 24 globally labeled episodes. Against a raw-future reconstruction baseline with the same receiver anchor and deployment capacity, CS-JEPA improves label-budget AUC for prediction error and inter-robot agreement.
• Generalization across swarm settings: The reported experiments include in-distribution settings, ring topologies, mutual-kNN graphs, and unseen swarm sizes up to 108 robots. In the five-seed follow-up, every measured effect favors CS-JEPA across all outer seeds.
• Planning relevance: A separate sealed eight-seed follow-up evaluates action-conditioned predictors that receive candidate four-step plans before producing receiver-local predictive representations. CS-JEPA reduces branch-value MSE by 45.5% and improves within-context candidate-score Pearson correlation by 0.1291, with favorable effects in all eight seeds, including an unseen N=32 case.

Why it matters

The main contribution is to turn “a shared future of the swarm” into a learnable representation target. In real robotic collectives, dense global labels are costly, communication is constrained, and the number of robots or graph structure may change. A method that can learn from unlabeled experience and then adapt with a small number of globally labeled episodes is therefore practically attractive.

The results also suggest that JEPA-style objectives may be better aligned with decentralized control than raw future reconstruction. Instead of forcing each robot to reproduce the future in full detail, CS-JEPA encourages a compact predictive representation that different robots can agree on and that can support value estimation for candidate actions. The work is still experimental and further validation in richer physical environments would be needed, but it offers a promising direction for swarm world models and distributed decision-making.

Source: Hugging Face Daily Papers