Reinforcement Learning

On-Policy Delta Distillation, or OPD², replaces direct imitation of a teacher’s full output distribution with a signal based on the difference between a reasoning-tuned teacher and its base model. The paper reports consistent gains across math, science, and code-reasoning benchmarks.

The paper uses chess as a controlled environment to study how pretraining choices shape reinforcement learning gains, and what RL changes inside a reasoning model.

SEED introduces an on-policy distillation framework for agentic reinforcement learning, converting completed trajectories into natural-language hindsight skills and distilling their behavioral effects back into the policy.

LongStraw targets the gap between million-token inference and much shorter RL post-training. By detaching shared prompt processing and replaying response branches, it makes ultra-long-context RL execution more memory-efficient.

Ring-Zero studies zero reinforcement learning at the 1T-parameter scale, using verifiable rewards rather than human-annotated reasoning traces. The paper argues that scaling improves sample efficiency and performance ceilings while revealing new reasoning behaviors.

A new arXiv paper examines why on-policy distillation works in LLM post-training—and why it can fail. The study argues that OPD is most useful as an exploration catalyst whose success depends on the faithfulness of its guiding signal.

The paper introduces GFlowRL, a streamlined GFlowNet-style reinforcement learning method that removes the learned partition network and replaces it with an in-batch Monte Carlo estimate.

A new arXiv paper argues that reporting a single post-training FLOP number hides the real allocation choices behind RL training. It studies how compute is split across model size, rollout search, policy learning, and reward feedback.

A new arXiv paper introduces Lighthouse RL, a reinforcement learning approach that restarts episodes from high-performing circuit configurations found during training. The goal is to reduce wasted exploration in sample-expensive analog circuit sizing.

A new arXiv paper proposes using the Lyapunov characteristic exponent as a physics-informed dense reward for stabilizing an inverted pendulum with vertical motion. The reported agent not only rediscovered Kapitza-like oscillatory stabilization but also damped the pivot motion into a strictly upright state.

A new arXiv paper introduces TRACE, a dense credit-assignment method for agentic reinforcement learning. It uses a frozen reference model to estimate state values and turn final-answer supervision into per-tool-call rewards.

A new arXiv paper introduces SIVA-RL, a framework that assigns visual-alignment supervision based on the observed effect of image interventions. The goal is to make vision-language models rely more reliably on visual evidence during reinforcement learning.

DAGR addresses a blind spot in goal-conditioned reinforcement learning: many goal embeddings do not account for the agent’s current state. The method improves navigation on OGBench, but its benefits do not consistently transfer to manipulation or puzzle tasks.