Large Language Models

This paper proposes TOPL, a training paradigm that does not directly imitate off-policy outputs. Instead, it teaches the model to classify whether each token is correct, which helps improve faithful generation under distribution shift.

A new paper featured on Hugging Face Daily Papers introduces Loopie, a Mixture-of-Experts looped Transformer family. The authors claim it outperforms standard Transformer baselines under the same compute budget and gains strong reasoning ability after post-training.

DeepLoop tackles a subtle issue in looped Transformer architectures: reusing the same physical blocks changes how residual updates behave. The paper proposes scaling rules that account for repeated parameter visits rather than only nominal layer depth.

GigaWorld-Policy-0.5 tackles a key bottleneck in World Action Models: costly video generation during inference. It keeps future visual dynamics as a training signal while switching to action-only decoding for deployment.

A new arXiv paper introduces Deep Interaction, a human-in-the-loop method that lets users edit faulty reasoning steps in an LLM’s original answer and reuse the corrected chain to steer future output.

A new arXiv paper reframes familiar Transformer feedforward-block choices as mechanisms for preserving gradient rank across depth. Skip connections, normalization placement, and width expansion are interpreted as part of a shared tradeoff among rank collapse, composition, and parameter cost.

DeltaMerge-LowRes explores whether low-resource NLP adaptation can avoid costly joint language-task fine-tuning. It trains a language delta and a task delta separately, then studies how different merge rules change model behavior.