Confucius4-R2T2 Brings Stable Prefixes to Streaming ASR

Introduction

For streaming automatic speech recognition, speed is only part of the problem. A system may produce text quickly, but if it repeatedly rewrites that text as more audio arrives, downstream software has to decide which version to trust. That behavior is tolerable in a transcript window, yet costly for voice agents, translators, and other systems that act on each token immediately. A revised token can trigger wasted computation or an action based on information that is no longer valid.

Confucius4-R2T2, introduced by NetEase Youdao, focuses on this stability problem. Its objective is to emit text that downstream applications can consume as soon as it appears, while keeping previously emitted text unchanged.

The central idea: Longest Stable Prefix

The model is trained with a new notion called Longest Stable Prefix, or LSP. Given the audio observed so far, LSP identifies the longest text prefix that is already consistent with a complete recognition result and is not expected to be overturned by additional speech. As the input grows, the stable prefix can stay the same or become longer, but it does not move backward and rewrite earlier output.

The design has several practical implications:

• Append-only transcription: emitted text is preserved rather than repeatedly revised.
• Immediate downstream use: agents and translation systems can process text while recognition continues.
• No architectural redesign: LSP is introduced as a post-training supervision target, allowing a full-context model to acquire streaming behavior without changing its network structure.
• Configurable streaming chunks: the implementation supports different chunk sizes, offering a way to balance responsiveness and available acoustic context.

Training and reported behavior

R2T2 starts from Qwen3-ASR-1.7B and is post-trained on 1,500 hours of Chinese and English audio. The pretrained audio encoder and projector are frozen during adaptation. The main learning task is therefore not to rebuild the acoustic front end, but to teach the model which portion of a partial recognition result is safe to commit.

According to the supplied paper summary, R2T2 brings streaming recognition close to the base model’s full-context quality while maintaining low emission lag. The reported gains also extend beyond the languages used for post-training, suggesting some cross-lingual generalization. The available material does not include detailed word-error rates, latency tables, or comparisons across deployment conditions, so those claims should be examined against the full paper and independent testing.

Why it matters

The broader contribution is a practical route from offline-style ASR to stable streaming ASR. Instead of assuming that streaming requires a new architecture or that low latency must come with frequent revisions, R2T2 changes the supervision target. The model learns to delay uncertain content while committing the portion that is already reliable.

That behavior is particularly relevant to applications where a transcript is an input to another system rather than merely a visual display. Stable output can simplify state management, reduce rollback logic, and make real-time integrations more predictable. The model weights, implementation, and a live demo have been released publicly, allowing developers to test the approach across languages, acoustic conditions, and interactive workloads.

Source: Hugging Face Daily Papers