Kimi K3 Debuts With 2.8T MoE Scale, Native Vision and a 1M-Token Context

Lead

The Kimi team has introduced Kimi K3 on Hugging Face Daily Papers, describing it as a new open-weight frontier model built around scale, long context and multimodal capability. According to the paper summary, Kimi K3 is a 2.8T-parameter Mixture-of-Experts model that activates 104 billion parameters per token, supports native vision, and offers a one-million-token context window.

That combination makes Kimi K3 more than a larger chat model. It is positioned for workloads where memory, planning and sustained execution matter: long-document understanding, long-horizon coding, agentic workflows, knowledge-intensive reasoning and vision-language tasks.

Key points

• A massive MoE design: Kimi K3 has 2.8T total parameters, but uses expert routing to limit active computation. The paper says Stable LatentMoE effectively activates 16 of 896 routed experts for each token.
• One-million-token context: The model extends context length to one million tokens. Kimi Delta Attention and Attention Residuals are presented as mechanisms that improve information flow across both sequence length and model depth.
• Native vision support: Kimi K3 is not framed as a text-only release. Its native vision capability expands the model’s target use cases across multimodal reasoning and visual understanding.
• Post-training for harder workflows: The authors highlight reinforcement learning across general, agentic and coding domains, along with multiple reasoning-effort levels. The stated goal is stronger compositional generalization and more reliable long-horizon execution.
• Infrastructure as part of the model story: At 2.8T scale, the paper emphasizes algorithm-system co-design for KDA, balanced expert-parallel training, efficient memory management, persistent rollout and sandbox states for million-token agentic RL, plus deployment optimizations.

Why it matters

The most important claim around Kimi K3 is not simply its parameter count. The release suggests that open-weight models are moving deeper into territory once dominated by closed systems: very long context, multimodal inputs, agentic reinforcement learning and large-scale MoE serving. The authors report an approximately 2.5x improvement in overall scaling efficiency over Kimi K2, attributing that gain to architectural, data, training and systems changes rather than to size alone.

On evaluation, the paper states that Kimi K3 reaches frontier-level performance across long-horizon coding, agentic, knowledge, reasoning and vision benchmarks. It also includes an important caveat: its overall performance still trails the strongest proprietary models named in the summary, Claude Fable 5 and GPT-5.6 Sol. Even so, the authors claim Kimi K3 consistently outperforms the other open and proprietary models evaluated in their suite.

Because the full model weights are being released, the broader research community can test those claims, inspect failure modes and build on the model for long-context agents, coding systems and multimodal applications. If independently reproduced, Kimi K3 could become a major reference point for open frontier-scale AI.

Source: Hugging Face Daily Papers