Intellifusion’s AI Inference Roadmap Shifts the Focus From Peak FLOPS to Token Cost

Lead

As large language models move from demos to continuous production use, inference cost is becoming a central metric for AI infrastructure. At WAIC, Intellifusion presented a roadmap for the next two-plus years: three cloud inference chips—DeepVerse100P, DeepVerse100D and DeepVerse100L—designed respectively for Prefill, Decode and Decode-stage FFN workloads, with deployment planned in large heterogeneous clusters.

Key points

• Inference is being split into finer stages. Prefill processes the input context in bulk and becomes especially compute-intensive in long-context scenarios. Decode generates tokens one by one and is more sensitive to memory bandwidth, data access and tail latency. With architectures such as MoE, even the Decode phase can be divided into Attention and FFN components with different resource profiles.
• Each chip targets a distinct bottleneck. DeepVerse100P is aimed at million-scale-context Prefill, where sharing compute and bandwidth with Decode can interfere with generation throughput. DeepVerse100D focuses on Decode, emphasizing substantially higher memory bandwidth than mainstream chips, support for 1024-card scale-up and an optical interconnect architecture designed to reduce multi-hop switching, congestion and long-tail delays. DeepVerse100L targets the compute-heavy FFN part of Decode, using a 3D Memory architecture and low-latency chip interconnects to improve overlap between computation and communication.
• The architecture is cluster-first. Rather than treating single-chip peak performance as the only goal, Intellifusion is proposing separate resource pools for Prefill, Decode and FFN, connected through high-speed interconnects. The intended result is a “token factory” that improves resource utilization across the full generation pipeline.
• Software will determine real usability. The IFWA software stack is positioned as the bridge between these chips and production models, with support for PyTorch ATen operators, vLLM, SGLang, quantization, compression, compilation and deployment. The company also points to its open-source Houmao multi-agent heterogeneous programming framework and its work on integrating Triton operator capabilities into FlagOS.

Why it matters

The roadmap shows how AI inference competition is moving beyond raw accelerator specifications. Longer contexts, complex reasoning workflows and agent-based applications create uneven pressure on compute, memory and networking. A stage-specialized cluster may reduce contention and improve stability if the scheduling and software layers are mature enough.

The challenge is that heterogeneous inference systems are harder to operate. Model migration, compiler support, operator optimization, interconnect behavior and cluster scheduling all affect whether the promised cost reduction appears in real deployments. Intellifusion’s long-term goal of making token generation dramatically cheaper should therefore be read less as a finished outcome and more as a direction: domestic AI compute must become not only usable, but efficient, easy to deploy and economically scalable.

Source: InfoQ 中文