Tencent’s Hunyuan team released the Hy4 model weights on November 20 and published a system card alongside them. According to the release, Hy4 handles text, images, and video, with a language backbone containing 770 billion parameters and 49 billion activated per token. Its stated context length is 1,048,576 tokens. The accompanying materials describe the model’s architecture, intended use cases, and benchmark results, offering a substantial amount of information for developers and researchers. The scores, however, are primarily vendor-reported and should be interpreted alongside independent replications.
Key details
- A sparsely activated MoE backbone: Hy4 has 78 layers. The first uses a dense feed-forward network, while the remaining 77 use a mixture-of-experts design. Each MoE layer contains 256 routed experts and one shared expert. For each token, the model selects eight routed experts in addition to the shared expert.
- Long context and sparse attention: The model uses Gated DeepSeek Sparse Attention and IndexCache to reuse sparse indices across layers. Tencent lists a maximum context of 1,048,576 tokens. That specification alone does not establish how reliably the model handles every long-context task; practical results will depend on inputs, serving setup, and inference configuration.
- Vision and speculative decoding: A vision encoder supports dynamic image resolutions. An adapter turns image and video-frame features into visual tokens that can be processed alongside text. Hy4 also includes a 3.3B-parameter DFly2 drafter for speculative decoding, which reuses the model’s KV cache and sparse indices. Tencent describes the added cost as low, but real-world throughput will depend on the deployment environment.
- A broad set of proposed uses: The release discusses computer-use workflows, frontend and game development, finance and office work, scientific research, and long-horizon tasks. It also presents scores across text and multimodal benchmarks. Results from company-run evaluations should not be treated as universal rankings across institutions or operating conditions.
How to read the evaluations
Hy4 Base posts strong results on some of the listed knowledge, reasoning, and multimodal tests. But the benchmarks vary in task design, sample size, and scoring method, so no single number can summarize the model’s overall performance. Tencent also cautions that the base model saw substantial synthetic instruction data during mid-training. In few-shot evaluations, it may occasionally fail to follow the expected answer format, potentially distorting scores. The company says that the highest score in a table row does not imply statistical significance. Its finance results come from specific internal workflows and expert evaluations, and are best understood in that context.
Why the release matters
Making the weights available gives researchers and developers an opportunity to inspect, deploy, and adapt Hy4, while putting its long-context, multimodal, and sparse-activation design into wider practical testing. The 770B total parameter count also points to substantial infrastructure requirements: activating 49B parameters per token does not mean that storage or serving costs are necessarily small. Actual requirements will depend on factors such as weight precision, hardware, and inference software. Publishing a system card with the weights provides additional detail about the model and its disclosed limitations. The next useful signals will include independent replications, long-context reliability, video-understanding quality, inference costs, and the practical constraints set by the license.
Source: Web Search
Comments
Checking sign-in status...
Loading comments...