Language Models

770B Parameters, 49B Active per Token: Tencent Just Opened the Full Hy4 Flagship Weights, and the Licence Is No Longer Apache 2.0

November 22, 20260 views阅读中文原文
770B Parameters, 49B Active per Token: Tencent Just Opened the Full Hy4 Flagship Weights, and the Licence Is No Longer Apache 2.0

TL;DR

On November 20, Tencent's Hunyuan team published the complete weights of its flagship Hy4 model to Hugging Face, ModelScope and GitCode — three checkpoints in all: the instruction-tuned Hy4, an 8-bit Hy4-FP8 build, and the pre-trained Hy4-base. An 82-page system card came with them.

Three sentences:

  • It is a very large, very sparse model: 770 billion parameters in the language backbone, but only 49 billion activated per token, plus a 1.23B vision encoder and a 3.3B spec-decoding drafter.
  • The context window is one million tokens, and it reads text, images and video — and can drive command lines and graphical interfaces.
  • The part to read first is not the benchmark table, it is the licence. This is not the Apache 2.0 of the August preview; it is Tencent's own Hy Licence, and Section 10.2 puts a gate in front of anyone reselling inference at scale.

The core facts here come from the official model card, the licence text itself, and measurements taken directly from the Hugging Face repository API. Secondary details are compiled from multiple media reports, and anything that is a media claim rather than a primary document is flagged as such.

What it actually is: a 770B multimodal MoE

From the official specification table:

  • 770B total parameters, 49B activated per token, across 78 layers — layer one is an ordinary dense feed-forward block, the other 77 are mixture-of-experts.
  • Every MoE layer holds 256 routed experts plus 1 shared expert; each token activates the top 8 routed experts together with the shared one. Hidden size 6,144, vocabulary 120,832.
  • Attention is Gated DeepSeek Sparse Attention with IndexCache, in MLA form, 64 attention heads. The indexer selects only 2,048 tokens per layer and reuses those sparse indices across layers.
  • 1,048,576-token context, the one-million figure everyone quotes.
  • Multimodal by construction: a 40-layer ViT with 16×16 patches and dynamic resolution; image and video-frame features are converted into visual tokens by an adapter and fed to the backbone alongside text.
  • DFly2 drafter: five layers, 3.3B parameters, built for speculative decoding.

DFly2 deserves its own paragraph. It reuses the backbone's top-k sparse KV cache and the exact 2,048 positions the indexer picked, so each draft token attends to at most about two thousand keys — a constant cost regardless of context length — and it adds no separate KV memory. In the official serving command it is switched on with --speculative-config '{"num_speculative_tokens":5,"method":"dfly"}'.

Get the timeline right: this is the fourth announcement, not the first

Hunyuan has pushed four separate messages in three weeks, and coverage keeps folding them into one story:

  1. August 28 — Hy4 preview ships, under Apache 2.0.
  2. November 4 — the full Hy4 launches as a paid service; weights are promised but not published.
  3. November 18–19 — the smaller Hy4 flash weights (roughly 220B total, 12B active) go up.
  4. November 20 — the flagship Hy4 weights finally land in the repositories.

So "Tencent open-sourced Hy4" only becomes literally true on November 20. Anything written before that about Hy4 weights was about the promise, the preview, or the smaller flash model.

How big are the three checkpoints? Measured, not repeated

I pulled the file listings from the Hugging Face API and summed every file size:

  • Hy4 (instruct): 294 files, about 1.55 TB; the repository's last commit is 2026-11-20 03:05 UTC.
  • Hy4-FP8: 294 files, about 786 GB.
  • Hy4-base (pre-trained): 328 files, about 1.54 TB.

There is a real pitfall here. At least one outlet that specialises in open-weight coverage reported the FP8 build at roughly 924 GB and noted that a 786 GB figure was also circulating, calling the two "conflicting". The measured number is 786 GB. A 770B model stored at one byte per parameter lands in exactly that range once indexer and vision-tower overhead are added. When you see 924 GB, you are looking at a number nobody checked against the file list.

The more consequential error is in TechNode's November 20 piece, which describes the release as "under the Apache 2.0 license". That is wrong. The model card's licence field reads other, and the licence name is tencent-hy-license. Copy that sentence into your procurement note and you will miss the clause that matters most.

Section 10.2 is the real gate

I read the LICENSE file line by line. The relevant part is Section 10:

  • 10.2(a): if you operate a model-as-a-service business — giving third parties remote access to inference or fine-tuning — and your aggregate worldwide gross revenue exceeds US$50 million in any consecutive 12-month period, you must obtain a separate written licence from Tencent first. Note the wording: *whether or not such revenue is derived from the Work or any Derivative Works*.
  • 10.2(b): the same applies before you use it in any product with more than 100 million monthly active users in a calendar month.
  • 10.3: purely internal use is exempt, provided neither the model, any derivative, nor their outputs are made available to third parties.
  • Licensing requests go to [email protected].

One sentence is easy to skim past: the terms travel with derivative works. Fine-tunes, quantisations, distillations, merges and prunes all carry Section 10.2 forward.

In other words, "open weights" and "open source" are two different things here. The weights are free to download, free to modify, free to run yourself — but reselling inference at scale requires a conversation first. Compared with the Apache 2.0 preview in August, that is a deliberate tightening. For a large cloud, the negotiation may be a bigger obstacle than the model; for small teams and internal deployments, it changes almost nothing.

The numbers Tencent reports about itself

On capability, the card offers a very large benchmark table. A representative selection:

  • Knowledge: MMLU 91.6, MMLU-Pro 73.9, C-Eval 95.0, CMMLU 93.1 — each slightly ahead of the corresponding DeepSeek-V4-Pro-Base figure of 91.1 / 73.4 / 93.4 / 90.7.
  • Factuality: SimpleQA 57.2, well above DeepSeek-V4-Pro-Base at 52.1 and the flash sibling at 37.5.
  • Multimodal: MMMU-Pro (standard, 10 options) 59.6, CVBench 86.7, TextVQA 78.2.
  • Finance: an evaluation run with Tencent Financial Technology using 213 finance professionals — 153 of them from leading institutions, more than 70% with over five years in the industry — across 579 real tasks. Hy4 averaged 3.04 out of 4, first among open-weight models; DeepSeek-V4.1-Flash-1017 scored 2.97, and both Kimi K3.1 Pro and GLM-5.3 scored 2.68.
  • Hallucination: Tencent says targeted search improvements cut its internal hallucination rate from 15.76% to 5.45% and speculative errors from 23.1% to 5.6%.
  • Versus its own preview: +19.1% on game development, +9.8% on game logic, and +12.03 percentage points on IRAB, an internal benchmark from the financial-AI agent company RabyteTech.

Every one of those numbers shares the same caveat: they come from Tencent's own evaluation suite, with no independent reproduction. The card is also candid about a distortion risk — because the base model saw a large amount of synthetic instruction data during mid-training, it occasionally fails to follow the expected answer format in few-shot evaluation, which can skew scores. That is an honest disclosure, and it is also a reason to discount the pre-training comparison table.

The stated weaknesses are equally direct: on research-level mathematics, long-horizon coding, model self-improvement and advanced visual question answering, Hy4 still trails the strongest proprietary models by a visible margin.

How to run it: eight GPUs minimum, forget single-machine

Tencent ships two official container paths:

  • vLLM: image vllm/vllm-openai:hy4, with --tensor-parallel-size 8, --attention-backend FLASHMLA_SPARSE, and --reasoning-parser hy_v4 --tool-call-parser hy_v4.
  • SGLang: image lmsysorg/sglang:hy4, with --tp 8 and --speculative-algorithm DFLY.

Two parameters are worth memorising. temperature=0.9 and top_p=1.0 are the recommended sampling values. And reasoning_effort accepts no_think or high — the default is high, meaning deep chain-of-thought is on unless you turn it off. If you want a direct answer, you must pass no_think explicitly, otherwise latency and token spend will both land above your estimate.

There is also a GGUF build (AngelSlim/Hy4-GGUF), but Tencent labels it for end-to-end testing, not production.

Who it is for

  • Teams building their own agents who want to stop paying per token: a 770B-class multimodal model with a 1M window and CLI/GUI control covers "reads the screen, reads the whole document, acts on it" in one download.
  • Anyone whose data cannot leave the building: finance, healthcare, public-sector document work. Self-hosting means no metered bill and no data egress, inside the Section 10.3 internal-use carve-out.
  • Long-document comparison and earnings-report analysis: a 1M window on sparse attention keeps KV pressure far below what a dense model of this size would need.
  • Game and front-end tooling: Tencent trained specifically on Unreal, Unity, Godot, Blender and CAD workflows — vertical capability that a general leaderboard will never show.

Who should wait

  • Solo developers with one consumer GPU: the 786 GB FP8 build needs eight-way tensor parallelism. This is not a download-and-try model.
  • Cloud providers already past US$50M in revenue that want to resell inference: negotiate the licence before you list it.
  • Anyone who needs an unambiguously permissive licence: this is not Apache 2.0, and if your compliance pipeline only accepts standard OSI licences, the model does not clear it.
  • Anyone expecting it to beat the best closed models outright: Tencent itself lists four categories where it loses, so do not treat self-reported scores as a verdict.

Pitfalls: four things that will trip you up

  1. Do not read "open weights" as "open source": downloadable weights are not unconditional permission to resell. Section 10.2 is enforceable language, not a suggestion.
  2. Do not quote the media figures: 924 GB is wrong; the measured FP8 size is about 786 GB, and 1.55 TB refers to the BF16 instruct build.
  3. Do not copy "Apache 2.0": at least one mainstream tech outlet got it wrong. The correct name is the Tencent Hy Licence.
  4. Do not forget the default reasoning mode: run it without no_think and latency will be markedly higher than you planned for.

Cost and time: how the arithmetic works

Tencent has not published a per-token API price for Hy4 — it has been a paid service on its own cloud since November 4 — so self-hosting is the only route where the cost is visible. A rough calculation: 786 GB of FP8 weights across eight 80 GB cards means the weights alone consume roughly 786 GB of VRAM, leaving headroom needed for KV cache and activations. That is precisely why the official recipe uses TP=8. This is rack-scale or whole-machine rental, not a spin-up-one-GPU-for-an-hour job.

For scale, the smaller Hy4 flash (about 220B total, 12B active) can be squeezed into materially smaller deployments. The two are not substitutes; they are two tiers of the same family.

How to use it: your first steps

  1. Read Section 10 of the licence first and decide which side of 10.2 your business sits on. Internal use clears immediately.
  2. Pick the right checkpoint: tencent/Hy4 or tencent/Hy4-FP8 for instruction capability, tencent/Hy4-base if you intend to post-train your own.
  3. Bring up serving with the official images at --tensor-parallel-size 8; run without speculative decoding first to establish a baseline, then add dfly and measure throughput.
  4. Benchmark your own tasks: treat the published scores as orientation only. Run your real long documents and agent jobs, and watch first-token latency under long context plus tool-call success rate.
  5. License up if you are over the line: if revenue or monthly actives cross the thresholds, email [email protected] before launch, not after.

Bottom line

This is one of the largest multimodal MoE models you can currently download: 770B total parameters, 49B active, a one-million-token window, and text, image and video input. What decides whether it enters your production stack is not the benchmark table — it is the 786 GB of VRAM and two lines in Section 10.

Sources

Comments

0 published
Log inorsign up to leave a comment.

Loading comments...