Tencent-Hunyuan/Hy-MT2

Overview

Hy-MT2 is a family of instruction-following translation models in three sizes: 1.8B, 7B, and 30B-A3B (MoE). All cover 33 languages and accept translation instructions in multiple languages. The release includes model weights, a report, an instruction-following benchmark (IFMTBench), and a training pipeline.

Beyond plain translation, the models are prompted for terminology constraints, style control, personalization, delimiter preservation, and structured data such as code keys or JSON-like values that must stay untranslated. That makes it relevant for localization pipelines and enterprise translation workloads, not only casual text conversion.

For on-device use, AngelSlim's 1.25-bit quantization is reported to cut the 1.8B model to about 440 MB and improve inference speed by 1.5x. Quantized variants include FP8 and GGUF builds for llama.cpp.

Quick start

Install transformers>=5.6.0 and PyTorch, then load a model from Hugging Face. The README's example uses tencent/Hy-MT2-30B-A3B, but the same pattern applies to the 1.8B and 7B checkpoints:

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_path = "tencent/Hy-MT2-30B-A3B"
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_path, dtype=torch.bfloat16, device_map="auto", trust_remote_code=True
)
model.eval()

prompt = "将以下文本翻译成英语,注意只需要输出翻译后的结果,不要额外解释:\n\n今天天气真好。"
messages = [{"role": "user", "content": prompt}]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_tensors="pt"
).to(model.device)

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=4096)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

For serving, the README shows vLLM (vllm serve tencent/Hy-MT2-30B-A3B --tensor-parallel-size 1), SGLang (python3 -m sglang.launch_server --model tencent/Hy-MT2-30B-A3B --tp 1), and llama.cpp for GGUF weights. The GGUF path depends on a custom STQ kernel from an unmerged llama.cpp PR, so it requires building llama.cpp from source. The published prompt templates show how to phrase terminology, style, and structure-preservation requests; both source and target languages should be full language names, in Chinese for Chinese prompts and English for English prompts.

Model downloads are listed on Hugging Face and ModelScope. The smallest GGUF variant is the 1.25-bit build, and the main 30B-A3B checkpoint is here.

Recommended settings

For 1.8B and 7B, the README recommends temperature 0.7, top_p 0.6, top_k 20, repetition penalty 1.05. For 30B-A3B, temperature 0.7, top_p 1.0, top_k -1, repetition penalty 1.0. Max tokens are 4096, max context is 8192, and there is no default system prompt.

Training and evaluation

The repository includes a training pipeline supporting full-parameter fine-tuning and LoRA, multiple DeepSpeed ZeRO configurations, and LLaMA-Factory integration; details are in the training guide. The IFMTBench benchmark is provided for evaluating translation instruction following, and the model report is on arXiv.

Constraints

• Loading with trust_remote_code=True executes model-repository code; this is normal for custom architectures but should be reviewed for untrusted forks.
• The 30B-A3B model requires substantially more memory than the 1.8B model; the README does not list explicit VRAM requirements.
• GGUF inference depends on a custom kernel not yet merged into upstream llama.cpp, so the documented build steps are required.
• The license is marked NOASSERTION in GitHub metadata; check the actual license before commercial deployment.
• The README does not include a complete evaluation table in text form; benchmark numbers are in figures and the report.

HotRepo's take

The project appeared in the HotRepo daily snapshot for the Python list on 2026-11-13 at position 3, gaining 49 stars that day. That is a single recorded snapshot, so it supports interest around release but not a sustained trend claim. The repository's breadth matters more than the number: weights at three scales, quantized variants including a 1.25-bit build, multiple serving backends, and a training pipeline make it a usable base for translation systems rather than a demo-only release.