inclusionAI/IDM

Project overview

IDM (Intrinsic Distribution Matching) is the official research release for a distillation method that turns large text-to-image diffusion backbones into generators that need only one or two conditional transformer evaluations. It combines internal distribution matching distillation (DMD) with moment matching on the generator's own EMA features, so no external feature encoder such as DINOv2 is required at inference time paper.

The repository provides the command-line inference path (inference.py), a unified sampler for 1-NFE and 2-NFE generation, Diffusers adapters and auxiliary-time wrappers, example prompts, and CPU tests. The paper reports results on Z-Image-Turbo 6B and Qwen-Image 20B backbones, both with full-parameter tuning and rank-32/64 LoRA. Reported peak figures include 94.25% LongText-CN and 90.20% LongText-EN at 2 NFE for Z-Image, with 0.25 s / 0.30 s latency on a single A100 in BF16.

This is aimed at researchers and engineers who want to reproduce or build on IDM-style few-step distillation and run its checkpoints, not at end users looking for a ready-made image app.

Quick start

Prepare Python 3.10 or newer and a CUDA GPU with enough memory for the selected backbone (the 20B model needs considerably more than the 6B one). Install the dependencies:

python -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt

Install a PyTorch build matching your CUDA environment if the default is unsuitable. FlashAttention, FSDP and training services are not required; Diffusers is pinned because the auxiliary-time wrappers depend on its model interfaces.

An IDM checkpoint is needed before you can generate an image. The checkpoint download links are marked "Coming soon" in the repository, together with their checkpoint-specific sampling settings, so the full end-to-end path cannot be completed yet from this release alone checkpoints. Once weights are available, point --model at them:

python inference.py \
  --backbone z-image \
  --model /path/to/IDM-Z-Image \
  --prompt 'A cafe menu with the heading "MORNING COFFEE" and three lines: "Americano $4", "Latte $5", "Mocha $6".' \
  --steps 2 --height 1024 --width 1024 --seed 42 \
  --output-dir outputs/z-image

Each run writes a PNG plus a JSON file recording the prompt, seed, weight paths and sampling configuration; existing outputs are not overwritten. Use --backbone qwen-image with a Qwen-Image checkpoint for the 20B variant, --transformer to load a transformer-only export against a base pipeline, or --lora to merge a Diffusers-format adapter transformer-only loading, LoRA loading.

Usage constraints

• Inference requires an IDM checkpoint; base-model and APEX weights do not reproduce the reported IDM results.
• --steps accepts 1 or 2 only, and those correspond to exactly one or two conditional transformer evaluations without classifier-free guidance.
• Height and width must be positive multiples of 16. --device cuda:1 selects another GPU; CPU is available for small compatibility checks.
• Transformer-only checkpoints must contain config.json and safetensors weights. LoRA adapters must match their base model and be in Diffusers pipeline format; raw optimizer states, FSDP shards and unconverted PEFT training checkpoints are not inference inputs.
• --local-files-only disables model downloads and uses local files or the Hugging Face cache.
• Code is Apache-2.0; base-model and future IDM checkpoint licenses apply separately to the weights.
• Reported benchmark numbers and latency figures come from the paper, not from running this repository, and the README notes that checkpoint-specific settings and end-to-end quality validation are still pending the weight release.

HotRepo's take

IDM entered HotRepo's new-project list on 2026-11-02 for Python, recording 58 stars across a window from 2026-11-01 to 2026-11-02, and its hourly samples show stars moving from 100 to 161 during that period. There is no recorded GitHub daily all-language rank, and no Hacker News discussion was found. The attention arrives while the headline artifacts — model weights and their sampling settings — are still marked as coming soon, so the present repository is effectively an inference scaffold plus a paper. That makes it most useful for readers evaluating the method or preparing to run it once checkpoints land, rather than for anyone expecting to generate images today.