speridlabs/idol-sdk

What it does

Idol takes one photo, render or generated image of an object and produces a 3D Gaussian splat model, including sides the input never showed. The pipeline removes the background with BiRefNet, conditions a 370M-parameter DiT on DINOv3 ViT-H+ features and a FLUX.2 VAE latent, then decodes 32,768 to 1,048,576 Gaussians (262,144 by default) into a standard 3DGS .ply. The full request uses about 1.8B parameters and peaks at 7.2 GB of VRAM, so it fits consumer GPUs.

The output is a .ply you can open in any Gaussian splat viewer, plus the preprocessed input image and the predicted input camera. It is aimed at 3D asset creation, reconstruction experiments and anyone who wants object splats from ordinary pictures.

Quick start

Requires Python >= 3.10 and PyTorch >= 2.5. Install the package from the repository root:

pip install -e .

Before the first run, request access to facebook/dinov3-vith16plus-pretrain-lvd1689m and black-forest-labs/FLUX.2-dev on Hugging Face, then log in with hf auth login; weights download from the speridlabs/idol repository on first use.

Run inference on a real photograph:

python -m idol.sample --image photo.jpg --mode real

This writes photo.ply, photo_input.png (the image after background removal and cropping) and photo_camera.json (the predicted input camera). Use --mode synthetic for computer-generated images; it is the default. The same call accepts a directory of images and an --out directory.

The Python API mirrors the CLI:

from idol import IdolPipeline

pipeline = IdolPipeline.from_pretrained("speridlabs/idol")
output = pipeline("photo.jpg", mode="real", seed=42, num_gaussians=1_048_576)
output.gaussian.save_ply("photo.ply")
output.preprocessed_image.save("photo_input.png")

To try it without installing, the Hugging Face Space runs the same demo with 93 example inputs and accepts your own images. For a local Gradio app, install the demo extra and run cd demo && python app.py.

Limits and requirements

Model access is gated: both DINOv3 ViT-H+ and FLUX.2 must be approved on Hugging Face before the first download. The GPU path is the intended one; flash-attn is used when installed, otherwise PyTorch's scaled dot-product attention gives the same result more slowly, and ATTN_BACKEND selects between them. --dinov3_dtype fp32 costs 1.7 GB more memory. Faster sampling is on by default on GPU, but the first image in a process is slower while it compiles. Reported request times range from 3.4 s on a B300 to 54.6 s on an L4 for 262,144 Gaussians at 50 steps. The code is Apache-2.0.

HotRepo's take

Idol is a young repository, and the available HotRepo data covers only its first days: from 131 stars at 2026-10-30 14:45 UTC to 385 at 2026-11-01 09:45 UTC. Over that window it gained 131 stars in the Python new-project list, with no recorded position on the all-language daily ranking. The README's per-GPU table is a useful reality check for the "consumer GPU" claim, since the same settings take roughly 14 s on an RTX 5090 and 19 s on an RTX 4090 versus 3.4 s on a B300. The comparison against TripoSplat and TRELLIS is the authors' own benchmark, so treat it as their reported numbers rather than an independent evaluation.