ubc-vision/solv3r

What it does

Solv3R is a pretrained 3D reconstruction model that takes a set of images (or video frames) and produces camera poses, depth maps, confidence values and world points. It is packaged as a Python package around a single model class, with inference, evaluation and visualization code. The README frames it as a multi-view solver backed by a looped Transformer, and the project is accompanied by an arXiv paper Solv3R: 3D Looped Transformer as a Multi-view Solver.

The distinctive feature is that solving is expressed as repeated calls of a looped block over a latent state z, rather than a single forward pass. This makes several usage modes natural: solving all views at once (offline), feeding in groups of images over time, joining views in a random order, and streaming video frames one at a time. The latent state can be reused across calls, so a solve can continue from a previous state or be updated when new images are added.

It is aimed at researchers and engineers working on multi-view geometry, camera pose estimation, dense reconstruction and visual localization. You need a suitable environment for PyTorch-based inference, and the model weights work on the provided pretrained checkpoint.

Quick start

Install by cloning the repository and setting up a Python 3.11 environment:

git clone https://github.com/ubc-vision/solv3r.git && cd solv3r
conda create -n solv3r python=3.11 -y && conda activate solv3r
pip install -e .

The pretrained weights are hosted on Hugging Face. Solv3R.from_pretrained() downloads them to the Hugging Face cache, or uses checkpoints/solv3r.safetensors if you place the file there.

A minimal inference loop:

from solv3r import Solv3R, cameras_from_pose_enc, list_images, load_images

model = Solv3R.from_pretrained()
images = load_images(list_images("path/to/images"))
ctx = model.encode(images)          # first image defines the world frame
z = model.solve(ctx, calls=16)      # 16 calls of the looped block
pred = model.decode(z, ctx)         # depth, depth_conf, rays, world_points, pose_enc
c2w, K = cameras_from_pose_enc(pred["pose_enc"], (ctx.H, ctx.W))

Images are resized so that the long side is 504 and both sides are multiples of 14, without cropping. To run the ready-made examples on the bundled sample data:

python examples/offline.py --images examples/data/park --out outputs/offline --sky-mask
python examples/streaming.py --frames examples/data/desk --out outputs/streaming

Each example writes cameras and depth maps as .npz, a point cloud as .ply, and a browser-viewable .html file. A result can be rendered as a video with python examples/render_video.py --input outputs/offline/solve --out outputs/offline/solve.mp4 --aerial; --aerial suits drone photos.

For continuation, reuse the returned state:

z = model.solve(ctx, calls=8, z=z)
new_images = load_images(list_images("path/to/more_images"), size=(ctx.H, ctx.W))
z, ctx = model.add_views(z, ctx, new_images)
z = model.solve(ctx, calls=8, z=z)

The README provides offline, incremental, any-order and streaming example scripts, plus evaluation entry points for the benchmark tables in the paper.

Evaluation and expected results

The evaluation code follows existing protocols without modification. The offline benchmark covers DTU, ETH3D, 7-Scenes, ScanNet++ and nuScenes, prepared with DVLT's scripts; the streaming pose benchmark follows CUT3R and TTT3R on Sintel, TUM-dynamics and ScanNet; and map-based relocalization uses 7-Scenes and Cambridge Landmarks. The README lists expected numbers such as DTU AUC@3 84.5 and ScanNet AUC@3 80.7 for the 16-call setting, and streaming ATE of 0.157 on Sintel with the L8 configuration. These are reproduction targets from the paper, not results of an independent test here.

Usage constraints

• The code is Apache-2.0, but the released model weights are built on DVLT and are distributed under the NVIDIA License, restricted to non-commercial research and evaluation use DVLT.
• The input handling assumes the long side is 504 pixels and both sides are multiples of 14, with no cropping; the first image defines the world frame.
• Benchmark preparation requires external datasets and third-party preprocessing scripts, so the evaluation commands are not self-contained without the corresponding data directories.
• Solver property examples and a demo are still listed as outstanding; the README points to the project page for a video of those properties.

HotRepo's take

Solv3R is a fresh research release: the repository was created on 2026-11-02, and the HotRepo sample shows Stars moving from 100 on 2026-11-04 to 158 by 2026-11-06, with the Python new-project list recording 48 Stars gained between 2026-11-05 and 2026-11-06. There is no best GitHub daily rank recorded, so the available data supports attention around the release rather than a sustained trend. The surrounding signals are coherent for a model release: an arXiv paper, a project page, Hugging Face weights and benchmark code all published alongside the inference code. The non-commercial NVIDIA license on the weights is the main practical constraint for anyone considering use beyond research and evaluation.