toyxyz/toyxyz_RGBMocap

What it does

RGB MoCap is a native Windows application that reconstructs people from video as 3D body meshes and animation bones, then turns that motion into usable character animation. Reconstruction runs locally through interchangeable AI models, and the app adds foot and palm contact correction, humanoid retargeting, and export to FBX for source animation or retargeted characters (including Mixamo and VRM/PMX humanoids) and to VMD for MMD. Projects keep separate results per AI model. A separate experimental PEAR preview shows one person's live body and finger bones from a webcam without recording.

It fits animators and technical artists who already work with Mixamo, VRM, PMX/MMD or Blender pipelines and want markerless capture from ordinary video without sending footage to a cloud service.

Install and run

Requirements are a 64-bit Windows machine with a compatible NVIDIA GPU/driver and OpenGL 4.3. Install Git for Windows, then clone into a writable folder:

git clone https://github.com/toyxyz/toyxyz_RGBMocap.git
cd toyxyz_RGBMocap
install.cmd
RGBMoCap.cmd

install.cmd prepares the runtimes for ExMR, FlowHMR and GVHMR plus PEAR preview support, Blender, FFmpeg and the native correction components. It does not download the AI weights; prepare those in Model Manager, and note that some providers require account registration and manual downloads. Keep the console window open while the app runs. To update, close the app and run update.cmd from the same folder.

In the app, select a model, use File > Open Video, optionally use Find person, then click Reconstruct 3D. Review the motion, edit contacts if needed and apply correction, then export FBX or import a character, retarget, and export. Multi-camera capture is a separate add-on: run Setup_MAMMA.cmd after the base install. Step-by-step coverage of controls, defaults and model files is in the user guide and model guides.

Storage and optional models

The five offline models with runtimes occupy about 63.1 GiB (~67.8 GB), measured without projects, caches or retained downloads. MAMMA (multi-view) and PromptHMR require separate setup scripts and a local native build with Visual Studio 2022 C++ Build Tools, Windows SDK and CUDA Toolkit 12.8; they are not installed by install.cmd or update.cmd. PEAR adds roughly 2.84 GiB and reuses the shared runtime. App source, compiler tools, install temporaries and saved work need additional space, and the README's measured figures are not a free-space guarantee — see storage and installation details.

Constraints and known limits

• Windows 64-bit only, with a compatible NVIDIA GPU, driver and OpenGL 4.3.
• Model weights are acquired separately and some require registration and license acceptance; third-party components keep their own licenses while the original RGB MoCap code is MIT.
• MAMMA and PromptHMR are opt-in and need their own compiler/CUDA toolchain; PromptHMR does not produce finger motion, and in the authors' subjective tests its overall motion quality was lower than ExMR, though it can improve relative placement and interaction between multiple people in a single video.
• PEAR live preview is experimental, may be unstable, and provides only a bone preview — no recording, correction or export.
• Resource Mode > Automatic is the recommended starting point; Low Memory trades slower loading and more system RAM for lower VRAM use. The README's VRAM and speed figures come from short paired/clip tests, not full-job guarantees.

HotRepo's take

The project appeared in HotRepo's new-project ranking for the Python list on 2026-11-01 with 74 stars gained over a roughly 24-hour window ending that morning. Hourly samples from 2026-10-30 14:45 UTC (365 stars) to 2026-11-01 09:45 UTC (505 stars) show steady accumulation rather than a single spike, with flat stretches such as 2026-11-01 01:45–03:45 UTC. There is no recorded all-language daily rank and no Hacker News discussion, so this is a narrow growth signal, not evidence of adoption or output quality. The practical differentiator is the local, multi-model pipeline with explicit contact correction and FBX/VMD targets — but the ~63 GiB footprint and separate registration-gated weights mean it is best evaluated by someone already committed to a Windows/NVIDIA character-animation workflow.