D3voz/openjutsu

Openjutsu is a local video-editing front end for swapping a person, object or background in an existing clip while preserving the original camera work, cuts, motion and soundtrack. It is aimed at people who have an NVIDIA GPU and a ComfyUI install and want an open-weights alternative to hosted character-swap tools, not a beginner video editor.

What it does

You upload a source video and a character photo, and press Generate. In Simple mode SAM 3.1 tracks the largest person, MiniMax H3 regenerates that area, and a grey silhouette of the tracked person guides the motion; anyone else in the shot is untouched. Advanced mode handles 1–4 characters with optional roles, matching each tracked person to a photo per shot and leaving unmatched people alone. Object swap changes, removes or replaces the background around a tracked thing (hair, a jacket, a car), and can add an object using manual placement boxes. A From picture path uses FLUX and SolarRenderer to stage the character in the source pose and generate a clip from one picture. Two post-processing steps repair finished video: Fix faces redraws a small face from a photo, and Fix colours grades the output without another H3 run project README.

How it works

The app is a Next.js interface over packaged ComfyUI API workflows @comfyorg/sdk, modellerad on Comfy's reference app comfy-reshoot. Each job is checked against ComfyUI's registered nodes and listed model files before upload, and missing packs or models are reported with install links or destination folders. That check does not cover Python dependencies, model integrity or available VRAM.

Requirements and limits

You need an NVIDIA GPU (tested on an RTX 4060 Ti 16 GB with 64 GB RAM), ComfyUI 0.41.2 or newer, Node.js 22.6 or newer, Python for the proxy, and a set of custom nodes and model files; the README lists each one requirements and setup. The local start script builds the app, starts the proxy and the production server, and opens the page. Windows users must set AIOHTTP_NOSENDFILE=1 before starting ComfyUI, or a partial-file request can stall all later file downloads until ComfyUI restarts. Uploads are capped at 512 MB per request and three concurrent arrivals; files stay in ComfyUI's input folder after a job. Generation continues in ComfyUI even if the web page is closed.

The README documents honest limitations. Mirrors and reflections do not work: the reflection keeps the original person and H3 copies their look back onto the character. Fix faces depends on InsightFace buffalo_l, licensed for non-commercial research only, and Ultralytics YOLO weights under AGPL-3.0; H3 and other model licences are separate from the app's MIT licence. A 6-second 480p clip takes roughly 6–13 minutes on the tested card, and 1080p is about 2.5× slower than 736p. Resuming relies on clip checkpoints in ComfyUI's output folder, so a fresh serverless worker will not resume. The Comfy Developer Platform deployment path is explicitly untested. When hosted beyond loopback, the app requires OPENJUTSU_PASSWORD of at least 12 characters and refuses to start without one otherwise; the login is single-owner, so everyone with the password shares job access, and the rate limiting depends on a proxy setting X-Forwarded-For.