pollen-robotics/microduck_rl

A narrow but unusually complete sim2real pipeline: train locomotion policies for the Microduck biped in simulation, then get them onto the physical robot. Worth a careful look if you own the robot or study small legged machines; not a general RL starter kit.

The problem

Getting a legged robot to walk with reinforcement learning normally means assembling an environment, robot model, actuator representation and training loop before you can see a gait. The harder part comes later: a policy that walks in simulation falls over on real servos. Here the robot model, task configs, PPO training, ONNX export and the runtime manifest format live together, so the step that changes is the hand-off between training and deployment. The result is a policy you can watch in a CPU MuJoCo viewer and then install on an actual Microduck.

What's different

The project's central claim is that actuator fidelity is most of the sim2real gap at this scale, and the design follows through. Every task uses the BAM M6 model for the Dynamixel XL330, including voltage control law, back-EMF and friction, plus per-environment randomization of battery voltage, voltage sag, command delay and friction. Each main task also has a Backlash variant: ±1° gear play per servo, with encoder readings taken through that play, matching where the physical encoder sits.

The catalogue is broader than walking — fall recovery, sit-stand, ground pickup, ball kick, a roulade and roller variants, with flat and rough terrain where noted. Task IDs are registered consistently, which makes variants easy to compare. The trade-off is coupling: the repo leans on mjlab, MuJoCo Warp and BAM, so changes in those dependencies can affect the training stack. Part of the documentation is written as an internal playbook (reward lessons in AGENTS.md) — useful context, not a general tutorial.

Who should try it

Good fit: someone with a Microduck who wants a custom gait, a Backlash versus non-Backlash comparison, or to publish a policy to the Hugging Face Hub so other owners can install it in one command. It also suits researchers wanting a concrete small-biped sim2real example. Poor fit: anyone without the robot or without a CUDA GPU who just wants to see locomotion RL.

A documented alternative covers the compute constraint: adding --hf-jobs runs training on Hugging Face Jobs instead of locally. That changes who can use it, but the repo still targets this specific robot.

Requirements and limitations

The quickstart requires a CUDA GPU because training goes through MuJoCo Warp, plus uv. On ARM boxes such as DGX Spark/GB10 or Jetson, the first sync pulls roughly 2 GB of CUDA wheels and needs a longer HTTP timeout. A CPU-only path exists for inference and tests, but it does not replace GPU training. Setup is a one-time sync that can be slow and machine-dependent; ongoing work is the training loop, quoted at about 1–2 hours at 4096 environments for a usable gait. See the project instructions for the exact commands.

Licensing has a practical boundary: code is Apache-2.0, but mesh files are CC BY-NC-SA 4.0 — non-commercial and share-alike. If you plan to reuse the models or build commercial work around them, check LICENSE-MODELS.md for scope and attribution. The README gives no hardware requirements beyond CUDA and says nothing about the quality of provided exported policies.

HotRepo's take

HotRepo has observed this repo only since 2026-10-24: 2,612 stars (+41) on 2026-10-24 and 2,617 (+34) on 2026-10-25. Two dated snapshots are not a trend, so persistence cannot be judged. The repo was created 2025-12-06, so it is young.

First verify whether the documented training path matches your machine — especially the ARM CUDA wheel download and the quoted training time. Those two claims most affect whether it fits your workflow.