Kimodo Motion Diffusion

作者 reason-machines2384a003145a无许可证83 个星标收录于 2026年10月8日更新于 2026年10月8日仓库3个月前更新

Generate high-quality 3D human and humanoid robot motions using Kimodo, a kinematic motion diffusion model controlled via text prompts and kinematic constraints.

仅含说明Design & Creative
AI 生成的概览

使用 Kimodo 扩散模型,根据文本提示和运动学约束生成 3D 人体与人形机器人动作。

功能
该技能说明如何安装和运行 Kimodo——一个运动学动作扩散模型,可根据文本提示以及关键帧、末端执行器位置、2D 路径等运动学约束生成 3D 人体和人形机器人动作。内容涵盖 kimodo_gen 命令行工具、kimodo_demo 交互式网页演示和底层 Python API,并说明 NPZ 动作文件、MuJoCo qpos CSV 以及兼容 AMASS 的导出格式。此外还涉及约束编写、批量生成、模型变体和故障排查。
适用场景
当需要生成、约束或导出 3D 人体或人形机器人动作时使用,例如文本生成动作片段、带关键帧或末端执行器约束的序列,或用于 MuJoCo 的 G1 机器人动作。也适用于以 SMPL-X 或兼容 AMASS 的输出为起点的重定向流程。
运行要求
需要约 17GB 显存的 GPU(建议 RTX 3090/4090 或 A100),Linux 系统,或通过 Docker 在 Windows 上运行,并从代码仓库安装 Kimodo 包。模型首次使用时自动从 Hugging Face 下载,因此需要网络访问。该技能本身仅包含说明文档,不附带脚本或资源文件。

Kimodo Motion Diffusion

Skill by ara.so — Daily 2026 Skills collection.

Kimodo is a kinematic motion diffusion model trained on 700 hours of commercially-friendly optical mocap data. It generates high-quality 3D human and humanoid robot motions controlled through text prompts and kinematic constraints (full-body keyframes, end-effector positions/rotations, 2D paths, 2D waypoints).

Installation

bash
# Clone the repositorygit clone https://github.com/nv-tlabs/kimodo.gitcd kimodo
# Install with pip (creates kimodo_gen and kimodo_demo CLI commands)pip install -e .
# Or with Docker (recommended for Windows or clean environments)docker build -t kimodo .docker run --gpus all -p 7860:7860 kimodo

Requirements:

  • ~17GB VRAM (GPU: RTX 3090/4090, A100 recommended)
  • Linux (Windows supported via Docker)
  • Models download automatically on first use from Hugging Face

Available Models

ModelSkeletonDatasetUse Case
Kimodo-SOMA-RP-v1SOMA (human)Bones Rigplay 1 (700h)General human motion
Kimodo-G1-RP-v1Unitree G1 (robot)Bones Rigplay 1 (700h)Humanoid robot motion
Kimodo-SOMA-SEED-v1SOMABONES-SEED (288h)Benchmarking
Kimodo-G1-SEED-v1Unitree G1BONES-SEED (288h)Benchmarking
Kimodo-SMPLX-RP-v1SMPL-XBones Rigplay 1 (700h)Retargeting/AMASS export

CLI: kimodo_gen

Basic Text-to-Motion

bash
# Generate a single motion with a text prompt (uses SOMA model by default)kimodo_gen "a person walks forward at a moderate pace"
# Specify duration and number of sampleskimodo_gen "a person jogs in a circle" --duration 5.0 --num_samples 3
# Use the G1 robot modelkimodo_gen "a robot walks forward" --model Kimodo-G1-RP-v1 --duration 4.0
# Use SMPL-X model (for AMASS-compatible export)kimodo_gen "a person waves their right hand" --model Kimodo-SMPLX-RP-v1
# Set a seed for reproducibilitykimodo_gen "a person sits down slowly" --seed 42
# Control diffusion steps (more = slower but higher quality)kimodo_gen "a person does a jumping jack" --diffusion_steps 50

Output Formats

bash
# Default: saves NPZ file compatible with web demokimodo_gen "a person walks" --output ./outputs/walk.npz
# G1 robot: save MuJoCo qpos CSVkimodo_gen "robot walks forward" --model Kimodo-G1-RP-v1 --output ./outputs/walk.csv
# SMPL-X: saves AMASS-compatible NPZ (stem_amass.npz)kimodo_gen "a person waves" --model Kimodo-SMPLX-RP-v1 --output ./outputs/wave.npz# Also writes: ./outputs/wave_amass.npz
# Disable post-processing (foot skate correction, constraint cleanup)kimodo_gen "a person walks" --no-postprocess

Multi-Prompt Sequences

bash
# Sequence of text prompts for transitionskimodo_gen "a person stands still" "a person walks forward" "a person stops and turns"
# With timing control per segmentkimodo_gen "a person jogs" "a person slows to a walk" "a person stops" \  --duration 8.0 --num_samples 2

Constraint-Based Generation

bash
# Load constraints saved from the interactive demokimodo_gen "a person walks to a table and picks something up" \  --constraints ./my_constraints.json
# Combine text and constraintskimodo_gen "a person performs a complex motion" \  --constraints ./keyframe_constraints.json \  --model Kimodo-SOMA-RP-v1 \  --num_samples 5

Interactive Demo

bash
# Launch the web-based demo at http://127.0.0.1:7860kimodo_demo
# Access remotely (server setup)kimodo_demo --server-name 0.0.0.0 --server-port 7860

The demo provides:

  • Timeline editor for text prompts and constraints
  • Full-body keyframe constraints
  • 2D root path/waypoint editor
  • End-effector position/rotation control
  • Real-time 3D visualization with skeleton and skinned mesh
  • Export of constraints as JSON and motions as NPZ

Low-Level Python API

Basic Model Inference

python
from kimodo.model import Kimodo
# Initialize model (downloads automatically)model = Kimodo(model_name="Kimodo-SOMA-RP-v1")
# Simple text-to-motion generationresult = model(    prompts=["a person walks forward at a moderate pace"],    duration=4.0,    num_samples=1,    seed=42,)
# Result contains posed joints, rotation matrices, foot contactsprint(result["posed_joints"].shape)       # [T, J, 3]print(result["global_rot_mats"].shape)    # [T, J, 3, 3]print(result["local_rot_mats"].shape)     # [T, J, 3, 3]print(result["foot_contacts"].shape)      # [T, 4]print(result["root_positions"].shape)     # [T, 3]

Advanced API with Guidance and Constraints

python
from kimodo.model import Kimodoimport numpy as np
model = Kimodo(model_name="Kimodo-SOMA-RP-v1")
# Multi-prompt with classifier-free guidance controlresult = model(    prompts=["a person stands", "a person walks forward", "a person sits"],    duration=9.0,    num_samples=3,    diffusion_steps=50,    guidance_scale=7.5,           # classifier-free guidance weight    seed=0,)
# Access per-sample resultsfor i in range(3):    joints = result["posed_joints"][i]   # [T, J, 3]    print(f"Sample {i}: {joints.shape}")

Working with Constraints Programmatically

python
from kimodo.model import Kimodofrom kimodo.constraints import ConstraintSet, FullBodyKeyframe, EndEffectorConstraintimport numpy as np
model = Kimodo(model_name="Kimodo-SOMA-RP-v1")
# Create constraint setconstraints = ConstraintSet()
# Add a full-body keyframe at frame 30 (1 second at 30fps)# keyframe_pose: [J, 3] joint positionskeyframe_pose = np.zeros((model.num_joints, 3))  # replace with actual poseconstraints.add_full_body_keyframe(frame=30, joint_positions=keyframe_pose)
# Add end-effector constraints for right handconstraints.add_end_effector(    joint_name="right_hand",    frame_start=45,    frame_end=60,    position=np.array([0.5, 1.2, 0.3]),   # [x, y, z] in meters    rotation=None,                           # optional rotation matrix [3,3])
# Add 2D waypoints for root pathconstraints.add_root_waypoints(    waypoints=np.array([[0, 0], [1, 0], [1, 1], [0, 1]]),  # [N, 2] in meters)
# Generate with constraintsresult = model(    prompts=["a person walks in a square"],    duration=6.0,    constraints=constraints,    num_samples=2,)

Loading and Using Saved Constraints

python
from kimodo.model import Kimodofrom kimodo.constraints import ConstraintSetimport json
model = Kimodo(model_name="Kimodo-SOMA-RP-v1")
# Load constraints saved from web demowith open("constraints.json") as f:    constraint_data = json.load(f)
constraints = ConstraintSet.from_dict(constraint_data)
result = model(    prompts=["a person performs a choreographed sequence"],    duration=8.0,    constraints=constraints,)

Saving and Loading Generated Motions

python
import numpy as np
# Save resultresult = model(prompts=["a person walks"], duration=4.0)np.savez("walk_motion.npz", **result)
# Load and inspect saved motiondata = np.load("walk_motion.npz")posed_joints = data["posed_joints"]       # [T, J, 3] global joint positionsglobal_rot_mats = data["global_rot_mats"] # [T, J, 3, 3]local_rot_mats = data["local_rot_mats"]   # [T, J, 3, 3]foot_contacts = data["foot_contacts"]     # [T, 4] [L-heel, L-toe, R-heel, R-toe]root_positions = data["root_positions"]   # [T, 3] actual root joint trajectorysmooth_root_pos = data["smooth_root_pos"] # [T, 3] smoothed root from modelglobal_root_heading = data["global_root_heading"]  # [T, 2] heading direction

Robotics Integration

MuJoCo Visualization (G1 Robot)

bash
# Generate G1 motion and save as MuJoCo qpos CSVkimodo_gen "a robot walks forward and waves" \  --model Kimodo-G1-RP-v1 \  --output ./robot_walk.csv \  --duration 5.0
# Visualize in MuJoCo (edit script to point to your CSV)python -m kimodo.scripts.mujoco_load
python
# mujoco_load.py customization patternimport mujocoimport numpy as np
# Edit these paths in the scriptCSV_PATH = "./robot_walk.csv"MJCF_PATH = "./assets/g1/g1.xml"  # path to G1 MuJoCo model
# Load qpos dataqpos_data = np.loadtxt(CSV_PATH, delimiter=",")
# Standard MuJoCo playback loopmodel = mujoco.MjModel.from_xml_path(MJCF_PATH)data = mujoco.MjData(model)with mujoco.viewer.launch_passive(model, data) as viewer:    for frame_qpos in qpos_data:        data.qpos[:] = frame_qpos        mujoco.mj_forward(model, data)        viewer.sync()

ProtoMotions Integration

bash
# Generate motion with Kimodokimodo_gen "a person runs and jumps" --model Kimodo-SOMA-RP-v1 \  --output ./run_jump.npz --duration 5.0
# Then follow ProtoMotions docs to import:# https://github.com/NVlabs/ProtoMotions#motion-authoring-with-kimodo

GMR Retargeting (SMPL-X to Other Robots)

bash
# Generate SMPL-X motion (saves stem_amass.npz automatically)kimodo_gen "a person performs a cartwheel" \  --model Kimodo-SMPLX-RP-v1 \  --output ./cartwheel.npz
# Use cartwheel_amass.npz with GMR for retargeting# https://github.com/YanjieZe/GMR

NPZ Output Format Reference

KeyShapeDescription
posed_joints[T, J, 3]Global joint positions in meters
global_rot_mats[T, J, 3, 3]Global joint rotation matrices
local_rot_mats[T, J, 3, 3]Parent-relative joint rotation matrices
foot_contacts[T, 4]Contact labels: [L-heel, L-toe, R-heel, R-toe]
smooth_root_pos[T, 3]Smoothed root trajectory from model
root_positions[T, 3]Actual root joint (pelvis) trajectory
global_root_heading[T, 2]Heading direction (2D unit vector)

T = number of frames (30fps), J = number of joints (skeleton-dependent)

Scripts Reference

bash
# Direct script execution (alternative to CLI)python scripts/generate.py "a person walks" --duration 4.0
# MuJoCo visualization for G1 outputspython -m kimodo.scripts.mujoco_load
# All kimodo_gen flagskimodo_gen --help

Common Patterns

Batch Generation Pipeline

python
from kimodo.model import Kimodoimport numpy as npfrom pathlib import Path
model = Kimodo(model_name="Kimodo-SOMA-RP-v1")output_dir = Path("./batch_outputs")output_dir.mkdir(exist_ok=True)
prompts = [    "a person walks forward",    "a person runs",    "a person jumps in place",    "a person sits down",    "a person picks up an object from the floor",]
for i, prompt in enumerate(prompts):    result = model(        prompts=[prompt],        duration=4.0,        num_samples=1,        seed=i,    )    out_path = output_dir / f"motion_{i:03d}.npz"    np.savez(str(out_path), **result)    print(f"Saved: {out_path}")

Comparing Model Variants

python
from kimodo.model import Kimodoimport numpy as np
prompt = "a person walks forward"models = ["Kimodo-SOMA-RP-v1", "Kimodo-SOMA-SEED-v1"]
results = {}for model_name in models:    model = Kimodo(model_name=model_name)    results[model_name] = model(        prompts=[prompt],        duration=4.0,        seed=0,    )    print(f"{model_name}: joints shape = {results[model_name]['posed_joints'].shape}")

Troubleshooting

Out of VRAM (~17GB required):

bash
# Check available VRAMnvidia-smi
# Use fewer samples to reduce peak VRAMkimodo_gen "a person walks" --num_samples 1
# Reduce diffusion steps to speed up (less quality)kimodo_gen "a person walks" --diffusion_steps 20

Model download issues:

bash
# Models download from Hugging Face automatically# If behind a proxy, set:export HF_ENDPOINT=https://huggingface.coexport HUGGINGFACE_HUB_VERBOSITY=debug
# Or manually specify cache directoryexport HF_HOME=/path/to/your/cache

Motion quality issues:

  • Be specific in prompts: "a person walks forward at a moderate pace" > "walking"
  • For complex motions, use the interactive demo to add keyframe constraints
  • Increase --diffusion_steps (default ~20-30, try 50 for higher quality)
  • Generate multiple samples (--num_samples 5) and select the best
  • Avoid prompts with extremely fast or physically impossible actions
  • The model operates at 30fps; very short durations (<1s) may yield poor results

Foot skating artifacts:

bash
# Post-processing is enabled by default; only disable for debuggingkimodo_gen "a person walks" # post-processing ON (default)kimodo_gen "a person walks" --no-postprocess  # post-processing OFF

Interactive demo not loading:

bash
# Ensure port 7860 is availablelsof -i :7860
# Launch on a different portkimodo_demo --server-port 7861
# For remote server accesskimodo_demo --server-name 0.0.0.0 --server-port 7860# Then use SSH port forwarding: ssh -L 7860:localhost:7860 user@server

来源与署名

来源:reason-machines/trending-skills位于skills/kimodo-motion-diffusion提交2384a00

许可证: 无许可证

内容归原作者所有。SourceWeft 从公开仓库中收录这些内容。

举报或申请下架