A joint team including UC Berkeley and NVIDIA introduces SuperMimic, a method that trains a humanoid robot controller on a single GPU in three hours, surpassing a 128-GPU baseline. The approach uses a differentiable world model to drastically reduce the compute required for large-scale motion libraries compared to traditional reinforcement learning.

UC Berkeley team proposes SuperMimic, outperforming 128-GPU baseline in 3 hours of single-GPU training

How much training compute does it take for a humanoid robot to master walking, jumping, and even backflips?

The conventional approach involves having the robot continuously trial and error in simulation, relying on reinforcement learning to optimize motion control policies. As the motion library expands, the associated training costs scale up accordingly.

On October 14, a joint team from NVIDIA, the Massachusetts Institute of Technology, the University of California, Berkeley, the University of Oxford, and Carnegie Mellon University published the paper "Robota-Embed: Unified VLA Embeddings for Robot In-Context Learning and Evolution." The paper lists 19 authors, with Yunze Liu and Jing Wang as co-first authors and NVIDIA research scientist Han Cai as the corresponding author.

The author list also includes Yuke Zhu, co-lead of the NVIDIA GEAR team; Trevor Darrell, founding co-director of the Berkeley Artificial Intelligence Research Lab (BAIR); Jim Fan, co-lead of the NVIDIA GEAR team; and Song Han, an associate professor at the Massachusetts Institute of Technology known for his research in model compression and efficient AI computing.

This research targets a major training bottleneck: as the motion library continues to expand, can a unified controller tailored to a large-scale motion set be trained without relying on cluster-level compute?

According to the paper, SuperMimic achieves a normalized return metric that outperforms the publicly released baseline model, SONIC, after just 3 hours of training on a single NVIDIA H100 GPU. In contrast, SONIC required approximately 7 days of training across 128 NVIDIA L40 GPUs. Based on the paper's calculations, SuperMimic achieves a more than 1,000-fold reduction in GPU-hour costs.

The team also deployed the trained control policy on a physical Unitree G1 humanoid robot, successfully executing jumps, lateral rolls, and backflips.

The core concept behind SuperMimic is not to train an independent controller for each individual motion. Instead, it relies on a single differentiable world model to support learning across the entire motion library.

First, learn how the body moves

To understand the high costs of previous approaches, we must first break down their root cause. Mainstream motion tracking today largely relies on model-free reinforcement learning, with Proximal Policy Optimization (PPO) being the representative algorithm. These methods treat the physics simulator as a black box, relying on massive sampling to evaluate motion quality. Because this sampling process is highly noisy, it requires large-scale simulation to reduce variance. As the motion library grows, the required simulation volume increases proportionally, ultimately forcing a brute-force scaling of compute.

The core improvement in SuperMimic is the introduction of a world model. The world model is a neural network that takes the robot's current state and action as inputs to predict the next state, while simultaneously outputting the immediate reward, robot observations, and the risk of falling. Thanks to its differentiable nature, the policy can backpropagate gradients through the network to every action step, directly determining the parameter update direction. This method is known as Backpropagation Through Time (BPTT).

SuperMimic uses a single world model to support learning across the entire motion library. All motions share the same robot body dynamics. Adding new motion segments merely supplements the model with samples of body and contact interactions, eliminating the need to expand model capacity.

The team did not modify the simulator. The world model learns from data generated by a standard rigid-body simulator, meaning the simulator itself does not need to support differentiable computation. Non-smooth physical processes, such as collisions and contacts, are still handled by the native simulator.

The diagram illustrates the three-step training loop of SuperMimic. First, the policy πθ runs in the simulator Sim, and the experiences are stored in a replay buffer where new data replaces old data. Second, segments are sampled from the buffer, and the world model fω predicts the next states based on actions and aligns them with the real trajectories. Third, the world model is frozen, and the policy imagines 32 steps within it, with the imagined returns backpropagated to update the policy. The loop also involves the loss function LWM and policy updates, providing an intuitive overview of the SuperMimic training process.

The overall training follows a three-stage iterative process.

In the first step, the current policy collects trajectories in the simulator and stores them in a replay buffer, where new data overwrites older samples;

Second, continuous eight-step segments are sampled from the buffer. Starting from the initial state, the world model makes autoregressive predictions based on historical actions and aligns them with the ground-truth trajectories.

Third, the world model is frozen while the policy performs 32-step imagined rollouts within it. The returns from these virtual rollouts are then backpropagated to update the policy.

These three steps are iterated alternately, jointly optimizing the world model and the policy. Data collected by older policies remains usable for training the world model because the robot's physical dynamics are independent of the data-collection policy, yielding higher data utilization.

During the data collection phase, noise is added to the actions to broaden the action distribution observed by the world model. Randomization variables in the simulation—such as ground friction, joint calibration errors, and torso center-of-mass offsets—are fed into the model as additional inputs. This allows the world model to remain deterministic, resulting in more stable predictions.

The virtual rollout predicts only 32 steps forward, while long-term returns are estimated by a critic network to evaluate the total subsequent reward. This requires the world model to maintain prediction accuracy only within the 32-step horizon. The team also noted that during training, the model autonomously corrects its prediction biases, effectively enabling stable rollouts over longer horizons.

The policy input consists of the robot's proprioceptive observations: 29 joint positions and velocities, IMU angular velocity, gravity direction, and the previous action, using the most recent 10 frames. The optimization target is the subsequent 10-frame trajectory of the reference action. All model networks are multilayer perceptrons; the world model dynamics network comprises three layers with 2,048 neurons each.

Two rounds of filtering yield 129,514 actions

Once the training scheme was finalized, a sufficiently large and physically feasible action library was still needed. The team selected action clips from the public BONES-SEED dataset that had been mapped to the G1 robot, starting with an original total of 142,220 clips. Many of these samples involved scenarios such as climbing stairs, carrying boxes, and sitting on chairs, or contained mapping errors. Because the G1 cannot execute these maneuvers in a flat environment, they required pre-filtering.

The first round consists of coarse keyword filtering. Each motion clip is named after the behavior, and any sample whose name contains keywords such as stairs, ladders, boxes, chairs, sitting, or climbing is directly discarded. The keyword list is adapted from the open-source data processing code of SONIC, with the team supplementing it to cover any missing terms. This round filters out 12,553 samples, leaving 129,667.

The second round performs physical feasibility checks to filter out invalid motions that defy logical naming, such as walking onto a box. The team runs each motion in MuJoCo for kinematic playback, monitoring the height of the lowest point on the robot's body. If this lowest point stays above 0.20 meters for 0.6 consecutive seconds, it is flagged as stepping on a foreign object. If it drops below -0.12 meters, it is flagged as the leg penetrating the ground. Additionally, if the feet are more than 0.02 meters off the ground while stationary for 0.5 seconds, the motion is also marked as infeasible. This round removes another 153 samples, ultimately yielding 129,514 valid motions with a total duration of approximately 264 hours.

This motion library corresponds to the publicly available training subset of SONIC. The total duration of the motions used by SONIC is 611 hours, whereas SuperMimic requires only 264.56 hours, reducing the dataset size by a factor of approximately 2.3.

During training, the robot does not always restart from the initial position of the motion. The team divides the motion timeline into 1-second segments and records the score for each segment. The environment reset strategy employs mixed sampling: a 50% probability of uniform random selection, a 40% probability of biased sampling toward low-scoring, difficult segments, and a 10% probability of sampling based on historical difficulty. This ensures that previously learned high-difficulty motions can be repeatedly reviewed.

For the training configuration, a single GPU launches 4,096 simulation environments, and each policy update uses 4,096 virtual rollout trajectories, training for a total of 320,000 iterations. The simulator uses the GPU-accelerated version of MuJoCo Warp, with a control frequency of 50 Hz. All compute costs were measured on NVIDIA H100 GPUs on Lambda Cloud. The team repeated the experiments using multiple random seeds, and the training process remained stable. The paper attributes this stability to the progressive reward shaping used in the motion tracking task.

Three hours on a single GPU, failures drop from 179 to 25

First, consider the direct comparison between SuperMimic and SONIC. Both methods consistently adopt the motion tracking setup from BeyondMimic, with SONIC serving as the most relevant public baseline currently available. The evaluation metric is normalized return, which represents the ratio of the total score for the entire motion to the perfect score. If the robot falls midway through the motion, the score is penalized.

This figure comprises three subplots. Subplot (a) is a normalized return comparison chart showing the scores of SuperMimic trained on one and two H100 GPUs, alongside the score of the SONIC model trained on 128 L40 GPUs, with compute costs for different durations annotated, clearly demonstrating that SuperMimic's three-hour single-GPU training outperforms the 128-GPU baseline. Subplot (b) is a line chart of the compute required to reach the target versus dataset length, and subplot (c) is a line chart of the 8-step prediction error versus dataset length. Together, they illustrate the changes in compute and prediction error, reflecting that as training time increases, the compute required to reach the target grows slowly, while the world model's prediction error remains stable.

After just three hours of training on a single H100, SuperMimic's score surpasses that of the SONIC model, which required 128 L40 GPUs trained for seven days to achieve a final score of 0.90. Wall-clock time is reduced by a factor of approximately 56, and GPU hours are reduced by a factor of approximately 7,200. The team's selection of the three-hour mark is conservative; from that point onward, every saved model checkpoint outperforms SONIC in both score and the number of failed motions. These three hours account for the complete training loop and exclude the time required for subsequent evaluation.

The team also noted hardware differences between the two experimental setups. The FP32 peak compute of the H100 is 0.74 times that of the L40, while its tensor core peak compute is 5.5 times higher. However, neither experiment maxed out the hardware's peak performance. The team concluded that the 2.3-fold difference in data volume and the up to 5.5-fold hardware difference cannot explain the more than 1,000-fold discrepancy in GPU-hour costs.

When tested individually across all 129,514 motions, SuperMimic failed on 25 motions after three hours of training. The publicly available SONIC model failed on 179 motions, while SONIC v1.1 failed on 239. The SONIC failures included all 44 backflips, all 22 side rolls, and eight segments of the full dance routine, as well as 15 failures out of 24 standing long jump samples. All these samples belong to the training sets of both models and are used to evaluate how completely each model covers the training motions.

SuperMimic's failures were concentrated in a few high-difficulty motion groups. The top five groups in the test comprised a total of 195 motions, accounting for 14 of the 25 failures. There was no motion group where SuperMimic's failure count exceeded that of SONIC. After training on two GPUs for 9.8 hours, the model achieved zero failures across all motions.

To eliminate interference from experimental settings, the team built a strict control baseline, PPO-Sim. This baseline runs the PPO algorithm directly in the simulator, with actions, observations, rewards, fall detection, parameter randomization, network size, and evaluation rules all aligned with SuperMimic. The number of simulation environments per GPU was even twice that of SuperMimic. The test selected 40 motion segments from the LAFAN1 dataset, totaling 2.45 hours.

This figure contains three subplots showing experimental results for the Berkeley team's SuperMimic. The left subplot (a) displays the normalized score over wall-clock time for SuperMimic (single GPU) and the PPO-Sim method with varying numbers of GPUs on the LAFAN1 dataset, showing that the single-GPU SuperMimic achieves the fastest score increase, far outperforming PPO-Sim across all GPU counts from the initial stage. The right side is divided into subplots (b) and (c). Subplot (b) shows the final scores with and without a critic network under different imagination steps H; at H=32, both configurations reach a high level of 0.8. Subplot (c) shows the time required for both configurations to reach a score of 0.82 under different imagination steps; at H=32, the times are similar, and both configurations achieve the target score only when H is no less than 16.

Experimental results show that SuperMimic achieves a score of 0.82 after just 1.2 hours of single-GPU training. PPO-Sim never reached this score within an 18-hour compute budget. Its final score with single-GPU training was 0.60, rising to 0.70 with four GPUs, and only improving to 0.72 with eight GPUs, indicating almost no benefit from scaling the compute from four to eight GPUs.

The team identified three core characteristics of the world model that significantly reduce compute requirements.

First, the model's predictions are stable, and its performance does not degrade as the data grows. When the training data was expanded from one hour to 264 hours, the eight-step prediction error remained stably within the 0.007 to 0.011 range, and the number of GPU hours required to meet the target only increased slowly.

Second, the rollout speed is significantly faster. Across 2,000 to 16,000 parallel environments, the rollout speed is more than 10 times higher than that of MuJoCo Warp, and in an environment with 64,000 parallel instances, the speedup reaches 62 times.

Third, it achieves higher gradient quality. For walking, running, and sprinting motions in the LAFAN1 dataset, operating within the same world model environment, SuperMimic requires approximately 64 times fewer rollout steps to reach optimal PPO performance. All comparisons in this set were conducted within the world model under identical hardware speed conditions, with the performance gap stemming from the first-order gradient update mechanism.

The team also independently verified the role of the critic network. Without the critic network, training failed outright when the maximum rollout steps were capped at 16. With the critic network included, just 16 rollout steps matched the performance of 32 steps in the critic-free version, which was the optimal training configuration. Increasing the rollout steps from 64 to 128 nearly doubled the time required to reach the target score while the final score dropped accordingly; 128 steps also represents the maximum VRAM capacity of a single H100 GPU.

Even when achieving identical scores, the critic network alters the robot's behavior. In simulation experiments, the robot was instructed to maintain a T-pose while being subjected to lateral pushes at speeds ranging from 0.5 to 4 meters per second. The policy equipped with a critic network recovered its balance within 0.5 to 2.4 seconds. In contrast, the policy without the critic network failed to maintain stable balance, stumbling under a 0.5 m/s push and falling over completely at 4 m/s. The root cause is that the rewards associated with balance recovery typically fall outside the 32-step rollout window.

This image shows motion demonstrations related to the Berkeley team's SuperMimic, featuring the Unitree G1 humanoid robot executing four different motions under a single universal tracking policy without task-specific tuning. The image is divided into four areas corresponding to the labeled motion categories: (a) a jumping motion, (b) a lateral roll with the torso and limbs on the ground, (c) a 90-degree backflip, and (d) a 360-degree stationary backflip. The time intervals for each motion sequence are marked on the right side of each area, with 0.2 seconds for a, c, and d, and 0.3 seconds for b, covering locomotion, full-body ground contact, and acrobatic motions.

In real-robot experiments, the same universal tracking policy was deployed directly onto a Unitree G1 without fine-tuning parameters for individual motions. The four demonstrated skills included hopping, lateral rolling with the torso and limbs on the ground, a 90-degree aerial flip, and an in-place 360-degree aerial flip, covering three categories of scenarios: locomotion, full-body ground contact, and acrobatic stunts. During training, only the physical parameters were randomized, which the team found sufficient to achieve sim-to-real transfer.

The team also explored a downstream application: controlling the robot to walk by receiving velocity commands. Using the same world model, they trained a velocity encoder with a velocity-tracking reward. The baseline used an action-tracking reward for knowledge distillation. In tests comprising 672 step commands spanning the full velocity range, the velocity-tracking approach yielded lower errors in 650 commands, while the distillation baseline performed better in only 15. The median relative errors for the two approaches were 0.16 and 0.49, respectively. Only the distillation method experienced falling issues, with failures concentrated in 7 commands that combined translation with rapid turning—commands for which corresponding samples were virtually nonexistent in the motion library.

The authors also outlined current limitations. Research on multi-interface joint training remains incomplete; the velocity reward currently updates only the velocity encoder and not the shared decoder. The world model was trained exclusively on simulation data and has not yet incorporated real-robot data. Furthermore, the model was trained only on flat-ground scenarios, while extensions to complex terrain and object interaction remain ongoing.