Qiyuan Embodied Research Center Unveils PrimeSim, Exploring Video Generation Models for Robot Simulation

On November 4, Qiyuan Embodied Research Center released PrimeSim, a world model simulator for robot manipulation policy training built on a pretrained video generation model. The team plans to open-source the model weights, inference code, and training code on November 6.

On November 4, the Qiyuan Embodied Research Center released PrimeSim, a world model simulator designed for policy training in robotic manipulation tasks. Built on a pre-trained video generation model and refined through mid-training and post-training, the system attempts to predict environmental changes in response to robotic action commands. The team stated that the relevant weights, inference code, and training code are scheduled to be open-sourced on November 6.

Robotic policy training often relies on simulation environments, but manually constructing these scenes demands substantial engineering effort. Although video generation models can produce highly realistic visuals, they do not necessarily respond accurately to actions, and they may fail to maintain consistent object and robot states across different camera perspectives. PrimeSim addresses these limitations by ensuring the generative model not only maintains visual coherence but also continuously simulates dynamics in response to action commands, functioning as a simulation environment for policy training and evaluation.

According to evaluation results published by the team, PrimeSim achieved an overall score of 6.09 on EWMBench, taking the top spot. In the five tests of WorldSimProbe, a benchmark designed based on LIBERO, it ranked first in three metrics: global trajectory coverage, action-source behavior retention, and interaction dynamics.

PrimeSim employs a two-stage training pipeline. The mid-training stage utilizes 30,110 hours of robot demonstration data spanning 22 robot platforms. It incorporates a unified action representation and multi-view processing to align control commands with the generated outputs. The post-training stage is optimized for continuous rollout, enabling the model to maintain state coherence even when exceeding the training context length, and supporting continuous rollouts of over two minutes.

Evaluations on the RoboTwin 2.0 held-out set using a single A100 GPU showed that the team reduced the model's denoising steps from 50 to 4, boosting the generation speed from 1.12 to 10.04 frames per second, while also improving visual quality metrics. A separate set of experiments compared PrimeSim against the native LIBERO simulator: across 300 shared initial states, the success rates of 8 OpenVLA-OFT policies yielded a correlation coefficient of 0.99 and a mean absolute error of 2.8. The team further reported that policies trained with reinforcement learning in PrimeSim achieved an 11.5% increase in success rate when evaluated on the native simulator.

These results demonstrate a high degree of consistency between the simulated trajectories generated by PrimeSim and policy performance in traditional simulation environments, providing preliminary evidence for training policies within neural simulators.