Beyond Prediction: Xiaomi Turns World Models into Data Engines

On July 14, Xiaomi released a video showcasing its self-developed robot performing continuous tasks in a factory. The footage demonstrates the robot's ability to maintain stable operations over extended periods despite handling workpieces of varying sizes.

However, the robot's stable performance in the factory is merely the visible outcome. What truly supports this result is the underlying model capability behind the robot. Every stage—from perception and decision-making to final action generation, all determined by the model's capabilities—impacts whether the robot can sustain task completion.

As robots transition to real production lines, Xiaomi's team has been simultaneously advancing model development and building an open-source ecosystem.

In this context, the Xiaomi Robotics team recently released and open-sourced Xiaomi-Robotics-U0 (hereinafter referred to as U0), a multimodal autoregressive model with 38 billion parameters designed for embodied synthesis.

U0 is the first unified embodied synthesis model that seamlessly integrates five core functions: 'basic image generation, image editing, embodied scene generation, embodied transfer, and embodied video generation.' It retains the world model's original visual generation capabilities while effectively transferring its generalization abilities to embodied generation.

Furthermore, through the first unified multi-view embodied generation framework, U0 decouples the workspace, background, foreground irrelevant objects, target objects, and lighting into independent dimensions. This achieves scalable embodied video enhancement while maintaining geometric consistency and interactive dynamics.

Extensive simulation and real-robot experiments have proven that the embodied video data generated by U0 can serve as a data source for downstream policy training, helping robots better adapt to complex and dynamic real-world scenarios.

Indeed, this marks the world model's further expansion of its application boundaries into the data engine domain, following its prior roles in policy evaluation, trajectory prediction, and action generation, thereby providing embodied AI with closed-loop capabilities for autonomous synthetic data creation.

Embodied capabilities are a "natural extension," not a "domain-specific specialization"

Currently, most mainstream embodied world models follow the path of "fine-tuning foundational generative models on robot data."

This approach may seem viable, but the cost is that the visual knowledge and generalization abilities learned from massive internet data during the pre-training phase will be largely erased during domain-specific specialization.

This leads to U0's core design philosophy: treating embodied generation as a natural extension of foundational image and video generation, rather than a specialized task requiring separate handling.

As shown in the figure above, U0 mixes data from both general-purpose and embodied domains and conducts continuous training within a unified autoregressive framework. The entire model architecture is built upon the open-source EMU3.5, a decoder-only Transformer based on Qwen-3-32B. Notably, U0 adopts next-token prediction as its sole training objective, with both model inputs and outputs encompassing visual tokens, text tokens, and control tokens, thus integrating multimodal information.

The specific design of training tasks is also noteworthy; U0 categorizes training samples into two main types: "single-step" and "sequence."

The first category involves single-step training that covers four core functions. Text-to-image (T2I), representing the model's basic capability; mixed image generation (X2I), which supports control via 1–3 reference images and text instructions; the core function, generating geometrically consistent multi-view initial observations of a robot given its type and scene description; and critically, generating multi-view RGB images that preserve the robot's interaction state, given source scene multi-view depth and a structured description of the target scene.

The second category is sequence training, responsible for modeling the time dimension. Training samples mainly include interleaved text-and-image subtask-subgoal sequences and multi-frame-rate embodied operation videos. The former enables the model to learn causal reasoning through text instructions and post-execution multi-view observations (instruction -> post-execution image -> instruction -> post-execution image). The latter allows the model to utilize cross-resolution joint training by sampling the same operation video at three frame rates (FPS 1/3/5), enabling it to learn planning, capture details, and understand dynamics.

Notably, when U0 learns to maintain geometric consistency across views using embodied transfer data, this 3D perception capability reciprocally enhances its general image editing quality. Meanwhile, the rich visual concepts in general text-to-image data help prevent embodied scene generation from collapsing into a few fixed desktop scenarios found in the training set.

Training is complete, but for the autoregressive model with 38 billion parameters to truly perform image generation, the critical bottleneck is inference acceleration.

U0's inference acceleration solution integrates 'FlashAR+ Extension' with vLLM. Traditional autoregressive generation is slow because it draws line by line, wasting computational resources while waiting to finish one row before starting the next. FlashAR+ adopts a diagonal drawing approach: what previously required serial execution for 'rows × columns' steps is reduced to just 'rows + columns' steps. On a single H20 GPU, generation time per image is compressed from 450 seconds to 16.6 seconds, a 27x speedup.

Combined with vLLM's continuous batching and KV cache management, this achieves an 82.9x acceleration in image generation, transforming U0's embodied data generation from merely usable to highly efficient.

As a Data Engine, U0 Treats Scene Generation Like 'Building Blocks'

A model with 38 billion parameters relies on data as its foundation. U0's training data spans six domains: general image-text, embodied manipulation, autonomous driving, egocentric vision, 3D reconstruction, and gaming, totaling 9.5 million single-step samples and 2.6 million video clips.

Thus, U0's four-branch decoupled annotation pipeline truly demonstrates Xiaomi's engineering prowess.

As shown in the figure above, the upper part illustrates scene description generation, while the lower part highlights trajectory-level sub-task decomposition. The entire data processing pipeline uses Qwen3-VL-235B as the annotation engine, with four branches performing their respective roles independently:

• General Scene Annotation. Sampling all video data without domain restriction, selecting high-definition key frames, and generating overall scene descriptions via VLMs, covering scene layout, object interaction, and spatial relationships. This builds general visual commonsense.

• Structured Embodied Scene Annotation. As the most original design in the data processing pipeline, this branch divides embodied scenes along five semantic dimensions: workspace, task objects, irrelevant objects, lighting, and background. Each dimension generates independent descriptions that do not depend on one another. As shown in the figure below, due to this decoupled division, subsequent random replacements can be generated for the background, workspace, and lighting dimensions using Qwen3-VL.

• Geometric Modality Extraction. Extracting temporally consistent inverse depth maps (brighter pixels indicate closer surfaces, darker ones farther) from all frames via Video Depth Anything. Through these depth maps, the model learns real 3D structures.

• Trajectory-level subtask decomposition. HDBSCAN clustering is performed using end-effector poses and gripper signals to automatically identify boundary signals for motion transitions and grasping transitions. Based on these signals, sampling is conducted in areas with dense physical interactions and sent to a VLM to generate unified-time subtask plans. Transition frames follow the scene annotation process of Branch 2, ultimately yielding training samples for embodied scene generation and embodied transfer tasks.

It should be noted that after the four branches of the data pipeline complete annotation, the annotated training data still needs to be regrouped according to three training objectives. Embodied transfer samples take system instructions, scene descriptions, and multi-view depth maps as inputs; scene generation samples pair system instructions with scene descriptions; video generation samples include system instructions, task instructions, and initial observation frames, along with robot actions and image sequences attached to the initial observation frames, collectively serving as guidance for video generation.

In fact, during data filtering, U0 prioritizes retaining samples with novel visual layouts, manipulation skills, or temporal stages, ensuring that while the training distribution eliminates redundancy, it also guarantees comprehensive coverage of manipulation skills.

Winning at Video Generation Without Sacrificing Physical Manipulation

U0's experiments cover four dimensions: video generation, embodied transfer, scene generation, and real-world policy enhancement. The results demonstrate that the model achieves video generation capabilities comparable to mainstream video models while showcasing physical reliability through specific embodied manipulation tasks, giving the overall experiments high persuasiveness.

First, regarding video generation capabilities, as shown in the figure above, on the World Arena benchmark, U0 (UNIS) achieves an overall EWMScore of 73.64, ranking first among over 100 models.

Specifically, as shown in the table above, U0 achieved the highest scores across all four sub-indicators: controllability (instruction following at 93.86), physical adherence (interaction quality at 87.30), 3D accuracy (perspective at 98.84), and motion smoothness (95.51). This demonstrates that U0's standout advantage extends beyond video generation to robot motion control.

Additionally, for embodied capability evaluation, the Xiaomi team constructed an embodied transfer benchmark comprising 300 samples (150 Easy + 150 Hard). The evaluation was conducted from three perspectives—depth consistency, structural fidelity, and semantic alignment—directly benchmarking against GPT-Image-2.0.

As shown in the table above, U0 comprehensively outperformed GPT-Image-2 across all eight metrics, with consistent advantages in both the Easy and Hard subsets. The root cause of this difference lies in GPT-Image lacking multi-view geometric constraints; it essentially generates three seemingly 'reasonable' images independently. In contrast, U0 embeds geometric conditions into the generation process through unified multimodal joint training targeting a single objective.

As shown in the image above, U0's architecture and training approach ensure that the model output strictly adheres to the depth geometric conditions of each view, maintaining consistency in object identity, position, and appearance. Consequently, both embodied scene generation and embodied transfer performance significantly surpass the video generation model GPT-Image.

The final and most compelling experiment was U0's performance in the real world. The Xiaomi team assigned three tasks to U0: storing headphones, folding towels, and packing boxes. Each task was divided into varying numbers of stages as shown in the figure below. More ingeniously, the Xiaomi team added an interference experimental group on top of the original basic experimental group for U0 (introducing previously unseen tablecloths, altering lighting conditions, etc.), creating a two-layer evaluation framework.

Originally, the strategy was based on π0.5 and deployed on the ARX dual-arm platform, with approximately 40 hours of real-world demonstrations collected for each task. As shown in the experimental results below, the baseline group (dashed fill) performed comparably to the original strategy, indicating that the model design represents a "natural extension" of the policy rather than "domain-specific specialization." However, in the perturbation group (solid fill), the enhanced strategy comprehensively outperformed the original one, achieving an overall Out-of-Distribution (OOD) completion rate of 63.2%, a net improvement of 26.3 percentage points.

The key behind this lies in invoking Qwen3-VL to generate 1,000 descriptions each for the workspace, background, and lighting conditions while keeping the task objects unchanged. After random combinations, U0's embodied transfer capability is utilized to generate multi-view RGB data based on the original depth maps. Finally, U0 mixes this batch of embodied transfer data with the original data for policy training, significantly enhancing generalization capabilities.

What's more interesting and intuitive is that, as shown in the figure below, the model demonstrates its closed-loop embodied data generation capability. U0 first uses scene generation capabilities to create an embodied scene, and then, using the initial frame of that scene along with a language instruction, it can directly generate an operation video. This so-called agent-style embodied data generation capability of U0 means that foundational world models can unfold through autonomous interaction, continuously producing increasingly diverse robotic experience data.

These experiments clearly show that U0 not only uses the foundational world model as a "world simulator" but also positions it as an "scalable data engine."

In Conclusion

In fact, beyond surpassing these important baselines and performing well in the real world, the technical route taken by Xiaomi's U0 and its demonstrated engineering implementation capabilities are equally worthy of attention.

Xiaomi U0 does not follow the traditional path of "foundation model → task-specific fine-tuning." It defines embodied generation as a natural extension of foundational generation, with five tasks—general text-to-image, image editing, embodied scene generation, embodied transfer, and embodied video generation—sharing a single set of parameters and one optimization objective. Visual knowledge from the general domain and geometric understanding from the embodied domain naturally merge during training without compromising either.

Furthermore, U0 employs a four-layer decoupled annotation pipeline and five-dimensional scene decoupling. This not only directly generates diverse data combinations but also proves that the method of "exchanging better data organization for more data" is viable and can alleviate the cost pressures traditionally caused by "exchanging more labor hours for data."

Of course, Xiaomi's confidence in developing U0, a 38 billion parameter model, stems from its robust engineering optimization capabilities.

As mentioned in the original article, a 38000000000-parameter autoregressive model requires approximately seven and a half minutes to generate a single 1,024×1,024 image on one H20 GPU. At this speed, deployment on the production line is impossible. However, the team managed to reduce latency to just 5.44 seconds through post-training with "FlashAR+" and vLLM engineering optimizations, achieving an 82.9-fold acceleration while maintaining essentially unchanged quality.

This ability to "run papers into usable systems," demonstrated through algorithmic innovation and inference stack integration in U0, is actually quite rare in the field of embodied models.

However, Xiaomi's manufacturing scenarios are precisely the best breeding ground for such engineering capabilities.

From making smartphones to Xiaomi home products, and now to cars, the differences in appearance between material batches on Xiaomi's factory production lines, changes in lighting over time, and layout adjustments for new workstations present countless challenges that are nearly impossible to fill through manual data collection alone. Yet, these exact scenarios serve as the ideal testing ground for robot capabilities.

This may be the reason why U0 insists on pursuing a generative approach. Compared to mainstream world models used for predicting and guiding robot motion control, U0 acts more like a data engine capable of amplifying limited real-world data into nearly infinite training samples. On July 16, Xiaomi simultaneously open-sourced Robotics-1, further completing the model landscape from data generation to robot control.

The two, appearing in sequence, correspond to data and strategy respectively, offering a new implementation path for the data closed-loop of Embodied AI.