NVIDIA Proposes ThinkWorld to Train VLAs on Unannotated Human Videos via Chain-of-Dynamics and Imagined Futures

NVIDIA and KAIST introduce ThinkWorld, a vision-language action framework that leverages chain-of-dynamics reasoning and imagined future states to extract training signals from unannotated human videos for robot policy learning.

Collecting robot demonstration data is expensive and covers only a limited range of scenarios and actions. While the internet contains vast amounts of first-person human video, this footage is scattered, unstructured, and lacks actionable labels that robots can directly execute. How can Vision-Language-Action (VLA) models absorb the manipulation experience embedded in these videos?

To address this challenge, NVIDIA and the Korea Advanced Institute of Science and Technology (KAIST) published the paper "ThinkWorld: World Imagination Reinforces Chain-of-Dynamics Reasoning" on October 27. They introduced ThinkWorld, a VLA learning framework that enables human videos without action annotations to provide training signals for robot action reasoning.

The paper's first author is Chaehyun Kim, and the corresponding author is Fu-En Yang, a research scientist at NVIDIA whose research focuses on robot learning and control, as well as embodied reasoning.

The research team translates action imagination into cross-embodiment chains of dynamics (Chain-of-Dynamics), using the alignment between the "imagined future" and the "actual future" as the scoring metric for reinforcement learning. This approach allows videos without action labels to supervise the model's reasoning.

Experiments on the real-robot dual-arm YAM platform show the model achieved a 60/60 success rate on in-distribution tasks and 52/60 on out-of-distribution tasks, yielding an overall micro-average success rate of 93.3%. Aggregated statistics on the project page report an out-of-distribution success rate of approximately 88.9%, significantly outperforming various baseline models.

Even when a robot has only learned isolated atomic skills and the complete task sequence appears exclusively in human video samples, the model can still execute the entire task pipeline in a single rollout.

From action chains to world imagination

Robot actions are typically defined in embodiment-specific control spaces, such as joint angles and end-effector poses. While these instructions can be executed directly, they are difficult to reuse across different robot embodiments and cannot directly interface with human demonstration data.

To address this, ThinkWorld introduces Chain-of-Dynamics (CoD): a sequence of discrete motion tokens that describe the evolution of world states over time, serving as a universal reasoning interface between human and robot actions.

The team first aligns both types of actions to a common scale. They map robot trajectories and annotated human hand trajectories to a unified, normalized end-effector delta representation. A motion tokenizer similar to FAST then segments these into corresponding CoDs from a shared vocabulary. The vision-language model (VLM) autoregressively generates this CoD by integrating task instructions, historical visual context, and current visual observations. This tokenization process does not require the kinematic structures of humans and robots to be identical; it only requires that both types of motion can be mapped to the same discrete vocabulary. The training time window is approximately one second. The motion dimensionality is six for single-arm and twelve for dual-arm tasks, comprising translation and rotation vectors but excluding gripper control.

However, merely generating a CoD does not enable the model to predict how these actions will alter the world.

The research team integrates a learnable future query module after the CoD. Conditioned on a teacher-forced CoD, the model contextually encodes the query and passes it through a lightweight decoder to output a future representation approximately one second ahead. Rather than reconstructing images in the RGB pixel space, the model performs this prediction within a frozen DINO feature space. DINO features preserve the complete spatial and object structures while suppressing appearance details, facilitating the comparison of structural environmental changes. The observed future frames are used exclusively to construct the supervision objective and are not fed back into the VLM during inference.

During the supervised fine-tuning (SFT) stage, the model simultaneously optimizes two learning objectives: first, generating the CoD from observed motions; and second, predicting future visual states based on the generated CoD.

Actual continuous control is delegated to an action expert module. This module receives the VLM representations shaped by the two aforementioned training objectives, rather than directly decoding or utilizing the CoD tokens.

The parameter freezing strategies for these two stages are exactly inverted. During the latter half of the SFT adaptation phase, only the action expert is trained while the VLM backbone remains frozen. Once the process transitions to world-imagination reinforcement learning, only the VLM is updated while the action expert is frozen.

This architecture decouples reasoning and planning from low-level control during training, preventing high-level planning and actual action execution from being entangled in the same parameter update step. Because future prediction objectives can be generated from any temporally ordered human or robot video, this design also lays the technical groundwork for subsequently scoring samples without action labels.

Relying solely on teacher-forced future prediction, the model still cannot independently plan the intermediate dynamics required to reach that future state.

Therefore, the second stage of the research adjusts the training logic: multiple candidate CoDs are sampled within the same visual context, and the system compares the match between the imagined future generated by each chain and the observed actual future. The dynamic chain with better alignment receives a higher reward score.

The reward relies solely on state transitions observed in the video, requiring neither action labels nor the online execution of these candidate actions.

The reward is calculated within the feature latent space. Simply put: the model first "imagines" the scene structure one second later and compares it with the actual next frame in the video; a high match yields a positive reward, while a poor match results in a penalty.

The research adopts a feature similarity calculation method incorporating motion weighting: image patches that undergo larger changes from the current frame to the future frame receive higher weights during scoring. At the same time, it maintains balanced contributions across the entire scene to prevent static backgrounds from dominating the reward results. All motion weights are derived entirely from the observed frames.

The optimization stage uses GRPO, updating the VLM based on normalized world imagination rewards while retaining a supervised replay mechanism to prevent the model from losing the physical grounding capabilities for actions learned during the SFT stage. All other modules remain frozen during RL training. During real-world deployment and inference, the model neither generates CoDs nor performs future predictions.

Two-stage training and single inference

Implemented in a specific configuration, this mechanism uses a Qwen3-VL-2B backbone, inherits the action expert from MolmoAct2, and extracts future representations using DINOv3.

Pre-training consists of two stages: supervised fine-tuning (SFT) and reinforcement learning (RL). The team first completed SFT using 128 NVIDIA H100 GPUs, followed by RL on 64 NVIDIA H100 GPUs, sampling K=4 trajectories per iteration and updating only the VLM. After pre-training, the model undergoes task-specific post-training tailored to the target robot body. ThinkWorld-LIBERO is adapted on the standard LIBERO training set, while ThinkWorld-YAM is adapted on MolmoAct2-BimanualYAM, consuming 32 and 64 NVIDIA H100 GPUs, respectively.

The training data mixes human egocentric videos with robot demonstrations. Human data sources include EgoDex and EgoVerse, while robot data sources cover DROID/BimanualYAM, Bridge/RT-1, BC-Z, and RoboCOIN. The combined source data comprises approximately 769,000 episodes and about 4,276 hours of video. Among these, EgoVerse and EgoDex together account for approximately 1,880 hours, or 44% of the total data; this portion lacks executable robot action labels. During the SFT stage, auxiliary visual question-answering samples are also mixed in to mitigate catastrophic forgetting.

Inference does not carry the computational overhead of training. CoD and world imagination only shape representations during the training phase and do not need to be re-run during deployment. Steady-state latency tests on a single H100 GPU show that ThinkWorld achieves a median latency of approximately 325 milliseconds and a peak memory usage of about 7.07 GiB, outperforming MolmoAct2-Think with its thinking module enabled under the same configuration.

During closed-loop execution, there is another easily overlooked detail: whether the next-step features anticipated by the model can keep pace with the actual environmental changes caused by its own actions. The team used principal component analysis (PCA) projections to compare the imagined future states with subsequently captured frames. The results show that the imagined features can update synchronously with environmental changes. This does not mean that contact-rich control problems have been solved, but it demonstrates that the future representations learned during training remain effective against distribution shifts induced by policy actions.

Real-robot, simulation, and skill composition

The real-robot experimental platform uses a dual-arm YAM equipped with three RealSense D405 cameras: one overhead camera and one on each wrist. Both training and inference stages use only RGB images. The policy performs inference on a single RTX 3090 GPU, with control commands uniformly queried and executed at 15 Hz.

The evaluation includes three manipulation tasks: color matching (placing fruits into bowls of the same color), cup hanging, and sorting. Each task covers at least six object configurations. Out-of-distribution scenarios modify object colors or object categories, and some scenarios also employ motion trajectories never seen during training. Five trials are conducted for each configuration. Before task-specific fine-tuning, the model is initialized using the embodiment-adapted ThinkWorld-YAM model.

The comparison baselines include Diffusion Policy, MolmoAct2, which possesses explicit action reasoning capabilities, and LaWAM, which constrains actions by predicting latent visual subgoals. The complete breakdown of results can be found in Figure 3 and Table 6 in the appendix of the paper.

Under in-distribution conditions, performance differences among the evaluated methods are minimal. ThinkWorld achieves perfect scores across all three tasks: 30/30 for color matching, 15/15 for cup hanging, and 15/15 for sorting. In out-of-distribution scenarios, the performance gaps widen significantly: 24/30 (80%) for color matching, 15/15 for cup hanging, and 13/15 (86.7%) for sorting. Overall, the aggregate in-distribution result is 60/60, and the out-of-distribution result is 52/60, yielding a macro success rate of 94.4% and a micro success rate of 93.3%. For comparison, MolmoAct2 achieves a micro success rate of only 60.0%, while Diffusion Policy and LaWAM perform even worse. The project page reports the in-distribution and out-of-distribution success rates as 100% and 88.9%, respectively, whereas the highest out-of-distribution score among the baselines is just 46.7%.

The research team posits that CoD reasoning uses the physical changes in the world caused by actions as its foundation. This helps the policy focus on core task attributes such as color and category, enabling flexible movement adjustments and reducing the tendency to memorize visual cues or mechanically replicate trajectories. However, there are still failure cases in the out-of-distribution setting, with six failures in color matching and two in sorting.

A more distinctive validation point on the real robot is skill composition. The team designed an oven task where the robot demonstrations covered only isolated atomic skills: opening the door, closing the door, placing the left object into the oven, and placing the right object into the oven. The complete execution sequence was only present in human videos.

Standard end-to-end VLAs learn executable control via robot action supervision. If an entire task trajectory does not appear in the robot data, the model struggles to piece together the complete procedure from human demonstrations. Executing such a task requires the robot to recognize the relevant atomic skills, maintain the correct temporal sequence, and seamlessly transition between them.

Referencing the human demonstration sequence of 'open -> place left -> close', ThinkWorld composed the corresponding robot skills within a single rollout to complete the entire process. During training, the robot was exposed only to isolated skill components, while the assembly blueprint for the full task came from human videos. The team attributes this transfer to a shared CoD reasoning space between humans and robots.

For the simulation evaluation, the study uses two benchmarks: LIBERO and the more challenging LIBERO-PRO. LIBERO encompasses spatial, object, goal, and long-horizon task suites. LIBERO-PRO introduces controllable perturbations in object properties, positions, instruction semantics, task goals, and environmental conditions. ThinkWorld-LIBERO is post-trained exclusively on the standard LIBERO training set and subsequently evaluated directly on LIBERO-PRO in a zero-shot setting.

Test results show that ThinkWorld achieves an average success rate of 97.3% on LIBERO and 55.6% on LIBERO-PRO, ranking first among all models in the tables, despite having approximately 2.9 billion parameters. The appendix reports a zero-shot average success rate of 71.7% on LIBERO-Plus, demonstrating an even more pronounced advantage when facing perturbations in the robot's initial state and object layout.

On the metrics for task perturbation and environmental perturbation, ThinkWorld achieves 31.9% and 60.9%, respectively. This outperforms VLA-JEPA, which also employs latent world modeling, by 11.1 and 19.9 percentage points.

Ablation studies use a baseline of the Qwen backbone combined with the MolmoAct2 action expert, progressively adding visual history, future prediction, CoD, and world imagination RL. The average success rate on LIBERO-PRO increases sequentially from 40.1% to 40.7%, 50.4%, 54.4%, and finally reaches 55.6%.

Future prediction yields the most prominent gains. CoD further improves performance across all shift dimensions, while world imagination RL additionally bolsters robustness to position, task, and environmental shifts. The team also conducted a control experiment, keeping the total volume of robot data constant while continuously increasing the proportion of human video. The overall success rate rose accordingly, with the 50% human data variant slightly outperforming the compute-matched pure robot data control group.

Evaluated in the SimplerEnv environment without task-specific fine-tuning, the model achieves an overall micro success rate of 46.5%. Cross-scenario transfer performance is measured using the YAM real-robot and LIBERO series simulation tests.

Features cluster by motion, not by embodiment

The study first examines what CoD retains between human and robot data. The team extracted internal features from the final thinking units of the model for dimensionality reduction visualization. In the plot, different colors represent the motion directions of the robot arm's end-effector, while marker shapes distinguish whether the data originates from humans or robots. Samples with similar motion trajectories cluster together, with human and robot data intermixed; even within the same data source, samples are re-partitioned according to motion patterns. Model features are primarily constructed around actions, unconstrained by the robot's physical embodiment.

A second control experiment tests whether future prediction depends on CoD. When conditioned on the CoD generated by the model, the predicted future visual features align more closely with actual environmental dynamics; however, if this chain is disrupted, an empty chain is fed, or CoD is turned off entirely, predictions tend to remain anchored to the current frame, and the overall matching similarity drops, with larger gaps in regions of highest variation. Attention visualization also shows that the computational units corresponding to CoD prioritize focusing on human hands, robot arms, and manipulated objects, rarely attending to the static background.

Placing ThinkWorld within the broader technical roadmap, it bridges scalable human video on one end and deployable robot policies on the other. Implicit action learning and explicit action redirection are two parallel approaches; world models and visual foresight can provide signals of the environmental consequences of actions. Past research often directly used future prediction as control output or solely for virtual rollouts. ThinkWorld's design choice is to use consequence alignment during training to reinforce shared dynamics reasoning, and to discard reasoning and imagination during deployment, retaining only the refined policy network.

For researchers, this framework provides a reproducible experimental paradigm: a unified shared motion vocabulary, visual scoring based on DINO features, paired with GRPO, while carefully controlling the data mix between humans and robots. For industrial deployment, the high cost of full-length real-robot demonstrations can be offset by combining "a small amount of real-robot atomic skills with a large volume of complete human manipulation videos", provided that the post-training and task fine-tuning pipelines converge stably.

With a score of 52/60 on out-of-distribution real-robot evaluations, ThinkWorld has transformed the concept of 'using imagined futures for reward scoring' into a verifiable experimental checklist. What remains to be seen is whether it can maintain this same stability when scaled to additional hardware platforms and deployed in noisier operating conditions.