NVIDIA Unveils GR00T 2 as Jim Fan Questions the Need for In-House Robot Foundation Models

Speaking at CoRL 2026, NVIDIA GEAR team leader Jim Fan introduced the GR00T 2 world-action model and a modular architecture, signaling a shift toward externally sourced and adaptable intelligence for robotics companies.

"I have never seen anything in human history that could shatter so many dreams."

On November 9, U.S. local time, at the 2026 Conference on Robot Learning (CoRL 2026), Jim Fan, head of the NVIDIA GEAR research team, opened his presentation with this joke.

He used the "five stages of grief" to describe his peers' reactions to GPT-6 Astra: denial, anger, bargaining, depression, and finally, acceptance. Upon reaching that final stage, he offered his conclusion: "Winter is coming."

Behind the joke lies an unavoidable industry question: now that general-purpose reasoning models can understand environments, plan autonomously, write code, and even invoke tools to control robots, how much need remains for robot foundation models dedicated specifically to real-time action?

Fan did not dwell on debating this premise. "Everything will become a skill, including System 1," he stated. In his view, the real question is: what constitutes a good skill? Can it be easily taught through demonstration, and easily guided through reasoning?

GR00T 2 is being developed precisely along these two axes: first, enabling robots to learn manipulation skills from human and device demonstrations; and second, allowing higher-level reasoning models to invoke, guide, and refine these operations via APIs.

NVIDIA refers to this collaborative paradigm as "Unified Cognition": dividing responsibilities between a reasoning model and a physical execution model, which then interact through APIs to invoke, guide, and correct one another.

During the same presentation, NVIDIA also announced it will gradually make available the GR00T 2 model, datasets, simulation tools, and agent frameworks, as well as the design schematics for its data collection hardware.

These developments collectively point to a shift: capabilities that robotics companies once had to develop in-house and tightly couple with their own hardware are increasingly becoming accessible externally, adaptable across different devices, and continuously improvable through software.

As the robot brain can now be decomposed, invoked, and reused, how necessary is it for robotics companies to train a complete intelligence system from scratch?

The robot brain doesn't need to do everything

For robots, understanding tasks and executing actions are two distinct processes.

Tasks like garbage sorting and solving the Tower of Hanoi require semantic understanding and logical planning; peeling label backing and inserting RAM modules into slots, however, demand continuous visual feedback and low-latency motor control.

A single model tasked with both often compromises one for the other. Jim Fan argues that it is entirely unnecessary to ask a 2-billion-parameter action model to write recursive code.

NVIDIA chose to assign these two tasks to different models, linking them via an interface.

In April of this year, Jim Fan half-jokingly held a "moment of silence" for the VLA paradigm at the Sequoia AI Ascent conference, pivoting instead to emphasize the potential of world-action models.

He argued that relying solely on representations derived from vision-language models to learn action outputs is not necessarily the optimal path for robots to understand physical interactions. Instead, models should learn the relationship between actions and the environment by predicting future state changes in the world. At the time, NVIDIA showcased world action model research such as DreamZero. Released over half a year later, GR00T 2 further advanced this approach.

Unlike many vision-language action (VLA) models that primarily learn the mapping from observations to actions, GR00T 2 simultaneously predicts future video frames and robot actions within a single diffusion Transformer. Jim Fan described this process as the robot "dreaming." This design allows the robot to learn not just how to execute a specific action, but also how that action will affect objects and the environment. The latter is especially critical for robots that require fine manipulation.

Building on this foundation, NVIDIA introduced the WAM API, enabling higher-level reasoning models to use GR00T 2 as a manipulation tool.

For example, in a waste sorting task, Astra identifies the object categories and directly marks the grasp and placement points in the camera feed, while GR00T 2 executes the actions based on these visual cues. This mechanism is known as Visual Steering.

In a series of target selection and placement tests using the YAM robot, GR00T 2 achieved a 40% success rate when given only task instructions. This improved to 66% with the addition of step-by-step instructions generated by a higher-level planner, and reached 92% when visual markers were also included.

Moreover, these visual markers can be extracted from the hand and gripper states in existing demonstrations, eliminating the need to collect new demonstration data specifically for this interaction modality.

NVIDIA also demonstrated another interaction mechanism called WAM Edit. Astra continuously monitors the camera feed and the robot's proprioceptive state, applying incremental corrections to the actions only when necessary, while GR00T 2 executes normally the rest of the time.

In a chemistry experiment demonstration, Astra determines whether the operation should proceed based on task requirements, guiding the robot to adjust the knobs via the microscope feed until focus is achieved.

Both mechanisms share the same underlying philosophy: the high-level reasoning model determines what to do and evaluates whether it is done correctly, while the robot action model handles the specifics of how to execute it.

At a deeper level, NVIDIA also needs to address real-time execution on the robot. To this end, it introduced GR00T 2 Turbo. This lightweight version of the 2-billion-parameter joint video-action Transformer enables world-action models to run directly on the robot body by compressing video representations, shrinking the model, and optimizing inference.

On the Unitree G1, the Turbo model runs on Jetson Thor with a single inference latency of 124 milliseconds, generating a full-body action plan of approximately 1.6 seconds every second; these plans are then decomposed into low-level joint commands at a 50 Hz frequency by the SONIC controller on Jetson Orin.

As a result, the entire system operates across three distinct timescales: Astra handles the relatively time-consuming tasks of comprehension and planning, GR00T 2 generates action segments, and SONIC translates them into continuous joint control. Reasoning, action prediction, and motor control each operate independently.

Similar hierarchical designs have also emerged in products from other companies.

In January this year, Figure released Helix 02, which also adopts a hierarchical structure of System 2, System 1, and System 0, linking semantic reasoning, visuomotor policies, and full-body control. The difference is that Figure places greater emphasis on deep synergy between its proprietary body, sensors, and models; NVIDIA, on the other hand, has further turned the collaboration between different levels into development interfaces, allowing high-level agents to invoke, guide, and correct action models, and to coordinate execution with external code skills.

However, NVIDIA has not dismissed end-to-end learning. Internally, GR00T 2 still jointly learns visual changes and actions; it is simply that at the level of the complete robot system, tasks of different natures and timescales are delegated to different modules.

If the robot brain can be partitioned, can manipulation skills be further freed from the constraints of specific hardware?

Learning new skills without starting from scratch every time

Robot training data has long been tightly coupled with the data collection devices and the target hardware. Teleoperation data for a specific robotic arm records the states of its particular joints, grippers, and control systems. If the end-effector is swapped for a different gripper or a dexterous hand with a different kinematic structure, the data may not be directly transferable. Even for the exact same task, deploying the data on a different robot may require collecting additional demonstrations, recalibrating, and retraining.

X-Adapt (Cross-Adaptation), introduced in GR00T 2, aims to reduce such repetitive costs.

The approach involves first learning manipulation skills on devices where data collection is easier, and then transferring those skills to the target robot. This eliminates the need to collect large amounts of task-specific demonstrations for every new task.

For example, a human operator uses a UMI handheld gripper to demonstrate the assembly of a toy dump truck. NVIDIA then fine-tunes the model using only these UMI task demonstrations and commands a YAM robotic arm equipped with a different gripper to execute the full assembly sequence.

In official tests, the system achieved a 60% success rate on this task, which takes up to approximately 4 minutes.

In another experiment, a human operator wears a camera and motion-capture gloves to perform chemical lab procedures. The robot learns tasks such as placing and wetting filter paper from this data, eliminating the need to collect target-robot teleoperation demonstrations for these specific tasks.

NVIDIA has also developed DexUMI, an exoskeleton device that captures more fine-grained human hand movements by matching the joint structure of the target dexterous hand.

These experiments demonstrate that the data required to learn a task does not necessarily have to come entirely from the robot that ultimately executes it.

This transfer capability is tied to the large-scale pre-training of GR00T 2. NVIDIA disclosed that its pre-training dataset comprises 402,730 hours of human egocentric video, 10,184 hours of UMI handheld device demonstrations, and 23,500 hours of robot teleoperation data. By raw duration, human video accounts for more than 90%. This video provides extensive interaction experience across diverse tasks, objects, and scenes, while the UMI and robot data supplement it with more explicit manipulation details and motor experience across different embodiments. In a series of ablation experiments, as the proportion of human video used by the model increased, the average task completion progress of the physical robot across 72 downstream tasks improved from 50% to 70%.

This difference is most pronounced when tackling novel tasks. NVIDIA selected 50 tasks that the target robot had not been specifically trained on, but which appeared in the human videos or other robot data. Without task-specific fine-tuning, GR00T 2 achieved a zero-shot average success rate of 40%. After providing a single demonstration from the target robot for each task and performing LoRA fine-tuning, the success rate increased to 71%. With five demonstrations, it reached 95%. Notably, the five demonstrations for each task totaled less than 3 minutes, and a single fine-tuning run on one RTX 5090 took approximately 30 minutes.

These experiments demonstrate that general capabilities acquired during pre-training can be repeatedly leveraged, thereby reducing the task-specific demonstrations required for new tasks. For robotics companies continuously deploying to new workstations and adapting to novel tasks, this approach could significantly reduce the workload associated with repetitive data collection and training.

However, mastering the task sequence is still far from reliably executing high-precision manipulations. In the few-shot experiments with GR00T 2, many failures occurred not because the model misunderstood the task, but due to insufficient precision in specific execution steps. For instance, the model knows it needs to insert a plug into a socket and can correctly sequence the operations, yet the final few millimeters of the insertion may still fail due to positional, postural, or contact errors.

NVIDIA therefore introduced real-robot reinforcement learning.

Across three contact-rich tasks, applying real-robot reinforcement learning improved the policy success rate from 40%–76% during the supervised post-training stage to 88%–96%. In a separate continuous manipulation experiment featuring agent task orchestration and recovery mechanisms, replacing the supervisedly trained manipulation policy with the reinforcement-learning-optimized GR00T 2 increased the number of fully successful trials—where five chargers were inserted consecutively—from 2 out of 20 to 20 out of 20.

These results clearly define the boundaries of general-purpose robot models: pre-training can provide broad manipulation knowledge, and a few demonstrations can help the model adapt to specific tasks, but precision issues during real-world physical contact still require targeted training and feedback.

Simulation has emerged as another tool to reduce these trial-and-error costs.

NVIDIA experiments demonstrate that supplementing unchanged real-world training data with simulated data boosts the success rate of GR00T 2 Turbo on a set of out-of-distribution real-robot tasks from 6.7% to 63.3%.

Meanwhile, NVIDIA is leveraging tools like SimFoundry to recreate simulation environments for real-world tasks. In a controlled evaluation spanning 25 YAM tasks and 14 GR00T 2 policies, the Pearson correlation coefficient between simulated and real-robot task success rates reached 0.92.

This allows researchers to pre-screen data processing methods and model inference configurations in simulation, determining which approaches warrant real-robot testing.

Consequently, the workflow for robots learning new tasks may gradually shift from collecting massive amounts of task-specific data to a paradigm of general pre-training, minimal adaptation, simulation validation, and real-world interactive refinement.

When robots begin writing and repairing their own skills

Mastering a single action is just one piece of a complete workflow. In real-world workstations, robots must not only know how to grasp, place, and assemble, but also handle various anomalies that arise during execution.

For instance, if a part drops during an assembly task, how should the subsequent process continue? If the grasp location shifts, does the robot need to reposition? If a part was installed backward in the previous step, can the system detect the error and recover without restarting the entire task?

Historically, engineers relied on their own experience and custom scripts to resolve these issues. NVIDIA is now shifting part of this burden to AI agents through its ASPIRE framework.

The ASPIRE agent framework employs a code-as-policy approach, enabling reasoning models to autonomously write, test, and debug robot control programs.

The process involves three stages. First, the system logs visual data, motion trajectories, grasp candidates, and collision feedback from the robot's execution, allowing the agent to pinpoint the cause of failure. Next, the agent either modifies the control program or generates alternative solutions, filtering for viable outcomes through multiple rounds of testing and parallel exploration. Finally, the validated programs and recovery methods are saved to a continuously expanding skill library for use in subsequent tasks.

In the toy assembly system demonstrated by Jim Fan, GR00T 2 handles most of the physically interactive actions. Meanwhile, a higher-level agent monitors task progress and invokes recovery skills learned via ASPIRE whenever errors occur.

Before each assembly trial, researchers scatter the parts back onto the table to randomize the initial layout. The higher-level agent must then organize the task based on the current scene and execute recovery procedures if the robot drops a part or encounters an execution blockage.

NVIDIA reports that across three toy assembly tasks, the introduction of the agent allowed the system to complete 100 trials each for an airplane, a tractor, and a car, achieving 98, 97, and 97 successes, respectively. Without the higher-level agent and relying solely on a post-trained action policy, the success rates for these three tasks ranged from 71% to 84%.

These results demonstrate that the reliability of long-horizon tasks depends not only on the underlying action model but also on the system's ability to monitor task progress, detect failures, and recover execution.

Independent research on ASPIRE further shows that as the skill library grows, the system achieves a 31% zero-shot success rate on a set of unseen long-horizon tasks, compared to just 4% for the baseline method. Furthermore, when skills acquired in simulation are transferred to real robots with different morphologies, the number of tokens the agent consumes to generate and debug functional programs decreases.

This means that knowledge previously scattered across engineers' experience, proprietary scripts, and on-site debugging can now be transformed into accumulative, reusable software assets.

However, the ASPIRE research team acknowledges that achieving fully autonomous continuous learning on physical robots still requires reliable success detection, safe resets, equipment calibration, and safety monitoring. Furthermore, automated debugging consumes significant model inference and simulation resources. Fixing an assembly error in a lab is not the same as ensuring long-term equipment stability on a customer's factory floor.

As foundation models adapt to new tasks from just a few demonstrations and AI agents assist in writing, debugging, and composing manipulation skills, portions of this repetitive development work can be automated.

For companies with long-term deployed fleets, the failure cases, process constraints, and maintenance experience accumulated during real-world operation could become key resources that differentiate them from general foundation model providers. Conversely, for companies lacking the capability to develop their own foundation models, external models and automated programming tools could lower the barrier to entry for complex manipulation scenarios.

This raises a broader industry question left by ASPIRE: as robot manipulation capabilities become increasingly software-defined, who will provide, manage, and continuously improve these skills?

NVIDIA aims to build the foundational platform for robot development

When considered alongside the other technologies announced at the same event, it is clear that NVIDIA's ambitions extend far beyond a single model.

In addition to the GR00T 2 and Turbo model weights scheduled for release in December, NVIDIA has already released or made publicly available several other projects. These include SimFoundry, which converts real-world videos into interactive simulation environments; ASPIRE, for developing and accumulating code-based skills; and Flora, for video annotation, task evaluation, and reward modeling.

NVIDIA has also released the first 100,000 hours of human video from the Human-1M dataset under a CC BY 4.0 license, permitting commercial use provided that attribution and other conditions are met. Meanwhile, the GEAR team has published the mechanical and electrical designs, along with manufacturing documentation, for data collection devices such as UMI and DexUMI, though some supporting resources are still being finalized.

These tools span multiple stages, from data collection, annotation, and model training to simulation validation, skill execution, and error recovery.

A major reason NVIDIA is openly releasing the design blueprints for its data collection equipment is that the robotics industry has yet to establish a development framework that can be widely reused across different hardware platforms.

General foundation models are just one component. Even with access to model weights, developers still need to resolve the discrepancies in training data formats, robot control interfaces, simulation environments, action adaptation, and task evaluation. To truly integrate these capabilities, developers need more than just a brain; they need a comprehensive suite of supporting tools.

Across the broader industry, different approaches to delivering robot intelligence are emerging.

On October 13, Physical Intelligence launched the beta version of the Pi API, allowing developers to upload robot data, fine-tune models on PI-hosted compute resources, and run the adapted policies via a cloud interface. Robot companies can now use and continuously improve cutting-edge robot models without having to independently build a complete foundation model training infrastructure.

Google DeepMind, meanwhile, is offering embodied reasoning, vision-language-action, and edge control capabilities in a layered fashion through its Gemini Robotics 2 series, covering various robot hardware. Figure has opted for a deeper hardware-software integration path. Its Helix 02 deeply integrates its proprietary robot body with vision, tactile sensing, and motion control, enabling perception and whole-body execution to coordinate within a single system.

Robot hardware companies can now assemble intelligent capabilities like building blocks. Technology choices at different stages can now have their investments and returns evaluated independently.

Take the Pi API as an example. Robot companies can bypass the need to build their own large-scale foundation model training infrastructure, but they still must collect task-specific data, perform fine-tuning, and conduct real-world testing. They also need to account for network reliability during cloud inference, operational costs, and the cadence of model updates. When PI announced its early partnership cases, it noted that some robots encountered network connectivity issues during actual deployment. Meanwhile, NVIDIA plans to release the weights for its GR00T 2 model, granting developers greater autonomy in model deployment and operation. However, developers will still need to handle the associated compute, adaptation, and maintenance themselves.

Companies operating large-scale fleets with continuous data feedback and ample R&D budgets have strong incentives to invest in developing their own foundation models over the long term, as enhanced model capabilities can in turn reinforce their hardware and data advantages. Conversely, for resource-constrained companies focused on specific use cases, it is more cost-effective to leverage external foundation models and concentrate their R&D efforts on hardware integration, proprietary skills, and customer delivery.

For NVIDIA, the ultimate winner among foundation models matters less than the fact that training, simulation, and deployment all consume massive compute. Whether developers choose to use GR00T 2 is secondary; what truly matters is whether they use NVIDIA GPUs, Isaac Lab, and Jetson. Opening up GR00T 2 is essentially about paving the road for its own compute ecosystem.

Early adopters are already testing NVIDIA's platform strategy. At CoRL, NVIDIA disclosed that Foxconn is deploying GR00T 2 for factory automation in server manufacturing, Siemens is evaluating it for industrial applications, and Nexuni is testing robotic laundry operations. Meanwhile, MicroAGI is using a version of GR00T 2 post-trained using reinforcement learning to enable its YAM robotic arm to plug and unplug Ethernet cables, leveraging Isaac Lab 3.0 and Newton to build and calibrate the simulation environments required for training.

However, most of these partnerships remain in the testing and evaluation phases. It remains an open question whether robot foundation models will eventually form an independent supply market akin to that of large language models.

Yet, several trends are already emerging from this competition. On one hand, the barrier for robot companies to access general intelligence capabilities is lowering. Models, datasets, and tools that previously had to be built from scratch are increasingly available through external suppliers. On the other hand, the widespread availability of these models and tools has not eliminated the challenges of real-world deployment. Mechanical design, tactile and force control, specific customer environments, process comprehension, exception handling, and long-term operations and maintenance still dictate whether a robot can transition from an experimental demo to an actual production workflow.

As this division of labor deepens, its impact will ripple across the industry value chain. Foundation model companies can commercialize their model capabilities without needing to manufacture their own robots. Development platforms can support multiple model architectures. System integrators have the opportunity to translate project experience into reusable software assets. Meanwhile, robot hardware companies must reassess which capabilities warrant their continued in-house investment.

At the end of his talk, Jim Fan returned to the theme of "winter" from his opening remarks. He argued that Astra has not shattered the dreams of robotics researchers, but rather broken down the walls between them.

He announced the open release of models, data, software, and hardware designs, concluding with a demonstration requiring high-frequency control to ignite a flame. "Summer is coming," he declared.

For the industry, whether summer has truly arrived will only be answered by real-world deployment and commercial results.

If the robot brain gradually evolves into a general-purpose capability that can be acquired, assembled, and replaced layer by layer, the competitive barrier for future robotics companies will shift to whether they can translate these capabilities into low-cost, reusable, and reliably stable real-world productivity.

Whether to develop in-house is no longer a matter of faith; it is a question of ROI.