Are Robots Abandoning Language? The Real Answer from GTC 2026

At this year's GTC 2026, Jensen Huang brought the classic Disney IP character Olaf to interact with the audience live.

Behind Olaf lies NVIDIA's comprehensive technological update for robotics, including the new Isaac Platform, multimodal foundation models, and a reinforced training framework that deepens the coupling between simulation and reality.

These systems build upon NVIDIA's accumulated expertise in robotics over recent years. However, across multiple demonstrations, a more specific engineering shift is evident: robotic systems are reducing their reliance on "explicit intermediate layers," allowing perception to directly influence action generation and thereby improving response speed.

In traditional VLA (Vision-Language-Action) models, robots typically receive perceptual information, use explicit language to understand tasks, and then decompose them into action sequences, with language serving as an intermediate layer.

Now, emerging approaches are targeting these "explicit representations" like language, feeding visual inputs, environmental states, and task conditions directly into policy models. The model reasons within a latent space to output continuous control signals.

Similar changes are also occurring in world models. In the recently published paper "Fast-WAM: Do World Action Models Need Test-time Future Imagination?" by Tsinghua University's Institute for Interdisciplinary Information Sciences and Galaxea AI, researchers addressed another type of intermediate structure—"future imagination." They raised the same question: whether these intermediate steps are truly necessary during inference.

Whether it is the VLA or world model approach, traditional "explicit intermediate layers" often suffer from excessive latency, which hinders the practical deployment of robots in the real world.

Thus, both point to the same underlying shift: robotic systems are moving from explicit reasoning structures that unfold in layers toward direct decision-making within a unified representation space.

Explicit intermediate layers have become a "burden" for robots

In the past one to two years, driven by the success of large language models, traditional VLA models inspired by their architecture typically adopt a hierarchical structure, roughly involving: language processing, semantic understanding, task decomposition, planning, and control execution.

This structure appears to have clear modularity and strong interpretability, but in practical deployment, some issues are gradually emerging.

• Inconsistent representation formats: Language models process discrete symbols, planning modules operate on structured tasks, while control systems rely on continuous time signals. Conversions between layers inevitably lead to error accumulation.
• Mismatched time scales: High-level decisions are typically low-frequency, whereas control execution requires high-frequency closed-loop responses. This hierarchical structure introduces latency and inconsistency.

Similar issues also exist in WAM, which typically assist decision-making by generating future video or state sequences, essentially operating on the principle of 'imagine first, then act.'

However, this process of "imagining the future" is also an intermediate layer that explicitly unfolds the decision-making process into "prediction, evaluation, and selection."

Therefore, whether it is language or future imagination, these intermediate structures share a common characteristic: they make the system easier to understand but increase computational and interface costs.

Robots Need Faster Response Times

A series of updates showcased by NVIDIA during this GTC also reveal an evolution from traditional pipelines to more compact structures, progressing from states to neural policies, and finally to continuous robotic actions.

The key change in VLA lies in the shifting role of language; it is gradually exiting the real-time control loop, serving primarily as supervision signals or high-level constraints during training rather than inputs for every action decision.

Meanwhile, policy models are beginning to output control values directly, such as joint angular velocities or end-effector poses, instead of intermediate semantic commands. The model must now complete within its internal architecture the decision-making processes previously handled by multiple modules.

This "convergence" is also evident in the WAM approach, with work on Fast-WAM demonstrating similar advancements.

Traditional WAMs typically consist of two key components:

• Training phase: learning dynamic representations from videos or states
• Inference phase: generating future trajectories (imagination), then selecting actions based on them

Fast-WAM decouples and validates these two parts, proposing a more simplified structure:

• Retains video modeling in the training phase
• Completely removes the future generation process during inference
• Outputs actions directly from current observations

The key experimental findings are also quite clear:

• Removing test-time imagination has minimal impact on performance
• Removing video-based training leads to significant performance degradation

This indicates that the step originally considered core—'imagining the future'—is not necessary during the inference phase for certain tasks. However, it is important to note that Fast-WAM does not remove the world model itself; rather, it removes the explicit future generation process during inference.

In terms of efficiency, this adjustment has also brought significant benefits. Fast-WAM reduced inference latency to approximately 190 milliseconds across multiple tasks, achieving a roughly fourfold speed improvement compared to approaches that require future imagination.

The Intermediate Layer is Being Internalized

Viewing the systemic changes at GTC alongside Fast-WAM reveals a more unified trend: robotic systems are reducing explicit intermediate layers by "internalizing" them into implicit representations within the model.

Language is one such intermediate layer, transforming complex decisions into interpretable semantic steps. Future imagination is another, unfolding the decision-making process into predictable trajectories.

The current shift is characterized by:

• No longer explicitly generating language instructions
• No longer explicitly predicting future trajectories
• Instead, directly completing decisions within a unified representation space

In other words, models no longer "first perform human-interpretable reasoning" and then execute actions. This logic aligns more closely with the natural form of control systems: input state, output action, without explicitly unfolding the intermediate process.

Why Now?

This trend of "reducing intermediate layers" is not the result of a single model design; the key lies in multiple foundational conditions maturing gradually over the past year.

Among these, the most significant change is in simulation and data generation capabilities. Based on NVIDIA Isaac Sim's large-scale simulation, robots can generate stable, controllable interaction data with feedback signals, including success and failure labels, in virtual environments.

This allows policy models to no longer rely entirely on expensive real-world data, making direct training from state to action possible.

This point is also reflected in the setup of Fast-WAM. The paper explicitly relies on video-level behavioral data for training and points out that performance mainly comes from dynamic representations learned during the training phase, rather than additional generative processes during inference.

That is to say, in many cases, when the data itself already contains sufficient temporal information, there is no longer a need to explicitly "generate the future again" during the inference stage.

Furthermore, changes in computing power and model architecture have created a more opportune moment for "reducing intermediate layers."

End-to-end strategies or implicit decision models actually require processing perception and control simultaneously within a unified representation space, which places higher demands on computational resources and model stability. The performance improvements in GPUs and edge computing platforms driven by the rapid development of AI over the past year or two have made it possible to run such models in practical systems.

The introduction of structures such as temporal Transformers and diffusion policies into control problems allows models to learn stable action distributions without explicitly unfolding intermediate steps. This is why Fast-WAM can maintain its performance even after removing "future imagination" during the inference phase; essentially, the model has already completed modeling of dynamic processes during training.

Another key point is the change in data closed-loop mechanisms. Compared to systems that rely on manual language annotation, policy models rely more on interaction data itself, including trajectories, reward signals, and environmental feedback. This enables the system to continuously optimize through a cycle of "execution, feedback, retraining," rather than relying on humans to provide structured intermediate representations.

Under this data mechanism, the role of "intermediate layers" is further diminished. Models no longer need to organize the decision-making process through language or explicit prediction, but instead directly learn mapping relationships from experience.

These conditions combined make a previously unrealistic choice feasible: compressing intermediate steps that were originally scattered across multiple modules into the model's internal representation, eliminating the need to unfold them layer by layer.

From Understanding-Centric to Control-Centric

The technical approach of weakening the "intermediate layer" makes robot task execution more engineering-oriented. However, the cost brought by this structural convergence is equally obvious:

• Decreased interpretability: In hierarchical systems, it is possible to pinpoint which stage a problem occurs in, whereas in end-to-end or implicit models, errors are often difficult to decompose.
• Changes in debugging methods: Problems no longer concentrate on module interfaces but shift more toward data and training processes, such as reward function design, differences between simulation and reality, and coverage of data distribution. These issues typically lack direct diagnostic tools.

Furthermore, explicit intermediate layers somewhat facilitate cross-task transfer, while implicit representations rely more heavily on the training distribution, leaving generalization capabilities uncertain.

Therefore, these methods currently perform better mainly in scenarios with stable structures, such as warehouse and manufacturing environments.

However, viewed from a system design perspective, this series of changes can be understood as a more fundamental shift: robots are moving from being "understanding-centric" to "control-centric".

Language still exists but plays more of a role in training and interaction; future imagination remains valuable but is no longer a necessary step during the reasoning phase.

At the execution layer, the system relies more heavily on direct mapping between continuous perception and action, along with feedback-based optimization processes—an engineering trade-off that aligns more closely with the fundamental nature of control problems.

The intermediate layer has not completely disappeared; rather, it has been 'compressed' into the model itself from the system architecture. While no longer present in explicit form, it continues to influence behavior generation in alternative ways.

Thus, in robotics—whether using VLA (Vision-Language-Action) models or world models—the field is following a path similar to autonomous driving: moving from rule-based symbols and perception-planning pipelines toward more direct perception-to-action mappings. The 'explicit intermediate layer' is becoming increasingly less significant.