Yann LeCun Praises INTACT: ZJU’s Sun Junhan and Tsinghua’s Zhao Hao Eliminate the 'Search Tax' from JEPA World Models

The core of a world action model lies in first predicting 'what happens next' and setting a goal for it, then having the robot look at the goal and independently decide each step to execute.

However, current mainstream approaches have an awkward design: during training, the model learns to 'predict what happens next given the current image and action'; but during deployment, the controller samples thousands of candidate action sequences from random noise, repeatedly unfolding, scoring, and filtering them to approximate a solution close to the goal, consuming significant time.

Take LeWM as an example: each decision requires evaluating 9,000 candidate sequences, with a single inference taking approximately 1.48 seconds. The main time consumption comes from searching for action sequences via CEM/MPPI.

On July 28, a paper titled 'Isomorphic Intent-to-Action Learning for Search-Free World Models' was published, with Ph.D. student Sun Junhan from Zhejiang University as the first author, and Professors Zhao Hao from Tsinghua University and Zhang Guofeng from Zhejiang University as corresponding authors, proposing the INTACT framework. INTACT uses a shared operator to simultaneously process 'what just happened' and 'what needs to be achieved next', aligning physical intent and goal intent under the same action semantics.

After this paper's results were posted on the overseas X platform, they received a like from Yann LeCun, Turing Award winner and proposer of the JEPA architecture.

The effect is obvious. The paper states that INTACT, trained for only 1 epoch, outputs action sequences with zero search, achieving a 95.33% macro-average success rate with millisecond-level inference that is 300 times faster. Furthermore, in 300 audits of Cube E5 starting from the same point, INTACT won with a 98.7% success rate against LeWM CEM, which ran 9,000 additional evaluations but achieved only a 67.0% success rate, thereby accelerating action sequence output while improving action success rates.

Taking this opportunity to publish the paper, we interviewed the first author of the paper, Sun Junhan from RoboParty Lab, focusing on several questions of particular interest to everyone, discussing the INTACT architecture centered around JEPA:

• JEPA has been focusing heavily on "prediction." How does it differ from architectures like Dreamer? How does INTACT help it achieve stronger control capabilities? And how exactly is that wall between prediction and action being dismantled?

• The core design in the paper is that "the same operator is called twice." Once for physical intent and once for target deployment, with a distinction between the two. The two calls share the same set of network weights; why is sharing better than separating them?

• The critical hurdle for JEPA to transition from academia to industrial deployment, and why this route is promising?

INTACT performed a complete comparison of zero-search, Guarded A*, and pure CEM search across four standard LeWM simulation tasks, while also validating the generalizability of representations under a shared-encoder multi-task setting. Based on Sun Junhan's interview and technical report, we draw the following brief conclusions regarding the aforementioned questions:

• Dreamer uses an Encoder→Decoder architecture, requiring pixel reconstruction; it has natural collapse prevention due to real-world supervision, but entails large data and computational loads. JEPA employs an Encoder→Encoder structure, predicting in abstract space without needing image reconstruction, making it more efficient yet prone to representational collapse. INTACT feeds two types of conditions—'physical successor' and 'goal intent'—into a single action operator, bridging the semantic gap between 'can predict' and 'can control.'

• The ablation study in the paper splits the two calls into two independent networks, resulting in a direct drop of 4.00 percentage points on PushT. This indicates that sharing is not about saving parameters, but about coupling physical intent and target intent within the same semantic space. Meanwhile, there is no forced point-to-point alignment between the two calls, allowing each to approach the true action in its own way while retaining reasonable deviations from each other. Because there is more than one successful path from the same starting point to the same goal.

• The first hurdle: JEPA only predicts and cannot control, but INTACT has already crossed it. The next two hurdles are: building high-fidelity simulation environments, and continuous optimization of the model itself, including extending from simulation to complex real-world robot tasks. The reason for being optimistic about this approach is that in real-robot deployment, VLA freezes semantic understanding (VLM) while fine-tuning action generation (Actor), which creates a disconnect; in contrast, INTACT freezes the Actor and adjusts the Encoder, enabling "true semantic understanding".

INTACT does not start from scratch outside the JEPA world model. It retains the forward predictor responsible for "given an action, deduce the next state," and adds a shared predictor responsible for "given a motion intent, directly output an action," so that the control interface of the world model does not need to be bound to external search modules. This transforms the JEPA world model from an academic architecture that "can predict but cannot control" into an engineering-deployable alternative route for VLA.

What is the World Model of the JEPA Architecture

In the embodied AI community, there are currently three technical approaches to robot control. Most people are already familiar with the "big data + large model" narrative surrounding VLA/Transformer; Dreamer also has several breakout works; but JEPA is the least discussed approach. Before explaining how INTACT works, it is necessary to introduce these three lines.

VLA models that are almost entirely based on the Transformer architecture follow the "pre-training → instruction fine-tuning → action output" pipeline. This is the most popular approach, including RT-2, the π series, OpenVLA, and GR00T N1. The core logic involves using massive amounts of internet text, images, and videos for pre-training to accumulate "world commonsense," followed by fine-tuning with robot demonstration data to produce actions end-to-end.

The advantage is that it scales with the data volume of LLMs, boasting strong semantic understanding and room for cross-task generalization. However, its shortcomings are also obvious: collecting real-robot trajectories via teleoperation one by one is expensive; inference parameters often reach several billion, making it heavy; crucially, semantic understanding and action execution are decoupled. While a VLM can describe tasks clearly, the Actor may not necessarily translate them into correct joint angles.

Dreamer follows the "encode → latent space rollout → decode back to pixels" approach. Representative models include PlaNet, DreamerV3, and TD-MPC2. The process involves an Encoder compressing observations into a latent representation, performing forward rollouts within that latent space, and finally using a Decoder to reconstruct images or reward signals.

Because the Decoder has real images for supervision, this architecture is naturally non-collapsing. But the Decoder also means that pixel information such as texture, lighting, and background, which have little to do with control, needs to be reconstructed. The computational and data volumes cannot be saved, and search is still indispensable during the inference phase.

The JEPA world model follows the paradigm of "encoding → direct prediction in latent space → no decoding." Representatives include LeCun's V-JEPA and LeWM, as well as INTACT from this paper. After the encoder compresses RGB into a latent representation, it directly predicts the next latent state within the latent space, with no decoder involved and pixel-level data never touched. RGB images contain excessive noise; slight changes in lighting cause all pixel values to shift, yet the underlying physical laws remain unchanged.

In fact, prediction is performed in abstract space, filtering out noise and retaining only the features of "what has changed." The benefit is that there is no need to reconstruct the image, and the number of parameters can be reduced to a very small size. The core risk is representation collapse: if the Encoder maps all inputs to 0, and 0 predicts 0, the loss function used for comparison may appear perfect, but the model is rendered useless. The technical heart of the entire JEPA family lies in preventing this collapse.

LeCun has been pushing JEPA for years, but very few in industry have actually used it. The reason is that while JEPA addresses "prediction," no one has yet solved "control." Industrial players cannot afford to wait for search-based action sequence methods like CEM. INTACT upgrades JEPA from a predictor into a "deployable action interface" for the first time.

Sun Junhan made a direct judgment: "LeCun's World Model, which predicts the next frame using state and action prediction, but this predictor in my eyes is the same as Dreamer's decoder; they are both generators. He hasn't truly leveraged the advantages of the encoder-encoder architecture."

Therefore, INTACT's main contribution is to align 'intent' and 'action' within the latent space by entrusting the prediction of the next frame's latent representation to the Jepa backbone.

The Same Network, Asked Twice — INTACT Turns Intent into Action

The architecture overview of INTACT is shown in the figure below, with its core broken down into three key design steps.

The first design defines two motion intentions for the current state. Physical intention (the difference between the actual next step and the current state, used to answer 'what happened just now'), and deployment intention (the difference between the target and the current state, used to answer 'which direction to go').

The former has real successors as a basis during training, but they are unavailable during deployment because the appearance of the next step is only known after action execution. The latter is directly usable during deployment, but it originates from goals rather than physical transitions. This is the starting point of the entire paper: each condition is useful on its own, but neither is complete.

The second key design involves both intents entering an identical input structure. This structure contains four slots: current state, motion intent, an interaction term between state and intent, and the previous action. This structure passes through the same set of network weights to output an action distribution. During training, the loss for each of the two calls is computed against the real action, but no loss is applied between the two outputs; deliberately, they are not forced to approximate each other.

In other words, given the same starting point and goal, there are countless paths taken historically. Forcing alignment would cause the model to remember only one 'average path,' whereas control requires that 'any path within a reasonable interval will suffice.'

The third design uses asymmetric gradients to ensure 'what should move does move, and what should stay fixed stays fixed.' When the physical successor serves as the anchor, it carries gradients to backpropagate and update the encoder, allowing visual representations to retain 'which changes are recoverable.' When the goal serves as the anchor, its gradients are truncated. The goal remains fixed, compelling actions to chase the goal rather than the goal accommodating the actions.

Sun Junhan provided a vivid analogy in an interview: 'If your senior year goal is Tsinghua or Peking University, you cannot change your goal to another school midway because it’s too hard. It might be easier, but it wouldn’t be the place you originally wanted to go.'

It is worth noting that among the four input slots in the second design, the most critical is the element-wise interaction between state and intent. The phrase 'move forward one meter' means something entirely different when in an open field versus at the base of a wall. Ablation experiments in the paper provide quantitative results: adding only this interaction term yields less than a one percentage point improvement, while fully matching all four slots yields nearly a four percentage point improvement.

This makes the inference loop during deployment simple: encode current and target frames → compute target intent → INTACT predictor outputs an action block → forward predictor unfolds one step → re-plan from the new observation. Compared to LeWM, the entire process evaluates no candidate sequences and invokes no terminal cost functions, significantly reducing inference time.

Searching for more does not necessarily mean better

The previous section discussed calling the same operator twice, where only a target needs to be provided during deployment to directly output actions. This raises the question: how well does this 'intent-to-action' scheme actually perform in practice, and more importantly, is there still a need for search methods such as CEM?

In fact, INTACT does not advocate completely abandoning search. What it changes is the positioning of search within world model control.

The paper systematically audits different search strategies across a complete matrix of 576 planner configurations. The core finding is that using Direct planning as the starting point for search and performing fine-tuning only within a very small neighborhood raises the macro-average success rate from ninety-five point five zero percent to a peak of ninety-six point seven eight percent; once the search range is further expanded, the success rate instead drops to ninety-two point eight six percent.

More notably, if we discard the Direct plan and revert to a full CEM search—generating 300 candidate sequences per decision randomly and iterating optimization for 30 rounds, resulting in 9,000 macro-average success rate evaluations of 93.78%—it still falls short of the 95.33% achieved without any search at all. The paper offers a precise summary on this: "Unconstrained iterative search reproduces the gap between training and deployment."

Moreover, as shown in the figure above, the required 'search configuration' varies across different tasks. PushT is a contact-intensive pushing task. The Direct mode achieved

85.78%, after making small local validations near the Direct plan, it was pulled to 92.33%, with search contributing nearly 7 percentage points of dividends.

Reacher, continuous control, trajectory smoothing. Direct already achieves 97.67%, but slightly expanding the search range drops it to 93.78%; further expansion leaves only 84.78%, which is 13 percentage points lower than zero-search performance. Search is purely counterproductive.

The stark contrast between these two tasks stems from the fact that PushT's Direct occasionally exhibits deviations near contact-switching points, where small-scale local search can correct them; whereas Reacher's Direct is already highly accurate, and a wider search range essentially injects noise next to the correct answer.

Based on this, the paper recommends Configuration Guarded A: using Direct planning as the search center, performing local validation with 128 candidates × 3 iterations within a very small neighborhood, totaling 384 candidates—only 4.27% of the full CEM set. This configuration achieves a macro-average accuracy of 96.86%, surpassing pure CEM by 16 percentage points.

This configuration principle lies in starting from the learned solution, narrowing the search scope, reducing the number of candidates to the hundreds level, and always retaining the original plan as a reference.

As for whether world models will still need search in the future, Sun Junhan’s response at the end of the interview was: 'Models should have the ability to search, but that doesn’t mean they must use it. Searching less or not searching at all is the direction we should pursue.'

An interesting finding appears in the appendix (as shown above). When using only physical successor inversion without target intent, Direct’s success rate on PushT after just one epoch is far lower than that of the complete INTACT target-displacement version. This indicates that when no real successors are available during deployment, the model must learn to generate actions directly from target intent.

Action Signals Can Also Influence Visual Representations

Direct mode has proven that INTACT learns to read actions directly from intent. However, the most enlightening finding in the paper is that even after completely removing the ability to read out actions and reverting to the same CEM blind search, the encoder jointly trained by INTACT still significantly outperforms an encoder trained solely on forward prediction.

As shown in Figure (a), a set of experiments disabling the action reader directly demonstrates this effect. On PushT, researchers removed INTACT's action-reading capability entirely, retaining only the encoder and forward predictor jointly trained by INTACT, thus returning to pure CEM blind search: pure LeWM (without action supervision) 42.22% → + physical inverse supervision 57.67% → + goal-intent supervision 61.44% → + reduced SIGReg 69.44%.

The key point is that the INTACT predictor was not used at all; performance in CEM alone based on the encoder-forward stack yielded a 27 percentage-point increase. This holds true for multi-task shared encoders as well: with INTACT's action reader disabled, CEM achieved 70.08%, compared to LeWM's 66.17%.

Action supervision affects encoders that did not directly participate in action training. Sun Junhan described this mechanism with a metaphor: "The mesh size of the Encoder's sieve adapts as the Predictor updates." The backpropagation of action likelihood gradients effectively tells the encoder: keep visual features useful for control; compress those irrelevant to actions.

Traditionally, the quality of a representation is often judged by how much information it retains. However, the paper finds that, as shown in the figure above, for control tasks, what matters is "what is stored correctly." Retaining too many pixel details unrelated to actions can instead mislead the search. What strongly correlates with success rate is whether the encoder can stably map similar intents to similar actions.

On this basis, multi-task experiments with shared encoders push generalization further. A single ViT-Tiny/14 encoder handles four tasks simultaneously, with each task retaining only a small task-specific head. After five rounds of training, the macro-average direct control success rate reaches 89.39%, outperforming pure forward models using CEM search under identical conditions across every task.

Whereas the complete model outperforms the branch using only target intent by nearly 9 macro-average points, indicating that physical inverse supervision is not an optional add-on.

Certainly, cross-task transfer and leave-one-task-out generalization have not yet been successful; the current universality of representations remains at the level of "one encoder learning four tasks simultaneously," rather than enabling deployment to task B after learning task A. However, it points to a fundamental judgment: the reverse shaping of the encoder by action supervision can be engineered for reuse. The next hurdle is whether the capability can be scaled up.

Beyond the Paper: Industrial Deployment and Roboparty

As is well known, the standard operation for VLA on real robots involves freezing the VLM and fine-tuning the action head based on a few real robot trajectories, yielding decent performance. However, INTACT does the opposite: it fine-tunes the encoder, including the Encoder itself.

The logic behind this is that INTACT's joint training has already refined the mapping from intent to action to be highly reliable. Compared to having an intent that moves closer to the goal, the new problems faced by real robots are often "which intent should the visual scene in a new environment be mapped to?"

When describing this scenario, Sun Junhan said: "Volcano 1 and Volcano 2 look different, but you want the model to understand them both as 'the volcano I am going to.' Changing the Encoder changes this understanding, not how you move." An additional benefit is that inference speed does not slow down due to fine-tuning, remaining at just a few milliseconds.

Meanwhile, the first hurdle for JEPA industrialization has been cleared. Interviews revealed that the team has already run INTACT on real robot training, covering a scope beyond the four simulation benchmarks in the paper. The next focus is on continuous refinement of high-fidelity simulation platforms and the model itself.

Sun Junhan's judgment is: "This architecture like INTACT will be a better future than VLA, as it truly understands semantics. In VLA, semantic understanding and action generation are disconnected—only when one can generate commands themselves and the body can adapt to those commands is it optimal."

Regarding INTACT's position in RoboParty Lab's technical landscape. RoboParty Lab is currently advancing four parallel tracks: Local Motion, VLA, World Model (where INTACT belongs), and Agent. INTACT is positioned as a World Action Model, with the long-term goal of gradually replacing VLAs.

The paper also explicitly states that if two condition families share supervised behavior labels, their shared conditional operator can align their quotient semantics, while 'asymmetric gradient' routing preserves the role of each family. This approach is more worthy of further exploration than debating 'whether world models need search'.

In Conclusion

In the past, world models were primarily used for prediction, while actual action control relied on search.

The former is responsible for demonstrating 'what the world becomes after doing A', which is not problematic. The issue lies with the latter, which is akin to groping in the dark to determine 'which A leads to the target B'.

INTACT attempts to solve this problem by aligning 'what needs to be achieved' and 'what actions should be taken' within the same semantic space inside JEPA.

The forward predictor continues to retain richer world representations, while the INTACT predictor is responsible for quickly translating intent into actions, reducing search from a mandatory interface to an optional verification step.

Therefore, INTACT's underlying philosophy is not merely using JEPA for a simple academic experiment; it aims to build it into the foundational infrastructure for perception and control.