Copper-Policy Replaces Pixel Prediction With Compact Latent Representations for Robot Control

The Copper-Policy model jointly learns compact latent world representations and action policies, bypassing the heavy computational cost of pixel-level video prediction. First author Zexin Feng from The University of Hong Kong explains how this asymmetric architecture achieves robust control without extensive embodied pre-training.

To endow robots with enhanced cognitive and physical interaction capabilities, researchers are exploring world action models (WAMs). By predicting scene evolution through visual dynamics, WAMs bridge the gap between what the world should become and how the robot can achieve this change. Notably, the benefits of joint training persist even when future states are not generated during testing.

Which aspect of WAM dynamics learning actually improves control? As demonstrated in Fast-WAM, video joint training remains beneficial even without generating future states during testing. More recent studies suggest that encoding useful future information into representations for policy learning can also enhance control performance. For instance, some research uses representations from a pre-trained joint embedding predictive architecture (JEPA) as policy inputs or fixed prediction targets, while other work learns predictable latent variables under specific constraints. Consequently, the value of a WAM appears to lie more in the representations shaped during training than in the actual futures it generates. This raises the question: can we predict future states directly within the representation space and co-evolve them with the policy?

This is precisely the core question Copper-Policy sets out to address.

Paper link: https://arxiv.org/abs/2609.32779

Project homepage: https://zexinfeng-cn.github.io/works/copper-policy/

The core concept behind Copper-Policy is to jointly learn the prediction target space through latent world prediction and the action policy, rather than relying on fixed representations from pre-trained encoders or staged learning. To achieve this, the model employs an asymmetric interface for prediction and control: the future expert focuses on learning task-relevant world evolution, while the action expert retains execution-specific details. This allows the prediction target space to concentrate on task-relevant dynamics without needing to preserve all spatial details required for execution. Experiments demonstrate that, even without extensive embodied pre-training, Copper-Policy achieves control performance on par with π0.5 while maintaining high training efficiency. Furthermore, the world representation exhibits remarkable robustness across a series of perturbation tests, with visual features from the video's current frame providing additional complementary information.

To explore the details behind Copper-Policy, 42HOW Robotics spoke with Zexin Feng, the first author of the study.

Zexin Feng, the first author of Copper-Policy, is a Ph.D. student in the Department of Civil Engineering at The University of Hong Kong (HKU), advised by Professor Xintao Yan. He earned his bachelor's degree in computer science from the Southern University of Science and Technology (SUSTech). His research interests span physical AI, autonomous driving, and machine learning. His accolades include the Best Undergraduate Thesis Award (2025) and the China National Scholarship (2024).

How to compress

During the early validation phase, the team experienced internal disagreements over the technical approach. Some members argued that video generation had already proven the value of the pixel-based approach, making it unnecessary to pivot to other methods. However, Feng Zexin noted that for research teams lacking the computational resources of major tech companies, the cost of pixel-level world modeling is prohibitively high; training a full-scale model could require approximately eight H200 GPUs. The team then began to explore whether world modeling could be performed directly in the compact latent space formed by self-supervised learning and jointly trained with the control policy, thereby avoiding the heavy overhead of pixel-level future prediction. Copper-Policy gradually took shape through the exploration of this very question.

The first question the team needed to clarify was: how much information in pixel-level future prediction is actually necessary for robot control? A pixel-level World Action Model (WAM) must generate video frames, requiring it to process details such as lighting, materials, and noise, sometimes even generating irrelevant disturbances in the scene. Yet this information is not necessarily relevant to the task the robot needs to complete. The team reasoned that pixel generation increases training costs and may reduce inference efficiency, but it is not strictly essential for learning a control policy. Consequently, the team shifted the prediction target to compact latent representations, testing whether world dynamics useful for control could be learned without reconstructing the full future visual scene.

Feng Zexin argues that, from the perspective of robot control, the differences between the approaches are also reflected in the space where the training signals reside. A WAM typically adds an action decoding expert on top of a video model. Compared to a Vision-Language-Action (VLA) model, it incorporates additional training signals from next-frame prediction, effectively adding an auxiliary loss from a learning perspective. More importantly, both the input and the prediction target of the WAM revolve around the observation space. In his words, the model is almost constantly "immersed in the observation space," enabling it to learn correlations between observation sequences more directly.

"Take picking up a cup and moving it from left to right as an example. A human does not need to mentally simulate the exact trajectory of every pixel on the cup during the motion, nor how the materials and water droplets in each frame move. Such redundant details are irrelevant; what matters is the information pertinent to the task."

Regarding the division of labor, Feng Zexin proposed the initial ideas and discussed and validated them with several collaborators. The earliest validation used toy tasks in a complex process that involved numerous benchmarks and experiments. As the second author of this paper, Feng Yixu from The University of Sydney handled the real-robot assistance and early validation. Her prior work published at ICML 2026, "See What Matters: Differentiable Grid Sample Pruning for Generalizable Vision-Language-Action Model," also introduced a compact scheme for the visual bypass. Consequently, the technical approach of Copper-Policy continues to inherit this compactness.

"That paper was highly inspiring for us. The visual pathway we added is equally compact, and nearly all information pathways within the model's backbone are abstracted or compressed."

Copper-Policy was also inspired by research from Yann LeCun's team, specifically LeWorldModel and LeJEPA. When modeling a task, LeWorldModel uses only 64 dimensions to represent it quite comprehensively. Calculated at 16 bits per dimension, 64 dimensions amount to only about 1,024 bits—far less than the information content of a single raw image frame. This demonstrates that the representation required to complete a task can be much more compact than the raw image, leaving room for more abstract extraction of task-relevant information. These studies provided the team with another approach worth validating: could world dynamics be learned in a more abstract latent space through single-stage joint training while retaining the information necessary for action control?

One direct benefit of compact representations is the reduction of unnecessary modeling and training overhead. Unlike pixel-level World Action Models (WAMs), Copper-Policy does not need to reconstruct extensive visual details or rely on a full video generation backbone. Within the resulting 2-billion-parameter model, the world modeling expert contains only about 870 million parameters, making it relatively lightweight compared to a WAM. Consequently, the World Representation encoder only needs to handle abstraction and compression. It is more abstract than a video-generation-based WAM, yet incorporates more dynamic information than a Vision-Language-Action (VLA) model, resulting in a more compact overall architecture.

Presenting the World

The Copper-Policy model architecture emphasizes three characteristics: compactness, intent-driven design, and decodable actions. The model first feeds multi-view camera images into a world representation encoder to obtain a compressed latent representation. This encoder consists of two parts: the first half is a frozen 300-million-parameter model primarily responsible for understanding the visual input, while the second half handles compression and encoding. The colored blocks in the architecture diagram represent the latent tokens output by the encoder. Information such as object relationships, contact states, and task progress is organized into 20 to 40 tokens, depending on the task configuration, eliminating the need for pixel-level image reconstruction. The model then predicts representation changes based on task instructions and generates executable actions for the robotic arm.

The future predictor takes the compressed current representation and task intent as inputs to learn how the world representation changes over time. Meanwhile, the action decoder combines this representation, the task intent, the current frame's visual information, and the robot's state to generate an action chunk. Executing multiple action chunks in sequence constitutes a complete operation.

When designing the architecture, the team did not assume any single technical route was inherently superior; instead, they selected methods based on practical problems. Feng Zexin notes that while the training objectives of methods like JEPA, DINO, and BYOL (Bootstrap Your Own Latent) differ, they all provide valuable insights for self-supervised representation learning. Copper-Policy draws on their joint-embedding prediction and target encoder mechanisms.

"Yann LeCun's team aims to have every state precisely represented and distinctly separated, using methods like cross-entropy or some heuristic to search for the path. We do not require the encoder to represent every state completely and in isolation, nor do we need that level of discriminative ability. Therefore, we borrowed JEPA's predictive structure and adapted it to the temporal domain, rather than copying it entirely."

Notably, while the team adopted JEPA's joint-embedding prediction concept by training the representation and policy together, their specific prediction targets differ. Because JEPA training requires masking in the temporal dimension to predict the future and map the present and future into the same space, "we essentially mask out the entire future and use the present to predict it, forming a joint-embedding prediction." This joint training allows the model to de-emphasize task-irrelevant details while retaining the information needed for action decoding, ultimately forming a control-oriented compact representation during the learning process.

The future expert primarily learns future state evolution from the compact world representation, while the action expert additionally reads the current frame's visual details and the robot's state, without directly reading the generated future tokens. Compared to methods that treat world information and actions equally, this asymmetric design emphasizes distinct specialized roles.

During training, ground-truth future frames are not used directly as pixel-level prediction targets. To mitigate the risk of representation collapse, Copper-Policy employs a target encoder updated slowly via an exponential moving average (EMA) to compress future frames into target representations. The current representation then generates predictions to align with these targets. If the target and prediction branches were updated synchronously and rapidly, the model could take shortcuts that bypass learning useful information, rendering the representations meaningless.

For its training methodology, Copper-Policy adopts conditional flow matching to supervise future representation prediction and action generation separately. The future expert learns to predict future representations within a compact space, while the action expert generates actions. Through joint optimization, both components collaboratively shape the world representations used for control. During inference, the model does not need to explicitly generate future representations; instead, the action expert directly decodes the actions.

Because the future expert DiT predicts the temporal evolution of representations and the action expert DiT generates actions, the two exchange information via joint attention. During inference, the action expert does not directly rely on the generated future representations; instead, future prediction aids control primarily through the representations shaped during training. During the training phase, both losses share a single backpropagation pass. The action loss also backpropagates to influence the world representation encoder, encouraging it to retain information necessary for operations such as grasping, aligning, and placing.

Consequently, in a representation similarity test spanning a 16-frame interval, Copper-Policy's similarity decreases more slowly over time, demonstrating superior temporal stability.

Another critical question is whether the model can distinguish between task-related changes and environmental disturbances. The Copper-Policy team argues that the key lies not in eliminating disturbances, but in isolating them.

The team calculated the principal directions of task changes and environmental disturbances within the representation space and compared their degree of overlap. The results show that with joint embedding prediction, the overlap is very low at a rank of 8, whereas without this objective, the overlap exceeds 50%. This indicates that disturbances severely corrupt the semantics of the entire scene. Rather than 'erasing' disturbances, the Copper-Policy model allocates them to different axes within the latent space. As the task progresses, the scene evolves accordingly. The abstract representations derived from token-level joint embedding prediction show significant improvements in mitigating the effects of noise, lighting variations, initial state differences, and camera jitter. In other words, this approach enhances Copper-Policy's resistance to interference, making it more robust.

How does it perform in practice?

Simulation tests show that on the standard LIBERO and LIBERO-Plus benchmarks, Copper-Policy achieves significantly better overall performance. Despite lacking embodied pre-training, it outperforms π0.5 on object and spatial tasks, and remains competitive with embodied pre-trained models like VLA-JEPA and X-VLA on other tasks. In the camera and noise perturbation tests within LIBERO-Plus, Copper-Policy demonstrates strong robustness. However, language instruction following remains a relatively weak area.

In the clean scenes of RoboTwin, Copper-Policy achieved 70.84%, closely trailing π0.5 at 70.70%. Across combined clean and randomized scenarios, it averaged 41.91%, ranking first among methods without embodied pre-training and outperforming EventVLA at 40.65% and Fast-WAM at 39.85%. However, under strong randomization conditions—where layout, distractors, background, and lighting change simultaneously—Copper-Policy's performance still drops significantly.

Across three real-robot tasks, Copper-Policy achieved an average success rate of 96.3%, generally outperforming π0.5 and Fast-WAM. Breaking down the tasks reveals that Copper-Policy holds a significant advantage in placing bananas and arranging bowls, while slightly trailing Fast-WAM in folding towels. These results indicate that the method excels in certain real-robot operations but still has room for improvement in long-horizon manipulation of deformable objects.

Feng Zexin believes that for researchers in VLA and WAM, getting things "right" is more important than making them "perfect." A solid codebase and well-designed experimental setups can significantly impact final performance metrics.

"So when simulation tests yield low scores, it doesn't always mean the model design is flawed. Ablation studies actually reveal the issues better—you can observe whether your module truly brings improvements, or conduct further probing analyses. As for the final score, it isn't highly instructive for the research itself. Often, what people are tuning is actually unrelated to the method's core design."

Training efficiency is another key advantage of Copper-Policy. The team used eight consumer-grade RTX 5090 GPUs to train a model with approximately 2B trainable parameters in about 9.67 hours. Compared to approaches relying on high-end compute like A100s, this cost structure is much more favorable for reproduction and iteration by small and medium-sized research teams. "In the team's latest progress, with further infrastructure-related optimizations, training speed has improved by an additional 30%," Feng Zexin revealed.

However, the current version of Copper-Policy still has some limitations in instruction following. Existing training data mostly uses episode-level task descriptions, resulting in limited fine-grained language supervision. Additionally, differences in prompts and embedding settings across various evaluation benchmarks pose a challenge for language instruction following in Copper-Policy. This is also a recent focus within the community.

"Language generalization cannot be achieved by just wishing for it. We cannot expect a VLA to naturally follow instructions well. Existing post-training datasets mostly treat language as a condition: you give it a task description, and it outputs a sequence of actions or performs a specific task. Ultimately, it just converges into a task ID, learning to distinguish between tasks without fine-grained differentiation. This problem cannot be solved simply by using a stronger foundation model."

On the other hand, the model's scores remain relatively low on generalization metrics such as background layout and language. The team attributes this directly to the lack of embodied pre-training.

Next, the team plans to focus on improving data utilization and instruction-following capabilities. On one hand, they aim to use more data to verify the stability of their representations, paying particular attention to whether embodied pre-training can further enhance performance. On the other hand, regarding instruction following, the team hopes to make further progress to address the pain points of control models such as VLA and WAM within the current research community.

Making technology accessible, just like copper

Discussing recent developments in embodied AI, the team believes that while some agentic policies can already decompose tasks effectively, reliable execution models are still needed. Combining such policies with control models like VLA or Copper-Policy could be a valuable direction. Another hypothesis is that if compact representations can fully retain task-relevant information, models based on imitation learning might develop stronger abstraction and action capabilities, eventually achieving effects similar to GPT-3. Whichever path prevails, Feng Zexin remains open to technological advancements in the field.

"Our current view is that execution is everything. In an era of such rapid technological iteration, there is no need to rigidly adhere to a single approach."

The team notes that training efficiency is one of the key advantages making Copper-Policy worth further development. They aim to maintain the model's generality while further reducing its reliance on external, complex modules. Under this vision, future iterations of Copper-Policy could more closely resemble a native in-context "imitation" model, rather than relying on additional agents, memory modules, or other complex components.

"We have always advocated for the imitation learning approach: taking a sequence of expert trajectories and using them to generate gradients to optimize the model. But it is also possible for a model to develop certain capabilities—similar to in-context learning (ICL)—without relying on gradient-based imitation. Ultimately, this capability paradigm might not require post-training fine-tuning to perform tasks."

The name Copper-Policy also reflects the team's aspiration to democratize this technology. They hope the model can be reproduced at a lower cost or used as a foundation by other researchers. The longer-term goal is to gradually translate embodied AI research into everyday products and services, bringing them into every home.

"The reason we named this research Copper is that copper was one of the earliest metals used by humans to make tools."