AgiBot's Rollback Method Uses World Model 'Rewind' to Surpass Human Averages in 8 Hours of Real-Robot Training

AgiBot, Beihang University, and the National University of Singapore introduced Rollback, a policy improvement method that backtracks within a world model to explore alternative actions. Trained on a single real robot for eight hours, the approach outperformed human baselines in ball tossing and tabletop curling by eliminating costly physical scene resets.

If a robot makes a wrong decision, can it roll back to a specific node, change its action, and try again?

This is highly difficult in the physical world; once a tower of blocks collapses, it does not rebuild itself. After every trial-and-error attempt, a human must manually reset the scene and return all props to their initial states before the next trial can begin. Vast amounts of time and money are consumed by scene resets, which is the core bottleneck making real-robot reinforcement learning expensive and iteration slow. Consequently, researchers have begun shifting training into world models, allowing robots to practice repeatedly in virtual simulations.

On October 26, AgiBot, in collaboration with Beihang University and the National University of Singapore, published the paper "Rollback: Experience Backtracking in World Models for Policy Improvement." The paper lists 16 authors, with Jiayi Luo, Yang Zhang, and Liliang Chen as co-first authors. The corresponding authors are Jianxin Li from Beihang University and Guanghui Ren, Executive Director of the Embodied AI Research Center at AgiBot.

The team introduced Rollback, a world model-based method for robot policy improvement. It adds a "rewind" capability to the world model: during a single interaction episode, the system can backtrack to a critical node, swap the action, and re-simulate the trajectory. By comparing the outcomes of different branches, it identifies the key steps that determine success or failure.

Core results from the study show that across three real-robot tasks—block pulling, ball throwing into a bucket, and desktop curling—Rollback achieved success rates of 69.33%, 40.00%, and 64.44%, respectively. These figures outperform the strongest baseline by 10.67, 11.11, and 11.85 percentage points, raising the average success rate across the three tasks from 46.72% to 57.93%.

More notably, this policy starts from teleoperated demonstration data that falls far short of human performance. Yet, after just eight hours of training on a single real robot, it surpasses the average success rate of human players in both the ball-throwing and curling tasks. It starts below human performance, but ends up outperforming humans in two tasks.

Simply put, this work teaches the robot how to "review its moves" within a world model. Building on real interaction trajectories, it backtracks to critical decision points, swaps the actions, and re-simulates the sequence. By comparing multiple branching outcomes derived from the exact same starting point, it directly evaluates the quality of each action without requiring the additional training of a separate value model.

Fresh data continuously collected by the real robot is fed back to constantly calibrate the world model, improving its accuracy in states frequently visited by the policy. The entire training pipeline is built on an asynchronous cloud-edge-device collaborative system, where the search, training, and real-robot execution modules run in parallel without waiting on one another.

Real-robot trial and error is too costly, so we train in imagination

The most direct way to improve a robot through trial and error is real-robot reinforcement learning (RL): the robot repeatedly attempts a task, receiving rewards for success and adjustments for failure. However, in the physical world, every trial incurs significant costs. These pain points essentially stem from the high cost of physical interaction and the heavy reliance on human labor for environment resets.

Consequently, the industry has begun shifting training into world models. A world model can be understood as a video simulator capable of "filling in the blanks": given the current frame and the robot's intended action, it predicts what will happen in the subsequent frames. The policy can then perform trial and error within this simulation, eliminating the need for real-robot interaction and manual environment resets.

This approach has advanced rapidly over the past two years. AgiBot previously introduced GE-Sim 2.0, a closed-loop video world simulator for robot manipulation. According to the original paper, it was retrained on thousands of hours of real-robot data. With just 2 billion parameters, it topped the public WorldArena leaderboard, and a single NVIDIA H100 GPU required only 2.3 seconds to generate 25 frames of inference.

However, the Rollback team argues that existing methods use world models too superficially. Most approaches still follow the standard real-robot training routine: starting from scratch every time, rolling forward to the end, and discarding the intermediate steps. This wastes a unique capability of world models that does not exist in the physical world: the ability to "rewind and redo." In physical reality, if you are halfway through pulling a block, you cannot rewind to the state before the action occurred. A world model, however, can retain the environment state at any point along a trajectory, instantly restore the scene, swap the action, and re-simulate.

To fully leverage the value of this rewind mechanism, three core challenges must be overcome.

First, not every step is worth redoing. Within a single trajectory, usually only a few decisions truly determine success or failure; altering the other steps yields similar outcomes. Blindly branching at every step would exhaust computational resources.

Second, the supervision signal is too sparse. An entire trajectory ultimately yields only a single binary outcome of "success" or "failure," making it impossible to identify which intermediate actions contributed to the result and which caused errors. This is the classic credit assignment problem in reinforcement learning. A common solution is to train an additional value model to estimate the quality of each step, but this introduces extra modeling assumptions and estimation bias.

Third, world models can be inaccurate. As the policy improves, it explores states rarely covered in the world model's training data. Prediction errors may accumulate, and the policy might even exploit these inaccuracies, preventing the skills learned in imagination from transferring to the real robot.

How does the world model rewind?

To address these three challenges, Rollback introduces three corresponding modules: empirical rewind tree search, tree-structured value estimation, and the co-evolution of the world model and policy.

Step 1: Construct a rewind tree from real trial-and-error trajectories. The system builds on a real interaction or demonstration trajectory, saving observation data at the boundary of each action chunk—a continuous sequence of actions output by the policy in a single step—to form a baseline reference path. Each node corresponds to 50 control steps at a frequency of 30 Hz. This approach uses GE-Sim 2.0 as the world model, which synchronously predicts the subsequent frames from the head and dual-wrist cameras, along with the robot body state, specifically the positions of seven joints and one gripper per arm. The team explains the necessity of predicting the body state: relying solely on generated images forces the policy to infer arm posture indirectly. For tasks requiring high precision in contact and release timing, such as extracting blocks or throwing balls, this level of information granularity is insufficient. The final outcome of the original real trajectory is propagated backward along the reference path, becoming the first evaluated branch on the tree.

Step 2: Use Monte Carlo Tree Search to select appropriate branching points. Drawing on the approach of AlphaGo, Rollback employs Monte Carlo Tree Search (MCTS) to allocate limited inference compute, referred to in the paper as the "imagination budget." Each node records its visit count and cumulative success count. Branches with higher success rates receive more exploration resources, while an exploration bonus ensures that less-visited branches still have room for trial and error. Because robot actions are continuous, high-dimensional vectors, it is impossible to enumerate all possible actions as in Go. Therefore, Rollback relies on the current policy to generate candidate actions by changing the random noise seed for each sample, an approach similar to Sampled MuZero. It then applies Progressive Widening to limit the maximum number of new actions added to a single node. Each new action branch is simulated continuously until the task succeeds or a preset depth limit is reached, at which point a task-fine-tuned reward model evaluates the outcome.

A further key design here is Disagreement-Driven Refinement. Effective training signals are produced only when branches derived from the same parent node yield different outcomes; observing result differences from just a few simulations is likely just noise from random chance. When the system detects that sibling branches under the same node fall into different value intervals, it adds the node to a priority queue and allocates additional simulations to under-visited branches until all branches meet the minimum visit requirement. Compute is thus concentrated on comparisons that generate supervision signals, which also reduces label noise.

Step 3: Directly evaluate individual action chunks from the rewind tree. Each action chunk generates a subtree, and the success ratio across all leaf nodes in that subtree defines the empirical value of the action. Rollback restricts comparisons strictly to sibling actions originating from the same parent node. It divides the value of a single action by the highest value within the same group and then categorizes it into K tiers, with the experiments using five tiers. The advantage of this mechanism is that it generates comparable supervision signals for both simple and highly difficult scenarios, as the baseline for comparison is always the performance of different actions from the exact same initial state. The paper notes that this intra-group normalization approach is conceptually similar to the group relative advantage logic used in Group Relative Policy Optimization (GRPO) for large model training. The distinction is that the groups here consist of candidate actions sampled from the same rewind context. If all sibling actions in a group fall into the same tier, it provides no discriminative information, and the sample is discarded.

Compared to training a value model, this approach offers a straightforward advantage. The reward model only needs to determine whether a complete simulation ultimately succeeds or fails, which is significantly easier to train than requiring a value model to predict the final success rate from an intermediate state.

Step 4: Update the policy using gear labels. The policy backbone is Physical Intelligence's π0.5. The training approach builds on the advantages of the team's RECAP scheme, specifically the Advantage-Conditioned Policy: it uses the gear corresponding to an action as a prompt condition and employs Flow Matching as the training objective, guiding the policy to learn the feature distributions of actions across different gears. The training dataset integrates samples generated from tree search, offline demonstration data, and real-robot trajectories. In this dataset, successful real-robot trajectories are labeled with the highest gear, while failed ones receive the lowest. During deployment and tree search, the policy is always conditioned on the highest gear, paired with Classifier-Free Guidance to reinforce the preference for optimal choices. Meanwhile, the candidate actions used for search are sampled from this same highest-gear policy. Once the policy parameters are updated, the candidate actions generated by subsequently created search trees are iteratively synchronized.

Step 5: Co-evolve the world model and the policy. This is a key point emphasized in the paper, which the authors term "Test-Time Physics Grounding." Here, "test-time" refers to the phase after the world model's pre-training, when it is used as a training environment. The initial world model is fine-tuned on expert data, having seen the contacts and object motions made by teleoperators, but it has rarely seen the consequences of a policy that "fails in a new way or succeeds via a new route." The paper gives an example: a block should fall over, but the world model keeps it standing, and the search might mistakenly learn this error as a success. Rollback's countermeasure is to continuously add newly collected successful and failed real-robot trajectories to the replay buffer. It then performs 400 steps of supervised fine-tuning on the world model using LoRA, followed by 100 steps of Distribution Matching Distillation (DMD) to restore fast sampling with few steps. Each real transition carries its own supervision objective, requiring no additional annotation.

What makes this capability special is that the dynamics in a physical engine are hand-coded, and system identification can only tune parameters; a learned simulator, however, can directly correct its own physical laws during use.

Search and training operate independently

Search within the video world model is highly compute-intensive, while real-robot interaction is slow and can only be parallelized by adding more robots. If the system alternated sequentially between data collection, search, and training, each resource would sit idle for long periods. Therefore, Rollback splits these tasks into concurrent processes that communicate only through data and versioned model snapshots.

The entire system is divided into a three-tier architecture:

The cloud handles all GPU-intensive tasks, including search, data storage, and training. The edge station sits right next to the robot and handles policy inference, keeping the cloud out of the robot's control loop.

The robot body itself is strictly responsible for perception and execution. Each search GPU simultaneously hosts a policy, a world model instance, and a context cache. An entire search tree is processed from start to finish on the same GPU; intermediate rollout states do not traverse the network, and only the supervised samples extracted after the search concludes are uploaded. The paper also notes that because search nodes do not share state, the theoretical throughput scales with the number of GPUs.

Another key detail is version pinning during the search task. At the start of each search tree, the system locks in the current latest versions of both the policy and the world model, maintaining them until the tree search is fully complete. This ensures that all branches within a single tree are proposed by the same policy and rolled out by the same world model, making the outcomes of sibling branches directly comparable while keeping the cached context valid throughout. In the experiments, the cloud cluster comprised 48 NVIDIA H200 GPUs: 40 dedicated to world model search and 8 to policy training, serving a single edge site and one physical robot.

Three real-robot tasks

The experiments were conducted using AgiBot's Genie-02 dual-arm robot, with each of the three test tasks focusing on different capabilities.

• Block Extraction mimics the game of Jenga, requiring the robot to push out a specific block in a single motion without toppling the tower, which tests fine manipulation in high-contact scenarios.
• Ball Tossing requires picking up a ball and tossing it into a floor bucket without it bouncing out, testing control over the release posture and timing.
• Tabletop Curling is set up as a human-versus-robot match where the robot plays second. Both sides slide two stones each, and later stones can strike earlier ones. After all four stones come to rest, the side with the stone closest to the center wins, combining strategic judgment with motion precision.

The policy input consists of three RGB camera feeds along with the robot's proprioceptive state, controlling 16 action dimensions at 30 Hz. Each task was evaluated from 6 initial positions (9 for Tabletop Curling), with 15 trials per position (25 for Block Extraction).

The study compares against three baselines, all initialized from the same π0.5 model: SFT, which relies solely on expert demonstrations for supervised fine-tuning; RECAP, which estimates advantages using a learned value model on real-robot data; and RISE, which the authors reproduced on the same GE-Sim 2.0. RISE is an 'imagined reinforcement learning' framework released earlier this year. According to its paper, it combines a controllable dynamics model with a progress value model to generate rollouts, estimate advantages, and update the policy within the imagined space. Both RECAP and RISE required approximately 8 hours of training, one physical robot, and a comparable number of GPUs.

Rollback outperformed the baselines across all three tasks.

• Block Extraction: 69.33%, 10.67 percentage points higher than RISE's 58.67%.
• Ball Tossing: 40.00%, 11.11 percentage points higher than 28.89%.
• Tabletop Curling: 64.44%, 11.85 percentage points higher than 52.59%.

The average across the three tasks reached 57.93%, outperforming SFT, RECAP, and RISE by 17.63, 13.71, and 11.21 percentage points, respectively. For the ball-throwing task alone, the improvement over RISE reached 38.5%.

A progressive comparison further highlights the incremental value of the approach. Moving from SFT to RECAP improved the average success rate by 3.92 percentage points; advancing from RECAP to RISE yielded an additional 2.49 percentage points. The 11.21 percentage point gain achieved by Rollback over RISE exceeds the sum of the previous two improvements. Because RISE was reproduced using the exact same world model, this set of differences eliminates variables introduced by the underlying simulation engine's performance, more accurately representing the performance boost brought by the "rewind and backtrack" mechanism itself.

Why is the improvement so significant?

First, the training process is more stable. The paper tracked the changes in real-robot success rates over the 8-hour training period. Starting from the RECAP policy, Rollback's ball-throwing success rate steadily climbed from 25.56% to 40.00%, a relative improvement of 56.5%; RISE, on the other hand, initially dropped to 24.44% at the 2-hour mark, fluctuated thereafter, and ultimately settled at 28.89%. In the curling task, Rollback improved from 51.11% to 64.44%, while RISE remained largely near its initial level, finishing at 52.59%, just 1.48 percentage points above its starting point. The gap between the two in curling widened at every evaluation checkpoint, and in ball-throwing, it continued to widen after the 4-hour mark.

Second, both components are indispensable. The authors conducted ablation studies on the block-pulling task. When the world model was frozen and prevented from updating alongside the policy, the success rate dropped to 64.67%, the number of consecutively pulled blocks decreased from 2.60 to 1.90, and the proportion of trials touching the target block fell from 89.33% to 78.67%. Replacing the tree-structured value with "directly using the reward from a single rollout as the label" caused the most significant drop, with the success rate falling to 59.33% and the target-touching ratio dropping to 77.33%. The paper explains that under direct reward supervision, each label depends on only a single random rollout rather than the average of the entire subtree; when the world model is frozen, prediction errors continuously accumulate over the long sequence of contact decisions required for block pulling. A similar phenomenon was observed in the ball-throwing task, where freezing the world model limited the success rate to just 26.67%.

Third, the search indeed identifies the decisive move. The paper visualized a search tree from the curling task. Instead of starting from scratch every time, Rollback rewinds to two decision points in the middle of a real episode to branch out again. At step 200, which corresponds to pushing the second stone, the real-world record shows that the push delivered the red stone into the house in about 10 steps; the imagined branch starting from the exact same moment pushed more lightly, causing the stone to slide for about 60 steps before stopping outside the house. Branching also occurs within the imagination: starting from the same imagined state at step 50, the actions of the two branches for pushing the first stone were nearly identical, with the difference lying in the release speed of the second stone—the faster one entered the house, while the slower one stopped short. Placing two trajectories that differ by only a single action side by side makes it immediately clear which move determines the outcome.

8 hours of training to catch up with and surpass the human average

Rollback's performance was benchmarked against human performance, which was divided into two groups: one where humans completed the tasks directly with their bare hands, and another where humans teleoperated the robot. Both groups adopted exactly identical initial conditions and success criteria, and the final results represent the average of three subjects.

In the ball-throwing task, the average success rate for bare-handed human execution was 31.48%, compared to an average of 16.53% for the teleoperated robot. Rollback reached 33.33% by the sixth hour of training, surpassing the bare-handed human average for the first time, and ultimately reached 40.00% after eight hours of training.

In the curling task, the bare-handed human average was 58.39% and the teleoperation average was 38.94%; Rollback overtook the human baseline at the fourth hour, ultimately reaching 64.44%. At the end of training, Rollback exceeded the bare-handed human averages by 8.52 and 6.05 percentage points across the two tasks, respectively, and outperformed the teleoperation baselines by 23.47 and 25.50 percentage points.

There is a noteworthy nuance in these results. The RECAP-initialized policy was already stronger than teleoperation (25.56% vs. 16.53% in ball-throwing, and 51.11% vs. 38.94% in curling), but it only crossed the human performance threshold after Rollback training. The paper emphasizes that the policy never observed how human players performed the tasks; the portion of its performance that exceeds human capability stems from comparing its own different actions within its imagined trajectories.

Of course, the claim of 'surpassing humans' must be understood in context. The human performance here represents the average of three participants, with fixed tasks and initial positions, which is entirely different from the capabilities of professional players or humans operating in open-ended scenarios.

The team also broke down the failure modes in the ball-throwing task: throwing too short, throwing too far, bouncing off the bucket rim, and releasing too late, causing the ball to drop next to the robot. The first three account for the majority of failures, each stemming from minor errors in release speed and direction. All four failure types occur within a very brief time window around the release; a single action chunk essentially determines the outcome of the entire round. This also explains why the ball-throwing task is particularly well-suited for learning by 'rewinding to the critical step and trying a different approach.'

Can virtual rollouts be trusted?

Whether the capabilities learned in a world model can be transferred to the real robot depends critically on the world model's accuracy. The team conducted two sets of experiments to evaluate this.

The first set of experiments focused on the block-removal task. Researchers selected eight real failure trajectories and fed the recorded action sequences directly into the co-evolved world model. Each trajectory was rolled out five times, yielding 40 sets of simulation results, which were then compared against the real-world video recordings. The average peak signal-to-noise ratio (PSNR) of the rollout results was 21.6 dB, with the head camera reaching 23.4 dB, benefiting from a static background and lower prediction difficulty. Per-pixel error accumulates progressively during the rollout, rising from 6.8 in the first quarter of the sequence to 11.8 in the final quarter, amplifying by 1.74 times over the rollout interval of 150 to 275 frames. The paper notes that this phenomenon is consistent with the method's design trade-offs: branching from real observation nodes avoids unbounded, ultra-long rollouts. More importantly, the reward model made zero misjudgments across these 40 failure rollouts, never identifying a failure branch as a success. If such a misjudgment were to occur, the incorrect label would pollute the relative value signals throughout the entire learning process.

The second set of experiments evaluated a curling task to compare the world model's performance before and after co-evolution. The team used 64 validation trajectories, running three rollouts per trajectory. Following the co-evolution update, the model's error decreased across all 64 trajectories, with the average error dropping from 19.52 to 12.96, a reduction of nearly one-third. Error divergence grew with rollout length: for the frozen, unupdated world model, the error rose from 12.48 in the early phase to 23.73 in the late phase, a 1.90-fold increase; the co-evolved version increased only from 10.50 to 14.47, a 1.38-fold increase. The variance across multiple rollouts also narrowed from 9.39 to 6.12.

The paper provides a concrete example: when the first stone stops at the center of the house, the frozen world model incorrectly shifts the stone to the edge of the house at step 130, and prematurely predicts the second stone as entering the house at step 215, even though the real-world footage shows the stone still sliding; the co-evolved model remains consistent with the real video. Such simulation deviations directly alter the score of a single rollout, thereby distorting the value labels of the corresponding branches.

Notably, these 64 validation trajectories are part of the real-world data used for adaptation during the co-evolution process, meaning the frozen model has never seen these samples. The paper also explicitly states that this comparison measures fidelity in states the policy would actually visit, which demonstrates the effectiveness of online calibration but cannot be directly equated with the model's generalization to entirely novel scenarios.

Next steps in world model training

Placing Rollback within the technical landscape of the past two years, it addresses the challenge of how to effectively leverage world models.

Substantial prior work exists in this direction. According to the Rollback paper's review, World-Gymnast uses vision-language models to evaluate imagined rollouts for RL and explores iteratively updating the world model with real-world experience; WoVR emphasizes building the world model into a reliable simulator for the RL post-training of vision-language action models; and RISE uses a compositional world model alongside a progress value model to achieve closed-loop self-improvement in the imagination space. Another line of work is Physical Intelligence's RECAP, which directly applies advantage-conditioned RL to real-robot deployment data. According to its paper, this approach can more than double task throughput and roughly halve the failure rate on some of the most difficult tasks.

Rollback differs in two key ways: first, it treats rewinding as a fundamental training operation, performing tree search from real-world experience and using comparisons between sibling branches in place of a value model; second, it isolates the online calibration of the world model to study it as an inherent capability of the simulator itself.

Conceptually, it resembles applying the classic "search plus distillation" paradigms of AlphaGo and Expert Iteration to video world models, and aligns closely with the Go-Explore approach of returning to previously visited states before exploring. The distinction is that these classic methods rely on exact simulators or rules, whereas Rollback's search operates within a learned, fallible, and continuously calibrated video model.

For the industry, the most noteworthy aspect may be its resource allocation. Search capability can be scaled by adding GPUs, whereas real-world data can only grow with the number of robots. Using one physical robot and 48 H200 GPUs for 8 hours of training (excluding prior task-level pre-training for the world model and policy), Rollback outperformed the strongest baseline under an equivalent budget by an average of 11.21 percentage points across three tasks. If this ratio can be replicated across more tasks, the bottleneck in robot reinforcement learning could partially shift from real-robot time to compute, which is much easier to scale.

Of course, its limitations are equally clear.

First, Rollback relies on a reward model trained for a single task, as well as the fidelity of the world model in long-horizon, contact-rich scenarios—and fidelity analysis has already shown that errors accumulate over the course of rollouts.

Second, each policy in the paper was trained for a single task using only one physical robot. Furthermore, the paper notes in its system section that multi-site, multi-robot configurations have not yet been evaluated.

The compute requirements for this entire pipeline are substantial. Beyond the 48 H200 GPUs used during the policy optimization phase, the earlier pre-training stages are also highly resource-intensive: the world model first undergoes 10,000 steps of supervised fine-tuning on 32 H200 GPUs, followed by 2,500 steps of distribution matching distillation on 16 H200 GPUs; during the policy warm-up phase, it also requires a full 100,000 steps on 16 H200 GPUs. This configuration is a heavy burden for most laboratories, concentrating the methodological barrier on compute and high-quality world models. Meanwhile, the three tests conducted are all desktop-level mini-games with explicit rules and simple success-or-failure criteria. When transferring to long-horizon tasks with ambiguous goals and sparse feedback, such as household chores or assembly, it remains to be verified through further experiments whether the reward model and world model can maintain stable performance.

In short, robots have learned to "take back their moves," but to play longer, more complex games, it ultimately depends on how far the world model can reliably roll out.