Robotics: New Over Old? HKU, Infinite Force, and Others Stress-Test VLAs on Real Robots

There is a long-standing problem in the neural networks of robots: catastrophic forgetting.

Today, the robot learns to organize tableware; tomorrow, it learns to fold towels; and the day after that, it learns to pack cardboard boxes. After learning multiple tasks, when it returns to the first one, it may degrade as if it had never learned it, unable to know how to handle it. For scenarios such as homes and factories, learning new things while forgetting old ones means that the maintenance cost of robots is extremely high.

So how can this problem be solved? On 7 month 28 day, a team from institutions including the University of Hong Kong, Infinity Force Robotics, and Ningbo Eastern Institute of Technology updated their latest paper titled 'Can Vision-Language-Action Models Learn from Real-World Data Continually without Forgetting?'

This research did not design a new VLA foundation model but instead placed an existing π0.5 into a real-world robotic task stream, allowing it to sequentially learn single-arm and dual-arm operations, and then observing how previously mastered skills changed.

The authors' experiments showed that if only fine-tuned item by item, old capabilities would significantly decline; however, retaining a small amount of historical demonstrations, interleaving replay at appropriate frequencies, and maintaining a unified action coordinate system could allow the model to continue retaining old skills while learning new tasks.

Here, 'continual learning' also needs to have its scope clearly defined. It does not mean the robot works while autonomously discovering tasks and updating online; rather, after obtaining teleoperation demonstrations for a batch of new tasks, it continues supervised fine-tuning on a GPU cluster, and then places the updated policy back onto the real robot for evaluation. This process connects data collection, model updates, and skill evaluation, providing a clearer engineering path for continuously adding new capabilities after robots go online.

Robots Also Forget Old Skills While Learning New Ones

The article shows that the research team built a self-constructed benchmark for 10 operations. The five single-arm tasks include stacking bowls, hanging cups, pressing buttons, folding towels, and pushing boxes; the five dual-arm tasks include placing bread into a pot, boxing two cans of cola, packing fruit in sequence, folding T-shirts, and packing paper bags, covering rigid body grasping/placing, contact manipulation, and flexible object folding.

The experimental platform consists of two PiPER-X six-degree-of-freedom robotic arms, parallel grippers, and four RealSense D435 cameras. For each single-arm task, 500 trajectories were collected; for each dual-arm task, 300 trajectories were collected, totaling 4000 teleoperation demonstrations and approximately 35GB of data.

The paper does not score long tasks using only 'success or failure,' but instead breaks tasks down into several observable intermediate steps. For example, placing two cans of cola into a cardboard box is broken down into four steps: right hand grasps, right hand places, left hand grasps, left hand places. Completing three steps yields 75 points, while completing all steps yields 100 points. Therefore, the 97.2 in the text refers to the 'average normalized task progress score' and cannot be directly written as a complete success rate of 97.2%.

The base policy uses π0.5 with approximately 2700000000 parameters, with all modules participating in updates. If fine-tuned independently for each task, the average baseline scores for single-arm and dual-arm tasks are 86.9 and 88.0 respectively. Without memory protection, continuing to fine-tune sequentially after learning each item, after completing the fifth item, the average scores drop to 31.4 and 38.3. In the single-arm sequence, the stacking bowl score drops from 100 to 20, and the hanging cup score drops from 97.5 to 0; the BWT metric measuring old task degradation also drops to -81.0 and -68.6 respectively. Skills learned earlier are more easily overwritten by later gradient updates.

These results refute an overly optimistic assumption: that VLAs with sufficiently large pre-training scales can naturally resist forgetting. The authors argue that tasks and action spaces in previous simulation benchmarks are closer to each other and may also overlap with the pre-training data of the base model; the distribution changes between real-world tasks are larger, causing the problems to surface.

A 'Recall Mechanism'

The main method adopted by the team is not novel: Experience Replay. After finishing learning one task, the system retains a small portion of complete trajectories from the demonstration data; when learning the next task, most training steps read new data, while a few steps extract historical trajectories, allowing the model to periodically 'review' while absorbing new skills. It adds no task-specific modules and does not require saving a complete set of models for each skill.

What truly determines the outcome is how replay is configured. When the buffer ratio is increased from 0.2% to 2%, the BWT for single-arm task flows rises from -8.95 to +0.25, indicating that forgetting has nearly disappeared. However, replay frequency cannot be simply increased indefinitely: raising it from 5% to 20% significantly improves the retention of old skills, but pushing it to 50% causes the model to sacrifice its ability to learn new tasks in order to review.

After adopting a 20% buffer ratio and a 20% replay frequency, the final average scores for both single-arm and dual-arm five-task streams reached 97.2, with BWT values of +1.5 and +1.9, respectively. When all five tasks were mixed together from the beginning for joint training, and the total number of updates was similarly controlled at 24000 steps, the average scores became 82.6 and 83.2. At least under this set of tasks and computational budget, learning in stages while interspersing old samples is more effective than optimizing all tasks simultaneously. The authors attribute this to gradient conflicts during multi-task training, but have not provided complete causal verification.

Another variable that is more easily overlooked is action normalization. The ranges of motion for robotic arm joints, end-effectors, and grippers differ significantly, so VLA training typically scales these values to a similar range. If this statistical calculation is redone for every new task, the same network output could correspond to different physical actions across tasks. This is akin to a model having just learned one coordinate language, only for the meaning of its 'words' to change in the next phase.

Among the three approaches, the best performance was achieved by freezing the action statistics of the first task and reusing them for all subsequent training and testing, with scores on all ten tasks no lower than 95. If each task used a separate set of statistics during training but unified to the first task's statistics during testing, the dual-arm sequences dropped below 0 for all tasks except the first. This does not imply that "always using the first task's statistics" is a universal solution; its validity relies on the premise that the action range of the first task can cover subsequent tasks. A more general insight is that continual learning must protect not only parameters but also the semantic consistency of the action interface.

VLA Starts Accumulating Its Own Experience

After successfully running both five-task flows, the team interleaved all 10 tasks: single-arm, dual-arm, single-arm, dual-arm. For the model, this is not just a doubling of task count, but requires repeated switching between completely different action dimensions, coordination methods, and contact modes. Based on this, the team designed an asymmetric replay plan: increasing the total replay frequency to 30%, while setting the sampling weight of dual-arm demonstrations in the buffer to twice that of single-arm ones, preserving more "review time" for skills with higher coordination difficulty.

After completing all ten tasks, the normalized scores of the ten items in the end-point matrix ranged between 80 and 100, with an average of approximately 92.9; the average backward transfer (BWT) was -2.7, and the average forward transfer (FWT) was +8.7. Skills learned early on, such as stacking bowls and hanging cups, were retained alongside skills added later, like pressing buttons and placing fruits, within a single continuously updated policy. This demonstrates a training approach closer to long-term usage: robots do not need to relearn all previous tasks from scratch every time a new capability is added.

These results transform "memory" into an actively manageable engineering object within VLA systems. Different skills exhibit varying sensitivities to parameter coverage. The training system can allocate replay budgets based on task structure, ensuring that complex skills receive greater protection. Experience replay thus evolves from a generic training technique into a mechanism for managing robotic capability assets—determining which experiences are worth retaining and which skills require more review, all within a continuous iterative process.

This has direct implications for Embodied AI. In reality, robots do not always face fixed workstations or fixed objects: production lines change products, warehouses introduce new goods, and service robots constantly receive new operational requests. If new tasks can be added sequentially to existing policies while old tasks are maintained through curated historical demonstrations, robotic software can shift from periodic holistic retraining to more flexible incremental upgrades. Data collection no longer serves only a single training session but gradually accumulates into a reusable experience library.

The paper and official code repository are now online, allowing access to core processes such as training, joint learning, and experience replay, providing a foundation for future research to expand across more tasks and robotic platforms. The value of this work lies not in inventing experience replay, but in integrating it into real-world robotic task flows to clarify how replay frequency, memory allocation, and action coordinate consistency collectively determine the effectiveness of continual learning.

The capability boundaries of general-purpose robots may depend less on how many tasks they know at factory launch and more on their ability to continuously accumulate new experiences after deployment. For robots to grow continuously, they must not only learn but also remember.