KEMO: Equipping VLA with a 'Memory Plugin' Boosts Long-Horizon Task Performance by 23.6%

On 2026, the month of 6, and the day of 17, robotics training infrastructure startup XDOF, founded in 2024, emerged from stealth mode and completed a funding round of 70000000 dollars, with investors including well-known venture capital firms such as Thrive Capital and Spark Capital.

Almost simultaneously with the completion of its funding round, XDOF released one of the largest open-source teleoperation datasets to date—** ABC-130K **: 130000 trajectories, 195 tasks, and 3,500 hours of real-world data.

However, larger-scale teleoperation data does not mean that VLA models can naturally acquire stable long-horizon manipulation capabilities.

Currently, the key bottleneck in long-cycle tasks lies in the time span between memory and decision-making—the model must remember critical states across hundreds or thousands of timesteps, ignore redundant frames, and make decisions at the right moment.

Therefore, on 2026 year 6 month 22 day, a paper titled 'KEMO: Event-Driven Keyframe Memory for Long-Horizon Robot Manipulation with VLA Policies' was published with Yihan Zeng as the first author. The list of authors includes Philipp Wu, one of the co-founders of XDOF.

The paper focuses on one of the most pressing issues in the current VLA field—the lack of memory in long-horizon manipulation tasks—and introduces KEMO as an "external module" for VLA to address this problem.

What KEMO Does for VLA

When robots perform long-horizon tasks, they often face the problem of "observing similar visual information at different stages of task execution."

In the past, to address this issue, methods such as "token merging," "recurrent states," or compressing all historical frames into fixed-size implicit representations, or using vision-language models (VLMs) to decompose long tasks, inevitably added extra inference costs and even introduced errors in sub-goal descriptions caused by the task decomposition process.

To address this, the paper introduces KEMO, a plug-and-play memory framework designed for long-horizon manipulation. The principle of KEMO is to identify "event keyframes" that mark state transitions between task stages, providing compact and explicit historical evidence to eliminate ambiguities caused by visual similarity across different stages, thereby supporting action prediction for each corresponding stage.

The KEMO framework mainly consists of two parts: "Keyframe Detection" and "Keyframe Integration."

• Keyframe Detection

First, an event saliency function is designed, taking robot joint position information as input. When key operational events such as grasping or placing occur, the joint movement speed slows down (typically, the joints intentionally decelerate when transitioning to the next stage of the task), causing the function's output value to increase.

Next, by detecting whether the output sequence of the aforementioned functions produces local peaks over a period of time, and identifying the moments when corresponding robot joints slow down as candidate event keyframes.

Finally, to avoid interference from motion deceleration signals that robots might generate due to posture adjustments or brief pauses, KEMO uses a frozen DINOv2 encoder to calculate the visual dissimilarity between candidate event keyframes and the most recent accepted event keyframe. A candidate frame is only accepted if its visual dissimilarity exceeds a certain threshold.

Through these three steps, KEMO can reliably identify event transition stages without task-specific annotations.

• Integrating Keyframes

First, encode the keyframe images using the same SigLIP vision encoder as the base policy, and apply spatial average pooling of 4×4 to reduce each keyframe encoding from 256 tokens to 16 tokens, thereby reducing computational cost.

Next, the 16 compressed tokens for each keyframe are stored in a temporal memory bank. Each keyframe slot in the bank receives a 'learnable temporal position embedding.' This 'embedding' broadcasts temporal information to the remaining 16 tokens, solving the problem where frames like 'the first cup-covering action' and 'the second cup-covering action' look visually similar but carry different temporal meanings.

Finally, memory is integrated via cross-attention: current observation 'visual tokens' retrieve 'event keyframe tokens' from the temporal memory bank. Through a gated residual structure, as training progresses and memory tokens become informative, the gate gradually opens, selectively fusing memory information into the VLA's language model backbone.

In essence, KEMO does not redesign the VLA, nor does it allow the language model to directly 'read' the memory. Instead, it performs a 'historical retrieval' using cross-attention on visual features before they enter the language model, conducts an 'information filtering' step using a gated residual, and then passes the enriched visual representation unchanged to subsequent training.

How Effective Is KEMO

All experiments were conducted on the real-world YAM dual-arm robot. YAM is a dual-arm collaborative robotic platform with multiple degrees of freedom per arm. The experiments used multi-camera RGB observations and joint positions as inputs for the robot's state.

The KEMO framework strategy initializes with a pre-trained VLA backbone network π0.5 and fine-tunes it via supervised fine-tuning (SFT) on collected demonstration data. To clarify the loss weighting shown in the figure: the weights for 3 time windows at each event keyframe are increased by a factor of 8.

This strategy was compared against two baselines: one that undergoes fine-tuning on the same demonstration data without memory enhancement (π0.5), and another memory-enhanced strategy called MemoryVLA, across 6 tasks.

A necessary note on MemoryVLA: inspired by cognitive science, it is a dual-memory framework that constructs a perception-cognition memory bank using traditional VLM encoding and generates actions through a retrieval-fusion mechanism. In contrast to KEMO, which relies on dense memory built from general VLM tokens, MemoryVLA is a lightweight plugin-level framework that stores only sparse memory.

The experimental results are also quite striking. Across all tasks, π0.5 achieved an average overall TSR (Task Success Rate) of 27.8%, while MemoryVLA achieved 1.4%. In contrast, our method achieved an average task success rate of 51.4%, representing an improvement of 23.6 percentage points over π0.5.

Our method also improved the SCR (Stage Completion Rate) from 42.3% to 76.4%, an increase of 34.1 percentage points. These results indicate that the proposed KEMO framework strategy has improved success rates and completion progress across tasks.

The final ablation experiments (experiments that remove a certain module to observe its impact on the overall task) demonstrate that both "gated residual fusion" and "loss weighting" significantly affect overall task completion.

In particular, after removing the loss gating, the task success rate was directly adjusted from 9 times to 0 times. At this point, memory features dominate visual representations, indicating that memory can only serve as a supplement to the current state rather than a replacement, which aligns with KEMO's plug-and-play design philosophy.

**Why It's Worth Noting

First, training a larger VLA base model (such as doubling the parameter scale of π0.5) requires not only computing resources but also more and more diverse training data for the larger model, and even more complex training infrastructure. Moreover, inference latency will grow linearly or super-linearly.

For a robotic system already running in a production environment, switching to a larger VLA model may mean re-evaluating the entire hardware-software stack.

The KEMO method in this paper is built on top of π0.5, but its architecture design features four components—the keyframe detector, memory encoder, gated fusion layer, and loss weighting—that do not depend on any specific attributes of π0.5.

Event-based keyframe detection relies solely on joint positions and RGB images; the memory encoder shares the VLA's visual encoder (SigLIP); and the gated fusion does not alter the VLA's language model backbone.

This means that theoretically, KEMO can be integrated into any VLA policy using SigLIP (or a similar visual encoder). However, one engineering detail must be noted: KEMO currently uses SigLIP as its visual encoder. If the target VLA employs a different visual backbone (such as CLIP or InternViT), the memory encoder needs to be adjusted accordingly. This is not an architectural limitation but requires engineering adaptation.

From an industrial perspective, the Cover Blocks, Box Refill, and Drawer Items Replacement tasks used to validate KEMO's capabilities are essentially variations of assembly tasks—'cover-uncover' corresponds to assembly-disassembly, 'refill' corresponds to category-based replenishment, and 'drawer item replacement' corresponds to part swapping.

In domestic scenarios, however, KEMO's current design validates short-term (single-task) memory based on event detection, which is still far from meeting the long-term memory requirements typical of household environments.

Specifically, the current keyframes in KEMO contain only RGB images. Tactile, force, and auditory modalities carry rich signals regarding task progress during fine manipulation, yet these are not currently utilized by the memory system.

But its core architectural principle—storing key events rather than densely recording all historical frames—may serve as the foundation for building larger-scale, longer-span memory systems.

In summary, KEMO efficiently meets the VLA's need to remember execution history in long-horizon operations through its plug-and-play lightweight plugin design and event-driven enhanced memory mechanism. This perhaps suggests that complex world models are not required; relying solely on refined key-frame memories can significantly enhance temporal reasoning capabilities.