500,000 Hours of First-Person Video: How to Add 'Touch' to Embodied World Models

A robotic hand plucks a grape from a bunch. The difficulty does not lie in "seeing" the grape, but rather in determining when to make contact, which finger should bear the force, and how much pressure to apply so as not to crush the fruit's flesh. Squeezing toothpaste, retrieving an obscured item from a soft bag, or writing with a soft pen are all problems of the same category: vision can describe objects and postures, yet it is difficult to directly ascertain whether contact has already occurred, or whether one should next reduce or increase the applied force.

On July 28, the general embodied foundation model company Zhi Wu Zai Jie released a technical report titled 'Being-H0.8: A Latent Tactile World-Action Model at Scale' and announced that it has collected more than five hundred thousand hours of raw first-person human videos.

This model continues its world-action modeling approach, but shifts the training focus from "predicting how the visual scene will change" to "how contact occurs and how actions should be adjusted based on feedback." The team positions it as the first model of this scale to introduce tactile pre-training on first-person human video datasets.

The news point of Being-H0.8 is not merely the addition of a tactile input channel. What truly deserves attention is its attempt to convert massive first-person videos, originally lacking tactile sensors, into 'pseudo-tactile' supervision capable of serving robot action learning, while placing human hands, dexterous hands, and grippers within a single state-action representation. For embodied models, this amounts to attempting to shift the most scarce contact experience—previously requiring expensive real-world data collection—partially forward to the video data processing stage.

500,000 Hours Is Not the End

The report states that the data foundation UniHand 3.0 of Being-H0.8 aggregates over five hundred thousand hours of raw first-person human videos, sourced from public datasets, partner data, and in-house data. The official page also explicitly acknowledged thirteen core data providers. The value here lies not only in volume: the team emphasized that each sample is traceable to its source, with formatting, metadata, duplicate clips, and quality issues addressed through a semi-automatic, human-in-the-loop workflow.

This data pipeline addresses a practical contradiction in embodied pre-training: while public videos are abundant, much of their content is irrelevant to manipulation; repeated clips of the same scene and action artificially inflate data scale; and occluded hands, rapid motion, or camera shake distort subsequent 3D hand trajectories. The report reframes the question from 'how much video was collected' to 'how many reliable embodied experiences were filtered out.'

The quantitative evidence provided primarily focuses on this step. In an evaluation of 5,000 randomly sampled videos by human annotators, its hand motion reconstruction and quality assurance process reduced severe tracking failure rates to 39%, lower than HaWoR's 67% and Dyn-HaMR's 57%; hand shape distortion rate was 11%, compared to 28% and 17% for the two baselines respectively; and the overall failure rate of the complete processing chain was 11%, versus 73% and 61% for the baselines. Automatic quality assurance could also identify over 92% of erroneous clips. This demonstrates that the data reconstruction and filtering pipeline is more usable, but does not equate to improved robot task success rates.

Where does 'touch' come from? Being-H0.8 does not treat ordinary RGB video as real pressure sensor readings. Its TactoHand module first learns on hand-object interaction data with geometric supervision: for a tracked hand, it predicts two dense maps on a unified MANO hand surface—a binary map of contact areas and a continuous field indicating proximity to the contact surface. The former answers 'where it touched,' while the latter describes whether 'contact is approaching or receding.'

Then, the system applies this model to unlabeled first-person videos, generating pseudo-touch labels aligned with hand topology; unreliable hand reconstructions are masked rather than crudely labeled as 'no contact.' When actual robot or gloved human data provides sensor readings such as contact and pressure, the system projects these signals of varying density and location onto the surface of the same 'standard hand.' In other words, video provides scalable contact and proximity supervision, while sensor data offers scarce but more direct physical observations; the two cannot be conflated.

This data system is not composed solely of human videos. The report also calibrates robot data from different bodies, cameras, and control interfaces to shared kinematic and coordinate specifications; transforms human operation videos generated under text or image conditions into paired robot demonstrations containing RGB, depth, and motion trajectories through 3D hand reconstruction, motion retargeting, and depth-aware synthesis; and separately collects bimanual human demonstrations matching downstream tasks, serving as an intermediate layer transitioning from general pre-training to robot adaptation. The former two address data heterogeneity and scale, while the latter pulls the model back to the operational semantics and contact behaviors required for real tasks.

Learning Actions After Contact in World Models

At the model level, Being-H0.8 retains the prior-posterior training approach of its predecessor. The prior branch available during deployment reads current instructions and visual observations; the posterior branch during training also sees future visual and tactile evidence. Both branches use the same latent variable positions, allowing the posterior to teach the prior to predict action-related consequences from current information. During actual operation, the posterior branch and future information are removed, so it does not generate future frames pixel by pixel, but rather leverages the learned latent states to condition action generation.

There is another problem with rich-contact manipulation: planning a long sequence of actions and executing them in an open-loop manner often fails to respond in time to situations where fingers have already touched the object, the object's stiffness changes, or the grip slips. To address this, the model incorporates "slow-fast" action experts: the slow channel computes and caches visual, language, and world states at the beginning of an action segment to provide a complete plan; the fast channel reads the latest proprioceptive state and tactile feedback before each shorter execution segment to regenerate the next short segment of actions. The team hopes that this division of labor will simultaneously preserve long-horizon planning capabilities and higher-frequency reactive abilities.

Another engineering challenge is that touch does not have a unified format. Some systems only have global contact labels, some only have a few fingertip sensors, and others possess per-vertex contact, proximity, or pressure measurements. Being-H0.8 accepts these signals using a coarse-to-fine "touch pyramid" on a unified hand topology, and employs a "missing touch" representation when tactile sensors are absent, thereby avoiding the need to switch model interfaces due to different sensor layouts.

The same unified approach also applies to actions. TopoHand decomposes wrist pose, hand morphology, and 20 standardized joint angles into separate representations, then uses an adapter to map them back to the control commands for human hands, dexterous robotic hands, or parallel grippers. It does not grant different hardware inherently identical manipulation capabilities; rather, it solves the alignment problem of 'the same action slot having different meanings across different hands' in training data. For foundation models requiring transfer across human videos, various dexterous hands, and grippers, this is more scalable than attaching a separate policy head for each type of hand.

Entered Real-World Task Scenarios

On real hardware, the project page showcases contact-rich manipulation tasks performed on two dual-arm platforms: including squeezing toothpaste, blind retrieval from soft bags, writing with a calligraphy brush, picking grapes, grabbing potato chips, and responding to human intervention during cleaning. The hardware setup includes LinkerHand L25 equipped with a 12×6 piezoresistive pressure array, DexHand 021 featuring fingertip visual-tactile sensors, and tactile grippers, among other combinations. These demonstrations cover scenarios involving deformable objects, occlusion, fine tool usage, bimanual coordination, and rapid response, indicating that the team is testing the model in real-world contact tasks rather than relying solely on video-based metrics.

The focus of Being-H0.8 lies in addressing the challenge of insufficient tactile data through a scalable data engineering pipeline: first, recovering contact cues from human videos; second, integrating observations from sensors with varying densities and different robot bodies into a unified interface; and finally, feeding these experiences back into real-time action correction. This approach consists of four interlocking components: 500,000 hours of first-person-view videos, pseudo-tactile annotations from TactoHand, a reactive action mechanism called Slow-Fast, and cross-body action representations from TopoHand.

For Embodied AI, what is truly scarce is not merely "how many objects have been seen," but knowing how actions should change after the hand makes contact with an object. Cameras have provided robots with large-scale visual experience; Being-H0.8 aims to advance this by incorporating contact, proximity, pressure, and action adjustments into the same scalable world model for learning. As human videos, robot data, and real tactile observations are integrated into a single training pipeline, the training objective for embodied models is shifting from "seeing how the world changes" to "understanding how contact alters subsequent actions."