Sharpa Open-Sources SharpaDex v1.0 Dataset Featuring Approximately 300 Hours of Real-World Tactile Manipulation Data Across 59 Tasks

On October 22, Sharpa open-sourced the SharpaDex v1.0 dexterous manipulation dataset. The dataset comprises approximately 300 hours of real-robot manipulation data, 28,993 demonstrations, and 32,396,172 frames, covering 59 manipulation tasks.

On October 22, Sharpa officially open-sourced the dexterous manipulation dataset SharpaDex v1.0. The dataset comprises approximately 300 hours of real-robot manipulation data, 28,993 demonstrations, and 32,396,172 frames, covering 59 manipulation tasks. According to Sharpa, SharpaDex v1.0 sets a new scale record for publicly available real-robot dexterous hand manipulation datasets featuring fingertip tactile feedback, establishing a new data foundation for tactile perception, visuo-tactile fusion, and complex dexterous manipulation policy learning.

To perform fine manipulation in the real world, robots must not only identify objects and their spatial locations but also perceive the physical changes that occur upon fingertip contact. Whether in precision assembly, tool use, or deformable object manipulation, contact states are intimately linked to action execution and task completion. Integrating tactile sensing, visual perception, robot actions, and task semantics remains a critical research direction in dexterous manipulation.

300 hours of real manipulation data with comprehensive multimodal recording

SharpaDex v1.0 was collected through real-robot teleoperation using a platform equipped with dual 7-DoF robotic arms, dual 22-DoF Sharpa Wave (W01) dexterous hands, and 7 torso joints. The dataset comprises 495 collection batches, totaling approximately 300 hours at 30 FPS.

For data recording, SharpaDex v1.0 synchronously captures visual information from a head-mounted stereo camera and left and right wrist cameras, while recording joint states, joint torques, and fingertip tactile signals for both arms and hands. Each demonstration also includes task descriptions, scene information, execution instructions, success criteria, and fine-grained action annotations, enabling researchers to correlate the robot's observed environment, executed actions, and actual contact states.

Each fingertip records 6-axis force signals, while preserving both raw tactile images and deformation images. Visual, tactile, action, and language information are aligned within a unified data framework, providing a more comprehensive observational foundation for studying real-world physical interactions.

Approximately 195 hours of fingertip contact, recording real physical interactions

The value of tactile data depends not only on its volume but also on whether it captures genuine, meaningful physical interactions. Statistics from SharpaDex v1.0 reveal that approximately 64.9% of the data frames feature at least one finger in a contact state, corresponding to roughly 195 hours of manipulation time.

Out of 28,993 total demonstrations, 28,992 recorded fingertip contact, yielding a cumulative total of 417,932 fingertip contact events. The data covers a range of contact scenarios, from light touches to forceful presses, and maintains temporal alignment with visual information, motion trajectories, and task semantics.

This enables researchers to analyze how contact states evolve throughout the manipulation process and explore tactile representation learning, contact state recognition, visuo-tactile fusion, and force-aware control, without relying solely on visual information to infer the interaction state between the robot and objects.

Multi-finger manipulation and bimanual coordination for complex task research

SharpaDex v1.0 extends beyond basic pick-and-place to encompass tool use, deformable object manipulation, precision insertion, screwing, and granular or small object manipulation. Specific scenarios include hammering nails, prying nails, pipetting with a dropper, ironing clothes, hanging clothes, loading batteries, plugging in modules, organizing items, and packing blueberries.

For bimanual coordination, approximately 18.3% of the data frames recorded simultaneous contact with objects by both hands, corresponding to about 55 hours of manipulation time. In 20 tasks, the duration of simultaneous bimanual contact accounted for more than 20% of the total task duration.

Some tasks also include operations where the left and right hands assume distinct roles, such as one hand stabilizing an object while the other performs insertion or tool manipulation. The data simultaneously records multi-finger contact conditions and hand joint movements, providing real-world manipulation samples for researching multi-finger coordination, bimanual role allocation, action composition, and cross-task transfer.

Three-tier language annotation from task descriptions to atomic actions

In addition to multimodal sensor data, SharpaDex v1.0 establishes a hierarchical language annotation system spanning tasks, subtasks, and atomic actions. This links continuous manipulation processes with natural language descriptions, temporal boundaries, and action labels.

Specifically, the task layer records manipulation goals, scenarios, instructions, and success criteria; the subtask layer divides continuous operations into segments with defined start and end times; and the atomic action layer provides finer-grained action semantics for the corresponding segments.

The dataset contains 224,087 subtask annotations, covering approximately 89.3% of the data frames, and provides 45 atomic action labels. These encompass general actions such as pick, place, insert, and press, alongside more task-specific operations like threading and hanging, fluid aspiration, grinding, and object handovers.

By mapping actions to natural language descriptions and temporal boundaries, researchers can further explore task decomposition, action semantic modeling, understanding of long manipulation sequences, and manipulation policy learning. Task-level success criteria also provide textual conditions for research into success detection and reward modeling.

Covering 59 tasks to provide a data foundation for dexterous manipulation research

SharpaDex v1.0 covers a total of six categories of manipulation tasks: 12 flexible or deformable object manipulation tasks, 14 tool-use tasks, 18 pick-and-place and organizing tasks, 8 precision insertion and assembly tasks, 4 granular and small-object manipulation tasks, and 3 screwing tasks.

Among these, tasks such as tool use, flexible object manipulation, precision insertion, and screwing total approximately 249 hours, accounting for approximately 83% of the total dataset duration. These tasks involve continuous contact, fine control, multi-finger coordination, or bimanual cooperation, providing rich samples for researching more complex real-robot manipulation.

As embodied AI research gradually shifts from basic grasping to complex real-world physical interactions, high-quality real-robot data is becoming a critical foundation for advancing robot manipulation capabilities. SharpaDex v1.0 integrates fingertip tactile sensing, multi-view vision, robot state, action trajectories, and hierarchical language annotations into a unified dataset, providing readily usable data resources for research on tactile perception, visuo-tactile fusion, bimanual coordination, and complex dexterous manipulation.

SharpaDex v1.0 is now publicly available on Hugging Face.