Tsinghua's Li Yi Team Releases PASSAGE for Humanoid Robot Navigation in Cluttered Environments

PASSAGE research has accumulated human motion data aligned with scenes to the scale of 100 hours. By combining a flow-matching planner with a perception-enhanced full-body tracker architecture, it enables humanoid robots to autonomously select and combine diverse movement actions—such as stepping over, passing underneath, and sidestepping—in cluttered and confined environments. Simulation test results intuitively demonstrate that the success rate of contact-free passage for the robot increases continuously as the scale of training data grows.

A joint team from Galbot, Shanghai Institute for Periodic Intelligence Research, ShanghaiTech University, Beijing Zhongguancun Academy, Shanghai Jiao Tong University, National University of Singapore, Tsinghua University, and Peking University officially released the related paper on September 16. The paper is titled "PASSAGE: Scaling Scene-Aligned Motion Learning for Perceptive Humanoid Traversal in Cluttered Environments." The paper features twenty authors, with Yuxuan Ma, Zicheng Zeng, and Chunlin Peng serving as co-first authors. Li Yi, Assistant Professor and doctoral supervisor at the Institute for Interdisciplinary Information Sciences, Tsinghua University, serves as the corresponding author. Li Yi previously participated in two classic 3D vision research projects, PointNet++ and ShapeNet, and has long been deeply engaged in research directions related to 3D perception and humanoid robot learning.

Prior to this, the ability of humanoid robots to navigate complex environments mainly relied on two technical paths. One path relies on training exclusive expert policies via reinforcement learning for specific scenarios, followed by model distillation to obtain a general controller. This approach incurs high overall R&D costs, and the range of actions the robot can perform is limited by artificially set motion goals. The other method involves collecting human demonstration motion data; however, requiring demonstration data to be highly aligned with scene geometric features so that it can be adapted to cluttered obstacle scenarios and autonomously called by the robot presents significant challenges. Previous public datasets have always had obvious limitations in volume. The core breakthrough of this team lies in expanding scene-aligned demonstration data to 100 hours, covering 1500 types of scenarios, while establishing a complete technical chain that connects the full-process perception, planning, and control system from onboard LiDAR to joint command output.

01 Robots must first decide how to pass

With their complete full-body joint structures, humanoid robots can achieve various traversal actions at the hardware level, such as stepping over knee-high obstacles, sidestepping through narrow passages, and ducking to pass under suspended beams. The true technical difficulty lies in enabling the robot to autonomously judge the appropriate traversal method for the scenario without human intervention, seamlessly combining diverse actions to complete the entire journey, with the entire decision-making and execution system relying entirely on onboard sensors.

Current mainstream technical solutions can be categorized into two types. The first trains expert policies for exclusive scenarios via reinforcement learning, then distills and integrates multi-scenario expert models into a general controller. While feasible in cluttered environments, this approach requires artificial pre-definition of motion goals, incurs huge computational overhead for training expert models, and existing general controllers only support driving twelve leg joints, significantly limiting the boundary of the robot's full-body coordinated movement capabilities. The second solution relies on collecting and training real-human motion data. Its core pain point is the extremely high threshold for action reuse in cluttered scenarios; only demonstration data precisely aligned with surrounding geometric structures can be effectively recognized and called by the robot.

In previous domain research, the volume of scene-aligned demonstration data was generally small. Public datasets had completed only 2.3 hours of data collection, covering 145 procedurally generated scenarios. Related research focus mostly remained limited to data construction and benchmark testing levels. Two core questions have always remained to be verified within the industry: whether scene-aligned demonstration data can support the end-to-end implementation of the full perception-planning-control chain, and to what extent the volume and scale of demonstration data affect the quality of robot action generation and downstream execution accuracy.

The research objective of PASSAGE is clear: it inputs only the destination target position to the robot, leaving the autonomous determination of the passage path and movement method entirely to the robot. This study focuses on combined passage scenarios involving spatial constraints from the ground, side, and overhead, temporarily excluding complex action scenarios such as climbing stairs and platforms, scaling obstacles, and crossing long-distance barriers. The team has targetedly built a complete technical chain from data collection, filtering, and augmentation to real-machine onboard deployment, and quantitatively verified the value of data scale in enhancing the whole-machine passage capability under a unified technical framework.

02 All 50 Real-Machine Trial Walks Reached the Destination

The team deployed the unified planner and tracker models onto the real machine, completing practical verification across fifty sets of differentiated physical scenario layouts. The test scenarios were divided into five major categories, including single crossing/passing, sideways squeezing, simultaneous upper-lower space restriction, and multi-action sequence combinations. Each category featured ten different layouts, with each layout executed for only one passage test. Throughout the testing process, only destination position commands were issued to the robot; there were no preset behavior labels or upper-level skill screening/scheduling modules, relying entirely on autonomous model decision-making.

In the fifty real-machine tests, the robot successfully reached the destination area in all instances, with no falls and no need for manual intervention. Among these, forty-five cases achieved zero-collision passage throughout the entire process, resulting in a non-contact passage rate of 90%. All twenty double-constraint and combined-action scenarios were completed successfully, with seventeen achieving zero contact throughout, and eight sequential combination scenarios achieving zero-contact results. Scenario contact situations were confirmed by on-site manual annotation. The team explicitly stated in the paper that the five cases with slight contact are sufficient to demonstrate that the core bottleneck of the current technology lies in ultimate obstacle avoidance capabilities, while the robot's path planning and full-process arrival capabilities are already fully mature.

Simulation tests further supplemented refined controlled experiments. The test set selected one hundred and fifty groups of brand-new scenarios never involved in training, evenly divided into three difficulty levels of fifty groups each. Each algorithm was tested five times in a single scenario, totaling seven hundred and fifty test rounds, with a maximum duration of sixty seconds per round. The criterion for successful arrival was the robot entering within a 0.5-meter range of the destination point; zero-contact arrival required satisfying no obstacle collisions throughout the process under this condition. Using the general version of the CAT algorithm without secondary training, the model achieved an arrival rate of 70.3%, a zero-contact arrival rate of 14.0%, a fall rate of 20.1%, and a contact time per unit walking distance of 0.3192 seconds per meter. The team also referenced the research scheme by Zhang et al. to build a diffusion-plus-tracker control technical pipeline, achieving an arrival rate of 86.0% and a zero-contact rate of 19.2% based on twelve hours of training data. Under the premise of identical twelve-hour training data, the average arrival rate of the PASSAGE model across three random seeds reached 90.0%, the zero-contact arrival rate increased to 48.6%, and the contact time per unit walking distance decreased to 0.1230 seconds per meter.

This controlled experiment has a clear scope definition. Although the test scenarios did not participate in the training of the PASSAGE model, they align with its training data distribution characteristics, whereas the CAT algorithm adapted to new scenarios in a zero-shot state throughout. Therefore, this comparison result is primarily used to verify the scene migration and generalization performance of the entire system, rather than a pure horizontal capability benchmark between the two action generation algorithms.

03 How Was the 100 Hours of Action Data Obtained?

All this data was obtained through human demonstration collection. The team built a procedural scenario generator, splicing block obstacles in a ten-meter-long corridor. Along the corridor, eight difficulty variables were selected, including step height, lateral offset, channel length and width, probability and height of ceiling appearance, and probability and height of ground protrusions, covering spatial constraints from the ground, side, and overhead.

In the data collection phase, operators wore Noitom PN Link inertial motion capture suits and VR headsets to navigate various scenes from a first-person perspective of a virtual character. If the virtual body collided with the scene environment, tactile feedback would be triggered, invalidating that demonstration sample. The team also required operators to actively vary their movements and navigation strategies to prevent everyone from adopting homogenized walking patterns.

The team performed collision removal and quality filtering on the collected raw samples, ultimately retaining 19,310 sequences totaling 17,470,128 frames, with a total duration of nearly 100 hours, distributed across 1,500 different scenes. The collected motions were then retargeted to the Unitree G1 robot, which has 29 degrees of freedom, while the accompanying scenes were scaled synchronously, maintaining the alignment relationship between motion and geometry.

The raw data underwent further augmentation processing. Motions and scene geometries were stored separately, allowing retargeting and augmentation operations to be conducted independently. For each motion segment, ten obstacle variants were generated, with scale ranges maintained between 0.5 and 1.5 times the original size, and rotation angles controlled within ±15 degrees. Uniform scaling was applied to match the retargeted motions, followed by kinematic playback using MuJoCo, keeping only action-scene pairs without collisions. After this verification round, the original 100-hour sample set was expanded to approximately 1,000 hours of action-scene pairs, with the entire augmentation process using the same batch of human demonstration actions. During the training phase, sample diversity was further increased, with up to 30 background layouts paired per scene, and left-right mirroring applied to actions with a probability of 0.5.

04 Planner, Tracker, and Onboard System

The PASSAGE framework divides motion generation and motion execution into two independent modules, trained separately and connected via a predefined interface. This architecture eliminates the need to train controllers for each specific behavior; onboard perception and computing units also connect to this unified interface. In real-robot operation, only a target point needs to be input to the robot.

How the Planner and Tracker Divide Responsibilities

The planner is responsible for determining the movement plan for the next half-second. It receives four frames of motion history, each being a 65-dimensional state encompassing torso height, postural tilt, planar velocity, yaw rate, as well as the positions and velocities of 29 joints. Two other inputs are local target points in the robot's coordinate system and a three-layer elevation map centered on the torso, sized 3×31×61, where the three layers record support surfaces available for stepping, side occlusion areas, and overhead clearance. The planner outputs reference actions for the next 25 frames (i.e., 0.5 seconds), generated using conditional flow matching, forming complete action blocks through integration starting from noise, with an inference frequency maintained at 6.25 Hz.

During the training phase, the team incorporated multiple constraints, including finite difference consistency for root state and joint velocity to reconstruct position and velocity, foot slip keypoint jitter, and bounding box penetration. Another constraint was derived from HumanoidPF: the team precomputed potential fields from scene signed distance fields, sampling at eleven anchor points located on the pelvis, torso, head, shoulders, palms, knees, and feet. Repulsive forces are generated when an anchor point enters a 0.2-meter range of an obstacle, and moving away from the guidance direction also incurs a loss. This potential field is used only during the training process; no additional inputs or computational overhead are introduced during inference.

The tracker is responsible for converting reference actions into joint commands. The team adopted the ScaleBFM full-body tracking interface, whose base checkpoint comes from another research output by the team, pretrained on approximately one thousand hours of general action corpus data, and adapted for the twenty-nine-degree-of-freedom G1 robot equipped with an AGX compute backpack. Terrain information, encoded as tokens via eight convolutional networks, is appended to the reference sequence, while the pretrained transformer and action head structures remain unchanged. The adaptation process employs an asymmetric actor-critic PPO algorithm: first, the terrain encoder and critic are optimized over two hundred iterations, followed by fine-tuning the overall policy using randomization of dynamic observations and external disturbances. The tracker operates at fifty hertz, outputting twenty-nine-dimensional residual joint position commands. Its training data is sourced from reference actions within the dataset; planner outputs do not participate in this training step.

Perception and Compute Are Onboard

Real-robot deployment uses the same planner and tracker. All computation tasks are performed on the Unitree G1 itself, supplemented by an NVIDIA Jetson AGX Orin module and a Manifold Tech Odin module. The entire system does not rely on pre-built maps, and no computation steps are offloaded to external devices.

Perception is handled by the onboard LiDAR. Registered point clouds and odometry are fused online and fed into ROG-Map to maintain a robot-centric three-dimensional occupancy grid. A three-layer elevation map is extracted directly from the grid. Each vertical voxel column outputs three types of information: the highest occupied surface, the lower boundary of obstacles supported by observed free space beneath that surface, and the supporting surface at a deeper level, corresponding respectively to walkable areas, side occlusions, and overhead clearance. Unobserved voxels are not directly classified as empty; this rule is critical for operational safety. The extracted surfaces are cached in the torso coordinate frame and resampled at twenty hertz using the latest odometry to mitigate pose staleness caused by gaps in point cloud updates.

The planner and tracker belong to separate processes. The planner is exported as an ONNX model and runs at FP16 precision using the TensorRT backend of ONNX Runtime. It requests a new planning solution once every eight fifty-hertz control steps, resulting in a planning frequency of 6.25 hertz. The tracker runs on the onboard CPU at fifty hertz. The latest joint targets are passed to an out-of-process C++ bridge module via shared memory and then dispatched to the robot body at five hundred hertz.

05 Data Scale and Four Ablation Groups

Data Increased from 6 Hours to 100 Hours

The research team conducted controlled variable experiments specifically targeting data scale, setting up five tiers of training data volumes: 6 hours, 12 hours, 24 hours, 48 hours, and 100 hours. For each tier, the planner was trained using three independent random seeds. Throughout the experiment, the model architecture, optimization budget, RTC mechanism potential field goal closed-loop reinforcement learning post-training method tracker checkpoints, evaluation episodes, and all other core conditions were kept constant; the only variable was the subset of pre-trained data used by the planner.

The test results from the five-tier gradient experiments clearly demonstrate that the optimization trend for obstacle avoidance performance is the most stable. The task arrival rate shows continuous improvement, rising steadily from 84.3% with 6-hour data training to 96.4% after 100-hour data training. The zero-contact arrival rate increased from 48.1% to 68.9%, and the contact duration per unit distance decreased from 0.3565 seconds per meter to 0.0499 seconds per meter. Within the 6-hour to 12-hour interval, fluctuations in zero-contact performance remained within the range of seed random error and lacked significant reference value; however, both the task arrival rates and contact durations measured across different seeds showed continuous optimization. Fall rates and foot slip probabilities did not exhibit a unified monotonic change pattern, indicating that the performance gains brought by expanding data scale are primarily concentrated in two dimensions: task arrival and autonomous obstacle avoidance.

The research team applied scene augmentation strategies on top of the 100-hour baseline data to further explore the model's performance limits. The optimized model ultimately achieved a task arrival rate of 98.7% and a zero-contact rate of 70.3%, with the contact duration per unit distance optimized to 0.0422 seconds per meter. Compared to the average of three seeds using only the original 100-hour data without scene augmentation, adding scene augmentation improved the task arrival rate by 2.2 percentage points, the zero-contact rate by 1.3 percentage points, and reduced exposure time per unit distance by 15.4%. This portion of performance gain came solely from variations in scene geometric forms, without increasing the overall volume of human motion demonstrations; therefore, the magnitude of performance improvement was far less than that brought by scaling up the original data volume.

What Each of the Four Ablations Measured

The research team conducted four independent ablation experiments to quantify and decompose the individual contributions of each technical module one by one. After removing the RTC mechanism, the model's comprehensive performance dropped significantly: the task arrival rate fell from 98.7% to 92.7%, the zero-contact rate plummeted from 70.3% to 24.5%, and the contact duration per unit distance rose from 0.0422 seconds per meter to 0.4038 seconds per meter, representing an overall error increase of nearly tenfold. The team analyzed that if geometric shape jumps occur during adjacent planning iterations, the lack of RTC consistency constraint regulation causes minor reference coordinate offsets to be continuously amplified by the execution layer, eventually evolving into collision issues during the robot's actual operation.

The model's performance showed a significant decline again after removing the post-training module for reinforcement learning on the planner side. The task arrival rate dropped to 94.4%, the zero-contact rate fell to 26.4%, the contact duration per unit distance increased to 0.1985 seconds per meter, and the foot slip probability rose from 0.0113 to 0.0185. This set of experiments demonstrates that, under the premise of locking the tracker's performance parameters, allowing the planner to continuously iterate and optimize based on deviations generated during actual robot operation can simultaneously improve task arrival effectiveness, obstacle avoidance capabilities, and overall execution precision.

The research team kept the planner and interaction interface configurations unchanged, replacing only the high-performance tracker with a basic ScaleBFM checkpoint, which caused a significant attenuation in the model's core performance metrics. The task arrival rate dropped to 84.4%, the zero-contact rate fell to 18.4%, and the fall rate increased significantly from 0.9% to 6.9%. This set of experiments precisely quantifies the pure benefit brought by perception enhancement on the tracker side; the optimization effect of local geometric real-time feedback is particularly prominent in complex limited-space scenarios such as narrow corridors. On this basis, separately removing the potential field loss constraint resulted in the model maintaining a task arrival rate of 84.8% and a zero-contact rate of 20.1%, while the contact duration per unit distance rebounded to 0.1538 seconds per meter, indicating a further obvious weakening of overall obstacle avoidance capability.

Combining the comprehensive results of the four ablation experiments, the research team clarified the modular division of labor logic within the entire technical system. Scene-aligned demonstration data can build an action resource library adapted to various geometric environments; potential field losses can establish explicit collision avoidance prior conditions; the RTC mechanism can ensure action continuity between adjacent planning iterations; reinforcement learning on the planner side allows the robot's actions to adapt to real operational deviations on the execution end and complete fine-grained optimization; and perception enhancement on the tracker side can perform local geometric corrections during real-time execution, ensuring the stability of the entire machine's movement.

06 In Conclusion

The core breakthrough of the PASSAGE solution lies in advancing the training volume of scene-aligned action data to a new magnitude of 100 hours. Simultaneously, relying on the combined architecture of the planner and tracker, it connects the complete link between behavioral decision-making and full-body motion execution, enabling the entire intelligent control logic to be fully implemented on real robot operations. Previously, the behavioral selection and motion execution of humanoid robots were in a state of mutual separation; behavioral strategies either relied on expert models trained via reinforcement learning or depended on manually organized fixed skill libraries. Both technical paths require substantial additional R&D costs when adapting to new behaviors and new scenarios.

The shortcomings of this solution are equally clear and explicit. During 50 real-robot tests, there were 5 instances of obstacle collision problems. The contact data required for training relies entirely on manual annotation, with samples covering only a single hardware platform and 50 scene layouts, showing obvious limitations in data richness and scene coverage. The perception layer retains only the geometric perception channel; LiDAR cannot identify special structures such as transparent surfaces and thin cables. This shortcoming will be addressed by supplementing RGB perception capabilities in the future. The existing action library only adapts to three types of spatial constraints: ground, side, and overhead, and temporarily cannot achieve complex motion behaviors such as climbing stairs, platforms, obstacles, or crossing gaps. Scene augmentation methods can only achieve diverse expansion of geometric forms; the upper limit of the model's generalization ability remains constrained by the richness of human demonstration data.

For the entire humanoid robot R&D field, this study has formed conclusions with reusable value. Scene-aligned action data and execution-layer perception feedback play independent and irreplaceable roles respectively; the performance gains brought by the RTC consistency mechanism and post-training reinforcement learning on the planner side can both be quantitatively verified. Although this framework currently cannot adapt to high-difficulty scenarios such as long-distance travel and complex terrain navigation, it completely outlines the positive correlation between data scale expansion and improvements in robot obstacle avoidance performance, providing a practical and reusable technical reference for the iterative upgrade of embodied intelligence and autonomous obstacle avoidance capabilities in humanoid robots.

Paper: https://arxiv.org/abs/2609.18732 Project Page: https://galaxygeneralrobotics.github.io/PASSAGE/