PokeBot Launches Free RoboAPI to Adapt a Single World-Action Model Across Multiple Robot Arms

On October 30, PokeBot released RoboAPI, a public interface for its proprietary Poke-Aurora world-action model, aiming to eliminate the need for repeated data collection and retraining when developers switch robot hardware.

On October 30, PokeBot launched RoboAPI on its official website, packaging its proprietary world-action model, Poke-Aurora, into a publicly accessible API.

The company states that RoboAPI addresses a key pain point in deploying embodied models: when developers switch robot hardware, they typically must repeat the entire cycle of data collection, model training, performance validation, and supplementary data gathering before the model can execute tasks reliably on the new device.

The calling logic is straightforward: developers upload the robot's observations and state, input a natural language instruction, and RoboAPI can predict and return a continuous action sequence in near real-time. The API is currently in a technical preview phase and is freely available to the entire industry, but it is restricted to non-commercial use and does not yet offer commercial services to for-profit third parties.

Image | Footage from the RoboAPI release video (Source: PokeBot)

PokeBot was founded by Huazhe Xu, an assistant professor at the Institute for Interdisciplinary Information Sciences at Tsinghua University. The team believes that the core bottleneck in deploying embodied models lies in the vast differences in mechanical structures, sensor configurations, and control schemes across different robots. Even if an embodied foundation model with reasonable generalization capabilities is open-sourced, it remains difficult to transfer it directly to other robotic hardware. RoboAPI targets zero-shot generalization, aiming to enable robots to execute tasks immediately upon receiving instructions without requiring additional data collection or secondary fine-tuning.

The company compares RoboAPI to the original GPT-3 API, which allowed developers to access cutting-edge capabilities via an interface without having to train and deploy large language models themselves. In a press release, Huazhe Xu noted that the team wants developers to do more than just watch demo videos; they want them to understand the model's underlying logic through actual API calls. The company's long-term vision is to bring general-purpose robots into households, and RoboAPI is viewed as the first step toward that goal.

The release included two sets of test videos. The first features internal real-world testing, where the robot was deployed across dozens of scenarios within the company. Employees issued random instructions based on their immediate surroundings, such as stacking all the cups, pulling open the bottom drawer, wiping up spilled water with a towel, and pairing white slippers to place them on the left. According to the company, as long as the instructions are clearly articulated, RoboAPI can successfully complete most tasks.

Image | During internal testing, the robotic arm executes the "wipe away spilled water with a towel" instruction (Source: PokeBot)

The second set of evaluations consisted of external joint tests. The company invited several beta developers to integrate their lab's existing robots and everyday objects with RoboAPI, design custom tasks, and submit execution videos. Testing spanned Beijing, Shanghai, Shenzhen, Xi'an, Xiongan, and Singapore, as well as Stanford, Princeton, and Boston in the United States. Tasks included opening an air fryer, loading bread into a toaster, placing an eggplant into an orange basket, and standing a bottle upright.

Figure | During external testing, a robotic arm executes the "stand the bottle upright" command in a beta tester's lab in Shanghai (Source: Poke Robotics).

Poke-Aurora, the underlying foundation supporting RoboAPI, is Poke Robotics' first world-action model. During pre-training, the model simultaneously predicts future visual frames and robotic actions.

In practice, the team found that a world-action model's ability to model the environment does not automatically translate into stable and reliable action execution. To address this, Poke-Aurora introduces a Modulated Residual Coupling (MRC) mechanism: it builds learnable cross-module interaction channels while preserving the original computational pathways of each module.

This mechanism aligns the gradient directions of the video prediction and action prediction branches, directly applying the model's understanding of object motion and physical interactions to action learning. The complete technical details of MRC will be released alongside the Poke-Aurora technical report.

During post-training, the team adopted an approach inspired by UMI to align the model to a "wrist-mounted fisheye camera + end-effector pose control" hardware paradigm. Significant variations in joint layouts, the number of visual sensors, and mounting positions across different robotic arms make models trained on one configuration difficult to transfer to another. Unifying observation and control logic around the end-effector substantially improves cross-embodiment transfer. Paired with consumer-grade action cameras, this also lowers the barrier to setting up the entire system.

Alignment yields two immediate results: on already-supported models, swapping the fisheye camera from an Insta360 Ace Pro 2 to a DJI Osmo Action 6 leaves model performance largely unaffected; the model can also transfer to entirely unseen embodiments with vastly different structures, such as moving from a UR7e to a standard YAM. As long as a robotic arm is equipped with a wrist-mounted action camera and possesses reliable inverse kinematics solvers, it can be integrated with RoboAPI.

The release page lists four hardware configurations and their relative performance benchmarks: a UR7e/UR5e with a Robotiq 2F-85 gripper is set at 100%, a Franka FR3/Panda with a Robotiq 2F-85 gripper at 87%, a standard YAM with a YAM gripper at 76%, and a Tianji floating base with a YAM gripper at 53%.

According to the RoboAPI developer documentation, the current hardware deployment guide supports the UR and Franka series of robot arms. Once the hardware is debugged, developers can issue commands directly from the local console or invoke the model's capabilities within their own applications using the OpenAI Python SDK, JavaScript, or HTTP interfaces.

For evaluation, the company developed a large-scale proprietary benchmark comprising nearly 1,000 instances of common manipulable objects. These were combined to generate approximately 10,000 distinct instructions, categorized into three main task types: object transfer and organization, pose and shape alignment, and environmental interaction and tool use. Evaluated on these unseen objects and tasks, Poke-Aurora achieved an average success rate of 74.2%.

Figure | Success rates of Poke-Aurora and baseline models across the three task categories (Source: Pokebot official website, compiled by 42HOW Robotics)

The company selected three models as baselines: GPT-6 Astra, π0.5, and MolmoAct2. The breakdown is as follows: in object transfer and organization tasks, Poke-Aurora achieved an 86.0% success rate, compared to 71.3% for Astra, 30.9% for π0.5, and 26.9% for MolmoAct2. In environmental interaction tasks, the success rates for the four models were 87.7%, 67.9%, 32.9%, and 23.6%, respectively. Pose and shape alignment was the lowest-scoring category across all models, with Poke-Aurora at 54.1%, Astra at 36.3%, π0.5 at 14.8%, and MolmoAct2 at just 7.9%.

The company also noted that although Poke-Aurora comprehensively outperformed the baselines in this proprietary evaluation, it cannot guarantee that all developers using RoboAPI will reproduce the same results. The purpose of opening RoboAPI is to subject the embodied model's generalization capabilities to testing in external, real-world scenarios, rather than relying solely on internal evaluation by the development team.

The documentation also provides several practical tips. Because this monocular setup lacks a global field of view, grasping an object can easily obstruct the camera and degrade performance. RoboAPI retains historical context and automatically retries after a failed task, so allowing more execution time increases the likelihood of success. Furthermore, English instructions generally yield better results; breaking down complex, lengthy tasks into short atomic steps and using precise adjectives to describe target objects can also improve the success rate.

Regarding safety, the company explained that while RoboAPI's action output speed is relatively fast, the generated commands generally fall within the safe operating envelope of mainstream robot arms. Users are advised to establish safety perimeters around the robot arm in advance and prioritize models equipped with compliant control. Commands that pose a risk of personal injury or environmental damage will be intercepted by the safety gateway. Since RoboAPI can still produce errors, users should report any dangerous robot movements to safety@pokebot.tech.