Universal Manipulation Exoskeleton

James (Jingxi) Xu^{*, 1, 2}, Xinda Qi¹, Yujun Cai¹, Houzhu Ding¹, Luqi Wang¹, Zhixin Sun¹,
Jyh-Herng Chow¹, Ming Yang¹, Mark Cutkosky²

Abstract

For robots to work safely in household environments, they need to be compliant and react to torque and force feedback during contact. However, the majority of existing data collection pipelines still lack the ability to capture force and torque data for learning active compliant policies. In this paper, we present Universal Manipulation Exoskeleton (UME), an upper-limb exoskeleton that provides real-time haptic torque feedback while recording whole-arm configurations and joint torque signals for teleoperation. With transparent torque feedback, human operators can even unsheathe kinematically constrained objects while blindfolded. UME is low-cost, lightweight, and portable. Equipped with an embedded IMU, it enables teleoperation for mobile manipulation. With our proposed universal retargeting algorithm, UME can teleoperate a range of robots, including the 7DoF OpenArm, 7DoF Franka, and 6DoF X-ARM. We demonstrate that this combination of capabilities enables learning bimanual, whole-body, and active compliant policies that operate effectively in highly constrained spaces. The learned robust autonomous policies achieve high success rates across a variety of tasks, including long-horizon mobile manipulation, force-mediated box flipping, visually occluded box pushing, and space-constrained tabletop manipulation.

Video

Task Definition: The robot initially faces the table and checks whether a drink can is present. If no drink is detected, the robot turns around, opens the fridge door, picks up a drink from the side holder of the door, transports it while using the other gripper to support the bottom of the can, and places it on the table. If the drink is removed afterward, the robot repeats the same procedure, ensuring that a drink can is always available on the table. Note that the can is unopened and full, not empty.

Challenges: Whole-body mobile manipulation. Long-horizon.

Task Definition: The robot must reach between two desktops to pick up a GPU card located at the back of the constrained space, and retrieve it to a location outside the enclosure.

Challenges: Highly space-constrained. Force-mediated. Requires whole-body movement.

Task Definition: Push a box against a vertically fixed surface to flip the box upright.

Challenges: Highly force-mediated.

Task Definition: Push a target box into the space between two desktop PCs until it reaches the very end, leaving no gap between the box and any hidden objects behind it. This task is inspired by how humans clean tabletops by shuffling objects all the way against the wall.

Challenges: Visually occluded. Highly force-mediated.

What is the cost of UME?

The entire UME system costs $1900. A detailed cost breakdown can be found in the paper.

What is the in-house-built mobile manipulator?

How many demonstrations did you collect to train the autonomous policies?

Box pushing was trained with 26 demonstrations. Box flipping was trained with 40 demonstrations. GPU picking was trained with 42 demonstrations. Fridge drink retrieval was trained with 157 demonstrations.