PanTact: Teaching Whole-Body Dexterity with a Wearable Contact Interface

PanTact system overview — **PanTact** gives a humanoid whole-body dexterity, learned from robot-free human demonstrations. (a) The data collection interface captures whole-body contacts alongside vision and motion. (b) Contact provides an interface for human-to-robot transfer: the visuomotor policy predicts contact goals, and the whole-body controller realizes them using suitable robot surfaces. (c) PanTact enables diverse whole-body dexterous skills on a humanoid.

Abstract

Humanoid robots have many degrees of freedom and contact surfaces, yet most loco-manipulation research restricts manipulation to the hands and avoids other body contacts with the environment. To extend manipulation beyond the hands, we present PanTact, a system that learns humanoid whole-body dexterity, the capability to manipulate with any suitable body surface, directly from robot-free human demonstrations. Our key idea is to use contact as manipulation-centric information that transfers across embodiments. Contact locations and timing specify the interactions needed to accomplish a task, while the robot chooses how to coordinate its body to realize them within its own physical constraints. Specifically, PanTact captures, learns, and executes whole-body contact via: 1) a wearable, robot-free data collection interface that augments egocentric vision and bimanual UMI grippers with whole-body modular contact patches to record body contact locations and states; 2) a visuomotor policy that mitigates the egocentric observation gap and predicts whole-body contact goals; and 3) contact-aware whole-body control formulations that realize active contact goals using suitable robot surfaces. We deploy PanTact on a Unitree G1 humanoid without additional body contact sensors, demonstrating whole-body dexterity involving body support, pushing, sweeping, and load-sharing.

Wearable Whole-Body Contact Interface

Egocentric-augmented bimanual UMI grippers plus modular contact patches on the forearms and knees record where, when, and how hard the demonstrator's body contacts the environment.

Robot-free demonstration. Head and wrist camera views recorded alongside live readings from the forearm and knee contact patches, captured in sync while the demonstrator performs the task.

PanTact modular contact patch — **Modular contact patch.** Sensor, connector, and strap modules snap together into patches of arbitrary shape and extent. We showcase arm and knee patches.

Tile-like modules. Each 44×44 mm module carries a single force-sensitive resistor (FSR). A custom PCB streams up to 16 FSR channels at 1 kHz; modules are daisy-chained over an RS-485 bus.
Magnetic tiling. Press-fit connectors reinforced by magnets let patches of any shape be assembled by snapping modules together. Strap connectors secure the patch to the body, and tracker connectors mount an HTC Vive tracker to track the patch pose.
Contact detection & localization. Contact is binarized against a per-channel adaptive baseline to handle FSR drift, and each sensor's 3D position follows from a rigid transform of its patch tracker.
Vision & pose. A head-mounted stereo camera and two fisheye wrist cameras at 60 fps, Vive trackers on the head and wrists, and gripper widths, all logged on a shared clock.

Whole-Body Contact Policy

A contact-predictive policy maps embodiment-agnostic observations to EEF trajectories and whole-body contact goals.

PanTact cross-embodiment contact interface — **Cross-embodiment contact interface.** The contact-predictive policy uses body-masked 3D egocentric observations, wrist images, and proprioception to predict EEF trajectories and contact goals (position and binary state). The contact-aware whole-body controller coordinates whole-body motion to track EEF commands and realize active contact goals using suitable robot surfaces.

Geometric body masking. Each gripper frame and contact patch defines a body-facing half-space; their intersection approximates the demonstrator's body, which is masked in the attention layers of the egocentric pointmap encoder. No appearance model of either embodiment is needed, and it scales to any number of patches.
Contact-aware action space. Each action step contains a head look-at point, EEF poses and gripper widths, and per-sensor contact goals, each a binary contact state and a 3D contact position.
Active vs. inactive goals. Only active goals request the robot to make contact; inactive goals do not force the robot to reproduce the human's free-space body motion, leaving the controller free to choose a robot-suited configuration.

Contact-Aware Whole-Body Control

The controller realizes active contact goals with any suitable robot surface, rather than assigning each contact to a fixed link.

Any-surface contact objective. The distance from a commanded contact to the robot is a soft-min over all candidate bodies, so whichever surface can best reach the contact satisfies it:

Virtual contact targets. Active contact targets are offset into the object along the pressing direction, so the residual tracking error produces a pressing force at contact.

Three Controller Formulations

Upper-Body IK. Differential IK QP jointly optimizing EEF and contact tasks. Requires no training data or simulation; limited to upper-body tasks with fixed legs.
WBIK + SONIC. Contact-aware whole-body IK whose dense reference is executed by the existing SONIC motion tracker. Enables lower-body contacts and locomotion.
Sparse Contact Tracker. RL goal-conditioned controller that directly tracks sparse EEF and contact goals, trained in simulation with hindsight contact labeling.

**Controller comparison.** Replaying demonstrations in simulation with and without contact goals, adding contact goals reduces contact surface distance for every controller at little cost to EEF tracking. Upper-Body IK achieves the lowest upper-body EEF error; for whole-body tasks, Sparse Contact Tracker reaches lower contact distance than WBIK+SONIC but higher EEF error.

Sweep-and-Stow

The robot sweeps tall objects from a counter into an open bin in a single stroke, then uses a lower-body surface to push the cabinet closed. Ours (WBIK + SONIC) reaches 90.63% end-to-end success; Humanoid UMI++ 0%.

Ours · WBIK + SONIC

Third-person view (left) and synchronized robot egocentric view (right)

Seen object, different layout

Unseen object

Two seen objects

Two unseen objects

Baseline · Humanoid UMI++

One object

Two objects

Pose-based baseline. Without explicit arm-contact objectives, the gripper passes above objects or knocks them outside the bin, and the robot fails to advance a leg to close the cabinet.

Sweep-and-stow results — (a) Our policy rollout with WBIK+SONIC, demonstrating arm sweeping and lower-body pushing, locomotion, and balance at the same time. (b) Test configurations with seen and unseen object combinations and varied cabinet openings. (c) Humanoid UMI++ failures: the gripper passes above an object or knocks it outside the bin, and the robot fails to advance a leg as its arms approach joint limits. (d) Stage-wise and end-to-end success rates.

Grasp-and-Embrace

The robot grasps a small object with the gripper, then embraces a bulky object with its arms and moves both simultaneously. Ours (Upper-Body IK) reaches 85% success; Humanoid UMI 0%.

Ours · Upper-Body IK

Rollout 1

Rollout 2

Baseline · Humanoid UMI

EEF-only baseline. Without arm-contact objectives, the arms never press on the bulky object, so it cannot be supported and moved.

Grasp-and-embrace results — (a) Autonomous rollout with Upper-Body IK, supporting a bulky object with the arms while grasping a small object with the gripper. (b) Overlaid test configurations with varied object placements and initial robot postures. (c) Humanoid UMI's typical failure: insufficient arm contact to support the bulky object. (d) Success rates. (e) Real-world tracking plots.

Carry-and-Push

The robot carries a box in both arms, walks towards a larger box on the floor, and pushes it with its shins and knees. Ours (WBIK + SONIC) reaches 90% success; Humanoid UMI++ 40%.

Ours · WBIK + SONIC

Rollout 1

Rollout 2 · box on right

Ours · Sparse Contact Tracker

Baseline · Humanoid UMI++

Carry-and-push results — (a) Rollout demonstrating arm load-sharing while walking and pushing with the lower body. (b) Test configurations with varied box placements and initial robot postures. (c) Humanoid UMI++ drops the carried box due to insufficient contact support. (d) Success rates. (e) Real-world tracking: Humanoid UMI++ continuously tracks knee poses with large error, while ours tracks leg contact goals accurately during active contact phases.

Citation

@misc{pantact,
  author       = {Xiaomeng Xu and Han Zhang and Zhian Ruan and Xiaofeng Guo
                  and Jinkun Cao and Ruihan Yang and Carmelo Sferrazza
                  and Shuran Song and Guanya Shi},
  title        = {PanTact: Teaching Whole-Body Dexterity with a Wearable Contact Interface},
  howpublished = {arXiv preprint},
  year         = {2026},
}