The glove captures what the hand does. The model shows the hand without the glove.
Glove2Hand (arXiv:2603.20850): sensing gloves equipped with tactile sensors and IMUs record hand motions and contact forces during object manipulation. The glove is excellent for sensing (pressure distribution, joint angles, finger dynamics) but terrible for vision — it occludes the hand, changes the hand's appearance, and breaks any visual model trained on bare hands.
Note (2026-11-09): I checked the abstract of the Glove2Hand paper, which has been revised since its first version. It says HandSense improves video-based contact estimation and hand tracking under severe occlusion; the essay says contact force estimation, and its closing claim that the synthesized hand is more useful than the real one would be is not in the abstract. The abstract also describes HandSense as the first multi-modal dataset of this kind.
The system synthesizes photorealistic bare-hand videos from glove recordings. A 3D Gaussian hand model maintains temporal consistency across frames — the hand doesn't flicker or deform unrealistically. A diffusion-based restoration technique integrates the synthesized bare hand into the scene, handling the complex interactions between fingers and objects (contact deformations, partial occlusion, shadow casting).
HandSense dataset: synchronized glove-to-hand video pairs with tactile and IMU data. The synthesized videos improve downstream tasks: contact force estimation and hand tracking under occlusion both benefit from having bare-hand visual data aligned with sensor recordings.
The structural insight: the glove creates a modality gap. Sensors measure dynamics (forces, angles) but destroy visual appearance. Cameras measure appearance but can't sense forces. The synthesis bridges the gap — converting sensor-space recordings into camera-space output — without requiring the actual bare hand to be present. You get the sensing of the glove and the appearance of the bare hand. The real hand is never recorded; the synthesized hand is more useful than the real one would be, because it comes with force data.