The camera can't see the edge. The finger can feel it.
Broken transparent fragments — glass, acrylic, crystal — defeat standard computer vision. Transparent edges don't generate the contrast that edge detectors rely on. Refraction and reflection create false contours. The visual information is unreliable precisely where precision matters most: at the fragment boundaries needed for reassembly.
Note (2026-11-13; read against the paper's full text): TransFrag27K is described in the paper as comprising over 27,000 annotated instances, not 27,000 images. The paper's framework has more parts than the essay gives: a segmentation network (TransFragNet) that finds the grasp position, a network that segments the contact region in the GelSight Mini images, a visual-tactile fusion classifier of the fragment's material, a contour estimation step, and a contour matching and reassembly algorithm; it reports contour alignment quality of 93.8 percent and 96.7 percent in real-world trials and a mixed reassembly task with 20 fragments of five materials. The paper trains the tactile contact-segmentation network and the tactile branch of the classifier; I did not find in it the essay's statement that the tactile part needs no training on transparent objects.
Visual-tactile fusion (arXiv:2603.20290): a robot grasps each fragment with a Gelsight Mini tactile sensor mounted on the gripper. The sensor deforms against the fragment edge and records the contact geometry. Combined with visual estimates (which are noisy but provide global context), the tactile signal corrects the edge estimate at the points of contact.
The workflow mimics human perception. When you try to piece together a broken glass, you don't rely on vision alone — you hold the fragments, feel the edges, rotate them to test fit. The camera provides the layout; the finger provides the geometry. TransFrag27K (27,000 synthetic images of fragmented transparent objects) trains the visual component. The tactile component doesn't need training on transparent objects specifically — it measures physical geometry, which doesn't depend on optical properties.
The structural point: multimodal sensing is not redundancy. It's complementarity. The camera and the finger don't measure the same thing with different precision — they measure different things entirely. The camera measures optical appearance (unreliable for transparent objects). The finger measures physical geometry (reliable regardless of material). The fusion works because the failure modes are orthogonal.