Track Deformable Objects from Point Clouds with Structure Preserved Registration

Welcome! This website supplements our IJRR submission, in which we present a novel state estimator for tracking deformable objects.

Manipulating deformable objects is a challenging but important problem for robots. To enhance the manipulation performance, this paper proposes a real-time marker-less state estimator to track deformable objects by stereo cameras. A novel registration algorithm is developed to align the object model towards the measured point clouds. Its performance outperforms other state-of-the-art non-rigid registration methods. A closed-loop framework which combines the probabilistic-based registration and dynamic-based simulation is provided which improve tracking robustness under massive occlusion. Experiments on object tracking and visual manipulation are performed to validate the effectiveness of the proposed method.

Track 1D deformable objects (flexible rope)

Comparison between tracking with CPD (before) and tracking with SPR (after)

Real-time tracking

Track 2D deformable objects (clothes)

Applications on Robot Manipulation

1. Manipulate the flexible rope to any sketched shapes

2. Automation and failure detection for robotic wire-harness-assembly