Motion Segmentation of RGB-D Videos via Trajectory Clustering

The goal in this project is to segment moving objects in a scene recorded by a RGB-D camera. Given a sequence of color and depth images, keypoints are tracked over time and labeled according to which moving object they belong to. The project was conducted as a Master thesis at Volumental AB and CVAP (KTH) in collaboration with the Smart Mobility Lab. Scenes from model trucks in the Smart Mobility Lab are used for testing the capabilities of the developed motion segmentation algorithm.

 

 

Motion segmentation of a video can be useful for the 3-d reconstruction of objects. Motion segmentation can be seen as image segmentation with time as an additional dimension. In this project, an approach is used where keypoints are tracked in the scene over time. The trajectories of these keypoints describe the local motion of individual landmarks in the scene. A similarity graph describes to which degree point pairs move similarly. Ultimately, spectral clustering is employed to partition the similarity graph, and thereby determine clusters of points that move similarly in the dynamic scene. These clusters are expected to correspond to moving objects.

Several dynamic scenes recorded in the Smart Mobility Lab are used to test the implemented motion segmentation algorithm. These scenes feature either one or two semi-trailer trucks recorded by a handheld, not necessarily static camera. The recordings provide various challenges, amongst them: (1) the semi-trailer trucks are articulated objects composed of two rigid parts, (2) the handheld camera is always shaking, (3) the background does not provide highly distinctive features for the keypoint tracker.

The implemented algorithm turned out to be not sophisticated enough for segmenting the scenes with two trucks and severe occlusion. This shows also that the scenes recorded in the Smart Mobility Lab are of sufficient difficulty to push the motion segmentation algorithm to its limits. In contrast, the method is able to segment videos of a single moving truck, even in the presence of a moving handheld camera. The videos below show the segmentations of videos with a single moving semi-trailer truck:

Julius Adorf

 

Your web browser can not show this movie

Your browser has javascript disabled.

You can not se movies inside the editor

To see the movie, use the page preview.

 

Your web browser can not show this movie

Your browser has javascript disabled.

You can not se movies inside the editor

To see the movie, use the page preview.

Top page top