Back to Jue Wang's project page

Video SnapCut: Robust Video Object Cutout Using Localized Classifiers

ACM SIGGRAPH 2009



Team

Xue Bai*                (University of Minnesota)
Jue Wang              (Adobe Systems)
David Simons       (Adobe Systems)
Guillermo Sapiro  (University of Minnesota)

*Xue did most of the work when he was an intern at Adobe.

Abstract

A variety of algorithms and systems have been recently proposed for interactive object cutout in images and video. Although tremendous success has been achieved in still images, accurately extracting dynamic objects in video remains a very challenging problem. Previous video cutout systems present two major limitations which prevent them from being applied in real production: (1) reliance on global statistics, thus lacking the ability to deal with complex and diverse scenes; and (2) treating segmentation as a global optimization, thus lacking a complete set of user controls and a practical workflow that can guarantee the convergence of the systems to the desired results.

In this paper we present Video SnapCut, a robust video object cutout system that significantly advances the state-of-the-art. In the center of the system lies a new localized segmentation paradigm, where segmentation is achieved by the collaboration of a set of local classifiers, each classifier adaptively integrating multiple local features such as color, edge, and on-line learned shape prior. By localizing the classifiers, our system achieves significantly better results than previous systems for complicated videos, including dynamic background and non-rigid foreground deformations. We further show how this segmentation paradigm naturally supports local user editing and propagates them across time. The object cutout system is completed with a novel coherent video matting technique. A comprehensive comparison with previous techniques is presented, demonstrating the effectiveness of the proposed system at achieving high quality results, as well as the robustness of the system against various types of inputs.

Paper



                                                    PDF (high res, 22M)       PDF(low res, 2M)

Video


                                                   Downloadable : SnapCut.mp4 (87 M)

             
                                     (This video contains audio, check your browser settings if no sound)

Productization

Update: 4/12/2010

Video SnapCut is all grown up! It has been transfered into the all-new feature "Roto Brush" released in Adobe After Effects CS5!


                  roto brush screenshot