RobustMatting : The robust tool for natural image segmentation and matting

Jue Wang

2006-07

 

1. Introduction

RobustMatting is a Windows program for extracting high quality mattes from natural images. Once a matte is extracted, the foreground object can be seamlessly composed onto a new background. An example is shown below.

From left to right: Original image, user roughly specified trimap, high quality matte the program extracted, new composite with a better background.

If the user provides a more accurate trimap, the program is capable of extracting a more accureate matte, as shown below. Alternatively, the program can also work with a few paint strokes the user specified. For this example the matte generated by strokes has slightly lower quality, but it makes sense since the input is very sparse.

The program also has the (limited) capability to work on a video sequence. As shown below, given an initial trimap for the frist frame of a simple video sequence, the progroam can automatically extract mattes over 40 frames by using optical flow for trimap propagation.

Compositional matting is supported in this program as an efficient tool to compose the foreground onto a new background image which is similar to the original one. As an interesting application, this technique can be used to enlarge a foreground object and re-compose it back to the original image, as shown below. More details can be found in this technical report.

 

A screenshot of the program is here:

A few notes you should read before downloading:

1. Keeping it in mind that the progam might not be stable. I am the only person maintaining the program and I am not writing it as a commercial software. It is only a TEST program, mainly for research purposes, and hasn't been tested thoroughly. So it may crash due to uncommon inputs and parameters.

2. The system is computationally expensive, so you don't want to try it on your laptop unless it is a powerful one.

3. If you find any bugs that you think should be fixed, please drop me an email at "juewang at adobe dot com".

4. If you have particularly interesting images or data set please share with me. Also, I may be able to adapt my algorithm for specific types of images in your applications.

5. Source code may be available for commercial purposes, we can talk.

6. This program is based on my unpublished research work so you won't be able to find out technical details on this webpage at this stage.

2. Download

The program has been commercialized by Digital Film Tools under the name of EZ Mask. However, if you want to try a free version for purely research purposes, please email me.

3. Footprints

Oct. 06, 2006

The product is released by digital film tools.

Meanwhile, the research version is upgraded to V1.45 for faster processing and other new features.

June 04, 2006 Version 1.4 is released. New features are:

1).Compositional matting and intelligent zooming are supported.

2). Switched from Gdi+ to OpenCV for more efficient image manipulation.

3). Fixed some bugs.

May 19, 2006 Version 1.25 is released. Add a new trimap propagation method for video matting.
May 16, 2006

Version 1.2 is released. New features:

1). Video Matting is supported. Look at the details in the manual, and download the new test data for testing.

2). Another 20% improvement on running time.

May 11, 2006 Version 1.1 is released: Optimized two key functions, the program is 30% faster.
May 09, 2006 Webpage is created.
May 09, 2006 Version 1.0 is released.