Expression Flow for 3D-Aware Face Component Transfer


ACM SIGGRAPH 2011

Fei Yang1  Jue Wang2    Eli Shechtman2    Lubomir Bourdev2   Dimitris Metaxas1     

1Rutgers University

2Adobe Systems


Abstract

We address the problem of correcting an undesirable expression on a face photo by transferring local facial components, such as a smiling mouth, from another face photo of the same person which has the desired expression. Direct copying and blending using existing compositing tools results in semantically unnatural composites, since expression is a global effect and the local component in one expression is often incompatible with the shape and other components of the face in another expression. To solve this problem we present Expression Flow, a 2D flow field which can warp the target face globally in a natural way, so that the warped face is compatible with the new facial component to be copied over. To do this, starting with the two input face photos, we jointly construct a pair of 3D face shapes with the same identity but different expressions. The expression flow is computed by projecting the difference between the two 3D shapes back to 2D. It describes how to warp the target face photo to match the expression of the reference photo. User studies suggest that our system is able to generate face composites with much higher fidelity than existing methods.

Citation

@ARTICLE{Yang_siggraph11_expresisonflow,
author = {Fei Yang and Jue Wang and Eli Shechtman and Lubomir Bourdev and Dimitri Metaxas },
title = {Expression Flow for 3D-Aware Face Component Transfer },
booktitle = {ACM Transactions on Graphics (Proc. SIGGRAPH)},
year = {2011},
}

Materials

Paper: Un-offical version with more compelling figures (PDF, 5.9MB) .   Offical ACM version: (PDF, 4.6MB)

Supplemental Material (PDF, 10.2MB)