Research

Frontal Face Synthesis from a Single Image

Face pose and illumination estimation is an important pre-processing step in many face analysis problems. we present a new method to estimate the face pose and illumination direction from one single image. The basic idea is to compare the reconstruction residuals between the input image and a small set of reference images under different poses and illumination directions. Based on the estimated pose and illumination direction, we develop a face synthesis framework to rectify the input image to the frontal view under standard illumination. Experiments show that our estimation method is both fast (less than one second per frame) and accurate (even less than three degrees) and our face synthesis method can generate visually plausible results, in particular for challenging inputs with large pose changes and poor lighting conditions. The synthesized frontal face views increase the face recognition rate significantly from 1:5% to 62:1%.

View synthesis Results.
(First row) Input; (second row) synthesized frontal-view;
(third row) the ground truth.

People:
Ruigang Yang, Xinyu Huang, Xianwang Wang, Jizhou Gao

Related Publication
Biometrics workshop of CVPR 2008

Sponsors
NSF, DHS