FACE RECOGNITION HOMEPAGE



Algorithms

 

Image-Based
Face Recognition Algorithms
Video-Based
Face Recognition Algorithms

Click here

 

Image-Based Face Recognition Algorithms

PCA|ICA|LDA|EP|EBGM|Kernel Methods|Trace Transform
AAM|3-D Morphable Model|3-D Face Recognition
Bayesian Framework|SVM|HMM|Boosting & Ensemble
Algorithms Comparisons

PCA

Derived from Karhunen-Loeve's transformation. Given an s-dimensional vector representation of each face in a training set of images, Principal Component Analysis (PCA) tends to find a t-dimensional subspace whose basis vectors correspond to the maximum variance direction in the original image space. This new subspace is normally lower dimensional (t<<s). If the image elements are considered as random variables, the PCA basis vectors are defined as eigenvectors of the scatter matrix.

Read more:

M. Turk, A. Pentland, Eigenfaces for Recognition, Journal of Cognitive Neurosicence, Vol. 3, No. 1, 1991, pp. 71-86
download here, 10.6 MB

M.A. Turk, A.P. Pentland, Face Recognition Using Eigenfaces, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3-6 June 1991, Maui, Hawaii, USA, pp. 586-591
link

A. Pentland, B. Moghaddam, T. Starner, View-Based and Modular Eigenspaces for Face Recognition, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 21-23 June 1994, Seattle, Washington, USA, pp. 84-91
link

H. Moon, P.J. Phillips, Computational and Performance aspects of PCA-based Face Recognition Algorithms, Perception, Vol. 30, 2001, pp. 303-321
download here, 1.61 MB

back to top

 

ICA

Independent Component Analysis (ICA) minimizes both second-order and higher-order dependencies in the input data and attempts to find the basis along which the data (when projected onto them) are - statistically independent . Bartlett et al. provided two architectures of ICA for face recognition task: Architecture I - statistically independent basis images, and Architecture II - factorial code representation.

Read more:

M.S. Bartlett, J.R. Movellan, T.J. Sejnowski, Face Recognition by Independent Component Analysis, IEEE Trans. on Neural Networks, Vol. 13, No. 6, November 2002, pp. 1450-1464
link | source code

C. Liu, H. Wechsler, Comparative Assessment of Independent Component Analysis (ICA) for Face Recognition, Proc. of the Second International Conference on Audio- and Video-based Biometric Person Authentication, AVBPA'99, 22-24 March 1999, Washington D.C., USA, pp. 211-216
download here, 158 kB

back to top

 

LDA

Linear Discriminant Analysis (LDA) finds the vectors in the underlying space that best discriminate among classes. For all samples of all classes the between-class scatter matrix SB and the within-class scatter matrix SW are defined. The goal is to maximize SB while minimizing SW, in other words, maximize the ratio det|SB|/det|SW| . This ratio is maximized when the column vectors of the projection matrix are the eigenvectors of (SW^-1 SB).

Read more:

K. Etemad, R. Chellappa, Discriminant Analysis for Recognition of Human Face Images, Journal of the Optical Society of America A, Vol. 14, No. 8, August 1997, pp. 1724-1733
download here, 4.04 MB

P.N. Belhumeur, J.P. Hespanha, D.J. Kriegman, Eigenfaces vs. Fisherfaces: Recognition using Class Specific Linear Projection, Proc. of the 4th European Conference on Computer Vision, ECCV'96, 15-18 April 1996, Cambridge, UK, pp. 45-58
download here, 662 kB

W. Zhao, R. Chellappa, A. Krishnaswamy, Discriminant Analysis of Principal Components for Face Recognition, Proc. of the 3rd IEEE International Conference on Face and Gesture Recognition, FG'98, 14-16 April 1998, Nara, Japan, pp. 336-341
link

A.M. Martinez, A.C. Kak, PCA versus LDA, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 23, No. 2, 2001, pp. 228-233
link

W. Zhao, A. Krishnaswamy, R. Chellappa, D.L. Swets, J. Weng, Discriminant Analysis of Principal Components for Face Recognition, Face Recognition: From Theory to Applications, H. Wechsler, P.J. Phillips, V. Bruce, F.F. Soulie, and T.S. Huang, eds., Springer-Verlag, Berlin, 1998, pp. 73-85
download here, 338 kB

J. Lu, K.N. Plataniotis, A.N. Venetsanopoulos, Face Recognition Using LDA-Based Algorithms, IEEE Trans. on Neural Networks, Vol. 14, No. 1, January 2003, pp. 195-200
link | source code, 652 kB

back to top

 

EP

Aa eigenspace-based adaptive approach that searches for the best set of projection axes in order to maximize a fitness function, measuring at the same time the classification accuracy and generalization ability of the system. Because the dimension of the solution space of this problem is too big, it is solved using a specific kind of genetic algorithm called Evolutionary Pursuit (EP).

Read more:

C. Liu, H. Wechsler, Evolutionary Pursuit and Its Application to Face Recognition, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 22, No. 6, June 2000, pp. 570-582
link

C. Liu, H. Wechsler, Face Recognition Using Evolutionary Pursuit, Proc. of the Fifth European Conference on Computer Vision, ECCV'98, Vol II, 02-06 June 1998, Freiburg, Germany, pp. 596-612
download here, 785 kB

back to top

 

EBGM

Elastic Bunch Graph Matching (EBGM). All human faces share a similar topological structure. Faces are represented as graphs, with nodes positioned at fiducial points. (exes, nose...) and edges labeled with 2-D distance vectors. Each node contains a set of 40 complex Gabor wavelet coefficients at different scales and orientations (phase, amplitude). They are called "jets". Recognition is based on labeled graphs. A labeled graph is a set of nodes connected by edges, nodes are labeled with jets, edges are labeled with distances.

Read more:

L. Wiskott, J.-M. Fellous, N. Krueuger, C. von der Malsburg, Face Recognition by Elastic Bunch Graph Matching, Chapter 11 in Intelligent Biometric Techniques in Fingerprint and Face Recognition, eds. L.C. Jain et al., CRC Press, 1999, pp. 355-396
download here, 735 kB

L. Wiskott, J.-M. Fellous, N. Krueuger, C. von der Malsburg, Face Recognition by Elastic Bunch Graph Matching, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 19, No. 7, 1997, pp. 775-779
link

back to top

 

Kernel Methods

The face manifold in subspace need not be linear. Kernel methods are a generalization of linear methods. Direct non-linear manifold schemes are explored to learn this non-linear manifold.

Read more:

M.-H. Yang, Kernel Eigenfaces vs. Kernel Fisherfaces: Face Recognition Using Kernel Methods, Proc. of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition, 20-21 May 2002, Washington D.C., USA, pp. 215-220
link

F.R. Bach, M.I. Jordan, Kernel Independent Component Analysis, Journal of Machine Learning Research, Vol. 3, 2002, pp. 1-48
download here, 482 kB

B. Scholkopf, A. Smola, K.-R. Muller, Nonlinear Component Analysis as a Kernel Eigenvalue Problem, Technical Report No. 44, December 1996, 18 pages
download here, 422 kB

M.-H. Yang, Face Recognition Using Kernel Methods, Advances in Neural Information Processing Systems, T. Diederich, S. Becker, Z. Ghahramani, Eds., 2002, vol. 14, 8 pages
download here, 265 kB

S. Zhou, R. Chellappa, B. Moghaddam, Intra-personal kernel space for face recognition, Proc. of the 6th International Conference on Automatic Face and Gesture Recognition, FGR2004, 17-19 May 2004, Seoul, Korea, pp. 235-240
download here, 104 kB

S. Zhou, R. Chellappa, Multiple-exemplar discriminant analysis for face recognition, Proc. of the 17th International Conference on Pattern Recognition, ICPR'04, 23-26 August 2004, Cambridge, UK, pp. 191-194
download here, 101 kB

J. Lu, K.N. Plataniotis, A.N. Venetsanopoulos, Face Recognition Using Kernel Direct Discriminant Analysis Algorithms, IEEE Trans. on Neural Networks, Vol. 14, No. 1, January 2003, pp. 117-126
link | source code, 749 kB

back to top

 

Trace Transform

The Trace transform, a generalization of the Radon transform, is a new tool for image processing which can be used for recognizing objects under transformations, e.g. rotation, translation and scaling. To produce the Trace transform one computes a functional along tracing lines of an image. Different Trace transforms can be produced from an image using different trace functionals.

source code, 149 kB

Read more:

A. Kadyrov, M. Petrou, The Trace Transform and Its Applications, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 23, No. 8, August 2001, pp. 811-828
link

S. Srisuk, M. Petrou, W. Kurutach and A. Kadyrov, Face Authentication using the Trace Transform, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'03), 16-22 June 2003, Madison, Wisconsin, USA, pp. 305-312
link

S. Srisuk and W. Kurutach, Face Recognition using a New Texture Representation of Face Images, Proceedings of Electrical Engineering Conference, Cha-am, Thailand, 06-07 November 2003, pp. 1097-1102
download here, 1.19 MB

back to top

 

AAM

An Active Appearance Model (AAM) is an integrated statistical model which combines a model of shape variation with a model of the appearance variations in a shape-normalized frame. An AAM contains a statistical model if the shape and gray-level appearance of the object of interest which can generalize to almost any valid example. Matching to an image involves finding model parameters which minimize the difference between the image and a synthesized model example projected into the image.

Read more:

T.F. Cootes, C.J. Taylor, Statistical Models of Appearance for Computer Vision, Technical Report, University of Manchester, 125 pages
download here, 1.28 MB

T.F. Cootes, K. Walker, C.J. Taylor, View-Based Active Appearance Models, Proc. of the IEEE International Conference on Automatic Face and Gesture Recognition, 26-30 March 2000, Grenoble, France, pp. 227-232
link

back to top

 

3-D Morphable Model

Human face is a surface lying in the 3-D space intrinsically. Therefore the 3-D model should be better for representing faces, especially to handle facial variations, such as pose, illumination etc. Blantz et al. proposed a method based on a 3-D morphable face model that encodes shape and texture in terms of model parameters, and algorithm that recovers these parameters from a single image of a face.

Read more:

J. Huang, B. Heisele, V. Blanz, Component-based Face Recognition with 3D Morphable Models, Proc. of the 4th International Conference on Audio- and Video-Based Biometric Person Authentication, AVBPA 2003, 09-11 June 2003, Guildford, UK, pp. 27-34
download here, 934 kB

V. Blanz, T. Vetter, A Morphable Model for the Synthesis of 3D Faces, Proc. of the SIGGRAPH'99, 08-13 August 1999, Los Angeles, USA, pp. 187-194
download here, 2.88 MB

V. Blanz, T. Vetter, Face Recognition Based on Fitting a 3D Morphable Model, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 25, No. 9, September 2003, pp. 1063-1074
link

B. Moghaddam, J.H. Lee, H. Pfister, R. Machiraju, Model-Based 3D Face Capture with Shape-from-Silhouettes, Proc. of the IEEE International Workshop on Analysis and Modeling of Faces and Gestures, AMFG, 17 October 2003, Nice, France, pp. 20-27
link

J. Lee, B. Moghaddam, H. Pfister, R. Machiraju, Finding Optimal Views for 3D Face Shape Modeling, Proc. of the International Conference on Automatic Face and Gesture Recognition, FGR2004, 17-19 May 2004, Seoul, Korea, pp. 31-36
download here, 1.32 MB

 

back to top

 

3-D Face Recognition

The main novelty of this approach is the ability to compare surfaces independent of natural deformations resulting from facial expressions. First, the range image and the texture of the face are acquired. Next, the range image is preprocessed by removing certain parts such as hair, which can complicate the recognition process. Finally, a canonical form of the facial surface is computed. Such a representation is insensitive to head orientations and facial expressions, thus significantly simplifying the recognition procedure. The recognition itself is performed on the canonical surfaces.

Read more:

A. Bronstein, M. Bronstein, R. Kimmel, and A. Spira. 3D face recognition without facial surface reconstruction, in Proceedings of ECCV 2004, Prague, Czech Republic, May 11-14, 2004
download here, 287 kB

A. Bronstein, M. Bronstein, and R. Kimmel, Expression-invariant 3D face recognition, Proc. Audio & Video-based Biometric Person Authentication (AVBPA), Lecture Notes in Comp. Science 2688, Springer, 2003, pp. 62-69
download here, 448 kB

More publications

back to top

 

Bayesian Framework

A probabilistic similarity measure based on Bayesian belief that the image intensity differences are characteristic of typical variations in appearance of an individual. Two classes of facial image variations are defined: intrapersonal variations and extrapersonal variations. Similarity among faces is measures using Bayesian rule.

Read more:

B. Moghaddam, T. Jebara, A. Pentland, Bayesian Face Recognition, Pattern Recognition, Vol. 33, Issue 11, November 2000, pp. 1771-1782
download here, 952 kB

C. Liu, H. Wechsler, A Unified Bayesian Framework for Face Recognition, Proc. of the 1998 IEEE International Conference on Image Processing, ICIP'98, 4-7 October 1998, Chicago, Illinois, USA, pp. 151-155
link

B. Moghaddam, C. Nastar, A. Pentland, A Bayesian Similarity Measure for Deformable Image Matching, Image and Vision Computing, Vol. 19, Issue 5, May 2001, pp. 235-244
download here, 1.19 MB

back to top

 

SVM

Given a set of points belonging to two classes, a Support Vector Machine (SVM) finds the hyperplane that separates the largest possible fraction of points of the same class on the same side, while maximizing the distance from either class to the hyperplane. PCA is first used to extract features of face images and then discrimination functions between each pair of images are learned by SVMs.

Read more:

G. Guo, S.Z. Li, K. Chan, Face Recognition by Support Vector Machines, Proc. of the IEEE International Conference on Automatic Face and Gesture Recognition, 26-30 March 2000, Grenoble, France, pp. 196-201
link

B. Heisele, P. Ho, T. Poggio, Face Recognition with Support Vector Machines: Global versus Component-based Approach, Proc. of the Eighth IEEE International Conference on Computer Vision, ICCV 2001, Vol. 2, 09-12 July 2001, Vancouver, Canada, pp. 688-694
link

K. Jonsson, J. Matas, J. Kittler, Y.P. Li, Learning Support Vectors for Face Verification and Recognition, Proc. of the IEEE International Conference on Automatic Face and Gesture Recognition, 26-30 March 2000, Grenoble, France, pp. 208-213
link

back to top

 

HMM

Hidden Markov Models (HMM) are a set of statistical models used to characterize the statistical properties of a signal. HMM consists of two interrelated processes: (1) an underlying, unobservable Markov chain with a finite number of states, a state transition probability matrix and an initial state probability distribution and (2) a set of probability density functions associated with each state.

Read more:

A.V. Nefian, M.H. Hayes III, Hidden Markov Models for Face Recognition, Proc. of the IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP'98, Vol. 5, 12-15 May 1998, Seattle, Washington, USA, pp. 2721-2724
link

A.V. Nefian, M.H. Hayes, Maximum likelihood training of the embedded HMM for face detection and recognition, Proc. of the IEEE International Conference on Image Processing, ICIP 2000, Vol. 1, 10-13 September 2000, Vancouver, BC, Canada, pp. 33-36
link

A.V. Nefian, Embedded Bayesian networks for face recognition, Proc. of the IEEE International Conference on Multimedia and Expo, Vol. 2, 26-29 August 2002, Lusanne, Switzerland, pp. 133-136
link

back to top

 

Boosting & Ensemble Solutions

The idea behind Boosting is to sequentially employ a weak learner on a weighted version of a given training sample set to generalize a set of classifiers of its kind. Although any individual classifier may perform slightly better than random guessing, the formed ensemble can provide a very accurate (strong) classifier. Viola and Jones build the first real-time face detection system by using AdaBoost, which is considered a dramatic breakthrough in the face detection research. On the other hand, papers by Guo et al. are the first approaches on face recogntion using the AdaBoost methods.

Read more:

Y. Freund, R.E. Schapire, A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting, Journal of Computer and System Sciences, Vol. 55, No. 1, 1997, pp. 119-139
download here, 642 kB

R. Meir, G. Raetsch. An Introduction to Boosting and Leveraging, In S. Mendelson and A. Smola, Editors, Advanced Lectures on Machine Learning, LNAI 2600, pp. 118-183, Springer, 2003
download here, 902 kB

P. Viola, M.J. Jones, Robust Real-Time Face Detection, International Journal of Computer Vision, Vol. 57, No. 2, May 2004, pp. 137-154
download here, 328 kB

J. Lu, K.N. Plataniotis, A.N. Venetsanopoulos, S.Z. Li, Ensemble-based Discriminant Learning with Boosting for Face Recognition, IEEE Transactions on Neural Networks, Vol. 17, No. 1, January 2006, pp. 166-178
link | source code, 766 kB

G.-D. Guo, H.-J. Zhang, S.Z. Li, Pairwise Face Recognition, Proc. of the Eighth IEEE International Conference on Computer Vision, ICCV 2001, Vol. 2, 09-12 July 2001, Vancouver, Canada, pp. 282-287
link

G.-D. Guo, H.-J. Zhang, Boosting for Fast Face Recognition, Second International Workshop on Recognition, Analysis and Tracking of Faces and Gestures in Real-time Systems, RATFG-RTS'01, in conjunction with ICCV 2001, 13 July 2001, Vancouver, Canada, pp. 96-100
download here, 130 kB

back to top

 

Algorithms Comparisons

Experimental design is an important (yet often neglected) part of face recognition research. Reporting detailed experimental setup parameters like:

  1. identification/verification performance,
  2. database/protocol used,
  3. number of images/classes in the training, gallery and probe set,
  4. possible overlap of images in gallery and training set,
  5. statistical significance of the reported improvements (preferably with hypothesis testing as well), etc.,

will make papers more readable and will make it possible for other researchers to easily evaluate reported results. Also, results will become independently reproducible.

Here are some papers that will help you design and conduct your experiments, and subsequently, improve the quality or your papers as they will be put on solid scientific basis.

Face Recognition Vendor Test

Evaluation of Face Recognition Algorithms

Read more:

J.R. Beveridge, K. She, B.A. Draper, and G.H. Givens. Parametric and Non-parametric Methods for the Statistical Evaluation of HumanID Algorithms, Third Workshop on Empirical Evaluation Methods in Computer Vision, Kauai, HI, December 2001
link

J.R. Beveridge, K. She, B. Draper, and G.H. Givens, A Nonparametric Statistical Comparison of Principal Component and Linear Discriminant Subspaces for Face Recognition, Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, December 2001, Kaui, HI, USA, pp. 535-542
link

B. Draper, K. Baek, M.S. Bartlett, and J.R. Beveridge, Recognizing Faces with PCA and ICA, Computer Vision and Image Understanding (Special Issue on Face Recognition), Vol. 91, Issues 1-2, July-August 2003, pp. 115-137
download here, 577 kB

P.J. Phillips, H. Moon, S.A. Rizvi, P.J. Rauss, The FERET Evaluation Methodology for Face-Recognition Algorithms, IEEE Trans. on Pattern Recognition and Machince Intelligence, Vol. 22, No. 10, October 2000, pp. 1090-1104
link

P.J. Phillips, E.M. Newton, Meta-Analysis of Face Recognition Algorithms, Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition (FRG'02), 20-21 May 2002, Washington, D.C., USA, pp. 224-230
link

S.A. Rizvi, P.J. Phillips, H. Moon, The FERET Verification Testing Protocol for Face Recognition Algorithms, Technical Report NISTIR 6218, Nat'l Inst. Standards and Technology, 1998
download here, 282 kB

S.A. Rizvi, P.J. Phillips, H. Moon, The FERET Verification Testing Protocol for Face Recognition Algorithms, Third IEEE International Conference on Automatic Face and Gesture Recognition, 14-16 April 1998, Nara, Japan, pp. 48-53
link

K. Delac, M. Grgic, S. Grgic, Statistics in Face Recognition: Analyzing Probability Distributions of PCA, ICA and LDA Performance Results, Proceedings of the 4th International Symposium on Image and Signal Processing and Analysis, ISPA 2005, Zagreb, Croatia, 15-17 September 2005, pp. 289-294
download here, 79 kB

K. Delac, M. Grgic, S. Grgic, Generalization Abilities of Appearance-Based Subspace Face Recognition Algorithms, Proceedings of the 12th International Workshop on Systems, Signals and Image Processing, IWSSIP 2005, Chalkida, Greece, 22-24 September 2005, pp. 273-276
download here, 337 kB

K. Delac, M. Grgic, S. Grgic, Independent Comparative Study of PCA, ICA, and LDA on the FERET Data Set, International Journal of Imaging Systems and Technology, Vol. 15, Issue 5, pp. 252-260
download here, 412 kB

J. Ruiz-del-Solar, J. Quinteros. Illumination compensation and normalization in eigenspace-based face recognition: A comparative study of different pre-processing approaches, Pattern Recognition Letters, Vol. 29, Issue 14, October 2008, pp. 1966-1979
download here, 1.58 MB link

More algorithms comparisons are also available on our "Vendors" page

back to top

 

Video-Based Face Recognition Algorithms

During the last couple of years more and more research has been done in the area of face recognition from image sequences. Recognizing humans from real surveillance video is difficult because of the low quality of images and because face images are small. Still, a lot of improvement has been made.

Read more:

G.J. Edwards, C.J. Taylor, T.F. Cootes, Improving Identification Performance by Integrating Evidence from Sequences, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, 23-25 June 1999, Ft. Collins, CO, USA, pp. 486-491
link

D. Gorodnichy, Video-Based Framework for Face Recognition in Video, Second Workshop on Face Processing in Video (FPiV'05), Proc. of the Second Canadian Conference on Computer and Robot Vision (CRV'05), 09-11 May 2005, Victoria, British Columbia, Canada, pp. 330-338
download here, 827 kB

S. Zhou, V. Krueger, R. Chellappa, Probabilistic recognition of human faces from video, Computer Vision and Image Understanding, Vol. 91, 2003, pp. 214-245
download here, 994 kB

S. Zhou, R. Chellappa, B. Moghaddam, Visual tracking and recognition using appearance-adaptive models in particle filters, IEEE Trans. on Image Processing, Vol. 13, No. 11, November 2004, pp. 1491-1506
link

S. Zhou, R. Chellappa, Probabilistic identity characterization for face recognition, Proc. of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR, 27 June - 02 July 2004, Washington, DC, USA, pp. II-805 - II-812
link

Z. Biuk, S. Loncaric, Face recognition from multi-pose image sequence, Proc. of the 2nd IEEE R8-EURASIP Symposium on Image and Signal Processing and Analysis, ISPA'01, 19-21 June 2001, Pula, Croatia, pp. 319-324
link

K.-C. Lee, J. Ho, M.-H. Yang, D. Kriegman, Video-Based Face Recognition Using Probabilistic Appearance Manifolds, Proc. of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2003, Vol. I, 16-22 June 2003, Madison, Wisconsin, USA, pp. 313-320
link

X. Liu, T. Chen, Video-Based Face Recognition Using Adaptive Hidden Markov Models, Proc. of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2003, Vol. I, 16-22 June 2003, Madison, Wisconsin, USA, pp. 340-345
link

G. Aggarwal, A.K. Roy-Chowdhury, R. Chellappa, A System Identification Approach for Video-based Face Recognition, Proc. of the International Conference on Pattern Recognition, 23-26 August 2004, Cambridge, UK
download here, 70 kB

back to top

© 2005-2014 VCL
Last update: 7 November 2008