Eigenfaces Group - Intro

Face recognition is a task which humans can perform with ease (most humans anyway - sufferers of proposagnosia can't recognize their own mother!). For computers, though, recognizing faces can be a Herculean task. Armed with some top-notch research, we attacked it anyway.

History:
The technique we used for face recognition was developed at Brown University in 1988 by Kirby and Sirovich and then expanded upon by a group at MIT. Our system is based on the work of Pentland, Turk, Moghaddam, and Starner of the Vision and Modeling Group of MIT.

Storage:
The face recognition system that we worked on builds a set of orthonormal basis vectors based on the Karhunen-Loève procedure for generation of orthonormal vectors. Using the best (highest eigenvalue, or most face-like) of these basis vectors which we call eigenfaces, we map images to "face-space". Using this representation, we can store each image as only a vector of N numbers where N is the number of eigenfaces. This results in huge storage savings as both the MIT group and we concluded that 50 eigenfaces forms a fairly comprehensive set of eigenvectors for characterizing faces. Thus, we can store our 80K images as 50 numbers.

Matching:
Using this stored representation of the images, when presented with a new image we can map it to face-space as well and quickly see which vector it most corresponds to or whether it corresponds to any of the vectors at all. By seeing if it corresponds to any of the stored vectors better than a certain threshold we can determine who the person is. If the image does not correspond to any of the stored vectors we conclude that we do not know (or fail to recognize) the person. Also, by taking the image to face-space and then back to image space we can see how good the reconstruction is and by this determine whether the image is in fact a face or not.

Reconstruction:
The ability to reconstruct the images from our stored vectors gives us both the ability for face-checking, the determination of whether the image is a face, and also image compression since the 50 values and corresponding set of eigenfaces are enough to reconstruct most any face.

Applications:
The face recognition system has a number of uses, some benign and some which cause apprehension in readers of Orwell's 1984:

• System (key) access based on face/voice recognition
• Tracking people either spatially with a large network of cameras or temporally by monitoring the same camera over time. (London is currently attempting to do both)
• Locating of people in large images

See Algorithms for more details.