ABSTRACT The face recognition is a fairly fairly controversial subject subject right now. now. A system such as this can can recognize and track dangerous criminals and terrorists in a crowd, but some contend that it is an extreme invasion of privacy. The proponents of large-scale face recognit recognition ion feel that it is a necessary necessar y evil to make make our country safer. It could benefit the visual visually ly impaired and allow them to interact more easily easily with the environment. environment. Also, a computer computer vision-based vision-based authentication system could be put in place to allow computer access or access to a specific room using using face recognition. recognition. Another possible possible application application would be be to integrate this technology into an artificial intelligence system for more realistic interaction with humans. We propose a face recognition method called the PCA (Principal Component Analysis). Principal Component Analysis (PCA) is a statistical method under the broad title of factor analysis. The purpose of PCA is to reduce the large dimensionality of the data space (observed variables) to the smaller intrinsic dimensionality of feature space (independent variables), which are needed to describe the data economically. This is the case when there is a strong correlation between observed variables. The jobs which PCA can do are prediction, redundancy removal, feature extraction, data compression, etc. Because PCA is a known powerful technique which can do something in the linear domain, applications having linear models are suitable, such as signal processing, image processing, system and control theory, communications, etc. The main idea of using PCA for face recognition is to express the large 1-D vector of pixels constructed from 2-D face image into the compact principal components of the feature space. This is called eigenspace projection. Eigenspace is calculated by identifying the eigenvectors of the covariance matrix derived from a set of f inge ingerprint rprint images (vectors).
1|Page
1. INTRODUCTION
1.1. PCA The Principal Component Analysis (PCA) is one of the most successful techniques that have been used in image recognition and compression. PCA is a statistical method under the broad title of factor analysis. The purpose of PCA is to reduce the large dimensionality of the data space (observed variables) to the smaller intrinsic dimensionality of feature space (independent variables), which are needed to describe the data economically. This is the case when there is a strong correlation between observed variables. The jobs which PCA can do are prediction, redundancy removal, feature extraction, data compression, etc. Because PCA is a classical technique which can do something in the linear domain, applications having linear models are suitable, such as signal processing, image processing, system and control t heory, communications, etc. Face recognition has many applicable areas. Moreover, it can be categorized into face identification, face classification, or sex determination. The most useful applications contain crowd surveillance, video content indexing, personal identification (ex. driver¶s licence), mug shots matching, entrance security, etc. The main idea of using PCA for face recognition is to express the large 1-D vector of pixels constructed from 2-D facial image into the compact principal components of the feature space. This can be called eigenspace projection. Eigenspace is calculated by identifying the eigenvectors of the covariance matrix derived from a set of facial images (vectors).
1.2.
MATHEMATICS OF PCA
A 2-D facial image can be represented as 1-D vector by concatenating each row (or column) into a long thin vector. Let¶s suppose we have M vectors of size N (= rows of image £ columns of image) representing a set of sampled images. pj¶s represent the pixel values.
(1)
The images are mean centred by subtracting the mean image from each image vector. Let represent the mean image.
m
(2)
And let be defined as mean centred image (3) ± m Our goal is to find a set of which have the largest possible projection onto each of the . We wish to find a set of orthonormal vectors for which the quantity (4) =
M
is maximized with the orthonormality constraint
2
|Page
(5) It has been shown that the and are given by the eigenvectors and eigenvalues of the covariance matrix
C =
(6)
where W is a matrix composed of the column vectors placed side by side. The size of C is N N which could be enormous. For example, images of size 64 64 create the covariance matrix of size 4096 4096. It is not practical to solve for the eigenvectors of C directly. A common theorem in linear algebra states that the vectors and scalars can be obtained by solving for the eigenvectors and eigenvalues of the M M matrix . Let and be the eigenvectors and eigenvalues of , respectively.
By multiplying left to both sides by
(7)
W
=
(8)
which means that the first M - 1 eigenvectors and eigenvalues of are given by Since we only and respectively. needs to be normalized in order to be equal to sum up a finite number of image vectors, M , the rank of the covariance matrix cannot exceed M - 1 (The -1 come from the subtraction of the mean vector m).
The eigenvectors corresponding to nonzero eigenvalues of the covariance matrix produce an orthonormal basis for the subspace within which most image data can be represented with a small amount of error. The eigenvectors are sorted from high to low according to their corresponding eigenvalues. The eigenvector associated with the largest eigenvalue is one that reflects the greatest variance in the image. That is, the smallest eigenvalue is associated with the eigenvector that finds the least variance. They decrease in exponential fashion, meaning that the roughly 90% of the total variance is contained in the first 5% to 10% of the dimensions. dimensions by computing A facial image can be projected onto
(9) where is the coordinate of the facial image in the new space, which came to be the principal component. The vectors are also images, so called, eigeni ages, or
m
eigenfaces in our case. They can be viewed as images and indeed look like faces. So, describes the contribution of each eigenface in representing the facial image by treating the eigenfaces as a basis set for facial images. The simplest method for determining which face class provides the best description of an input facial image is to find the face class k that minimizes the Euclidean distance
(10) where is a vector describing the face class. If is less than some predefined threshold =
, a face is classified as belonging to the class k.
3
|Page
2.
FACE RECOGNITION
Once the eigenfaces have been computed, several types of decision can be made depending on the application. What we call face recognition is a broad term which may be further specified to one of following tasks: y y
y
identification where the labels of individuals must be obtained, recognition of a person, where it must be decided if the individual has already been seen, Categorization where the face must be assigned to a certain class.
PCA computes the basis of a space which is represented by its training vectors. These basis vectors, actually eigenvectors, computed by PCA are in the direction of the largest variance of the training vectors. As it has been said earlier, we call them eigenfaces. Each eigenface can be viewed a feature. When a particular face is projected onto the face space, its vector into the face space describes the importance of each of those features in the face. The face is expressed in the face space by its eigenface coefficients (or weights). We can handle a large input vector, facial image, only by taking its small weight vector in the face space. This means that we can reconstruct the original face with some error, since the dimensionality of the image space is much larger than that of face space. In this report, let¶s consider face identification only. Each face in the training set is transformed into the face space and its components are stored in memory. The face space has to be populated with these known faces. An input face is given to the system, and then it is projected onto the face space. The system computes its distance from all t he stored faces. However, two issues should be carefully considered: y y
What if the image presented to the system is not a face? What if the face presented to the system has not already learned, i.e., not stored as a known face?
The first defect is easily avoided since the first eigenface is a good face filter which can test whether each image is highly correlated with itself. The images with a low correlation can be rejected. Or these two issues are altogether addressed by categorizing following four different regions: 1. 2. 3. 4.
Near face space and near stored face known faces Near face space but not near a known face unknown faces Distant from face space and near a face class non-faces Distant from face space and not near a known class non-faces
Since a face is well represented by the face space, its reconstruction should be similar to the original; hence the reconstruction error will be small. Non-face images will have a large reconstruction error which is larger than some threshold . The distance determines whether the input face is near a known face.
4
|Page
3. REQUIREMENTS 3.1.
HARDWARE REQUIREMENTS o
3.2.
Camera integrated system
SOFTWARE REQUIREMENTS o o o
Windows OS Visual Studio Open-CV version 2.10
5
|Page
4. UNDERLYING PR INCIPLE 4.1.
Main idea behind eigenfaces
- Suppose G is an 1vector, corresponding to an N N face image I . - The idea is to represent mean face) into a low-dimensional space: (K<
4.2.
Computation of the Eigenfaces
Step 1: obtain face images (training faces) (very important: the face images must be cantered and of the same size)
Step 2: represent every image as a vector Step 3: compute the average face vector :
Step 4: subtract the mean face:
Step 5: compute the covariance matrix C :
where A =[ ] ( M matrix) Step 6: compute the eigenvectors of A The matrix A is very large --> not practical !! Step 6.1: consider the matrix (M xM matrix) Step 6.2: compute the eigenvectors of = What is the relationship between and ? A or where A C =
Thus, A and A have the same eigenvalues and their eigenvectors are related as follows: =A !! Note 1: A can have up to eigenvalues and eigenvectors. Note 2: A can have up to M eigenvalues and eigenvectors. Note 3: The M eigenvalues of A (along with their corresponding
6
|Page
eigenvectors) correspond to the M largest eigenvalues of (along with their corresponding eigenvectors). Step 6.3: compute the M best eigenvectors of : A (important: normalize such that || || =1) Step 7: keep only K eigenvectors (corresponding to the K largest eigenvalues)
7
|Page
4.3.
REPRESENTING FACES ONTO THIS BASIS
- Each face (minus the mean) in the training set can be represented as a linear combination of the best K eigenvectors:
(we call the ¶s eigenfaces)
- Each normalized training face
is represented in this basis by a vector:
, i
=1,,2.......,M
8
|Page
4.4.
FACE RECOGNITION USING EIGEN FACES
- Given an unknown face image (centered and of the same size like the training faces) follow these steps: Step 1: normalize
Step 2: project on the eigenspace
Step 3: represent
as:
Step 4: find Step 5: if ,< then is recognized as face l from the training set. - The distance , is called distance within the face space (difs)
Comment: we can use the common Euclidean distance to compute er , however, it has been reported that the M ahalanobis distance performs better:
(variations along all axes are treated as equally significant)
9
|Page
4.5.
FACE DETECTION USING EIGENFACES
- Given an unknown image Step 1: compute
Step 2: compute
Step 3: compute
, then is a face. is called distance from face space (dffs)
Step 4: if < - The distance
10 | P a g e
5. 5.1.
TECHONOLOGY USED MICROSOFT VISUAL C++
Visual C++ is IDE of Microsoft to develop application using C++ & its STL (Standard Template Library). Application developed by visual C++ can be of different types such as MFC application, win console application etc. In our case, we will be developing win console application using OpenC V. 5.2.
OPEN CV
OpenCV [OpenCV] is an open source (see http://opensource.org ) computer vision library available from http://SourceForge.net/projects/opencvlibrary. Th e library is written in C and C++ and runs under Linux, Windows and Mac OS X. Th ere is active development on interfaces for P ython, Ruby, Matlab, a nd other languages. OpenCV was designed for computational efficiency and with a strong focus on realtime applications. OpenC V is written in optimized C and can take advantage of multicore processors. If you desire further automatic optimization on Intel architectures [Intel], you can buy Intel¶s Integrated Performance Primitives (IPP) libraries [IPP], which consist of lowlevel optimized routines in many different algorithmic areas. OpenC V automatically uses the appropriate IPP library at runtime if that library is installed. One of OpenC V¶s goals is to provide a simple-to-use computer vision infrastructure that helps people build fairly sophisticated vision applications quickly. The OpenC V library contains over 500 functions that span many areas in vision, including factory product inspection, medical imaging, security, user interface, camera calibration, stereo vision, and robotics. Because computer vision and machine learning often go hand-in hand, OpenC V also contains a full, general-purpose Machine Learning Library (MLL). This sub library is focuse d on statistical pattern recognition and clustering. The MLL is highly useful for the vision tasks that are at the core of OpenC V¶s mission, but it is general enough to be used for any machine learning problem.
11 | P a g e
5.2.1
OpenCV Structure and Content
OpenCV is broadly structured into fi ve main components, four of which are shown in Figure. The CV component contains the basic image processing and higher-level computer vision algorithms; ML is the machine learning librar y, which includes many statistical classifiers and clustering tools. HighGUI contains I/O routines and functions for storing and loading video and images, and CXCore contains the basic data structures and content.
12 | P a g e
6. 6.1.
USER DEFINED FUNCTIONS AND THEIR PURPOSE printusage This function is used to control the program from the command line user interface. Depending upon the user input it transfers the control of program to either training phase or testing phase or online recognition of a face from camera.
6.2.
learn This function is used for offline recognition mode where it trains from the data given in text file and store the trained data into the file µfacedata.xml µ.
6.3.
doPCA This function is used to perform PCA operation on the training faces.
6.4.
StoreTrainingdata This function is used to store the recognition data as an xml file.
6.5.
loadTrainingdata This function is used to load the training data from the file 'facedata.xml'.
6.6.
Findnearestneighbour This function is used to find the most likely person based on detection. It returns the index of image, and stores the confidence value into variable pConfidence.
6.7.
loadfaceImgArray This function is used to read the names & image of people from a text file, and load all those images listed.
6.8.
Recognizefilelist It is used to recognize the face in each of the test images given, and compare the results with the truth.
6.9.
Getcameraframe It is used to grab the next camera frame. It waits until the next frame is ready, and provides direct access to it, as a result it do NOT modify the returned image or free it. It automatically initializes t he camera on the first frame.
13 | P a g e
6.10.
Convertimageto grayscale
It is used to return a new image that is always greyscale, irrespective of the input type of image. It free the returned image using cvReleaseImage() when finished. 6.11.
cropImage It returns a new image that is a cropped version of the original image.
6.12.
resizeImage
It creates a new image copy that is of a desired size. Remember to free the new image later. 6.13.
convertFloatImagetoUcharImage
It gets an 8-bit equivalent of the 32-bit Float image. It returns a new image, so remember to call cvReleaseImage() on the result. 6.14.
SaveFloatImage
It stores a greyscale floating-point CvMat image into a BMP/JPG/GIF/PNG image,since cvSaveImage() can only handle 8bit images (not 32bit float images). 6.15.
detectFaceInImage
Perform face detection on the input image, using the given Haar cascade classifier. It returns a r ectangle for the detected region in the given image. 6.16.
Retrainonline
It is used to re-train the new face recorded in a database without shutting down.Depending on the number of images in the training set and number of people, it might take 30 seconds or so.
14 | P a g e
.
Snapshots of project on running mode
Figure : It represents start of the project where user have to enter a command, according to which it wil run further.
Figure : It represents status of the program when user has entered r ecognize as its command.
15 | P a g e
Figure : It represents status of the program where the progra m is trying to recognize a face
Figure: It represents a camera frame which has been captured by camera. After detecting a face in the frane the program identifies it and returns name of the function and its confidence level.
16 | P a g e
8.
Future Work
There are many interesting problems that remain in the area of face recognition. One problem is image preprocessing prior to the application of the eigenface method. It may be possible to gain better accuracy in classification if one segments the spectrum of people into different spaces. For example, if one was able to determine if an image was of a man or a woman, one could use this categorization to send an image to one of two classifiers, each specifically trained with that type of individual in mind. This would mean that there would be a set of eigenfaces specifically for males and one specifically for females (face spaces with gender, so to speak). Work in this area has been done by Lizama, Waldoestl and Nickolay [4], however it would be interesting to extend it to use eigenfaces to act as the gender classifier as well. A general face-space would be created in addition to the male and female face-spaces, with the sole purpose of being used to classify an image as male or female. Another area of future work is improving our neural network classifier. As mentioned in the previously, it is possible to construct the network to take its input directly from the image data rather from the vector that results from an images projection into face-space. Perhaps learning the face projection function could increase the accuracy of the neural network classifier. Additionally, more experiments are needed to see if there are other ways to tweak the network configuration to produce better results.
17 | P a g e
9.
R eference y
Learning OpenC V ±Computer Vision with the openC V Library O¶Reilly Publication Gary Bradsk1 and Andrian Kehlar
y
y
M.A. Turk and A.P. Pentland, ³Face Rec ognition Using Eigenfaces´, I EEE Conf. on Computer Vision and Pattern Recognition , pp. 586-591, 1991. Kyungnam Kim ´ Face Recognition using Principle Component Analysis´
18 | P a g e
