Face Recognition Door Lock

Attach the servo to a GPIO (we selected GPIO 17 here) of the Raspberry pi A and control its rotation utilizing pulse-width modulation. The servo is powered by a 6V-battery pack.

Figure 3: Quick-Start Circuit

Below is the final outlook of our face recognition door lock model.

Figure 4: System Framework

Pi B works as server waiting for connection from Pi A and its user interfaces only show up when Pi B receives an image. The image will be displayed on its PiTFT screen plus 2 buttons ‘Yes’ or ‘No’, by pressing which the corresponding response will be sent to Pi A. See details in 2 ‘main.py’ scripts in code appendix.

Figure 6: User Interface of Pi B

Test

Since there are many levels of touch buttons and we need to display various messages in different circumstances, we observed pygame window on computer monitor and debugged our user interfaces layer by layer. We tested the sensitivity and reaction time of each touch button and experimented with different sleep time to make sure buttons all work well. Besides these, we also did some corner tests to see what would happen.

• Face Detection and Haar Cascade Classifier

After preprocessing, like resizing and cropping, the image will be used as input of Haar Cascade Classifier to detect whether there is a single face detected in this image.

Face detection is a process of finding out the face area in the image. In the project, we use Haar Cascade to detect faces. Haar-like features such as edge features, line features and center-surround features are used and they are inputs of classifiers. Cascade classifiers test the image by cascade features. Since the amount of features is large, instead of applying all features on the window, the features are divided into different stages. The window will be tested stage by stage and initial stages usually have less haar-like features. If the window fails in a stage, it will be discarded and the following stages won’t be tested. The window which successfully passes all the stages is considered to be face image. Haar cascade classifiers han an advantage of its fast detection speed compared to other classifiers.

Test

We tried to resize captured images to different sizes and different classifier tuning parameters like scale factor, minimum number of neighbors, to find out the best configuration for face detection. See details in ‘face.py’ in code appendix.

• Face Recognition and Eigenface Classifier

If a single face is detected by Haar Cascade classifier, the face will be cropped out of the scene. Then the Eigenface classifier that has already been trained by prestored image library, will try to recognize the cropped face and return the confidence of its prediction at the same time. By setting a threshold for the predicting confidence, we can determine when to acknowledge that the face is truly recognized.

Face recognition is matching the input signal with the prestored library. Though the input signal is noisy due to the different angle, position and intensity of light, the image could be recognized according to the position of eyes, face and mouth in the face, and their relative distances between each other. These features are called eigenfaces and they could be extracted from original image data by principal component analysis. Each face is represented by a subset of eigenfaces and the face could be reconstructed if eigenface could be correctly calculated for each proportion. The new image will be recognized based upon eigenvectors and Euclidean distance between eigenvectors. See details in ‘train.py’ and ‘classify.py’ in code appendix.

Test

We tested the eigenface classifier with different confidence threshold to get the best recognition accuracy. After the system could recognize one face successfully, we also considered to make the system recognize multiple faces by setting up several classifiers and let them do recognition work in turns or in multi-thread way. As a result, due to the limited time, we continued on the former choice which is simpler than multi-thread.

• Passcode Implementation

In the implementation of keypad, the approach for deciding the pressed button is firstly setting outputs (column pins) and inputs (row pins) as high. When a button is pressed, it will produce low signal. Then each column is scanned in a ‘for’ loop. If the low signal is detected, we could know which button is pressed. See details in ‘passcode.py’ in code appendix.

Test

In the test of passcode function, the system worked well on the monitor screen at beginning. But in the condition that all buttons and layers were displayed on PiTFT screen,, the PiTFT screen would display sliding messy codes when we entered the password ending with ‘#’. The TA helped us debugging and told us the problem was caused by wrong circuit connection. We checked our circuit and found the reason is that we attached one of the keypad interface pin to GPIO 25 which is used for special function. We changed the connection to GPIO 21 and the problem was solved.

• Communication over TCP Sockets

The two Raspberry pis in the project are set to communicate using TCP sockets via wifi modular and a wifi router. The Raspberry pi A, which displays main buttons, is the client and the other Raspberry pi B, for remote confirmation, is the server. With specific IP addresses, the client could connect to the server and send files like images over TCP socket. Then the server will send response back to client and once response received, the client is going to close connection. We developed python script code for above communication routine by leveraging python library ‘socket’. See details in ‘server.py’ and ‘client.py’ & ‘client2.py’.

Test

When we started this communication part, there were 2 choice when deciding which Pi worked as a server or client. We tested on both 2 methods and found that Pi B needs to wait for images sent by Pi A at any time, so it’d better to be a server. By doing so, whenever Pi A, the client, is trying to send an image to Pi B, it could be connected to Pi B successfully since the server keeps on all the time.