Han Yu (hy532)
Yingjie Li (yl2988)
In this project, we designed a system to control the electrical appliances and arm the house with a tracking camera. There are two parts in the system. In the ‘Controller’ part, users can turn on the lights and the fan by touching the screen. The temperature and humidity of the house show on the ‘Controller’ page. In the ‘Security’ part, users are required to set the PIN for their house. With the correct PIN, users can arm or disarm the house. When the house is in ‘Arm’ status, magnetic door sensor turns on. Once the door is open, the camera will start to track the face of intruder, taking pictures and sending an alert email to the user.
In our design, we separate the power supply for RPi and the tracking camera to make sure the power supply we use is stable. We use GPIO.6, GPIO.23, GPIO.16 to control three lights, GPIO.19 to control the fan and GPIO.12 and GPIO.13 to control the servos we use for tracking camera. GPIO.5 and GPIO.26 is used as input GPIO. GPIO.26 is used to sense the status of the door and GPIO.5 is used to read in the data for temperature and humidity sensor.
To get a more stable control of our servos using PWM, we use the hardware PWM in RPi. The RPi does support the hardware PWM but only several pins are configurable. We use GPIO.12 and GPIO.13 to control the servos by hardware PWM. The hardware PWM is more stable than the software PWM as the software PWM will result in significant wiggle and shiver on the pulse as the RPi is not a real-time system and the RPi will deal with many other processes at the same time. So, in our design, we use the hardware PWM to control the servos.
The power supply for the fan and the servos should be separated from the RPi. In our first attempt to combine the control and the security function, we use the 5V on RPi to power the servos and the fan, the servo broke down. The TA told us that the fan needs the current to power it and so does the servos. However, the current provided by the RPi is small and fixed, if you want to power the servos and the fan at the same time, the current will be unstable and may cook something unexpected. So, in our design, we separated the power supply.
We created three interfaces in our project including the homepage, the controller page and the security page. On the home page, we will show the date and two choices ‘Security’ and ‘Controller’ for the user to choose from.
Fig.1 Home page
If we choose the ‘Security’, the page will turn to the security page and prompt the user to enter the PIN to arm or disarm your house.
Fig.2 Security page
You can navigate to home page by pressing the ‘HOME’ button.
When you choose ‘Controller’ button, you can navigate to the controller page on which you can send your commands to the electrical appliance in your house. The temperature and the humidity of the house will refresh every time you click on it.
Fig.3 Controller page
In this function, you will know the temperature and the humidity in the house by clicking the space where shows the temperature and humidity. The data will be read from GPIO.5, the temperature and humidity sensor. What’s more, you can turn on and turn off the light in different room by press the corresponding buttons shown on the screen. Additionally, you can turn on and turn off the air conditioner (the fan) via the interface.
There are two statuses of the house, one is ‘Arm’ and the other one is ‘Disarm’. When used for the first time, you can press ‘*’ to set your own PIN number and end with ‘#’ to confirm your PIN. Then, every time you want to change the status of the house, there will be a prompt prompting you to enter your password and end your entering with ‘#’, you are required to type in the right PIN to change the status. When you choose the ‘Arm’ status, the door sensor is ready. Once the door is open, the tracking camera will work and take pictures of the intruder. When you choose ‘Disarm’, you are required to enter the authentic PIN. Once the PIN is verified authentic, the house is in the status of ‘Disarm’ and when the door is open, the tracking camera will not work.
The camera we use is hold by a rotating platform of two servos. One is used to rotate horizontally and the other one is used to rotate vertically. For the first time we use the servos, we should calibrate them. Since the rotation angle is controlled by the duty cycle of the PWM sent to it, we can change the duty cycle of the PWM signal and find the angles it rotates corresponding to the duty cycle. One thing to mention here is the duty cycle corresponding to an angle will change some time.
Then, after we calibrate the servo, we can start to compose the Python code for the tracking camera. We use OpenCV to realize the recognition of human's face and track the face. First, the camera will capture a face and calculate the center coordinate of the face. Then it will compare the face's center coordinate with the center coordinate of the frame. Then the difference will be used to adjust the duty cycle of the PWM signal sent to the servo so that the servo can rotate in the corresponding angles. The recognized human’s face will always be located in the center of the frame.
After we captured the pictures we save them locally. When we save five images, we will send all of them to the user and clear them all locally. The camera captures one picture every 50 times tracking the face. We use SMTP in Python code to send email. We use Gmail to send and receive emails in our code. The images will be sent to the users as attachments of the email.
After building the hardware part, we test the control function first. We set the GPIO for the device at HIGH to turn them on and at LOW to turn them off. Every time the command is executed successfully, we will get a feedback message in our terminal.
First, we tested the servo we use for the camera. We tested them to calibrate them. The direction of the servo is controlled by the duty cycle of the signal sent to it. So, for the first thing, we wrote a Python code to generate the signal with different duty cycles and recorded the corresponding performance of the servo to find the relation between the duty cycle and the rotation angle of the servo.
Then, we composed the Python code for the camera. First, the camera should capture the picture and recognize the human face in the picture and mark it out.
Combine the code for camera with the servo control code. when the object marked in the picture moves, the camera will move with it, marking it continuously to capture the picture. The pictures will be captured every ten times marked.
After the camera can capture and save the pictures, we tested the code for sending an email with pictures attached and an ‘Intruder’ message in the body of the email. We use Gmail to send and receive the email.
Our system can display most of the functions expected at the beginning. There are mainly three parts in our system.
First, for the homepage, the date and a ‘WELCOME HOME!’ will show on the screen. There are two buttons on the screen, one is ‘Security’ and the other one is ‘Controller’. When you press ‘Security’, the page will turn to the ‘Security’ page. When you press ‘Controller’, the page will jump to the ‘Controller’ page.
Fig.4 the model house
Fig.5 the security page with prompt prompting to enter the PIN Fig.6 the camera tracking the face Fig.7the alert email received by the user
Fig.8 the controller page to control your lights
We finished our project successfully after 6 weeks. The results we get basically completed our expected goals. we encountered and solved many problems during the 6 weeks.
For the first thing, in hardware design, we cooked one of our servos because of the wrong power supply. We powered the servos and the fan with 5V on RPi at the same time. Since the current provided by the pin is very small while the fan and the servo need more current to start, the results of the connection may be unexpected. We changed the power supply to an external power supply after being suggested by the TA and it worked as expected.
Additionally, the servo we use may change its corresponding duty cycle some time. In many cases, every time we rebooted our system, the servo always acted strangely in an unexpected way. We thought it was cooked again but when after we get help from TA, we found that the servo is good, but the corresponding duty cycle of the angles may change for some time. So, every time we reboot our system, we should first check the servo to make sure it works as expected.
At our first attempt to run our system on piTFT, we found the piTFT just did not respond to our touch. We checked our code and hardware connection step by step and finally found that that’s because we occupied the GPIO.18 which is used to acquire the touch to information from the piTFT. We solved this problem by changing the GPIO we used to connect our hardware components.
We also found our system had a great latency at the first attempt, which is uncommon for our system. We tried to comment out the function of the GPIO we use to test which GPIO caused the latency. Finally, we found that the GPIO which reads data from Temperature and Humidity Sensor has the greatest latency as the speed for data transferring is very slow. We set every click will allow the data to refresh once. However, when the data is refreshing, the entire process will be blocked, thus there will be a lot of latency.
We will try to make the temperature and the humidity refresh automatically without causing latency in our system. To do this, we can try to run the data reading process in the background and communicate with the main process.
We can design a Web interface or mobile phone APP to make it possible to control the system remotely.
We can add more sensors and functions to our system like the PIR motion sensor and a buzzer to alarm the intruder.
With machine learning, we can add a face recognition function to our system which can recognize the user’s face and the stranger’s face. We can build a library and train the recognition model, then use the model to recognize.
The project cannot be completed successfully without anyone’s help. We have a very enjoyable cooperation with each other.
The software design, including piTFT interface design, camera tracking design and security page design. Assembled the model house.
The hardware design and connection. Composed the Python code for sending emails. Wrote the weekly reports and final report. Assembled the model house.
During the project, Professor Joseph Skovira helped a lot with our project. He provided us with components and gave us a lot of useful suggestions. TAs in ECE 5725 including Xitang Zhao, Mei Yang and Joao Pedro Carvao also helped us a lot. They cared much about our programs and did solve us lots of problems.