"Home is where the
After the inventions of PCs, the Internet and personal digital devices, such as smart phones and wearable devices, loT is now believed to be the trend of the world. Through the loT everything in people's life are connected to the Internet. A smart home system is a mini loT network of people's personal in which all the electronic devices, including sensors of all kinds, household appliances and smart digital devices in a house are connected to each other and work cooperatively and automatically.
The smart home system this project builds mainly includes a temperature sensor, a fan, a LED light, a microphone and a control unit. The control unit building on a raspberry Pi connects the microphone ,which is used for sound detecting, through the USB, connect all the other parts together through Wi-Fi and is in charge of all the controlling and data processing.
The fan and the LED light are remotely controlled by the raspberry Pi while the fan can not only be manually controlled by the control console on a TFTscreen, but can also be auto controlled by the raspberry Pi according to the temperature value sending from the temperature sensor. The temperature sensor sends the temperature value to the raspberry Pi all the time, and the fan runs faster when the temperature goes high and stops when the temperature is under 25 centigrades.
The sound detecting part is used to record and analyze sound appeared in home, if there are any explosions or other unusual sounds, the raspberry Pi could report it to user. Fast Fourier Transform is applied to the sounds recorded to analyze frequency information, from the frequency information we could identify if the sound is unusual or not.
The objectives of this system can be separated into two parts, those for the remote control part and those for the sound detecting part.
Part I. Remote Control
To accomplish a remote control system based on Wi-Fi, the following objectives should be achieved:
1. Tempreture value continually sent from the temperature sensor to the raspberry Pi every second.
2. The speed the fan runs can be remotely controlled according to the input temperature value in the auto control mode.
3. Draw a control console displayed on the TFT screen to control the fan and the LED light in manual control mode.
4. The operating mode can be switched between auto control mode and manual control mode manually.
Part II. Sound Detecting
To detect and identity a piece of sound, the following objectives should be achieved:
1. Accurately record the sound from outside.
2. Apply Fast Fourier Transform to the recorded sound.
3. Analyze the frequency information and identify if there is unusual sound.
Design and Testing top
The design and testing part can also be separated into the same two parts as above.
Part I. Remote Control
In the remote control part, the first thing is to set up a Wi-Fi module for the fan and the LED light so that they can be control through Wi-Fi signal. We chose to use the esp8266 Wi-Fi breakout which can be programmed through the Arduino IDE. The esp8266 Wi-Fi breakout is a Wi-Fi shield that offers a complete and self-contained Wi-Fi networking solution allowing it to host the application itself without external clock or storage . And also this shield has eight GPIOs, which can wire the temperature sensor, the LED light and the fan to it at the same time.
Since the fan we used needed a 5V power supply and the GPIOs of the esp8266 Wi-Fi breakout can only provide 3.3V voltage, so we used an external 5V power supply for it. To still control the fan using a 3.3V GPIO, we used a BD139 MOSFET to control its path to the ground as the circuit diagram shows below.
To control the whole system without needing a keyboard and a mouse plugging to the raspberry Pi, we also installed a 2.8-inch TFT touchscreen on the raspberry Pi for the GUI control console. We configured the TFT touchscreen according to its installation instructions .
The speed the fan runs and the ON/OFF of the LED light should be controlled according to the commands sent from the control unit. But we need first test the function in local before starting the communication. The Wi-Fi shield can outputs different PWM signal through the GPIO to the fan to control its speed, and the LED light is control by the digital voltage of the GPIO it connects to. So to test the hardware we wrote a simple script to turn on the LED light by giving its GPIO a high value, and outputting a PWM signal, which changed from time to time, to control the fan. All of them worked just as expected. And the TFT screen displayed the desktop successfully shown in the figure below.
To set the raspberry Pi as a hotspot using its Ethernet cable network, we used a miniature Wi-Fi module (802.11b/g/n) to establish a wireless network. We followed the instruction on the the link  to set a Wi-Fi hotspot named Pi_AP using the USB Wi-Fi module and bind the hotspot with a static IP address, 192.168.42.1.. The hotspot could distribute at most ten slave IP addresses to the devices connected to it.
To test the Wi-Fi network that the raspberry Pi set, we connected our cell phone to the network and checked the IP address it got. As we can see from the figure below, the IP address of the router, which referred to the raspberry Pi, is 192.168.42.1, and it distributed the IP address, 192.168.42.10, to the cell phone successfully.
After setting the hotspot successfully, we needed to set up the Wi-Fi connection between the Wi-Fi shield and the raspberry Pi. We wrote the Wi-Fi configuration code in C using the esp8266 library provided by Arduino and printed out the IP address of the shield after it successfully accessed the hotspot. The IP address of the esp8266 Wi-Fi shield is necessary for the future communication between the raspberry Pi and the shield.
We downloaded the program to the Wi-Fi shield through Arduino IDE, and monitored the log printed through the UART port. When the connection is set successfully, we can see the Wi-Fi status which contains the name of the network, the shield’s IP address and the signal strength through the log just like the figure showing below.
Temperature Value Receiving
When the Wi-Fi connection was set, we wired the temperature sensor, which we used the ds18s20 digital temperature sensor, to another GPIO of the esp8266 Wi-Fi breakout. The ds18s20 digital temperature sensor provides 9-bit Celsius temperature measurements ranging from -10°C to +85°C over a 1-Wire bus that only requires one data line. Before transmitting the data to the control unit, we first programmed the Wi-Fi shield to let it keep receiving the temperature value from the temperature sensor every second.
To receive the data from the ds18s20 temperature sensor, we used the <OneWire.h> library and the < DallasTemperature.h> library provided by Arduino. We received and printed the current temperature the sensor measured every second. The result shows below.
After the Wi-Fi shield successfully received the temperature value from the temperature sensor through the 1-Wire bus, the next thing to do is to transmit the data to the control unit and receive command from it, which means building a two-way communication between the control unit and the Wi-Fi shield.
We used the “socket” to realize the communication based on Wi-Fi signal, since we know the IP addresses of both sides of the communication. We chose the UDP socket instead of the TCP type because it’s faster, and it is also easier for either part of the communication to switch its role between a server and a client.
In the auto control mode, every second the Wi-Fi shield receives a value from the sensor, it transmits the data to the raspberry Pi as a client and listen for the feedback command. The program runs in the raspberry Pi first keep listening for the message from the Wi-Fi shield as a server and give the corresponding command to the Wi-Fi shield as a client after receiving and processing the data. Both parts of the communication switch their roles between the server and the client all the time. While in manual control mode, the raspberry Pi keeps running as a client and only hears from the Wi-Fi shield for a feedback after sending the commands. The switch of the two modes can be accomplished by pressing a button on the raspberry pi. Since we need to run a non-stop loop on both sides of the communication while the project is running, so we need an interrupt when the button is pressed.
We wrote the socket program in python on the raspberry Pi and in C on the Wi-Fi shield. To test the whole communication function, we ran the program through the terminal on the raspberry Pi. We used the GPIO #22 on the TFT screen to bring an interrupt to switch the mode from the auto control to the manual control.
We start the program in auto control mode. At first the Wi-Fi shield kept transmitting the temperature value to the control unit every second as a client and the raspberry Pi processed the data after receiving it as a server. After processing the temperature value, if the fan speed needed to change, the program running on the raspberry Pi would change to a client and generate a corresponding command sending back to the Wi-Fi shield, which now change to a server, to adjust the fan speed. We could see the result through the log from the terminal of the raspberry Pi and from UART port of the Wi-Fi shield.
And then we pressed the button on the TFT screen to switch the mode to manual control. We used the terminal to input commands to control the LED and the speed of the fan. The log on the raspberry Pi side printed the command content and the target IP address after we input the command. When the Wi-Fi shield successfully received its commands it sent a feedback message back to the raspberry Pi, which is “acknowledged” in this case, and the raspberry Pi would print this feedback on its log telling people the command was received successfully by the other side. The log on the Wi-Fi shield side printed the size and the content of the command it received and the IP address of the control unit. The result shows below.
After the all the functions were verified, we built a GUI control console for the manual control mode where people can turn the LED light on and off through a virtual button and change the speed of the fan by sliding a bar. The control console will appears whenever the button on the TFT screen is pressed, and when people want to quit the manual control mode letting the system go back to the auto control mode, they can just click another virtual button.
We wrote the GUI in python using the Tkinter library. We drew two buttons, one for the LED light control and one to quit the manual control mode, and also a bar valued from 0 to 100 to control the fan speed. The result shows below.
Part II. Sound Detecting
This part of the project contains mainly two parts, which are sound recording part and frequency analysis part. The sound recording part is composed of microphone, sound card, and Linux audio library. The frequency analysis part is composed of wav file parser and Fast Fourier Transform library.
The noise measurement requires high-resolution and wide-frequency microphones, after some investigation, we found that condenser microphones meet our requirements. On the other hand, raspberry Pi does not have an audio card itself, which means an external audio card is needed. So we found CAD Audio U37 USB Cardioid Condenser Studio Recording Microphone is just the product we need. It is a condenser microphone and owns an internal sound card, providing direct USB connection to the raspberry Pi.
ALSA (Advanced Linux Sound Architecture) library provides audio and MDI functionality to the Linux operating system. By the APIs the library provides, we could control the microphone and the raspberry Pi playback system. After installing ALSA library successfully, we need to configure the alsamixer to set up the microphone port on the card.
Then we could use arecord command to record the sound. After recording the sound, we could use aplay command to playback.
Type command alsamixer to enter the GUI. First press F6: Select Sound Card, the select USB Device PNP, which represents the USB microphone. And we could also adjust the gain from the microphone in this GUI. Then we could use arecord command to record the sound. The result shows below.
We Ran the command in terminal:$ arecord -D plughw:1 --duration=1 -f cd music.wav. This command creates a music-x.wav file every 1 second, x is the number of the files.
Installing FFTW and Libsndfile Library
In order to do frequency analysis, we need to apply Fast Fourier Transform (FFT) to the data we collected. And FFTW library is an excellent implementation of FFT with very fast computing speed. Luckily, we could also use apt-get install command to install this library.
FFT is applied to complex values, so we need to read out the raw data from wav file. Libsndfile is a C library for reading and writing files containing sampled sound.
We could use sndfile-mix-to-mono command to convert the multi-channel wav file to a mono-channel wav file, making it suitable for doing FFT.
Applying the FFT
To finally applying the FFT, first, we need to read the wav file in a buffer. Then build two complex arrays, which are input and output of FFT function respectively. Next we executed the FFT and wrote the output to a file named wav_FFT.txt. The code is attached in code appendix part.
The whole system we finally built shows below. For the remote control part, we successfully achieved all the objectives we set including control both the LED light and the fan through Wi-Fi in both auto control mode and manual control mode.
For the sound detecting part, we recorded the sound from the microphone and generate a wav file, converted the wav file to raw data and applied Fast Fourier Transform to it. The results were stored in a text file for future analysis. The analysis of the sound needed to be complete in future progress.
Results vs. Expectations
Although we achieved most of the objectives, there are still some parts of the project need further improvements:
1. The sound detecting part hasn’t been finished; we still need to compare the result of the FFT to the frequency spectrum of normal noise to identify a sound is a noise or not.
2. As the Figure.12 shows the moment when switching the mode from the auto control mode to the manual control. When the GUI appeard, the CPU occupancy rate raised sharply from nearly 0% to 25%, which meant the GUI took much system resources while running. Due to the luck understanding of the Tkinter library, we still need to optimize our code to make it less resource intensive.
3. Now we can only store the audio in local when detecting a noise, in further study, we want to upload the audio to a server where the audio can be access through the Internet so people will know exactly what happened when the sound alarm alert by download the audio wherever has the access of the Internet.
ESP8266 Wi-Fi Breakout Adafruit
B. Parts List and Costs
|CAD U37 USB Microphone||Lab Stock||$45.00||1||$45.00|
|ESP8266 Wi-Fi Breakout||Adafruit||$9.95||1||$9.95|
|Miniature Wi-Fi Module||Adafruit||$11.95||1||$11.95|
|2.8-inch TFT Touch Screen||Lab Stock||$34.95||1||$34.95|
|DS18S20 Temperature Sensor||DigiKey||$4.95||1||$4.95|
|Servo Motor||Lab Stock||$11.95||1||$11.95|
|LED light||Lab Stock||$0.05||1||$0.05|