ECE 5725 Smart Trash Can

Software Design 

As we mentioned in the hardware section, we have 1 ultrasonic sensor outside the trash can measuring the distance between the trash can and any object in front of the trash can; we have 3 sets of IR break beam sensors measuring how full the trash can is. When the ultrasonic sensor reads an input less than our threshold value, we will open the lid of the trash can and close the lid when the user has left. The 3 sets of IR break beam sensors were placed at a different level inside of the trash can. If the trash block the beam and the sensors will know that something is in the trash can. When the trash can is fairly empty, only the sensor placed at the bottom will be blocked and change its status; when the trash can is full, all 3 sets of sensors will be blocked. We used a PiTFT to display the trash can's emotion based on how full the trash can is from sensor readings. When the status of sensors change, our PyGame program will read the status and draw emojis and status bar based on the reading. 

For our voice recognition system, we used a software called SOPARE which is capable of performing offline speech recognition on the Raspberry Pi. We trained SOPARE with 2 hotword commands "open" and "close", by saying the corresponding command to the microphone and labeling the command. SOPARE will covert our voice into chunks of arrays representing our voice characteristics (tone, volume, pitch and etc.). We trained each word 15 times and each word had 15 voice records. In the recognition phase, we spoke to the microphone and SOPARE would still convert the voice to chunks of arrays. Then it would go to its record and tried to find a match with the current input. We set an 87% threshold for the similarity of records to be considered a match, in other words, only if the similarity of the current input and an existing record has 87% match, the current input will be considered to be a corresponding hotword. 

The system would ideally return either ['open'], ['close'] or [' '] in an array named "readable_result". [' '] means no match found with ['open'] and ['close'] are our hotwords. However, SOPARE would detect consecutive words as a sentence. We might have something like ['open', 'open', 'close'], ['close', ' '] or [' ', 'open'] as a result if we speak too fast or the testing environment is noisy. These command result caused problems in our system because the system would confuse and not performing actions due to extra words in the readable_result. Instead of only detecting the first word (array index 0) and use that as the command, we wrote something like:

if (' keyword' in readable_result):

    do the first matched keyword_action

and this solved our problem by reducing the impact of extra words recorded.