xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en" ECE 5725 | Gesture Controlled Music Box

Four by Four Matrix Design Ideology

Delin Cao

Design and Testing

With the idea of incremental development and test-driven development. Design and implementation phases are three:

 

Phase 1: Testing and implementing photo sensors

In this project, we used a type of infrared photo sensor named OPL530. This sensor receives infrared signals and flip its voltage level at its middle pin. Our first goal is to light it up. A simple circuit for only one sensor on protoboard was developed.

This circuit simply receives only sensor’s signal and transmitted to another GPIO pin, which toggles a LED.

After realizing one sensor, sensors matrix was designed for receiving more accurate hand signals. For forming the matrix, there are many factors that need to be considered. The distance between sensors (S), the range of light source (R) and the distance between hand and sensors (H) are three main factors.

By doing lots of experiments, we found that R is a flexible factor as long as there is direct full spectra light source right on the top of the sensors. Factor S needs to be with 2-3 centimeters to avoid conflict among sensors and keep considerable accuracy of array. Last but not least, H factor in this projects is in the range of 1-2 centimeters.

Our test is limited in the lab environment for this project. The results of factor are only reference. There should be other options for the factors.

 

Phase 2: Inventing algorithms for gesture analysis

Four by four design ideology

This design is to implement a 4x4 sensor matrix for scanning the gesture data continuously.

 

4x4 matrix have advantages of raw data accuracy and extendable ability. In my observation, with a 16 nodes matrix, this design is capable of reading more specific hand gestures rather than palm swipe. But the design difficulty is really high and needs far more time than our project limit.

To realize 4x4 matrix design, first thing to do is to collect groups of data from sensors and analyze the patterns of different gestures. After making enough observation of the data, a database for different patterns was written. Unfortunately, our time is limited and only a relatively small database is made for now. When the database is finished, similarity check was made between raw data and database to recognize specific gestures. For now, this algorithm is capable of recognize simple gestures like swiping in different directions and hold right above the matrix.



Phase 3: Designing user interfaces

A music box surely needs a friendly user interface. We design the graphical user interface with inspiration of mp3 players.  

On this screen, current song name, playing status and volume level are shown. And they change dynamically with the gesture control.

To realize the graphical thing, we chose pygame display for arranging layouts and status updates. And for the music playing part, pygame mixer was applied for music selection, volume control and so on.

Results

This design finished all the goals we set at the beginning. The photo sensor performs beyond our expectations which expedited our design process in no doubt. There is a pity that the accuracy of the gesture detection is not as sensitive as other commercial products performed.

Conclusion

In this project, the four by four deisgn realized our initial goals like recognizing song change, vlume setup, pause and start.

 

The 4x4 design is able to recognize simple gesture as cross one design capable of. What is more powerful is that, with a larger gesture pattern database, this design is going to learn far more gestures than just simple swiping.

Disadvantage of this design is obvious. First, huge amount of time will be needed to observe the data pattern and form database. Second, For receiving signals, nearly all of the GPIO pins are used, which caused lots of resource usage.

Further Work

If I have time, I'll have some extra thing to do. First thing to do is to solider all the sensors together. Since there are too many wires and connections in the circuits, the robustness is a huge problem. Making a firm connection will indeed decrease the error rate. Also, when the circuit is soldiered, there will be more experiments on the selections of factor H, R and S. Second thing we care about is optimizing our gesture recognizing algorithms. With higher accuracy, this project can further support more gestures than what we realized now.

 

Code Index

W3C+Hates+Me Valid+CSS%21 Handcrafted with sweat and blood Runs on Any Browser Any OS