xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en" ECE 5725 | Gesture Controlled Music Box

Gesture Controlled Music Box

Delin Cao, Tianze Jiang


In this project, we designed and implemented a gesture controlled music box on a Raspberry Pi Kit. The music box uses photo detectors for receiving hand gestures to further implement functions like music playback, pause, start and volume control. With self-invented algorithms, this music box has good human interactions.



The goals of this project are two: gesture detection and analysis, music box interface on R-Pi.

To detect gestures there are two steps that needs to be done: gesture signal receiving and gesture pattern analysis. We chose infrared photo sensors for signal collection. And self-invent algorithms were applied for pattern recognition.

The music box is implemented with pygame on R-Pi for user interface display and music playback. The core of this part is music play. Pygame mixer was chosen for music control. It’s clear and simple to realize.

Two Designs

During the initial phase, we have different ideas about algorithms inventing and sensor matrix forming. Hence there are two complete designs for the project. We made parallel developments. And both of them are functional well.


Design 1: Four By Four Sensor Matrix Design


Design 2: Cross One Dimensional Sensor Matrix Design


Here is one crazy drawing from the very beginning of our brainstrom step. We thought about two-dimensional and three-dimensional models and also different sensor selections. And finally our design worked out.

Crazy Drawing


Group members (alphabet order):

Delin Cao                                 dc884@cornell.edu

Tianze Jiang                             tj273@cornell.edu


Course Advisor:

Joe Skovira                               jfs9@cornell.edu


First thanks to our course advisor, professor Skovira. He help us with ideas comming up, model formation and python support.

Especially thanks to our teaching assistant, Jingyao Ren

W3C+Hates+Me Valid+CSS%21 Handcrafted with sweat and blood Runs on Any Browser Any OS