Pi Game Player for iPhone!

Wen-Yu(ww424), Hanchen(hj424)

Our raspberry Pi

What Goals We want

- Build a robotic system to play mobile game app

- High accuracy (high scores) is guaranteed

- High speed to play the game apps is also guaranteed

Game app layout

Get to know Our Idea

Raspberry Pi is an ideal platform to build a small robot. We get the idea from Lab3 which we can build a robotic car on a raspberry pi system by writing some controlling programming. But, in Lab3, our robot does not have “eyes”. So, here is our idea – we want to build up a robot with eyes and make it play some simple game apps, such as Lumberman. Lumberman is a mobile game app, where you need to avoid the branches when you chop the wood. We implemented the algorithm to detect where the branches are and make our raspberry pi to chop the wood happily!

Keyword: Raspberry Pi, PiCamera, Image Processing, GPIO output control

Setup environment For PiCamera and Libraries

Before we start our project, we need to set up the environment for raspberry pi.


Sofeware and Hardware work together :)

Our project can be split into three categories: platform design, software design and hardware design. Platform design is to provide the spaces for our smartphone and the raspberry pi, also, make sure the relative position of the camera and our smartphone can be fixed. Then software part can be analogous as the mind of our raspberry pi. We apply the image-processing algorithm to detect the branches and decide where to chop the woods on the screen. Last but not least, we build the "hands" for the raspberry pi to trigger the touch events on the capacity screen of the smartphone when it needs to move to the other side. This project is lots of fun! Join us and see how each part works by scrolling down!

Now our Pi Game Player can play the game! Come join us and see if you can beat it!!


How Good We Achieved

Until now, out powerful Pi Game Player have reached a score up to 2855! It is pretty high which even beat one of the developers who is really good at this game (1000). So we think our robot can defeat any human players! Here we will introduce some timing specification of our awesome Pi Game Player to analyze why it can defeat human players quantitatively.



What strategies To improve it

Even if our Pi Game Player is able to defeat all human players, the accident game-over caused by miss tapping on the touch screen is an annoying but important issue for this project. If this problem can be resolved, our Pi Game Player can get an infinite score in theory. Another problem is that now our Pi Game Player is a little fragile which means its performance is not stable. Therefore this part is going to analyze the reasons of these problems and covers some methods to improve or resolve it.

What we have created / referenced For this project

Here is the code appendix for our project. We've created several versions of codes here.

Parts List

Name Quantity Price
Raspberry Pi2 B 1 35$
PiCamera Board Module 1 19$
H11F1 2 3.3$
Breadboard 2 5$
Wires+Resistors+Tape some 1$
Total 71.6$



PiCamera Document
Scipy installation
Python Image Library Document
The Python Imaging Library Handbook
ECE 5760: Final Project: Flappy Bird Player
H11F1 Datasheet
One-page bootstrap template


At last, We want to say ...

This project is lots of fun and challenging!! At first, the motivation for this project is that we want to build a system that can help us to play the game and get high scores. When developing this project, we found it challenging. The difficulties we faced includes environmental lighting condition, the stability of the tapping electrodes, the slow speed of pi-camera, the low-quality of images taken by pi-camera. We faced these difficulties and came up with the solutions accordingly, at last, we made it!! Combining the techniques of software and hardware, we help our raspberry create its eyes and hands. Finally, we made it can score almost 3000 points.



Wen-Yu Wang (ww424) :
In charge of software part, including developing the algorithm for detection of branches (testing and debugging), cooperating with hardware part, and the template, formatting of the report. She loves this project and she thinks this project is the most interesting project she has worked during her student life. In this project, she thinks they have learned how to developing a "project" and how to solve problems systematically. )


Hanchen Jin (hj424) :
In charge of hardware part, including the circuit design, testing and debugging. And he also provides some ideas and assistance for implementing the algorithm of detecting branches. Actually at first, He thinks it is easy to build the circuit to tapping on the touch screen, however, selecting material to build the stable hardware requires lots of experiments thus is really tough. Furthermore, he was impressed by the powerful Python libraries and he got some interests towards software development.

course staffs


Thanks professor and TAs for the helpful advices. In the process of developing this project, professor Joe gave us lots of useful information, great ideas and cared about us very much. We really appreciated that. And the advices from TAs were helpful as well. Thanks for their efforts which made this project more perfect. Besides, Wen-Yu wanted to give a special thank to her boyfriend for the creative idea for this project :)