Using a Brain Controlled Interface for Simple Robotic Motion
By Mira Bhut and Seetha Kolli.

Demonstration Video


An EEG, or electroencephalogram is a test that measures the electrical signals output by the brain through the use of electrodes placed on the head of the subject. EEGs were historically reserved for use in the medical field as a diagnostic test. However, in recent years, many engineers and have discovered that there exist many exciting applications for EEG technology in research, control engineering, and consumer products among many others. Thanks to the growth in this field, technology has not only improved, but so have resources for independent developers and price points of commercial headsets. One such headset, the MindFlex, is available at a very low cost as part of a fun family game. However we think that this headset is capable of much more than just a game controller. With “MindBot” we hope to show some of the possibilities that exist for use of EEG interfaces in modern technology and future advancements.

Generic placeholder image

Project Objective:

  • Hack a mind flex headset for general EEG use.
  • Show proof of concept for applications of low cost EEG technology.
  • Control a mechanical system using two distinct signals.


Our project was broken down into 4 main phases: Hacking the Mindflex headset, Obtaining Data via Bluetooth, and Data Processing (in the form of the robot), and the GUI. In figure 1 you can see the Mindflex headset with the attached bluetooth module. In figure 2 you can see the robot with the attached raspberry pi and TFT touchscreen.

Mindflex Headset

Fig 1: Headset with attached HC05

Hacking the Headset

Out of the box, the chip inside the mind flex headset is only set up to communicate with the board inside the base of the game. In order to communicate with the chip, we had to attach our own bluetooth module directly to the board. We got this idea from a tutorial on instructables.com. This involved soldering jumper wires to the TX, RX, V+ and GND of the chip. We then connected the V+ and GND accordingly; the TX of the chip went to the RX of our bluetooth module, and the RX of the chip went to the TX of our bluetooth module. During this process we considered multiple different bluetooth modules, including the HC06 and the Adafruit Bluefruit LE UART Friend. Between the two HC models, the HC05 is much more user friendly. The Friend is actually the most user friendly, as it is programmable via standard FTDI cables and comes with many accessible libraries. However, in order to dynamically switch between receiving and transmitting data to over bluetooth, it requires an external circuit. The Neurosky chip within the headset has multiple modes of data transmission, out of the box, the chip in the mindflex is automatically configured for mode 1, which is data that has already been processed via fast fourier transform and is transmitted at a baud rate of 9600. In order to get raw data, the chip needs to be switched to mode 2 and will transmit at a baud rate of 57600. One way to do this is to remove a surface mount resistor (see the Instructable for more information). Another way to do this is to transmit a hex byte from the pi to the Neurosky chip. We were wary of the first method as we did not want to irreperably damage the headset, but the second method does not survive power cycling. This means that we would have to transmit the byte every time the headset was powered up. For this reason, we felt it would be more prudent to go with a chip capable of dynamically switching modes.


Fig 2: Raspberry pi attached to robot


The trickiest part of implementation of the MindBot was in creating a reliable bluetooth connection that we could read the data from. After a significant amount of trial and error, the best route ended up being to create a socket connection through the rfcomm port using the RPi as a host and the HC05 as a client. There is no standard UUID protocol for the Neurosky chip, so we used existing mindwave libraries to read directly from the buffer, and altered the libraries as needed to fit our application.

Data Processing

In order to move the bot, we read the wearer's attention level and correlate that to a speed. We wanted to switch between moving straight and turning when the user blinks. When there is a peak the user has blinked. The first part of this involved reading in the data from the neurosky chip. Each data byte is proceeded by a hex row code indicating what type of data is contained within that byte. There are two row codes available for attention level and blink strength. Other row codes include raw data, meditation level, an EEG power spectrum, and a value indicating a power electrical connection between the electrode in the headseat and the wearer. The libraries we were using to process the data created an object with a class based on the row code for each data point. We would read in the byte, check the class, and if it was an attention level data point, we would update the speed and the gui then move the robot. If the data point was a blink, we would update the direction of one of the servos, then move the bot.


Fig 3: Start Screen

According to the servo datasheets, a neutral position occurs when the servo is sent a pulse width of 1.3 ms with a 20 ms gap between each pulse. This corresponds to a 7.5 percent duty cycle. Maximum clockwise rotation is achieved with a pulse width of 1.5 ms, which corresponds to an 8.5 percent duty cycle, while a maximum counterclockwise rotation corresponds to a 6.5 percent duty cycle. The attention level received has a range of 0 to 100. we multiply this percentage by the difference between the maximum speed duty cycle and the neutral duty cycle (1%) and add that value to the neutral signal. When we do this, we make sure that the servo continues rotating in the same direction, but this allows the speed to vary linearly with concentration level.

As we worked, we discovered that the mindflex headset is not configured to perform blink detection, meaning that although the chip is capable of doing so, the chip was configured so that it never output a byte with a row code corresponding to a blink. However the EEG power spectrum byte contains values representing each of 8 different types of waves, delta, theta, low alpha, high alpha, low beta, high beta, low gamma, and mid gamma. Upon researching, we found that a blink causes a spike in the amplitude of brain waves between the 8 types of brain waves. In order to detect a blink, we calculate the average value of the wearer's mind waves over the time they've been wearing the headseat and compare that to the current average of their brain waves. If the difference is above a certain threshhold, we consider that event a blink. Therefore we adjusted our algorithm to enter the blink detection function when we received a data point with a row code indicating an EEG power spectrum byte. The implementation of this code may be seen in the appendix.

One of the tough parts about the blink detection is that the data is received from the headset at a rate of roughly 1 Hz. This means if the user blinks in between data points being received, the data indicating a spike in brain activity will not be transmitted to the raspberry pi. This is a hardware limitation of the mindflex headset and could possibly be combatted using a more expensive model that is capable of continuous data streaming, however that would lie outside the scope of this project as our goal is to show the possiblities of a low cost module.


Fig 4: Screen displaying concentration level


The program starts as soon as the pi is powered up. So in order to make it easier for the user, we wanted them to be able to start the bot when they were ready. The start screen features two buttons that the user may use to either start the bot or to quit the application. In the event that the touch screen is in-operable, the user may use the physical buttons on the side of the pi TFT as well. Button #27 is to quit, while button #17 is to start. Once the user starts the robot, the default is for it to begin moving forward.

At this point we switch the GUI to display the current state of the user. We want the user to know their current attention level in order to help them focus, or unfocus, their thoughts in order to move the bot the speed they wish, so we print the attention percentage to the screen eery time it is updated. This also helps the user know which techniques to focus or distract themselves are working, such as focusing on the motion of the bot, doing math problems. or letting their mind wander. The quit button on screen is still available to exit the application, and the physical quit button, #27, still works in this mode as well.


We were able to successfully use the users attention level for variable speed control of the robot, with a statistically significant degree of accuracy in using blinking to toggle the degree of freedom of the motion. Moving forward, we want to use the raw data to perform more accurate blink detection, potentially using machine learning to create a blink threshhold that is more accurate to the current user as they wear the headset, as opposed to the current method which employs a global threshhold for all users. We would also like to add in features that display the state of all 6 of the users brain waves. We would also potentially like to build a maze for wearers to move the bot through and perform a study on how effective the module could be for brain training.

Work Distribution

Generic placeholder image

Seetha Kolli


Data processing and GUI.

Generic placeholder image

Mira Bhut


Hardware, Bluetooth Protocols and Interfacing, Testing.

Parts List

  • Raspberry Pi $35.00
  • PiTFT touch screen $35.00
  • MindFlex Headset - $17.00
  • HC05 Bluetooth Module - 8.99
  • LEDs, Resistors, Robotic testing platform and Wires - Provided in lab

Total: $99.50

Code Appendix

#!/ussr/bin/env python 

""" MINDBOT.py : 


Python script to stream data at 9600 Baud from wireless single channel 
EEG DEVICE, Mindflex. MINDBOT.py performs fourier transform of streaming data for 
feature extraction. It implements ML library PiScy to predict and classify EEG 
signals with high accuracy. MINDBOT.py utilizes multi-processing python library for


The python script will return and log all raw data points passed at 9600 BAUD in file EEG_RAW#
MINDBOT.py will also return and log all data points post feature extraction and classification. 
Lastly, utilizing the processed and extracted data, MINDBOT will continously control GPIO pins 
on Rasperry Pi for autonomous motor control of Mind controlled Robot. TFT Screen Implemented for 
on board console. 


This is the implementation for multi-processing feedback controls 
Based on the work of Xitang Xhao (xz522@cornell.edu) and Francis Yu (fy57@cornell.edu)
This model will use pipeline in which Master Process (stream raw data) is passed to Dual Master/Slave process to 
filter data, removing noise and eye movement signals. It is then passed to 3 Slave Processes to compute respective
feature extraction and identification. These contours are then sent to master process queue to control Robot feedback. 


__authors__ = "Mira Bhut (MAB586) | Seetha Kolli (SAK295)"
__course__  = "Cornell ECE Embedded OS 5725 | Joe Skovira (JFS9)"

## Importing Libraries

## LIBRARIES                            ## FUNCTIONALITY
#from multiprocessing import Process, Queue, Valve, Lock, Array

import pygame                           #GUI for TFT Display     "
import os                               #RPI3 OS: LINUX          "
import time                             #TIME CONTROL/REF        "
import subprocess                       #ROUTINE SUB             "
import RPi.GPIO as GPIO                 #RPI3 GPIO Pins           "
import bluetooth                        #HC05 and RPI3 BT        "
import RawReader                        #BT COMMUNICATION BUFFER "
from MindflexDataPoints import RawDataPoint, AttentionDataPoint, EEGPowersDataPoint
from MindflexDataPointReader import MindflexDataPointReader

mindflexDataPointReader = MindflexDataPointReader()             # CONNECT TO HC05
mindflexDataPointReader.start()                                 # START READER
dataPoint = mindflexDataPointReader.readNextDataPoint()         # READ NEXT BYTE

os.putenv('SDL_VIDEODRIVER', 'fbcon')
os.putenv('SDL_FBDEV', '/dev/fb1')
os.putenv('SDL_MOUSEDRV', 'TSLIB')
os.putenv('SDL_MOUSEDEV', '/dev/input/touchscreen')

GPIO.setup(27, GPIO.IN, pull_up_down = GPIO.PUD_UP)
GPIO.setup(17, GPIO.IN, pull_up_down = GPIO.PUD_UP)


#setting up screen
size = width, height = 320, 240
screen = pygame.display.set_mode(size)

#initialize fonts
largeText = pygame.font.SysFont('Comic Sans MS', 30)
smallText = pygame.font.SysFont('Comic Sans MS', 15)

#initialize colors
black = (0, 0, 0)
white = (255, 255, 255)
red = (255, 0, 0)
green = (0, 255, 0)
blue = (0, 0, 255)

# Define buttons
quitSM_area = (185, 50, 60, 20)
quitGM_area = (260, 220, 60, 20)
start_area = (65, 50, 60, 20)
butt_width = 0

attn = 0  #data(2)

# render text
start = smallText.render("START", 0, black)
quit = smallText.render("QUIT", 0, white)
#conc = largeText.render("Concentration Level: %d" %conc_level, 0, white)

# define text location
start_loc = (80, 55)
quit_loc = (203, 55)
conc_loc = (10, 110)

# initialize game state variables
st = 0 # indicates game has started

pos = (0,0)

period = 20
dc_stop = 0
freq = (1*1000)/period

dc_max = 0.2
dc_min = 0

blink = 0
oldSum = 0
countSum = 0
countUp = 0
#bl.theta = 0
#bl.lowAlpha = 0
#bl.highAlpha = 0
#bl.lowBeta = 0
#bl.highBeta = 0
#bl.lowGamma = 0
#bl.midGamma = 0

attn  = 0
ran = 100

#       SERVO1     #
# Pin13 as output PWM
GPIO.setup(13, GPIO.OUT)
servo1 = GPIO.PWM(13,freq)

#       SERVO2     #
# Pin19 as output PWM
GPIO.setup(19, GPIO.OUT)
servo2 = GPIO.PWM(19,freq)
def GPIO27_callback(channel):
        global start_time
        start_time = -100

def GPIO17_callback(channel):
        global st
        if (~st):
                st = 1


def blinkDetect(data):

        global blink
        global oldSum
        global countSum

        #find sum of all wave values
        curSum = data.delta + data.theta + data.lowAlpha + data.highAlpha
        curSum = curSum + data.lowBeta + data.highBeta + data.lowGamma + data.midGamma

        if (countSum == 0):

                #find time average
                old_mean = curSum/(8)
                old_mean = oldSum/(8*countSum)

        cur_mean = curSum/8

        diff = cur_mean-old_mean

        print"cur first then old"
        print cur_mean
        print old_mean
        #print countSum

thresh = 150000    ###set threshhold from testing
        if (abs(diff) >thresh):
                blink = 1
                blink = 0
                oldSum = oldSum + curSum
                countSum = countSum + 1

def startScreen ():   ## draw start screen ##
        global quit_butt
        global start_butt
        gameName = largeText.render("Mind Bot", 0, white)
        screen.blit(gameName, (110, 110))
        quit_butt = pygame.draw.rect(screen, red, quitSM_area, butt_width)
        start_butt = pygame.draw.rect(screen, green, start_area, butt_width)
        screen.blit(start, start_loc)
        screen.blit(quit, quit_loc)

def updateStatus(con): #data(2)
        global quit_butt
        conc = largeText.render("Concentration Level: %d percent" %con, 0, white)
        screen.blit(conc, conc_loc)
quit_butt = pygame.draw.rect(screen, red, quitGM_area, butt_width)
        screen.blit(quit, (278, 225))

def changeDir(val):     #change direction when user blinks
        global blink
        global dc2
        val = float(val)
        neg = -(dc2-7.5)
        dc2 =  7.5+neg
        val = int (val)
        blink = 0

def runBot(val):        #change bot speed based on effort
        global dc1
        global dc2
        global countUp
        val = float(val)
        if (dc2>=7.5):
                dc2 = 7.5+val/100
        elif (dc2<7.5):
                dc2 = 7.5-val/100
        dc1 = 7.5+val/100

        val = int(val)

        if (countUp % 10 ==0):
                countUp = 0
                #print val
                #print dc1
                #print dc2
                countUp = countUp +1

def stopServos():       #Stop servos
        global cur1
        global cur2
        cur1 = 0
        cur2 = 0

servo1.start(dc_stop)                                   # SERVO1 STARTS NEUTRAL
dc1 = 8.5                                        # CURRENT STATE: NEUTRAL
servo2.start(dc_stop)                                   # SERVO2 STARTS NEUTRAL
dc2 = 6.5                                          # CURRENT STATE: NEUTRAL

# set pins as callbacks
GPIO.add_event_detect(27, GPIO.FALLING, callback=GPIO27_callback, bouncetime=300)
GPIO.add_event_detect(17, GPIO.FALLING, callback=GPIO17_callback, bouncetime=300)

event = pygame.event.poll()

start_time = time.time()


while(time.time()-start_time <100):

        if(event.type == pygame.MOUSEBUTTONUP):
                pos = pygame.mouse.get_pos()
                if (start_butt.collidepoint(pos)):
                        pos = (0,0)
                        st = 1

                elif (quit_butt.collidepoint(pos)):
                                pos = (0,0)
                                start_time = -100
                                pos = (0,0)
                dataPoint = mindflexDataPointReader.readNextDataPoint()  #read datapoint

                if (dataPoint.__class__ is AttentionDataPoint):   #if it is an attention value
                        attn = dataPoint.attentionValue   # get the attention value
                #       print attn
                elif (dataPoint.__class__ is EEGPowersDataPoint):       # if it is a blinkvalue
                        #print bl.delta         # get the blink value
                        bl = dataPoint

                if (not dataPoint.__class__ is RawDataPoint):
                        print dataPoint
                if (blink):
                        print "you blinked"
                        changeDir(attn)         # uses attn because needs a servo speed after direction change, blink is global
                #if (not attn.__class__ is RawDataPoint):
                #       print attn
        event = pygame.event.poll()