Franklin


ECE 5725 SP 19 Final Project

Caitlin Stanton (cs968) and Brandon Quinlan (bmq4)

Objective

Many dog owners need to be away from the house for long periods of the day for work or school, and may worry about their companion while they’re out. The goal of Franklin is to alleviate some of these worries by keeping a dog happy and healthy automatically. Franklin has three features that can assist a dog owner in the daily task of caring for their pet.

Introduction

We designed and implemented three different systems for automatic dog maintenance. The first of which is a food refill system. At a pre-programmed time of day, the food tank will open up and release food into a bowl until its full, indicated by an IR sensor embedded in the bowl. The second part is a fetch playing system. We use a DC motor to accelerate a ping pong ball inside a cylindrical cage. Using a magnet attached to the rim of the cage and a hall effect sensor we can determine when the cage has reached its maximum speed. At the maximum speed we use a servo motor to lift the ball over the edge of the cage, causing it to fly in a random direction. The final system is an automatic petting arm. When something comes within range of the arm, it begins making a petting motion. The arm has to joints, giving two degrees of freedom. The three parts combined compose a comprehensive system for automatic dog maintenance.

Figure 1: System diagram for Franklin

Design and Testing

Construction

We built our project physically in the same pattern that we wrote our software, incrementally. We started off with the food refill system, for which a few different physical components were needed. The first was a reservoir to hold the food that will be used to refill the bowl. This tank has to be larger than the food bowl so it can hold multiple refills at once, and also had to be shaped in a way that would facilitate easy dispensing of its contents. Our original plan was to make this tank out of wood, however to save time and material cost, we decided instead to use a 2 liter soda bottle, with both ends cut off. The next step was to build a sort of valve which we could use to control whether food is allowed to leave the tank or not. To fill this purpose we cut a hole into the bottom of the bottle and embedded a mini servo motor into it. We then made a small door out of index cards and attached it to the arm of the servo. Then by controlling the position of the servo motor we could either allow the food to fall into the bowl or not. For the bowl, we cut the bottom off of a cardboard box. We then screwed a pair of IR sensors directly across from each other, one emitter and one receiver. The line between the two is broken when the food fills above that height of the bowl, and we know we can close the door again.

For the fetch part of our project, the first step of the construction was to create cylindrical piece to place the ball in as it accelerates. Rather than design one, we decided to buy a squirrel cage blower on the suggestion of Prof Skovira. To keep the ball in place as the cage accelerates, we cut a piece of cardboard to size and wedged it into one of the slits of the cage and secured it with tape. We also built a small platform to keep the ball level inside the cage because the original bottom of the cage was sloped. We found that this change helped the ball move more freely to the wall we had built in when the cage was spinning. In order to lift the ball up over the edge of the cage after reaching top speed, we cut a hole in that platform and replaced it with a small lever attached to a servo. With this adjustment the ball would now fall onto the lever instead of remaining on the platform, so we could lift the ball out of the cage using the servo motor. To control the servo we decided to use a Raspberry Pi Zero, which would communicate wirelessly to our other Pi. The only space that the Pi Zero would fit was on the outside of the cage, so we taped it to the side with electrical tape. To power this Pi we attached a portable battery pack with a USB power connection. However the battery was fairly heavy, so we decided to counter-balance it with another battery on the opposite side of the cage. We decided that the other components were light enough to not cause much of a balance issue.

To actually spin the cage, our plan was to use the motor of a box fan. However we found that it was difficult to securely attach the cage to the motor, and additionally the motor struggled to actually spin the cage when it was attached by itself. Instead we found that we could tape the cage to the center of the fan blades which worked very well. To cover up the fan blades, we decided to put the motor back into the fan and just replace one side of the cover with cardboard, with an opening big enough for the cage attached to the motor to protrude. The body of the fan also gave us a base to build the rest of our project off of. We attached a plank of wood to the side of the fan and then attached the food tank to that, and placed the bowl underneath.

For the last part of the project, the petting arm, we used a long strip of cardboard which we cut into two pieces are the arm, and two standard servo motors as the joints. We used the cross-shaped attachment for the servo and screwed it to the cardboard. We were worried that the servo might not have the torque required to hold up both arms and the other servo, so we made the “upper” arm shorter to reduce the required torque. We attached the full arm to the body of the fan on the corner near the food tank, with the idea that a dog could be pet as it eats.

Figure 2: Finished Franklin

Notifications

Every action in Franklin provides a notification to the user with regards to whether the food is being refilled, your dog is playing fetch, or Franklin is petting your dog. This is all done with the Twilio library, which provides the ability to send SMS messages to any phone number. The library itself is very simple to use (the tutorial is linked below in “Resources”) and only required a Twilio account to get a free phone number from which to send the SMS messages. The overview of messages.py is that an authorized connection to the Twilio Client object is made using a valid account’s SSID and authorization token. Once that’s been done, messages can now be sent using a REST API call.

The only drawback is that messages on a free Twilio account can only be sent to the phone number on the account itself. Additionally, Twilio can be easily hacked if your authorization token and SSID are posted publicly (i.e. on GitHub)—we learned this the hard way. Make sure to keep your tokens safe, whether through encryption or a private code repository.

Food Refill

The idea behind the food refill system is that there can be a specific time hard coded into a python script which will open the food tank at a specific time, and leave it open until the food bowl is full. To get the time, we use the datetime Python library. We call the datatime.now() function, and extract the hour and minute from the tuple that gets returned and compare to the pre-programmed time. If they match up we open the tank by changing the PWM signal using the RPi.GPIO library. Once opened, we begin polling the GPIO pin connected to the linebreak IR receiver. Once we read that the pin value is low, indicating the bowl is full, we again change the PWM signal to the original value.

Figure 3: Food dispenser system

We tested the different parts of this system incrementally, beginning with correctly identifying a specific time. We then moved on to calibrating the PWM to move the servo to the correct positions. Once we had the servo opening the tank at the correct time, we started working with the IR sensors. Again we begin by just seeing if we could identify a break by printing to the display. Once this was working we integrated it into the rest of the code so we could change the PWM signal upon the line break.

Fetch

The system we built to implement the fetch functionality of Franklin involved a motorized container to spin and catch the ball, as well as a mechanism to throw the ball out. The former was accomplished with a cage, typically used in cars, attached to a motor from a box fan. With a little extra cardboard and duct tape, the cage was made to effectively contain the ping pong ball by maintaining it at a height level that wouldn’t get the ball thrown out naturally while also allowing some range of motion to help our mechanism throw the ball. This mechanism involved a triangle of cardboard attached to a mini servo located in the central axis of the cage.

The flow of the system first started with turning on the fan motor to get the cage spinning. Attached to the cage was a stack of magnets, and these were used to determine how fast the cage was spinning. As the cage spun, these magnets passed by a hall effect sensor, which outputs a voltage based on the presence of a magnetic field. If a magnetic field was detected by the hall effect sensor (within a range of about two centimeters), it would output a low voltage into one of the GPIO pins of the Pi 3.

Figure 4: Given circuit from hall effect sensor datasheet

Every time the hall effect sensor registered a magnetic field, the time at which this occurred was stored in a two-element array. This array was continually updated to hold the two most recent times the sensor was triggered. These readings allowed for us to calculate the velocity of the cage using our function calculate(); classical mechanics tells us that velocity is equal to distance divided by time, so we only needed those two values at any point to calculate. The distance was simply the circumference of the cage, found to be 15.24 centimeters. The period was the difference between the two times stored in the array mentioned earlier.

Figure 5: Close-up of soldered circuit

The calculate() function was called using fetchCallback(), a GPIO callback function attached both the rising and falling edge of GPIO 12, the pin connected to the output of the hall effect sensor. This callback function was linked to both edges to increase the efficiency of the hall effect sensor—later on in the project, we noticed that if many sensors and servos were running simultaneously, the hall effect sensor would only trigger if no magnet was present. Having the callback function triggered on both the rising and falling edge could allow for us to change the code around and have all of the functionality surrounding fetch be initiated when the hall effect sensor left the stack of magnets, not when it first saw it.

Within fetchCallback() the velocity of the cage was calculated and then a conditional was placed on it to see if the conditions were right to throw the ball. In our case, that was done by checking to see if the global variable velocity was holding a value greater than 200 cm/s. If that was true, then a series of steps were taken to throw the ball.

While the main Pi, our Pi 3, was running the code in fetch.py to read the hall effect sensor and calculate the cage’s velocity, there was another Pi on Franklin running its own program, client.py—this was a Raspberry Pi Zero W, a smaller Pi with WiFi functionality. The Pi Zero was headless, meaning that it wasn’t attached to a keyboard, mouse, monitor, or Ethernet connection. Instead, it was hooked up to a handheld power supply and attached to the side of the cage. Using an additional Pi meant that the servo within the cage could be controlled without the fear of tangled wires or broken equipment.

The sole purpose of the Pi Zero was to trigger the micro servo, specifically an SG90, and throw the ball. However, our design relied on the Pi 3 to determine if the cage was spinning at a high enough rate. This meant that some sort of data connection needed to be established between the two: this was done using a socket. Sockets allow connection between two nodes, one listening and one establishing the connection for data transfer, on the same network to communicate with each other. The Pi Zero was the server, meaning that it opened up a specific port and was waiting for a connection, while the Pi 3 was the client, the node that would reach out to establish the socket connection. Once client.py was run, it bound itself to port 5725 and waited for a connection, which would be created by the Pi 3 saw that the cage was spinning at a rate faster than 200 cm/s. The socket connection occurring then alerted the Pi Zero that it was time to flip the ball out of the cage, and the micro servo was moved using a PWM signal.

Figure 6: Rotating cage with Pi Zero attached

Petting Arm

This feature of Franklin was the simplest to implement: it only needed to detect if a pet was nearby and then respond accordingly. We used a Pololu digital distance sensor to output a low voltage if it saw an object within the range of two to ten centimeters. Similar to the fetchCallback() function, the GPIO pin connected to the distance sensor’s output was set to call the function petCallback() on both the rising and falling edges of its output. Within that function was a while loop to check and see if the voltage output was still low; this was done to ensure that the petting motion of our arm was continuous while the object was still in range. The motion of the arm was controlled using two standard parallax servos, each connected to a separate part of the arm. The lower servo had a more limited range of motion, oscillating between PWM pulse widths of 1.45 ms and 1.55 ms, while the upper servo switched between 1.3 ms and 1.7 ms.

Figure 7: Petting arm with distance sensor circuit

Results

We were glad to be able to implement all of the systems we had originally outlined in our project proposal. We were also able to extend this to include the text notifications using Twilio. While all of the systems did work, not all of them worked exactly as intended. For our demo, we used balled up pieces of paper in place of dog food to show the food refill system. However, we found that the balls frequently clogged up in the tank and wouldn’t fall out even if the tank was open. Additionally, when anything broke the line the tank would close, even if it was just a stray ball. We have a few ideas that remedy this observations discussed in the Future Work section. For the fetch system we were actually pleasantly surprised that it worked as well as it did. We expected it to be the hardest part of our project, which we were right about, but we were very pleased with how we were able to throw the ball. However we were disappointed that we could not make it fully automated. We had wanted the motor to start up automatically, in response to a sensor. We were not able to separate the power cord from the fan, which meant we had to manually switch it on and off. With respect to the petting arm, we always expected that it would be difficult to replicate the motion of a human arm using two servos, but we were pleased with the results that we got.

Our demo video, outlining what we were able to accomplish with Franklin, can be found below:

Conclusion

While we were able to get all of the different systems working in isolation, we were disappointed not to be able to integrate all of the systems together. We found that because of the way we read the time for the food refill system, we could not attach that trigger to a hardware interrupt as we were for the other sensors. This meant we had to poll the time. We found that this caused issues with the PWM signals to the servos, causing a lot of jitters, to the point where they were no longer really usable. We also found that it caused issues with the reading from other sensors, in particular the hall effect sensor, meaning that we were not able to integrate all of the systems together. Otherwise everything worked as expected.

Future Work

If we had more time to work on this project, our first goal would of course be integration of all of the different parts. We think that the piece holding back the integration was the polling of the time for food refill. To get around this our idea is to use the crontab to schedule a task which would open the tank and wait for the IR beam to be broken. We suspect that this would lower the computational requirements such that they wouldn’t interfere with the PWM or other GPIO readings. This should allow us to integrate the different systems more cleanly.

We also would like to improve the functionality of a few of the systems themselves. The first is that we would change the way the food tank closes. We want to change it so that the tank will only close if the beam stays broken for some contiguous amount of time. This would be a better indicator that the bowl is actually full, and be more robust to a stray piece of food falling in front of the beam.

One other thing we would like to do is try to implement our stretch goals. One such goal was to integrate a camera into our system that would allow a pet owner to see a live feed of their dog. This would have involved many additional steps, namely getting the camera working, getting the Pi able to communicate the camera using WiFi, and then developing an app that a user could download to access the data. With more time, we would have liked to finish integrating all of these systems together and making these improvements.

Code Appendix

All programs used are located below or in our Github repository.

feed.py

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
# Brandon Quinlan (bmq4) and Caitlin Stanton (cs968)
# ECE5725, Final Project

import datetime
import time
from twilio.rest import Client
import RPi.GPIO as GPIO
import messages

# Update PWM signal of p
def change_PWM(on_time, p):
    freq = 1000.0/(20.0+on_time)
    dc = 100.0*(on_time/(20.0+on_time))
    p.ChangeFrequency(freq)
    p.ChangeDutyCycle(dc)

# Set up GPIO
GPIO.setmode(GPIO.BCM)
GPIO.setup(27, GPIO.IN, pull_up_down=GPIO.PUD_UP)
GPIO.setup(13, GPIO.OUT)
GPIO.setup(5,GPIO.IN, pull_up_down=GPIO.PUD_UP)

# Define constants
GPIO_pin = 13
on_time = 1.2
freq = 1000.0/(20.0+on_time)
dc = 100.0*(on_time/(20.0+on_time))
p = GPIO.PWM(GPIO_pin, freq)
p.start(dc)

# Initialize desired food time
food_time = (18,24)
current_time = (datetime.datetime.now().time().hour, datetime.datetime.now().time().minute)
while (food_time != current_time):
    # wait for feeding time
    current_time = (datetime.datetime.now().time().hour, datetime.datetime.now().time().minute)
messages.send_sms("Time for food!")

# open tank
change_PWM(2, p)
while GPIO.input(5):
    # wait for line break
    pass
# Close tank
change_PWM(1.2, p)
time.sleep(3)

p.stop()
GPIO.cleanup()

fetch.py

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
# Brandon Quinlan (bmq4) and Caitlin Stanton (cs968)
# ECE5725, Final Project

import time
import datetime
import RPi.GPIO as GPIO
import math
import socket
import messages

GPIO.setmode(GPIO.BCM)

# Variables for fetch
timeVal = [0,0]
diameter = 15.24 #centimeters
circumference = math.pi*diameter
numPasses = 0
velocity = 0 #cm/s

# ISR to be called
def fetchCallback(channel):
  # Called if sensor output changes
  stamp = time.time()
  if GPIO.input(channel):
    # No magnet
    print("Sensor HIGH")
  else:
    # Magnet
    print("Sensor LOW")
    calculate(stamp)
    global velocity
    if velocity > 200:
        print "hall"
        # Connect to Pi zero
        messages.send_sms("Playing fetch!")
        s = socket.socket()
        s.connect(('10.148.12.144',5725))
        print "connected"
        server = s.recv(1024)
        s.close()
        velocity = 0

# Update velocity
def calculate(magnetPass):
  global numPasses
  numPasses = numPasses + 1
  timeVal[0] = timeVal[1]
  timeVal[1] = magnetPass
  period = timeVal[1]-timeVal[0]
  global velocity
  velocity = circumference/period

def main():
  # Wrap main content in a try block so we can
  # catch the user pressing CTRL-C and run the
  # GPIO cleanup function. This will also prevent
  # the user seeing lots of unnecessary error
  # messages.

  # Get initial reading
  fetchCallback(12)

  try:
    # Loop until users quits with CTRL-C
    while True :
        time.sleep(0.1)

  except KeyboardInterrupt:
    # Reset GPIO settings
    GPIO.cleanup()

# Set Switch GPIO as input
# Pull high by default
GPIO.setup(12, GPIO.IN, pull_up_down=GPIO.PUD_UP)
GPIO.add_event_detect(12, GPIO.BOTH, callback=fetchCallback, bouncetime=200)

if __name__=="__main__":
   main()

client.py

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
# Brandon Quinlan (bmq4) and Caitlin Stanton (cs968)
# ECE5725, Final Project
# Python script to be run on the Pi Zero
import socket
import RPi.GPIO as GPIO

servo = False
s = socket.socket()
# Connect to Pi 3
def connection():
    global s
    s.bind(('',5725))

# Setup GPIO and PWM
GPIO.setmode(GPIO.BCM)
GPIO.setup(3,GPIO.OUT)
freq=1000.0/21.5
p=GPIO.PWM(3,freq)
dc=100.0*(1.5/21.5)
p.start(dc)

# Wait for connection
def waiting():
    s.listen(5)
    c,addr = s.accept()
    print "got connection %s %s" %(c,addr)
    global servo
    servo = True

# Change servo position
def playFetch():
    print "yeet"
    p.ChangeFrequency(1000.0/21.0)
    p.ChangeDutyCycle(100.0*(1.0/21.0))

if __name__=="__main__":
    connection()
    while True:
        if servo:
            playFetch()
            servo = False
        else:
            waiting()

petting.py

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
# Brandon Quinlan (bmq4) and Caitlin Stanton (cs968)
# ECE5725, Final Project

import RPi.GPIO as GPIO
import time
import messages

# Set up GPIO
GPIO.setmode(GPIO.BCM)
GPIO.setup(20,GPIO.OUT)
GPIO.setup(21,GPIO.OUT)

# Set initial PWM
on_time = 1.5
freq = 1000.0/(20.0+on_time)
dc = 100.0*(on_time/(20.0+on_time))
pet1 = GPIO.PWM(20, freq)
pet2 = GPIO.PWM(21,freq)
pet1.start(dc)
pet2.start(dc)

# Changes PWN signal of p to be high for "on_time" microseconds
def change_PWM(on_time, p):
    freq = 1000.0/(20.0+on_time)
    dc = 100.0*(on_time/(20.0+on_time))
    p.ChangeFrequency(freq)
    p.ChangeDutyCycle(dc)

# ISR for petting arm
def petCallback(channel):
  motion2 = [1.3,1.7]
  motion1 = [1.45,1.55]
  i = 0
  messages.send_sms("Pet in range")
  while not (GPIO.input(channel)):
      change_PWM(motion1[i%2],pet1)
      change_PWM(motion2[i%2],pet2)
      i = i + 1
      time.sleep(0.2)

def main():
	petCallback(19)

	try:
	  # Loop until users quits with CTRL-C
	  while True :
	      time.sleep(0.1)

	except KeyboardInterrupt:
	  # Reset GPIO settings
	  pet1.stop()
          pet2.stop()
	  GPIO.cleanup()

# Set up ISR
GPIO.setup(19,GPIO.IN,pull_up_down=GPIO.PUD_UP)
GPIO.add_event_detect(19,GPIO.BOTH,callback=petCallback,bouncetime=200)

if __name__=="__main__":
   main()

messages.py

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
# Brandon Quinlan (bmq4) and Caitlin Stanton (cs968)
# ECE5725, Final Project

from twilio.rest import Client

account_sid = 'AC4bda313b35fa302d7ec0e1ae151c8505'
auth_token = 'aff8936c41459ba1b0f98452ead78c4d'
client = Client(account_sid,auth_token)

# Send "text" to Brandon's phone
def send_sms(text):
	message = client.messages.create(
		body=text,
		from_='+17326482248',
		to='+17326093709'
	)
	print(message.sid)

References

Work Distribution

During the implementation of this project, Caitlin worked on the code and circuitry for the fetch and petting systems, as well as the Trilio communication. Brandon spent more time working on the construction of the physical components of the lab and the code for the food refill system. For the report we each wrote about the parts that we worked on and Brandon wrote the other parts while Caitlin did the HTML formatting.

Materials and Budget

Meet the Team

Here's the team behind Franklin! To the left is Brandon Quinlan, senior double ECE/CS major, and to the right is Caitlin Stanton, junior ECE major.

Project Name

Lorem ipsum dolor sit amet, consectetur adipisicing elit. Mollitia neque assumenda ipsam nihil, molestias magnam, recusandae quos quis inventore quisquam velit asperiores, vitae? Reprehenderit soluta, eos quod consequuntur itaque. Nam.

Close Project

Project Name

Lorem ipsum dolor sit amet, consectetur adipisicing elit. Mollitia neque assumenda ipsam nihil, molestias magnam, recusandae quos quis inventore quisquam velit asperiores, vitae? Reprehenderit soluta, eos quod consequuntur itaque. Nam.

Close Project

Project Name

Lorem ipsum dolor sit amet, consectetur adipisicing elit. Mollitia neque assumenda ipsam nihil, molestias magnam, recusandae quos quis inventore quisquam velit asperiores, vitae? Reprehenderit soluta, eos quod consequuntur itaque. Nam.

Close Project

Project Name

Lorem ipsum dolor sit amet, consectetur adipisicing elit. Mollitia neque assumenda ipsam nihil, molestias magnam, recusandae quos quis inventore quisquam velit asperiores, vitae? Reprehenderit soluta, eos quod consequuntur itaque. Nam.

Close Project

Project Name

Lorem ipsum dolor sit amet, consectetur adipisicing elit. Mollitia neque assumenda ipsam nihil, molestias magnam, recusandae quos quis inventore quisquam velit asperiores, vitae? Reprehenderit soluta, eos quod consequuntur itaque. Nam.

Close Project

Project Name

Lorem ipsum dolor sit amet, consectetur adipisicing elit. Mollitia neque assumenda ipsam nihil, molestias magnam, recusandae quos quis inventore quisquam velit asperiores, vitae? Reprehenderit soluta, eos quod consequuntur itaque. Nam.

Close Project