Alarm robot

A project by Juexiao Wang (jw2534), Joo yeon Chae (jc2464)

Objectives

-A smart alarm robot that plays music and moves in random direction at given time
-A piTFT display and touchscreen quit button
-A friendly webAPI for musician selection and setting alarm time
-Music selection based on weather information

Introduction

Are you a late sleeper who somehow misses or turns off ten phone alarms? We introduce you to Alarm Robot system built in the Cloud, which wakes you up in an entertaining and effective way. Users are allowed to select alarm music of your favorite artist and set the alarm time on the OFFICIAL Website before they go to bed. The robot will wake up in time in the morning playing music according to the Real-Time weather condition. With music, everyone can have a good mood towards a brand new day ! In order to turn the alarm off, users have to catch the robot moving in random direction and press on the quit button on the touchscreen. On the screen, user would also find the weather, the alarm time, and the song information on the alamr. After nice stimulation from music and some body movement, you would be ready to start off fresh morning.

Hardware Design

Design Overview

The alarm robot had four peripherals connected: a IR distance sensor (Sharp GP2Y0D810Z0F), two servo motors (Parallaz Standard Servo), a Real Time Clock (PCF 8523), and a speaker. The hardware modules were connected as below. The IR distance sensor was used to detect an object in front of the robot to avoid collision during random movement. The servo motors were use to move the robot with two wheels. The real time clock was used to get accurate time information on real time basis. There were two different voltage sources. One was 5V battery pack, which was connected to the IR sensor and te servo motors. The other one had 3.3V output and was connected to power the Raspberry Pi and the real time clock. The signals from the IR sensor and the motors were connected to the GPIO pins of Raspberry Pi. The signal output for the servos were connected to resistors for safety. The real time clock was communicated with the Raspberry Pi using I2C. The SDA and SCL ports of the RTC were connected to the SDA and SCL ports of Raspberry Pi. The speaker was used for alarm music output and was connected to the 3.5mm audio jack.

Implementations of the Alarm Robot System

Real Time Clock

Raspberry Pi gets time from the internet. Unfortunately, the internet connection of Raspberry Pi is not the best. In order to ensure that we get the most accurate time, we integrated PCF8523 Real Time Clock into Alarm Robot. The real time clock used I2C to communicate with RPi. We tested the connection with Ã¢ÂÂi2cdetect -y 1Ã¢ÂÂ command. We had to load modules and add some configurations following the tutorial on the adafruit website (the second link in the Reference section). After the configuration, the system time was automatically synced to the real time clock at boot. We could also get the time directly from the real time clock using Ã¢ÂÂsudo hwclockÃ¢ÂÂ commands.

Power Save Mode for Alarm Clock

Register Overview from the Reference.

Our original plan was to use the interrupt signal from the real time clock and a relay circuit to power up the Raspberry Pi in order to save power over night. PCF 8523 Real Time Clock has a SQW pin which is used as either CLKOUT or INT1. We changed the value of COF control bit of Tmr_CLKOUT_ctrl register to 111 to disable CLKOUT and, therefore, enable interrupt signal. Setting the AIE bit of Control_1 register changed the interrupt signal to follow the status of big AF, alarm flag. This means that the interrupt signal is high when it is not alarm time yet and becomes low after the alarm time. We used CircuitPython library and adafruit_pcf9523 library for modifying registers and testing. The register information was from the datasheet in the Reference.

Power Cord Splitted

Power Control Circuit

With power cord connected to the relay, we were able to power on and off Raspberry Pi using a voltage input signal. We were also able to observe active low interrupt signal using an oscilloscope. However, the voltage level of the high output of the interrupt signal was not high enough to move the switch of the relay. It would have been possible to implement the power save mode with slight changes to the hardware. We could either use a transistor that matches the voltage of the high interrupt signal from RTC or use another RTC with higher interrupt signal output.

Software Design

Elements in the Alarm Robot System

The Alarm Robot system is built on Amazon cloud platform based on Flask web application framework. There are six main elements communicating with each other through http protocal, Amazon EC2, Amazon DynamoDB, weather server, website hosted on EC2 and Raspberry pi.

Amazon EC2 is the center controller of the system, which will response to the http request from the other five elements in the network.
A relational database is stored on Amazon DynamoDB, which is composed of four attributes, weather,singer, songtitle and music url.
The weather server is a weather api called OpenWeatherMap. User can send http request to the server and get back real-time weather information including temperature, weather condition.
An official website for the alarm robot is designed and hosted on Amazon EC2. User can set their preferences of favorite singer, alarm time via this visualized GUI interface. and click set your robot button, according to which a http POST request will be sent back and stored on EC2.
Raspberry pi serves as the client of the system, which will send gettime, getmusicinfo, gettempinfo, playmusic requests to EC2. The response result will be stored on Raspberry pi and prepared for piTFT display and sleep time.

Implementations of the Alarm Robot System

System built in the Cloud

The whole system is built on Amazon EC2 cloud platform. An instance is lauched under Amazon Linux AMI environment. To start with, basic dependent packages will be installed on the instance, such as python, pip, requests, boto3, flask. Then a server is developed under Flask web application framework. There are several responses for the HTTP requests are defined, such as a request to retrieve music in the database according to weather information and the selected singer. The weather information is the response from the OpenWeatherMap weather API. A request from user to set the alarm time and their preferred singer via the website. A request to play music according to the url retrieved from the database and a request from Raspberry pi to get the alarm time. A request from Raspberry pi to get the weather and music information.

Real-Time Weather Information

OpenWeatherMap is a weather api providing users with flexibility of watching for current temperature and weather conditions in your city or any other location on the interactive global map. We choose to access the weather information according to the zipcode of the area. A HTTP GET request containing the zip code of Ithaca is sent to the weather api. The response of the weather information containing the temperature, humidity, weather condition etc is returned in JSON format. We extract the values of two keys in the response, the temperature and weather conditon. To narrow the size of the database, the weather condition is classified into three catogories, Clouds, Clear and NG. NG is the shortcut for NotGood, which represents a broad range of weather conditions including rainy, snowy, windy and so on.The selected weather information is stored in the query preparing for the retrievel of music information in the database.
.

Relational Database in the Cloud

The relational database containing weather and music information is built on Amazon DynamoDB, which is a key-value and document database that delivers single-digit millisecond performance at any scale. Each entry in the relational table has three attributes, weather, singer and info. The weather attribute serves as the partition key and the singer attribute serves as the sort key. The info attribute contains the title of the music and the YOUTUBE url of the music. To select music according to the weather, we define a temperature range [-2,4 ] corresponding to [-20,40] degree centigrade. Taken the three weather conditions in to consideration, there are 21 combinations of weather conditions in granularity. Each combination should relate to a song of specific style to balance the mood of the user in every morning. For example, when the weather condition is NG and it's in the temperature range [-2,-1], a pop rock music such as Shake it Off by Taylor Swift should be played to stimulate the secretion of epinephrine of user, thus achieving the goal of refreshing ! Also, as we offer two singers Taylor Swift and Linkin Park for user to select on the website, there should be 42 entries in the relational table. The music in each entry is well selected based on Psychology lol.

Official Website hosted in the Cloud

The official alarm robot website is designed and hosted on Amazon EC2. Here is the ip address of it. When user open search for the ip address, EC2 will render template of the designed website in response. Before bedtime, users can select their favorite singer(though there are only two options...we will try to build a larger database in the future), set their alarm date including year, month, date(wake up the sleeping beauty a thousand years later...), set their alarm time. After finishing all the settings, users can click the set your robot button to send a HTTP POST request to EC2 to store the information, which will be a component in the query.

Raspberry pi as a Client

As the Alarm robot system is built in the cloud platform, the system is rather portable and flexible. In this project, Raspberry pi will serve as the client and utilize the system to realize several fancy alarming functionabilities with piTFT and servos. As a client, Raspberry pi will send HTTP request to EC2 to get the information of alarm time, weather information, music information, which will be displayed on the piTFT in real-time. The alarm time will be compared to the present time, which is obtained from the real time clock, to calculate the sleeping time of the raspberry time. After the Raspberry pi is waken at the alarm time, two_wheels program will be executed to motivate the servo to move randomly and avoid obstacles with IR sensor. Music will be downloaded from YOUTUBE and played on the OMXPlayer.

Results

Snapshot of our official website !

This is our official website designed and hosted in the cloud, have a Try by Clicking: Alarm Robot

Here is our Robot !

The Alarm Robot system turns out to be consistent with our expectation. The standard operation workflow is as follow: user open the official website, select their favorite singer and set the alarm time by clicking the set your robot button. Raspberry pi running client program is keeping requesting time, weather and music information until the time is set on the website. At the specific alarm time, the robot will wake up and play the music of the selected singer and real-time weather information. The Robot will run randomly and avoid being catched easily with the IR sensor. When users catch the robot and touch the quit button on the piTFT, the music and wheels will be stopped immediately.

Conclusions

The final project is planned, scheduled and demonstrated as expected. In the software design, firstly the real-time weather information is obtained from the OpenWeatherMap api through HTTP request. Then a relational database was established based on the weather information and the music information on Amazon DynamoDB, with which users can retrieve the music according to the weather. Thirdly, an official website was designed for users to set the alarm robot. In the hardware desing, firstly the real-time clock was wired and connected to the Raspberry pi. Secondly the IR sensor is connected to the Robot to avoid obstacles. The most challenging part is the integration of the software design and the hardware design, Raspberry pi wants to get time information from the website hosted on EC2 and display weather and music information on piTFT. The part of playing music with the derived youtube url is the most exhausting part. In the beginning, we plan to use web browser on the Raspberry pi to open the url. However, it turns out that the web browser can not open the link and play music while the piTFT is working. What's more, the music can not be stopped because we are using a web browser. Then we change our thoughts to download the music according to the url and download an omxplayer to play the musicon Raspberry pi. In this way not only did we solve the problem of playing music while displaying on piTFT, but also we can control the play and stop of the music with program. In conlcusion, there is always a gap between ideality and reality.

Future Work

There still remains some modules to be done in our project. We have installed the face_recognition module and plan to add the function that when the robot recognized the face of the user, it will automatically be stopped. This will be an improvement of the quit botton on piTFT. Also, we plan to accomplish a power energy saving system with RTC, which means that when the robot get the information of the alarm time, raspberry pi will enter a sleeping mode, thust the battery will be saved to a large extent.

Work Distribution

Group Photo

Joo Yeon Chae(jc2464)
Take charge of hardware integration including IR sensor and RTC module, robot movement, and power management system.

Juexiao Wang(jw2534)
Take charge of building the alarm robot system on Amazon Cloud platform, carrying out the design of the official website and displaying information on piTFT.

Parts List

All of the parts were provided in the lab.
Raspberry Pi 3 Model B $35.00
Adafruit PiTFT $34.95
Parallax Standard Servo (x2) $14.99
Sharp GP2Y0D810Z0F IR Sensor $7.95
PCF8523 Real Time Clock $4.95
Panasonic DS1E-S-DC24V $7.52
Total $120.35

References

RTC integration Tutorial

PCF8523 Datasheet

OpenWeatherMap

Amazon DynamoDB

#Server, Client

Server.py

from flask import Flask, Response, render_template
from datetime import datetime
import query

app = Flask(__name__)
wake_time = ["0"]

@app.route("/", methods=["GET"])
def show_homepage():
return render_template("web.html")

@app.route("/music/", methods=["GET"])
def query_music(zip_code):
dic = query.DynamoDB_Query(zip_code)
return Response(headers={"url": dic["music_url"],"temperature":dic["temperature"],"music":dic["Music"],"singer":dic["Singer"],"weather":dic["weather"]}

@app.route("/singer/", methods=["POST"])
def choose_singer(singer):
query.singer = singer.replace("-", " ")
return Response(headers={"Singer":singer})

@app.route("/set_time/", methods=["POST"])
def set_time(time):
wake_time[0] = time
print("ALARM TIME SET TO "+wake_time[0])
return Response(headers={})

@app.route("/get_time", methods=["GET"])
def get_time():
return Response(headers={"time":wake_time[0]})

if __name__ == "__main__":
app.run(host="0.0.0.0", port=1722)

Client.py

from __future__ import unicode_literals
import requests
import webbrowser
import time
import rtc
import display
import subprocess
import os
from omxplayer.player import OMXPlayer
import youtube_dl

class Client:
def __init__(self, ip, port, zipcode, path):
self.ip = "http://" + ip + ":" + port
self.zipcode = zipcode
self.path = path
requests.post(url=self.ip + "/set_time/0")

def update(self):
self.info = requests.get(url=self.ip + "/music/" + self.zipcode).headers
#print(self.info)

def getmusicinfo(self):
#print("SINGER: " + self.info["singer"])
#print("PlAY MUSIC: " + self.info["music"])
return self.info["music"], self.info["singer"]

def gettempinfo(self):
#print("TODAY'S WEATHER: " + self.info["weather"])
#print("TODAY'S TEMP: " + self.info["temperature"] + "ÃÂ°C")
return self.info["weather"], self.info["temperature"]

def getmusicurl(self):
return self.info["url"]

def gettime(self):
r = requests.get(url=self.ip + "/get_time")
return r.headers["time"]

def downloadmusic(self):
opts = {"outtmpl": self.path,
"format": "bestaudio/best",
"postprocessors": [{
"key": "FFmpegExtractAudio",
"preferredcodec": "mp3",
"preferredquality": "192",
}]}
with youtube_dl.YoutubeDL(opts) as ydl:
ydl.download([self.info["url"]])

def playmusic(self):
self.player = OMXPlayer(self.path)
#subprocess.Popen(['omxplayer $(youtube-dl -g best ' + self.info["url"] + ')'])
#webbrowser.open(self.info["url"])

def stopmusic(self):
self.player.quit()

def sleep(sleep_time):
start = time.time()
while time.time() - start < sleep_time:
if d.should_quit():
exit(0)
d.blit()
d.tick()

if __name__ == "__main__":
temp_path = "/home/pi/youtubemusic.mp3"
if os.path.isfile(temp_path):
print "Removed old file"
os.remove(temp_path)
c = Client(ip="34.220.186.40", port="1722", zipcode="14850", path=temp_path)
d = display.Display()

while True:
c.update()
music, singer = c.getmusicinfo()
weather, temp = c.gettempinfo()
alarm_time = c.gettime()

if alarm_time == "0":
print("Not yet changed")
d.set_content(weather + " " + temp + "\xb0C", "Not set", music, singer)
sleep(5)
else:
show_time = rtc.convertpattern(alarm_time, "%Y.%m.%d.%H.%M", "%Y/%m/%d %H:%M")
d.set_content(weather + " " + temp + "\xb0C", show_time, music, singer)
print("Alarm clock set to " + show_time)

c.downloadmusic()

alarm_time = rtc.gettimestamp(alarm_time, "%Y.%m.%d.%H.%M")
rpi_time = rtc.getrpitimestamp("%Y-%m-%d %H:%M:%S")
sleep_time = max(alarm_time - rpi_time, 0)
print("Will sleep for {} seconds...".format(sleep_time))
sleep(sleep_time)

print("Wake up")
try:
c.playmusic()
except:
pass

import two_wheel
two_wheel.runrobot(c, d)
exit(0)

#Relational Database

MusicsCreateTable.py

from __future__ import print_function # Python 2/3 compatibility import boto3

dynamodb = boto3.resource('dynamodb', region_name='us-west-2')

table = dynamodb.create_table(
TableName='MusicMulti',
KeySchema=[ { 'AttributeName': 'weather', 'KeyType': 'HASH' #Partition key },
{ 'AttributeName': 'singer', 'KeyType': 'RANGE' #Sort key } ],
AttributeDefinitions=[ { 'AttributeName': 'weather', 'AttributeType': 'S' },
{ 'AttributeName': 'singer', 'AttributeType': 'S' } ],
ProvisionedThroughput={ 'ReadCapacityUnits': 10, 'WriteCapacityUnits': 10 } )

print("Table status:", table.table_status)

MusicLoadData

from __future__ import print_function # Python 2/3 compatibility
import boto3
import json
import decimal

dynamodb = boto3.resource('dynamodb', region_name='us-west-2')
table = dynamodb.Table('MusicMulti')
with open("/Users/wangjuexiao/Desktop/playDynamodb/musicdata.json") as json_file:
musics = json.load(json_file, parse_float = decimal.Decimal)
for music in musics:
weather = music['weather']
singer = music['singer']
info = music['info']

print("Adding music table:", weather, singer)
table.put_item(
Item={ 'weather': weather,
'singer': singer,
'info': info, } )

Query.py

from __future__ import print_function # Python 2/3 compatibility
import boto3
import json
import decimal
import requests
import webbrowser

from boto3.dynamodb.conditions import Key, Attr # Helper class to convert a DynamoDB item to JSON.
singer = "Taylor Swift"
class DecimalEncoder(json.JSONEncoder):
def default(self, o):
if isinstance(o, decimal.Decimal):
if o % 1 > 0:
return float(o)
else:
return int(o)
return super(DecimalEncoder, self).default(o)

def DynamoDB_Query(zip):
dynamodb = boto3.resource('dynamodb', region_name='us-west-2')
table = dynamodb.Table('MusicMulti')

URL = "http://api.openweathermap.org/data/2.5/weather?zip="+zip+",us&APPID=f238cb34dcf2de6576414883ce95b492"
r = requests.get(url=URL)

weather = r.json()["weather"][0]["main"]
if weather not in ["Clouds", "Clear"]:
weather = "NG"

temperature = r.json()["main"]["temp"] - 273.15
temprange = int(round(temperature/10))
print("temprange "+str(temprange))
print("Music for "+ weather +" Days") response = table.query(
KeyConditionExpression=Key("weather").eq(weather+" "+str(temprange)) & Key("singer").eq(singer) )
music_url = response["Items"][0]["info"]["music_url"]
songtitle = response["Items"][0]["info"]["songtitle"]
print(songtitle[0])
return {"music_url":music_url,"temperature":int(round(temperature)),"Singer":singer,"Music":songtitle[0],"weather":weather}

#RTC module

RTC.py

import subprocess
import time
import datetime
import client

def gettimestamp(timestr, pattern):
timestamp = time.mktime(datetime.datetime.strptime(timestr, pattern).timetuple())
return timestamp

# Get time from readint the result from command line
def getrpitimestamp():
result = subprocess.Popen(['sudo','hwclock','-r'],stdout=subprocess.PIPE,stderr=subprocess.STDOUT)
stdout,stderr = result.communicate()
rpttimestr = stdout[:-1].split(".")[0]
return gettimestamp(rpttimestr, "%Y-%m-%d %H:%M:%S")

# Servo

two_wheels.py

import RPi.GPIO as GPIO
import time
import random

GPIO.setmode(GPIO.BCM)# Set for broadcom numbering not board numbers...
GPIO.setup(18, GPIO.OUT)
GPIO.setup(23, GPIO.OUT)

GPIO.setup(12, GPIO.IN, pull_up_down=GPIO.PUD_UP)

#create a PWM instance
frequency = 1/21.5*1000
p1 = GPIO.PWM(18, frequency)
p2 = GPIO.PWM(23, frequency)

#start PWM
p1.start(0) #start left servo pwm, idle
p2.start(0) # start right servo pwm, idle

backward = False

def direct_servo(servo,direction): #function for changing the direction of a servo
if direction == 1: # clockwise
width = 1.3
elif direction == 0: # idle
width = 0.0
elif direction ==2: # counter-clockwise
width = 1.7
dc = width/(20+width)*100 # duty cycle
frequency = 1/(20+width)*1000

if servo == 1: # left servo
p1.ChangeDutyCycle(dc) # change dc and frequency
p1.ChangeFrequency(frequency)
elif servo == 2: # right servo
p2.ChangeDutyCycle(dc) # change dc and frequency
p2.ChangeFrequency(frequency)

def GPIO12_callback(channel): # change the direction of left servo every time pressed
print('obstacle detected')
direct_servo(1,2)
direct_servo(2,1)
print('back')
global backward
backward = True
time.sleep(1)

def cleanup(client):
p1.stop()
p2.stop()
client.stopmusic()
GPIO.cleanup() #clean up GPIO on CTRL + C exit
exit(0)

def sleep(client, display, sleep_time):
start = time.time()
while time.time() - start < sleep_time:
if display.should_quit():
cleanup(client)
display.blit()
display.tick()

def runrobot(client, display):
GPIO.add_event_detect(12, GPIO.FALLING, callback = GPIO12_callback, bouncetime = 300) # add event for button
while not display.should_quit():
global backward
rand_dir = random.randint(1,2)
print(rand_dir)
direct_servo(1,rand_dir)
direct_servo(2,rand_dir)
sleep(client, display, 2)
direct_servo(1,1)
direct_servo(2,2)
print('forward')
sleep(client, display, 5)
cleanup(client)