Jingyi Shen(js2825) and Yunhe Shao(ys566)
ECE 5725 Final Project
We designed a smart guest management system that is intended to help public buildings such as apartment complexes and office buildings to improve management efficiency.
Unlike modern smart locks for homes, our systems provide administrators with more flexibility and with a wide range of functionality for various daily needs, especially in the covid era, such as the acquisition of express and takeaway information, management of personnel entry and exit, activation of building alarms, etc..
Keywords: YOLOv3 tiny, motion sensor, hall effect sensor, telebot
Part 1: Identification of delivery package
We all know that an office building will receive a large number of express delivery every day. If the door management system can automatically identify the express brand and save relevant information, it can improve management efficiency.
Step 1: Use labelimg for image annotation, generate express image data set and corresponding annotation file
Step 2: Use YOLOv3 for model training based on the Darknet framework, save configuration files and the best weights
Step 3: Use the motion sensor, when an object is detected to move, call the camera to take a picture, and transmit it to the background. Use the trained model for real-time express type detection
The following examples show some of the types of express that we can detect:
Part 2: Access for Maintenance personnel
Maintenance personnel, such as plumbers, weeders, cleaners, etc., need to enter the building regularly, so we set the gate password. If the password is entered correctly, you can enter, and three incorrect entries will not be able to enter, and will be reported to the administrator.
Enter the 4-digit password:
If correct, the door will be open and the administrator will be notified
If you enter the error three times, it cannot be entered, and the administrator will receive the relevant information
Part 3: Storage and Retrieval of Food Delivery
Food delivery package is totally different from express package. The storage time of food delivery is relatively short. When the delivery arrives, administrator need to know its location and delivery time immediately, so that we can deal with it if there is a problem later.
Therefore, we use hall effect sensor, which can detect the presence of a magnetic field and is often used for positioning objects or measuring rotation. We use two magnets. When the food is placed on the shelf, gravity causes the two magnets to approach each other, and the sensor returns to a high voltage, indicating that it has detected that there is a takeaway delivery.
Step 1: Food Delivery Package Recognition (have one in Part 1)
Step 2: the food delivery staff will place the food delivery in the designated location
Step 3: Administrator can check in real time whether there is food delivery on the delivery shelf and the delivery time
Part 4: Alarm System
When an emergency occurs, the administrator can trigger the alarm to urge the people in the building to evacuate as soon as possible. This feature will be shown in the next section
Our complete design is based on telebot and telegram. Telebot is a bot framework for Telegram Bot API. This package provides the best of its kind API for command routing, inline query requests and keyboards, as well as callbacks. We set up an administrator group chat on telegram and set the group chat main menu. The administrator can send commands on the menu to implement all the functions we mentioned in the previous part.
/start: The administrator will receive a message: Welcome to the ultimate control panel
/history: The administrator will receive a message about all the express delivery and its classification information (Fedex, UPS, DHL)
/latest: The administrator will get the information of the most recently received package, which is convenient for the administrator to check
/door: The password function will be turned on, and maintenance workers can enter the building by entering the password. When someone successfully enters the building, the administrator will receive: "the mechanics have entered; If the password is entered incorrectly three times, the door cannot be closed and the administrator will receive a message: Potential risk of unknown person trying to enter
/alarm: When the administrator sent this command, the building’s alarm was turned on to remind people to evacuate quickly
/alarm_stop: When the danger has been eliminated or all personnel have been evacuated, the administrator sends this command to disarm the alarm
/food: The administrator will receive the current status of the takeaway shelf, whether the takeaway has been delivered and the delivery time
The picture on the right is an example of the functions we have implemented. The administrator receives different information in real time by sending different commands
The building administrator can remotely control through telegram, and can receive real-time delivery information, takeaway information, and personnel entry and exit information. At the same time, it can remind building personnel to evacuate in an emergency. After testing, our system has strong stability, so we basically realized all the functions in the proposal, and finally realized a relatively complete smart guest management system.
Problem 1: The camera cannot detect the express in real time. At first, we tried to directly call yolov3 to perform target detection on real-time video to determine whether there is an express delivery, but the real-time and accuracy did not meet our expectations, so we changed to using a motion sensor to take pictures when an object was detected in motion. Object recognition on the picture. In the future, we plan to replace yolov3 with other methods, such as feature detection to identify the type of express. In contrast, there should be a smaller delay
Problem 2: When the administrator enters the history command, it should return the type and quantity of all the current express. However, when the same type of express appears multiple times, the history will not be repeated. In the follow-up, we will modify the content of the reply to ensure that it will be returned every time there is a new package, and the order will be sorted according to the arrival order of the package
Problem 3: The password update module still needs to be improved. It is currently in the program. It is forbidden to enter when the same user's password is detected three times incorrectly. In the future, we will realize that the administrator can manually update the password through commands to ensure the controllability of the door password
Responsible for Delivery Identification, Access for Maintenance Personnel, and alarm system
Design Storage and Retrieval of Food Delivery module, our complete system design and system testing