Project By Sichen Liu (firstname.lastname@example.org) Ruichen Tan(email@example.com)
The color feature of the color ring resistor, it’s possible to detect the value of resistor based on computer vision. The object of this project is to design an auto-resistor sorter machine that can tell the value of the resistors.
Pi Resistor Sorter is a machine that detect the input resistors. The control unit of Pi Resistor Sorter is Raspberry Pi, and the detection of resistor value is based on hardware camera. OpenCV-Python library is used to analyze video frame. We implemented the mechanical hardware via 3D printed. To control mechanical movement, a servo is used to control convey belt unit, and another servo is used to control the rotation of the plate that allowed resistors to be sorted. A simple web page graphical user interface is implemented to show the status of the Pi Resistor Sorter.
The color code indicates the value of the resistors. Due to this feature, Computer Vision is a potential approach to tell the value. There are more than one color standards of color ring resistor such as 4-band, 5-band or 6-band type. For our project, 4-band resistor is used for testing. The background color of our testing resistor is light yellow.
The tradition method of measuring the resistor is electrical measurement, but this electrical measuring only tells the actual value of the resistor. The value of what this resistor supposes to be is hidden. Also compared with electrical measurement, the computer vision detection can tell the tolerance of the resistor.
1. Convey Belt Movement – Control the movement of the convey belt, this convey belt moves the resistor under camera before sorting. After sorting, convey belt move the resistors to sorted plate.
2. Resistor Recognition – Detect the color of the resistor rings, and calculate the value of resistor based on the color of rings and its order.
3. Plate Movement – Rotate specific angles according to the result of the computer vision measurement.
This device is for sorting the resistors on the conveyor belt. With this roller, we can put the resistors simutaniously. But it did not work. Because the resistors are not perfect cylinder and there are only two points that can actually contact with the sorting wheel and the conveyor. And once there is a tiny un-smooth part in any of those two parts. It will make the direction of the resistor change and lead to the result that the resistors will be stucked in the slot between the the wheel and the conveyor
This part can store the different resistors. When the resistors fall from the belt, they will fall into the 6 divided areas.
Foudamental support of the whole machine
This is one of the rollers for the belt. The servos provide power to the conveyor through these rollers
OpenCV (Open Source Computer Vision Library) is a library of programming functions mainly aimed at real-time computer vision. We chose OpenCV-python to detect the color of the resistor. The color images are represented as three-dimensional arrays, we use Numpy to finish detection assignment.
GRB color model HSV color model
The HSV model describes colors in terms of hue, saturation, and value. The hue indicates the primary color, and value indicates the brightness of the color. The change of the shadow only influences the value attribute, and the hue attribute will not change so much. Hence the HSV model is better for measuring the image where shadows always change.
Working flow of value detector
UnSharp mask: The UnSharp mask adds the detail of resistor that lets us recognize the its color features. Sharpening filter emphasizes the differences between adjacent light and dark sample points in an image.
Color Area detection is the method we used to find the color ring of the resistor. After the UnSharp mask filtering, the color feature of resistor will focus on specific areas. In this stage, we transfer the RGB color model to HSV color model. Set specific color bound for different color, and filter the image. A 2D Numpy array is shown after color filtering operation.
Erosion: Due to the limit of real life light environment, the color noise is unavoidable. Erosion is one of the basic operations in the area of morphology. This operation erodes away the boundaries of regions of foreground pixels. Pi resistor sorter uses the erosion algorithm to filter noises on the image.
Using the data gathered from color detection algrithom, we could find how many rings on the resistor target. When we were able to calculate the value of resistor based on these color data. The algrithom we implemented is based on counting the points and calculate the median location of the resistor. The value of resistor is depending on the color ring and its location.
According to our test, the result of color detection changes for different frame. To improve the accuracy and tolerance of resistors sorting, we marked a resistor value as valid only when three repeat results happen.
The above image processing took long take to finish, to guarantee the accuracy, we implemented an edge detection algrithom to test if there was a resistor coming. The edge detection algorithm we implemented is canny detection. MinVal and MaxVal were set to fit number to find the edge between resistor and white background page.
GUI is used to convey the information of Pi resistor sorter. Since Pi owns operating system, we chose to run Pi as a website server, thus other electronic devices can access it using website. Compared with using normal PiTFT, the webpage is more convenient and intuitive.
The web framework is python micro framework library, flask. This framework can handle http request as python functions. Our web page is straightforward, which enable users to see the information of sorting when users access Pi’s IP address. The History window shows the sorting history combined with resistor value and the time. The Statistics window shows the counting number of different resistors.
Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker. In this project, we chose Redis due to its lightweight feature. When resistor sorting python scrip detect a valid resistor value, the history information of this resistor will add to Redis. The web scrip accesses the database, and puts the resistor data on web page from Redis.
We successfully detect the color feature using OpenCV. The result of color detection based on threshold is shown as following image.
Our project proved that computer vision is a potential method for resistor sorting. Pi resistor sorter performs partly of the function it was designed for. Pi resistor sorter can tell the resistors with specific color feature as we expected. The website GUI and movement control meets the goals we set early. But our design has some limits for mechanical staffs. And the result of color detection is influenced by the changeable environment.
Early our expectation is to design a machine based on convey belt model that can sort many resistors at a time. The performance of mechanical roller did not meet our expectation since the irregular shape of resistors. Hence, we have to put the resistor one by one to finish the detection assignment. Moreover, the movement of convey belt did not work as us supposed to be. The unstable movement sometimes causes the resistor cannot stop to the right location after Pi detects the edge of the resistor.
Our Pi resistor sorter performs bad for detecting some specific colors like gray, silver and white. Also since the color bound of orange and red is very closed, sometimes the Pi resistor sorter will take the orange color as red. The mechanical design also limits the accuracy. The paper material we used to build for convey belt works poorly after long time testing. Sometimes the movement of convey paper will not on the same horizon, which means the focus will change.
We can improve the conveyor by using a true belt. Then we can maximize the use of this device by making the sorting and feeding devices, which means that we can change the part to sort different staffs at anytime we want. Like the candies screws and nuts (But this version one is still mainly used for the resistors and the detachable parts are the potential uses detachable
Replace servo motor with stepper motor
Stepper motors have a large number of poles, magnetic pairs. Servo motor often has 4 to 12 in total. To move the stepper motor to specific position, it requires a position encoder, which map an angle to specific time. Our implementation is based on servo motor based on position encoder. The servo rotation location is not very accurate based on our result. The design of the stepper motor provides a constant holding torque without the need for the motor to be powered. The torque of a stepper motor at low speeds is greater than a servo motor. Hence, we think the stepper motor is better for using for plate rotation control in our design.
Stable computer vision Environment
To color detection algrithom is sensitive to the light environment. In this project, we did a lot to make the light environment stable like blocking the environment light and adding LED rings around the camera. However, due to the limits of convey belt, the focus for camera will change. A black box environment and conveyor belts with gear may be a potential solution.
Now the algrithom of sorting is combined with edge detection and direct color detection. This algorithm is a tradeoff between speed and accuracy. A better way for color detection is detecting the edge of resistor first, and then analyze image and color detection only inside the edge of resistor. The background color of testing resistor is light yellow, which is closed to the background color.
Special thanks to Cornell Rapid Prototyping Lab (RPL) providing us 3-D printer environment.
Computer Vision, Webpage GUI, Movement Control, and Hardware circuit setup.
Hardware design and build, program optimization, web production
We would like to thank Prof. Joesph Skovira for his advice for this project. He provided many useful suggession for system deisgn and color detection algorithm. In addition, we would like to thank all TAs for answering our questions.