ECE434 Project-Image recoginition with openCV and Siri

Revision as of 08:42, 17 November 2020 by Wangh9 (talk | contribs) (Future Work)
Jump to: navigation, search

Team members: Haoxuan Sun, Heda Wang

Executive Summary

In this project, we will use OpenCV to do image recognition. A servo will make the camera centered at the desired target. Also, it is controlled by Siri through shortcuts and ssh.

Currently, this project only supports three colors. But you can easily add colors you want to find. This project is limited by the processing speed of the beagle bone. Because the processing takes a long time for each frame, we can only point to the color instead of tracking. With enough optimization and computational power, this can be easily turned into an image tracker.

To achieve this project you need: 1. BeagleBone Black 2. A servo 3. A USB webcam 4. LED or other colorful things for target 5. (optional). iPhone or iPad Supports Siri

A demo video is available on YouTube:


Connect the USB camera to the USB port. Connect the servo to 5V on the Beaglebone. If the servo is not powered, you might need to plug in the 5v power supply on the beagle bone. The servo signal is connected to pin 8_13

Tie or glue the servo and the camera together. We will use the camera for other use laster, so we tie the camera and servo using some wire.

Installation Instructions

We need to install openCV To install it, connect to the BeagleBone and type:

   sudo apt install python-opencv
   sudo pip3 install opencv-python

and if error occurs, type:

   pip3 install opencv-python 
   sudo apt-get install libcblas-dev
   sudo apt-get install libhdf5-dev
   sudo apt-get install libhdf5-serial-dev
   sudo apt-get install libatlas-base-dev
   sudo apt-get install libjasper-dev 
   sudo apt-get install libqtgui4 
   sudo apt-get install libqt4-test

User Instructions


Theory of Operation

Work Breakdown

Heda Wang: Learn and config Siri shortcuts, Wrote the servo code and the scan algorithm, Integrate the servo code with image recognition code.

Haoxuan Sun: Install Open-CV, Wrote the image processing and recognition code. Figure out how to find ip address on campus network.

Other things we done them together.

Future Work

Our initial plan was to make the servo track the object. But the image processing speed is so limited. Every frame need several seconds to process. If we can optimize the algorithm, we can achieve track the boject.

Also, The short cut on Siri can accept parameter input. With more development, we could let siri accept any color input with only one python file.


Give some concluding thoughts about the project. Suggest some future additions that could make it even more interesting.