Difference between revisions of "ECE434 Project-Image recoginition with openCV and Siri"

From eLinux.org
Jump to: navigation, search
Line 63: Line 63:
== Highlights ==
== Highlights ==
This is a demo video with our explaination.
== Theory of Operation ==
== Theory of Operation ==

Revision as of 10:26, 17 November 2020

Team members: Haoxuan Sun, Heda Wang

Executive Summary

In this project, we will use OpenCV to do image recognition. A servo will make the camera centered at the desired target. Also, it is controlled by Siri through shortcuts and ssh.

Currently, this project only supports three colors. But you can easily add colors you want to find. This project is limited by the processing speed of the beagle bone. Because the processing takes a long time for each frame, we can only point to the color instead of tracking. With enough optimization and computational power, this can be easily turned into an image tracker.

To achieve this project you need:

1. BeagleBone Black

2. A servo

3. A USB webcam

4. LED or other colorful things for target

5. (optional). iPhone or iPad Supports Siri

A demo video is available on YouTube: https://youtu.be/_aZgPR_QLXI


Connect the USB camera to the USB port. Connect the servo to 5V on the Beaglebone. If the servo is not powered, you might need to plug in the 5v power supply on the beagle bone. The servo signal is connected to pin 8_13

Tie or glue the servo and the camera together. We will use the camera for other use laster, so we tie the camera and servo using some wire.

Installation Instructions

We need to install openCV To install it, connect to the BeagleBone and type:

   sudo apt install python-opencv
   sudo pip3 install opencv-python

and if error occurs, type:

   pip3 install opencv-python 
   sudo apt-get install libcblas-dev
   sudo apt-get install libhdf5-dev
   sudo apt-get install libhdf5-serial-dev
   sudo apt-get install libatlas-base-dev
   sudo apt-get install libjasper-dev 
   sudo apt-get install libqtgui4 
   sudo apt-get install libqt4-test

Connet your beagle bone to your router though a cable Also, make sure there is shortcut on your ios device. You can install it from APP store. Click create add shortcut and scripting. Then select the Run Script Over SSH. Touch the show more to see all the option. Go to your router config page and find your beagle bone's IP address. Enter the IP address to "Host" and the port as 22. The user is usually Debian and also enters the password. In the script enter sudo python3 red.py. Click the + under the block you just added and select scripting again. Select show Notification and configure it as "I am pointing red". Finally, rename the shortcut as "point red" or whatever you want Siri to recognize. If you want to deal with more color, just copy this shortcut and change the code it run.

After configer all the shortcut the screen should look like this. figure here

User Instructions


This is a demo video with our explaination. https://youtu.be/ch4CfHEJSlM

Theory of Operation

Work Breakdown

Heda Wang: Learn and config Siri shortcuts, Wrote the servo code and the scan algorithm, Integrate the servo code with image recognition code.

Haoxuan Sun: Install Open-CV, Wrote the image processing and recognition code. Figure out how to find ip address on campus network.

Other things we done them together.

Future Work

Our initial plan was to make the servo track the object. But the image processing speed is so limited. Every frame need several seconds to process. If we can optimize the algorithm, we can achieve track the boject.

Also, The short cut on Siri can accept parameter input. With more development, we could let siri accept any color input with only one python file.


There is a lot of potential with the Siri shortcut. Adding more fun things to the shortcut will make this project connect better to ios devices.