Jetson/Tutorials/Vision-controlled GPIO

< Jetson
Revision as of 12:09, 2 July 2014 by Shervin.emami (talk | contribs) (Created the tutorial for turning an LED on whenever a face is detected)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Turn an LED on/off based on whether a face is detected by a camera

This tutorial will explain how to detect a face using OpenCV, and how to turn an LED on/off based on whether the face is detected.

Testing your camera

  • Connect a USB webcam to the full-sized USB 3.0 port on Jetson TK1 (or into the micro-AB USB 2.0 port through a micro-B to female USB-A adapter).
  • If you haven't tried using your webcam on Jetson TK1, check if it works first such as by running this to show the camera preview in a graphical window:
sudo apt-get install luvcview

(If you can't see the camera stream, then try plugging your webcam into the micro-AB USB 2.0 port through a micro-B to female USB-A adapter, or looking at supported cameras on the Cameras reference page).

Detecting a face using OpenCV

  • If you haven't done so already, install OpenCV to the home directory on your device, by following the Installing OpenCV page. If you used the pre-built version of OpenCV, then you will also need to download the OpenCV source code in order to get the OpenCV sample programs. So if you don't have the OpenCV samples then open a web-browser to "" & clicking on "OpenCV for Linux/Mac", or from the command-line you can run this on the device:
  • Now make your own "faceActivatedGPIO" folder for this project, eg:
mkdir ~/faceActivatedGPIO
cd ~/faceActivatedGPIO
  • Now copy the "cascadeclassifier" face detection sample from OpenCV's GPU-accelerated samples folder and build it:
cp ~/opencv/samples/gpu/cascadeclassifier.cpp .
g++ cascadeclassifier.cpp -lopencv_core -lopencv_imgproc -lopencv_highgui -lopencv_calib3d -lopencv_contrib -lopencv_features2d -lopencv_flann -lopencv_gpu -lopencv_legacy -lopencv_ml -lopencv_nonfree -lopencv_objdetect -lopencv_photo -lopencv_stitching -lopencv_superres -lopencv_video -lopencv_videostab -o cascadeclassifier
  • Try running it with your attached webcam:
./cascadeclassifier --cascade ~/opencv/data/haarcascades/haarcascade_frontalface_alt.xml --camera 0

(Hit 'm' to search for just a single face instead of multiple faces, then hit Spacebar if you want to compare CPU vs GPU performance).

Testing GPIO on Jetson TK1

  • If you haven't done so already, follow the GPIO tutorial so that you can manually turn an LED on/off using root-permissions.
  • Now we want to control GPIO from our own code. Download Derek Molloy's SimpleGPIO library and get the 3 source files in the "gpio" folder. eg:
sudo apt-get install git
git clone git://
mkdir gpio
cp ../boneDeviceTree/gpio/*.cpp gpio/.
cp ../boneDeviceTree/gpio/*.h gpio/.
  • Let's build & run the sample TestApplication to make sure GPIO works. Remember that it must be run with root-permissions in order to access GPIO:
g++ gpio/TestApplication.cpp gpio/SimpleGPIO.cpp -I gpio -o testGPIO
sudo ./testGPIO
  • You should now be seeing your LED flash several times (or see the voltage going up & down on your multimeter).

Controlling the LED based on face detection results

  • Rename "cascadeclassifier.cpp" to "faceActivatedGPIO.cpp", eg:
mv cascadeclassifier.cpp faceActivatedGPIO.cpp
  • Now it is just a matter of combining the cascadeclassifier sample code with the SimpleGPIO library. Near the top of the "faceActivatedGPIO.cpp" file, add:
#include "SimpleGPIO.h"
  • Then near the bottom of the file after it says imshow("result", frameDisp); (line 277), add this code:
// Turn the LED on or off depending on whether a face was detected or not.
const int GPIO_LED = 57;  // GPIO_PH1 (Pin 50 on J3A1 of Jetson TK1 board)
gpio_export(GPIO_LED);    // Start using the LED pin
gpio_set_dir(GPIO_LED, OUTPUT_PIN);   // The LED is an output
if (detections_num > 0) {
    gpio_set_value(GPIO_LED, HIGH);   // Turn the LED on
else {
    gpio_set_value(GPIO_LED, LOW);    // Turn the LED off
gpio_unexport(GPIO_LED);  // Stop using the LED pin for now
  • Now build the project. eg:
g++ faceActivatedGPIO.cpp gpio/SimpleGPIO.cpp -I gpio -o faceActivatedGPIO
  • And finally, run the project with root-permissions. eg:
sudo ./faceActivatedGPIO

You should now see your camera preview on your screen, and if your face is detected then your LED should be on, and if not then it should be off!

Next steps

You can improve this project in many ways, such as turning on an electro-mechanical relay that opens a door when a face is detected, instead of just flashing an LED. Or turn multiple LEDs on based on how many faces were detected or their size, by using multiple GPIO pins. And obviously, now that you know how to perform some vision-based GPIO, you could apply the same principles to activate GPIO when the full-body of a person is detected (using OpenCV's HOG sample as explained in the Full Body Detection tutorial) or when a car is detected (using the same HOG or CascadeClassifier code but with an XML file trained for car classification), or thousands of other potential computer vision or software based detections.

This tutorial is also the basis for the vision-controlled robot tutorials such as the SCOL walking robot tutorial.