ECE597 Project Auto HUD

Revision as of 19:39, 20 May 2010 by Routhcr (Talk | contribs) (OpenCV Haar Training)

Jump to: navigation, search

Project Overview

The goal of this project is to use the beagle board to run image recognition on a camera feed located inside a car, and then signaling to the driver via a pico projector various objects of interest.

Team Members

Chris Routh

J. Cody Collins

Greg Jackson

Keqiong Xin


  • Create Minimal Linux Image that can run OpenCV and run the display
  • Determine hardware needed for the project
  • Work on getting a camera functioning on the beagle board
  • OpenCV running nativly on beagle with min config
  • OpenCV working on video stream
  • Projector working on Beagle
  • Car integration (power)
  • Algorithm development

Installing OpenCV (Development Machine)

This is a script that will install OpenCV on a debian-based development machine. The script will add the debian testing repositories and install OpenCV and its dependencies. The repositories are then removed to avoid conflicts with existing packages during regular updates.


echo "deb testing main" > /tmp/opencv-temp.list 
echo "deb-src testing main" >> /tmp/opencv-temp.list
sudo mv /tmp/opencv-temp.list /etc/apt/sources.list.d/opencv-temp.list

sudo apt-get update
sudo apt-get install -y --allow-unauthenticated libcv4 libcv-dev libhighgui4 libhighgui-dev libcvaux4 libcvaux-dev

sudo rm -f /etc/apt/sources.list.d/opencv-temp.list

sudo apt-get update

Installing OpenCV on the Beagle

Probably the easiest place to start is by using narcissus. Choose beagleboard as the machine type and unstable for the release. In order for highgui to work (necessary for camera capture unless you are using GStreamer), you must build an image with X11 support. Therefore, choose X11 for the user environment. The choice for the X11 desktop environment is not critical, but it would be wise to choose something fairly lightweight, such as Enlightenment. It took several hours for Gnome to configure upon first boot. Once the filesystem has been extracted to a properly formatted SD card with an appropriate kernel on the boot partition (we tested this using 2.6.29), you should be able to boot. Upon boot, you will need to run opkg update. After this, you will need to run opkg install with the following packages:

  • gcc
  • gcc-dev
  • binutils
  • binutils-dev
  • opencv
  • opencv-dev
  • g++

You should now be able to compile using:

g++ signdetect.cpp -o signdetect -I /usr/include/opencv/ -L /usr/lib -lm -lcv -lhighgui -lcvaux

OpenCV Haar Training

Gather Samples

Due to the large volume of sample data needed to create a effective Haar Cascade (about 1000 positive images) it is easier to gather video of a positive target and then break apart the video frame by frame and use the results as images. There are 2 two types of images, good and background. Both types of images are important in order for the cascade to be trained accurately.

Create Index File

There are two index files needing to be created in order for the system to train on the images, a background index file creating a list of filelocations, and a positive index file containing the positive file locations, the number of objects in the picture and the rectangular locations for the object.

Creating the Negative Index File

Use the following automated script:


find ./*.jpg -maxdepth 2 -print  > background.idx
find ./*.png -maxdepth 2 -print >> background.idx
find ./*.bmp -maxdepth 2 -print >> background.idx
find ./*.jpeg -maxdepth 2 -print >> background.idx

Generate Samples

Using the positive samples the creasamples cammand can apply transforms to the images and add them to the background images creating a wider range of images to train on. The syntax for this command is:

Usage: ./createsamples
  [-info <description_file_name>]
  [-img <image_file_name>]
  [-vec <vec_file_name>]
  [-bg <background_file_name>]
  [-num <number_of_samples = 1000>]
  [-bgcolor <background_color = 0>]
  [-inv] [-randinv] [-bgthresh <background_color_threshold = 80>]
  [-maxidev <max_intensity_deviation = 40>]
  [-maxxangle <max_x_rotation_angle = 1.100000>]
  [-maxyangle <max_y_rotation_angle = 1.100000>]
  [-maxzangle <max_z_rotation_angle = 0.500000>]
  [-show [<scale = 4.000000>]]
  [-w <sample_width = 24>]
  [-h <sample_height = 24>]

Display Buffer Mapping

In order to get access to the display buffer on the beagle, you will need to run the following in U-Boot

setenv bootargs console=ttyS2,115200n8 root=/dev/mmcblk0p2 rw rootwait omapfb.mode=1024x768MR-16@60 omapfb.debug=y omapdss.def_disp=dvi omapfb.vram=0:10M,1:10M vram=20M

Other sources have mentioned setting a value for mmcargs. However, we were not able to get it to work properly until the options were applied directly to the bootargs variable.

Pico Projector Integration

As of revision C4 of the Beagleboard there is no necessary configuration needed to display native resolution on the projector.

Future Directions

GStreamer on the DSP

There is a package available for the beagle called gst-dsp, which is a native GStreamer plug-in to give it access to the DSP. Along with gst-opamfb and the dsp-bridge driver, this should allow us to access the DSP directly and output video directly to the framebuffer. OpenCV can interact with GStreamer, so this appears to be a very promising direction for the project. See this article for more information and a demonstration. That article also has a link to a minimal beagle image that provides a native framebuffer video player without requiring X.