ECE597 Project Auto HUD
The goal of this project is to use the beagle board to run image recognition on a camera feed located inside a car, and then signaling to the driver via a pico projector various objects of interest.
- Create Minimal Linux Image that can run OpenCV and run the display
- Determine hardware needed for the project
- Work on getting a camera functioning on the beagle board
- OpenCV running nativly on beagle with min config
- OpenCV working on video stream
- Projector working on Beagle
- Car integration (power)
- Algorithm development
Installing OpenCV (Development Machine)
This is a script that will install OpenCV on a debian-based development machine. The script will add the debian testing repositories and install OpenCV and its dependencies. The repositories are then removed to avoid conflicts with existing packages during regular updates.
#!/bin/bash echo "deb http://mirrors.kernel.org/debian/ testing main" > /tmp/opencv-temp.list echo "deb-src http://mirrors.kernel.org/debian/ testing main" >> /tmp/opencv-temp.list sudo mv /tmp/opencv-temp.list /etc/apt/sources.list.d/opencv-temp.list sudo apt-get update sudo apt-get install -y --allow-unauthenticated libcv4 libcv-dev libhighgui4 libhighgui-dev libcvaux4 libcvaux-dev sudo rm -f /etc/apt/sources.list.d/opencv-temp.list sudo apt-get update
Installing OpenCV on the Beagle
Probably the easiest place to start is by using narcissus. Choose beagleboard as the machine type and unstable for the release. In order for highgui to work (necessary for camera capture unless you are using GStreamer), you must build an image with X11 support. Therefore, choose X11 for the user environment. The choice for the X11 desktop environment is not critical, but it would be wise to choose something fairly lightweight, such as Enlightenment. It took several hours for Gnome to configure upon first boot. Once the filesystem has been extracted to a properly formatted SD card with an appropriate kernel on the boot partition (we tested this using 2.6.29), you should be able to boot. Upon boot, you will need to run opkg update. After this, you will need to run opkg install with the following packages:
You should now be able to compile using:
g++ signdetect.cpp -o signdetect -I /usr/include/opencv/ -L /usr/lib -lm -lcv -lhighgui -lcvaux
OpenCV Haar Training
Due to the large volume of sample data needed to create a effective Haar Cascade (about 1000 positive images) it is easier to gather video of a positive target and then break apart the video frame by frame and use the results as images. There are 2 two types of images, good and background. Both types of images are important in order for the cascade to be trained accurately.
Using the positive samples the creasamples cammand can apply transforms to the images and add them to the background images creating a wider range of images to train on. The syntax for this command is:
Usage: ./createsamples [-info <description_file_name>] [-img <image_file_name>] [-vec <vec_file_name>] [-bg <background_file_name>] [-num <number_of_samples = 1000>] [-bgcolor <background_color = 0>] [-inv] [-randinv] [-bgthresh <background_color_threshold = 80>] [-maxidev <max_intensity_deviation = 40>] [-maxxangle <max_x_rotation_angle = 1.100000>] [-maxyangle <max_y_rotation_angle = 1.100000>] [-maxzangle <max_z_rotation_angle = 0.500000>] [-show [<scale = 4.000000>]] [-w <sample_width = 24>] [-h <sample_height = 24>]
Display Buffer Mapping
In order to get access to the display buffer on the beagle, you will need to run the following in U-Boot
setenv bootargs console=ttyS2,115200n8 root=/dev/mmcblk0p2 rw rootwait omapfb.mode=1024x768MR-16@60 omapfb.debug=y omapdss.def_disp=dvi omapfb.vram=0:10M,1:10M vram=20M
Other sources have mentioned setting a value for mmcargs. However, we were not able to get it to work properly until the options were applied directly to the bootargs variable.
Pico Projector Integration
As of revision C4 of the Beagleboard there is no necessary configuration needed to display native resolution on the projector.
GStreamer on the DSP
There is a package available for the beagle called gst-dsp, which is a native GStreamer plug-in to give it access to the DSP. Along with gst-opamfb and the dsp-bridge driver, this should allow us to access the DSP directly and output video directly to the framebuffer. OpenCV can interact with GStreamer, so this appears to be a very promising direction for the project. See this article for more information and a demonstration. That article also has a link to a minimal beagle image that provides a native framebuffer video player without requiring X.