Difference between revisions of "ECE497 Project BeagleBoard Kinect with Usable Framerates"
m (→User Instructions) |
m (→Highlights) |
||
Line 113: | Line 113: | ||
== Highlights == | == Highlights == | ||
− | + | This is a work in progress. Will hopefully be updated when we have something cool to show. | |
− | |||
− | |||
== Theory of Operation == | == Theory of Operation == |
Revision as of 10:16, 17 May 2011
Team members: Aaron Bamberger, Kyle Green, Ty Strayer
Contents
Executive Summary
There is currently no good solution for using a Kinect sensor with a BeagleBoard and a Linux build suited for embedded use (we are using Angstrom). Most current examples use a desktop-grade Linux build (Ubuntu chief among them) and run at horrendously slow frame rates (< 5 FPS is normal). We believe most of the problem is due to the Kinect drivers not being optimized to utilize the hardware the BeagleBoard has to offer. This project aims to improve the performance of the Kinect video display demo on the BeagleBoard by porting it to use the SGX graphics accelerator on the BeagleBoard using OpenGL ES. The Kinect video display demo uses a framework called GLUT (The OpenGL Utility Toolkit) to handle the creation of operating system shells and keyboard and mouse interaction. Unfortunately, OpenGL ES and the BeagleBoard do not support GLUT. We will need to replace the functionality of GLUT with the functionality of PVR, a toolkit with similar intentions to GLUT that is provided by the SGX hardware manufacturer and will run on the BeagleBoard.
Our progress to date includes configuring Angstrom with the proper video display mode so that the SGX graphics driver can run, installing the Kinect drivers under Angstrom and demonstrating that we have connectivity with the Kinect (through manipulation of its motors and LED), and getting some of the pre-compiled SGX demo programs to successfully run.
The issues still facing us include resolving some header and library dependencies in order to get the SGX demo programs compiling successfully on the BeagleBoard. Once we have succeeded with this, the only remaining step will be to modify the Kinect video display demo source code, replacing all GLUT functionality with its equivalent PVR functionality.
In conclusion, we believe we are very close to having a working Kinect video display demo running on the BeagleBoard with increased performance owing to its use of the BeagleBoards hardware graphics acceleration.
Installation Instructions
Kinect Drivers--libfreenect
Full information about libfreenect and its installation can be found at its site, here
Here I've reproduced just the sections that are useful for getting the drivers set up and working on the BeagleBoard
- Clone the git repository with all of the libfreenect source code in it
- Get all of the packages you need for building the drivers
- Run the cmake utility to generate the necessary makefiles
- Make the tiltdemo
- Run the tiltdemo
git clone git://github.com/OpenKinect/libfreenect.git
opkg install git
opkg install cmake libusb-1.0-0
cd libfreenect mkdir build
cmake ..
make tiltdemo
./tiltdemo
chmod +x tiltdemo
SGX Graphics Drivers
The PowerVR SGX graphics drivers are already installed as part of the SpED BeagleBoard image. We need to install the TI development package to get access to the development headers and example code. This package can either be installed on your host computer or on the BeagleBoard directly. When installed, it defaults to being set up to cross-compile, but for this project we are going to natively compile on the BeagleBoard to make our lives a bit simpler. This requires a couple of changes in the Makefiles, which will be detailed later in the instructions.
- Download the TI Omap SGX drivers
- Make the installer executable
- Install the Graphics SDK
- Explore the source tree a bit
chmod +x Graphics_SDK_setuplinux_4_03_00_02.bin
./Graphics_SDK_setuplinux_4_03_00_02.bin
Configuring the SpED Image to Work with the SGX Drivers
One slightly annoying feature of the SpED image is that even though it comes with the SGX drivers pre-configured, it defaults to a video mode that isn't compatible with the drivers, specifically it starts with 720p HD resolution with 32-bit color-depth. As far as we can tell, the SGX drivers won't run correctly unless the video mode is 640x480 resolution with 16-bit color depth. Clearly, this is an area that needs a bit more research, but for now it's just easier to set it up so that SpED boots at 640x480 with 16-bit color, so that the SGX drivers will work correctly.
There are two main ways of getting this set up.
- Modify the boot.scr script to set up the appropriate video mode at boot
- Get rid of the boot scripts and let the defaults take over
bootargs=console=ttyS2,115200n8 mem=80M@0x80000000 mem=384M@0x88000000 mpurate=1000 buddy=none camera=lbcm3m1 vram=16M omapfb.vram=0:8M,1:4M,2:4M omapfb.mode=dvi:hd720 omapdss.def_disp=dvi root=/dev/mmcblk0p2 rw rootfstype=ext3 rootwait mmc init setenv dvimode hd720 setenv vram 16M omapfb.vram=0:8M,1:4M,2:4M if test "${beaglerev}" = "AxBx"; then setenv optargs mem=80M@0x80000000 musb_hdrc.fifomode=5 elif test "${beaglerev}" = "Cx"; then setenv optargs mem=80M@0x80000000 mem=128M@0x88000000 musb_hdrc.fifomode=5 else setenv optargs mem=80M@0x80000000 mem=384M@0x88000000 fi run loaduimage run mmcboot
omapfb.mode=dvi:640x480MR-16@60 setenv dvimode 640x480MR-16@60
mkimage -A arm -O linux -T script -C none -a 0 -e 0 -n 'Some Descriptive Text Here' -d boot.cmd boot.scr
Once you've done either of the above methods, you can check if the mode got set properly by looking at the kernel command line. Go to /proc and cat cmdline. Somewhere in the command line you should see omapfb.mode=dvi:640x480-16@60. If you do, you should have the right video mode. As a final check, you can try and run one of the pre-compiled demo programs that came with the SGX drivers installation. Go to SDKPackage/Binaries/CommonX11/Demos and try running one of the demos you find there. I suggest ChameleonMan, because he runs like a chameleon and doesn't afraid of anything. If you can get one of the demos to run, congratulations, you have your video mode set up correctly.
Modifying the Kinect Display Program to Use PVR and SGX
This section is a work in progress. It will be updated as progress is made
User Instructions
Once the Kinect display program has been successfully ported from GLUT to PVR, using the program should be as simple as running the display program.
Highlights
This is a work in progress. Will hopefully be updated when we have something cool to show.
Theory of Operation
Give a high level overview of the structure of your software. Are you using GStreamer? Show a diagram of the pipeline. Are you running multiple tasks? Show what they do and how they interact.
Work Breakdown
List the major tasks in your project and who did what.
Also list here what doesn't work yet and when you think it will be finished and who is finishing it.
Conclusions
Give some concluding thoughts about the project. Suggest some future additions that could make it even more interesting.