ECE497 Project Chronos Controlled Audio Effects

Team members: Joel Carlson, Brian Hulette

Executive Summary
With this project, the BeagleBoard is given an audio input, applies various effects to that audio, and then outputs the audio. A Texas Instruments Chronos watch is used to control which effects are applied, and the watch's accelerometer data is used to adjust the parameters of the effects.

At this point, the project is working with three effects. The three effects are a normal pass-thru, a lowpass filter, and a bandpass filter. The watch's accelerometer data controls the cutoff frequency for the lowpass filter, and it controls the center frequency and bandwidth for the bandpass filter. GStreamer is used to provide the audio effects.

At this point, the only things "not working" would be that we did not implement the highpass filter, reverb, or flanging that we initially planned on including. These would be fairly easy to integrate into the program as long as one can implement it with GStreamer.

Further information about GStreamer and the TI Chronos Watch can be found below.

Git Repository
The project is stored on GitHub: https://JoelsonCarl@github.com/JoelsonCarl/Chronos-BeagleBoard-Audio

Other URLs:

Read+Write Access: git@github.com:JoelsonCarl/Chronos-BeagleBoard-Audio.git

Read+Write Access: https://JoelsonCarl@github.com/JoelsonCarl/Chronos-BeagleBoard-Audio.git

Read Only Access: git://github.com/JoelsonCarl/Chronos-BeagleBoard-Audio.git

Compilation and Installation
The program was compiled on a host machine running Ubuntu 10.04 64-bit. The program was cross-compiled using the angstrom-2011.03-x86_64-linux-armv7a-linux-gnueabi-toolchain-qte-4.6.3 toolchain. To compile, the  variable within the file makefile_profile.mak must be changed to point to.

The  target within makefile_profile.mak may need to be changed. We had it  the executables to   on the BeagleBoard. Essentially, whether or not you use, you need some way to get the chronos-beagle_DEBUG.beagle and/or chronos-beagle_RELEASE.beagle file to the BeagleBoard.

Additional BeagleBoard Packages
The following additional packages were needed from those already on the SPEd2 image.
 * none

Kernel Modifications
The kernel included with the SPEd2 image does not require these modifications.

(The kernel we used is included in the git repository as uImage-chronos)

The kernel we used is the version pulled via the OpenEmbedded bitbake process (linux-omap-psp-2.6.32-r100a+gitr5fc29e7b2a76a64a739f857858ef0b98294aa155). After we had that kernel, we did the following.

Clean up the kernel, create a config file with the default settings, and then make the default kernel. $ make mrproper $ make omap3_beagle_defconfig $ make

After the kernel has compiled, run. In the configuration options, go into. Within there, put a "Y" on the following: Exit menuconfig and save the changes.
 * USB Modem (CDC ACM) Support
 * USB Serial Converter Support
 * USB TI 3410/5052 Serial Driver

Compile these changes and then build the uImage to boot from. $ make $ make uImage

Extra Hardware
The TI Chronos watch and RF access point can be purchased from http://focus.ti.com/docs/toolsw/folders/print/ez430-chronos.html?DCMP=Chronos&HQS=Other+OT+chronos.

User Instructions

 * 1) Kill any running PulseAudio processes. (Run  )
 * 2) Run the chronos-beagle_RELEASE.beagle file.  The output should say.
 * 3) On the Chronos watch use the bottom left (M2) button to navigate between various watch modes.  Go to either ACC or PPT mode.
 * 4) * PPT Mode
 * 5) ** Press the bottom right (S2) button to enable PPT mode and connect to the RF access point (a wireless symbol should start flashing on the watch).
 * 6) *** Pressing the top right (S1) button will start or stop the audio.
 * 7) *** Pressing the top left (M1) button will switch between different effects.
 * 8) **** 0 = Pass Thru
 * 9) **** 1 = Lowpass Filter
 * 10) **** 2 = Bandpass Filter
 * 11) *** Pressing the bottom left (M2) button currently has no functionality.
 * 12) *** Pressing S2 again will stop PPT mode.
 * 13) * ACC Mode
 * 14) ** Press the S2 button to enable ACC mode and connect to the RF access point (a wireless symbol should start flashing on the watch).
 * 15) *** Pass Thru - the accelerometer has no effect
 * 16) *** Lowpass - tilting the watch away from you lowers the cutoff frequency, tilting towards you increases the cutoff frequency
 * 17) *** Bandpass - tilting the watch away lowers the center frequency, tilting towards increases the center frequency, tilting left narrows the bandwidth, tilting right widens the bandwidth
 * 18) ** Press the S2 button again to stop ACC mode.
 * 19) Press Ctrl-C to stop the program.

Highlights
We have nothing to highlight or brag about beyond what was already described.

Chronos Watch
The chronos.c file contains the "main" running loop. This program opens up the serial port to interact with the RF access point. It then enters a loop where it repeatedly writes a command to the serial port that will request data from the watch. It then waits for that data to be returned. If the data is a button press or accelerometer data, the program will send the appropriate commands to the GStreamer pipeline. The relationship between watch events and GStreamer commands is consistent with that described in the User Instructions. After those commands are sent (or if no button press or accelerometer data was supplied), the program sends out another request for data and repeats the process over and over until an interrupt (Ctrl-C) is seen.

GStreamer Pipeline
We decided to use GStreamer to create an audio pipeline. GStreamer is a tool that allows you to connect various blocks (called elements) together to create a pipeline which multimedia data (in our case, just audio) flows through. Our pipeline is actually quite simple, it reads audio data using ALSA, then applies some effect to it (using one of the many built-in audio processing plugins), and finally outputs the audio using ALSA:

alsa-src -> audioconvert -> EFFECT -> audioconvert -> alsa-sink

The EFFECT element can be any number of GStreamer plug-ins. The three that are currently implemented are:

The sample pipelines show an example of how we are using each filter. You can run each of these at the command line on a computer with gstreamer-0.10 installed to get an idea of what each effect does.

The creator of the pipeline has the ability to:
 * Start the pipeline
 * Stop the pipeline
 * Restart the pipeline with a different effect block
 * Edit the effect parameters while the pipeline is running

Work Breakdown
For the audio effects, Brian looked into DSPLIB, GStreamer, and fidlib. He ended up using GStreamer in the project.

Only a pass-thru, lowpass, and bandpass audio effect were implemented. Further effects (using GStreamer) could be implemented very easily.

Conclusions
The program is fully functional. Other than the watch occasionally registering a single button press as multiple presses, it is generally an easily usable program. The switching between audio effects happens quickly and smoothly enough that it is not too unpleasant to listen to.

Figuring out how to interface with the Chronos RF access point/watch was not easy. The example programs we found were useful, but it was still not the same as a good set of documentation on what commands the RF access point expected as well as how the watch would return the data. We also had a lot of difficulty finding and linking a library for running audio filters. As mentioned earlier, we investigated several different alternatives for the audio processing. In the process of trying to get these alternatives working we learned a lot about cross-compiling and linking cross-compiled libraries.

Future additions could include adding more audio effects using GStreamer. Also, the bottom left watch button (M2) did not end up getting used, so someone could probably come up with a good use for that.

TI Chronos Watch
We are using the default watch firmware.

We found a Python script that pulls button presses (found here). We placed the original Python script in the git repository. We used this as a start for writing C code to get the button presses.

Chronos RF Access Point Commands
The RF access point takes certain commands over the serial port to communicate with the watch. Some of these commands are listed below. These were found here.

All values listed are hexadecimal.
 * Background Polling (functionality unknown)
 * PC to AP: FF 20 07 00 00 00 00
 * Response: FF 06 07 xx xx xx xx
 * xx xx xx xx: 4 byte watch address
 * Start Access Point
 * PC to AP: FF 07 03
 * Response: FF 06 03
 * Check AP Command (functionality unknown)
 * PC to AP: FF 00 04 00
 * Response: FF 06 04 03
 * It may be returning the state of the watch or AP
 * Request Data
 * PC to AP: FF 08 07 00 00 00 00
 * Response: FF 06 07 tt xx yy zz
 * tt: data type
 * 12 (M1 pressed)(xx,yy,zz = 00)
 * 22 (M2 pressed)(xx,yy,zz = 00)
 * 32 (S1 pressed)(xx,yy,zz = 00)
 * 01 (valid acc data)(xx,yy,zz = acc data)
 * FF (no/invalid data)
 * Stop Access Point
 * PC to AP: FF 09 03

Cross-Compiling with GStreamer
Cross-compiling code that uses GStreamer can be somewhat of a pain. The difficulty arises because you must find libraries that have been compiled for the ARM, and not for your host machine. After looking at some GStreamer documentation I found that compiling on the beagle is possible using the command: $ gcc -Wall $(pkg-config --cflags --libs gstreamer-0.10) pipe.c -o pipe The  portion of this command is replaced with the c-flags necessary to link GStreamer when it is run. Unfortunately, running this command on your host machine causes it to point to libraries compiled for your host, not the ARM. I had to figure out what C flags are actually being generated here so that I could modify them to point to the appropriate ARM libraries. Running the  command by itself yields:

$ pkg-config --cflags --libs gstreamer-0.10 -pthread -I$/usr/include/gstreamer-0.10 -I$/usr/include/glib-2.0 -I$/usr/lib/glib-2.0/include -I$/usr/include/libxml2 -pthread -lgstreamer-0.10 -lgobject-2.0 -lgmodule-2.0 -lxml2 -lgthread-2.0 -lrt -lglib-2.0 <<< Formatting added by author for clarification >>>

We can use all of this as-is except for the include flags. Versions of these libraries compiled for the ARM can be found deep in the toolchains directory. On my machine they are here:. So, in order to create a Makefile which properly compiles with GStreamer you must create the c-flags like so (with the directories modified slightly for your machine, of course): ARM_TOOLCHAIN_PATH := /home/hulettbh/Beagleboard/toolchains/usr/local/angstrom/arm

ARM_INCLUDE	   := $(ARM_TOOLCHAIN_PATH)/arm-angstrom-linux-gnueabi/usr/include ARM_LIB		   := $(ARM_TOOLCHAIN_PATH)/arm-angstrom-linux-gnueabi/usr/lib

ARM_CFLAGS += 	-Wall -pthread \ -I$(ARM_INCLUDE)/gstreamer-0.10 \ -I$(ARM_INCLUDE)/glib-2.0 \ -I$(ARM_LIB)/glib-2.0/include \ -I$(ARM_INCLUDE)/libxml2 \ -pthread -lgstreamer-0.10 -lgobject-2.0 -lgmodule-2.0 \ -lxml2 -lgthread-2.0 -lrt -lglib-2.0