Difference between revisions of "EBC Exercise 17 Using ALSA for Audio Processing"

From eLinux.org
Jump to: navigation, search
m (Part c - Audio Loopthru)
(Modify audio_thread.c: Removed lots of details that may not have been right.)
Line 236: Line 236:
 
Note: Since lab06c is a combination of lab06a and lab06b - and only file that differs between them is audio_thread.c. Sure, you could have copied the files in reverse order, but we highly recommend copy them in the order above so that the following steps are consistent with your files/directories.
 
Note: Since lab06c is a combination of lab06a and lab06b - and only file that differs between them is audio_thread.c. Sure, you could have copied the files in reverse order, but we highly recommend copy them in the order above so that the following steps are consistent with your files/directories.
  
=== Modify audio_thread.c ===
+
=== Modify the files ===
  
* Open audio_thread.c in a text editor.
+
Modify the '''audio_thread.c''' and the other files as needed to make it work.
Navigate to lab06c_audio_loopthru/app/, then:
+
<pre>
+
gedit audio_thread.c &
+
</pre>
+
* Begin by removing the #define statement that sets INPUTFILE. While not absolutely necessary, we might as well clean up anything that will not be used later, and removing it here will help us catch any errors if we forget to remove some filerelated commands. (In the place of the #define INPUTFILE, in the next step we’ll add the proper command needed for the ALSA driver.)
+
 
+
* In audio_thread_fxn() in the declarations, modify the #define bit settings for the initMask to have three initialization states:
+
** ALSA_DRIVER_INITIALIZED
+
** INPUT_BUFFER_ALLOCATED
+
** OUTPUT_BUFFER_ALLOCATED
+
It doesn’t matter which bit you allocate to each, as long as they are each independent bits in the mask, i.e. 0x1, 0x2, 0x4, 0x8, etc.
+
 
+
* Remove the input file initialization code
+
** Remove the following code section in audio_thread_fxn() of audio_thread.c:
+
<pre>
+
/* Open input file */
+
...
+
</pre>
+
And this one, too:
+
<pre>
+
/* Record that input file was opened in initialization bitmask */
+
...
+
</pre>
+
* Modify the mode field of the sound device attributes to enable bidirectional operation. Before editing the sAttrs.mode variable, let’s first check what options are available in
+
  This needs to be fixed.
+
 
+
* Declare a new Buffer_Handle hBufIn and initialize it to NULL.
+
Since we’re doing a pass through application we could get by with a single buffer; reading into it using Sound_read(), then writing it back out with Sound_write().
+
 
+
However, when we add our audio processing to this thread, it will be helpful to have separate input and output buffers. So, in this lab we’ll go ahead and create a separate input buffer. Add the new buffer handle next to the one already allocated for hBufOut.
+
Buffer_Handle hBufIn
+
(Note: In place of audio processing – which we’ll add in a later lab exercise – in step 28 we’ll use the memcpy() function to copy the data from the input buffer to the output buffer.)
+
 
+
  Fix the above too.
+
 
+
* Allocate an audio input buffer.
+
Because we copied lab06b (audio playback) into lab06c, we already have a section of code that creates the audio output buffer. Inspect the code after the following banner:
+
<pre>
+
/* Initialize the output audio buffer */
+
</pre>
+
We still need this code. However, we need to add the INPUT side; i.e. the audio input buffer. The simplest way to do this is to copy this section of code (thru and including setting the initMask bit) from lab06a. When pasting, place it BEFORE the output section and modify it appropriately for the INPUT audio buffer.
+
Copy the entire section from:
+
<pre>
+
/* Create input read buffer */
+
</pre>
+
…TO…
+
<pre>
+
/* Record that the output buffer was allocated in bitmask */
+
initMask |= INPUT_BUFFER_ALLOCATED;
+
</pre>
+
INCLUSIVE. Then, paste it right ABOVE the output section.
+
Change the hBufOut handle in the copied section to hBufIn and …
+
… change the INITMASK bitmask to INPUT_BUFFER_ALLOCATED.
+
 
+
* Prime the Pump using Sound_read() rather than fread().
+
You’ll find the call that needs to be changed in the “// Prime the Pump” section just before
+
the while() loop.
+
As earlier in step 23, you canmcheck the Sound.h header file for the Sound_read() prototype.
+
For convenience, we’ve reprinted here:
+
Int Sound_read (Sound_Handle hSound, Buffer_Handle hBuf)
+
Also, don’t forget to change the DBG() statement that follows to reference hBufIn, instead of
+
inputfile.
+
 
+
----
+
Note: Following the Prime-thePump Sound_read(), you’ll notice two single writes. These
+
writes avoid any potential underflow condition on the output driver and does not add any
+
noticeable distortion. There is nothing you need to modify here – just FYI.
+
 
+
----
+
* Within the while() loop, replace fread() call with Sound_read() call.
+
If you need a hint, you can reference the audio_thread_fxn() in lab06a_audio_recorder.
+
Once again, don’t forget to change the DBG() statement to use hBufIn vs. inputfile.
+
 
+
* Create an audio pass-thru using memcpy() to copy data from the input to the output. You will need to use the Buffer_getUserPtr() function to get the memory pointer associated with each buffer. (The handles hBufIn and hBufOut are pointers to structures, not pointers to the underlying memory buffers).
+
 
+
For example, if you wanted to specify the pointer to the input buffer, you would use:
+
Buffer_getUserPtr(hBufIn)
+
You will also need to use the Buffer_getSize() function to determine how many bytes to
+
transfer. In this case, use the size of the output buffer:
+
Buffer_getSize(hBufOut)
+
The memcpy() prototype is as follows:
+
memcpy(void *write_to, void *read_from, int num_bytes);
+
In your code, substitute the functions provided above into the memcpy call to perform the
+
audio pass thru.
+
* Replace the file cleanup code with the Buffer_delete cleanup on hBufIn.
+
 Locate the Thread Delete Phase after the “cleanup:” tag in the audio_thread_fxn
+
of lab06c. Remove the code section labeled:
+
/* Close input file */
+
 Replace the file cleanup code’s fclose() with the proper Buffer_delete() of hBufIn.
+
* If you opened audio_thread.c in lab06a_audio_record, you can close it now.
+
Note, you should not need to save audio_thread.c from lab06a_audio_record because
+
you should not have modified this file, only copied sections from it to paste into lab06c.
+
31. Update the variable declarations at the beginning of audio_thread_fxn.
+
After cutting-and-pasting the code in the last few steps, a few new variables have been added
+
and one was removed. Update the variable declarations at the beginning of
+
audio_thread_fxn.
+
The following can be removed:
+
FILE *inputFile = NULL;
+
Make sure the following variable is declared:
+
Buffer_Handle hBufIn = NULL;
+
* Save and close audio_thread.c.
+
  
 
=== Build and Test ===
 
=== Build and Test ===

Revision as of 22:29, 9 December 2011


This lab demonstrate the Linux ALSA driver as well as basic file I/O. Parts a and b are inspection labs. While c requires you to combine the previous two parts.

  • Part a analyzes the function calls necessary to record audio from line input to a file.
  • Part b examines the function calls necessary to playback audio from a recorded audio file.
  • Part c combines a and b into a single application that loops the audio from input to output (i.e. audio loop-thru) without recording to a file. For an extra challenge, advanced users may want to try to build c without referring to the procedure.

Much of this lab was adapted for the Beagle from OMAP™/DaVinci™ System Integration using Linux Workshop.

Part a - Audio Record

The goal of this part is to analyze the function calls necessary to record audio from the Beagle's a line input to a file.


Note: The 1/8 inch audio in jack on the Beagle is a line in jack, not a microphone in. It doesn't not have the extra amplification needed for a microphone.


  • Copy the AudioThru files from here to your Beagle.
  • Change directories to AudioThru/lab06a_audio_record
  • List the files used to build this application:

File Inspection

  • Use a text editor to examine the new files in this application.
$ gedit *.c *.h

main.c

This is the entry point for the application. main() does the following:

  1. Creates a signal handler to trap the Ctrl-C signal (also called SIGINT, the interrupt signal). When this signal is sent to the application, the audioEnv.quit global variable is set to true to signal the audio thread to exit its main loop and begin cleanup.
  2. Calls the audio_thread_fxn() function to enter into the audio function.
  3. Upon completion of this function, the main routine checks – and reports – success or failure returned from the audio function.

audio_thread.c

audio_thread_fxn( ) encapsulates the code required to run the audio recorder. The lab06a_audio_recorder application is single-threaded, so the motivation for encapsulation in this manner may not be initially obvious. We will see in later labs – when combining audio and video in a multi-threaded program – why declaring this function (as opposed to running everything from main) is useful.

audio_thread_fxn utilizes the following:

audio_io_setup
Opens and configures the audio input driver. See audio_input_output.c for details.
while()
will execute until the envPtr->quit global variable is set to true.
    • Inside the while() loop, snd_pcm_readi() is used to read data from the audio input driver (ALSA) and fwrite() is used to write the data into a file. The i in snd_pcm_readi() means the left and right channels of the stereo sign are interleaved. At is, one 16-bit value is the left channel, the next 16-bit value is the right channel.
    • When the envPtr->quit variable is set to true (occurs when the user presses Ctrl-C in the terminal) this capture (record) process exits and the application proceeds to the cleanup phase before exiting.
initMask

It goes without saying, writing robust code – and debugging it – can be a tedious chore; it is further exasperated when using printf() statements as the primary means of providing debug information back to the programmer. To this end, we have employed an initMask to help keep track of resources opened (and closed) during the program.

The audio_thread_fxn() uses an initialization mask (initMask) to keep track of how many resources have been opened and initialized. Each bit in the mask corresponds to a resource; the bit positions in the initMask variable are #defined towards the top of the file.

/* The levels of initialization for initMask */
#define ALSA_INITIALIZED 0x1
#define INPUT_BUFFER_ALLOCATED 0x2
#define OUTPUT_FILE_OPENED 0x4
/* Only used to cleanup items that were initialized */
unsigned int initMask = 0x0;
    • When you OR the initMask with a #define'd value, the associated bit will get set in the initMask variable. For example,

InitMask.png

This is useful so that if an error occurs, the application will not attempt to close or free resources that were never opened or allocated. If you look down at the cleanup part of our audio_thread.c, you’ll see how we used the initMask variable to accomplish this.

Build and Run the application

  • Build the application using make, i.e. “make all”.
  • Test your audio connection:
arecord –f cd | aplay –f cd

This command uses the arecord (“ALSA recorder”) utility to capture audio and, instead of sending to a file, uses a Linux process pipe to send the data to the aplay (“ALSA player”) application. This will loop audio through the board. If you have a working audio input and the board is connected to a speaker, you should hear the audio play over the speakers. Press ctrl-c to quit.

On arecord and aplay, the –f option lets you change the format:

Quality # Channels Bits Rate
default mono 8-bit 8 KHZ
cd stereo 16-bit 44.1 KHz
dat stereo 16-bit 48 KHz
  • Execute the ./app_DEBUG.Beagle application.

The application is hard-coded (using a #define statement in audio_thread.c) to save the audio data to the file /tmp/audio.raw. Execute the application.

  • Press Ctrl-C to exit the application.

After a suitable amount of time press Ctrl-C in the terminal to exit from the application. You can list the /tmp/audio.raw file with the –lsa options to see the size of the file and verify that it has recorded properly:

ls –lsa /tmp/audio.raw

Recall that a signal handler was placed in main.c to trap the SIGINT (Ctrl-C) signal. When Ctrl-C is placed, this signal handler will execute, signaling the audio thread to exit its main loop, proceed to cleanup, and then exit.

  • Use the Linux aplay utility to confirm successful recording.

Because this application saves the audio as a raw stream you may also check that the record has operated properly using the aplay utility.

$ aplay –f dat /tmp/audio.raw

DBG vs ERR

Let’s explore the debugging features we’re using in our lab files. We are using two macros defined in the file debug.h. They are DBG() and ERR() – essentially, they are wrapper functions around an fprintf() function.

  • Add a new debug statement to your file.

In main.c, immediately after the signal handler function, add a DBG() statement:

// Set the signal callback for Ctrl-C
pSigPrev = signal( SIGINT, signal_handler );
DBG( "Registered SIGINT signal handler.\n" );
  • Build both debug and release profiles on the Beagle, run then and compare their outputs.
$ make                  // Defaults to making the DEBUG version
$ PROFILE=RELEASE make  // Make the RELEASE version
$ ./app_DEBUG.Beagle    // Runs the DEBUG version
$ ./app_RELEASE.BEAGLE  // Runs the RELEASE version

Does your new statement show up in the terminal when you execute the program app_DEBUG.Beagle? Does it show up when app_RELEASE.Beagle?

  • Switch from DBG() to ERR(), then once again, build, run and compare both profiles. What is the difference between DBG and ERR?
  • Either Delete the new ERR() statement, or switch it back to DBG().

We don’t really need this statement, so feel free to remove it. On the other hand, if you want to leave the new debugging statement, we recommend that you, at the very least, change it back to a DBG() statement.

Part b - Audio Playback

The goal of this part is to analyze the function calls necessary to play back audio from a recorded file to the driver.

  • Copy the AudioThru files from here to your Beagle.
  • Change directories to AudioThru/lab06b_audio_playback
  • Use a text editor to examine audio_thread.c.

Only audio_thread.c has changed from the lab06a_audio_record application. Let’s look at some of the differences:

    • The ‘create’ part of the audio_thread_fxn():
      • Uses the fopen() function call to open a file for playback.
      • Uses audio_io_setup() and configure the audio output driver.
      • The malloc() function allocates a RAM buffer to store the audio data from the input file before it is written to the audio driver.
    • Inside the while() loop:
      • fread() method is used to read audio data from the input file (/tmp/audio.raw)
      • snd_pcm_writei() method is used to write the data to the ALSA driver. The i, like in the snd_pcm_readi(), mean the data is interleaved.
      • When the envPtr->quit variable is set to true, the loop exits. (This occurs when the user presses Ctrl-C in the terminal.)
    • Finally, review the “cleanup” phase, which runs right before exiting. (This basically undo’s the steps in the create/setup phase).

Build and Run the Application

  • Build the application using make.
  • Make sure that audio.raw was created properly.

List the contents of /tmp with the “-lsa” flags setting to verify that /tmp/audio.raw exists and has a greater than zero filesize.

The application is hard coded (using a #define statement in audio_thread.c) to read data from the file /tmp/audio.raw.


Note: If the Beagle is rebooted after running the lab6a_audio_record application, the /tmp/audio.raw file is erased. If this happens, run it again in either DEBUG or RELEASE mode to re-record the /tmp/audio.raw file

Finally, you can return to lab06b_audio_playback and run the playback utility.


  • Execute the ./app_DEBUG.Beagle application.

The application should play back the audio that was recorded in lab06a_audio_record and then exit. If you do not wish to hear all of the audio, press Ctrl-C to exit.

Questions about: audio_thread.c

  1. Which function call is used in the while() loop to read audio data from the line input via the ALSA driver?
  2. In the while( ) loop there is an fread() function (similar to fwrite() function, lab06a). For the file read (or write) function, a FILE pointer is the last parameter passed. What is the purpose of the FILE pointer, and where does it come from? (In other words, what function is used to generate valid FILE pointers from which read and write operations can be made?)

Part c - Audio Loopthru

In this part, you will combine parts a and b into a single loopthru application.

What do we need to change?

Before we start copying, cutting, and pasting files and code, let’s think about what must be done to get the loop-thru lab to work.

  • In our Lab06a_audio_record application, we used fwrite() to PUT (i.e. write) the audio data to audio.raw. Which function was used to GET (read) the audio data from the ALSA driver?
GET audio data: ________________________________________________
PUT audio data: fwrite() inputBuffer -> audio.raw
  • Similarly, in the Lab06b_audio_plaback application we used the snd_pcm_writei() function to PUT the data to the ALSA driver. But, how did we GET (i.e. read) the audio data?
GET audio data: ________________________________________________
PUT audio data: snd_pcm_writei() outputBuffer -> hSound
  • Now, in Lab06c_audio_loopthru we want to create an audio pass-thru application. Which two functions should be used to read and write data to/from ALSA driver?
GET audio data: ________________________________________________
PUT audio data: ________________________________________________

File Management

  • Begin by copying all files from lab06b_audio_playback into lab06c_audio_loopthru with the following:
$ cd ..
$ mkdir –p lab06c_audio_loopthru
$ cp –R –f lab06b_audio_playback/* lab06c_audio_loopthru

The mkdir “–p” option prevents an error if the directory already exists.

The cp “-R” options says to recurse directories, while the “-f” option forces over write if the file already exists.


Note: Since lab06c is a combination of lab06a and lab06b - and only file that differs between them is audio_thread.c. Sure, you could have copied the files in reverse order, but we highly recommend copy them in the order above so that the following steps are consistent with your files/directories.

Modify the files

Modify the audio_thread.c and the other files as needed to make it work.

Build and Test

  • Build and run the application.
$ make
$ app_DEBUG.Beagle

You should hear audio playing.

Assignment

There are many interesting things you can do now that you have the framework for bringing live audio into and out of the Beagle.

  1. What is the delay, in ms, from the input to the output?
  2. How does BLOCKSIZE (initially 48000) impact this delay. Try changing it and keep detailed records of your experiments.
  3. In audio_input_output.c there is a parameter called periods which is initially 4. How does changing this impact the delay?
  4. What's the minimum delay you can get an not drop samples? Report values for 8k and 48k sampling rates.
  5. Use htop to esitmate the total % of CPU time being used to pass through the audio.
  6. Try changing the sampling rate. How does the % CPU change when switching from 48K to 8K sample per second?
  7. Verify that the left input is going to the left speaker and the right input to the right. Replace the memcpy() with code to switch the left and right channels.
  8. Write code the independently change the gain on the left and right channels.
  9. Write code to delay one channel by N samples. Be careful, You'll need to carry some samples over from one buffer to the next.
  10. Apply a filter to the audio.