Friday, August 31, 2012

Show and tell



Video time!  I figured that it might be helpful to make a few videos explaining how my project works and showing it in action.  The first one is pretty basic.  It shows off the display handling white noise, a frequency sweep, and a few random songs.




This second video is a bit long, but I feel it's a pretty good explanation of my entire project, from signal capture all the way up to figuring out which LEDs to turn on.  It contains a brief demonstration of the project running with real input, but it's meant more as a condensed explanation of how everything works.









Saturday, August 25, 2012

All Together Now!

My last big step was to combine my signal conditioning circuit, my audio capture functions, and my DSP functions and get them to play nicely together.  This was very, very surprisingly easy.  I expected to spend days, possibly weeks debugging the integrated code, but surprisingly, that never happened.

First, I grabbed an audio splitter from Fry's so I could hear the signal going into my conditioning circuit via my headphones.  Next, I modified the main loop of the project that was running my audio capture code to include a signal processing function.  This function would check to see if the uDMA interrupt had set the "data ready" flag, and if so, process the data using the method described in my last post, then update my debug UART to display the frequency range of the current peak.

Once this was compiled and running, I used a java based web applet (found at http://web.mit.edu/jorloff/www/beats/beats.html ) to play a sine wave at a frequency I could specify.  Much to my amazement, my integration required very little debugging, and in about an hour's time, I was able to see my code properly responding to signals ranging from 40 Hz to 20 kHz.  Next, I found a youtube video that contained a frequency sweep, and verified that as the pitch got higher, the observed peak increased as well.  Success!

The remainder of the necessary code was pretty easy.  The shift registers on the Olimex booster pack communicated over SPI, and a coworker of mine already had code running on that board to display an arbitrary character array (each array element is a column, each bit of the char is the row).  I copied his code into my project with some minor modifications, and got it working without much issue.

The final hurdle was figuring out what data should go into the display matrix.  My previous audio experience was extensive enough for me to realize that the frequency should be displayed on a logarithmic scale.  I wrote another program in C to calculate which fft frequency bins would correspond to which LED based on the minimum and maximum display frequency, then stored these frequency breakpoints in a static array.  I then used the statistics portion of the CMSIS DSP library to find the average power in each LED range.

I modified my debug UART to keep track of the maximum magnitude I observed for a frequency bin when running the frequency sweep youtube video, and used that to determine what the maximum "loudness" value would be.  I figured I'd display power on a logarithmic scale as well, so I calculated power breakpoints using the value I observed when no audio was being played as the minimum power to display, and the maximum loudness value as the maximum power to display.  This gave me an array of power breakpoints.  So each time I went through my signal processing loop, I compared the average power observed in each LED range to these power breakpoints, and updated the display array based on that value.

The result was a far cry from ideal, but it was good enough to get me very excited.  My LED array flickered,  the middle and high bands didn't look like they were responding right, but I was able to play a random song and see the lower frequency LEDs light up whenever the kick drum or bass guitar hit.  It wasn't good enough to show off yet, but it was certainly good enough for me to call it a finished proof of concept.

Tuesday, August 21, 2012

Perfection never was a requirement, although some might say we desired it.

My proof of concept was complete, and I was excited.  Now, it was time to give the display some polish and make it actually look presentable.

Step one was getting rid of the LED flicker.  The olimex booster pack headers weren't quite compatible with our launchpad (the SPI TX pin to the olimex was sitting on an SPI RX pin on the launchpad), so originally I was having to use our software SPI driver to communicate with the booster pack's shift registers, as the software driver allowed for using generic GPIOs as opposed to SPI specific pins.  I already had most of the launchpad<--->olimex connectors blue wired though, so adding three more to allow my project to use the hardware SPI peripheral was no big deal, and made the display refresh function much less software intensive.  Next, I decided to allocate one of my unused timers to act as a display timer.  I set it up to throw an interrupt based on a predefined refresh rate macro, and update one column of the display every time the interrupt occurred.  The result was a much, much better looking display that had no flicker whatsoever.  I found that I needed to be careful about setting my refresh rate too high though; my signal processing was able to keep up pretty well, but the fact that I had no smoothing implemented in my software meant that if the refresh rate was too high, any sudden peaks in the frequency spectrum (like that resulting from the attack of a kick drum) would only appear as though the upper LEDs were briefly dimly illuminated.

The next step was cleaning up the fft results.  The biggest hurdle here was in figuring out how to calculate the "maximum loudness" value.  In my proof of concept, the power LED breakpoints were all calculated based on observed values and statically defined.  This was a bad approach as it did not account for any variance in  the volume of the incoming signal (like switching from a loud song to a soft song) and, more importantly, did not account for the fact that a tone at 200 kHz sounds much, much quieter than a tone of the same power at 12 kHz.  Also, the logarithmic frequency display meant that my lower frequency LED was the average of only 2 frequency bins, where my upper frequency LED was the average of about 400 frequency bins.  Overall, this caused the lower frequency LEDs to display much more power than the upper frequency LEDs.

I played around with a few different possible solutions for this, and finally came up with something that looked good to me.  I created an array of eight maximum power values, corresponding to the eight frequency ranges I was displaying.  Each time through the processing loop, I divided the current power value for the frequency range by the maximum power I'd observed on that range (updating the maximum power if the current power was bigger than the maximum, of course).  This gave me a normalized value between 0 and 1 of how "loud" that frequency range was in relation to what it had been in the past.  At that point, I just checked to see if the normalized value was greater than 1/8, 2/8, 3/8, ... and set the number of LEDs to illuminate for that range accordingly.  To account for frequency spikes or drastic changes in volume, I multiplied each maximum by a decay factor every time I went through my signal processing loop.

The resulting display looked much, much better.  The downside to this method was that I was displaying power values relative to previous values for that frequency range, as opposed to the power in one frequency range relative to the current power in all the other frequency ranges.  I'm not convinced this is the best method to use for computing the display values, but it is the best that I was able to come up with.

The final step to getting a good looking display was to figure out what frequency range I should display.  I found that even with my normalized power display method, every time a cymbal crash was made on the drums or a sibilant sound was made by a vocalist, the top three to four frequency LEDs would peak.  This was because I was displaying the entire range of human hearing (20 Hz to 22 kHz) scrunched into 8 LED ranges.  I probably should have used some scientific method to figure out which ranges to display, but I decided instead to use the trusty "try stuff until it looks good" approach.  I ended up finding that using 40 Hz as a minimum and about 12 kHz as a maximum gave me a really good looking LED display.  With these values, I was able to finally visually map which LEDs were corresponding to which instruments when a song was playing in real time.

After about two months of spending inordinate portions of my weekends and free evenings to blink LEDs, I had finally created something that I was proud of :)

Remember remember, the... whenever your last DSP class was

After ensuring that I had a good, valid stream of data coming in, I figured it was time to brush off the DSP textbook that had remained undisturbed on my shelf since college and start doing some analysis on the input audio signal.  I started by reading through the documentation and fft example that CMSIS ships with.  The fft example showed how to calculate a 1024 point transform on a complex input signal, which was not exactly what I wanted to do, but was close.  I added the example code to my project and verified that I was able to properly link to the CMSIS .lib that I had previously created when I wrote my app note on the topic (found at http://www.ti.com/lit/an/spma041a/spma041a.pdf).  That done, I switched over to cygwin and wrote a quick c program to emulate what the ADC would see if it was observing a perfect 440 Hz sine wave:


#include <stdio.h>
#include <math.h>

#define PI 3.141592653589793
#define ARR_SIZE 2048
#define SAMPLE_FREQ 44600
#define FREQ 440

int main(int argc, char** argv)
{
        float period = 1.0/SAMPLE_FREQ;
        float time = 0;
        float freqRadians = 2*PI*FREQ;
        unsigned long x[ARR_SIZE*2];
        int i, j = 0;
        printf("const float ti_sample_sine_vector[%d] = {\n", ARR_SIZE);
        for(i=0; i< ARR_SIZE; i++)
        {
                j++;
                //
                // ADC sees signal centered at 0x800 w/ 0x280 max swing
                //
                x[i] = (unsigned long)(640 * sin(freqRadians*time) + 0x800);
                printf("0x%04x,\t", x[i]);
                if(j == 8)
                {
                        printf("\n");
                        j = 0;
                }
                time += period;
        }

        printf("};\n\n\n");
        return 0;
}

Next, I wrote a similar snippet of code to generate the parameters of a windowing function.  I had read in a number of places that an fft is designed to work on a continuous, infinite signal, so when you instead use it for frequency analysis on a finite, non-continuous signal, it is best practice to multiply the samples by a window to reduce the amount of noise across frequency bins.  I think.  I'm nowhere close to a DSP expert, but this sounded like good information to me, so I wrote another snippet to generate a vector that would represent a hamming window:

#include <stdio.h>
#include <math.h>

#define PI 3.141592653589793
#define ARR_SIZE 2048

int main(int argc, char** argv)
{

        double x[ARR_SIZE*2];
        int i, j = 0;
        printf("const float ti_hamming_window_vector[%d] = {\n", ARR_SIZE);
        for(i=0; i< ARR_SIZE; i++)
        {
                j++;
                x[i] = 0.54 - 0.46*cos(2*PI*i/ARR_SIZE);
                //printf("%.18f,\t%.18f,\t", x[i], x[i]);
                printf("%.18f,\t", x[i]);
                //if(j == 4)
                if(j == 8)
                {
                        printf("\n");
                        j = 0;
                }
        }

        printf("};\n\n\n");
        return 0;
}

I copied the output of these two programs into two C files, and used them to write static vectors into the lm4f's flash.  Once that was done, I wrote a bit of code in my launchpad project that would multiply the sample by the hamming window, take a 2048 point real fft of that sample (which is implemented in CMSIS by doing a 1024 point complex fft, then running the result through a split fft process.  I don't know exactly what that means, but I think the result is a real fft of 2048 points).  I then took the complex magnitude of the results, found the maximum value in the result array, determined what frequency range that bin corresponded to, and used my debug UART to print that frequency range.  I had to massage the data and play around a bit with the input flags, but eventually I saw what I had been striving for: a message stating that the maximum frequency component of the signal I ran my DSP function on was between 435 and 458 Hz!

#define NUM_SAMPLES 2048
#define SAMPLING_FREQ 44600
#define INVERT_FFT 0
#define BIT_ORDER_FFT 1

extern q15_t ti_sample_sine_vector[NUM_SAMPLES];
float32_t nextTest[NUM_SAMPLES];

unsigned long g_ulADCValues[NUM_SAMPLES];

float32_t g_fFFTResult[NUM_SAMPLES * 2];


arm_rfft_instance_f32 fftStructure;
arm_cfft_radix4_instance_f32 cfftStructure;

float HzPerBin;

void
SampleSineTest(void)
{
unsigned long i;

arm_rfft_init_f32(&fftStructure, &cfftStructure, NUM_SAMPLES, INVERT_FFT, BIT_ORDER_FFT);
HzPerBin = (float)SAMPLING_FREQ / (float)NUM_SAMPLES;

for(i=0;i<NUM_SAMPLES;i++)
{
//
                // Recenter the signal at 0... might not be necessary
                //
g_fFFTResult[i] = ((float)ti_sample_sine_vector[i] - (float)0x800);
}

UARTprintf("test processing using sample 440 Hz sine wave\n");

arm_mult_f32(g_fFFTResult, ti_hamming_window_vector, g_fFFTResult, NUM_SAMPLES);
UARTprintf("Done with Hamming window multiplication\n");

UARTprintf("Calculating FFT\n");
arm_rfft_f32(&fftStructure, g_fFFTResult, g_fFFTResult);

UARTprintf("Calculating complex magnitude of each frequency bin\n");
arm_cmplx_mag_f32(g_fFFTResult, g_fFFTResult, NUM_SAMPLES * 2);


UARTprintf("Finding largest frequency bin\n");

arm_max_f32(g_fFFTResult, NUM_SAMPLES, &maxValue, &i);


UARTprintf("Peak is between %06d and %06d Hz: %06d\n\n", (int)(HzPerBin*i), (int)(HzPerBin*(i+1)), (int)maxValue);
}


My next step was to very, very anxiously put everything together...

Monday, August 20, 2012

To condense fact from the vapor of nuance

My first challenge was capturing the audio signal using the analog to digital converter running on lm4f120xl controller.  A quick bit of datasheet reading told me that there are two hardware limitations to the ADC that I have to worry about.  For one, the ADC expects a signal range of 0 to 3.3V.  Any voltages that dip below ground cause problems or are ignored.  The second limitation is that the ADC uses a switched capacitor array to implement its successive approximation register.  As a result, we have to have a low impedance signal feeding the ADC.

I talked with a few hardware knowledgeable friends of mine about this, and eventually came up with the following circuit to handle the above constraints:
The idea here is that the resistors will bias any input signal to between the ADC's maximum input voltage and ground.  The opamp will then amplify the signal enough to easily charge the switched capacitor array, giving us a good enough signal for our ADC to work with.  I dusted off my wiring kit, made a quick trip to Fry's, fired up the oscilloscope, and poked around enough to get myself confident that my line level input (courtesy of a male to male 1/8 inch audio cable, a stereo headphone jack, and my laptop's audio out) was being properly biased and conditioned.

I couldn't find a rail-to-rail opamp at Fry's, so I used this little number instead, which has a maximum output of Vcc-1.7.  Fortunately, I had a 5V bus I could tie into.

Hardware being handled, it was time for software!

I had played around with the ADCs on an lm4f232 a bit, so it didn't take much to get me to a point where I could capture the signal and print its level in Volts on one of the UARTs.  I figured there was no need to get fancy (yet), so I just used sequencer 3 of the ADC (which is limited to a one sample FIFO) to capture based on a software trigger.  Throw it in a while loop and UARTprintf ad nauseum.

Next was figuring out how to set up a controlled sampling frequency.  My first thought was to set up a continuous timer to countdown at 44.6 kHz (for some reason I picked 44.6 instead of 44.1... friends don't let friends drink and divide), throwing an interrupt every time it hit 0.  I was poking through the driverlib api for the timer getting this set up when I stumbled across the TimerControlTrigger function, which according to the documentation "Enables or disables the ADC trigger output."  It turns out that the hardware is already capable of doing what I had planned on doing through software: use a timer to trigger a capture in the ADC module.  Happy day!

A bit of debugging, a bit of re-reading, and another night of playing around with the launchpad got me to the point where I could use a timer to trigger an ADC capture, then on the ADC interrupt move the data into a sample array, then use a second timer to print out that array once per second.  The data looked good, though I couldn't think of a good way to verify that I was capturing with the expected sampling rate.

A normal person might have called it a day at that, but I long ago decided that the kind of person who spends his weekends and free nights messing about with microcontrollers is a far cry from a normal person.  The method of having the ADC interrupt copy the sample from the FIFO to the sample array reeked of inefficiency, especially when the documentation for the uDMA engine was tantalizingly sitting in the periphery of my vision.

For those not familiar with it, a uDMA engine can automatically move data from one place in memory to another, without needing the main processor to intervene.  As a result, you can do things like copying data from a peripheral's register map directly into a variable that's been allocated somewhere on your stack, which happens to be exactly what I wanted to do.

More datasheet reading, searching through old forum posts, and snooping around random example code led me to the confident belief that I could use the sequencer capture complete signal to trigger a uDMA transfer at the hardware level that would leave me with a much more efficient system.  My goal was to have a single "go" bit hit, then receive an interrupt SAMPLE_SIZE/44600 seconds later (44,600 samples/sec, SAMPLE_SIZE samples per signal processing loop), at which time the data would all be nicely arranged in my buffer.  Another night spend coding, and I was able to realize this goal.  The only limiting factor was that the uDMA engine has a max transaction limit of 1024, meaning that I would have to re-initiate the uDMA transfer after each 1024 samples until my buffer was full.  Irritating, but it only took a few lines of code, and was easily managed inside the interrupt handler.

After all of that, which took me about a week and half to get through (one weekend on the hardware, about another week on the software), I was very efficiently capturing audio data at an easily configured sampling frequency.  Woo and hoo!


In summation, my signal chain at this point was:
Timer0 triggers ADC0 capture; ADC0 capture is fed by custom circuit; ADC0 capture complete triggers uDMA transfer; uDMA transfer triggers interrupt, at which point our signal has been captured and moved into our sample buffer.

The Journey from There to Here

I've finally decided to start a blog for the fun little side project I've been playing with lately.  So, a condensed history of how I've spent my free time over the past few months follows herein:

February of 2012, I finished my first major, publicly visible task as an engineer on TI's Stellaris Microcontroller team; I published an application note detailing how to build ARM's CMSIS DSP library using our (in the corporate sense) preferred IDE, Code Composer Studio v5.  The app note is available at http://www.ti.com/lit/an/spma041a/spma041a.pdf and required a decent amount of work between myself and the engineers in Dallas who make the tms470 compiler to get fully running.  CMSIS uses a number of direct assembler calls and compiler intrinsic, the support for which required some work on my part.

I felt the application note was fairly well done, but I was left with a feeling of unfulfilled remorse, as I spent a very long time writing it, but never actually got to use the CMSIS DSP lib outside of the default test cases that CMSIS ships with.  As such, I started brainstorming ideas for projects that would incorporate heavy dsp functions.

I spent many years during and after college running soundboards and digital mixing/recording equipment, so I soon decided that I wanted to do something involving audio signal processing with CMSIS.  The obvious idea for me was to use the fft functionality to make the display for a graphic equalizer.  By April 2012, I had a spare lm4f232 evaluation kit in my office and seven hours' time to kill (compliments of a day of travelling for a friends' wedding), so I spent the time in the air learning how graphics lib worked and exploring the analog inputs of the 232.  The experience left me more knowledgeable but discouraged, as the 96x64 OLED just didn't look as visually impressive as I had hoped.


Around July 2012, several of my coworkers were talking about and working on the Stellaris Launchpad that will be releasing in a few months.  It took a bit of finagling, but I managed to secure an early run beta version to play with.  A bit of googling and I came across an MSP430 booster pack that Olimex released a few years ago: https://www.olimex.com/dev/msp-led8x8.html  In short, it's an 8 by 8 array of LEDs that uses SPI for communication.  Not nearly as much resolution as the 232 kit's OLED, but much more interesting to me.  Plus, blinking LEDs is what microcontrollers are all about!


Hardware in hand, I started to define the parameters of my project.