Saturday, December 15, 2012

The Writeup 2: Electric Boogaloo


Hello everyone!  I have some new, very exciting updates to my project!

For those who aren't familiar with my previous works, my last big project was a frequency analyzer I made using a Stellaris Launchpad and an 8x8 LED panel MSP430 Launchpad booster pack that I repurposed for Stellaris.

Well, I've made a few upgrades since then :)








Project Overview

  • Timer triggered ADC for signal capture at user specified frequency (4 KHz - 80 KHz, defaults to 26 KHz)
  • Smart uDMA function determines, based on sampling frequency, whether to do signal processing on a fresh array of new data or rotate in a smaller amount of new data for each DSP loop
  • 2048 point FFT for frequency bin calculation
  • ARM CMSIS DSP library used for Hamming Window application and FFT calculation
  • Touchscreen used for variable minimum display frequency, maximum display frequency, sampling frequency, number of display elements on screen (aka number of bars), debug information, and visual effects
  • Dynamic calculations made based on current sampling frequency and max/min display frequencies to generate a logarithmic scale for frequency axis.
  • Timer controlled 18 FPS display rate (adjustable to up to 30 FPS at compile time)

Hardware

Audio Capture

The first big update to my project is on the signal capture side.  I've always been much more of a software guy, but I figured this would be an excellent opportunity to brush the rust off of my board layout skillset.

In order to get the Launchpad to be able to read line level audio, it is necessary to add a DC bias to the audio signal.  Line level audio is centered at ground and has a ~1.6 V swing, which is a problem because the ADC peripherals on the Launchpad can only take input signals between 3.3 V and ground.  To fix this, I use a simple circuit (a capacitor and two resistors) to get the signal centered at 1.65 V with a 1.6 V swing.


Instead of using a breadboard for this, though, I decided to draw up the schematic using Eagle's schematic capture tool, then do a simple board layout (again using Eagle).


Once the board layout was done, I uploaded the schematic to Batch PCB.  I was originally planning on ordering a few boards from there, but one of my coworkers found out about what I was doing and offered to pay for my board to be fabricated by Advanced Circuits instead.  As much as I like supporting batch PCB, I'm not one to turn down free fabrication :)







Overall, I was very happy with how the board ended up turning out.  For anyone interested in using/modifying/viewing my raw board design files, they can be found along with the bill of materials for my board in the hardware directory of my github.

Display

The next big change, and arguably the most impressive addition to my project, was the display.  Kentec just recently released a 3.5 inch, 320 x 240 16 bit color display, complete with resistive touchscreen overlay on a booster pack for about $50.  Needless to say, as soon as I found out about its existence, I incessantly pestered my contact on the Stellaris applications team until he let me have one to play with.  I had to make some changes to the sample driver provided for the display.  Specifically, the touch screen uses a timer to trigger ADC captures, which interfered with my audio capture functions.  With a few modifications, though, I was able to integrate the Kentec booster pack into my project pretty easily.

Software

The addition of the touch screen allowed for a much, much wider scope of functionality for my software.  I had originally designed my code such that the sampling frequency and display parameters were easy to change at compile time via a few pre-processor macros, but it took a bit of work to get those options all configurable at run time in an intuitive fashion.  That said, the software I wrote for the LMF120 is now responsible for the following functions:
  • ADC Sampling at a user defined frequency frequency
  • Digital Signal Processing on the captured audio data (2048 point FFT)
  • GPIO based communication with the Kentec display
  • Graphical User Interface for changing display and DSP parameters at runtime
I'm a software developer by day, so I did all that I could to make those steps run as efficiently as possible.  I worked primarily in Code Composer Studio, and my source code is available on github.

Audio Capture

For the ADC interaction, I ended up using three separate peripherals to give me an incredibly software efficient audio capture process.  First, I set up a timer to count down at the sampling frequency (defaulted to 26 KHz).  I set up the timer to trigger an ADC capture at the hardware level.  The ADC capture complete signal was set up to initiate a uDMA transfer, which was configured to move the data from the ADC capture FIFO to a global array of samples.  The result is very, very software efficient; One line of code starts the sampling timer, and SAMPLE_SIZE/26000 seconds later a software interrupt occurs, letting you know that the data has been captured and is ready in the sample array.  I was very proud of the efficiency of this process :-)

I did encounter some difficulty when trying to run my code at low sampling frequencies.  The flow of the above method is basically
  1. Capture Data
  2. Halt Capture
  3. Process Data
  4. Start Next Capture
  5. Display Data
The problem with this becomes obvious in the example of using 4 KHz for the sampling frequency.  The 2048 point FFT requires the use of 2048 samples, and the above method means that every time we want to do a new signal processing loop, we have to get a fresh 2048 samples.  If we're sampling at 4 KHz, that means we have at most about 2 sets of fresh samples per second to process.  That means we can only update our display at 2 frames per second.
To get around this, I have two different uDMA algorithms.  The first (fast) method is as described above.  The second (slow) method decreases the uDMA size to 256 and uses a ping pong buffer to store the data.  When the uDMA transfer for the ping buffer is complete, the uDMA engine switches to storing data in the poing buffer.  While the pong buffer is filling, the data in the sample array is shifted left by 1792 samples, then the data from the ping buffer is copied into the top 256 spots of the sample array.  The joys of flexibility :)

Digital Signal Processing

I am fortunate in that I've been playing around with audio hardware for years in the form of running soundboards and messing around with studio mixing and recording.  My last DSP class was a long, long time ago, but my interest in audio gave me a decent foundation in audio based DSP.  I am also fortunate in that ARM has a DSP library specifically designed to take advantage of the M4F's floating point instructions.  TI even has an application note detailing how to build that DSP library in Code Composer Studio.  From what I've heard, the amazingly handsome, charismatic, genius of an engineer who wrote the note did an incredible job on it, and it is wonderfully thorough and well written.  It might also be a little out of date (as ARM released a new version of CMSIS since the app note was published), but it's still quite useful.

With those tools at my disposal, the DSP portion of my code wasn't too difficult to figure out once I wrapped my head around CMSIS.  Step one is to multiply the samples by a windowing function.  I chose a hamming window, because I vaguely remember hearing that that was good for audio.  Next, I take the FFT of the input data.  I'm pretty proud of this part as well; the M4F ended up being powerful enough to take a 2048 point fft on the input data, which gives you 1024 frequency bins, each of which represents the energy found in a 12.7 Hz wide frequency band.  So once I have the fft output, I take the complex magnitude of each sample, giving me the total power found in each frequency bin.

Touch Screen/Configuration

Once I had the data captured and processed, I needed a way for the user to configure how it would be displayed.  I ended up using three screens total for my project: the main "Frequency Display" screen and two Configuration screens.  The first configuration screen is used to modify various frequency parameters and the number of display elements to use.  This is done using StellarisWare's graphics library, using slider widgets and pushbutton widgets.  Each parameter has a slider that can be used for setting the value, and a + and - button that can be used to fine tune the values.  On this screen, the user can change the maximum display frequency, the minimum display frequency, the sampling frequency, and the number of display elements (aka the number of bars) used on the display.  Once these have been entered, my code determines which bars need to correspond to which bins in order to best facilitate a logarithmic frequency display.  This normally means the lower 1/3 of the bars each represent less than 10 bins worth of FFT data, and the upper bars contain the summation of at times hundreds of bins.  If the user ever tries to get the screen to display at a greater granularity than possible (like having only 10 Hz of separation between lower and upper frequency while running at a 40 KHz sampling rate), the display will automatically be set to use as many bars as possible while still not displaying over the user's requested frequency range.

Display

The communication to the Kentec display is handled via a parallel GPIO interface.  I started out using Stellaris' Graphics Library (grlib) for everything, and was quite impressed with it.  Going into this project, I had almost no experience working with grlib.  Despite that, it ended up only taking me three hours worth of work to go from using my 8x8 LED display to having 8 bars of generic height behaving as desired on the Kentec display.  I like to think that I'm a pretty decent programmer, but I honestly feel that getting that far that fast is much more a sign of a well designed library than it is a sign of an intelligent coder.

For actually sending data to the display, I was able to use a driver that Kentec provided.  Sadly, I had to make a few modifications; the Kentec driver uses the same timer to control the same ADC sequencer that I was using for my audio capture, so I ended up changing to a different timer that performed a processor triggered ADC capture for the Kentec display.  Also, the display's draw functions all operated by drawing horizontal lines, which is nonideal if you're trying to draw lots of thin vertical lines, so I added in a function to allow for drawing images and lines by column as opposed to by row.  In hindsight, I think the display allows for changing from "portrait" to "landscape" orientation, so I probably could have just rotated my image and set the display to landscaped, but it's a bit late for that realization now.


With my modifications to the Kentec driver, I was able to get around 30-40 frames per second (depending on sampling frequency).  I found that the display looked a bit better if I cut it back to 15 frames per second though.  The instantaneous nature of the FFT meant that if I updated as fast as data was being processed, impulses in various frequency ranges would not remain on the display long enough for my eye to catch them and make sense of them.  I use a simple timer peripheral to keep the frame rate constant over time.

Credits, development time, and shaggy hair

All photographs on this post were taken by my wonderful wife, whose work can be found on her website.

In total, this project took about a month and a half to get from my last release to its current state.  I could likely have finished it in less time, but I have also been working on another big project in my free time lately; raising my first daughter :)


She was born in mid October, meaning she was about 3 weeks old when I started working on the code for this release.  She tends to fall asleep pretty easily when she's in a baby bjorn-style harness, so most of my coding was done between 9 and midnight while wearing a ten pound child on my chest :)  The unfortunate downside to this has been that between my real 8-5 job at TI (which actually has nothing to do with Launchpad), working on this project, and keeping that little lady happy, I haven't quite been able to maintain the sharp look I used to sport.  So my apologies if I look a bit shaggy in the explanation video for this one... it's been a few days since I've had a chance to have a proper shave :)

Sunday, December 9, 2012

Feature Complete!!

At 2:30 this morning, the source code for my next update officially hit feature complete.  It will take me a few days still to clean up everything for a release, but I should be able to make a public git push, a new writeup, and a few new videos by the end of the week.  This is a big one, folks :)

Sunday, September 9, 2012

The Secret Sauce

Source code is up!
https://github.com/EuphonistiHack/launchpad-freq-analyzer.git

I'm assuming that TI will follow the standard procedure for Stellaris installs for Launchpad, which is to say that when you install StellarisWare, the file structure will be such that you have a root (defaulted to C:/StellarisWare) in which driverlib will be copied.  That root will also contain a  boards directory, which in turn should contain a directory for the lm4f120.  If you do a git clone off of the above address, you'll find a similar file structure.  You should just be able to copy the led_equalizer project into the lm4f120 board directory, and copy dsplib into the stellarisWare root.

In it's current state, the only thing I've verified working is the Code Composer Studio build.  I have included project files that should at least be 95% good for all the other Stellaris supported environments: a makefile for you command line GCC enthusiasts and project files for Keil uVision, IAR Embedded Workbench, and CodeSourcery's IDE.  I'm guessing that if you tried to import and build into any of those as is, you will hit two problems.

For one, there's a config array that the uDMA engine uses that needs to be stored on a 1024 byte boundary.  I have some pragmas in place to handle that for CCS and IAR, but I haven't yet figured out the syntax for handling this in other compilers.  Shouldn't be too hard to figure out, and hopefully I'll have that done soon.

The second issue will be with the CMSIS library.  The current release contains a precompiled .lib and the headers necessary to interface with it.  This was compiled with the tms470 compiler, though, so it won't work with anything other than CCS.  If you download CMSIS for yourself, it should come with precompiled binaries for use with uVision, GCC, and CodeSourcery.  You'll want to add those to dsplib, then edit your project's library include path/list to point to the proper binary.  I am considering including these in dsplib, but I'm not really sure what ARM's policy is on redistributing their binaries, and I'd prefer not get sued out of existence :)

Enjoy!

Wednesday, September 5, 2012

The writeup

Hi, everybody!

I like playing with microcontrollers.  When I heard that TI was going to release a hobbyist centered Stellaris evaluation kit, the ek-lm4f120XL, I got excited.  When I convinced a coworker to let me have a rev0 board to work with, I got even more excited.  I decided that I would combine my love of microcontroller projects with my love of all things audio and make an LED frequency analyzer.

Many thanks to my wife, Heather Wills, for taking a decent photo of the board for me!

Hardware


The total cost of my setup is about $25 (the majority of which is from the LED display).  The Stellaris ADC requires that its input signal be a low impedance signal that is between ground and 3.3V.  This is not consistent with line level audio, so I implemented a very simple circuit on a breadboard to condition the input signal using an NTE987 opamp that I grabbed from Fry's, two 47k resistors, and a 1 uF capacitor.  The output of this circuit goes to one of the ADC input pins on the launchpad.  I found an MSP430 booster pack that is essentially an SPI controlled 8x8 LED array that I decided to use for the display.

Bill of Materials

ek-l4f120XL: $5
olimex MOD-LED8x8: $15
Opamp: $2
Resistors/Capacitors: Negligible (had them lying around in my wiring kit)

Total Cost: $22

Implementation

The signal condition circuit has to perform two functions.  First, it must bias the signal to center at ~1.65V and be limited to a 1.65V swing.  Line level audio is centered at ground and has at most around a 1.5V swing, so a simple resistor network can be used to bias this up to 1.65V.  Second, the output must be a signal with low enough impedance to power the switched capacitor array that the ADC input uses for its signal capture.  A low grade opamp can be wired up at unity gain to accomplish this.  The final result looks something like this:


For the input, I also realized that I'd need a headphone jack with exposed leads, which I picked up at Radio Shack for something like $1.50.  Were I a more patient engineer, I'm sure I could find something cheaper on digikey or mouser.  I slapped all of that together on my breadboard, which gave me the following:


The last piece of hardware work was connecting the Olimex booster pack to the Launchpad.  Unfortunately, I had to blue wire this, as the SPI RX pin on the booster pack was not aligned with the SPI TX pin on the launchpad.  Not a big deal, though, as the interface to the board is only five pins (SPI_CLK, SPI_TX, SPI_LATCH, VCC, GND).

Software


The software running on the Stellaris was responsible for the following functions:
  • ADC Sampling at a specific frequency
  • Digital Signal Processing on the captured audio data
  • SPI communication with the LED array
I'm a software developer by day, so I did all that I could to make those steps run as efficiently as possible.  The source code I'm using is up on github at https://github.com/EuphonistiHack/launchpad-freq-analyzer.git

Audio Capture

For the ADC interaction, I ended up using three separate peripherals to give me an incredibly software efficient audio capture process.  First, I set up a timer to count down at 44.6 kHz.  I set up the timer to trigger an ADC capture at the hardware level.  The ADC capture complete signal was set up to initiate a uDMA transfer, which was configured to move the data from the ADC capture FIFO to a global array of samples.  The result is very, very software efficient; One line of code starts the sampling timer, and SAMPLE_SIZE/44600 seconds later a software interrupt occurs, letting you know that the data has been captured and is ready in the sample array.  I was very proud of the efficiency of this process :-)

Digital Signal Processing

I am fortunate in that I've been playing around with audio hardware for years in the form of running soundboards and messing around with studio mixing and recording.  My last DSP class was a long, long time ago, but my interest in audio gave me a decent foundation in audio based DSP.  I am also fortunate in that ARM has a DSP library specifically designed to take advantage of the M4F's floating point instructions.  TI even has an application note detailing how to build that DSP library in Code Composer Studio.  From what I've heard, the amazingly handsome, charismatic, genius of an engineer who wrote the note did an incredible job on it, and it is wonderfully thorough and well written.  It might also be a little out of date (as ARM released a new version of CMSIS since the app note was published), but it's still good.

With those tools at my disposal, the DSP portion of my code wasn't too difficult to figure out once I wrapped my head around CMSIS.  Step one is to multiply the samples by a windowing function.  I chose a hamming window, because I vaguely remember hearing that that was good for audio.  Next, I take the FFT of the input data.  I'm pretty proud of this part as well; the M4F ended up being powerful enough to take a 2048 point fft on the input data, which gives you 1024 frequency bins, each of which represents the energy found in a 20.3 Hz wide frequency band.  So once I have the fft output, I take the complex magnitude of each sample, giving me the total power found in each frequency bin.

LED Display

To figure out how to translate 1024 frequency bins into 8 LEDs, I had to do some trial and error to figure out what looked the best.  I ended up splitting the 8 LEDs into logarithmicly spaced frequency ranges.  Each time I execute my signal processing loop, I calculate the average power found in the bins for a given frequency range (again using CMSIS for efficiency).  I divide this value by the maximum value I've seen in that frequency range.  Then, I just say if the result is greater than 1/8, turn on one LED; Greater than 2/8, turn on 2, etc.  If the result is bigger than the max, then turn on all 8 LEDs, and make that the new maximum. I also multiplied the current maximum by 0.999 to have a decay factor, which made it so my output wouldn't be decimated by a single spike in the audio.

To actually turn on the LEDs, I used a second timer for an easily configured refresh rate.  Each time the timer hits 0, I use the hardware SPI module to update the next column of the 8x8 LED display based on the value found by the above method.  The results are, if I say so myself, pretty impressive.

Future Work


I have a few ideas of where to go from here.  For one, the Olimex LED tile has an expansion header that can be used to have multiple tiles connected to the same SPI bus.  My code is theoretically written such that I can change a single macro to use a different number of frequency display elements (so I can go from 8 to 16 frequency LED columns with a simple recompile).  It'd be pretty fun to try expanding my LED tile to 16 columns wide.

A coworker of mine built what he calls his LED cube a few months ago.  Basically, it's an 8x8x8 cube of LEDs that functions very similarly to the olimex booster pack.  I have some cool ideas for how to do 3 dimensional frequency displays :)

I also found a booster pack that contains a 160x128 LCD display.  I'm pretty sure I can talk to the LCD just using Stellaris' graphics library without much work at all, and I think it'd be really visually impressive to expand my display to be contain 160 frequency bins.

Finally, it's been a long time since I've done a board layout from scratch, so I've been toying with the idea of making a custom PCB to get rid of the protoboard and blue wires that I currently have to use to make everything work.

Showing Off

For those interested in seeing how this works, I have a rather short video showing off the display!  For those who don't care much about the setup, you can fast forward to about the 2:05 mark to see the demo with music :)

I will likely add one more in the future that just shows off the finished working product with the audio overlayed directly on top of the video (as opposed to captured as it comes out of my headphones).

Friday, August 31, 2012

Show and tell



Video time!  I figured that it might be helpful to make a few videos explaining how my project works and showing it in action.  The first one is pretty basic.  It shows off the display handling white noise, a frequency sweep, and a few random songs.




This second video is a bit long, but I feel it's a pretty good explanation of my entire project, from signal capture all the way up to figuring out which LEDs to turn on.  It contains a brief demonstration of the project running with real input, but it's meant more as a condensed explanation of how everything works.









Saturday, August 25, 2012

All Together Now!

My last big step was to combine my signal conditioning circuit, my audio capture functions, and my DSP functions and get them to play nicely together.  This was very, very surprisingly easy.  I expected to spend days, possibly weeks debugging the integrated code, but surprisingly, that never happened.

First, I grabbed an audio splitter from Fry's so I could hear the signal going into my conditioning circuit via my headphones.  Next, I modified the main loop of the project that was running my audio capture code to include a signal processing function.  This function would check to see if the uDMA interrupt had set the "data ready" flag, and if so, process the data using the method described in my last post, then update my debug UART to display the frequency range of the current peak.

Once this was compiled and running, I used a java based web applet (found at http://web.mit.edu/jorloff/www/beats/beats.html ) to play a sine wave at a frequency I could specify.  Much to my amazement, my integration required very little debugging, and in about an hour's time, I was able to see my code properly responding to signals ranging from 40 Hz to 20 kHz.  Next, I found a youtube video that contained a frequency sweep, and verified that as the pitch got higher, the observed peak increased as well.  Success!

The remainder of the necessary code was pretty easy.  The shift registers on the Olimex booster pack communicated over SPI, and a coworker of mine already had code running on that board to display an arbitrary character array (each array element is a column, each bit of the char is the row).  I copied his code into my project with some minor modifications, and got it working without much issue.

The final hurdle was figuring out what data should go into the display matrix.  My previous audio experience was extensive enough for me to realize that the frequency should be displayed on a logarithmic scale.  I wrote another program in C to calculate which fft frequency bins would correspond to which LED based on the minimum and maximum display frequency, then stored these frequency breakpoints in a static array.  I then used the statistics portion of the CMSIS DSP library to find the average power in each LED range.

I modified my debug UART to keep track of the maximum magnitude I observed for a frequency bin when running the frequency sweep youtube video, and used that to determine what the maximum "loudness" value would be.  I figured I'd display power on a logarithmic scale as well, so I calculated power breakpoints using the value I observed when no audio was being played as the minimum power to display, and the maximum loudness value as the maximum power to display.  This gave me an array of power breakpoints.  So each time I went through my signal processing loop, I compared the average power observed in each LED range to these power breakpoints, and updated the display array based on that value.

The result was a far cry from ideal, but it was good enough to get me very excited.  My LED array flickered,  the middle and high bands didn't look like they were responding right, but I was able to play a random song and see the lower frequency LEDs light up whenever the kick drum or bass guitar hit.  It wasn't good enough to show off yet, but it was certainly good enough for me to call it a finished proof of concept.

Tuesday, August 21, 2012

Perfection never was a requirement, although some might say we desired it.

My proof of concept was complete, and I was excited.  Now, it was time to give the display some polish and make it actually look presentable.

Step one was getting rid of the LED flicker.  The olimex booster pack headers weren't quite compatible with our launchpad (the SPI TX pin to the olimex was sitting on an SPI RX pin on the launchpad), so originally I was having to use our software SPI driver to communicate with the booster pack's shift registers, as the software driver allowed for using generic GPIOs as opposed to SPI specific pins.  I already had most of the launchpad<--->olimex connectors blue wired though, so adding three more to allow my project to use the hardware SPI peripheral was no big deal, and made the display refresh function much less software intensive.  Next, I decided to allocate one of my unused timers to act as a display timer.  I set it up to throw an interrupt based on a predefined refresh rate macro, and update one column of the display every time the interrupt occurred.  The result was a much, much better looking display that had no flicker whatsoever.  I found that I needed to be careful about setting my refresh rate too high though; my signal processing was able to keep up pretty well, but the fact that I had no smoothing implemented in my software meant that if the refresh rate was too high, any sudden peaks in the frequency spectrum (like that resulting from the attack of a kick drum) would only appear as though the upper LEDs were briefly dimly illuminated.

The next step was cleaning up the fft results.  The biggest hurdle here was in figuring out how to calculate the "maximum loudness" value.  In my proof of concept, the power LED breakpoints were all calculated based on observed values and statically defined.  This was a bad approach as it did not account for any variance in  the volume of the incoming signal (like switching from a loud song to a soft song) and, more importantly, did not account for the fact that a tone at 200 kHz sounds much, much quieter than a tone of the same power at 12 kHz.  Also, the logarithmic frequency display meant that my lower frequency LED was the average of only 2 frequency bins, where my upper frequency LED was the average of about 400 frequency bins.  Overall, this caused the lower frequency LEDs to display much more power than the upper frequency LEDs.

I played around with a few different possible solutions for this, and finally came up with something that looked good to me.  I created an array of eight maximum power values, corresponding to the eight frequency ranges I was displaying.  Each time through the processing loop, I divided the current power value for the frequency range by the maximum power I'd observed on that range (updating the maximum power if the current power was bigger than the maximum, of course).  This gave me a normalized value between 0 and 1 of how "loud" that frequency range was in relation to what it had been in the past.  At that point, I just checked to see if the normalized value was greater than 1/8, 2/8, 3/8, ... and set the number of LEDs to illuminate for that range accordingly.  To account for frequency spikes or drastic changes in volume, I multiplied each maximum by a decay factor every time I went through my signal processing loop.

The resulting display looked much, much better.  The downside to this method was that I was displaying power values relative to previous values for that frequency range, as opposed to the power in one frequency range relative to the current power in all the other frequency ranges.  I'm not convinced this is the best method to use for computing the display values, but it is the best that I was able to come up with.

The final step to getting a good looking display was to figure out what frequency range I should display.  I found that even with my normalized power display method, every time a cymbal crash was made on the drums or a sibilant sound was made by a vocalist, the top three to four frequency LEDs would peak.  This was because I was displaying the entire range of human hearing (20 Hz to 22 kHz) scrunched into 8 LED ranges.  I probably should have used some scientific method to figure out which ranges to display, but I decided instead to use the trusty "try stuff until it looks good" approach.  I ended up finding that using 40 Hz as a minimum and about 12 kHz as a maximum gave me a really good looking LED display.  With these values, I was able to finally visually map which LEDs were corresponding to which instruments when a song was playing in real time.

After about two months of spending inordinate portions of my weekends and free evenings to blink LEDs, I had finally created something that I was proud of :)

Remember remember, the... whenever your last DSP class was

After ensuring that I had a good, valid stream of data coming in, I figured it was time to brush off the DSP textbook that had remained undisturbed on my shelf since college and start doing some analysis on the input audio signal.  I started by reading through the documentation and fft example that CMSIS ships with.  The fft example showed how to calculate a 1024 point transform on a complex input signal, which was not exactly what I wanted to do, but was close.  I added the example code to my project and verified that I was able to properly link to the CMSIS .lib that I had previously created when I wrote my app note on the topic (found at http://www.ti.com/lit/an/spma041a/spma041a.pdf).  That done, I switched over to cygwin and wrote a quick c program to emulate what the ADC would see if it was observing a perfect 440 Hz sine wave:


#include <stdio.h>
#include <math.h>

#define PI 3.141592653589793
#define ARR_SIZE 2048
#define SAMPLE_FREQ 44600
#define FREQ 440

int main(int argc, char** argv)
{
        float period = 1.0/SAMPLE_FREQ;
        float time = 0;
        float freqRadians = 2*PI*FREQ;
        unsigned long x[ARR_SIZE*2];
        int i, j = 0;
        printf("const float ti_sample_sine_vector[%d] = {\n", ARR_SIZE);
        for(i=0; i< ARR_SIZE; i++)
        {
                j++;
                //
                // ADC sees signal centered at 0x800 w/ 0x280 max swing
                //
                x[i] = (unsigned long)(640 * sin(freqRadians*time) + 0x800);
                printf("0x%04x,\t", x[i]);
                if(j == 8)
                {
                        printf("\n");
                        j = 0;
                }
                time += period;
        }

        printf("};\n\n\n");
        return 0;
}

Next, I wrote a similar snippet of code to generate the parameters of a windowing function.  I had read in a number of places that an fft is designed to work on a continuous, infinite signal, so when you instead use it for frequency analysis on a finite, non-continuous signal, it is best practice to multiply the samples by a window to reduce the amount of noise across frequency bins.  I think.  I'm nowhere close to a DSP expert, but this sounded like good information to me, so I wrote another snippet to generate a vector that would represent a hamming window:

#include <stdio.h>
#include <math.h>

#define PI 3.141592653589793
#define ARR_SIZE 2048

int main(int argc, char** argv)
{

        double x[ARR_SIZE*2];
        int i, j = 0;
        printf("const float ti_hamming_window_vector[%d] = {\n", ARR_SIZE);
        for(i=0; i< ARR_SIZE; i++)
        {
                j++;
                x[i] = 0.54 - 0.46*cos(2*PI*i/ARR_SIZE);
                //printf("%.18f,\t%.18f,\t", x[i], x[i]);
                printf("%.18f,\t", x[i]);
                //if(j == 4)
                if(j == 8)
                {
                        printf("\n");
                        j = 0;
                }
        }

        printf("};\n\n\n");
        return 0;
}

I copied the output of these two programs into two C files, and used them to write static vectors into the lm4f's flash.  Once that was done, I wrote a bit of code in my launchpad project that would multiply the sample by the hamming window, take a 2048 point real fft of that sample (which is implemented in CMSIS by doing a 1024 point complex fft, then running the result through a split fft process.  I don't know exactly what that means, but I think the result is a real fft of 2048 points).  I then took the complex magnitude of the results, found the maximum value in the result array, determined what frequency range that bin corresponded to, and used my debug UART to print that frequency range.  I had to massage the data and play around a bit with the input flags, but eventually I saw what I had been striving for: a message stating that the maximum frequency component of the signal I ran my DSP function on was between 435 and 458 Hz!

#define NUM_SAMPLES 2048
#define SAMPLING_FREQ 44600
#define INVERT_FFT 0
#define BIT_ORDER_FFT 1

extern q15_t ti_sample_sine_vector[NUM_SAMPLES];
float32_t nextTest[NUM_SAMPLES];

unsigned long g_ulADCValues[NUM_SAMPLES];

float32_t g_fFFTResult[NUM_SAMPLES * 2];


arm_rfft_instance_f32 fftStructure;
arm_cfft_radix4_instance_f32 cfftStructure;

float HzPerBin;

void
SampleSineTest(void)
{
unsigned long i;

arm_rfft_init_f32(&fftStructure, &cfftStructure, NUM_SAMPLES, INVERT_FFT, BIT_ORDER_FFT);
HzPerBin = (float)SAMPLING_FREQ / (float)NUM_SAMPLES;

for(i=0;i<NUM_SAMPLES;i++)
{
//
                // Recenter the signal at 0... might not be necessary
                //
g_fFFTResult[i] = ((float)ti_sample_sine_vector[i] - (float)0x800);
}

UARTprintf("test processing using sample 440 Hz sine wave\n");

arm_mult_f32(g_fFFTResult, ti_hamming_window_vector, g_fFFTResult, NUM_SAMPLES);
UARTprintf("Done with Hamming window multiplication\n");

UARTprintf("Calculating FFT\n");
arm_rfft_f32(&fftStructure, g_fFFTResult, g_fFFTResult);

UARTprintf("Calculating complex magnitude of each frequency bin\n");
arm_cmplx_mag_f32(g_fFFTResult, g_fFFTResult, NUM_SAMPLES * 2);


UARTprintf("Finding largest frequency bin\n");

arm_max_f32(g_fFFTResult, NUM_SAMPLES, &maxValue, &i);


UARTprintf("Peak is between %06d and %06d Hz: %06d\n\n", (int)(HzPerBin*i), (int)(HzPerBin*(i+1)), (int)maxValue);
}


My next step was to very, very anxiously put everything together...

Monday, August 20, 2012

To condense fact from the vapor of nuance

My first challenge was capturing the audio signal using the analog to digital converter running on lm4f120xl controller.  A quick bit of datasheet reading told me that there are two hardware limitations to the ADC that I have to worry about.  For one, the ADC expects a signal range of 0 to 3.3V.  Any voltages that dip below ground cause problems or are ignored.  The second limitation is that the ADC uses a switched capacitor array to implement its successive approximation register.  As a result, we have to have a low impedance signal feeding the ADC.

I talked with a few hardware knowledgeable friends of mine about this, and eventually came up with the following circuit to handle the above constraints:
The idea here is that the resistors will bias any input signal to between the ADC's maximum input voltage and ground.  The opamp will then amplify the signal enough to easily charge the switched capacitor array, giving us a good enough signal for our ADC to work with.  I dusted off my wiring kit, made a quick trip to Fry's, fired up the oscilloscope, and poked around enough to get myself confident that my line level input (courtesy of a male to male 1/8 inch audio cable, a stereo headphone jack, and my laptop's audio out) was being properly biased and conditioned.

I couldn't find a rail-to-rail opamp at Fry's, so I used this little number instead, which has a maximum output of Vcc-1.7.  Fortunately, I had a 5V bus I could tie into.

Hardware being handled, it was time for software!

I had played around with the ADCs on an lm4f232 a bit, so it didn't take much to get me to a point where I could capture the signal and print its level in Volts on one of the UARTs.  I figured there was no need to get fancy (yet), so I just used sequencer 3 of the ADC (which is limited to a one sample FIFO) to capture based on a software trigger.  Throw it in a while loop and UARTprintf ad nauseum.

Next was figuring out how to set up a controlled sampling frequency.  My first thought was to set up a continuous timer to countdown at 44.6 kHz (for some reason I picked 44.6 instead of 44.1... friends don't let friends drink and divide), throwing an interrupt every time it hit 0.  I was poking through the driverlib api for the timer getting this set up when I stumbled across the TimerControlTrigger function, which according to the documentation "Enables or disables the ADC trigger output."  It turns out that the hardware is already capable of doing what I had planned on doing through software: use a timer to trigger a capture in the ADC module.  Happy day!

A bit of debugging, a bit of re-reading, and another night of playing around with the launchpad got me to the point where I could use a timer to trigger an ADC capture, then on the ADC interrupt move the data into a sample array, then use a second timer to print out that array once per second.  The data looked good, though I couldn't think of a good way to verify that I was capturing with the expected sampling rate.

A normal person might have called it a day at that, but I long ago decided that the kind of person who spends his weekends and free nights messing about with microcontrollers is a far cry from a normal person.  The method of having the ADC interrupt copy the sample from the FIFO to the sample array reeked of inefficiency, especially when the documentation for the uDMA engine was tantalizingly sitting in the periphery of my vision.

For those not familiar with it, a uDMA engine can automatically move data from one place in memory to another, without needing the main processor to intervene.  As a result, you can do things like copying data from a peripheral's register map directly into a variable that's been allocated somewhere on your stack, which happens to be exactly what I wanted to do.

More datasheet reading, searching through old forum posts, and snooping around random example code led me to the confident belief that I could use the sequencer capture complete signal to trigger a uDMA transfer at the hardware level that would leave me with a much more efficient system.  My goal was to have a single "go" bit hit, then receive an interrupt SAMPLE_SIZE/44600 seconds later (44,600 samples/sec, SAMPLE_SIZE samples per signal processing loop), at which time the data would all be nicely arranged in my buffer.  Another night spend coding, and I was able to realize this goal.  The only limiting factor was that the uDMA engine has a max transaction limit of 1024, meaning that I would have to re-initiate the uDMA transfer after each 1024 samples until my buffer was full.  Irritating, but it only took a few lines of code, and was easily managed inside the interrupt handler.

After all of that, which took me about a week and half to get through (one weekend on the hardware, about another week on the software), I was very efficiently capturing audio data at an easily configured sampling frequency.  Woo and hoo!


In summation, my signal chain at this point was:
Timer0 triggers ADC0 capture; ADC0 capture is fed by custom circuit; ADC0 capture complete triggers uDMA transfer; uDMA transfer triggers interrupt, at which point our signal has been captured and moved into our sample buffer.

The Journey from There to Here

I've finally decided to start a blog for the fun little side project I've been playing with lately.  So, a condensed history of how I've spent my free time over the past few months follows herein:

February of 2012, I finished my first major, publicly visible task as an engineer on TI's Stellaris Microcontroller team; I published an application note detailing how to build ARM's CMSIS DSP library using our (in the corporate sense) preferred IDE, Code Composer Studio v5.  The app note is available at http://www.ti.com/lit/an/spma041a/spma041a.pdf and required a decent amount of work between myself and the engineers in Dallas who make the tms470 compiler to get fully running.  CMSIS uses a number of direct assembler calls and compiler intrinsic, the support for which required some work on my part.

I felt the application note was fairly well done, but I was left with a feeling of unfulfilled remorse, as I spent a very long time writing it, but never actually got to use the CMSIS DSP lib outside of the default test cases that CMSIS ships with.  As such, I started brainstorming ideas for projects that would incorporate heavy dsp functions.

I spent many years during and after college running soundboards and digital mixing/recording equipment, so I soon decided that I wanted to do something involving audio signal processing with CMSIS.  The obvious idea for me was to use the fft functionality to make the display for a graphic equalizer.  By April 2012, I had a spare lm4f232 evaluation kit in my office and seven hours' time to kill (compliments of a day of travelling for a friends' wedding), so I spent the time in the air learning how graphics lib worked and exploring the analog inputs of the 232.  The experience left me more knowledgeable but discouraged, as the 96x64 OLED just didn't look as visually impressive as I had hoped.


Around July 2012, several of my coworkers were talking about and working on the Stellaris Launchpad that will be releasing in a few months.  It took a bit of finagling, but I managed to secure an early run beta version to play with.  A bit of googling and I came across an MSP430 booster pack that Olimex released a few years ago: https://www.olimex.com/dev/msp-led8x8.html  In short, it's an 8 by 8 array of LEDs that uses SPI for communication.  Not nearly as much resolution as the 232 kit's OLED, but much more interesting to me.  Plus, blinking LEDs is what microcontrollers are all about!


Hardware in hand, I started to define the parameters of my project.