The Writeup 2: Electric Boogaloo

Hello everyone!  I have some new, very exciting updates to my project!

For those who aren't familiar with my previous works, my last big project was a frequency analyzer I made using a Stellaris Launchpad and an 8x8 LED panel MSP430 Launchpad booster pack that I repurposed for Stellaris.

Well, I've made a few upgrades since then :)

Project Overview

• Timer triggered ADC for signal capture at user specified frequency (4 KHz - 80 KHz, defaults to 26 KHz)
• Smart uDMA function determines, based on sampling frequency, whether to do signal processing on a fresh array of new data or rotate in a smaller amount of new data for each DSP loop
• 2048 point FFT for frequency bin calculation
• ARM CMSIS DSP library used for Hamming Window application and FFT calculation
• Touchscreen used for variable minimum display frequency, maximum display frequency, sampling frequency, number of display elements on screen (aka number of bars), debug information, and visual effects
• Dynamic calculations made based on current sampling frequency and max/min display frequencies to generate a logarithmic scale for frequency axis.
• Timer controlled 18 FPS display rate (adjustable to up to 30 FPS at compile time)

Hardware

Audio Capture

The first big update to my project is on the signal capture side.  I've always been much more of a software guy, but I figured this would be an excellent opportunity to brush the rust off of my board layout skillset.

In order to get the Launchpad to be able to read line level audio, it is necessary to add a DC bias to the audio signal.  Line level audio is centered at ground and has a ~1.6 V swing, which is a problem because the ADC peripherals on the Launchpad can only take input signals between 3.3 V and ground.  To fix this, I use a simple circuit (a capacitor and two resistors) to get the signal centered at 1.65 V with a 1.6 V swing.

Instead of using a breadboard for this, though, I decided to draw up the schematic using Eagle's schematic capture tool, then do a simple board layout (again using Eagle).

Once the board layout was done, I uploaded the schematic to Batch PCB.  I was originally planning on ordering a few boards from there, but one of my coworkers found out about what I was doing and offered to pay for my board to be fabricated by Advanced Circuits instead.  As much as I like supporting batch PCB, I'm not one to turn down free fabrication :)

Overall, I was very happy with how the board ended up turning out.  For anyone interested in using/modifying/viewing my raw board design files, they can be found along with the bill of materials for my board in the hardware directory of my github.

Display

The next big change, and arguably the most impressive addition to my project, was the display.  Kentec just recently released a 3.5 inch, 320 x 240 16 bit color display, complete with resistive touchscreen overlay on a booster pack for about $50.  Needless to say, as soon as I found out about its existence, I incessantly pestered my contact on the Stellaris applications team until he let me have one to play with.  I had to make some changes to the sample driver provided for the display.  Specifically, the touch screen uses a timer to trigger ADC captures, which interfered with my audio capture functions.  With a few modifications, though, I was able to integrate the Kentec booster pack into my project pretty easily.

Software

The addition of the touch screen allowed for a much, much wider scope of functionality for my software.  I had originally designed my code such that the sampling frequency and display parameters were easy to change at compile time via a few pre-processor macros, but it took a bit of work to get those options all configurable at run time in an intuitive fashion.  That said, the software I wrote for the LMF120 is now responsible for the following functions:

• ADC Sampling at a user defined frequency frequency
• Digital Signal Processing on the captured audio data (2048 point FFT)
• GPIO based communication with the Kentec display
• Graphical User Interface for changing display and DSP parameters at runtime

I'm a software developer by day, so I did all that I could to make those steps run as efficiently as possible.  I worked primarily in Code Composer Studio, and my source code is available on github.

Audio Capture

For the ADC interaction, I ended up using three separate peripherals to give me an incredibly software efficient audio capture process.  First, I set up a timer to count down at the sampling frequency (defaulted to 26 KHz).  I set up the timer to trigger an ADC capture at the hardware level.  The ADC capture complete signal was set up to initiate a uDMA transfer, which was configured to move the data from the ADC capture FIFO to a global array of samples.  The result is very, very software efficient; One line of code starts the sampling timer, and SAMPLE_SIZE/26000 seconds later a software interrupt occurs, letting you know that the data has been captured and is ready in the sample array.  I was very proud of the efficiency of this process :-)

I did encounter some difficulty when trying to run my code at low sampling frequencies.  The flow of the above method is basically

1. Capture Data
2. Halt Capture
3. Process Data
4. Start Next Capture
5. Display Data

The problem with this becomes obvious in the example of using 4 KHz for the sampling frequency.  The 2048 point FFT requires the use of 2048 samples, and the above method means that every time we want to do a new signal processing loop, we have to get a fresh 2048 samples.  If we're sampling at 4 KHz, that means we have at most about 2 sets of fresh samples per second to process.  That means we can only update our display at 2 frames per second.
To get around this, I have two different uDMA algorithms.  The first (fast) method is as described above.  The second (slow) method decreases the uDMA size to 256 and uses a ping pong buffer to store the data.  When the uDMA transfer for the ping buffer is complete, the uDMA engine switches to storing data in the poing buffer.  While the pong buffer is filling, the data in the sample array is shifted left by 1792 samples, then the data from the ping buffer is copied into the top 256 spots of the sample array.  The joys of flexibility :)

Digital Signal Processing

I am fortunate in that I've been playing around with audio hardware for years in the form of running soundboards and messing around with studio mixing and recording.  My last DSP class was a long, long time ago, but my interest in audio gave me a decent foundation in audio based DSP.  I am also fortunate in that ARM has a DSP library specifically designed to take advantage of the M4F's floating point instructions.  TI even has an application note detailing how to build that DSP library in Code Composer Studio.  From what I've heard, the amazingly handsome, charismatic, genius of an engineer who wrote the note did an incredible job on it, and it is wonderfully thorough and well written.  It might also be a little out of date (as ARM released a new version of CMSIS since the app note was published), but it's still quite useful.

With those tools at my disposal, the DSP portion of my code wasn't too difficult to figure out once I wrapped my head around CMSIS.  Step one is to multiply the samples by a windowing function.  I chose a hamming window, because I vaguely remember hearing that that was good for audio.  Next, I take the FFT of the input data.  I'm pretty proud of this part as well; the M4F ended up being powerful enough to take a 2048 point fft on the input data, which gives you 1024 frequency bins, each of which represents the energy found in a 12.7 Hz wide frequency band.  So once I have the fft output, I take the complex magnitude of each sample, giving me the total power found in each frequency bin.

Touch Screen/Configuration

Once I had the data captured and processed, I needed a way for the user to configure how it would be displayed.  I ended up using three screens total for my project: the main "Frequency Display" screen and two Configuration screens.  The first configuration screen is used to modify various frequency parameters and the number of display elements to use.  This is done using StellarisWare's graphics library, using slider widgets and pushbutton widgets.  Each parameter has a slider that can be used for setting the value, and a + and - button that can be used to fine tune the values.  On this screen, the user can change the maximum display frequency, the minimum display frequency, the sampling frequency, and the number of display elements (aka the number of bars) used on the display.  Once these have been entered, my code determines which bars need to correspond to which bins in order to best facilitate a logarithmic frequency display.  This normally means the lower 1/3 of the bars each represent less than 10 bins worth of FFT data, and the upper bars contain the summation of at times hundreds of bins.  If the user ever tries to get the screen to display at a greater granularity than possible (like having only 10 Hz of separation between lower and upper frequency while running at a 40 KHz sampling rate), the display will automatically be set to use as many bars as possible while still not displaying over the user's requested frequency range.

Display

The communication to the Kentec display is handled via a parallel GPIO interface.  I started out using Stellaris' Graphics Library (grlib) for everything, and was quite impressed with it.  Going into this project, I had almost no experience working with grlib.  Despite that, it ended up only taking me three hours worth of work to go from using my 8x8 LED display to having 8 bars of generic height behaving as desired on the Kentec display.  I like to think that I'm a pretty decent programmer, but I honestly feel that getting that far that fast is much more a sign of a well designed library than it is a sign of an intelligent coder.

For actually sending data to the display, I was able to use a driver that Kentec provided.  Sadly, I had to make a few modifications; the Kentec driver uses the same timer to control the same ADC sequencer that I was using for my audio capture, so I ended up changing to a different timer that performed a processor triggered ADC capture for the Kentec display.  Also, the display's draw functions all operated by drawing horizontal lines, which is nonideal if you're trying to draw lots of thin vertical lines, so I added in a function to allow for drawing images and lines by column as opposed to by row.  In hindsight, I think the display allows for changing from "portrait" to "landscape" orientation, so I probably could have just rotated my image and set the display to landscaped, but it's a bit late for that realization now.

With my modifications to the Kentec driver, I was able to get around 30-40 frames per second (depending on sampling frequency).  I found that the display looked a bit better if I cut it back to 15 frames per second though.  The instantaneous nature of the FFT meant that if I updated as fast as data was being processed, impulses in various frequency ranges would not remain on the display long enough for my eye to catch them and make sense of them.  I use a simple timer peripheral to keep the frame rate constant over time.

Credits, development time, and shaggy hair

All photographs on this post were taken by my wonderful wife, whose work can be found on her website.

In total, this project took about a month and a half to get from my last release to its current state.  I could likely have finished it in less time, but I have also been working on another big project in my free time lately; raising my first daughter :)

She was born in mid October, meaning she was about 3 weeks old when I started working on the code for this release.  She tends to fall asleep pretty easily when she's in a baby bjorn-style harness, so most of my coding was done between 9 and midnight while wearing a ten pound child on my chest :)  The unfortunate downside to this has been that between my real 8-5 job at TI (which actually has nothing to do with Launchpad), working on this project, and keeping that little lady happy, I haven't quite been able to maintain the sharp look I used to sport.  So my apologies if I look a bit shaggy in the explanation video for this one... it's been a few days since I've had a chance to have a proper shave :)

Feature Complete!!

At 2:30 this morning, the source code for my next update officially hit feature complete.  It will take me a few days still to clean up everything for a release, but I should be able to make a public git push, a new writeup, and a few new videos by the end of the week.  This is a big one, folks :)

The Secret Sauce

Source code is up!
https://github.com/EuphonistiHack/launchpad-freq-analyzer.git

I'm assuming that TI will follow the standard procedure for Stellaris installs for Launchpad, which is to say that when you install StellarisWare, the file structure will be such that you have a root (defaulted to C:/StellarisWare) in which driverlib will be copied.  That root will also contain a  boards directory, which in turn should contain a directory for the lm4f120.  If you do a git clone off of the above address, you'll find a similar file structure.  You should just be able to copy the led_equalizer project into the lm4f120 board directory, and copy dsplib into the stellarisWare root.

In it's current state, the only thing I've verified working is the Code Composer Studio build.  I have included project files that should at least be 95% good for all the other Stellaris supported environments: a makefile for you command line GCC enthusiasts and project files for Keil uVision, IAR Embedded Workbench, and CodeSourcery's IDE.  I'm guessing that if you tried to import and build into any of those as is, you will hit two problems.

For one, there's a config array that the uDMA engine uses that needs to be stored on a 1024 byte boundary.  I have some pragmas in place to handle that for CCS and IAR, but I haven't yet figured out the syntax for handling this in other compilers.  Shouldn't be too hard to figure out, and hopefully I'll have that done soon.

The second issue will be with the CMSIS library.  The current release contains a precompiled .lib and the headers necessary to interface with it.  This was compiled with the tms470 compiler, though, so it won't work with anything other than CCS.  If you download CMSIS for yourself, it should come with precompiled binaries for use with uVision, GCC, and CodeSourcery.  You'll want to add those to dsplib, then edit your project's library include path/list to point to the proper binary.  I am considering including these in dsplib, but I'm not really sure what ARM's policy is on redistributing their binaries, and I'd prefer not get sued out of existence :)

Enjoy!

The writeup

Hi, everybody!

I like playing with microcontrollers.  When I heard that TI was going to release a hobbyist centered Stellaris evaluation kit, the ek-lm4f120XL, I got excited.  When I convinced a coworker to let me have a rev0 board to work with, I got even more excited.  I decided that I would combine my love of microcontroller projects with my love of all things audio and make an LED frequency analyzer.

Many thanks to my wife, Heather Wills, for taking a decent photo of the board for me!

Hardware

The total cost of my setup is about $25 (the majority of which is from the LED display).  The Stellaris ADC requires that its input signal be a low impedance signal that is between ground and 3.3V.  This is not consistent with line level audio, so I implemented a very simple circuit on a breadboard to condition the input signal using an NTE987 opamp that I grabbed from Fry's, two 47k resistors, and a 1 uF capacitor.  The output of this circuit goes to one of the ADC input pins on the launchpad.  I found an MSP430 booster pack that is essentially an SPI controlled 8x8 LED array that I decided to use for the display.

Bill of Materials

ek-l4f120XL: $5

olimex MOD-LED8x8: $15

Opamp: $2

Resistors/Capacitors: Negligible (had them lying around in my wiring kit)

Total Cost: $22

Implementation

The signal condition circuit has to perform two functions.  First, it must bias the signal to center at ~1.65V and be limited to a 1.65V swing.  Line level audio is centered at ground and has at most around a 1.5V swing, so a simple resistor network can be used to bias this up to 1.65V.  Second, the output must be a signal with low enough impedance to power the switched capacitor array that the ADC input uses for its signal capture.  A low grade opamp can be wired up at unity gain to accomplish this.  The final result looks something like this:

For the input, I also realized that I'd need a headphone jack with exposed leads, which I picked up at Radio Shack for something like $1.50.  Were I a more patient engineer, I'm sure I could find something cheaper on digikey or mouser.  I slapped all of that together on my breadboard, which gave me the following:

The last piece of hardware work was connecting the Olimex booster pack to the Launchpad.  Unfortunately, I had to blue wire this, as the SPI RX pin on the booster pack was not aligned with the SPI TX pin on the launchpad.  Not a big deal, though, as the interface to the board is only five pins (SPI_CLK, SPI_TX, SPI_LATCH, VCC, GND).

Software

The software running on the Stellaris was responsible for the following functions:

• ADC Sampling at a specific frequency
• Digital Signal Processing on the captured audio data
• SPI communication with the LED array

I'm a software developer by day, so I did all that I could to make those steps run as efficiently as possible.  The source code I'm using is up on github at https://github.com/EuphonistiHack/launchpad-freq-analyzer.git

Audio Capture

For the ADC interaction, I ended up using three separate peripherals to give me an incredibly software efficient audio capture process.  First, I set up a timer to count down at 44.6 kHz.  I set up the timer to trigger an ADC capture at the hardware level.  The ADC capture complete signal was set up to initiate a uDMA transfer, which was configured to move the data from the ADC capture FIFO to a global array of samples.  The result is very, very software efficient; One line of code starts the sampling timer, and SAMPLE_SIZE/44600 seconds later a software interrupt occurs, letting you know that the data has been captured and is ready in the sample array.  I was very proud of the efficiency of this process :-)

Digital Signal Processing

I am fortunate in that I've been playing around with audio hardware for years in the form of running soundboards and messing around with studio mixing and recording.  My last DSP class was a long, long time ago, but my interest in audio gave me a decent foundation in audio based DSP.  I am also fortunate in that ARM has a DSP library specifically designed to take advantage of the M4F's floating point instructions.  TI even has an application note detailing how to build that DSP library in Code Composer Studio.  From what I've heard, the amazingly handsome, charismatic, genius of an engineer who wrote the note did an incredible job on it, and it is wonderfully thorough and well written.  It might also be a little out of date (as ARM released a new version of CMSIS since the app note was published), but it's still good.

With those tools at my disposal, the DSP portion of my code wasn't too difficult to figure out once I wrapped my head around CMSIS.  Step one is to multiply the samples by a windowing function.  I chose a hamming window, because I vaguely remember hearing that that was good for audio.  Next, I take the FFT of the input data.  I'm pretty proud of this part as well; the M4F ended up being powerful enough to take a 2048 point fft on the input data, which gives you 1024 frequency bins, each of which represents the energy found in a 20.3 Hz wide frequency band.  So once I have the fft output, I take the complex magnitude of each sample, giving me the total power found in each frequency bin.

LED Display

To figure out how to translate 1024 frequency bins into 8 LEDs, I had to do some trial and error to figure out what looked the best.  I ended up splitting the 8 LEDs into logarithmicly spaced frequency ranges.  Each time I execute my signal processing loop, I calculate the average power found in the bins for a given frequency range (again using CMSIS for efficiency).  I divide this value by the maximum value I've seen in that frequency range.  Then, I just say if the result is greater than 1/8, turn on one LED; Greater than 2/8, turn on 2, etc.  If the result is bigger than the max, then turn on all 8 LEDs, and make that the new maximum. I also multiplied the current maximum by 0.999 to have a decay factor, which made it so my output wouldn't be decimated by a single spike in the audio.

To actually turn on the LEDs, I used a second timer for an easily configured refresh rate.  Each time the timer hits 0, I use the hardware SPI module to update the next column of the 8x8 LED display based on the value found by the above method.  The results are, if I say so myself, pretty impressive.

Future Work

I have a few ideas of where to go from here.  For one, the Olimex LED tile has an expansion header that can be used to have multiple tiles connected to the same SPI bus.  My code is theoretically written such that I can change a single macro to use a different number of frequency display elements (so I can go from 8 to 16 frequency LED columns with a simple recompile).  It'd be pretty fun to try expanding my LED tile to 16 columns wide.

A coworker of mine built what he calls his LED cube a few months ago.  Basically, it's an 8x8x8 cube of LEDs that functions very similarly to the olimex booster pack.  I have some cool ideas for how to do 3 dimensional frequency displays :)

I also found a booster pack that contains a 160x128 LCD display.  I'm pretty sure I can talk to the LCD just using Stellaris' graphics library without much work at all, and I think it'd be really visually impressive to expand my display to be contain 160 frequency bins.

Finally, it's been a long time since I've done a board layout from scratch, so I've been toying with the idea of making a custom PCB to get rid of the protoboard and blue wires that I currently have to use to make everything work.

Showing Off

For those interested in seeing how this works, I have a rather short video showing off the display!  For those who don't care much about the setup, you can fast forward to about the 2:05 mark to see the demo with music :)

I will likely add one more in the future that just shows off the finished working product with the audio overlayed directly on top of the video (as opposed to captured as it comes out of my headphones).

Show and tell

Video time!  I figured that it might be helpful to make a few videos explaining how my project works and showing it in action.  The first one is pretty basic.  It shows off the display handling white noise, a frequency sweep, and a few random songs.

This second video is a bit long, but I feel it's a pretty good explanation of my entire project, from signal capture all the way up to figuring out which LEDs to turn on.  It contains a brief demonstration of the project running with real input, but it's meant more as a condensed explanation of how everything works.

All Together Now!

My last big step was to combine my signal conditioning circuit, my audio capture functions, and my DSP functions and get them to play nicely together.  This was very, very surprisingly easy.  I expected to spend days, possibly weeks debugging the integrated code, but surprisingly, that never happened.

First, I grabbed an audio splitter from Fry's so I could hear the signal going into my conditioning circuit via my headphones.  Next, I modified the main loop of the project that was running my audio capture code to include a signal processing function.  This function would check to see if the uDMA interrupt had set the "data ready" flag, and if so, process the data using the method described in my last post, then update my debug UART to display the frequency range of the current peak.

Once this was compiled and running, I used a java based web applet (found at http://web.mit.edu/jorloff/www/beats/beats.html ) to play a sine wave at a frequency I could specify.  Much to my amazement, my integration required very little debugging, and in about an hour's time, I was able to see my code properly responding to signals ranging from 40 Hz to 20 kHz.  Next, I found a youtube video that contained a frequency sweep, and verified that as the pitch got higher, the observed peak increased as well.  Success!

The remainder of the necessary code was pretty easy.  The shift registers on the Olimex booster pack communicated over SPI, and a coworker of mine already had code running on that board to display an arbitrary character array (each array element is a column, each bit of the char is the row).  I copied his code into my project with some minor modifications, and got it working without much issue.

The final hurdle was figuring out what data should go into the display matrix.  My previous audio experience was extensive enough for me to realize that the frequency should be displayed on a logarithmic scale.  I wrote another program in C to calculate which fft frequency bins would correspond to which LED based on the minimum and maximum display frequency, then stored these frequency breakpoints in a static array.  I then used the statistics portion of the CMSIS DSP library to find the average power in each LED range.

I modified my debug UART to keep track of the maximum magnitude I observed for a frequency bin when running the frequency sweep youtube video, and used that to determine what the maximum "loudness" value would be.  I figured I'd display power on a logarithmic scale as well, so I calculated power breakpoints using the value I observed when no audio was being played as the minimum power to display, and the maximum loudness value as the maximum power to display.  This gave me an array of power breakpoints.  So each time I went through my signal processing loop, I compared the average power observed in each LED range to these power breakpoints, and updated the display array based on that value.

The result was a far cry from ideal, but it was good enough to get me very excited.  My LED array flickered,  the middle and high bands didn't look like they were responding right, but I was able to play a random song and see the lower frequency LEDs light up whenever the kick drum or bass guitar hit.  It wasn't good enough to show off yet, but it was certainly good enough for me to call it a finished proof of concept.

Perfection never was a requirement, although some might say we desired it.

My proof of concept was complete, and I was excited.  Now, it was time to give the display some polish and make it actually look presentable.

Step one was getting rid of the LED flicker.  The olimex booster pack headers weren't quite compatible with our launchpad (the SPI TX pin to the olimex was sitting on an SPI RX pin on the launchpad), so originally I was having to use our software SPI driver to communicate with the booster pack's shift registers, as the software driver allowed for using generic GPIOs as opposed to SPI specific pins.  I already had most of the launchpad<--->olimex connectors blue wired though, so adding three more to allow my project to use the hardware SPI peripheral was no big deal, and made the display refresh function much less software intensive.  Next, I decided to allocate one of my unused timers to act as a display timer.  I set it up to throw an interrupt based on a predefined refresh rate macro, and update one column of the display every time the interrupt occurred.  The result was a much, much better looking display that had no flicker whatsoever.  I found that I needed to be careful about setting my refresh rate too high though; my signal processing was able to keep up pretty well, but the fact that I had no smoothing implemented in my software meant that if the refresh rate was too high, any sudden peaks in the frequency spectrum (like that resulting from the attack of a kick drum) would only appear as though the upper LEDs were briefly dimly illuminated.

The next step was cleaning up the fft results.  The biggest hurdle here was in figuring out how to calculate the "maximum loudness" value.  In my proof of concept, the power LED breakpoints were all calculated based on observed values and statically defined.  This was a bad approach as it did not account for any variance in  the volume of the incoming signal (like switching from a loud song to a soft song) and, more importantly, did not account for the fact that a tone at 200 kHz sounds much, much quieter than a tone of the same power at 12 kHz.  Also, the logarithmic frequency display meant that my lower frequency LED was the average of only 2 frequency bins, where my upper frequency LED was the average of about 400 frequency bins.  Overall, this caused the lower frequency LEDs to display much more power than the upper frequency LEDs.

I played around with a few different possible solutions for this, and finally came up with something that looked good to me.  I created an array of eight maximum power values, corresponding to the eight frequency ranges I was displaying.  Each time through the processing loop, I divided the current power value for the frequency range by the maximum power I'd observed on that range (updating the maximum power if the current power was bigger than the maximum, of course).  This gave me a normalized value between 0 and 1 of how "loud" that frequency range was in relation to what it had been in the past.  At that point, I just checked to see if the normalized value was greater than 1/8, 2/8, 3/8, ... and set the number of LEDs to illuminate for that range accordingly.  To account for frequency spikes or drastic changes in volume, I multiplied each maximum by a decay factor every time I went through my signal processing loop.

The resulting display looked much, much better.  The downside to this method was that I was displaying power values relative to previous values for that frequency range, as opposed to the power in one frequency range relative to the current power in all the other frequency ranges.  I'm not convinced this is the best method to use for computing the display values, but it is the best that I was able to come up with.

The final step to getting a good looking display was to figure out what frequency range I should display.  I found that even with my normalized power display method, every time a cymbal crash was made on the drums or a sibilant sound was made by a vocalist, the top three to four frequency LEDs would peak.  This was because I was displaying the entire range of human hearing (20 Hz to 22 kHz) scrunched into 8 LED ranges.  I probably should have used some scientific method to figure out which ranges to display, but I decided instead to use the trusty "try stuff until it looks good" approach.  I ended up finding that using 40 Hz as a minimum and about 12 kHz as a maximum gave me a really good looking LED display.  With these values, I was able to finally visually map which LEDs were corresponding to which instruments when a song was playing in real time.

After about two months of spending inordinate portions of my weekends and free evenings to blink LEDs, I had finally created something that I was proud of :)