Moodbar

This was a weekend of tinkering. One of the things I discovered was Moodbar --- a cute blend of signal-processing and UI design. The motivation for this toy was that intra-song navigation is still largely guesswork since one has to know in minutes/seconds where one would like to jump to in a song. Now why would you want to jump around a song? Well, if you're like me, you might want to jump to an interesting few bars of a musician's solo, for example. So rather than dealing with this antiquated interface:

this is what I can work with instead:

I'll point you to their paper for all the interesting details, but the gist of it is that for each few-millisecond slice of the song, the intensity of the power spectrum in the low, mid and high ranges is translated into R, G and B values respectively. What this means for those of you (yes you Jayita) who've forgotten signal-processing basics, bright shades mean loud bits, red parts denote bass-dominated parts of the song, while green and blue indicate more stuff in the soprano and higher regions. Here's what a few of my favorites look like:

"2 and 2" has 3 distinct regions corresponding to solos by Miguel Zenon (soprano sax), Joshua Redman (alto sax), and Eric Harland (drums). The horn solos are green, indicating a strong mid-range response, while Eric's solo is a mixture of bass (red) and high-frequency (blue) cymbal-work. It's also interesting to see the pattern of tension buildup and release in each solo; the colors start out dark, and build up to a bright glow towards the end of each solo (a common idiom in jazz).

Dream Theater's "Caught In A Web" has nice green highlights where James LaBrie's voice dominates the midrange. Now I know how to jump around those bits since I think he's typically the weakest element of the band's music (sorry Arjun!).

Surprisingly, jazz tends to display more structure in the moodbars --- probably because of the long solo sections featuring instruments in different tonal ranges. Bill Evan's "All of You" shows nice demarcations with Scott LaFaro's bass solo taking up the dark-red section in the middle followed by a short solo by Paul Motian on drums. Joe Henderson's blistering saxophone solo also shows up in bright green on McCoy Tyner's "Passion Dance".

Pop and rock moodbars are less organized, but song structure such as choruses (such as the bright loud cyan regions where "We Will Rock You" is yelled by the crowd) are easily identifiable too.

I should mention that the moodbar is available as a plugin to Amarok.