OCR Advisor and software mathematician Mark Fisher received some recent coverage in Wired and Co.Design for his fantastic renderings of whale, dolphin, bird, and insect sounds using a diagnostic function called “wavelets.” Wavelets are typically used to evaluate large data sets in a way that can look at the entire set while highlighting fine inner details. In this case Mark’s large data sets are sounds.
In most visual sound analysis either the spectral or the amplitude components of a sound are evaluated over time – as found in the OCR Sound Library Audiograms (rendered through Cornell’s “Raven” software). The amplitude (how loud a sound is) is fairly easy to render as it just requires creating a visual correlation of “louder sound to more of something.”
The spectral (frequency) components are a bit more complicated because you have to measure the amplitudes through a discreet set of “frequency bins” or filters and have the outputs show up in a way that is easy to understand. For example higher frequencies can appear “higher up” on the “y” axis, or “more to the right side” of an “x-y” graph.
While this common processing-and-display convention is useful and easy to read once you get the hang of it, it discards the finer details by processing the sound in big chunks.
Wavelets are a more complicated process. Instead of running a sound through a set of stationary filter bins and measuring the outputs of the bins, wavelet analysis (mathematically) throws little bursts of sound or “wavelets” at a sound and evaluates the interference. If you know the precise shape and timing of the wavelets, the interference patterns will tell you a lot about what you’re bouncing them off.
False killer whale wavelet mandala
There is still frequency on the “y axis” but the “x axis” displays interference in the time domain; and “how much” (or “how little”) is represented by how bright the patterns are. In this manner large and complicated sounds can yield meta-patterns that also reveal the tiny details.
This may seem a bit difficult to wrap your head around, but these patterns can tell volumes about the specifics of a sound – stuff that we might be able to hear but would have a hard time describing.
Mark is taking his wavelet analysis and bending them into circular “mandalas.” These mandalas can be useful, but they are also beautiful while telling us a lot about the details of the sounds that produced them. The complexities of the patterns also reflect the complexity of sound production and reception. Wavelet analysis might hold the key to how you can identify the voice of someone you haven’t heard in 20 years, or how a pod of dolphins instantaneously sort out their complex bioacoustic world.
You can also dispense with the heady stuff and just look at the gorgeous patterns on Mark’s AguaSonic website.
Very cool Mark!