Clavilux 2000 – Interactive instrument for generative music visualization from Jonas Heuer on Vimeo.

Think of playing musical notes for a moment, or close your eyes while fingering a piano keyboard. Odds are, some visual – however abstract – pops into your mind. Visualizing musical notes is second nature in the digital realm, once a note and an image can each be represented with numbers.

Clavilux 2000 by Jonas Friedemann Heuer is one of the latest works to run with the idea. As you play notes, beams of color drift up from the keyboard. In 3D mode, those beams take on a lovely, subtle quality. The model itself isn’t new, owing the notes-as-lines model to player pianos (or even music boxes), and recalling light organs. But there is something intuitive about this model – and I can imagine it being a terrific way to encourage someone to practice. (Well, that or else it could be distracting while practicing!)

Description. Thanks to Yifan Mai for the link; via infosthetics.com, a fantastic resource for exploring ways of visualizing information.

Clavilux 2000 is a music visualization installation that produces generative real-time animations of music. It consists of a computer running vvvv patch hooked up to a MIDI keyboard and projector. Every note played on the keyboard produces a stripe, whose proportions and color correspond to how the note was played. For instance, the color is mapped to the tonality of the note via the circle of fifths, thus visualizing harmonic consonance and dissonance. Besides looking really cool, it also thus creates unique “fingerprints” of each performance.

Clavilux 2000 is extremely close in design to a key 1995 work by media artist Toshio Iwai, known most recently for the Yamaha Tenori-On and Nintendo-published ElectroPlankton DS (each of which uses ideas from the earlier project). Piano–as image media and related works employed both inputs and outputs. (in the installation, visitors could use a trackball to enter note events visually on a screen; in performance with Ryuichi Sakamoto, the work used a piano. In each, events fly off perpendicular to the piano keyboard as beams of light, just as in the work here. That’s not a criticism, incidentally – even without seeing Iwai’s work, it’s a logical solution, because the keyboard organizes notes into an array of thin rectangles (the keys).

Unfortunately, Iwai’s work is not well-documented online; videos of these pieces have been removed. I do have a few resources for you, however. At bottom, there is a video of a 2006 Ars Electronica talk on the visual interface for music. (I have some video of Toshio’s similar thoughts around the launch of the Tenori-On which I should publish.) And for more:

Toshio Iwai talking about the visual-musical interface[artintelligence]

Toshio Iwai keynote at Futuresonic [pixelsumo]

http://www.nydigitalsalon.org/10/artwork.php?artwork=57

Soundbooth Beta 2 Article

Given that Audition is already bundled in the Windows video production suites, maybe this means Mac bundle customers and lower-end customers will also see the app as an included product, which would be nice. We’ll know soon enough.

Equally interesting is the reaction Soundbooth is already generating. Apparently me comparing an audio editor to Photoshop pricked up some ears:

Adobe Soundbooth Beta 2: Now Easier, More Photoshop-y [digg]

And, of course, digg’s trolls immediately took to the comments. One good idea out of the discussion: OGG export, which has a lot of appeal to me and wouldn’t even require a license fee for Adobe. Mac users are still understandably upset that there’s no PowerPC version, but given the availability of Sound Studio and Peak LE, and the Intel-specific optimizations in the Intel-native Soundbooth, this argument seems like a waste of time.

Are Graphics Tools Intuitive?

More interesting, though, a lot of readers were upset that I called Photoshop intuitive. Personally, I think the basic lasso editing tool and graphical painting metaphors are quite intuitive, and I think a spectral view is one of the best ways to visualize sound. These metaphors have become so familiar to computer users, in fact, that we forget they weren’t the creation of Adobe Photoshop at all. The lasso tool, and most other paint tools that are now as second-nature to us as windowed interfaces, are the invention of Bill Atkinson, while developing MacPaint for the original Mac (pictured at right, courtesy folklore.org):

MacPaint Evolution [Andy Hertzfeld, folklore.org]

It’s rare that someone really invents something, but in the case of the now-ubiquitous graphic editing metaphors on which most modern apps are based, Bill really did invent some of the basic tools. And I’ll argue with the commenters on digg: this doesn’t just make sense to graphics people. I remember being in school and using the original MacPaint and Hypercard for the first time. I didn’t need anyone to show me: I started messing around with the mouse, I saw what the tools did, and I’ve been using these tools ever since.

There are other lessons here, too. Apple in the early days took smart interface concepts and applied them universally. MacPaint tools were in HyperCard; shortcuts worked everywhere. Apple doesn’t do that nearly as well any more; even Final Cut Pro and Logic Pro feel like they were developed by different companies. Adobe has a lot more integration work to do, too, and certainly apps are much more complicated than they once were — not always a bad thing, as I’d rather use Photoshop CS3 than MacPaint. But leveraging good ideas across products is absolutely a good idea.

The other lesson, though, is that the possibilities are still open for new interface inventions. Before MacPaint, there was no MacPaint: Atkinson made it up. Why not make up new ways of using our computer? Of course, Apple had an advantage, in that they could build software around the hardware most people weren’t yet using (the mouse). That concept has deep implications for those of us in music, because physical activity has always been what music is about.

Bill Atkinson was one programmer with an idea, of course, not a huge corporate development effort. I suspect we’ll see more good ideas come from individual developers. Some of these interface ideas will be intuitive, others more subtle. But there are still plenty of ways of exploring sound and image that we haven’t discovered yet.

Oh, incidentally, digg doesn’t usually bring down CDM. We’re having serious server problems that seem to bring CDM crashing to a halt if we look at it the wrong way. We’re moving to a new, dedicated server next week. At that point, please digg us, slashdot us, tell everyone we’re hosting free nude photos (synth pr0n?), whatever. (Just don’t send any more spam than the 2,000 spam comments we get per day. We hate that.)