Clavilux 2000 – Interactive instrument for generative music visualization from Jonas Heuer on Vimeo.

Think of playing musical notes for a moment, or close your eyes while fingering a piano keyboard. Odds are, some visual – however abstract – pops into your mind. Visualizing musical notes is second nature in the digital realm, once a note and an image can each be represented with numbers.

Clavilux 2000 by Jonas Friedemann Heuer is one of the latest works to run with the idea. As you play notes, beams of color drift up from the keyboard. In 3D mode, those beams take on a lovely, subtle quality. The model itself isn’t new, owing the notes-as-lines model to player pianos (or even music boxes), and recalling light organs. But there is something intuitive about this model – and I can imagine it being a terrific way to encourage someone to practice. (Well, that or else it could be distracting while practicing!)

Description. Thanks to Yifan Mai for the link; via infosthetics.com, a fantastic resource for exploring ways of visualizing information.

Clavilux 2000 is a music visualization installation that produces generative real-time animations of music. It consists of a computer running vvvv patch hooked up to a MIDI keyboard and projector. Every note played on the keyboard produces a stripe, whose proportions and color correspond to how the note was played. For instance, the color is mapped to the tonality of the note via the circle of fifths, thus visualizing harmonic consonance and dissonance. Besides looking really cool, it also thus creates unique “fingerprints” of each performance.

Clavilux 2000 is extremely close in design to a key 1995 work by media artist Toshio Iwai, known most recently for the Yamaha Tenori-On and Nintendo-published ElectroPlankton DS (each of which uses ideas from the earlier project). Piano–as image media and related works employed both inputs and outputs. (in the installation, visitors could use a trackball to enter note events visually on a screen; in performance with Ryuichi Sakamoto, the work used a piano. In each, events fly off perpendicular to the piano keyboard as beams of light, just as in the work here. That’s not a criticism, incidentally – even without seeing Iwai’s work, it’s a logical solution, because the keyboard organizes notes into an array of thin rectangles (the keys).

Unfortunately, Iwai’s work is not well-documented online; videos of these pieces have been removed. I do have a few resources for you, however. At bottom, there is a video of a 2006 Ars Electronica talk on the visual interface for music. (I have some video of Toshio’s similar thoughts around the launch of the Tenori-On which I should publish.) And for more:

Toshio Iwai talking about the visual-musical interface[artintelligence]

Toshio Iwai keynote at Futuresonic [pixelsumo]

http://www.nydigitalsalon.org/10/artwork.php?artwork=57

Our friend and interactive hero Gustavo Bravetti must have been inspired by all the talk of hexagonal sequencers, because he’s come through with a brilliant prototype of a new interactive sequencer design. He writes:

I just wanna share mi first very unfinished and at ultra alpha stage, hexagonal sequencer prototype!

Between many things, I have planed to include many automatic scale definition tools, follow actions, you’ll can easily change the hexagon density, and multi-touch support via IR (wiimote or cams) is planned also.
This is just a sneak peak.

For an “alpha” version, as you can see, there’s already a lot of goodness going on. The visuals and interaction are powered by vvvv, the free-for-non-commercial use (and otherwise affordable) Windows-only patching language. Max is great, but vvvv is capable of some very powerful features of its own, including particularly nice hooks into Windows’ DirectX rendering engine.

vvvv Site + Wiki + Community

More on vvvv at Create Digital Motion, as it’s most often used on the visual side:
http://createdigitalmotion.com/tag/vvvv

As with so many of these things, vvvv’s community is more valuable than even the tool itself; we’re seeing lots of work on doing clever things with the environment. And vvvv has gotten some powerful music features like VST plug-in support, meaning you could build your sequencer in vvvv and skip something like Live altogether.

Previously on this topic:
Music on the Game Grid: Interactive Arpeggiators Al-Jazari, reacTogon
Alternative Sequencers: Elysium Generative Mac App and the Joy of Hex

And for more of the Awesomeness of Gustavo (pay close attention to that interview, especially):
Live + FM8 = Drum Kit Love: Free FM8 Drum Kit Download
Weekend Inspiration: Ableton Live Follow Actions, Dummy Clips, Making Snares
Interview: Gustavo Bravetti, Playing Music with Light and Interactive Gloves

NYCPatchingCircle @ Pd wiki (thanks to Hans-Christoph for making this happen)

See you there if you’re in the area. 540 W. 21st Street, (between 10th and 11th Avenues).

We asked to see inside the Ableton Live sets you use in live performance, and you’ve responded with an overwhelming variety of responses. There are plenty of very practical submissions, from beginners and advanced users alike, which should give us a real sense of the ways in which people are playing Live as an instrument. Naturally, there are also some more unusual entries.

At top: Mark Gutierrez has used the Live arrangement grid as a palette for animated pixel art, with 8-bit game characters from Space Invaders and Super Mario Brothers dancing across the screen. At bottom: Manuel Palenque has connected Live to the patching environment and 3D visual tool vvvv for live, animated visuals. (Manuel, maybe you can tell us – do you output those visuals to a screen, or use them as feedback during your set?)

Insane examples, yes, but they do illustrate what’s possible. Videos after the jump.

Keep your Live sets coming. Grab a screenshot or video and send to:

• our Live set Flickr group or
• email to pictures (at) createdigitalmusic.com

Ableton Live 8 bit Space Invaders from mark gutierrez on Vimeo.
TP2 from Manuel Palenque on Vimeo.

As seen in Japanese on Hideyuki Fukasawa’s blog.

Related:

vvvv Adds Music Features; Get Your Synesthesia Patching On, Free on Windows

vvvv, the free-for-non-commercial-use patching environment on Windows, already has a cult following among visualists. Now, it’s looking more interesting for music, too, with the 4.0 beta 17 release.

• VST plug-in support for adding audio/music instruments and effects
• Multichannel waveplayer
• eCue Lighting Control Support

In case you haven’t worked this out yet, what this means is that you can now add powerful visual interaction with a VST plug-in. That could be a huge boon to audiovisual shows. Max and Pd (among others) have had this ability for some time, so it’s not revolutionary as an idea – but it is nice to get this feature in this powerful, eye-candylicious app. (Thanks to Bjorn from vvvv for the heads-up.)

I may have to try out Kore, since Kore runs easily as a VST and hosts other instruments / effects in a way that can work live. FL Studio could be interesting, too, for the same reason – and, like vvvv, has a solid following as a Windows exclusive.

Details:

http://vvvv.org/tiki-view_blog_post.php?blogId=3&postId=256
http://vvvv.org/tiki-index.php?page=Change+log
http://vvvv.org/tiki-index.php?page=VST
http://www.ecue.de/products/interfaces/butler.html

vvvv Tag @ createdigitalmotion.com

vvvv also recently added the ability to develop your own objects (“nodes” in vvvv speak). Development looks unusually easy, with baked-in C# support, so there’s good stuff happening in vvvv-land in general.

Everywhere I go, people bug me about when they’ll see better support for OpenSoundControl (OSC) in applications. Why use fancy-schmancy OSC when MIDI does the job? Well, OSC supports higher resolutions of data when needed, maps variables elegantly (when you’re controlling something like visuals and descriptions like musical pitch or filter cutoff make no sense), and plays nice over networks and with multiple computers. In other words, go ahead and use MIDI when it does the job — but we need something else when it doesn’t.

Here’s one way to get OSC from your favorite app: hack it in. Spotted today as Gav tells Create Digital Motion about gluing together Isadora and Ableton Live, the OSCGlue plug-in is a simple VST insert that listens to MIDI and sends OSC.

And, yes, this is another free Windows VST that doesn’t work on Mac. But before you get too excited, more disclaimers from the creators:

OSCGlue is a simple 24hours hack and may contain bugs. It seems to work under Ableton Live and VVVV. The Plugin will work only under Windows. Receiving messages under other platforms should be possible. Please report your findings.

It’s also written in Delphi with a Windows-only wrapper, so don’t bother asking about a port. (Mac users, for the record, have the wonderful OSCulator for which there’s no good Windows equivalent — it maps input controls to OSC.)

Still, it’s well worth checking out, and I’m definitely going to be giving it a try:

OSCGlue 1.0alpha at the vvvv wiki [let's just hope that's "alpha" for "awesome!"]

The reason the plug-in comes from the vvvv folks is important: this could be ideal for routing sonic stuff in Live to vvvv for visuals. And I think interfacing software to software in this way could become OSC’s killer app. What we really need is not a "24hours hack" but better support in the host programs — and Ableton Live has to be right at the top of the list. On the Native Instruments side, it’d be great to see Reaktor support input, especially given that that could mean doing bizarre things with granular instruments using high-resolution controller data. When someone hooks up their brain to a granular soundmaker and invents a new kind of music using only their mind, it’ll be worth it — really. Yes, Reaktor supports OSC now, but the implementation is wacky and undocumented. It’s amazing as many people use is as do, given that NI doesn’t really even tell anyone about it.

But for some reason, instead of telling NI and Ableton and whatnot that they need better OSC support, you keep telling me. So don’t tell me. Write your friendly software vendor.

Expect a more organized letter-writing campaign soon — and I think there’s also an opportunity to work with developers to make implementation work better. It has the word "open" in it for a reason.

A team of artists has turned the Nintendo Wiimote into a controller for an adaptive, 3D sound environment. Claudio Midolo, Edgar Castellanos, Natan Sinigaglia, and Pedro Mari worked together on the project, and have posted extensive technical details if you fancy trying something like this yourself.

The basic setup:

• Mac for control input via Bluetooth and music generation
• aka.wiiremote external for Max/MSP (by Masayuki Akamatsu, freely available though Mac Universal only) for communication with the Wiimote
• Propellerheads Reason for sound
• Windows PC for 3D visualization, via visual software vvvv (Windows-only)
• UDP for data transmission from Mac to PC

More details and lots of video:
wiiwiiwiiwii (Thesis Project)

I’m not totally convinced from the perspective of musicality, but it’s a promising project, and it demonstrates how a controller like the Wii can allow a reduction of controls. As it happens, I’ve been really aware of this as I play EA’s Madden Football. They even have a new single-controller scheme, because you can do so much gesturally, which previously required lots of buttons or keyboard hotkeys. I think this kind of gestural control could be relevant not only to the Wii but other controllers — there’s no reason only the Wii needs to have tilt sensors. Even my digital camera has a tilt sensor for adjusting the orientation of image previews.

So there you have it. Students, tell your computer music / new media teacher I told you have to spend time playing Madden football, in order to experience the possibilities of gestural interactive control mapping. Seriously. You’ll just have to take on my Giants.

Thanks to Chris O’Shea for the link. He’s got so much awesomeness on his site Pixelsumo that I don’t know where to begin.