It’s hard to assemble an April Fool’s Joke involving technology these days, because actual inventions keep proving stranger than fiction. When Google created a prank involving gestures for controlling email, it was only a matter of time before someone whipped up a prototype that actually did the job.

The Moog Music company, therefore, may be asking for trouble. Their highly-entertaining polyphonic Theremin is spot-on parody, down to the “Stairway to Heaven” solo. And part of the geekier joke for Theremin players is the knowledge that the technology behind this instrument makes what they’re describing safely impossible.

But what’s impossible with conventional Theremin technology could be very possible with computer vision – even the goofy gestures in Moog’s faked video. Artist, inventor, and musician Tim Thompson has been at the bleeding edge of new music instruments for some time. It wouldn’t be overstatement to say Tim was using multi-touch before multi-touch was cool. When I shared a booth with him at Maker Faire a few years ago, he had with him FingerWorks hardware, a now-discontinued tactile, multi-touch pad, and was using it to play visuals live. In a pattern too often repeated in technology, the independent niche tool was snapped up by a larger player. In this case, that larger player was Apple – and, apparently backed at least in part by FingerWorks’ know-how and patents, Apple made history.

In a new project filmed by the superb Modulate This!, Tim works instead with touch-less control, using the Kinect to track multiple areas of expression. (Tim is using the free environment Cinder, which joins tools like Processing and OpenFrameworks as well-liked options for Kinect hackers. In this case, the Kinect support itself comes from libfreenect, the open-source drivers for Mac, Windows, and Linux.)

What he’s built, in other words, is a true polyphonic Theremin – able to play more than one line and employ more than a monophonic gesture, all without touch. The joke may be on Moog.

Read the full story on Modulate This, Mark Mosher’s all-original repository for great writing on music making.
An Exclusive First Look at Tim Thompson’s Kinect-Based Instrument: MultiMultiTouchTouch
(Thanks to Tim and Roger Linn for sending this my way!)

Part of the value of trying extreme ideas is to demonstrate not only advantages, but disadvantages. And I still find some reason to express healthy skepticism. The similarity to the Theremin isn’t accidental in the Kinect experiments. These projects also inherit the Theremin’s weaknesses. A lack of tactile feedback means it’s difficult to orient pitch or achieve precise control, without the resistance a physical object provides. Reliance on gestural control also opens the opportunity for accidental input and calibration challenges. (The Kinect fares better than the Theremin, but it’s not immune to similar problems, if for different reasons.) Taking a page from the Theremin, Tim’s physical frame makes a big difference – while it doesn’t provide tactile resistance, it at least creates a point of reference in physical space.

The Kinect also adds a new problem the Theremin didn’t face: latency. All of this means if you still like knobs, keys, strings, or even physical multi-touch (which can in certain variations provide excellent tactile feedback via deformable meshes), you needn’t worry. Your revolution may not be Kinect-ified.

But if there were one perfect design for musical instruments, we’d all play just one instrument. Instead, the history of instrument design across the world is an evolutionary explosion of different tradeoffs, different playing styles, and resulting different musical idioms. Any joke can become an instrument, just as any instrument – to someone – can seem like a joke. And that means if you’re looking for something new, you might just celebrate every day as if it’s April Fool’s Day. No kidding.

Updated: Tim offers some comments. He says what other musicians experimenting with Kinect have told me – that while it has certain restrictions as a solo instrumental controller, there’s tremendous potential for multi-user scenarios like installations. And that is itself significant (back to the question of choosing tradeoffs in order to accomplish goals). Tim writes:

Folks whose goal is to replace conventional instruments are sure to be disappointed, as you describe. You could add more detail on other goals:

Goal: using it for art installations at events like Burning Man, creating new and “casual” instruments which are unusual yet inviting and easy to play. Matt Bell ran an experiment related to that goal: http://www.youtube.com/watch?v=mQiyKFDvzkU

Goal: creating controllers which have a much larger visual appeal to an audience, who deserve performers more interesting to look at than someone hunched over buttons and sliders. That’s the reason why musicians like Mark Mosher are interested, in the same way he’s interested in the Percussa Audiocubes, for their visual appeal in performances.

Goal: provide an instrument that dancers can use in performances. I’ll be exploring this in the fall, with a choreographer friend.

Good food for thought; feel free to discuss more in comments.

Sonic scientist Peter Neubäcker of Melodyne has been wowing Internet audiences for some time with the automagical powers of the company’s Direct Note Access (DNA). The vision: manipulate individual pitches as easily as MIDI notes, even in polyphonic passages of a single instrument. At NAMM last month, the company showed the first product, Melodyne editor, due to ship in the spring for US/EUR 349.

There’s just one little catch: a solo developer has beaten them to the punch, at least prior to them shipping their DNA flagship editor tool. And if you want it right now, it’s yours for 25 Euros. (The final version will cost 99 Euros.)

Jonathan Schmid-Burgk, sole developer and a student at Harvard, announces:

The time has come to announce the release of the world’s first published polyphonic tone manipulation software. The dream of musicians to isolate single notes out of chords and so to manipulate most forms of recorded audio has come true on the 20th of January 2009.

Shell out EUR25, and you get a Mac VST plug-in that can manipulate audio easily. With monophonic audio, you can create polyphonic harmonizations. You can isolate and manipulate individual harmonics – meaning not only can you do pitch manipulations, but presumably sound design, as well. You can change individual notes or chords in recorded audio, to fix mistakes or (more interesting) actively recompose audio.

I feel about this the same way I do about Celemony: this gets really interesting when you use it for sound design. For some inspiration, skip this post and head straight for the sound samples on the site:

VisualVox polyphonic 0.9 [improvisator.de]

Via the awesome rekkerd.org

Also check out his Harmony Improvisator which creatively generates harmonies from MIDI input – an interesting thing to mess around with even for those of us who know / have taught (ahem) classical harmonic theory

Now, VisualVox Polyphonic isn’t without some catches, as you’d expect from the solo-student cheap alternative:

• It’s Mac-only and VST-only for now. (Just keep in mind, though, that means not only Cubase works but things like Ableton Live on the Mac, as well.)
• Jonathan warns that there are still some bugs, there are big temporary files, there are occasional glitches in the algorithm, and it doesn’t quite sync properly with hosts, requiring exports.
• The interface isn’t as slick or fancy as Celemony, and may be slightly less convenient for editing.
• Update – people are having some significant hosting issues with this. (Your mileage may vary, though the fact that this was tested in Cubase tells me those folks may have better luck.) So stay away if you don’t like bugs – this isn’t a final release. We’ll have more as this evolves, okay?

But here’s why I’m anxious to test it, nonetheless:

• It’s cheap – meaning it might be more open to people just curious to experiment. (I actually wonder if he shouldn’t keep the EUR25 price and make up for it on volume.)
• No iLok dongle, as required for Celemony. Correction: Celemony also allows challenge/response — so you have a choice.
• There are some interesting fine-tuning options for the algorithm and manipulation parameters. That could make this more interesting for sound design – and something even Celemony users might want to run alongside the more polished tool.

In other words, none of this is likely to dampen enthusiasm for Celemony, but that’s a good thing: more is more better. Could this mean the floodgates are about to open for creative sound design tools that mess with audio in new ways? Hey, I hope so.

If anyone tests this, we’d love your report – and sound examples.

Celemony is showing their new technology at Musikmesse, and they’ve got demos online you can check out:

Direct Note Access

Grab a note inside a chord, and you can manipulate that note directly. Retune it, change timing, adjust formants, change amplitude — timbre, time, and pitch are all accessible. Celemony is largely pushing this as a corrective tool, as that’s an obvious market, but needless to say, creative applications — even creative abuse — become interesting, too.

Melodyne Studio costs US$399 (349 EUR), with various discounts for upgraders, and the technology will be making more limited appearances elsewhere in Celemony’s product line. Now, it is a plug-in — clearly, someday this sort of thing will just be integrated directly in your host of choice, and I’m particularly excited about the day when it becomes a live performance tool. But for now, it could well be worth the cost of ownership.

You’ll have to wait a bit: the new version is scheduled to ship in the fall, though if you buy now, you’ll get the update free. Celemony, I’ll be seeing you at AES, I think.

Compatibility: Mac (Intel/PowerPC), Windows (XP/Vista)

Thanks to everyone who sent this in (Alex, Karsten, Eric, and others)! By popular demand, the demo video SonicState grabbed at Messe, because they’re organized enough to actually be in Frankfurt while I chill out here in NYC:

MESSE08:Melodyne Blows Our Minds