As computer music practice – part composition, part instrumental play – spreads, the idea of software interface as performance tool is becoming second nature. Putting those opposable thumbs and sensitive fingertips to work, multitouch controllers are growing in number, variety, and sophistication. Berlin-based artist Marco Kuhn shows off his beautiful creation, the PlayBox multitouch hardware, and its first app, PlayLive. That first software focuses on Ableton Live performance, but Live could be just the beginning – Marco has worked with Pd in the past and promises other apps to come. He’s interested in selling this device in the future, and he shares with us the tools he used to create this work for those of you doing development along similar lines.
Imagine drawing an interface on paper, then being able to use it as a musical interface. Or, heck, don’t imagine it – do it. Unfortunately, the kinds of intelligence necessary to make the music video in yesterday’s post just aren’t practical yet. (That is, you could draw a picture of a keyboard, and even use the picture as a music controller, but while you or I could recognize a keyboard from a drum pad and know that line is a fader, a computer would need some sort of advance structure for any recognition to work.) But you can do some really clever things, as folks have shared in comments.
And using some basic paper interfaces, you can make entire instruments for just a few dollars.
Of course, the awesomest way to do anything is with LAZORS. Greg Kellum and Alain Crevoisier presented a paper at last year’s NIME (a conference for new interface designs for music) proposing a system for making any surface a control surface. Like the music video yesterday, you can configure your surface to function however you like – even dividing it up into pads and faders.
By now, you’e likely seen plenty of multi-touch interfaces or means of tracking hands. But, to paraphrase the NIME paper, these either require a special surface (or transparent surface), or they can’t actually detect when you’re touching. You can even use multiple cameras or an IR beam, but there are limitations to accuracy and the size of the usable surface that would result. Kellum and Crevoisier use an infrared camera and two illuminators, each built by pointing a laser at a mirrors.
Yawn, you say, been there, done that, seen Jeff Han’s video… The advantage of this system is that you can use any surface, like your dining room table. And you can configure that surface however you like. There’s even a freely-downloadable Surface Editor you can extend in Java and Processing. The creators claim they can even get input latency down to a reasonable 10 ms using high-speed cameras.
db3ll has created a keyboard out of paper, and of course it works better than those flimsy rubber “roll-up” pianos you see for sale. “Conductive ink is what I used,” he says, “painted on as traces on the non-printed side of the paper.” That’s the twist – I had assumed you’d use the top of the paper, but the trick is to use the reverse side to provide the “wiring.” He also offers advice for making a fader:
It’s still anyone’s guess exactly what fruit parent technology maker Stantum may soon ship, but the JazzMutant Lemur touchscreen is getting a component update soon. Nat Lecaude points to a quiet MySpace post from JazzMutant with the details of a coming manufacturing change.
“…the next batch of Lemur will feature the latest generation of our multi-touch technology: better optical performances, higher precision, greater accuracy and responsiveness. It will be clearer and have brighter colors. We plan on launching the new Lemur in early October, and of course we will keep you updated as we get closer to launch date. We once again thank you for your patience, and look forward to sharing the excitement early October!”
It’s actually quite remarkable to me that JazzMutant remains alone in this market – and with Stantum focused on the mass market, that could be the case in the future, too. The issue is that doing multi-touch well still costs some money. There are basic implementations on computers that are cheaper, but that restricts you to a few computer models, because slapping multitouch overlays on displays remains pricey. So HP can get a few computers to the mass market, but not without cutting some corners and not even on that company’s full range. The iPhone has brilliant multi-touch control, but a mobile form factor makes this much easier.
I’ve got some videos demonstrating what’s possible with the Lemur coming soon, as well as some notes on how the software has evolved since I first saw it in its initial release. Even if you don’t want or can’t afford a Lemur, it’s a fascinating demonstration of interaction design and OSC, with lessons (inspiring and tough alike) for other interfaces.
There’s plenty of rightful skepticism about the use of mainstream displays for multitouch in general purpose computing. And why not? As a full-time replacement for other input, multitouch probably doesn’t make sense. But for music, the equation is changing. Multitouch capabilities are showing up on commodity-priced PC computers like the multi-touch enabled HP laptop models – the tx2z seen here starts, incredibly, at US$850. And because computer musicians are looking for more control, having a touch-enabled display (even single-touch) just makes sense.
The screen for a laptop musician is a huge piece of real estate. Finally, instead of sitting dumbly in front of you glowing, it can become an X/Y controller or give you shortcuts for controls or provide additional parameters. Yes, using a touchscreen exclusively can result in the dreaded “gorilla arm.” The ergonomics of using a vertically-oriented screen are extremely poor – if you use it exclusively for an extended period of time. But if you look at the way people are using these touchscreens, for incidental control in combination with other things – and the ability of convertible laptops to transform into a horizontal orientation – I think this is no longer the deal killer it once was.
At top, an HP laptop ($850) plus the free version of Sensomusic’s Usine is all you need to create a multitouch interface for Ableton Live. Correction: right now this is limited to single touch only, but multitouch is supported in the hardware, in drivers, soon in Windows 7, and support is promised for a future version of Usine. The point still stands — as does the ability to optimize controls for your fingers. Being able to use more than one at once will, of course, be that much better.
Fractal (see Myspace) uses the combination to play Ableton Live with some simple controls. If you get hooked on Usine, you can get the full “Pro” version for EUR70 with additional patches and objects.
The one major remaining obstacle to multitouch, at least, is cost. If you don’t especially fancy buying a new HP laptop, add-on kits still run in the range of US$800-900 (meaning, ironically, you might as well just buy the HP instead). Laptop vendors are still slow to adopt the technology, though that could change when Windows 7 ships later this year. (On the other hand, tablet PCs, even when they were shipping in relative quantity, often were constrained in available configurations and either skimped on specs or demanded a significant premium.)
But let’s not complain too much. The simple reality is you can add an HP laptop now to a live rig as a performance instrument for under a grand.
Okay, Google Android fans — your apps are starting to arrive, especially as Google continues to improve the SDK.
First up, here’s a demo of the new app FingerPlay MIDI, which turns Android into a simple touch controller. True, we’ve seen similar things on iPhone, and Apple’s platform has precise multitouch which Android lacks, but it’s nice to see the concept extended across platforms. Big thanks to postromantic on Twitter for the tip! (Follow cdmblogs for more.)
I can’t tell yet if this will be open source – that would actually be nice, I think, as it’d allow the community of developers to have a shared set of tools. (In fact, it seems a logical model would be to cash in on general-market apps and open source the more music-specific, niche stuff.)
In other Android mobile news, Christopher Souvey continues to work on his Musical application and the Musical Pro desktop app. The desktop client works with MIDI over Wifi, and thanks to the Cupcake OS update, latency is greatly reduced after a complete rewrite and the creation of custom drawing and event handling and controls. Check out the slick new tuner and UI Christopher has been developing, too.
This play-along piano is probably not something any of you folks desperately need, but it is a good demonstration of what’s possible.
Another interesting thread to follow will be the growing power of Web apps. On Android, you’re already able to combine a Web app with the Java APIs, and going another level, mobile apps with native ARM code for the processor. Translation: while phones still have a fraction of the power of your computer, it’s getting easier for developers to work across platforms and to take advantage of what power is there. That’s leading to trends that could be of use not only to a single platform (Android, iPhone), but to mobile devices in general.
12
