What if you could mash, mangle, mush, and morph sounds with your fingers on a screen, watching the waveforms dance in response in three dimensions? That “what if” is expressed beautifully in a project by musician-developer Christian Bannister of Portland, Oregon, who works as Subcycle Labs.
The result is like being able to touch sound directly.
Three-dimensional forms morph and vibrate using visuals programmed in Processing, making architectural-organic shapes and spaces that really begin to “look” like sound. These forms can represent synthesis and effects parameters (Christian has done some work with the Massive synth from Native Instruments), or can allow navigation through loops using touch. Gestures remap offsets and duration for audio, scrub and slice, and apply granular resynthesis.
As computer music practice – part composition, part instrumental play – spreads, the idea of software interface as performance tool is becoming second nature. Putting those opposable thumbs and sensitive fingertips to work, multitouch controllers are growing in number, variety, and sophistication. Berlin-based artist Marco Kuhn shows off his beautiful creation, the PlayBox multitouch hardware, and its first app, PlayLive. That first software focuses on Ableton Live performance, but Live could be just the beginning – Marco has worked with Pd in the past and promises other apps to come. He’s interested in selling this device in the future, and he shares with us the tools he used to create this work for those of you doing development along similar lines.
There’s plenty of rightful skepticism about the use of mainstream displays for multitouch in general purpose computing. And why not? As a full-time replacement for other input, multitouch probably doesn’t make sense. But for music, the equation is changing. Multitouch capabilities are showing up on commodity-priced PC computers like the multi-touch enabled HP laptop models – the tx2z seen here starts, incredibly, at US$850. And because computer musicians are looking for more control, having a touch-enabled display (even single-touch) just makes sense.
The screen for a laptop musician is a huge piece of real estate. Finally, instead of sitting dumbly in front of you glowing, it can become an X/Y controller or give you shortcuts for controls or provide additional parameters. Yes, using a touchscreen exclusively can result in the dreaded “gorilla arm.” The ergonomics of using a vertically-oriented screen are extremely poor – if you use it exclusively for an extended period of time. But if you look at the way people are using these touchscreens, for incidental control in combination with other things – and the ability of convertible laptops to transform into a horizontal orientation – I think this is no longer the deal killer it once was.
At top, an HP laptop ($850) plus the free version of Sensomusic’s Usine is all you need to create a multitouch interface for Ableton Live. Correction: right now this is limited to single touch only, but multitouch is supported in the hardware, in drivers, soon in Windows 7, and support is promised for a future version of Usine. The point still stands — as does the ability to optimize controls for your fingers. Being able to use more than one at once will, of course, be that much better.
Fractal (see Myspace) uses the combination to play Ableton Live with some simple controls. If you get hooked on Usine, you can get the full “Pro” version for EUR70 with additional patches and objects.
The one major remaining obstacle to multitouch, at least, is cost. If you don’t especially fancy buying a new HP laptop, add-on kits still run in the range of US$800-900 (meaning, ironically, you might as well just buy the HP instead). Laptop vendors are still slow to adopt the technology, though that could change when Windows 7 ships later this year. (On the other hand, tablet PCs, even when they were shipping in relative quantity, often were constrained in available configurations and either skimped on specs or demanded a significant premium.)
But let’s not complain too much. The simple reality is you can add an HP laptop now to a live rig as a performance instrument for under a grand.
One of the problems with touchscreens is that, even as they have become more sophisticated about tracking multiple fingers at once, they still generally don’t respond to pressure. To make touchscreens really useful for music, we need genuine pressure sensitivity.
For that reason, you may be intrigued to see this video of Zen Piano, a demo app for the iPhone and iPod touch. The idea: respond not only to the position of your finger taps, but also to how hard you’re tapping the phone That promises “velocity-sensitive” tapping, which would make touchscreen interfaces more powerful.
Here’s the somewhat overheated description by GreatApps, who say their “patent-pending,” “cutting-edge” technology is the result of “having gone through the research and development phases.”
TapForceTM has been developed from the ground up to provide a completely intuitive way of interaction for users. It can detect more than a hundred different levels of force, and has an accuracy that has to be seen to be believed. And all this can now be done in software, no hardware modifications are necessary. Hundreds of millions of devices currently on the market can make use of the TapForceTM technology today.
A whole new range of games and apps has just been made possible.
Ever feel like you’ve found the seam dividing past and future?
The past: restrictive UI frameworks requiring pages and pages of code to produce dated-look 2D displays. Proprietary software with rigid interfaces. Input bottlenecked through the x and y coordinates of a single mouse pointer.
The future: UIs whipped together graphically or with a few lines of code. 3D mixed with 2D. Open-source, friendly frameworks. Creating your own interface or drawing upon a community of creative software makers. Input that uses multitouch for gestures, collaborative input, manipulation of 2D and 3D space, and … well, just a lot more fun.
There’s no need to wait around for the future. Creative software inventors are building it for themselves. Here are two of the most promising multitouch interface projects I’ve seen in my inbox.
In no time at all, you’ll be painting a cow! (Okay, more on that in a moment…)
36
