Software doesn’t have to mean virtualizing everything and letting go of physical objects. On the contrary, it can create all sots of imaginative, new ways of mapping musical ideas to the physical world. And that’s how we wind up with a walnut drum sequencer.

There’s something about virtual drum machines and snacks. We’ve seen bubblegum and Skittles, beer bottle caps, soda bottles, and now walnuts. Don’t stop now: someone has to do Cheetos, even if it means dealing with orange stuff all over your fingers.

That said, it’s not walnuts that make d-touch an important project. Built by Enrico Costanza back in 2003, the project is now available for free download as an open source library, a server (in case you don’t want to get into the C++ code but might want to use this in your own projects), a free, usable drum machine, and a set of documentation that can help you make your own stuff easily. Enrico worked on the original reacTable prototype and has done some really important work in this field. Right now, Enrico and co are looking for feedback, but if you’re ready to just be a tester and play with this – and see what you can do musically – now’s your chance.

d-touch also combines high levels of computer readability for accurate tracking with the ability to make your own tags. Instead of using ugly-looking glyphs, you can make patterns that make sense to human beings as well as computers. Oh, yeah – and mobile fans, this runs at a full 14 fps even on S60 phones.

For more, check out the d-touch site:
http://d-touch.org/ [Register first to make the download available]
and follow them on Twitter:
http://twitter.com/audiodtouch

Thanks to Martin (of reacTable, which is moving toward a commercial product) for sending this our way. Thanks, too, to Ben, who’s working on tangible interfaces with special needs students. I really look forward to hearing how that’s going.

Free software, a webcam, and some stickers printed on an inkjet can turn any object into a real-world controller. That’s what Paul Rose of Institut Fatima and his team did with a soccer ball (translation for the civilized world: football). The software is powered by the same framework used for the reacTable, but in this case there’s no table and no projector: just a ball.

Institut FATIMA uses a Fussball as (des-)controller for triggering drumsamples. The camera detects the symbols on the ball, kicks numbers into the sequencer, the sequencer matches goals. The goal is always music. Software used is reactivision and ableton live. Do it at home.

As it happens, reacTIVision just got a significant update, with more improvements planned. You can read up on the full details on Create Digital Motion:

Free Tangible Tracking: reacTIVision 1.4 Here, TUIO2 Coming Soon

Martin Kaltenbrunner, co-creator of the framework (and the reacTable), has some tips for working with tangible interfaces and music, and where to find more inspiration.

In addition to TUIO, reacTIVision also has an alternative MIDI mode, where you can map the appearance of fiducial symbols to note ON and OFF events, as well as their X,Y and rotation angle to a control channel value. Quite a few people have been using this for the creation of cheap web-cam based MIDI controllers.

http://www.youtube.com/results?search_query=reactivision+midi

Using TUIO, you have more alternatives though, you can currently use Max/MSP, Pure Data, Quartz Composer, Processing, Java, C++, C# and so on to receive the object & finger tracking data. Here are a few cool musical projects, that have been built using reacTIVision:

http://modin.yuri.at/tangibles/?list=7

Patrick H. Lauke (patch pictured, from Flickr) has a video on YouTube that shows some of the basic workflow for combining the free patching environment Pd with TUIO and reacTIVision. He cautions:

this may not be pleasant from a musical point of view, but it only serves as a first test for further experimentation.

Hopefully this gives folks some ammunition if you’re getting involved in the tangible interface hackday! [Project site | on CDMu]

MSAFluid for processing (Controlled by iPhone) from Memo Akten on Vimeo.

TUIO is a simple but powerful emerging protocol for multitouch control for live music and visuals, as used on the powerful live tangible synth reacTable. Apparently no one told Apple, however. While the App Store rubber-stamps useless toys like fake cigarette lighter flames, they bizarrely rejected a powerful application by a leading digital artist that would enable standardized TUIO control – for free. (More back story below; see an example in action above.)

As a blogger, my reaction is usually to whine and pontificate, for better or worse. The engineering approach would be to find some hack away the problem. That’s what Andrew Turley did with the TUIO protocol. So, Apple won’t allow an app that does the trick? Why not go back to what developers did before the SDK, and just use the iPhone browser?

As Andrew explains it:

After reading the story I started thinking about seeing how far one could push Safari as an application platform, using web apps to get around Apple’s tight control of the app store. Since you would be connecting to another computer anyway to use an OSC application, why not just have the app be a web app running on a web server somewhere on the local network? The web server can then take care of things like sending out OSC messages or playing music or doing whatever it is people want to do.

To that end I created a little system that implements the TUIO protocol. You use an iPhone to run a web app, which in turn talks to the web server, which in turn sends OSC messages.

Jasuto envelope example from Jasuto on Vimeo.

Imagine friendly creation of custom synths and sounds by dragging visual nodes. Now imagine you can do that on a mobile device and your computer – and eventually combine the two. That’s the vision of Jasuto, and while it’s not quite there yet, it’s incredibly promising.

The laws of combinatorics predict that, on a regular basis, you’ll see countless soft synths that are slight variations of one another. With the iPhone/iPod touch gold rush in full swing, we’re starting to see the pattern repeat itself, just as it did in Windows and Mac plug-ins. Some are brilliant; others are just the usual variations on a theme.

Of course, even better is the ability to build exactly what you want out of the same buildings blocks. Powerful toolkits like Max/MSP, Pd, Reaktor, SuperCollider, SynthMaker and the like let you do this, but they qualify as the more-sophisticated Erector Set of synthesis. Sometimes you just want some simple, LEGO-style building blocks that cover the basics.

That’s why Jasuto looks so promising. It’s actually two pieces of software – a plug-in for Mac and Windows VST. Combine basic modules, and you get some powerful features, even on the iPhone:

c

Ever feel like you’ve found the seam dividing past and future?

The past: restrictive UI frameworks requiring pages and pages of code to produce dated-look 2D displays. Proprietary software with rigid interfaces. Input bottlenecked through the x and y coordinates of a single mouse pointer.

The future: UIs whipped together graphically or with a few lines of code. 3D mixed with 2D. Open-source, friendly frameworks. Creating your own interface or drawing upon a community of creative software makers. Input that uses multitouch for gestures, collaborative input, manipulation of 2D and 3D space, and … well, just a lot more fun.

There’s no need to wait around for the future. Creative software inventors are building it for themselves. Here are two of the most promising multitouch interface projects I’ve seen in my inbox.

In no time at all, you’ll be painting a cow! (Okay, more on that in a moment…)