Dig into humanity’s past, and alongside the earliest tools, you’ll find some of the earliest instruments. Designing objects for expression seems to be an essential part of civilization.

Martin Kaltenbrunner, a co-designer of the Reactable tangible music interface, is also a professor in Interface Culture at the Linz University of Arts in Austria. There, in the land of Mozart and Haydn, he works with students to explore what interface design is.

So, when I got to spend some time with Martin in New York in September, I was interested in more than just the flashy coolness of the Reactable, the futuristic table-with-blocks interface for music. We got a chance to talk about instrument design generally. The funny thing about the Reactable is that it is closer to the experience of working with a modular synthesizer and oscilloscope than anything else, with the sense of physical connections of sound to object you’d get from classic synths. It is something unique, truly, but that’s its pedigree.

Martin and I got to give a talk together at the Austrian Cultural Forum New York, a terrific hub in which Austrian artists frequently are paired with New York-based folks, all in a lean, tall modern landmark building in Midtown. We also performed together, which for me was a real pleasure; Martin claims not to be a musician as such, but was good fun as an improvisation partner.

The next day, we headed to Manhattan music education center Dubspot, where Martin’s creation quickly attracted crowds of interested students and educators. Dubspot filmed our encounter for the video at top. Amusingly, the prominent synth sounds you hear at the beginning are not the Reactable, but our own MeeBlip open source synth, which I brought along to illustrate conventional tangible instrument design with switches and knobs. (If you’ve been impatiently waiting for news on the MeeBlip, believe me, I’m even more impatient – more announcements on that this week and next, following a production quality issue with a contractor that required us to reboot the run of new instruments.)

Reactable Live @ Dubspot! Interactive Sound Design Workshop Video Recap [Dubspot Blog]

YouTube commenters, that subtle and thoughtful bunch, are complaining that the tangible Reactable will set you back thousands of Euros. But at ACF and Dubspot, I was also equipped with the far more economical and portable alternative: Reactable Mobile runs on both Android and iOS. (I was pleasantly surprised to discover the app runs perfectly on a Galaxy Tab 10.1 from Samsung; I’m still a long, long way from being able to recommend buying an Android tablet, but if you’ve got one, I can certainly recommend this app.) Now, don’t get me wrong: the experience was nowhere near as fun as using the table. On the other hand, you can’t fit the table into a seat-back pocket on easyJet, and the savings in cash is proportional to the sacrifice in experience. What impresses me is that the design of the physical Reactable “flattens” so nicely onto the screen; I think it’s a user experience triumph that you can make that translation. And I was able to load up a few loops of my own music and jam along with the MeeBlip and Martin on the (real) Reactable.

Looking beyond the Reactable, Martin addressed these larger issues of tangible interface design at TEDx Vienna – a fitting locale for talking the history and future of music. His whole presentation, and a sweeping concept map of what he discussed, is available.

Martin Kaltenbrunner: Tangible Music

If you just want to get your Reactable on and can’t afford the table, see Reactable Mobile below. (Seen here on an iPad 2, but I’ve run successfully on the original iPad and the Galaxy Tab – the experience is more or less identical, thanks to portable code.)

And if you can afford the table, you rockstar, uh, can we be your friend?

http://www.reactable.com/products/mobile/
[the awesome table version]
Martin Kaltenbrunner website [with plenty of academic links]

Checkerboard Step Sequencer V2 from Josh Silverman on Vimeo.

Shall we play a game?

Have your checkers chops ready, because Josh Silverman’s Checkerboard Step Sequencer, a tangible interface for music, will test both your game mettle and your grooves.

Built with the open source coding tool OpenFrameworks and Ableton Live as sound source, the checkerboard fuses computer vision technology and … well, some beats.

This video should make obvious the relationship between the position of the checkers pieces and the noises they represent and trigger. It’s still a work in progress, but for now I won’t subject you to the cacophony that is the sound of an actual game of checkers.

Aside from the kick drum, which just keeps pace on every beat, all other drum samples are triggered off the board.

In this version, I’ve implemented a Mute Region that surrounds the board. When the application sees activity in the mute region, it disables the updating of the sequencer. This way, my hand doesn’t trigger a cacophony when I move the pieces.

More technical explanation on Josh’s blog:

How it works [prettyextreme]

As it happens, you can meet this project in person if you’re in the NYC area. We’ll be hosting Josh on Sunday night at Handmade Music, at Culturefix’s Lower East Side. 4:30-6p is an open lab, a chance to check out this project and others (including MeeBlip!), followed by cacophonous demos and raucous music starting at 7p.

Detailed Lineup; Handmade Music site
Event on Facebook

Historical precedent: It’s fairly hard to top John Cage and Marcel Duchamp playing chess, with or without sonification, but apparently sonified they were:
John Cage Playing Chess [Uncertain Times]

No information on what the chess game sounded like, however. Backgammon, anyone? Thanks, SkyRon, for the tip! Also, from 1997, a grid game with Toshio Iwai and Ryuichi Sakamoto; thanks, Ctrlsave. (Interesting to reflect on how much easier this is to do in 2010, thanks to more accessible software and greater, cheaper horsepower.)

Software doesn’t have to mean virtualizing everything and letting go of physical objects. On the contrary, it can create all sots of imaginative, new ways of mapping musical ideas to the physical world. And that’s how we wind up with a walnut drum sequencer.

There’s something about virtual drum machines and snacks. We’ve seen bubblegum and Skittles, beer bottle caps, soda bottles, and now walnuts. Don’t stop now: someone has to do Cheetos, even if it means dealing with orange stuff all over your fingers.

That said, it’s not walnuts that make d-touch an important project. Built by Enrico Costanza back in 2003, the project is now available for free download as an open source library, a server (in case you don’t want to get into the C++ code but might want to use this in your own projects), a free, usable drum machine, and a set of documentation that can help you make your own stuff easily. Enrico worked on the original reacTable prototype and has done some really important work in this field. Right now, Enrico and co are looking for feedback, but if you’re ready to just be a tester and play with this – and see what you can do musically – now’s your chance.

d-touch also combines high levels of computer readability for accurate tracking with the ability to make your own tags. Instead of using ugly-looking glyphs, you can make patterns that make sense to human beings as well as computers. Oh, yeah – and mobile fans, this runs at a full 14 fps even on S60 phones.

For more, check out the d-touch site:
http://d-touch.org/ [Register first to make the download available]
and follow them on Twitter:
http://twitter.com/audiodtouch

Thanks to Martin (of reacTable, which is moving toward a commercial product) for sending this our way. Thanks, too, to Ben, who’s working on tangible interfaces with special needs students. I really look forward to hearing how that’s going.

Fritzcrate Project / lusidLearn Early Demo from Michael Schieben on Vimeo.

Knobs and faders can be rigid. Fancy multitouch devices can be expensive. But for the cost of a webcam and some spare materials, you can build computer interfaces with objects around the house, thanks to the power of open source software.

In just one day, a group of artists in the CDM community, from Austria and Germany to New York to Australia, got quite a lot working with tangible interfaces. At top, Michael Schieben and Christophe Stoll experimented with using soda bottles to control software like Future Audio Workshop’s lovely Circle. (Ableton Live works, too – as does any MIDI software.) As Precious Forever, these guys are responsible for some of the best UIs in music software, from FAW to recent Native Instruments designs, so it’s lovely to see them experimenting with this idea.

As you add more people to the mix, you get ideas you might otherwise never have imagined, from a game involving blocks of the Tokyo skyline to an interface built into floor toms.

We also got a lot of real-world data on what works, what needs work, and what causes trouble for beginners, which we’ll be documenting. (Adam and Martin from the Trackmate and reacTIVision projects, respectively, were both tuned in to see progress and provided lots of help – and are also collecting that data to improve their own documentation and libraries.) More commentary on all these side benefits, as well as a discussion with visitors from Argentina on the scene around the world, at Create Digital Motion.

Musical Resources

We also got some really helpful tips for working with the free, powerful, tri-platform synthesis tool SuperCollider:
Charles Martin wrote up an easy SC test script for receiving Trackmate messages (and also had the clever idea of using a floor tom)

And for connecting Trackmate to MIDI and working with Processing, lots of tips are available on Michael Schieben’s noisepages blog:
http://fritzcrate.noisepages.com/

Get Involved

More documentation:
Tangible Interface Hackday: The Projects (So Far)
http://hackday.noisepages.com/

http://trackmate.sourceforge.net/
http://reactivision.sourceforge.net/

So, what’s next? You can join discussion and brainstorming for how to proceed, and how to get in on another hackday (formal or ongoing), even if you missed the first. Stop by the Tangible and Multi-Touch Interface group on noisepages:
Tangible + Multi-Touch noisepages Group

Our noisepages community is still in “alpha” state, but it’s usable – we’ve just fixed avatar uploading, which was the biggest problem. We’ll have more features, functionality, and improvements down the line, as well as more extensive documentation for how to get started. But if you’re a bleeding edge sort of person, join up free and give us some advice on what you’d like out of it.

I look forward to more work on these projects. Stay tuned for more, including some additional documentation (I’m developing some stuff around my own project).

2008 has been an amazing year for music technology. But I can’t bring myself to look back on it on this New Year’s Eve: not when there’s so much to look forward to in 2009. Case in point? An extraordinary, innovative new controller that in a matter of hours was already spreading among connected music technologists around the planet.

At the end of the day, it’s not hard to describe what you might want out of an expressive music controller. Most people would agree on that. The challenge is really an engineering problem. Solve the engineering problem in an artful way, and you can spend the rest of your time just practicing playing your invention. That’s what makes the above video so exciting.

Randall Jones has built a really elegant and wonderful multi-touch hardware controller, as reported by MAKE:blog (and picked up on Hack a Day). With $50 in parts and a lot of clever hardware design and software coding, Jones has built an interface that responds to both touch and pressure and, using some smart sonic mapping, can realistically reproduce instruments like the dumbek and guiro.

Intimate Control for Physical Modeling Synthesis [Project Page / Paper Abstract]

PDF, Randall Jones MSc Research Paper

Who needs a “top 10 technologies of 2008” post for CDM when this particular instrument could pretty easily top the whole list? Let’s just call it done, and uncork the champagne: major congrats, Randy! (This is a master’s thesis!)

Jones’ work does have some precedent, but just to review how much he’s accomplished here: he’s innovated in terms of the sensing, the form factor, the software interpolation, and the way in which the control data is mapped to a synthesis method. (Whew!) That has had a number of specific achievements:

• A clever form factor: The basic design here is elegant and could be adaptable to other form factors. Most importantly, the use of polypropylene and silicone rubber means the interface deforms nicely as you press it, giving you feedback. And that opens lots of other similar design possibilities. Translation: being squishy rocks.
• Multidimensional / force-sensitive input: “Multi-touch” usually only means something that can take multiple touch inputs at one time, as in, from multiple fingers. But as I’ve complained in the past, the problem is that most multi-touch interfaces, like the Lemur and iPhone, don’t respond to the amount of pressure or deform (providing tactile feedback) when you use them. That makes them feel a bit like an ATM screen that happens to take more than one finger at a time. Jones’ prototype responds to how hard you’re pressing or hitting it, and it’s more sensitive – more like a real-world instrument. Jones calls this “multidimensional,” but force-sensitive would be another way to look at it. Translation: it’s more organic than entering your ATM PIN code.
• Audio-rate resolution: By choosing to use audio signal for control rather than … well, anything else, Randy gets an extremely responsive control signal. The controller itself is passive, requiring no power. Everything is done by processing audio created by the sensors. That’s not a new idea, but by returning to it in this context, Randy makes a much more responsive controller than most touch and multi-touch controllers before it, and returns to some of the analog-style control of instruments like the Theremin and acoustic instruments. Translation: it don’t mean a thing if it ain’t got that swing.
• Smart centroid processing: To make touch points accurate, Randy wrote a new object for Jitter that does centroid calculation. Translation: Randy worked through the details.
• Waveguide mesh modeling for deep sonic results: None of this would be meaningful without meaningful sounds coming out of it. Here, Randy builds on previous work in creating a physical model of a drum head (back to the dumbek), but both refines the model and works intelligently through how to match it with the controller. Translation: it makes good noises.

There are some other related touch devices, each with their own strengths and weaknesses, but rather than reinvent his work, I’d suggest reading through Randy’s own research paper. He does a great job talking about what’s great and not-as-great about other research and products, and makes a terrific argument for his design choices. (Basically, see also the Haken Continuum, the Korg WaveDrum, the Tactex MTC Express, the CNMAT multitouch controller, and the Audio-Input Radio Drum.)

Now, if this kind of development had come about a few years ago, the next thing we’d be talking about is how this might be commercialized. Part cost is cheap – that’s good news. I don’t know Randy’s plans for licensing and future development. But we can already compare how this might have gone had this been introduced, say, a few years ago.

The old model:

1. Spend a couple of years more refining the prototype on your own.

2. Show off the work at an academic conference in Italy. People can see it if they’re associated with an academic institution that’s also willing to pick up their airfare and let them leave for a week. (Anyone who’s had to wrestle with their department to get a couple hundred dollars for a thousand dollars in actual expenses knows what I’m talking about here.)

3. Publish in an academic journal a handful of people get, so they can try to figure out what it’s like from diagrams and grainy black-and-white photos (and no sound). Oh yeah – no one can actually comment on the story, either, so advanced researchers lack feedback and newcomers lack context and commentary.

4. Get the prototype ready for NAMM. Blow a bunch of cash on a hotel in Anaheim and a NAMM booth.

5. Work out a complex, expensive production and distribution scheme that in turn jacks up the price. Hope someone finds it and can afford it.

6. To justify the cost, make the product as finished as possible. Sell it as a “Digital Dumbek” to eccentric, loaded rock stars.

Now, I don’t want to sound grumpy or naive. The truth is, some really brilliant instruments have been down this road. But I’m not sure the description above really did a whole lot for their brilliance. I’m not necessarily saying there aren’t benefits to some of the above techniques, but clearly they won’t work for everyone.

The new model:

“Where’s my flying car?” That’s the question that often comes up when technology and a New Year coincide. With music, though, I think what we’ve really been waiting for is a convergence of new technology and new communities. Flip through the Computer Music Tutorial’s section on new instruments, and you’ll see some fantastic, exotic, and often familiar new interfaces. So what has 2009 got that the last few decades didn’t? Think cut-rate, faster tech, and a connected Web community to develop, evangelize, and use new technologies.

1. Get 20,000 Vimeo views the moment you post your first demo. No one has to fly anywhere.

2. Connect with people doing research and experiments worldwide and get instant feedback and ideas and potential collaboration – even if they’re not publishing research or flying to conferences or are even academics.

3. Instead of assuming you’ll build a finished, closed product (hello, “Digital Dumbek”), figure people will want to hack the result. (Randy has already posted parts lists, so it’s possible to build this thing right now if you’re so inclined. And he also says in Vimeo contents he’s thinking about doing a version that transmits OpenSoundControl data, so you could use it to control other instruments, music, or even visuals.)

You’ve already skipped a lot of the steps that were previously necessary just to find interested parties. And that network could in turn be used to figure out how to make the thing around the world at a lower cost and get it directly to people who might want to buy it. It also assumes those people might hack the tool for very different applications, instead of having to target one particular kind of person.

Nothing about this is a panacea for innovation: most of the hard engineering problems remain, and this doesn’t mean you can magically create new products. But there’s no question that even changing the hurdles means there’s new potential. If 2009 is about anything, I hope it’s about people finding new solutions to taking that potential and tapping into it. You can bet it’ll be a major focus of this site.

Congrats, Randy – we’ll be watching.

Happy New Year.

Minitek is a massive “electronic music + innovation festival” coming to New York this weekend. If you’re anywhere in the area, I definitely recommend finding a way to get out here. And if you’re far away, stay tuned, as we’re planning some epic coverage here on CDM. The events are split between Midtown Manhattan and the eccentric shores of Coney Island (above).

The music lineup includes some big names — think Richie Hawtin, Magda, M.A.N.D.Y., Audiofly, Audion (big highlight for me), Adultnapper, Heidi, Guy Gerber, and the like, with Francois K closing out the evening. But for those of you whose tastes tend in the more experimental, there’s quite a bit of that, too.

Under the “innovation” category, we’ll see works like James Patten on the tangible Audiopad interface, Norman Fairbanks jamming on the Tenori-On, Audiocube players, and more. There’s also a big lineup of generative visuals for the nights; we’re covering that on our sister site Create Digital Motion. You’ll also be able to chat up record labels and play with the tech during the day in the village on Coney Island.

Minitek Festival Site; ticket info (you will want to get a pass if you can!)

Set times are finally up, though that includes only the main acts, not the innovation. (I know some of the, ahem, “innovators” so when they find out themselves when they’re playing, I’ll pass it on!)

Details are constantly coming in, so you can watch the blog or mailing list on the site. And yes, Astroland just closed, but Coney Island’s other rides are up, in case you want to hop on the Cyclone between sets. (Artists playing, if anyone wants a CDM interview on one of the rides, I’m game. I think it’s be fun to do an interview mixed with screams on the coaster.)

Tangible Roots

One of the projects I’m most excited about is Roots, a new music/visual interactive installation for the Brick interface. Co-creators Jordan Hochenbaum and Owen Vallis took home a Tenori-On the last time they ran into CDM (video of that after the jump); this time they team up with London-based designer Memo Akten for some beautiful generative visuals. (They’re also behind the Arduino-based Monome clone we saw a couple of weeks ago.) Here’s a short teaser video; we’ll have more on this after the weekend.

Roots Multi Touch Tangible Installation Teaser from BricK Table on Vimeo.

And that’s just a taste of one of the innovation day projects, every single one of which have gotten a mention (or three) somewhere in the pages of CDM in the past (really)!
Here’s Jordan and Owen in the heart-stopping finish to CDM’s Futuristic Design Challenge, in the moments that won them the Tenori-On prize we gave up.

Brick Table @ Yuri’s Night Bay Area 2008 Lightning Round from BricK Table on Vimeo.

The ground breaking MBRP-101 from Fisher Price is the best all in one music playback solution for mobile and club djs that need affordable and reliable performance. Visit http://www.djtechtools.com/?p=202 for more info and release dates on this exclusive release.

World Famous Dj Mei Lwun (5 time dade county YMCA battle champ) was kind enough to review the unit for us. Check him out at:
http://www.mei-lwun.com/

Brilliant. Of course, this being CDM, I fully expect:

• Commentary on the terrific tangible interfacing aspects of this design (seriously — look at the tactile quality, the module for storing records, the accessible interface, the minimal design, the bright colors)
• Someone to use this live
• Someone to hack this into a usable DJ controller — maybe adapting it to Ms. Pinky control vinyl, or putting sensors into the plastic tone arm

Brings back great memories as I used to play with this as a kid at day care. Oldskool Fisher Price was the greatest.