Tenori-On and iPad apps, hardware designs and visual creations: set against the beautifully-generative mind of Japanese/New York artist Yayoi Kusama, the flurries of dots and circles and patterns in musical interfaces take on a richer meaning. This video, from a workshop hosted at the Tate Modern alongside an exhibition of Kusama’s work, needs little introduction. Instead, the dizzying cuts of geometric abstraction, the array of visual ideas for musical interface begin to take on the same personality of her expansive creations. The galaxies produced out of the minds of musicians somehow overlap with this iconic artist. I hadn’t really made the connection before, even as a fan of her work, but with this workshop, the sympathetic vibrations – intentional or not – become clear. Description:

Sonic Kusama:
Workshop exploring connections between the work of Yayoi Kusama and creation and representation of new music & sound art through visual audio interfaces.
Presented by Simon Little and Kelvin Brown with Chase Lane.
Audio track by Capstone Music
Video production by Territory Studio

If you’re in London, Infinite Kusama is on view now at the Tate Modern.

Today, I’m in London doing a hands-on workshop on visual metaphors for music, and covering various topics filed under “synesthesia” at Music Tech Fest. It seems appropriate, with the subject matter on the brain, to revisit the topic of visuals and music in a series of posts.

When you make hardware, with knobs and faders, you’re constrained by physical space – the amount of room on a circuit board, the radius of a knob cap, the size of your fingers. But before you get there, the first step is to sketch an idea. Imagine if you could do that with a marker and paper and ink.

SketchSynth is the latest attempt at a drawable set of controls, letting you turn an illustration on paper into something you can actually use to make music. It’s not the first – this dream of being able to make things come alive with nothing more than a magical pen is an old one – but the execution looks superb. There are two basic approaches to the idea: one is to use some sort of conductive ink to turn the drawing itself into a sensor, and the other is to point a camera at the drawing and calculate where a user makes contact with the drawing. SketchSynth opts for the computer vision approach, by way of OpenFrameworks and the old standby of free and open source vision, OpenCV. (Kyle McDonald’s ofxCV does the heavy lifting.)

Conventional vision fits the task well: faders and knobs respond as expected, even though they’re only ink on paper. While it’s a drawn interface, and could look like anything, the behavioral metaphors all come from hardware: there are sliders, momentary buttons, and pots. Place the tool in “edit” mode, and the computer analyzes what you’re drawing; in “play” mode, the camera tracks your finger as you manipulate what you’ve drawn. The project goes one step further than many that have come before, by overlaying a projection calibrated with your drawing for interactive visual feedback as well as sonic. Sound in this case is provided by Pd, but OpenSoundControl (OSC) messages let you connect to other musical (or visual) tools. See more in the making-of vid, at bottom.

Creator Billy Keyes is working with the right mentor, too, completing this as research for Golan Levin, who has long explored the relationship of drawing and musical gesture. His Sonic Wire Sculptor was a seminal creation in connecting drawings to sound, using a tablet to produce three-dimensional “wire” structures and corresponding sound synthesis. His Messe di Voce performance piece neatly reversed the relationship, using the voice as the input to animate drawings and illustrations. More of Levin’s work at flong.com. See also: composer Xenakis’ UPIC, which lives on today as IanniX, a tool getting a lot of development attention.

Keyes’ project, already getting lots of blog buzz, is notable for its practicality and immediacy; it seems a tool many others might use and build upon rather than a single piece of art.

It’s lovely to see projects, particularly academic projects, come to some form of completion and clarity. Speaking of completion and clarity, at some point a proper survey of drawn musical interfaces seems a must, but that will have to come another day. Where can I get a nice full English breakfast?

I’m late to the party, but hat tip in particular to Creative Applications

Project Page at golancourses.net; Linux source is promised soon

I’ve covered a number of these sorts of projects over recent years.

Paper, Drawing as Musical Controller: A Round-Up (including a number of paper examples)

Drawing Sound: Crazy Touch Interface Sound Experiments with Usine, PC (using only a screen – and thus producing a very different experience of drawing

Imaginary Instruments: Marker and Paper as Controller (this 2009 project is almost a direct analog to SketchSynth, minus the projection)

iPhones, Pencils: Hand-Drawn Music Interactions, Tokyo Subway Mobile Jam (this one isn’t quite the same as the others, using pencil and paper to design an interface for a screen – but it’s nice to recall that you can do that, as well)

Pen and Paper as Graphical, Digital Music Score (from earlier this year, a more open-ended design)

Video: Sonic Wire Sculptor, 2003:

It starts as just another toy to play around with in a few minutes of distraction in your Web browser – as if the Web were short on distraction. But then, something amazing can happen. Like a musical Turing Test, you start to get a feeling for what’s happening on the other side. Someone’s stream of colored dots starts to jam with your stream of colored dots. You get a little rhythm, a little interplay going. And instead of being a barrier, the fact that you’re looking at simple animations and made-up names and playing a pretty little tune with complete strangers starts to feel oddly special. The absence of normal interpersonal cues makes you focus on communicating with someone, completely anonymously, using music alone.

Dinah Moe’s “Plink” is the latest glimpse of what Web browser music might be, and why it might be different than (and a compliment to) other music creation technology. You can now create private rooms to blow off steam with a faraway friend, or find new players online. It’s all powered with the Web Audio API, the browser-native, JavaScript-based tools championed by Mozilla. That means you’ll need a recent Chrome or Firefox (Chrome only at the moment; this is a Chrome Experiment), and mobile browsers won’t be able to keep up. But still, give it a try – I think you may be pleasantly surprised. (Actually, do it right now, as you’ll probably be doing it with other CDM readers. I expect greater things!)

http://labs.dinahmoe.com/plink/

Thanks to Robin Hunicke, who worked with multiplayer design and play at That Game Company’s Journey on PS3 and now on the browser MMO Glitch. I think her friends were more musical than most, because the place came alive after she linked from Facebook.

The browser is becoming a laboratory, a place to quickly try out ideas for music interaction, and for the code and structure that describe music in a language all their own. As in Plink, it can also benefit from being defined by the network and collaboration.

Dinah Moe’s experiments go in other directions, as well. In Tonecraft, inspired by the 3D construction metaphor of Minecraft, three-dimensional blocks become an alternative sequencer.

http://labs.dinahmoe.com/ToneCraft/

There are many reasons not to use Web tools. The Web Audio API still isn’t universal, and native options (like Google’s Native Client) have their own compatibility issues, stability concerns, and – because of security – they don’t do all the things a desktop application will. Desktop music tools are still more numerous, more powerful, and easier to use, so if you’re a reader out there finishing a thesis project, you might look elsewhere. (Actually, you’re probably in trouble, anyway, by any nation’s academic calendar, given it’s the First of May, but I digress.)

But think instead of this as another canvas, and the essential building blocks of interface design, code, and networking as shared across browsers and desktop apps. Somehow, in the light of the Internet, its new connectedness, and its new, more lightweight, more portable code and design options, software is changing. That transformation could happen everywhere.

If you need something to help you meditate on that and wait for a revelation to occur to you, I highly recommend watching a soothing stream of dots and some pleasing music as you jam with your mouse.

Of course, in the end, like a digital mirror, it might inspire you to go out to the park with a couple of glockenspiels and jam the old-fashioned way. But maybe that’s another reason to make software.

(Here’s a video, in case you’re not near a browser that supports the app!)

More, plus reflections on adaptive music:
http://labs.dinahmoe.com/

Through the eyes of satellites, roving Google trucks, aerial imagery, and more, we have plenty of eyes on our planet. But what does it sound like here on Earth?

In a Web application and accompanying art installation, the world turns as it echoes sounds recorded around the world on Creative Commons-licensed site Freesound.org. It’s stunning to hear our world’s acoustic diversity – in some strange way, even more than seeing it, in that sounds can instantly give you a sense of place and time. You can load a version on your browser or on the iPad; then, from the world’s cities, listen as sounds mix automatically from one locale to another in an ambient sound score.

Browser Version (animates a bit slow for me, but works)
iPad World Sound Mix app [free | iTunes]
(via Hermann Helmholtz – great tip!)

The basic notion is something we see repeated regularly, even with this visualization; this is a fantasy those of us who work in sound routinely entertain. But it’s doubly worth mentioning, in that it’s an excuse to mention the lovely Japanese label/artist/laboratory 43d.

43d engages sound through a variety of tools. In the 43d laboratory, the spinning Earth interface finds its way into an installation (video below), iPad app, and browser app, as workshops send participants into the field to listen to their environment and gather more sounds. Such exercises have an added bonus for us electronic musicians, of course, as collected sounds can easily become the raw materials of music in any genre through the wonderful alchemy of our machines.

http://labs.43d.jp/

The installation and sound mix project:

“World Sound Mix for BankART LIFE3″ is a sound visual installation, generating new soundscape around the world. This work continues mixing the sounds at selected two points somewhere in the world from the database of huge quantities of environment sounds and generating new soundscape.

For this exhibition, we set up a magic box that resonates mixed soundscape in Sapporo and somewhere in the world. During the exhibition, a globe in the box keeps turning and resonating sounds in real time.

About sounds data:
World Sound Mix is based on a sound database from Freesound project, its sounds have been recorded and gathered by sound hunters around the world. The use of sound data is under the CreativeCommons Sampling+ 1.0 License. By the username and “freesound sound ID” shown on the globe, listener can refer to original content.

http://www.43d.jp/wsm2011/

Freesound.org, a terrific source of sounds:

http://www.freesound.org/

But what I especially like about all of this is that the environmental sounds don’t have to exist in a vacuum. 43d is also an ambient music label, the work of artist Junichi Oguro:

A sound artist who widens the realm of music. Born in Sapporo in 1974.
He started to compose music since his childhood, and received a grand prize at a national contest. In 2006 he visited Berlin for making music in various fields from commercial music for TV spots to sound space design in various areas of Europe. He also showcases sound art pieces in the realm of the contemporary art. He manages an ambient label “43d” which was established for creating leading edge sounds.

The just-released “Unfield” is breathtaking, turning effortlessly from rough-shod digital glitches to icy-sweet ballads and intimate, gorgeous vocals by Malloy Nagasawa. It combines custom software and control with more conventional recording techniques:

http://www.43d.jp/releases/

Have a listen:

Hope to hear more from this whole project.
43d.jpg

By the 1990s, the notion that computer software could be a means of delivering interactive digital art more personally was enjoying a Renaissance. This was the age of the Voyager CD-ROM, which catered to new multimedia PCs and Macs with titles from the likes of Laurie Anderson and Morton Subotnick, the decade in which Brian Eno released Generative Music as software and Monolake – before Ableton – included a Max/MSP patch with an album. But the reach of these experiments was doomed to be relatively limited.

Now, of course, things are different. First, we saw some widely-available audiovisual toys, coinciding in particular with the debut of the iTunes App Store. But now, those fairly one-dimensional experiments are beginning to blossom into something else. When these particular gadgets and app stores are forgotten, the question is whether those aesthetic adventures, the personalization of the digital art experience, will endure.

Joshue Ott, co-creator of Thicket for iOS, points to a review of that application on Apple’s App Store. “I always want to touch the masterpieces in museums,” a user says in that review. “I’ll use Thicket instead of getting arrested!”

“It’s the democratization of our own performance works,” muses Ott. “It’s a way people can play along with us,” he says. “We’re constantly creating processes to create sound and music; it’s what we’ve done for ten years or so,” chimes in Ott’s creative partner, Morgan Packard. “Now people can own the processes, not just the results.”

Ott and creative partner Packard have long each been visual and music performers, respectively. That meant what it has traditionally meant: the artist gets up in front of an audience, the real work hidden behind an onstage laptop. With Thicket, by contrast, the raw materials of that performance became embodied in the software itself, and thus in the hands of the audience, who can double as performer. At first, this software included only a simple mode or two, each with a specific sound, musical ambience, and visual look. Even in those versions, Thicket made some appearances in an occasional gallery show or performance – the app you download could also be the art.

As Thicket has added modes, though, it has evolved in a kind of platform of its own. Ott and Packard produce new works that can be distributed as in-app purchases (more on how they contend with that in a bit). The sum total of those modes has created a massive audiovisual playground, a compendium of ideas and aesthetic.

A new version released this week adds three new modes, seen in the video at top here, building atop modes added in late December. For the first time, you can use Thicket on an iPhone and not just an iPad; it’s a Universal app. Screenshot sharing is available, too. But the addition of all these modes, unveiled with a “reboot” of the app at the end of last year, represents a shift in thinking as these artists and developers reevaluated what it was they were doing.

“We found that the modes were becoming so different, so much deeper,” says Ott:

We were having such fun using it as a big sketchbook that we decided to ditch the ‘rotate to change modes’ system so that we could handle lots of modes, rather than just four or five. The modes in Thicket reboot are completely new, and each one is a lot more complex than the older modes. They’re all very different, and each have separate methodologies behind how you control them. We’re playing with different concepts in user interaction design, searching for the right intuitive feel to make a true audiovisual instrument (among quite a few other things).

In other words, if you haven’t played with Thicket lately, it’s a different animal. It’s a Long Play album to the first version’s single cut. The work is immersive, too; you can transmit video output via HDMI or VGA on the iPad, and get up to 1920×1200 HD output, with no menu intervening. (One of the many significant current drawbacks of Android for the moment for artists: the move to a soft menu on Android tablets means menu detritus that never goes away. Artists were intensely relieved this week when Apple’s new iPad kept its signature, dedicated hardware menu button.)

Morgan Packard says he has some strong feelings about why this kind of experience has value in his work:

I’d say where we both overlap is our shared interest in how abstract sound and picture, plus interactivity, all can work together. Thicket is a bit of a research sketchbook for us. There’s something very magical about just twiddling your fingers and having sound and visuals spring to life. Frankly, we don’t entirely understand this medium yet. But we like not knowing, trying to understand it in different ways.

The gestural thing is huge with us, and is at the core of what thicket is. It’s partly why I’m a bit resistant to the idea of layering features on to Thicket. Of all the different people who give us feedback, I get the most gratification from parents of special needs kids.The non-fiddly, large-motor interaction style is very accessible to a huge range of minds and hands. I want to explore this more, to give people new ways of feeling expressive and creative with movement and gesture. In my mind, that’s what’s really special about what we’re doing.

The duo did get a chance to try the app with people with different user needs. Ott explains:

Morgan and I actually toured a special needs school earlier this year and observed autistic kids using Thicket. A very special music teacher is using Thicket (among a couple of other technologies) to teach the kids music, and had found that it seemed to really empower them. He offered to let us visit and we happily agreed… really really amazing experience.

As Subotnick hoped years ago in “All My Hummingbirds Have Alibis” for Voyager, the distribution of art as software can create a new kind of “chamber” art, in which the work is personal, enjoyed by a few people. It can be a family or a couple of friends on a couch.

Of course, somewhere in all of this, these artists are looking for revenue in order to be able to devote the massive amounts of development and testing time the application demands. (Neither has quit day jobs, which means finding a way to devote resources to development.) Thicket easily climbed in download counts, but only after the application was made free. In-app purchases have been a tough mountain to climb, but have at least allowed some revenue to trickle in; the challenge was finding a way to make them appealing to users, says Ott:

I think in general people hate In-App Purchasing (IAP), because, in general, I think IAP is usually not handled so well. We have thought a lot about how to show people exactly what they are buying before they buy it, and I’m really pleased with what we’ve come up with. Every mode in the new Thicket has a pre-recorded “demo” of one of us playing the mode. Before you buy a mode you can watch this demo, learn what the mode can do, watch someone use it in an interesting way, and decide if that’s something you’re interested in or not. You can of course watch the demos even after you’ve purchased the mode (and the free Sinemorph mode also includes a demo as well). The demos are a great way for us to show users different tricks and techniques.

So the reboot is really about making Thicket a platform rather than just a single art piece. Something that we can keep adding to (with a financial structure that makes sense for us to keep adding to). Something that we can start collaborating with other artists on – we are talking to a couple of different people about releasing modes within the Thicket system. So yeah, that’s what the platform part is. We’re really excited about it, and what it will become in the future.

But these concerns aside, the developers aren’t just creating Thicket for users; they’re building something they use themselves. As Josh explains:

I’ve performed with Thicket now a couple of times, once at the excellent SONiC festival, and another at Issue Project Room in a program curated by Ryan Lott (AKA Son Lux), and have started to really feel like it has the potential to be a new form of audiovisual instrument. I want to see more stuff like it- things that generate graphics and audio intertwined, and I want to continue to explore these relationships in different ways. I’m actually pretty excited about performing with Thicket more, and I think doing so will push it even further in that direction.

“That’s really what an audiovisual instrument is to me,” says Ott. “It’s something that you can bang on and make something interesting, but you can touch it subtly, as well, to shape it, to express with it. That’s what I want to make. We’re right at the beginning of that exploration, and I think we have something that is a promising vehicle for it.”

You can try out the new Thicket now, as seen in CDM Apps:

Thicket @ CDM Apps
[Says iPad, is actually now Universal. PS - music and beauty flow from my fingers all the time - no app needed - but I'm glad now the rest of you get the chance.]

FRACT is a curious combination of music studio and puzzle game, merging elements of games like Myst with the sorts of synths and pattern editors you’d expect somewhere like Ableton Live. You have to make sounds and melodies to solve puzzles; by the end of the game, say the creators, you’re even producing original music. The work of a small student team out of Montreal, FRACT looks like it has all the makings of an underground indie hit – at least for music nerds.

As the creators describe it:

FRACT is a first person adventure game for Windows & Mac much in the vein of the Myst titles, but with an electro twist. Gameplay boils down to three core activities: Explore, Rebuild, Create. The player is let loose into an abstract world built on sound and structures inspired by electronic music. It’s left to the player to explore the environment to find clues to resurrect and revive the long-forgotten machinery of this musical world, in order to unlock its inner workings. Drawing inspiration from Myst, Rez and Tron, the game is also influenced by graphic design, data visualization, electronic music and analog culture.

The hub of the game is a virtual studio, collecting patterns and timbres. It’s right now in prototype phase, but it already looks visually stunning, an alien, digital world in which more-conventional step-sequencer views seem to emerge from futuristic landscapes. And you can spot Pd in the background (the free and open source patching tool, Pure Data). Update: the developers confirm that they’re working with the embeddable Pd library, libpd. That enables synths with sounds like phase modulation and classic virtual analog sounds, all modulating and generating sounds in-game.

The developers have also published plenty of sound samples so you can experience the musical side of this. Via SoundCloud:

While never released, one place some similar ideas has shown up is a prototype game inspired by Deadmau5. As in this title, two-dimensional editing screens and synth parameters are mapped to a first-person, three-dimensional environment. However, FRACT appears to take this concept much further, expanding upon the world, building more instruments, and actually turning those interactions into gameplay elements. The video of the Deadmau5 project – apparently done in-house for fun and not endorsed by the mouse-headed artist:

That title was the work of a game house called Floaty Hybrid; music blog Synthtopia got the scoop on this in August:
http://www.floathybrid.com
Mau5Bot Sequencer Lets You Make Music In A 3D World [Synthtopia]

We’ll be watching this one develop, certainly; good luck to the team!
http://fractgame.com/

Trust your ears.

It seems a simple instruction, in the teaser video for this project by Stockholm-based producer/composer Håkan Libdo. But for those of us used to having vision, focusing in on one sense – even the sense on which we rely for music and sound – can be an extraordinary experience. If digital interface design has done anything, it has forced new ways of looking at design across senses, and not just in a weary repetition of “we always do it this way.”

Playing tennis in InvisiBall becomes a new experience. Sonic cues along direct the ball from side to side. And apart from experiencing the game – blindfolded or even as a non-seeing person – the result can be a performance. Developed with programmers Magnus Frenning and Jonatan Liljedahl, the game relies on three-dimensional sound and senses user play, a new musical game.

A computer, while tracking the players, seamlessly mixes sonic cues with a musical soundtrack, so that the results are melodic and not only sound effects. Played in a darkened room, the rackets emit infrared light. Wiimotes here don’t move, as they do when playing games like Nintendo’s Wii Tennis; instead, they’re just IR sensors.

An audience gets to “see” the position of that invisible ball, so that spectating is a slightly different sensation; a TV displays where the ball would be.

More information on Håkan Libdo’s site; he tells us this could be making appearances at festivals this year. (So… if you’ve got a festival, and you’re reading… uh, send CDM some tix, too, and we’ll visit?)

http://hakanlidbo.com/archives/2075

I’m not sure how everyone who owns an iPad uses it for music, but I find myself strangely drawn, more than anything else, to analog step sequencers. With MIDI connections – via a special interface or a standard USB MIDI interface connected via adapter to the tablet – you can even drive hardware. For me, the app of choice has been Little MIDI Machine. Developer Chris Randall has a new application in the analog-style sequencing category, though, called Phaedra.

If you haven’t grabbed it already, you have until the New Year to get it for US$4.99 before the price jumps to ten bucks. And you get an impressive array of features:

• Multiple buses, with 32 steps max each
• Programmable note, velocity, gate time, and two MIDI CC outs for each step
• Send or receive MIDI Clock for sync
• Use MIDI hardware (via Core MIDI), other apps (using “background MIDI” or OMAC), or your computer (networking via a MIDI Network Session

Phaedra for iPad [iTunes Store Link; you'll need iOS 5.0]

I wanted to know more about the creation of Phaedra. Developer and musician Chris Randall, known for his work with boutique plug-in maker Audio Damage, released this under a new moniker, Naughty Panther, which does iOS and MIDI development. Chris has been known to mix old and new, as with his musical use of the Apple II. Here, he gives us some insight into how he went through the design process on this new tool.

CDM: How do you sketch out the UI on a project like this? Paper and pencil?

Chris: Normally I just make notes in a little Field Notes notebook as I think about them — I carry one everywhere — then once I have a clear mental image of what I want to make, I just bust it straight out in Photoshop (or, for skeuomorphic interfaces like this one, a combination of Photoshop for the panel and 3D Studio Max for the knobs and buttons).

What inspiration did you consider as far as hardware?

Adam sent me the page for the Moon Modular 568 “Sequential Trigger Source” and quipped that it would make a fun iPad app, and I kind of ran with it. (Their site is a frame-a-thon, but ’tis here: http://www.lunar-experience.com/home.html) As you can see, the UI for Phaedra closely follows the Moon Modular design. The functionality diverges quite a bit, though. Once I had the initial look, I just started adding features I personally thought would be nice in a step sequencer. Then the beta testers had some more input. And now that it is out, I’m getting a ton of great feature requests from the users. The next update will have some cool tricks in it, provided I can figure out how to code them.

What hardware would you recommend for connecting MIDI?

The best MIDI I/O situation for Phaedra is an Alesis iO Dock. Hands down. I have tested it with a half dozen MIDI interfaces via the [Apple] Camera Connection Kit [for connecting driverless USB MIDI interfaces], and they all work fine except that $5 cheap one that people buy from Amazon, which unsurprisingly, has problems. Phaedra also automatically creates a virtual port and connects to all virtual endpoints running on the iPad, so it can drive NLog Pro, Animoog, whatever. And it also is [Apple MIDI] Network Session aware, so it can just work over wi-fi.

I’m going to add OSC output as an option in the next update, I think. I’m still thinking about how to implement that, but it seems like it would be handy, and then Phaedra’s abilities would be greatly increased.

I’m also adding trigger outputs via the audio ports, so you can use Phaedra to clock an external modular sequencer like Makenoise René, or a pre-MIDI drum machine. I’m giving it 48-ppq clock for Korg drum machines, 24-ppq clock for most every other drum machine, and arbitrary rates from 16-ppq on up to 1-ppq for driving modular sequencers and your Monotribe. This should be handy, and will be sample-accurate in line with the MIDI clock output.

See also Chris’ blog entry on the app at Analog Industries:
Some Cool Shit

Dig into humanity’s past, and alongside the earliest tools, you’ll find some of the earliest instruments. Designing objects for expression seems to be an essential part of civilization.

Martin Kaltenbrunner, a co-designer of the Reactable tangible music interface, is also a professor in Interface Culture at the Linz University of Arts in Austria. There, in the land of Mozart and Haydn, he works with students to explore what interface design is.

So, when I got to spend some time with Martin in New York in September, I was interested in more than just the flashy coolness of the Reactable, the futuristic table-with-blocks interface for music. We got a chance to talk about instrument design generally. The funny thing about the Reactable is that it is closer to the experience of working with a modular synthesizer and oscilloscope than anything else, with the sense of physical connections of sound to object you’d get from classic synths. It is something unique, truly, but that’s its pedigree.

Martin and I got to give a talk together at the Austrian Cultural Forum New York, a terrific hub in which Austrian artists frequently are paired with New York-based folks, all in a lean, tall modern landmark building in Midtown. We also performed together, which for me was a real pleasure; Martin claims not to be a musician as such, but was good fun as an improvisation partner.

The next day, we headed to Manhattan music education center Dubspot, where Martin’s creation quickly attracted crowds of interested students and educators. Dubspot filmed our encounter for the video at top. Amusingly, the prominent synth sounds you hear at the beginning are not the Reactable, but our own MeeBlip open source synth, which I brought along to illustrate conventional tangible instrument design with switches and knobs. (If you’ve been impatiently waiting for news on the MeeBlip, believe me, I’m even more impatient – more announcements on that this week and next, following a production quality issue with a contractor that required us to reboot the run of new instruments.)

Reactable Live @ Dubspot! Interactive Sound Design Workshop Video Recap [Dubspot Blog]

YouTube commenters, that subtle and thoughtful bunch, are complaining that the tangible Reactable will set you back thousands of Euros. But at ACF and Dubspot, I was also equipped with the far more economical and portable alternative: Reactable Mobile runs on both Android and iOS. (I was pleasantly surprised to discover the app runs perfectly on a Galaxy Tab 10.1 from Samsung; I’m still a long, long way from being able to recommend buying an Android tablet, but if you’ve got one, I can certainly recommend this app.) Now, don’t get me wrong: the experience was nowhere near as fun as using the table. On the other hand, you can’t fit the table into a seat-back pocket on easyJet, and the savings in cash is proportional to the sacrifice in experience. What impresses me is that the design of the physical Reactable “flattens” so nicely onto the screen; I think it’s a user experience triumph that you can make that translation. And I was able to load up a few loops of my own music and jam along with the MeeBlip and Martin on the (real) Reactable.

Looking beyond the Reactable, Martin addressed these larger issues of tangible interface design at TEDx Vienna – a fitting locale for talking the history and future of music. His whole presentation, and a sweeping concept map of what he discussed, is available.

Martin Kaltenbrunner: Tangible Music

If you just want to get your Reactable on and can’t afford the table, see Reactable Mobile below. (Seen here on an iPad 2, but I’ve run successfully on the original iPad and the Galaxy Tab – the experience is more or less identical, thanks to portable code.)

And if you can afford the table, you rockstar, uh, can we be your friend?

http://www.reactable.com/products/mobile/
[the awesome table version]
Martin Kaltenbrunner website [with plenty of academic links]

Compare the complex model of what a computer can use to control sound and musical pattern in real-time to the visualization. You see knobs, you see faders that resemble mixers, you see grids, you see – bizarrely – representations of old piano rolls. The accumulated ephemera of old hardware, while useful, can be quickly overwhelmed by a complex musical creation, or visually can fail to show the musical ideas that form a larger piece. You can employ notation, derived originally from instructions for plainsong chant and scrawled for individual musicians – and quickly discover how inadequate it is for the language of sound shaping in the computer.

Or, you can enter a wild, three-dimensional world of exploded geometries, navigated with hand gestures.

Welcome to the sci fi-made-real universe of Portland-based Christian Bannister’s subcycle. Combining sophisticated, beautiful visualizations, elegant mode shifts that move from timbre to musical pattern, and two-dimensional and three-dimensional interactions, it’s a complete visualization and interface for live re-composition. A hand gesture can step from one musical section to another, or copy a pattern. Some familiar idioms are here: the grid of notes, a la piano roll, and the light-up array of buttons of the monome. But other ideas are exploded into spatial geometry, so that you can fly through a sound or make a sweeping rectangle or circle represent a filter.

Ingredients, coupling free and open source software with familiar, musician-friendly tools:

• Two projectors
• A monome
• Processing, the elegant and artist-savvy free software for visual code
• Ableton Live and Cycling ’74′s Max for Live, acting as the interactive glue with the sound world
• Drumaxx, Image-Line’s tasty physical-modeled drum synth
• Native Instruments Battery, the sampled drum engine
• Eclipse, the free IDE, for Java coding in this case
• Community Core Vision and reacTIVision (based on our previous info, at least), free and open source community-based projects for making the interfaces you see in movies happen in real life.

Another terrific video, which gets into generating a pattern:

Now, I could say more, but perhaps it’s best to watch the videos. Normally, when you see a demo video with 10 or 11 minutes on the timeline, you might tune out. Here, I predict you’ll be too busy trying to get your jaw off the floor to skip ahead in the timeline.

At the same time, to me this kind of visualization of music opens a very, very wide door to new audiovisual exploration. Christian’s eye-popping work is the result of countless decisions – which visualization to use, which sound to use, which interaction to devise, which combination of interfaces, of instruments – and, most importantly, what kind of music. Any one of those decisions represents a branch that could lead elsewhere. If I’m right – and I dearly hope I am – we’re seeing the first future echoes of a vast, expanding audiovisual universe yet unseen.

Previously:
Subcycle: Multitouch Sound Crunching with Gestures, 3D Waveforms

And lots more info on the blog for the project:
http://www.subcycle.org/