Imagine a nightmarish, dark-world, alternative-reality version of Wii Music, one that sends Miyomato-san screaming. That’s what you get from tokoloten, in a very un-Nintendo noise performance, as found on comments. The Wii is just one of his tools:

tokoloten uses a variety of objects such as magnet motors, infrared devices, game controllers… in order to hide his lack of conventional technic. Depending on the venue, the show might be ambient-like, experimental or electronica with weird cinematographic references. But it most often combines all of this.
tokoloten is based in Lausanne, Switzerland.

It’s proof that the controller – any controller – is in the hands of the creator, and what it sounds like is entirely undetermined.

Mapping a hardware input to a sound means making an abstract connection between one physical action and another sonic reaction. What that relationship is is entirely up to you. I was honestly a bit surprised by some of the impassioned critical reactions to yesterday’s brief mention of the use of the Wiimote as a studio recording. Of course, that proves the creed of the blogger – post first, ask questions later, and when in doubt, just post. Amidst some of the frustration, there are some good discussions, though I do dream of an Internet on which we criticize content without name-calling.

But the reality remains: controllers are always abstracted from the sound, by definition, and whether they’re satisfying to you depends on how you’ve mapped them. I don’t know what qualifies as innovative, but then, there have been times when I’ve very much enjoyed turning a knob, so “innovation” isn’t always what matters to me. I tend to fall back on Duke Ellington – “if it sounds good, it is good.” For controllers, that means “if it feels good, it is good.” You’re the one with the controller in your hands.

For an alternative example, musician/artist Kassen has an excellent session on improvising with custom software and game controllers. Below, you can catch some of his talk from Amsterdam’s famed STEIM research center, which has a long history of researching the controller-music connection. After all these years asking that question, what we have is …more questions. But that’s a beautiful thing.

Kassen (DJ, performer, ChucK programmer) from STEIM Amsterdam on Vimeo.

LaDiDa Demo from khush on Vimeo.

There’s no question iPhone/iPod touch development – really, just clever mobile development – has gotten a bit overhyped lately. But that’s all the more reason to do a round-up of genuinely interesting stories, real innovation happening on the platform. So, I’m clearing out my inbox with some of the more creative tools appearing recently on Apple’s mobile gadgets. There’s no better way to kick off today’s festivities than with this unusual “reverse karaoke” creation.

Sure, people may think they’re tone-deaf. But even the layperson has extraordinary powers of musical perception. So how could you train your iPhone to perceive and respond to music? That’s the question asked by LaDiDa for iPhone, the first of a new line of “intelligent” music applications for mobile devices. A “reverse karaoke” tool, the idea is to listen to singing and fake accompaniment, rather than having you sing along to canned backing tracks. Nothing is pre-programmed; everything is generated on the fly on the device.

It’ll even make up a Bollywood accompaniment to your singing:

LaDiDa Bollywood Duet from khush on Vimeo.

Of course, to me, it’s interesting not only what the iPhone is able to musically, but also what these algorithms are unable to make sound musical. Both reveal a whole lot about how we hear and conceptualize music. I think the team deserves real credit for making this fun, though, and on constrained hardware.

The app’s creator Khush follows in the footsteps of Smule in that it takes hard-core academic music research and uses mobile devices as a vessel for getting that tech in the hands (literally) of the general public. (See my interview with Smule founder and ChucK originator Dr. Ge Wang.)

Parag Chordia, developed at professor at the Georgia Institute of Technology and the gentleman you see in the video, spoke to CDM about what’s happening behind the scenes. He tells us about how this application was developed, and how the intelligent algorithms work (or at least try to work, as music analysis and auto-accompaniment remain at early stages).

First, an explanation of the app.

For many, mobile technology and developing for the iPhone and the iPod touch is a fad and a Gold Rush. Good designers, though, take a longer view of how interaction can be expressive. And there are few people with a better sense of the big picture of small devices than Dr. Ge Wang. The co-founder, CTO, and Chief Creative Officer of Smule has a background that goes well beyond the latest Apple platform. Along with Perry Cook at Princeton, Ge Wang is the co-originator of ChucK, a real-time programming language for synthesis so efficient some people use it live onstage. (ChucK, as an open source project, now has a terrific team of people behind it.) ChucK is the sonic engine that powers Smule’s projects. Ge Wang also teaches at Stanford, working with students and fellow researchers to explore new ways of interacting with music technology.

Ge Wang joined me for a lengthy phone conversation recently. He really contextualized why the iPhone is important in the grand scheme of things, but also how the people at Smule and Stanford (and Princeton) can approach technology for musical interaction, focusing on what devices are rather than what they’re not.

(The audio here, believe it or not, is extensively edited – Ge Wang is that easy to talk to. I hope the next time it’s over beers rather than Skype.)

The full interview can be played below, or downloaded directly.

Play this track:

Download MP3 of the interview

Thanks to KORG and the Nano Series for their support of programming on createdigitalmusic.com.

Lastly: a video of the Smule team headquarters and playing around with Leaf Trombone for a Zelda duet!

More information:
The Mule Chronicles [Smule Blog]
http://smule.com/
Center for Computer Research in Music and Acoustics (CCRMA), Stanford University

Previously, for more on Ge Wang and CCRMA:
Make:TV Meets Stanford Musical Inventors, Feedback Piano

Video + Audio Subscriptions, iTunes Podcast

CDM is now launching regular audio content on the artists and inventors we cover as part of our series CDM Sounds. You can subscribe (and review the podcast) via iTunes, where you’ll also find our new video series:

cdm Sounds Podcast [audio]
cdm TV [video]

Or using your software of choice, subscribe directly to RSS. (I like to follow podcasts with Banshee and Winamp this way.)

We’ve fixed some transcoding issues for iPod touch/iPhone on the video podcast. Please do test this and let us know if you have any issues on your software/hardware.

Barry Threw demos the K-Bow at Maker Faire from The Amazing Rolo on Vimeo.

Yann Seznec aka The Amazing Rolo brings CDM his coverage of
music tech at the Maker Faire in three episodes today.

As long as there have been computers, violinists have looked for ways of extending the nuances of their physical performance into the digital realm. (Us keyboardists have it easy – we’re used to pressing an array of levers, and a lot of the gestures we make are, arguably, superfluous.) Many of these concepts return to the idea of the bow.

The K-Bow by Keith McMillen Instruments is a Bluetooth-enabled bow with sensors that read bow angle, length, acceleration, grip pressure, and even hair tension. It’s accompanied by software developed in Max/MSP. The bow itself is one of those “if you have to ask, you can’t afford it situations,” at US$4000-5000 retail, though they claim the bow itself – specially-designed kevlar and carbon graphite, anyone? – can compete with more expensive bows even before you add in the sensors.

As an addendum to the last story, Ivica Ico Bukvic sends along an example of the [myu] Max/MSP + Unity game engine combination in action. Here’s the surprise: Unity isn’t generating visuals. Instead, Unity simulates ripples created by movement in the space, and builds physical models that are sonified and spatialized by Max/MSP.

Speaking of work involving art museums and the combination of Max and Unity, VJ Anomolee notes in comments his own work with the pairing. Lightbent Synth is an in-progress piece with alternative controllers and sensors that produces sound with a novel visual representation (sound’s very quiet in this preview — more hopefully once it progresses):

Lightbent Synth from VJ Anomolee on Vimeo.

Ivica explains the top work: