Create Digital Music » Wii Controller as Complete Audiovisual Musical Instrument, and How Less is More

Oct 15 2007

Wii Controller as Complete Audiovisual Musical Instrument, and How Less is More

Wii controller

A team of artists has turned the Nintendo Wiimote into a controller for an adaptive, 3D sound environment. Claudio Midolo, Edgar Castellanos, Natan Sinigaglia, and Pedro Mari worked together on the project, and have posted extensive technical details if you fancy trying something like this yourself.

The basic setup:

Mac for control input via Bluetooth and music generation

aka.wiiremote external for Max/MSP (by Masayuki Akamatsu, freely available though Mac Universal only) for communication with the Wiimote

Propellerheads Reason for sound
Windows PC for 3D visualization, via visual software vvvv (Windows-only)

UDP for data transmission from Mac to PC

More details and lots of video:
wiiwiiwiiwii (Thesis Project)

I’m not totally convinced from the perspective of musicality, but it’s a promising project, and it demonstrates how a controller like the Wii can allow a reduction of controls. As it happens, I’ve been really aware of this as I play EA’s Madden Football. They even have a new single-controller scheme, because you can do so much gesturally, which previously required lots of buttons or keyboard hotkeys. I think this kind of gestural control could be relevant not only to the Wii but other controllers — there’s no reason only the Wii needs to have tilt sensors. Even my digital camera has a tilt sensor for adjusting the orientation of image previews.

So there you have it. Students, tell your computer music / new media teacher I told you have to spend time playing Madden football, in order to experience the possibilities of gestural interactive control mapping. Seriously. You’ll just have to take on my Giants.

Thanks to Chris O’Shea for the link. He’s got so much awesomeness on his site Pixelsumo that I don’t know where to begin.

Want to Get Started Circuit Bending?

Tiny PCs for Music: UMPC Runs SONAR, Fruity Loops Like a Real PC

15 Comments

Leee

i wish there was an easier (already coded!) way of doing this so that there was just one program on my mac that could do audio and video with the wiimote like this.

October 16, 2007 @ 1:17 am

Peter Kirn

Actually, there’s no reason you couldn’t do this inside one app. For instance, you could build a standalone Mac-only app using Max that did visuals and sound generation within a Max patch. I think for these guys, doing the extra pieces actually made it simpler for them … and may have also related to how they broke up the collaboration.