Playing Music with Light Pens, Flourescent Bulbs, Brought to You By … Sony?

The urgency of being way behind a single dominant player can make electronics makers do some odd stuff to promote their products. iPod, once an icon of digital cool, has achieved such ubiquity that it doesn’t even try to be hip any more. The thing is being promoted with American Idol, for crying out loud — not exactly indie cred. We saw Microsoft enlisting indie musicians and animators to sell Zune, of course.

But here’s where things get surprisingly amazing: Sony is using weird and wonderful Japanese experimental music to promote Walkman.

Now we’re talking.

And whether or not Walkman is cool again, this is for sure: Japanese experimental musicians? Mind-blowingly cool. And, apparently, in love with using light as a controller for sound.

Atsuhiro Ito uses contact mics on a fluorescent bulb he dubs the Optron. Instead of just being stage eye candy, the bulbs are really making the sounds here; coupled with guitar effects, he can solo on the bulbs. It’s what the Knitting Factory will be like after the nuclear winter. I can’t wait.

Taeji Sawai uses a light pen to draw melodic lines and rhythmic onto a screen. The basic effect – track light from a single source – is old. Yet he’s clearly got a brilliant aesthetic mind that makes it all work; the elements are strikingly simple but never fail to be engaging. And there’s a strong connection to work by his fellow sonic inventor Toshio Iwai.

Thanks to our friend Donald Bell of cnet, aka very talented and (cool) musician Chachi Jones, who has a great write-up:

Sony Walkman promos are awesome, confusing

Confusing? No, I’d say Sony is confusing; the real question is why their Walkman can’t be more like these ads. Plus, since neither Don nor I can read Japanese, how do we know those characters don’t say something like “Hey, guys, sorry for that bit with the lousy boring electronics – we’re coming back from the dark side to make awesome things again”? Okay, maybe not. (Do let me know if the next one says “Fine, you damned snarky blogger, I’d like to see you run a giant multinational corporation.”)

Admittedly, the problem here is this makes me want to toss my iPod touch out the window and build my own open source MP3 player with Popsicle sticks and wire, or, at best, mod an original Walkman so I can play circuit-bent OGG files using power from a bicycle. At the very least, I’m ready to add to my Atsuhiro Ito and Taeji Sawai collection. And I don’t think their full body of work is on iTunes. That’s just as well.

So, Sony, thanks. Now, will you let us run homebrew music apps on your PSP? Please?

Open-Source 3D Webcam MIDI Controller

Interested in using webcams to translate on-screen motion to MIDI? Want x, y, and z 3D tracking? Ben Tan writes to let us know about his in-development software project called Peripheral MIDI Controller (pmidic) which does just that. The current build is still a work in progress, but has added enough stability and features that it should be worth a look.

Peripheral MIDI Controller

Grab your pen light and start waving it around for filter cutoff and resonance — whoo!

Right now, it’s Windows-only, but the libraries on which it’s built are cross-platform and could be ported to both Linux and Mac. (He’s using Intel’s OpenCV, which is the most popular, open platform for computer vision — odds are if you’ve seen slick webcam tricks, OpenCV was involved, because writing these algorithms from scratch requires a heavy-duty math and computer science background. And the app itself is built with the superb super-cross-platform wxWidgets library.)

If you’ve been looking to experiment with webcam control, this could be a good start — practicing interpreting the control data, experimenting with lighting, and experimenting with inputs all take time. And if you’re a coder, the project is open, so all of us C++ beginners can slog a little collectively.

Tonight in New York: Detecting and Visualizing Motion, Free Workshop

Tonight here in New York, I’ll be presenting a free workshop on detecting and visualizing motion from camera inputs, which may be of interest not only for those of you eagerly anticipating the new Create Digital Motion site, but also anyone who’d like to use cameras as controllers for music. Full details after the jump.

If you’re not in New York, don’t fret; I plan to organize this stuff and have online examples/tutorials in the near future. But if you are there, say hi! And yes, I finally plan to deliver on my New York CDM get-together promises in June; my schedule will finally allow that.

Still from a recent performance with Eric Dunlap and Mare Hieronimus at Eyewash, the visual performance series in NYC.

read more

@AES: “Subversive” Hipno Plug-ins Use Color, Game Pads, Webcams

Cycling ‘74 has released a new collection of plug-ins I’ve been eagerly anticipating, developed by a team from Electrotap. These aren’t your run-of-the-mill audio effects and virtual instrument: think spectral and granular sound, wild filter/delay stuff, morphing color-palette interfaces, and input from gamepads and webcams:

Cycling ‘74 Hipno “Subversive” Plug-in Collection

US$199 for a whole mess of plug-ins; works with everything. (Mac RTAS/VST/AU, Windows RTAS/VST.) More after the jump.


read more

Run Guitar Effects With Your Face; More on Motion Tracking and Gestural Control


If you haven’t yet seen commentary on this at Music Thing or (originally) WWMNA, David Merrill of the MIT Media Lab has a project for controlling guitar effects with his face. (It’s not new, incidentally — dates to 2003, and you can look forward to more of this sort of thing at the annual New Instruments for Musical Expression aka NIME, due next in Paris in June 2006.)


So, uh, aside from being weird, could this ever actually be useful and not just freaky? Possibly: gestural mapping gets especially interesting, as David uses a nod to trigger an event, for instance. As video processing gets less intensive relative to computer speed, and gestural processing gets more intelligent, you could eventually interact with your tech like a musician. Hell, if your musician friends ignore you the way mine do, you may have even more luck with the tech.


What’s mysterious here is the software David mentions — check the PDF for more project details. The software is a modification to something called FaceSense, running on Linux. Don’t think this is something you can just go download, though. See also:


Vicon, providers of pro-level motion capture and control systems


Visual Tracking of Movements for a Gesture-Based Interface, a detailed paper by Daniel Hunnisett


Mouthesizer, which reads mouth positions (also from NIME 2003, demoed mapping to guitar effects)


Motion capture does promise to ultimately provide performance and expression without the aid of physical objects, using full-body motion as performance. Now go build something that lets you control your synth with your eyebrows, okay?