Create Digital Music » Open-Source 3D Webcam MIDI Controller

Mar 15 2007

Open-Source 3D Webcam MIDI Controller

Interested in using webcams to translate on-screen motion to MIDI? Want x, y, and z 3D tracking? Ben Tan writes to let us know about his in-development software project called Peripheral MIDI Controller (pmidic) which does just that. The current build is still a work in progress, but has added enough stability and features that it should be worth a look.

Peripheral MIDI Controller

Grab your pen light and start waving it around for filter cutoff and resonance — whoo!

Right now, it’s Windows-only, but the libraries on which it’s built are cross-platform and could be ported to both Linux and Mac. (He’s using Intel’s OpenCV, which is the most popular, open platform for computer vision — odds are if you’ve seen slick webcam tricks, OpenCV was involved, because writing these algorithms from scratch requires a heavy-duty math and computer science background. And the app itself is built with the superb super-cross-platform wxWidgets library.)

If you’ve been looking to experiment with webcam control, this could be a good start — practicing interpreting the control data, experimenting with lighting, and experimenting with inputs all take time. And if you’re a coder, the project is open, so all of us C++ beginners can slog a little collectively.

Digidesign Public Beta for Structure Sampler

Hacked MIDI Support for Nintendo DS: DSerial

9 Comments

Ben T

Thanks Peter!
Just to let y’all know…that screenshot is from the previous release (Alpha). The current version (0.5) has a slightly better looking UI. I’ll put up some better screenshots on the pmidic website soon. Videos of pmidic in action can be expected soon as well :)

March 15, 2007 @ 7:05 pm

Peter Kirn

Well, the editor (cough, cough) is supposed to do a screen grab — I was away at my Mac. :) I’ll fire it up on the PC and give it a go and snap something myself.

March 15, 2007 @ 11:13 pm

Paolo V

It’s interesting!
Can you give me an advise for a webcam to buy? I need to check the specifications or all webcam is good for your software?
Thanks for your work Ben!

March 16, 2007 @ 3:54 am

Olivier Seb

Nice Application Ben, I hope we will see it on mac OSX very soon.
BTW, there is a similar tool for OSX:
http://web.mac.com/pierre.couprie/iWeb/Logiciels/midiWebcam.html

Midiwebcam is not Universal Binary but runs on my macbookPro.
Calibrating is not easy and I didn’t find the way to keep the settings each time I quit.
So there is room for a competitor ;-)

Olivier

March 16, 2007 @ 9:18 am

Peter Kirn

@phoban: Definitely check out EyesWeb, though I think it’s also based on OpenCV.

There’s also cv.jit for Jitter users:
http://www.iamas.ac.jp/~jovan02/cv

And JMyron for Java / Processing / Director / etc.:
http://webcamxtra.sourceforge.net/

Though neither of those is quite as extensive as the OpenCV-based stuff.

@Paolo: I’m still trying to find a reliable driver-free webcam that would work across platforms. See the discussion here:

http://createdigitalmotion.com/2006/11/02/creative-labs-live-optia-and-class-compliant-webcams-driver-free-on-windows-linux-and-mac-too/

March 16, 2007 @ 10:04 am

Paolo V

Thanks peter,
Creative Labs Live Cam Optia seem to be a good choice if it works without driver on XP sp2

March 16, 2007 @ 10:52 am

ConcenCrate

YAY! some food for the tech-nerd in me!

March 19, 2007 @ 9:23 pm

Gav Smith

This software is great. Although I’ve not yet managed to set it up as a midi input device. I want to use it to drive Absynth 8 and point the cam at glass box full of ants. I think ants would generate some interesting synth music.

then I’d like to point it at a lava lamp, then I’d like to find out how to translate the midi into data that I can set to the parallel port to a IO card and use it to control stepper motors that control a humanoid sculpture, which you could control with your body movement.

Then I’ll try using it as a weather monitor by pointing the cam at the sky and listening to the approaching weather fronts.

I’m sure I’ll think of some more ideas.

Cheers

Gav