Therenect – Kinect Theremin from Martin Kaltenbrunner on Vimeo.

Who says technology has to move fast and die young? Leon Theremin may have been a full century ahead of his time, before computers, before transistors, before jet engines or atomic power or rockets.

ReacTable creator Martin Kaltenbrunner has a virtual Theremin prototype built with Microsoft’s depth-sensing, 3D Kinect camera. And what he really needs is some players of the real Theremin to help develop it. Martin writes CDM:

I just finished my first musical instrument based on the Kinect controller. The Therenect is a virtual Theremin, which defines two virtual antenna points that allow controlling the pitch and volume of a simple oscillator. The distance to these points can be controlled by freely moving the hand in three dimensions or by reshaping the hand, which allows gestures that are quite similar to playing an actual Theremin.

At the moment I am getting in contact with some trained Theremin players in order to tune the application to fully simulate the behavior of an actual Theremin. We will then publish some additional videos with a more musical experience … The software has been developed using the Open Frameworks and OpenKinect libraries and will be released under an open source license when it is more mature.

On our sister site Create Digital Motion, we’ve also noted that Martin’s new library allows OSC communication anywhere, so if a virtual Theremin isn’t inspiring your Kinect dreams, you can make something else.

Kinect with Anything: TUIO Gestures from Kinect

Kinect may be popular at the moment, but lest you feel rushed, just remember – a hundred years later, people still play the Theremin. So maybe if your idea is worthwhile, you’ve got some time. (Erm, not to enable any more procrastination.)

TUIO is a simple but powerful emerging protocol for multitouch control for live music and visuals, as used on the powerful live tangible synth reacTable. Apparently no one told Apple, however. While the App Store rubber-stamps useless toys like fake cigarette lighter flames, they bizarrely rejected a powerful application by a leading digital artist that would enable standardized TUIO control – for free. (More back story below; see an example in action above.)

As a blogger, my reaction is usually to whine and pontificate, for better or worse. The engineering approach would be to find some hack away the problem. That’s what Andrew Turley did with the TUIO protocol. So, Apple won’t allow an app that does the trick? Why not go back to what developers did before the SDK, and just use the iPhone browser?

As Andrew explains it:

After reading the story I started thinking about seeing how far one could push Safari as an application platform, using web apps to get around Apple’s tight control of the app store. Since you would be connecting to another computer anyway to use an OSC application, why not just have the app be a web app running on a web server somewhere on the local network? The web server can then take care of things like sending out OSC messages or playing music or doing whatever it is people want to do.

To that end I created a little system that implements the TUIO protocol. You use an iPhone to run a web app, which in turn talks to the web server, which in turn sends OSC messages.

Details:

touchy feely [Pillowsopher Blog]

Needless to say, there are some downsides: you have to run the Python server, you’re more limited in input and control than you would be in a real app, and you’re stuck inside the Safari browser, which could be a bit inconvenient. So I’m not backing down from my original complaint – I don’t see Apple doing anyone a service by blocking this kind of app, and the only rational explanation seems to be that the folks doing the review process don’t understand what the app is.

Most importantly, I do know that all our griping was sent to Apple, so I’m hopeful the App Store will reconsider the decision once they get it.

But I love solutions, too, and this can be one for many applications.

It also illustrates an important point: the browser on mobile devices (Apple and otherwise) could be a powerful outlet. It doesn’t always make sense to build an entire application; there will be various cases in which a little browser tool will do a job. Need a quick remote control for a live performance / art installation / club lighting rig? You might try the easy solution with the browser first.

Thanks for the great hack, Andy, and I’m curious to hear if anyone uses or extends this.

Previously:

Apple Rejects Free iPhone Tool For Artists Because of “Minimal User Functionality”

Since the dawn of computing, developers have been free to create whatever software they can imagine for computers. Windows, Mac, UNIX, Linux, Atari, Amiga, Apple II, Commodore 64 – it doesn’t matter. Come up with an idea, and short of doing something destructive on the system, you can make it work on a computer. It’s this freedom that has made the computer age possible. Game consoles have been different, a relic of the days when cartridges were physical objects you put in the machine. But mobile devices have generally acted more or less like computer platforms – look at Windows Mobile, Symbian, Linux, Android, Palm OS, Palm’s Web OS, and so on. It wouldn’t be odd to expect the same of the iPhone or iPod touch, which is essentially a Mac running on a low-power platform with a mobile-optimized set of libraries. The iPod doesn’t even connect to a wireless phone network; it uses WiFi just like your computer.

As musicians and artists, this sort of freedom has given us the freedom to make expressive music and art using powerful tools. That same freedom hasn’t applied to comparatively restrictive game platforms, which is why music apps for platforms like PSP and Nintendo DS require hacking hardware and software.

But then there’s the iPhone / iPod touch. Apple claims that they create a superior user experience by controlling quality, and they use that control to pick and choose which applications they think are appropriate for their phone. Never mind that a whole lot of what’s available on the iTunes store is simply worthless crap. And, frankly, that’s okay – users pick and choose the good stuff, and a lot of it’s really great.

But far from simply protecting mobile carriers like AT&T from abusive apps, it’s clear from developer experiences that Apple has extended that supposedly superior judgment to second-guessing developers on design and application purpose.

The latest victim: a fully free wireless multitouch server that would empower iPod touch and iPhone users to control live art and perform, created by one of the world’s leading interactive artists, Memo Akten. It seems Apple’s powers that be rejected the app because they simply don’t understand what the heck it is.

The story so far:

I received the following response from Apple: “Thank you for submitting your application “MSA Remote”. We need clarification in regards to the functionality of the application as we do not have access to the interactive installations that the application controls. If possible, please provide login information for a server application we can use to review your application.”

So I sent them 3 desktop apps: a standard TUIO client, a MSATouch client (multiple devices can control a single client without interference), and an OSC Dumper so they could see all the messages being sent in detail.

Even after receiving that documentation, Apple decided that they knew better than their users and developers. Result: no app.

Apple’s response to this app is “We’ve reviewed your application, MSA Remote, and we have determined that this application contains minimal user functionality and will not be appropriate for the App Store.”

I find it hard to believe that a FREE TUIO SERVER for iPhone and iPod touch is not suitable for the App Store!! Please leave comments below if you think it is suitable and I will get back to them to try and sort it out.

MSA Remote for iPhone

TUIO, for the record, is the open multitouch platform employed in the powerful, expressive reacTable tangible interface that has been used by the likes of (Mac user) Bjork.

Let’s be clear. The point here is not just to rant. Memo is looking for comments from users. I’m hopeful that an upswell of support could show Apple the error of their ways and get them to correct course on this one, and I’ll applaud them if they do that.

But let’s also be clear: restrictive platforms are bad for artists. Apple is setting a dangerous precedent, and I’m frankly tired of the conventional assumption that they’re always right. I think the restrictiveness of the platform – well beyond what is “safe” for users or what might endanger Apple’s relationship with carriers like AT&T – is simply wrong-headed. The reason we love platforms like the Mac is that they have empowered us to express ourselves freely. And having seen the power of the Mac as a platform over the years, I’d be disingenuous not to point out that the iPhone has lost a big part of that soul.

The good news is, complaining and whining and griping can be productive. Tweetie, a powerful Twitter app that was censored because it might expose users to profanity on the open Web service, did make its way to the store after massive public outcry.

So, as I say, I don’t rant just because I like the sound of my own typing. I hope that the rest of you will join in, and it’ll make a difference.