The good news: Microsoft is taking multi-touch, camera tracking, and gestural technologies seriously, and they have what looks like a very nice implementation that will be one of the first commercial implementations. The bad news: it’ll cost US$10,000 out of the gate. That high price will mean you’ll see at places like T-Mobile stores and Sheraton hotel lobbies first. But what you need to know: you can build your own version, thanks to available open source tools, with is likely to be more useful for music.

Good sources of commentary:
New Media Initiatives Blog at Walker Arts Center, which notes this could be museum-friendly tech.

Chris O’Shea @ Pixelsumo, who has built a device something like this himself.

The video does show what’s cool about Surface — and it’s easy to imagine these same techniques being applied to live visual and music performance. (People have already tried experiments in that, and I think there’s a lot more to be done — once you’re talking music rather than just digital snapshots, you get into deeper questions about how to model the interface.)

But let’s get a few things out of the way:
1. Enough about Minority Report, already! Minority Report was a terrific movie from an interface design perspective. (Slightly less so from a … well, movie perspective, but that’s another story.) But this has nothing to do with Minority Report‘s free-form, table-free, gestures in 3D space with gloves interface, unless all vaguely futuristic interfaces will now be compared to that movie. Actually, this thing is more practical, unless you want to share gloves with people at the Sheraton. (Ewwww … oh, and incidentally, there’s no logical reason why in the future we need gloves to do tracking.)

How about other comparisons? My apartment looks kinda like Blade Runner. This ATM is exactly like Lost in Space. If I start selling giant rubber costumes with fins, I can be the first to bring commercially-viable Doctor Who monster technology to a mainstream public. Oh, never mind.

2. “First commercial implementation” is sometimes meaningless. Since the dawn of time, pricey, first-to-market versions have often failed. Look at Apple: Lisa? Newton? Even the Mac initially lost out to the Apple II. That’s just one company. Now, there’s the additional reality of readily-available tools allowing people to take concepts like this and do whatever they want, freely experimenting without market restrictions. The commercial implementations may be more successful in that case — but the independent efforts could be sexier.

3. Tables take up space. Part of the reason you’ll be seeing this in hotel lobbies is that the tracking here presumably requires a large physical object in order to work, much like other interactive tables we’ve seen. If you want gestural or multi-touch technologies to be portable, or work with much smaller computer form factors, you need a different design. I’m still intrigued by what cameras may be able to do with enhanced computing resources.

None of this is intended to criticize Microsoft. I’m personally excited this stuff is catching on. I just want to make sure we remain tethered to the larger realities here.

That said, I’ll be eager to try out the demo version in New York in a couple of weeks, which just happens to coincide with a conference of people interested in musical interface design. Heck, maybe we can convince the Microsoft engineers to come over for a few drinks.

DIY Surface Computing

Be Your Own Bjork: If you want to build your own device like this, the reacTable software is available open source. It’s even available as a library for the free tool Processing, which is an ideal environment in which to learn how to code this stuff. reacTable is a little different from Surface, in that reacTable has objects on the surface to manipulate. That rules out some gestures you might make with your hands, but it does add additional tactile feedback, and, hey, playing with blocks is fun. Actual touch might be more difficult, though other computer vision tools that enable tracking are available.

Chris O’Shea (who comments on today’s announcement on his blog Pixelsumo) had also begun an open source software and hardware project using blocks on tables, called Sonic Forms. That project ultimately didn’t take off, but Chris has gone on to write about, speak publicly about, and do other wonderful things, like a table installation with music boxes.

Of course, my significant other doesn’t live in the world of blogs, and says, wisely, “but would you really want to do that?” That’s not a bad question to answer. It’s worth weighing this against other interface possibilities, and, for that matter, balancing the time you spend on the interface with the time you spend on music. More on both those topics soon.

And yes, obviously whoever shoots photos of these things doesn’t really get this concept. Here’s a hot tip to save you US$10,000:squeeze someone you love right now, huddle next to your computer in front of Google Maps, and you, too, can have “collaborative computing.” (Make sure they point meaningfully at the screen and laugh with delight at … the … directions … to the airport.) Since you can do this with a normal laptop, you don’t have to put a giant, clunky digital table in your living room. Which is good, because those coffee table books might screw up the tracking.

Computing couple

  • Nat

    Check out this site for DIY multitouch screens :
    http://www.nuigroup.com

  • http://www.jatunmusic.com dead_red_eyes

    Great writeup Peter. I saw this on Gizmodo today … and reading thru the comments gave me a freaking headache. Somehow they ended up getting in a Apply fanboy vs. Microsoft fanboy war. Ugh.

    I'm pretty excited to see this, and can only hope that there's not going to be a monopoly on this stuff in the future. Han has already been building these interfaces for big businesses and the military, and I can't help but wonder if he helped Microsoft with this. I think that the idea is really sweet tho, but I can't help but feel that I'd like one of these to be propped up at a 45 degree angle … but then you wouldn't be able to set stuff on it, which kinda defeats the purpose I guess.

    And there's some scare mongering to be had with this as well. 4 infared cameras in your living room? Talk about big brother / 1984.

    I REALLY want a Lemur so bad, and a reacTable … but there's just no way in hell that I can afford either of them. I can only hope that someday, this stuff will be less expensive.

  • http://www.createdigitalmusic.com Peter Kirn

    Right; I was equally frustrated … let's compare a tiny phone with touch sensitivity to a huge table using cameras and compare all of that to a movie using gloves in a system that's imaginary. Then, let's ignore the actual people who did the designs and see only the corporate entities (Apple, Microsoft) that paid for their research and own the patents, and turn the whole thing into a Mac vs. Windows war even though these particular products aren't even about those PC platforms.

    It's just utterly stupid. And do we really live in a world where technology isn't real to people until it has Apple's or Microsoft's name on it? I live on those two companies products myself, but surely that doesn't mean you have to be blind to everything else.

    The reality is, these are all dramatically different technologies, even if they all incorporate some kind of multi-touch. I think ultimately "multi-touch" will be as broad a category as "display." If you think about it, your QWERTY and MIDI keyboards are multi-touch. Multi-touch makes sense because we have hands with fingers on them. Actual implementation will probably vary wildly — and should.

    Now, people in comments say all kinds of silly things, of course. I just hope the larger tech blogs and writers will take a broader view and fight some of these misconceptions, to look beyond some of the novelty at the underlying tech and what it means.

    Oh, and I'm totally for Pac-Man on the Microsoft Surface. With multi-touch, you could have multiple Pac-Mans at once, or try to play all the ghosts — with a friend, it'd be like arm Twister. Namco?

  • Pingback: New Media Initiatives Blog » Microsoft Surface

  • http://ocp.pt.vu ocp

    I want a multitouch flexible screen : )

    http://www.youtube.com/watch?v=Rn10mRyBqAE

  • blueshifter

    "UGH!!! transferring photos from my camera to my phone is such a pain!! can't someone invent a table to make it easier?!?!?"

  • http://www.createdigitalmusic.com Peter Kirn

    blueshifter, you win the comments contest.

  • blueshifter

    <blockquote cite="there’s no logical reason why in the future we need gloves to do tracking.">

    I always assumed that was for the feedback, you want the interface to 'push back' to let you know you are moving something, or have reached a surface, etc. Man, that Minority Report was AWESOME!

  • http://www.createdigitalmusic.com Peter Kirn

    Oooh, actually, that's a really good idea — haptics in the gloves. I just figured it was a Hollywood thing so you understood there was an interface there. But all designs are in a sense fictional before they're real; there's always something to be learned from great design, imaginary or real.

  • nonstatic

    just for the record, microsoft did in fact internally develop this technology. they didn't just license it from someone who did the "real" work. they certainly weren't the first to think of it, but they haven't been claiming that. i worked on this project 3 years ago when they were demoing the technology internally but weren't really sure what it was FOR yet.

    people who are yapping about MS stealing/paying for the technology from someone else should get their facts straight. and besides, when i first took a look at this thing a few years back, it wasn't the multitouch aspect which blew me away, it was the object recognition.

  • http://www.createdigitalmusic.com Peter Kirn

    @nonstatic: Right, and unlike Apple, Microsoft does support significant amounts of research independent from product development. Apple, of course, used to have that with their larger research groups and advanced technology groups and whatnot, but ultimately decided it wasn't worth the expenditure and wasn't yielding better research. Not having worked in such a capacity, I can't comment on which system is better, if one is better; I expect that will remain in debate.

    But that wasn't my point — Microsoft is able to support researchers, internal or acquired, because they're a big, rich company. There's nothing wrong with that. What's wrong to me is the endless obsession of the blogosphere with the corporate entity at the cost of the research and technology itself. So, yes, let's absolutely talk about object recognition, etc., and the guts of the tech.

    When I say let's focus on the actual people, I mean the actual people at those companies. Corporate PR doesn't always let you talk to them or even find out who they are, but you get the idea. "Microsoft" doesn't do anything, because Microsoft isn't a person or a technology. So debating Microsoft vs. Apple *all the time* can miss the important details.

  • poopoo

    The ergonomics on this thing are pretty dreadful. If you thought hunching over a laptop for any amount of time was painfull, this thing will make that seem comfortable.

  • Pingback: Create Digital Music » More DIY Music Tables: MultiTouch Console, Built in Processing

  • Pingback: Create Digital Music » Interactive Table as Synth, Via New, Better Bjork Tour Vids; Microsoft Surface Snickering