“Augmented Reality” is a fancy term for describing ways of using computer vision to overlay digital intelligence on images. In other words, you can, for instance, scratch a vinyl record using a camera – plus a tag for identifying the object’s position in 3D space.

Cambridge-based designer Todd Vanderlin put together an elegant demonstration of the possibilities here, and his video has accordingly been making the rounds. (See: Synthtopia – and I actually heard about it this morning from a high school friend. The power of the Internet.)

Todd has more details on his site, which includes all kind of wonderful projects, like laser sound fountains and, always favorite around here, creepy circuit-bent baby dolls.

AR Scratching [Todd Vanderlin]

There’s actually some work to this: you need to figure out how the album is spinning. And of course, because this is augmented reality and not reality, there’s real potential here to imagine a new kind of vinyl DJing in which normal physics don’t apply.

From the video description:

I was playing around with some AR markers the other day and came up with this idea. taking just a plain old vinyl record and attaching an AR marker to the label you can track the record in 3D space. The next question was, can you scratch the record?

So by figuring out the velocity of the records rotation and applying it to the payback of the audio you can scratch. There is some digital noise that needs to bee worked out, but sounds pretty good. Its still really hard to scratch, it takes some practice but is super fun. The next step is to figure out some nice triggers for different modes. I like the idea of not needing a turntable but the actual spinning of the record helps with the scratching and playback. I made a couple modes, one where the record is paused and you can just scratch through the song. The other looks for zero velocity for x time and then continues on with the song. If there is velocity you then are scratching and the audio is affected. I think that this project has some legs can’t wait to play more.

I Want My Augmented Reality TV

So, this has sufficiently inspired you and you want more augmented reality? We’ve got more for you.

Digital artist and magician Marco Tempest has just demonstrated what happens when you do card tricks with augmented reality – and he shares some details of his rig:

Virtual Magic: Augmented Reality Card Tricks with Marco, OpenFrameWorks [Create Digital Motion]

We’ve even seen augmented reality climbing walls.

If you’re ready to do this yourself, we have a number of resources:

CDMotion has Andy Best’s tutorial on OpenCV with Processing (not an AR-specific library, but relevant): Getting Started, Popping Bubbles

We’ve also got an in-progress library for use with mapping projections in space for “spatial augmented reality”

A New Year’s video with a library for Flash, Java

A tutorial on getting started with augmented reality using Flash

openFrameworks, a library for C++ coding (which in turn supports multitouch, augmented reality – you’ll see some projects on that page)

Bryan Chung is working on a library for Processing

Auditorium is a fascinating free Flash game that turns interactive music arrangement into a series of puzzles. The center of the game is what the creators call “flow” – a visual stream of particles that can be directed to audio “containers” to create sound. The user places circles with icons signifying direction in the stream to redirect the particles where desired. As the stream hits the containers, it produces musical patterns. The results aren’t entirely open-ended – that is, there is a fairly fun puzzle game here, in that you can only “clear” a level by directing the flow of particles through all the objects. But the creators do claim that:

Auditorium is about the process of discovery and play. There are no right or wrong answers; there are many ways to solve every puzzle. To get started, fill up the first audio level.

playauditorium.com

The game so far is just a “demo”; the goal is to flesh out the game and deliver a more fully-functioning version. There’s a precedent for that: games like N+ (formerly N), Crayon Physics, World of Goo (formerly Tower of Goo), fl0w, and Da Blob – even the prototype for breakout hit Portal – began their life as free games or research prototypes before becoming officially-published titles, just to name a few. In fact, a significant chunk of what’s happening in game design these days is beginning its life in research and indie projects. That’s likely because hardcore and casual gamers alike are hungry for new concepts, and A-list developers are saddled with epic projects and bone-crunchingly huge ambitions and budgets. I’m not certain Auditorium will be the next big hit, but this kind of model could generate the interactive music games that future-minded music lovers have been anticipating.

My only criticism here, as with many similar games, is that the actual music content is fairly static. It’s a good prototype, but it’d be great if these particles had more influence on music. That creates a new problem for game designers: the music itself is really part of the game mechanics. Part of the fault here is Flash, whose sonic capabilities are fairly limited without a significant investment of effort. I’d love to see a game environment in which it’s easier to prototype musical ideas, with live-generated musical materials and synthesis.

The prototype here is promising, though. Found other interactive music games out there you like? Do let us know!

(Thanks, Brent!)

SoundCloud: The Tour from SoundCloud on Vimeo.

SoundCloud, an online sharing community for sound and music, is now hours from public launch. I’ve been playing around with a closed beta for several months, and have to say, I’ve been really impressed. SoundCloud isn’t the first attempt to provide places to share music files with others, but previous attempts have been lackluster when it comes to easy sharing, features necessary to make music listening more enjoyable, and upload capacity. Most importantly, none has accomplished the community “stickiness” that has been the cornerstone of successful media services like Flickr, Vimeo, and YouTube. In fact, there’s been so much of a noise-to-signal problem with the Web space, I expect a lot of you have simply tuned out new Web services. There are some good reasons to pay attention to SoundCloud, though:

• Singing telegram, anyone? Music on SoundCloud acts more like a messaging service. Tracks from people you’re following appear in an inbox for you to sort through. You can even create a DropBox for other people, so this could be huge for people running labels or live events. (That’s especially welcome now that a lot of people have given up on individually clicking myspace links to hear what someone sounds like.)
• Easier uploads and sharing: Getting files on the service in your favorite format, with whatever length you want, is a whole lot easier than on competing services.
• Smart player interaction: Services like this now live and die on their embeddable player. SoundCloud’s is really clever and attractive. Download links are everywhere. Also, SoundCloud attacks the biggest problem with music – it’s invisible. There’s a waveform view, and people can comment on specific points in your music. That feature has been annoying in a lot of video players, but here comments appear only if you want them to, and I’ve found them really helpful in getting feedback. (See my example track below, for instance.)
• Open API: A full API means you can built interesting apps atop SoundCloud. Check out the lovely Radioclouds by Matas Petrikas for an example of how interesting this can be; source code is available.

SoundCloud, indeed, seems to have all the kinds of features that made Flickr stand out from a crowd of photo services. Labels are already onboard, too: Compost, BPitch Control, and Goldie are already making it part of their workflow, says SoundCloud.

One very significant missing feature: it’s not yet possible to embed licenses in your music, so it’s not possible to tell what’s copyrighted and what’s Creative Commons-licensed. SoundCloud founder Alexander Ljung tells us this feature is coming very soon, though, and as on Flickr, you’ll be able to set a global default for your music, so if you want to release everything under a CC license, you can. That should make SoundCloud absolutely explode as a place for CC-licensed remixing.

Now, the only bad news: pro accounts are a bit pricey when compared to a service like Flickr, priced at EUR9-EUR59 each month. The EUR9/mo account is actually probably what most people will need: 15 tracks per month (that seems like plenty), plus basic stats, dropbox, and support. If you were a beta tester like me, you probably got a complimentary Light account through the beginning of next year. And importantly, there’s no limit on file sizes on any of the accounts. That means you could easily upload 15 live/DJ sets a month, so no complaint here. My guess is that the rates are a reflection of what it costs now in bandwidth for an ad-free site. And you can always go for a free account and see how the service works. But I do expect price to be the major obstacle to this service’s popularity. Update: comment to that effect in, what, five minutes of me posting?

Alex and the team say they’re CDM readers, and I’ll get to meet with them next week while I’m in Berlin. So if you’ve got any questions or feedback, let us know. I’d also love to see this stuff integrated more tightly in other communities; Facebook is there, for a start. Stay tuned.

SoundCloud

SoundCloud Tour

SoundCloud Blog

Here’s an example of my player (just an informal live improv set, not a full track). I need to go upload more content but plan to do that over the coming weeks. But it gives you a sense of how this works in action.

Here’s the upload interface, which I find quite usable. Note that you can also use SoundCloud for private files:

It’s Flash 909, and Amiga Flash.

Code wizard Andre Michelle has already made a name hacking audio capabilities into Adobe Flash and ActionScript 3. We got to see his work in the form of real-time audio effects processing in the GarageBand-like online sample-and-compose interface for Splice:

Interview: How Splice.com Has Taken Music Real Audio Processing to the Web

Well, there’s more, well into the “Things Adobe Wouldn’t Normally Expect People to Do With Flash” category. There’s 8BitBoy (warning: link autoplays music), a Flash-based player for Amiga MOD tracker tunes. There’s a 909 emulation (cutely named FL-909). There’s open ActionScript 3 source called popforge [@ Google Code] with all the Flash-hacking tricks needed to do audio.

Now, the most tantalizing bit yet: Andre has a new music environment coming, and to tease its arrival, he’s put up a little application with Roland emulations and stompboxes — and it’s all part of the Rich Internet Application of the Future:

The Hobnox Audio Teaser is a new approach to give you a great opportunity to create music [compositions] online in your browser. The underlying audio engine is based on dynamic digital signal processing to provide very complex audio compositing in Flash. The graphical user interface is based on an unlimited sized desktop to layout your chosen devices as you wish to operate.

The first application design delivers you some Roland emulations and effect floorboards to play with. Since this is a Teaser, you don’t have the option to save your environment. However, already at this early state we provide you with some well-known software features as history, clipboard and auto-alignment of the devices.

“Hobnox” will be about more than some TB-303 toys; it promises to be an “online entertainment and rich media publishing platform.” I assume we’ll find out just what that means later. For now, there is a limited beta sign-up with the audio application; it wasn’t quite online when I looked for it, but hopefully it’ll show up soon:

Hobnox Audio Teaser – Come and register for private beta

Hobnox.com

It’s not all sunshine and happiness. These are hacks, not fully-supported features of Flash — and Adobe, like so many developers of supposedly “Rich Internet Applications,” often doesn’t have a clue about sound. Andre has run into problems with his hacks for that reason. Now, I know what you’re saying — why should Adobe, a hugely-successful business, care about your MOD player? And of course, they shouldn’t. But that’s not the point. The vision of Rich Internet Applications is software that is expressive across media, and that includes sonic capabilities. Maybe people don’t care about the ability to play old Amiga tracker tunes — well, unless Timbaland is looking for some new tracks. But they might care when speech and creative music applications and interactive sonic interfaces come into play. We’ll never know for certain if the RIA developers keep ignoring audio completely, because then we lack the platform for people to do interesting stuff.

In the meantime, though, it’s comforting to have guys like Andre hacking in brilliant ideas. I hope Adobe takes note.

Thanks to Chris O’Shea for this link. Be sure to check out his all-new Pixelsumo blog.

Mark Nelson didn’t manage to make this his fifth portable flash recorder review in Hawaii, but he made up for it in depth. His review of the Zoom H2 is almost 5,000 words and contains surround-sound links galore as well as some nifty audio examples.

What’s especially cool was that it became a collaborative review after I asked readers what features they wanted us to test when the H2 finally shipped. They piled on with questions, driving my blog to #1 on the whole O’Reilly Network. At last count, I had close to 300 comments. One reader even wrote a Mac plugin to convert the H2’s quad recordings to 5.1.

So, there you have it: mobile recording geekery can have mass appeal!

Review

Pre-discussion

As readers look for the ideal mobile recording device, we’ve had ongoing, vibrant discussion about the Zoom H4. Its built-in stereo mic pair, real XLR jacks, and low price are big pluses — enough so that at least some are willing to overlook fidgety mic level settings. But one major problem could be a deal-breaker: an audible buzz in recordings.

Fortunately, it seems there’s a fix: if you don’t want to have to plug in the AC adapter all the time (which would defeat the purpose of a mobile recorder), make your own DIY battery pack. Zoom forums and RC car enthusiasts to the rescue!

Brad Linder, whom I met today at the NYC Jelly coworking session out in Brooklyn, is a blogger (Weblogs, Inc.) and NPR freelancer. Needless to say, NPR’s audio requirements will be higher than the typical amateur podcaster. Brad extensively documents his DIY RC battery pack.

Building an external battery pack for portable electronic devices [Brad Linder's Digital Home]

Good stuff, and now that I’ve just gotten my own H4, I’ll be trying this out myself.

A couple of other questions come to mind, though: have others had trouble using external mics (which is part of the point of getting an H4)? And, power gurus, I wonder if a LiPoly battery (Polymer Lithium Ion), as used in cell phones and sold via vendors like SparkFun Electronics, could do the trick? They’re not cheap, but the slim design is a big improvement over the Ghostbuster-esque design above. Thoughts?

Previously:
Zoom H4 Mobile Recording: Useful for Movie Production?

Zoom H2 Portable Flash Recorder Coming Soon; Mic Design Delay (with loads of comments from Zoom owners and folks who went with competitive devices like the Edirol mobile recorder)

For field recording, sampling, recording practices and performances, video production, and a lot of other purposes, just about everyone wants an ideal digital mobile recorder. If you haven’t been following comments, we’ve had an extended discussion by readers on the Zoom H4 mobile recorder, its upcoming smaller sibling the H2, and competitive devices like Edirol’s R09.

Now, the excellent new blog bleeps has had some hands-on time with the H4 in movie production:
10 reasons a Zoom H4 is handy on a movie set!

Interestingly, the main issue other readers have had with the H4 — difficulty accessing mic level controls — wasn’t really a problem in this application. The basic internal stereo mics did just fine for stereo imaging. And there’s no shortage of uses:

• Recording primary on-location sound in stereo (even with the built-in mics, though the H4 also has XLRs)
• On-the-fly foley / sound effects
• Ambient audio, room tone, etc.
• On-location audio playback
• Lots of storage for interviews, extra tracks
• It’s not a camera.

Be sure to check out the full writeup for more details.

“It’s not a camera” may seem an obvious point, but you might be surprised by how big an issue this is, even on larger productions. Video production staff have a nasty habit of trying to record everything to cameras. That means pricey cameras dedicated to nothing but recording audio, with non-standard batteries (the H4 does AA’s) and extremely limited storage on arcane storage formats. That and, of course, most cameras are lousy audio recorders.

Those of you doing on-location video production, what do you use? Some readers here are using higher-end recorders for audio from Marantz and others. What readers are not doing is fancy SMPTE sync to the cameras, because virtually no productions have the time or interest in fiddling with something like that.

In other news, I bought the bullet and bit an H4 — wait a minute. Reverse that. Anyway, I’ll let you know how it goes; I’ll be using it on location for a couple of things later this week I think you’ll enjoy. More soon.

Everyone is looking for the Zoom H2, a nifty portable flash recorder with a merciful street of around US$200 and a built-in mic. We know because you’ve all started hitting our ancient page on the H2. It in fact isn’t shipping yet; Sweetwater has an explanation on their site:

The Zoom H2 is expected to begin arriving in stores in August (slightly delayed by an improvement to the microphone design).

Just to demonstrate I’m not linking to them for affiliate cash, I’ll, erm, not link to them. But I expect you can find them (add a dot-com to their name) or try your favorite retailer and perhaps get a similar story.

It looks very nice, and I have to admit I’m myself looking for a new portable recorder. It seems like it’d actually be worth spending the extra US$100 and getting a Zoom H4, which adds XLR/line ins and phantom power for when you need them. (It’s also got four-tracking capability and effects, but the external mic in to me is the big draw.) Advice?

There’s such interest in this, I think we may need to make a portable recorder mega-page. Leave your suggestions here.

We broke the story on the H2 and deferred to our friends on O’Reilly on reviewing the H4. But you will find lots of reader comments (surprisingly, more H4 feedback on the H2 story — check out comments). Could help you make a smart purchasing decision; I’ll be using it to make mine:

Recording@NAMM: Zoom’s $199 USB H2 Mobile Mic/Recorder, Found in the Wild
O’Reilly Reviews Zoom H4 in Hawaii: Good, with Quirks

Okay, Akai. You’re going to have to get on the ball, fast. Sure, the new MPCs have all kinds of sophisticated music production capabilities. But can they remix the sultry, animal call of The Cheat, or make the ladies swoon with a solo by Strongbad?*

Onetwo, short video made with looping … uh … things [Homestarrunner.com]
Onetwomixer: MixMastah 800 for mixing it yourself

Yes, a lot of these Flash-based remix tools / mash-up gimmicks have been somewhat … limited … musically-speaking. But this is strangely fun. I dare you to use it live. (If they had only played up the “mash-up” angle, maybe they might have gotten mentioned in Wired. Sigh.)

*Oh, and Akai/MPC fans: I’m kidding.

Visual Acoustics is an online musical toy built in Flash designed by Alex Lampe (”Ample Interactive”) of the UK. (Via Music Thing.) The motion visuals are beautiful, and the music and interface is very reminiscent of Toshio Iwai’s work (see Nintendo’s ElectroPlankton, for instance). As with Iwai’s designs, just about anything you play will sound good and ambient. Now, there are two schools of thought on that. One suggests that these kind of futuristic interfaces make music accessible to anyone. The other would hold that part of what makes traditional musical instruments lovely is that, while they take a long time to learn, the rewards are much deeper. I’m not sure one is inherently better than the other, but I still wonder if it isn’t possible to build visual interfaces that are harder to master but deeper to play.

If you want some inspiration for moving in either direction, Visual Acoustics certainly shows potential. Now you just need a Wacom tablet-enabled version that, rather than conventional sliders for parameters, adjusts to gesture and pressure.