Perhaps you’ve seen the demo videos, as people do astounding things by moving their body around and using the Kinect camera to make music. Now, a set of Max for Live devices makes it reasonably easy to access your body as input inside Ableton Live.

FaceOSC Mapper

Pictured at top, this builds on Kinect superstar coder Kyle McDonald’s face-tracking tool and lets you use your face – position and even facial movements – to control Ableton Live parameters.

Kinect Camera

For use with the V-Module and vizzABLE systems, you can plug in one or more Kinect cameras, and get tilt, distance filtering (to remove backgrounds), depth maps, RGB and IR modes, and plug in your depth-sensing camera for more goodness.

Kinect – OSCeleton

The home run: look at skeletal tracking for extremely precise human control of parameters, as seen in the video. It only gives your left and right hand, but stay tuned for further developments. See also this example patch.

Example video shows how to “track hand positions and translate to volume and send levels.” Not impressed? Remember, it’s a proof of concept: you can assign to other parameters, practice your movements, change the musical content, and even modify the patch to make it work better.

It’s Electronic Body Music! (Sorry, couldn’t resist. Kids, ask your … parents jeez we’re getting old, aren’t we?)

The tenth anniversary of the iPod debut means you’ll find plenty of commentaries on Apple’s iPod and how it has changed music. It’s an issue that’s been talked to death enough, continuously, in the past ten years that I’m literally uncertain there’s more I can say about it. Here’s one good, compact commentary from Daring Fireball, inspired by Macworld’s sharp review from the 2001 debut of the hardware.

Instead, let’s consider what hasn’t happened: Apple hasn’t discontinued the standalone iPod, as distinct from the iPad and iPhone and other general devices. For music lovers, that’s a big deal. The sad news is, the category itself has all but entirely imploded.

The last ten years has been in almost every category a kind of battle between dedicated devices and convergence devices. Anecdotally and statistically, you’ve seen people abandon dedicated video cameras, still cameras, audio recording gadgets, and audio players for something like their iPhone. Little wonder: unless you have enormous pockets, if the integrated device does the job – and its battery doesn’t give out – it means something that’s always at the ready.

Apple’s legacy in music players is curious: they both defined the category, and wiped out all the competition. And that’s true even before Apple changed the category again with the iPhone. That’s not the normal pattern: typically, in electronics or any other tech, the pioneer defines a space in which other competitors come and play. Not so with the iPod: a combination of shifting consumer trends, the profound success of the iTunes “ecosystem,” and the general ineptness of competitors to make quality, differentiated alternatives has led to the iPod standing more or less alone. The iTunes issue shouldn’t be overlooked: recall that when the iPod launched, record labels were still concerned about copy protection. The result was an iTunes-iPod relationship that ultimately kept consumers from working out the complexities of moving their music library to another, rival player. (The fact that most of the rival players weren’t any good didn’t help, so we can’t ever really know how much of a factor this was.)

Two things have happened this fall. Microsoft did discontinue the Zune, in what seems the final death knell for any major dedicated music player that isn’t made by Apple:
Microsoft confirms Zune HD is dead

But, secondly, even as various analysts predicted Apple would kill the dedicated iPod players or even the iPhone-with-no-phone iPod touch, Apple didn’t discontinue anything.

My favorite player remains Apple’s iPod Classic. It’s beautifully designed, holds an absurd amount of music no phone can match (160 GB), and has a simple, clean interface for getting to your music. It’s sad to me only that it’s the only choice, particularly because the one thing rivals did have going for them was easier, more open sync rather than iTunes-only solutions. In fact, even the original iPod had as a major selling point the ability to work as a dedicated hard drive. As a purchaser of the first iPod, one of my favorite features was the ability to easily tote around a big file or two atop the music library.

Oh, yeah, and it’ll still run when your phone battery is dying, and it costs just US$249 – no phone contract required. Ahem.

http://www.apple.com/ipodclassic/

Phones as playback devices are pretty great. But remember that the original dream of the iPod was something different: it was the ability to put your whole music library on one device and take it anywhere. My main question is how that legacy will pan out. Dedicated music devices give you distraction-free access to nothing but music, and ongoing storage innovations mean that something that’s just a music device may long exceed what the convergence devices can do, surviving for the reason SLR cameras do.

Apple’s iPod series will last so long as people keep buying them; Apple seems in no hurry to walk away from extra revenue. (It’s part of the reason why they’ve got all that cash, folks.) But I wonder in the long term what will happen to the category. To me, the major gaping hole is something a lot of us wanted even when we saw the first iPod: a dedicated, pro-quality music player, a kind of audiophile iPod. It doesn’t need any fancy features or silly gold-plated jacks, just something dedicated to playing music and nothing else. I wonder if we’ll ever see that, or if it’ll be another casualty of the explosion in consumer gadgets. In the meantime, long live the iPod Classic.

And for the record, if you do have an original iPod from ten years ago, you can still make it sing: install Linux and it’ll even run Pd.

The gaming industry has made their bet, and it’s that touchscreens go better with tactile controls. Might digital musicians reach the same conclusion?

A funny thing has happened on the way to the touch era. The vision of a device like the iPad is minimalist to the extreme: an uninterrupted, impossibly-slim metal slate, as impenetrable as some sort of found alien scifi object. The notion is that by reducing physical controls, the software itself comes to the fore. It’s beautiful conceptually … and then you find yourself tapping and stroking a piece of undifferentiated glass. For navigating interfaces – and even, I’d argue, exploring sound design and composition – it works brilliantly. But for live digital performance (what to game lovers is called “gaming”), for anything that wants tactile feedback, it can be imprecise or unsatisfying, or both.

Watching this shake out as a design problem is fascinating, especially coming from the perspective of music. Digital musicians were exploring alternative interfaces since before it was cool. Given the ability to make any sound we can possibly imagine, the question of how you design an interface around sound is compositional, philosophical, essential.

Whatever winds up working in the marketplace, there are some fascinating ideas for combining touch with tactile. Since both are good at certain tasks, why not do both?

We’ve seen several examples among musicians and researchers exploring how to augment the touchscreen with physical input:

Mike Kneupfel’s research at NYU’s ITP program, in the video at top, investigates adding additional inputs. See: Extending the Touchscreen.

We saw that kind of extensibility in an iPad dock concept by Livid Instruments.

While it lacks additional tangible controls, I/O extensibility is featured in a still-as-yet-unreleased “pro” dock by Alesis, and most recently in a DIY dock by circuit bending pioneer Reed Ghazala.

Now, game vendors are moving in the same direction – even with prototypes that look quite a lot like the research project above. (Sometimes, arriving at the obvious conclusion is necessary for a great design.)

Sony’s PlayStation Vita, successor to the PSP mobile game platform, augments touch input with tactile controls in much the same way as Michael Knuepfel’s work does. Notably, it also proposes how these inputs can coexist in a form factor that’s larger than a phone, but smaller than a tablet – scaled roughly to a comfortable holding distance between your two hands. (Microsoft and Apple each unveiled standard split keyboards on Windows 8 and iOS 5, respectively. The era of thumb ergonomics is now fully underway.)

Nintendo’s Wii U controller combines a lot of sensing capabilities into one device. Like Sony’s effort, the centerpiece is the combination of the interactive touch display with analog controls. But true to its Wii heritage, Nintendo is packing other sensing technology, too. While its evolution has been more piecemeal, the same is true of the Xbox 360 in the Kinect era. The Kinect camera is really a bundle of mic and stereoscopic camera sensing with software intelligence for motion analysis and even speech analysis via a variety of methods. While Kinect is touchless, the conventional gamepad still plays a role.

What’s the relevance of all of this evolution to music? Digital music’s demands parallel gaming, requiring precision, accessibility, scalability from beginners to hardcore experts, and real-time interaction. Also, music research has often been at the forefront of experimentation with a variety of means of translating sensory data to expression. And since musical practice itself is roughly as old in human evolution as language, if not older, it’s a key way of glimpsing how ubiquitous interfaces can become meaningful.

Let me put that another way: the stuff game companies are doing now looks a heck of a lot like what computer musicians have been doing for years.

While much of the acclaim for platforms like the iPad has been for their transparency and unadorned interfaces – and while I believe those are valuable concepts – bundles of capabilities for interacting with the world can become powerful. That means efforts like Apple’s addition of USB MIDI connectivity to the iPad, or Google’s nascent work to standardize USB host mode and open hardware development (based on Arduino), take on new meaning. Add to this additional connectivity via Bluetooth and wifi, and it may be that we only really see what these platforms do when, like the PC, they start geting sociable with a range of other gear.

This could also mean that communities like the music community have a chance to prove that the “post-PC era” is a little different than it’s been described in the mainstream press – and maybe a little less a radical departure. The “post PC era,” we’re told, is less about being a hub for a lot of hardware. But as people look for tactile feedback, some of the coolest applications of these platforms may not be in the mainstream use as “consumption” devices, but at the fringe.

I’ve just come from the launch of the Samsung Galaxy Tab 10.1 in New York. You’re not missing much; there were a handful of people snapping up the tablets. (I think the 10.1, and a few other Honeycomb-based tablets, do have a bright future, though their growth may be a bit slow at first as developers get their hands on them and give people a reason to buy them.)

What was most compelling to people at the launch, though, was a planned appearance by pop star Ne-Yo (at least according to some staffers to whom I spoke).

But the connection was, at best, tenuous. It may be when devices like these tablets are made more viable for musicians onstage that that connection starts to make sense. And that may mean that Apple and Google/Android vendors alike need to start to think more aggressively about the larger ecosystem and hardware applications. Remember all those futuristic promises from Apple about hardware accessories? Right now, the most significant hardware is the Square payment add-on, and it uses a hack to make it work through the audio jack. Both Apple and Google can do more work to open up hardware development.

It’s all well and good for the tablet to be a “post PC” device, to be different from PCs, to be better. But they may simultaneously need some of the openness to other gadgets that made the PC age so revolutionary.

If your music and sound collections seem like outwardly-expanding universes, two new tools promise to bring order by representing media as virtual planets and stars. One works on albums and tracks on the iPad; the other uses computer-aided analysis of loops and samples (not just music) on Windows. One will make your eyeballs pop; one might help you manage gigs of samples for a game design project.

Built in the open-source framework Cinder by an all-star team of media artist-designers (Ben Cerveny, Tom Carden, Jesper Sparre Andersen and Robert Hodgin), Planetary should satisfy space nuts and eye candy lovers. The metaphor is pretty direct: artists are stars, albums are planets around the artists, tracks are moons around the planets, and you can filter “constellations” by letter. That means the actual structure is heavily hierarchical, actually, in the tradition of iTunes (and, before it, its predecessor SoundJam). I’m not sure what happens with, say, compilations. But let’s face it: the real draw is that it’s incredibly beautiful to look at. I’d be just as entertained looking at a visualization of my system folder if it looked this pretty.

For now, Planetary is some fascinating eye candy with at least basic playback capabilities, iPad-only. That brings some good news – Airplay wireless works, and since it makes use of standard media code, even features like Last.fm scrobbles function. It also brings some bad — while Apple added support for libraries to third-party apps, Home Sharing isn’t included, so you’re limited to what’s on your iPad. Playlists aren’t supported, either. But hook this up to a projector or large screen TV with some of your favorite music, and I don’t think you’ll be complaining. And as a free tool, it’s incredible.

Planetary is available now; free for the iPad. As seen on creativeapplications.
http://planetary.bloom.io/
iTunes link

Less pretty, but with greater facilities on the utility side, is the Windows-only Soundtorch. (Thanks to Kristian Gohlke for the tip!) Visually, it offers a similar metaphor: media assets live on a continuous plane. Functionally, though, it’s more algorithmic than hierarchic, using something called the Computer Aided Sound Exploration engine (C.A.S.E.). The set of algorithms, which the creators say were based on evaluation of human listening, performs a sophisticated set of extractions of some 600 features from each sound file.

Rather than limit itself to albums and tracks, C.A.S.E. is tuned for audio files and loops. It’s fast enough that it can plow quickly through gigs of material. So, if you’re on Windows and have amassed an enormous collection of loops, samples, field recordings, sound effects, and the like, Soundtorch will use C.A.S.E. to first map all those relationship, then visualize them. You can use the mouse to produce new collections of assets, map relationships visually, export those relationship to XML, copy sounds to the clipboard, export to WAV, or open them in Windows Explorer. That is, all that eye candy is a genuine interface, not a barrier between you and what you might do (as so often happens with these sorts of experimental interfaces).

In fact, you might argue that, despite outward appearances, Soundtorch is entirely different from Planetary, but they share one common conceptual assumption. Related media “orbit” or attract to common materials. The difference is that Soundtorch is relational. In Soundtorch, if you “magnetize” a file, it – and any similar files – become attracted to attractors called “magnets.”

As is appropriate searching for media, the “torchlight” metaphor shines a light through files. Everything under the light plays back simultaneously, so you don’t have to audition sounds one at a time. (That sounds slightly terrifying to me, but I have to spend more time with it in an actual library.)

The creators describe the magic thusly:

Have you ever listened to a sound and felt that there was a similar one somewhere on your hard disk? And the sound you can’t find would just work so much better right now? Well, Soundtorch also remembers all sounds that you ever listened to. Just select any sound on Soundtorch, and let the system suggest the most similar ones from your whole collection.

In other words, SoundTorch is as much about what you can’t see as what you can – the intelligence to determine similarity behind the scenes. Check out the tech talk in the video above for more information on how “aurally and visually-enhanced audio search” could also apply this technology. More research at:
http://www.accessive-tools.com/

Soundtorch 2.0 entered a free public beta last week. It was developed in Microsoft’s C#-based XNA framework.

Grab the download:
http://soundtorch.com

Finally, if you want to hear the “Optimist” track by Zoe Keating without that voiceover and just enjoy Planetary’s gorgeous visuals, here you go:

From innovation in the visual interface to the intelligence underneath that changes how the computer interprets relationships between files, finally, there’s hope. Music and sound might not forever be trapped in views borrowed from spreadsheets, tables modeled on the needs of accountants 30 years ago.

Therenect – Kinect Theremin from Martin Kaltenbrunner on Vimeo.

Who says technology has to move fast and die young? Leon Theremin may have been a full century ahead of his time, before computers, before transistors, before jet engines or atomic power or rockets.

ReacTable creator Martin Kaltenbrunner has a virtual Theremin prototype built with Microsoft’s depth-sensing, 3D Kinect camera. And what he really needs is some players of the real Theremin to help develop it. Martin writes CDM:

I just finished my first musical instrument based on the Kinect controller. The Therenect is a virtual Theremin, which defines two virtual antenna points that allow controlling the pitch and volume of a simple oscillator. The distance to these points can be controlled by freely moving the hand in three dimensions or by reshaping the hand, which allows gestures that are quite similar to playing an actual Theremin.

At the moment I am getting in contact with some trained Theremin players in order to tune the application to fully simulate the behavior of an actual Theremin. We will then publish some additional videos with a more musical experience … The software has been developed using the Open Frameworks and OpenKinect libraries and will be released under an open source license when it is more mature.

On our sister site Create Digital Motion, we’ve also noted that Martin’s new library allows OSC communication anywhere, so if a virtual Theremin isn’t inspiring your Kinect dreams, you can make something else.

Kinect with Anything: TUIO Gestures from Kinect

Kinect may be popular at the moment, but lest you feel rushed, just remember – a hundred years later, people still play the Theremin. So maybe if your idea is worthwhile, you’ve got some time. (Erm, not to enable any more procrastination.)

Ben X Tan writes to let us know he’s working with hacks for Microsoft’s Kinect 3D camera system for Xbox to perform MIDI control. Result: depth-sensing, gestural musical manipulations! It’s just a prototype, but since today I cover the larger landscape of what’s happening with Kinect, it’s well worth teasing. From the description:

Coded in C#.net using this: http://codelaboratories.com/nui

Very hacky ugly, yucky, alpha prototype, source code available here:
http://benxtan.com/temp/pmidickinect.zip

Next project is making a version of pmidic that uses Kinect. Then, you can control Ableton Live or any other MIDI software or hardware with you limbs. Isn’t that amazing!!!

If you are interested, you should also check out:
http://pmidic.sourceforge.net/
http://benxtan.com

He tells me, “It’s just proof of concept for now, but want to make something nicer in the long run The community on IRC for this stuff is great. Very supportive. I love all the hacking going on at the moment. Especially after watching the Social Network. Very inspiring!”

So, what’s this whole Kinect thing about, why should you care, why might it be useful to artists and musicians and designers generally, and where do you go to find the code? I’ve rounded up various hackers working on the project to answer those questions on Create Digital Motion (in this case, Capture Digital Motion):
Kinect Hacking and Art Round Table: Why it Matters, What You Need to Know

Games really are reshaping music. Despite their relatively simple gameplay, the Guitar Hero and Rock Band franchises originated by developer Harmonix are stimulating interest in real music making. It’s no accident that you can walk into a Best Buy and, next to aisles of video games, find a growing selection of serious musical instruments and technology.

These titles are also stimulating interest in music and artists and producing a new distribution outlet, at a time when the distribution picture for music can seem bleak. But until now, that outlet has been limited to big acts, big tracks, and big deals with big labels. It has only promoted music you already know, not the discovery of new music. Rock Band Network could change all that.

We took a detailed look in August at how Rock Band Network worked technically, and how authoring a song for RBN could give you the same level of gameplay and choreographed graphics that the official Rock Band tracks get. But now here’s the big news: at long last, RBN is opening to the general public, starting with an open beta for artists and play-testers.

What it is: Rock Band Network is a new set of authoring tools (built around Reaper), a submission process (built around Microsoft’s Xbox 360 XNA Ceators Club), and an upcoming store to host indie tracks called the Rock Band Network Music Store.

What it costs: Rock Band Network membership is free, but you’ll need a $99/year XNA Creators’ Club Premium account to submit or test music.

What you’ll need: To author titles, you need an Xbox 360, a copy of the Reaper software, a set of free plug-ins for Reaper for RBN, the XNA account, and either a Windows PC or Mac. (You’ll need Windows, either virtualized or on another machine, in order to actually load the tracks for testing, but you can author on either; see below for more.)

What it gets you (as an artist): If you make it through the peer-reviewed submission process, you stand to set your own pricing and receive 30% royalties (retail, excluding tax) on everything you sell.

What it gets you (as a peer reviewer): With the XNA Creators’ Club membership, you can play as many tracks as you want without any additional charge, in exchange for your feedback. Tired: squeezing into sweaty, overcrowded bars at CMJ and South by Southwest to hear new acts. Wired: Scouting for new acts on your cough with your Xbox 360. And that could make a nice community of music, depending on how this evolves.

Where the tracks will be distributed: Anyone with a copy of Rock Band 2 (and presumably future versions of Rock Band) can play your tracks. Releases will initially debut on the Xbox 360 store for 30 days. A “selection” of tracks will also appear on the PS3 and Wii stores after that. (The approved songs will stay on the RBN Store on Xbox 360, regardless.)

When does all of this happen? The open beta launches today for peer reviewers and artists. The store is due, um, “real soon now.” (No specific date yet.) The game itself is ready to go, at least on Xbox 360: a patch introduced way back in September added the ability to play RBN tracks.

CDM Talks to Harmonix

John Drake, Program Manager for Rock Band Network, took some time out to answer my questions on the eve of launch.

CDM: What will the Rock Band Network Store look like? Where will you get access to it? Will it be a similar store on the PS3 and Wii?

John: The RBN store will run in parallel to the existing Harmonix DLC store, and will be in the same menu location within Rock Band 2. The RBN store has more info about each song than our existing DLC store does, and it has more ways to discover new music: you can search by subgenre, album, country of origin, record label, even the author of the song.

The PS3 store will be very similar to the Xbox 360 store. Details of the Wii RBN presence are still being worked out.

Ed. note: It’s especially nice to see the RBN store on equal footing. I had high hopes for the XNA-produced games on Xbox Live, but those titles aren’t displayed or listed in exactly the same way, which I think has hurt the initiative a bit.

CDM: In addition to the XNA Premium subscription, you still need Windows to support testing your own tracks, yes? Do you need a Windows PC to be a playtester?

John: You need to run Windows in order to transfer song files to the Xbox 360, because we use Games for Windows Live to manage the transfer. We have informally tested running Windows on a Mac on a number of virtual machines, as well as BootCamp, and most of them work perfectly for transferring files.

Ed.: I can add, a number of the Harmonix guys are Mac fans, so you can believe they tried the virtualization approach!

CDM: Since we last talked, there has been a private beta. Were there any additional improvements / changes since our August conversation? What kind of feedback have you gotten?

John: The closed beta has been absolutely invaluable to help us shape the experience for the new members just now joining the program. We’ve cleaned up and organized the documents section of the website, added a great deal of new information, clarified policies for submitting songs, and generally made sure that the pipeline is running smoothly. None of the major processes are any different than initially designed, but we have changed a million small details to make it better.

It’s important to note that the members that have been in the beta have been absolutely extraordinary: patient, intelligent, hard working, thoughtful, and helpful to each other as they worked through the inevitable issues that cropped up as we readied the site for launch.

CDM: Have any currently-available tracks come through the private beta process? (Jonathan Coulton’s?)

John: We currently have nearly 40 approved tracks, including tracks by the inimitable JoCo, and a bunch more up for playtesting and peer review. We’re expecting even more great content to go up for testing in the next few days, and we’re excited for people to join our playtesting ranks to get even more songs through the pipeline!

CDM: I see TuneCore is offering track preparation services. Have you seen similar offerings begin to appear? (For some of us, doing the authoring may actually be satisfying – we’re weird that way!)

John: There’s a great variety of services cropping up from authoring houses offering with different programs to create songs for bands. These range from straight, up-front fee structures to a $0 down, pay us out of your royalties deal. It’s really exciting to see how different groups are responding!

*PS, I’m with you on the satisfaction of authoring. I’ve been working with my band to put our whole last and current record (17 songs in total) up for RBN. It’s a lot of work, but it’s super rewarding to get involved in the process! And it’s really doable if you’re used to making music as a passion!

CDM: Outside RBN, are these tools beginning to be used on Harmonix’s own tracks? (I believe that was in the works when we last spoke.)

John: It was always the intention that the tools we developed for the Rock Band Network would be integrated internally at Harmonix and that has begun to happen. With the industry leading amount of content we produce (over 1000 songs and counting) anything that makes the job of our unparalleled Audio Team easier is welcome, and in most cases the Rock Band specific tools were built by members of the Audio Team themselves!

CDM: Okay, enough of the nit-picky details… what’s it mean for you that you finally get to take this to public beta? Now with a few months more perspective on it, what do you think this will mean for musicians to get on this platform, revenue aside?

John: As our Senior Producer Matthew Nordhaus said about Rock Band Network, “It completes me.” We’re already thrilled with the community working within RBN and we’re hopeful to see a lot more great content and enthusiastic playtesters signing up at Creators.RockBand.com now that we’re open!

Additionally, we’re really proud of our teams here at Harmonix and MTV Games, who have designed a really smart way of getting great music into the hands of fans. Empowering musical groups of all sizes and genres to be able to post their own content for sale is really a dream come true at Harmonix. Adding the great variety of music for our passionate fanbase only makes it that much sweeter. We’ll be even more excited when the store turns on and those first tracks sell!

Go Check it Out

I hope to help document both how artists are using RBN and the technical process for doing this yourself over the coming weeks. In the meantime, you can hop on the beta yourself if you’re interested:

How to submit a song: http://creators.rockband.com/docs/Website
Scroll down to “Adding a song to the pipeline.”

How to become a peer reviewer?
http://creators.rockband.com/docs/Playtest_Process

And yes, I still want to see an Amplitude/Frequency Network that’s friendly to electronic music, minus drums + guitar. I think Harmonix knows a few of us feel that way.

Jonathan Coulton on Rock Band Network, from the awesome PAX.

PAX ’09 Rock Band Network Panel #3 from Harmonix on Vimeo.

Everything you need:
http://Creators.RockBand.com

Video interview by G4:

There’s a lot of hype around the latest schemes for changing how artists get their music to fans, but not actually a whole lot of news. (It always seems to boil down to a website with some unpronounceable name.)

Well, this is news: Harmonix is opening up Rock Band to anyone who wants their music in it, and giving you the same sophistication of tools they use themselves. That’s a real game-changer – literally.

And I don’t mean just for the actual game Rock Band. Sure, Harmonix was the house that made music games a phenomenon in the US. They learned well from Japan’s Masaya Matsuura, perfected music games’ mechanics in Amplitude and Frequency, popularized the formula by launching Guitar Hero, then rocked collaboration with Rock Band before convincing the infamously-guarded Beatles to finally embrace digital tech. But the sad reality of game music in general is that it’s been a playing field for the old guard – it’s licensing deals with major labels to promote music you’ve already heard. It’s the top hits on the radio, redigested onto your game console. There’s commercial calculation behind even the tune that’s in the background while you’re paging through a screen in Madden. Harmonix has already changed some of the economics, and disrupted even what could be a hit, as kids discover classic metal for the first time or geeks grab music by Jonathan Coulton and Stephen Colbert. But that’s not quite the disruptive shift in game music so many people have expected.

I think Rock Band Network could be the first real sign of that shift.

Rock Band Network promises to be something really different. How?

• Anyone can get their music in the game. You don’t even need a label. You need a few (cheap) software tools, a computer, and some basic MIDI chops, and for a fraction of the cost of pressing a couple hundred CDs, any artist can get their work into Rock Band 2.
• It’s a real community-driven process. Your A&R people don’t have to shmooze with MTV. You don’t have to enter into some complex developer agreement with Microsoft or Sony. There isn’t even a shady, mysterious review process like the Apple iTunes App Store. Actual Rock Band fans will get to play your music and tell you that the animation needs fixing and the difficulty level needs to be fixed on the drums.
• You use Reaper – an actual music production tool for grown-ups. Harmonix could have given us some weird in-game tool they cobbled together themselves. Instead, they give us a special verison of Reaper, the brilliant, full-blown Digital Audio Workstation that inexplicably costs just US$60 but blows the pants off a lot of better-known tools. So you actually get to assemble your music the way Harmonix has been doing for years, with a real tool. Fortunately, the process has been made much easier and copiously documented, but it’s nice to be treated like adults for a change.
• If it works, Rock Band is just the beginning. It’s impossible to see into the future. RBN is a leap of faith both in the artists and the game fans, in terms of their taste and the amount of effort they’ll invest. But if it works, Rock Band Network could change the way people think about interactive user-created content, well beyond just furniture in the Sims or Little Big Planet.

Anyway, enough of the big picture – let’s talk details. I got to sit down with the Rock Band Network team from Harmonix high above Times Square in MTV’s offices this week to get a full-blown demo – including some seriously fun nerding out with composer/sound designer Caleb Epps, plus Senior Producer Matthew Nordhaus and MTV’s games man, Paul DeGooyer. (In a sign that the big media world still doesn’t quite get what’s going on in this field, no one at the Viacom security desk had even heard of Harmonix.)

The team was extremely generous with technical details of Rock Band Network, and walked me through the process of how artists would get going with RBN. Here’s a first look at that process.

What You Need to Get Started

$60 Reaper + free plug-ins + a computer + Windows to beam over the music + an Xbox 360 to test on.

1. Reaper (Mac, Windows) For the authoring itself, you may be surprised: you don’t need some special tool. You use Cockos’ brilliant, lightweight, Reaper. It’s not even Reaper Rock Band Edition. Reaper for Mac will work, too. Cost: US$60 for the standard license, or US$225 if you’re already a huge rockstar.

http://www.reaper.fm/

2. Reaper plug-ins (Mac, Windows) Reaper plug-ins: this download is the real magic, adding everything from shortcuts for making tempo maps to color-coding tracks to helping you add lyrics, animations, and everything else that makes your song into a Rock Band track… game. Gamesong? Songgame? Cost: Free.

3. MAGMA Packaging Tool (Windows) MAGMA is a simple tool that facilitates getting those files packaged up with artwork and keywords and such, and moving them over to the Xbox 360 for testing yourself and for sharing with the rest of the community. It is Windows-only because it relies on Microsoft’s networking functionality with the console, but Harmonix says they’ve had no problem using it on the Mac via an emulator or Boot Camp. Cost: Free. (or the cost of Windows if you’re on the Mac).

4. Xbox Creators’ Club Membership: Join Microsoft’s game development community, and you get access to a special Rock Band creators area that lets you upload and share your tracks – and other tracks from other users (which is where item #5 comes in). Cost: $99 /year (Note that there are some discounted ways to get at this for shorter terms, and you get all the game developing features of the community, too, in case you want to try to make your own game in XNA.)

5. An Xbox 360 and Rock Band 2: You do want to actually play the results, right? (Unfortunately, because of the reliance on Creators Club, Sony’s PS3 isn’t yet supported, though some sort of PS3 distribution is planned for the future.) Cost: About to come down thanks to sales – and now you get to write off an Xbox 360 on your taxes.

Total cost: as little as $100-160 or so with the various pieces, or a little more if you need to pick up an Xbox 360 and the game and/or equip your Mac to run Windows.

By the way, Ars Technica claimed this month, based on the experience of one developer, that Xbox Live [is] not ready for user-generated content. That claim is simply wrong. Sure, LittleBigPlanet is cool on PS3, but the infrastructure for moderating content is there, on the community created for the XNA game development platform. And the tracks for Trails HD (the game mentioned in that article) or even LittleBigPlanet really pale in comparison to what Harmonix is about to unleash. It’s the first time a game has really been a platform, which was long the vision of Harmonix’s founders.

Now, let’s get into actually making your music.

The Tempo Map

Since Rock Band is assuming …well, a rock band, you’ll need to allign a tempo map with the audio so the software knows where the bars are. Caleb Epps showed me some of the nifty shortcuts that make moving from bar to bar snappy and automagical. Reaper itself has actually incorporated feature enhancements to accommodate the Rock Band workflow – which, in turn, means that the wider Reaper community may find improvements that impact them outside of preparing tracks for the game. I’ll cover this process in more detail once Harmonix unveils the wider beta.

MIDI Mapping and Animation

Here’s where the real work begins. When I visited Harmonix in Cambridge as they were developing the first Rock Band game, I found one guy hunched over a copy of Cubase doing just this: adding MIDI events for the game play at different skill levels. Now, in Reaper, you’re doing a process that’s just as sophisticated – it’s just much more user-friendly and quicker. (Harmonix says they’re gradually adopting the tools for the Rock Band Network internally, and some of their work already uses it.)

Especially nice: you’ll see color coding that matches the different game controllers.

MIDI isn’t just used for the notes in gameplay, though. You also add notes for the vocals, with the “+” key signifying a syllable extending across notes and another character designating notes that can’t be sung. (Bob Dylan, I’m looking at you.)

Most interestingly, you can tightly control animations, down to when the onscreen drummer chokes a hat or the camera cuts to the singer or the lighting in the venue activates, all using MIDI events. Check out the “Text Events” dropdown in the screen grab above.

If this sounds like a lot of work, it is. Fortunately, Harmonix says that the finished release will include tools that, say, allow the software to intelligently generate the animations. You can come back and tweak those if you wish, but you won’t necessarily have to manually add every single camera move – even though that’s traditionally how Harmonix does it.

All of this gets saved as standard MIDI files, so theoretically DAWs other than Reaper could perform the task, too – though for now, I can’t imagine wanting to leave Reaper, given the level of integration and documentation. But it’s nice that Harmonix hasn’t invented some crazy closed format, because if this takes off, I could see people creating other tools.

The Simulation

Now, if you had worked at Harmonix up until recently – as I saw when I did that first office tour – you’d then have to figure out how to get this song over to an Xbox console to play test it. Happily, you don’t have to do that any more. A convenient plug-in will pop up a graphical representation of any of the four parts. You can watch them animate through and get a real sense of what it’s like playing the game.

This is implemented as a standard plug-in, but the UI requires Reaper to work properly, so for now, it’s restricted to Reaper.

MAGMA and Play Testing

Good game design is all about play testing. So, when you’re distributing your music as a game, it’s essential that you actually play it as a game.

Yep, that’s right. This is the stage of the process where you have to play your Xbox. (Shame.)

MAGMA is the tool that packages in artwork and beams the track over to your Xbox 360 console. Provided your computer and your console are on the same network, the process of getting a built track to the Xbox is nearly instantaeous.

You can “audition games” locally, thanks to a patch to Rock Band 2 allows anyone with a Creators Club membership to play the games. That means you can easily test your own tracks on your Xbox, but also explore what other people are doing. And the community will ultimately determine which tracks are good enough to be approved.

In other words, if you don’t want to make your own Rock Band tracks, but want to become a virtual Xbox music “scout,” you could sign up for a membership and look for the next big thing by playing their music – interactively – on Rock Band.

That’s got to be better than dealing with all the CDs that usually show up in your mailbox.

The best part of all of this to me is that people can offer feedback. You can get through the first pass of your music, but then see how it’s playing with other people. Need to fix a camera angle? Dial down the difficulty on one level? Now you’ll get real feedback.

Interestingly, this also complements Microsoft’s other purpose for the Creators’ Club, which is to encourage independent game development using their elegantly-designed XNA game tools, some of which ultimately make it to Xbox Live Arcade. I think there’s actually a chance this could breathe some life (and users) into that service. Now, if only Microsoft would build more robust audio tools into the game toolkit so some crazy indie developer can built the next Frequency or Amplitude …but I digress.

Q&A

Anticipating the kind of questions you may be asking yourself…

When does it all happen? The network is now in closed beta. A larger beta is planned for next month, with a full launch expected around October.

So who will use all of this? I think there will be several groups:

1. Indie bands with tech savvy.

2. Indie bands who aren’t tech savvy, who will learn Reaper to get this working – and wind up using Reaper and other computer audio tools to produce their next album. (Harmonix promises extensive documentation to give them a hand. I’m sure CDM can help, too.)

3. Electronic artists who build a cottage business around prepping other people’s tracks.

4. Game developers and game fans who pick this stuff up because they love Rock Band, and wind up getting further into music.

And while 1-3 are certainly interesting to CDM, I hope we get to interract with people in that fourth category.

Isn’t this going to be too hard for some people?

Yup. Yup, it is. On the other hand, Harmonix is going to great lengths to make this easier – and if you are a skilled MIDI sequencer, you’ve just found a business opportunity.

I’ve got a Mac and a PS3.

Don’t sweat it. A lot of the Harmonix folks are Mac users, alongside the happy Windows users. It could be well worth running in an emulator or a second partition, and you can still do all your music production on the Mac. As for the PS3 – well, you can either make friends with an Xbox owner, or watch for the sale I hope is coming. You do need a hard drive, but otherwise this seems a reasonable investment.

Will I get paid?

We’ll talk more about this in a future story, but yes – thanks to the Xbox Creators Club payment infrastructure, you can expect to get paid early and often (payments arrive quarterly), meaning this could be a decent revenue stream at a time when they’re hard to find.

Performance licensing is apparently not applicable to Rock Band (I did ask about that); that’s, again, a topic for a separate article.

What if my instrumentation doesn’t fit Rock Band?

Check out the Rock Band catalog. There’s some flexibility here, as long as the game play works. You just need to make it work for the default setup so that people with a mic, a guitar, a bass, and a drum kit in front of their TV can have a good time.

And as I talked to Harmonix, we talked about the fact that previously unavailable genres could look really fantastic in the game – yes, Norwegian Death Metal, your time has come! (Now, if we just got vocal harmonies as in The Beatles…)

I also expect some really, really odd submissions in the community. (“The World’s Hardest Rock Band Track,” anyone?)

Meanwhile, I’m hoping that Harmonix will re-release their back catalog, Frequency and Amplitude, on Xbox Live Arcade, and then doubly hoping they’ll let people author for them, for all of us fans of electronic music with unusual instrumentations, and the unusual gameplay mechanic of those games. (Their new PSP game, incidentally, quietly returns to that game style.)’

On the other hand, I don’t expect Harmonix to do everything here. If this works, Rock Band could be just the beginning.

Changing the World of Music

Harmonix has long talked about wanting to create a “platform” for music, but I think it’s really Rock Band Network that could get them there. Rock Band alone can’t be the exclusive future of interactive music – that’d be boring. But if Harmonix pulls this off, it could be a real catalyst for transforming all recordings into an interactive experience – not just the established hit parade we’ve already seen. And that’s utterly huge.

I also think it’ll be well worth the time of CDM to watch as this evolves. We talk a lot about alternative controllers, about interaction design, about the merging spheres of games and music, but also about musical integrity and creativity and new outlets for spreading musical material. Rock Band Network could bring all of those ideas into mainstream consciousness in new ways.

And, oh yeah – it’ll be a heck of a lot of fun to play those tracks, and to get people playing your music. Stay tuned.

Sign up for the beta and get more information here:

http://creators.rockband.com/

I hesitate to share this, because a) YouTube numbers suggest you may have seen it already and b) it’s pretty depressing. On the other hand, it’s not like the fact the economy is depressing is news, exactly, so I suggest we employ the time-tested coping method that is laughter. Thanks (?) to Paul Norheim for this.

It also suggests a pleasing solution: the world economy just has the pitch control set wrong! Just start that turntable up again.

Or, more disturbingly, the fall of the economy is all part of some deep Schenkerian urlinie, a global capitalistic descent to the tonic. (What? No one up for some Friday afternoon theory humor?)

And yes, with apologies to the very-talented Microsoft Songsmith team, your product is becoming the new Hitler meme.

That’s it. We’re out for the weekend. I got nothin’.

Okay, so you’ve seen the painful demo video for Microsoft Research’s Songsmith software – it was intended to me tongue-in-cheek, I think, but the self-parody didn’t quite work. But the idea of auto-accompaniment software that interprets your recorded singing remains impressive. And I’ve gotten some tips that it is possible to make Songsmith sound good. Naturally, the biggest variable will be the quality of your own singing. But to make the software side of the equation more interesting, it is possible to extend the tool.

Garritan, maker of the samples in the tool, has two add-ons. There’s an orchestral pack with the usuals, and Garritan’s sampled orchestras do sound very, very good. Better yet, there are some analog synths to add, including some bass, J-60, Jupiter, and other action. These don’t come with styles, but they do give you some new sounds. Whether you use them for more evil and cheese is up to you. US$9.99 each.

Band-in-a-Box maker PG Music also has Style PAKs that are compatible with Songsmith, too. The key with these is adjusting variables in the accompaniment, and tweaking chord progressions.

I can’t say I’m entirely sold yet because I’ve never been a fan of auto-accompaniment – though, okay, I did pass some enjoyable hours messing around with electronic organ and Casio keyboard presets as a youngster, so I take that back.

Here’s my challenge to you, if you are a Windows user and give Songsmith a try. Go. Make something really great. Maybe it takes this in a new direction — sample Hatebeak’s heavy metal parrot screeches. Maybe you just happen to be a brilliant singer. Report back. The world’s ears thank you in advance.

Image: roadsidepictures.

Breaking News: If you were David Lee Roth, and you decided to use Songsmith, you would sound something like this. (Thanks, Neal Johnson! Actually, what’s a word that means not so much “thanks” but “please, never, ever send anything like this again, for the love of all that is good?”)

Warning: The following link may cause permanent hearing loss, after you gouge out your ears.

Runnin’ With The Songsmith [Metafilter Music]