There’s a lot of hype around the latest schemes for changing how artists get their music to fans, but not actually a whole lot of news. (It always seems to boil down to a website with some unpronounceable name.)

Well, this is news: Harmonix is opening up Rock Band to anyone who wants their music in it, and giving you the same sophistication of tools they use themselves. That’s a real game-changer – literally.

And I don’t mean just for the actual game Rock Band. Sure, Harmonix was the house that made music games a phenomenon in the US. They learned well from Japan’s Masaya Matsuura, perfected music games’ mechanics in Amplitude and Frequency, popularized the formula by launching Guitar Hero, then rocked collaboration with Rock Band before convincing the infamously-guarded Beatles to finally embrace digital tech. But the sad reality of game music in general is that it’s been a playing field for the old guard – it’s licensing deals with major labels to promote music you’ve already heard. It’s the top hits on the radio, redigested onto your game console. There’s commercial calculation behind even the tune that’s in the background while you’re paging through a screen in Madden. Harmonix has already changed some of the economics, and disrupted even what could be a hit, as kids discover classic metal for the first time or geeks grab music by Jonathan Coulton and Stephen Colbert. But that’s not quite the disruptive shift in game music so many people have expected.

I think Rock Band Network could be the first real sign of that shift.

Rock Band Network promises to be something really different. How?

• Anyone can get their music in the game. You don’t even need a label. You need a few (cheap) software tools, a computer, and some basic MIDI chops, and for a fraction of the cost of pressing a couple hundred CDs, any artist can get their work into Rock Band 2.
• It’s a real community-driven process. Your A&R people don’t have to shmooze with MTV. You don’t have to enter into some complex developer agreement with Microsoft or Sony. There isn’t even a shady, mysterious review process like the Apple iTunes App Store. Actual Rock Band fans will get to play your music and tell you that the animation needs fixing and the difficulty level needs to be fixed on the drums.
• You use Reaper – an actual music production tool for grown-ups. Harmonix could have given us some weird in-game tool they cobbled together themselves. Instead, they give us a special verison of Reaper, the brilliant, full-blown Digital Audio Workstation that inexplicably costs just US$60 but blows the pants off a lot of better-known tools. So you actually get to assemble your music the way Harmonix has been doing for years, with a real tool. Fortunately, the process has been made much easier and copiously documented, but it’s nice to be treated like adults for a change.
• If it works, Rock Band is just the beginning. It’s impossible to see into the future. RBN is a leap of faith both in the artists and the game fans, in terms of their taste and the amount of effort they’ll invest. But if it works, Rock Band Network could change the way people think about interactive user-created content, well beyond just furniture in the Sims or Little Big Planet.

Anyway, enough of the big picture – let’s talk details. I got to sit down with the Rock Band Network team from Harmonix high above Times Square in MTV’s offices this week to get a full-blown demo – including some seriously fun nerding out with composer/sound designer Caleb Epps, plus Senior Producer Matthew Nordhaus and MTV’s games man, Paul DeGooyer. (In a sign that the big media world still doesn’t quite get what’s going on in this field, no one at the Viacom security desk had even heard of Harmonix.)

The team was extremely generous with technical details of Rock Band Network, and walked me through the process of how artists would get going with RBN. Here’s a first look at that process.

What You Need to Get Started

$60 Reaper + free plug-ins + a computer + Windows to beam over the music + an Xbox 360 to test on.

1. Reaper (Mac, Windows) For the authoring itself, you may be surprised: you don’t need some special tool. You use Cockos’ brilliant, lightweight, Reaper. It’s not even Reaper Rock Band Edition. Reaper for Mac will work, too. Cost: US$60 for the standard license, or US$225 if you’re already a huge rockstar.

http://www.reaper.fm/

2. Reaper plug-ins (Mac, Windows) Reaper plug-ins: this download is the real magic, adding everything from shortcuts for making tempo maps to color-coding tracks to helping you add lyrics, animations, and everything else that makes your song into a Rock Band track… game. Gamesong? Songgame? Cost: Free.

3. MAGMA Packaging Tool (Windows) MAGMA is a simple tool that facilitates getting those files packaged up with artwork and keywords and such, and moving them over to the Xbox 360 for testing yourself and for sharing with the rest of the community. It is Windows-only because it relies on Microsoft’s networking functionality with the console, but Harmonix says they’ve had no problem using it on the Mac via an emulator or Boot Camp. Cost: Free. (or the cost of Windows if you’re on the Mac).

4. Xbox Creators’ Club Membership: Join Microsoft’s game development community, and you get access to a special Rock Band creators area that lets you upload and share your tracks – and other tracks from other users (which is where item #5 comes in). Cost: $99 /year (Note that there are some discounted ways to get at this for shorter terms, and you get all the game developing features of the community, too, in case you want to try to make your own game in XNA.)

5. An Xbox 360 and Rock Band 2: You do want to actually play the results, right? (Unfortunately, because of the reliance on Creators Club, Sony’s PS3 isn’t yet supported, though some sort of PS3 distribution is planned for the future.) Cost: About to come down thanks to sales – and now you get to write off an Xbox 360 on your taxes.

Total cost: as little as $100-160 or so with the various pieces, or a little more if you need to pick up an Xbox 360 and the game and/or equip your Mac to run Windows.

By the way, Ars Technica claimed this month, based on the experience of one developer, that Xbox Live [is] not ready for user-generated content. That claim is simply wrong. Sure, LittleBigPlanet is cool on PS3, but the infrastructure for moderating content is there, on the community created for the XNA game development platform. And the tracks for Trails HD (the game mentioned in that article) or even LittleBigPlanet really pale in comparison to what Harmonix is about to unleash. It’s the first time a game has really been a platform, which was long the vision of Harmonix’s founders.

Now, let’s get into actually making your music.

The Tempo Map

Since Rock Band is assuming …well, a rock band, you’ll need to allign a tempo map with the audio so the software knows where the bars are. Caleb Epps showed me some of the nifty shortcuts that make moving from bar to bar snappy and automagical. Reaper itself has actually incorporated feature enhancements to accommodate the Rock Band workflow – which, in turn, means that the wider Reaper community may find improvements that impact them outside of preparing tracks for the game. I’ll cover this process in more detail once Harmonix unveils the wider beta.

MIDI Mapping and Animation

Here’s where the real work begins. When I visited Harmonix in Cambridge as they were developing the first Rock Band game, I found one guy hunched over a copy of Cubase doing just this: adding MIDI events for the game play at different skill levels. Now, in Reaper, you’re doing a process that’s just as sophisticated – it’s just much more user-friendly and quicker. (Harmonix says they’re gradually adopting the tools for the Rock Band Network internally, and some of their work already uses it.)

Especially nice: you’ll see color coding that matches the different game controllers.

MIDI isn’t just used for the notes in gameplay, though. You also add notes for the vocals, with the “+” key signifying a syllable extending across notes and another character designating notes that can’t be sung. (Bob Dylan, I’m looking at you.)

Most interestingly, you can tightly control animations, down to when the onscreen drummer chokes a hat or the camera cuts to the singer or the lighting in the venue activates, all using MIDI events. Check out the “Text Events” dropdown in the screen grab above.

If this sounds like a lot of work, it is. Fortunately, Harmonix says that the finished release will include tools that, say, allow the software to intelligently generate the animations. You can come back and tweak those if you wish, but you won’t necessarily have to manually add every single camera move – even though that’s traditionally how Harmonix does it.

All of this gets saved as standard MIDI files, so theoretically DAWs other than Reaper could perform the task, too – though for now, I can’t imagine wanting to leave Reaper, given the level of integration and documentation. But it’s nice that Harmonix hasn’t invented some crazy closed format, because if this takes off, I could see people creating other tools.

The Simulation

Now, if you had worked at Harmonix up until recently – as I saw when I did that first office tour – you’d then have to figure out how to get this song over to an Xbox console to play test it. Happily, you don’t have to do that any more. A convenient plug-in will pop up a graphical representation of any of the four parts. You can watch them animate through and get a real sense of what it’s like playing the game.

This is implemented as a standard plug-in, but the UI requires Reaper to work properly, so for now, it’s restricted to Reaper.

MAGMA and Play Testing

Good game design is all about play testing. So, when you’re distributing your music as a game, it’s essential that you actually play it as a game.

Yep, that’s right. This is the stage of the process where you have to play your Xbox. (Shame.)

MAGMA is the tool that packages in artwork and beams the track over to your Xbox 360 console. Provided your computer and your console are on the same network, the process of getting a built track to the Xbox is nearly instantaeous.

You can “audition games” locally, thanks to a patch to Rock Band 2 allows anyone with a Creators Club membership to play the games. That means you can easily test your own tracks on your Xbox, but also explore what other people are doing. And the community will ultimately determine which tracks are good enough to be approved.

In other words, if you don’t want to make your own Rock Band tracks, but want to become a virtual Xbox music “scout,” you could sign up for a membership and look for the next big thing by playing their music – interactively – on Rock Band.

That’s got to be better than dealing with all the CDs that usually show up in your mailbox.

The best part of all of this to me is that people can offer feedback. You can get through the first pass of your music, but then see how it’s playing with other people. Need to fix a camera angle? Dial down the difficulty on one level? Now you’ll get real feedback.

Interestingly, this also complements Microsoft’s other purpose for the Creators’ Club, which is to encourage independent game development using their elegantly-designed XNA game tools, some of which ultimately make it to Xbox Live Arcade. I think there’s actually a chance this could breathe some life (and users) into that service. Now, if only Microsoft would build more robust audio tools into the game toolkit so some crazy indie developer can built the next Frequency or Amplitude …but I digress.

Q&A

Anticipating the kind of questions you may be asking yourself…

When does it all happen? The network is now in closed beta. A larger beta is planned for next month, with a full launch expected around October.

So who will use all of this? I think there will be several groups:

1. Indie bands with tech savvy.

2. Indie bands who aren’t tech savvy, who will learn Reaper to get this working – and wind up using Reaper and other computer audio tools to produce their next album. (Harmonix promises extensive documentation to give them a hand. I’m sure CDM can help, too.)

3. Electronic artists who build a cottage business around prepping other people’s tracks.

4. Game developers and game fans who pick this stuff up because they love Rock Band, and wind up getting further into music.

And while 1-3 are certainly interesting to CDM, I hope we get to interract with people in that fourth category.

Isn’t this going to be too hard for some people?

Yup. Yup, it is. On the other hand, Harmonix is going to great lengths to make this easier – and if you are a skilled MIDI sequencer, you’ve just found a business opportunity.

I’ve got a Mac and a PS3.

Don’t sweat it. A lot of the Harmonix folks are Mac users, alongside the happy Windows users. It could be well worth running in an emulator or a second partition, and you can still do all your music production on the Mac. As for the PS3 – well, you can either make friends with an Xbox owner, or watch for the sale I hope is coming. You do need a hard drive, but otherwise this seems a reasonable investment.

Will I get paid?

We’ll talk more about this in a future story, but yes – thanks to the Xbox Creators Club payment infrastructure, you can expect to get paid early and often (payments arrive quarterly), meaning this could be a decent revenue stream at a time when they’re hard to find.

Performance licensing is apparently not applicable to Rock Band (I did ask about that); that’s, again, a topic for a separate article.

What if my instrumentation doesn’t fit Rock Band?

Check out the Rock Band catalog. There’s some flexibility here, as long as the game play works. You just need to make it work for the default setup so that people with a mic, a guitar, a bass, and a drum kit in front of their TV can have a good time.

And as I talked to Harmonix, we talked about the fact that previously unavailable genres could look really fantastic in the game – yes, Norwegian Death Metal, your time has come! (Now, if we just got vocal harmonies as in The Beatles…)

I also expect some really, really odd submissions in the community. (“The World’s Hardest Rock Band Track,” anyone?)

Meanwhile, I’m hoping that Harmonix will re-release their back catalog, Frequency and Amplitude, on Xbox Live Arcade, and then doubly hoping they’ll let people author for them, for all of us fans of electronic music with unusual instrumentations, and the unusual gameplay mechanic of those games. (Their new PSP game, incidentally, quietly returns to that game style.)’

On the other hand, I don’t expect Harmonix to do everything here. If this works, Rock Band could be just the beginning.

Changing the World of Music

Harmonix has long talked about wanting to create a “platform” for music, but I think it’s really Rock Band Network that could get them there. Rock Band alone can’t be the exclusive future of interactive music – that’d be boring. But if Harmonix pulls this off, it could be a real catalyst for transforming all recordings into an interactive experience – not just the established hit parade we’ve already seen. And that’s utterly huge.

I also think it’ll be well worth the time of CDM to watch as this evolves. We talk a lot about alternative controllers, about interaction design, about the merging spheres of games and music, but also about musical integrity and creativity and new outlets for spreading musical material. Rock Band Network could bring all of those ideas into mainstream consciousness in new ways.

And, oh yeah – it’ll be a heck of a lot of fun to play those tracks, and to get people playing your music. Stay tuned.

Sign up for the beta and get more information here:

http://creators.rockband.com/

I hesitate to share this, because a) YouTube numbers suggest you may have seen it already and b) it’s pretty depressing. On the other hand, it’s not like the fact the economy is depressing is news, exactly, so I suggest we employ the time-tested coping method that is laughter. Thanks (?) to Paul Norheim for this.

It also suggests a pleasing solution: the world economy just has the pitch control set wrong! Just start that turntable up again.

Or, more disturbingly, the fall of the economy is all part of some deep Schenkerian urlinie, a global capitalistic descent to the tonic. (What? No one up for some Friday afternoon theory humor?)

And yes, with apologies to the very-talented Microsoft Songsmith team, your product is becoming the new Hitler meme.

That’s it. We’re out for the weekend. I got nothin’.

Okay, so you’ve seen the painful demo video for Microsoft Research’s Songsmith software – it was intended to me tongue-in-cheek, I think, but the self-parody didn’t quite work. But the idea of auto-accompaniment software that interprets your recorded singing remains impressive. And I’ve gotten some tips that it is possible to make Songsmith sound good. Naturally, the biggest variable will be the quality of your own singing. But to make the software side of the equation more interesting, it is possible to extend the tool.

Garritan, maker of the samples in the tool, has two add-ons. There’s an orchestral pack with the usuals, and Garritan’s sampled orchestras do sound very, very good. Better yet, there are some analog synths to add, including some bass, J-60, Jupiter, and other action. These don’t come with styles, but they do give you some new sounds. Whether you use them for more evil and cheese is up to you. US$9.99 each.

Band-in-a-Box maker PG Music also has Style PAKs that are compatible with Songsmith, too. The key with these is adjusting variables in the accompaniment, and tweaking chord progressions.

I can’t say I’m entirely sold yet because I’ve never been a fan of auto-accompaniment – though, okay, I did pass some enjoyable hours messing around with electronic organ and Casio keyboard presets as a youngster, so I take that back.

Here’s my challenge to you, if you are a Windows user and give Songsmith a try. Go. Make something really great. Maybe it takes this in a new direction — sample Hatebeak’s heavy metal parrot screeches. Maybe you just happen to be a brilliant singer. Report back. The world’s ears thank you in advance.

Image: roadsidepictures.

Breaking News: If you were David Lee Roth, and you decided to use Songsmith, you would sound something like this. (Thanks, Neal Johnson! Actually, what’s a word that means not so much “thanks” but “please, never, ever send anything like this again, for the love of all that is good?”)

Warning: The following link may cause permanent hearing loss, after you gouge out your ears.

Runnin’ With The Songsmith [Metafilter Music]

CDM has been publishing for over four years.

For once, I actually can say … nothing. Please provide your own snark in the comment space below. The CDM family thanks you.

Beamz ad, there’s a new sheriff in town.

Gizmodo: Beamz Infomercial Is Most Stupid Promo Video in History [Published in the more innocent times of April 2008]

Updated: Yes, this is the Microsoft Research team doing the video. So, in their defense, they don’t do promo videos or music for a living. We love you, Microsoft Research. It’s not even worth saying “Don’t Quit Your Day Job.” As we have previously learned, the best way for researchers to look really cool is either to involve head crabs or, if they must do vocals, be sure to involve particle physics.

In a surprise announcement (well, surprising me, at least), the experimental MySong shown by Microsoft Research earlier this year will be available for sale. US$29.95 will buy you a downloadable auto-accompaniment tool. Windows-only, but it sounds as though a Mac release is in store (seriously). It’s a bit like Band-in-a-Box for singers: sing in a line, and the software will generate accompaniment to your singing with styles of your own choosing. There are thirty styles included, and apparently Microsoft focused on the content end in bringing this product to market: there’s a 1 GB space requirement and partnerships announced with PG Music and sample house Garritan.

I’m guessing PG Music, the makers of aforementioned Band in a Box, have helped smooth out the slightly unmusical arrangements generated by the first version. Now, okay, admittedly I was skeptical of the output I heard of the first version. Maybe I’m scarred because I had a high school jazz teacher who player trumpet, not piano, and therefore insisted on running Band-in-a-Box over top of me while I tried to comp on keys. But there are reasons this is cool:

• Garritan’s sample content sounds great.
• PG Music has made its auto-accompaniment a lot more musical over the years.
• The thing could be a decent sketchpad for people who find this helps them imagine musical ideas – realizing there’s no substitute for the real thing.
• Most importantly, bringing research to market is a great thing.

And let me emphasize that last point. I love that Microsoft has made this available. Too often, R&D achievements get one demo, a patent filing, and then languish in some dark closet, never to be seen again. Sure, some of them probably were never meant for the light of day, but very often people love the demo and want to give the thing a chance – and why not let you decide?

Songsmith at Microsoft Store, via istartedsomething

So a big congrats to the Microsoft R&D team. And here’s to more research seeing that light of day, whether through open source availability or commercial release (or, where appropriate, both).

So Songsmith will accompany your vocals, Apple will get Sting to teach you to play and explain how he wrote Roxanne – okay, as if this week, you really have no excuse not to graduate from Rock Band, ye casual musicians!

Updated: Oh, wait. (*&(*&$#&*. The promo video is … ?

(*&(*&$#&*. Can Microsoft just let Sparrow do all the promotion from now on, please?

I’m going to make this a minimalist post because I’ve said what I’ll say about Kodu, the one really cool part of Microsoft’s keynote yesterday, on Create Digital Motion. (Am I the only person who wishes Sparrow had just done the whole keynote?)

But have a look at the shot above. One of the complaints about generative and algorithmic music software (and music software in general) is that the interface has been so complex. Clearly, there are many other ways to design these interfaces, and in turn, to shape the way we use these to compose and perform music. Forget for a moment that games are “games,” and this this thing is “for kids,” and I think you’ll agree – there are lots of areas to explore, and lots of potential.

It doesn’t even require some futuristic music software. Imagine more complex rules in Ableton Live’s follow actions, made graphically.

Excuse me, I’m going to pick up some Tinker Toys to think about interactive design.

You Know, For Kids: Game Design, World Creation as Microsoft Research Previews Kodu [Create Digital Motion]

PS, I believe now more than ever that Music and Motion deserve separate sites, but have a look and I think you will find some overlap.

I actually hadn’t had time to watch my tech RSS feeds yesterday when I said I “lost half an hour dreaming of my new lounge-style studio where I adjust envelope breakpoints from a giant aluminum sphere like the one in Sleeper.”

But, anyway – wish granted!

*Disclaimer: The following video, while demonstrating some insanely cool tech, may bore you to tears. In response to reader requests, we feel it’s important to warn you.

Microsoft’s multi-touch Sphere plays crazy Pong [Boing Boing Gadgets]

Now, of course, researchers being researchers, Microsoft R&D has taken a massive sphere controller and turned it into a mind-achingly dull slide show. I, on the other hand, could imagine kinky sci-fi electronica being made with massive hand gestures, particles spinning through space representing sonic grains, and the like. Microsoft, if you’re looking to hire someone to do something interesting with your giant sphere, I’m sure I or any one of the readers of this site can make something that couldn’t be replicated with a Flickr account, a toy bouncy ball, and a projector. This is the power of musicians. You try to make something absurd useful, but not really. We make the absurdly useless awesome. (Case in point: modular synthesis. Hey, is anyone using these giant telephone switchboards? Mind if we invent a new kind of party and welcome aliens to our planet?)

That said, let’s talk about just how much this is like Woody Allen’s sci-fi parody classic Sleeper.

Woody Allen, tech visionary that he was, clearly foresaw rubbing giant balls as a major future breakthrough in interface design. (For what it’s worth, the same film also predicted the 180 in fad diets that would exonerate protein and fat.)

Not only did Woody Allen’s character get strangely high with a large sphere, but the movie even suggested how to productize a more portable version of the sphere, as seen at right. Microsoft Sphere Nano, anyone?

In the classic manner of the future aging, then becoming hip and retro again, I’m not the only one now coveting a Sleeper sphere. It seems none other than Kanye West is gathering lots of nude models and, in elaborate choreography for his stage show, having them enact a giant be-in with their spheres, inspired again by Woody Allen. See some photos on Flickr, not quite safe for work, depending on where you work. (Oddly, I think this now means Woody Allen and Kanye West have exactly the same fantasy.)

And it’s back to the future in other ways, too.

The classic Keracolor TV has come back into fashion, and (as of last summer, at least) was being remade in a new edition. You probably can’t afford one, but it would look great with the Orion synth you can’t afford – and notice that its hull is more properly rounded. (At least one reader yesterday complained the Orion wasn’t as rounded as it could be.)

Ironically, I think these 1968 designs look far more stylish and futuristic than Microsoft’s Sphere.

So, sorry, Microsoft. The future just isn’t what it used to be. Even Woody Allen’s future. And he was kidding.

Let this be a lesson to you, purveyors of online music. If you do DRM-lock digital music, be prepared to continue to support it well into the future, lest users rebel. Microsoft announced earlier this year that its MSN Music service, defunct now for some time and never terribly popular, would cease to function as of August 31, 2008. This would mean that people who bought tracks from MSN Music would no longer be able to authorize files to play on new PCs and devices. The only workaround: burn to CD and re-rip.

Even on a relatively unsuccessful service, though, that caused a major outcry. Result: Microsoft has backpedaled, extending the deadline to "at least the end of 2011" and possibly even beyond. (By then, you may have to appeal to our new robotic overlords anyway, after the Great Cyber Rebellion of August 4, 2011. Oops, sorry, the people of your time aren’t supposed to know about that yet.)

The lesson here seems clear to me: the cost of DRM is ultimately exacted on the vendor. It’s especially ironic as video sellers move toward more DRM rather than less, but DRM in music seems utterly dead. And whereas the DRM controversy began as a discussion of piracy issues, it’s now centered on sales. The simple fact of the matter is, online music has proven to have real profit potential, even if it’s been slow to catch up with the late 90s CD bubble. True, DRM does live on in subscription services, though I think the comparison there isn’t entirely valid — the point of subscription models is unlimited access to music, not necessarily building permanent collections. And even there, we’ve seen a migration away from DRM, as in the streaming/purchase model on Lala.com, which I examined earlier this month.

Tomorrow night, I’ll be attending the launch party for the Wax Poetics digital download store, and they’re a print magazine for vinyl buffs, for crying out loud. (Oh yeah — guess that bit about print and vinyl being dead was also wrong.)

Conventional wisdom: 0. Music lovers: score.

When I reviewed JazzMutant’s Lemur at the end of 2005 (printed in the February 2006 Keyboard Magazine), I wondered if what we were really waiting for wasn’t a computer screen. At the time, I wrote:

There’s no question that multi-touch touchscreens represent the future of computer interfaces, and the Lemur is the biggest leap yet toward that science fiction future. For now, the challenge is that the Lemur’s features lie somewhere between a computer display and music controller, without effectively supplanting either one. The Lemur sacrifices the sensitivity and tactile feedback of physical controls in the name of flexibility, but that payoff is limited by the restrictions of its pre-built interface objects and the difficulty of configuring new layouts and assigning them to software controls.

If the Lemur could be truly fused with the computer display, rather than requiring an entirely independent interface, it would become a must-buy.

JazzMutant Lemur Review

At the same time, I marveled at what multi-touch could mean: interfaces that were as flexible as software, powerful live performance capabilities, and the ability to navigate sound spatialization and timbre in new, freer ways. Rather than a solution in search of a problem (as multi-touch image resizing is, arguably), these were tasks that just weren’t possible via any other interface.

The video above, showing multi-touch integrated with the next version of Windows 7 (expected at the end of next year), demonstrates one thing to me: multi-touch is coming, and it’ll be mainstream. And that’s huge for creative performance.

Microsoft demonstrates Multi-touch at D: All Things Digital Conference [Windows Vista Team Blog]

When Touch Makes Sense

Ironically, because Microsoft is the first to show off this technology in something resembling a consumer-ready, standard computer, people are lukewarm. (Do you think the reaction would have been this way if it had been Apple showing the same demo?) Now, I’m all for skepticism. It’s nice to see Lifehacker asking its readers whether touch is really necessary. That was the question I asked in regards to the Lemur, as well: touch isn’t the answer to everything. You lose tactical feedback, and a certain amount of accuracy. On the multi-touch iPhone, this is an especially big deal: I can easily out-type any iPhone user on my Blackberry, and multi-touch doesn’t mean a whole lot on a small form factor that can only comfortably accommodate one or two fingers at a time. Lastly, no technology can change the physical size of your finger relative to, say, a stylus.

But when it comes to music performance, I’m convinced multi-touch can be very powerful. Forget Microsoft’s lame piano demo or obligatory but meaningless photo resizing. Onstage, a multi-touch display is ideal. You can make quick gestures, quickly point at stuff without taking your eyes off the screen, and use large-scale interfaces built for performance. Imagine reaching over to quickly swap instruments, or switch between song sets, or make a rapid gesture to adjust the timbre of a sound, or navigate surround sound spatialization. And imagine that you’ll be able to do this without having to content with another piece of gear, as on the Lemur, but on a mainstream laptop, with any software you like.

Beyond Microsoft

What’s ultimately fantastic about the Microsoft announcement is that it should have implications beyond just Windows. Unlike the proprietary, one-device iPhone, having Windows 7 support multi-touch means lots of hardware should follow, with the economy of scale and access that everyone may benefit. Even Microsoft’s commitment to the relatively niche-oriented tablet PC has driven down digitizer prices (a step, not incidentally, toward this announcement). You can buy an affordable tablet PC right now with Linux installed, if you like. While Microsoft has a leg up in the enabling software for multi-touch, I don’t think it’ll be impractical for other frameworks or open-source frameworks to follow. In fact, the real challenge is to think about interface design in a new way. (In an interview with CDM, the developers of the upcoming Circle soft synth specifically mentioned thinking about making touch work in future as a design goal, and they use the cross-platform JUCE framework.)

And while they didn’t make a specific announcement, I would expect Microsoft to be likewise aggressive about promoting multi-touch capabilities in their own application development frameworks. Ultimately, I believe the most interesting multi-touch interfaces will continue to come from individual developers and researchers, not the likes of Microsoft and Apple. That’s been true already, so imagine what will happen when those folks have cheap hardware ready to go and can focus on design. The OLPC project, of course, promised a multi-touch laptop replacement, as well; that’s basically just a mock-up and I’ll believe it when I see it, but someone is going to deliver a multi-touch machine soon. (It’ll be interesting to see if we hear anything from Apple, as well.)

Yep, I Want It

Don’t get me wrong: tangible, hardware controls aren’t going anywhere. On the contrary, I think the experience of using multi-touch displays, which even with haptics are a long way from giving real tactile feedback, reminds us of the range of ways in which software design and hardware interface can fuse. But by going beyond QWERTY and mouse/trackpad, multi-touch displays could make for an exciting future.

And in answer to Mary Jo Foley’s question, do I want multi-touch in a laptop? Not only do I, but stand next to me or any other digital musician struggling with a tiny trackpad onstage, and you’ll see why.

More multi-touch coverage from CDM

The vocal input goes well, and illustrates the number of different inputs beyond the mouse you can expect in The Future. Here’s the problem: harmony is extraordinarily difficult to model on a computer because of the number of variables, the amount that’s driven by instinct and art. And let’s be blunt: it doesn’t work right.

In short: if you’re planning to build a Jerome Kern robot, the technology may not be there just yet.

If you’ve got a strong stomach, you can watch the application lay waste to “The Way You Look Tonight.” Speaking of tears: composer Kern actually drove the lyricist, Dorothy Field, to tears with the original. MySong might make you cry … in a different way. It chooses chords that fit a key and fit the melody, but completely unravels when it comes to making chords work horizontally with each other with the melody — which, when you think about it, isn’t all that easy even for experienced musician. The funny thing is, the harmonic structure of the song isn’t that complex (well, until MySong gets cranked to its avant-post-bop setting later in the demo). Harmony is perhaps just harder than the technologists may realize.

The researchers do compare their tool to Band-in-a-Box’s automatic harmony selection module, and this works better than that — but that’s not saying much.

I also have to admit, I’m getting a little fatigued of all these tools that want to dumb down music, as if somehow it’s music’s obligation to be push-button easy. Do we build giant robotic armatures so people can play basketball without practicing? Isn’t it the struggle that makes it fun? The researchers in this point seem to have missed the point: all those hours you spend sitting with an instrument working out chords are perhaps what music is about. There’s not some musical secret the experts are keeping from everyone else. The songwriter with the guitar very likely received very little training. All of that tweaking of melody and harmony is part of the process that eventually yields things like, well, “The Way You Look Tonight.” Jerome Kern and Cole Porter and Richard Rogers did it very quickly; amateurs may do it more slowly. But it may not be possible to reduce to rules in a way that the current generation of computing intelligence can even understand — and even if it does, it may require more than one or two sliders to adjust.

The best part of the video is the editable parameters: sliders for Jazz factor and Happy factor settings. (Theory fans: the approach seems to be for Happy factor to lobotomize to major I/V chords and Jazz factor to eventually turn everything into sus13.) I’d like to suggest a few additional settings for reproducing a broader variety of music:

• Emo angst factor
• Tone deaf factor
• Pretentious techno chords factor
• Stoned factor
• Saccharine-sweet triteness factor
• Community theater audition accompanist factor
• Went to a liberal arts college where everyone on my floor played Ani DiFranco way too much factor

It’s well worth watching the demo. And, of course, this is the reason to tackle artificial intelligence — even if you’re unsuccessful, you’re learning. My guess is, we’ll need genuine AI before we can successfully harmonize melodies.

MySong, from Microsoft Research, makes your singing sound a lot better than it really does  [istartedsomething]

MySong: Automatic Accompaniment for Vocal Melodies [Explanation, Demos, Academic Paper]