The gaming industry has made their bet, and it’s that touchscreens go better with tactile controls. Might digital musicians reach the same conclusion?

A funny thing has happened on the way to the touch era. The vision of a device like the iPad is minimalist to the extreme: an uninterrupted, impossibly-slim metal slate, as impenetrable as some sort of found alien scifi object. The notion is that by reducing physical controls, the software itself comes to the fore. It’s beautiful conceptually … and then you find yourself tapping and stroking a piece of undifferentiated glass. For navigating interfaces – and even, I’d argue, exploring sound design and composition – it works brilliantly. But for live digital performance (what to game lovers is called “gaming”), for anything that wants tactile feedback, it can be imprecise or unsatisfying, or both.

Watching this shake out as a design problem is fascinating, especially coming from the perspective of music. Digital musicians were exploring alternative interfaces since before it was cool. Given the ability to make any sound we can possibly imagine, the question of how you design an interface around sound is compositional, philosophical, essential.

Whatever winds up working in the marketplace, there are some fascinating ideas for combining touch with tactile. Since both are good at certain tasks, why not do both?

We’ve seen several examples among musicians and researchers exploring how to augment the touchscreen with physical input:

Mike Kneupfel’s research at NYU’s ITP program, in the video at top, investigates adding additional inputs. See: Extending the Touchscreen.

We saw that kind of extensibility in an iPad dock concept by Livid Instruments.

While it lacks additional tangible controls, I/O extensibility is featured in a still-as-yet-unreleased “pro” dock by Alesis, and most recently in a DIY dock by circuit bending pioneer Reed Ghazala.

Now, game vendors are moving in the same direction – even with prototypes that look quite a lot like the research project above. (Sometimes, arriving at the obvious conclusion is necessary for a great design.)

Sony’s PlayStation Vita, successor to the PSP mobile game platform, augments touch input with tactile controls in much the same way as Michael Knuepfel’s work does. Notably, it also proposes how these inputs can coexist in a form factor that’s larger than a phone, but smaller than a tablet – scaled roughly to a comfortable holding distance between your two hands. (Microsoft and Apple each unveiled standard split keyboards on Windows 8 and iOS 5, respectively. The era of thumb ergonomics is now fully underway.)

Nintendo’s Wii U controller combines a lot of sensing capabilities into one device. Like Sony’s effort, the centerpiece is the combination of the interactive touch display with analog controls. But true to its Wii heritage, Nintendo is packing other sensing technology, too. While its evolution has been more piecemeal, the same is true of the Xbox 360 in the Kinect era. The Kinect camera is really a bundle of mic and stereoscopic camera sensing with software intelligence for motion analysis and even speech analysis via a variety of methods. While Kinect is touchless, the conventional gamepad still plays a role.

What’s the relevance of all of this evolution to music? Digital music’s demands parallel gaming, requiring precision, accessibility, scalability from beginners to hardcore experts, and real-time interaction. Also, music research has often been at the forefront of experimentation with a variety of means of translating sensory data to expression. And since musical practice itself is roughly as old in human evolution as language, if not older, it’s a key way of glimpsing how ubiquitous interfaces can become meaningful.

Let me put that another way: the stuff game companies are doing now looks a heck of a lot like what computer musicians have been doing for years.

While much of the acclaim for platforms like the iPad has been for their transparency and unadorned interfaces – and while I believe those are valuable concepts – bundles of capabilities for interacting with the world can become powerful. That means efforts like Apple’s addition of USB MIDI connectivity to the iPad, or Google’s nascent work to standardize USB host mode and open hardware development (based on Arduino), take on new meaning. Add to this additional connectivity via Bluetooth and wifi, and it may be that we only really see what these platforms do when, like the PC, they start geting sociable with a range of other gear.

This could also mean that communities like the music community have a chance to prove that the “post-PC era” is a little different than it’s been described in the mainstream press – and maybe a little less a radical departure. The “post PC era,” we’re told, is less about being a hub for a lot of hardware. But as people look for tactile feedback, some of the coolest applications of these platforms may not be in the mainstream use as “consumption” devices, but at the fringe.

I’ve just come from the launch of the Samsung Galaxy Tab 10.1 in New York. You’re not missing much; there were a handful of people snapping up the tablets. (I think the 10.1, and a few other Honeycomb-based tablets, do have a bright future, though their growth may be a bit slow at first as developers get their hands on them and give people a reason to buy them.)

What was most compelling to people at the launch, though, was a planned appearance by pop star Ne-Yo (at least according to some staffers to whom I spoke).

But the connection was, at best, tenuous. It may be when devices like these tablets are made more viable for musicians onstage that that connection starts to make sense. And that may mean that Apple and Google/Android vendors alike need to start to think more aggressively about the larger ecosystem and hardware applications. Remember all those futuristic promises from Apple about hardware accessories? Right now, the most significant hardware is the Square payment add-on, and it uses a hack to make it work through the audio jack. Both Apple and Google can do more work to open up hardware development.

It’s all well and good for the tablet to be a “post PC” device, to be different from PCs, to be better. But they may simultaneously need some of the openness to other gadgets that made the PC age so revolutionary.

Load up an Ableton set or mix samples, and you’re already in the domain of interactive music. With joysticks and arcade buttons and other controls, the blending of game and musical interface into generative compositional fusion is even clearer. It’s little wonder many electronic musicians take an interest in the nexus of gaming and music.

Any discussion of interactive music scores for games would be incomplete without Jonathan Mak. His self-produced title Everyday Shooter used classic top-down space combat as a musical experience: not only do sound effects in the game act as musical elements, but even the flow of the game itself fits into a generated song structure. Mak even imagined the title as an album. Playing through it, once you get into the groove of the action, the roles of gamer and listener merge into a single flow. (See video, at end.) Another nice feature – breaking from cliche, it’s a music game that employs guitar lacks in place of, say, a pounding trance soundtrack.

Now, Mak turns his attentions from space shooter to platformer, with SoundShapes, on the console Sony announced this week, the PlayStation Vita. It’s best to watch the video to see what’s going on. It’s just a taste; we’ll have to see the final title.

There’s more good news, too: on of our favorite artists, the inventive i am robot and proud, is the musical collaborator. And you can now look to Toronto as a hotbed of indie game action – take that, Montreal and New York – with i am robot and proud and the team behind iPad album-as-game-as-album Swords & Sworcery.

The effects of making ever game event musical can be cartoonish at times – though, perhaps in a game, that’s part of the pleasure and aesthetic. But in Mak’s best moments, it was as though he was composing with gameplay – canonical gameplay forms as a modern, digital parallel to musical forms like a sonata.

We’ll be watching. Thanks to Metehan Korkmazel for the tip!

Via GameSetWatch, who have some nice analysis. (One point of disagreement – they speculate this will be onstage “in the hands of at least one chiptunes performer.” I think that’s actually relatively unlikely; chip music artists continue to prefer dedicated music tools, not games. Laptop artists would occasionally feature an ElectroPlankton cameo. But I’ve been fiddling with some Processing sketches that try to make game interfaces for music. It’s fun, if really hard; I’ll keep trying. Seems a good airport layover project.)

Previously on CDM:
Portal 2′s Musical World, Available Free, in Non-Adaptive Form “For Testing” [ Also makes use of interactive musical accompaniment ]

Game Meets Album: Behind the Music and Design of the iPad Indie Blockbuster Swords & Sworcery

Jim Guthrie was a rockstar long before the iPad was. Paired with pixel-intense artist Craig D. Adams (aka Superbrothers) and the co-design and coding effort of a crack team of video game “wizards” at the indie studio capy, he’s made a soundtrack that’s destined to be a gaming classic. But if you don’t want to play it, you can still listen to it. And if you’re playing it, you may find that it feels as though you’re listening to it, and gazing into its artwork.

From the moment you tap to launch it, Swords & Sworcery plunges you into a world that’s part game, part interactive album. Yes, there’s the obvious presence of a spinning vinyl record you can scratch and brake, right there on the title screen. And yes, there’s the conspicuous “EP” in the title, or the just-released LP (a real LP, on digital but also now sold out on vinyl).

But it’s once you navigate the expansive digital forests of the title, once Jim Guthrie’s moody soundtrack taps away at your brain, that you begin to get it. Sword & Sworcery will certainly get the dreaded (or is that coveted?) “arty” title, but it’s the way in which it spins out audiovisual entertainment that makes it special.

It’s pure aesthetic deliciousness, a brew that makes your head buzz.

And it’s finding that aesthetic sense – neither retro nor modern, neither low-fidelity nor slick – that makes this title relevant beyond even the world of gaming. Jim Guthrie’s songs and the lush pixel art graphics are the perfect fusion of old and new. It’s telling that Guthrie himself crafts his tracks in a combination of a PlayStation music game (MTV-branded, no less), GarageBand, and then high-end Universal Audio plug-ins. (See video above, and have fun gear-spotting familiar toys through the jump cuts.) It’s sort of studio garage, in the way digital music can be now. Its unabashedly synthetic instrumentation gives voice to a generation that grew up with computer-produced music. The musical score itself sometimes nods to Philip Glass, sometimes to punk rock, very often a mixed-up, intimate fantasy folk cinema, with sounds both shiny and flat.

But happily, this isn’t just a game with a clever soundtrack, or a release of game music. It’s a real fusion of album and game, music and visuals. And, lest we get to carried away with the Art label – capital a – music and game alike are good fun.

CDM managed to pry co-creators Craig D. Adams and Jim Guthrie from an adoring gaming press long enough to talk to us in depth about the making of the music and release, down to every last technical and artistic detail. They said so much – and crossed two media so completely – that I’ve broken up their ideas into two stories, across Create Digital Music and Create Digital Motion. Their reasoning for committing to those two media has a lot in common, I think, with why we run these two sites and why a lot of you read and contribute to them.

Out now: both an LP music release on Bandcamp and iPad version. Coming this month: recent-gen iPod touch and iPhone versions of the game, too.

Jim Guthrie: Sword & Sworcery LP – The Ballad of the Space Babies @ Bandcamp
http://www.swordandsworcery.com/project/

Let’s begin with the notion of this as musical-visual collaboration. Obviously, some of our favorite game experiences have used music effectively. What’s different about this project?

Craig:The iPhone & iPod Touch, and the iPad to some extent, don’t have an input style that lends itself to precise inputs. So, it seems to me that a lot of traditional video games seem to fall a bit flat on these platforms. The thing is, these machines are great music and video players, so we knew going in that we wanted to make something that was as open and as laid-back as a record-listening experience matched with a naturalistic visual presentation inspired by film, so that was really the starting point. We also felt that a more relaxed, more occasional, less punishing, more interesting experience would be a better fit, something that was closer in pace to browsing the Internet or whatever. Early on we were calling S:S&S EP “a brave experiment in Input Output Cinema.” I/O Cinema is kind of an intentionally absurd nonsense buzzword but I think it’s perfectly apt for this type of entertainment, it’s a heckuva lot more descriptive than ‘videogame’ anyways, in that it gets away from the idea of a program with rules and win/lose conditions and it puts the focus more on the conversation the audience has with the creators while the audience pokes, prods & problem-solves an authored audiovisual creation.

How did you work together, Superbrothers and Jim, to combine music and visually? What was that collaboration like?

Craig: When we looped Jim into the project in we told him the name, described the aesthetic, talked a bit about The Legend of Zelda & Castlevania, and then Jim dug around and found a few songs he thought might fit. I went ahead and tried to generate art & narrative concepts using Jim’s songs or else stand-ins to set the mood. As we started to mix things together we’d evaluate, iterate & improvise. Eventually we’d get into situations where me and Kris, Capy’s creative director and co-designer on S:S&S EP, would have a plan for an environment or a scene or a situation, and we’d get the art & the mechanics together and then pass along a rough build to Jim with some kind of suggestion like ‘go John Carpenter on this one’ or whatever, and then Jim’d work his magic, filter the concept through his music-making mind and barf up something totally beautiful & shockingly perfect. So yeah, it was a messy process, but towards the end we kind of got a feel for it, I think it all worked out super well.

Jim: It wasn’t always clear if the art needed to inspire more music or the other way around, but it was a very necessary process considering the relation the two elements share in the game.

Technically speaking, is there anything unique to the way the music integrates with game play? How did you approach the technical challenge there, in other words?

Craig: For the music integration aspect, we really just made things up as we went along. We tried some things; some of them worked, some of them didn’t. Then we’d iterate on them or revise them as necessary. We tried chopping things up into a million loops and then stringing them back together with logic, and it kind worked, but was kinda rough, so then we’d revise it or refine it. Eventually we started to figure out a bit of a groove – we learned what the limits were with the machines & the quirks of fMOD [the game sound engine]. We’re a whole lot wiser now, but I think it was a positive thing going into something like this a bit naive.

Jim: Technically, there’s nothing in this game that hasn’t been done before. We sort of ‘stood on the shoulders of giants’ and made it our own. It’s more about the mood and atmosphere that the music and art create that is special. Like Craig said, we made things up as we went.

From the beginning, we knew it was very possible that this would be released digitally as an album, but it wasn’t until a little later on that the idea of vinyl struck us as a good idea. You would think it was all planned from the beginning considering how often the image of the record appears in the game but it sort of willed itself in that direction over time.

It’s always tough to describe the process of summoning one’s art. After we had sort of figured out what the first few tracks were going to be, I just let Craig’s art and ideas lead the way and I reacted. It also really comes down to knowing your craft and what tools you use to create with. Once you figure that out the tools don’t get in the way when you’re hot on the trail of a fleeting melody. There’s noting worse than loosing that spark because a technical issue. Computers have robbed me of so many musical sparks, but to be fair, they have given it back tenfold.

I will give into the temptation to ask one obvious question – what does it mean that it’s an EP? Obviously, it’s a reference to the notion of a game release as being akin in some way to an album, but anything beyond that you wish to say?

Craig:The EP concept goes back to the start of the project – we wanted to put the sound component right out front. We wanted the whole project to feel like a musical composition, and at first we wanted to make something small and acknowledge that this was a tentative first release by a new videogame ‘band.’ The project grew from ther,e and it goes well beyond the 37 minute running-time we had originally envisioned, but everything else fits.

We had always planned to prepare a record release to accompany the project and when the time came to commit to this we basically had to make a vinyl edition, and Jim basically just put that into gear on his own… so that became Jim Guthrie’s Sword & Sworcery LP – The Ballad of the Space Babies. While the record is a smaller component of the project in terms of man-hours, the music on its own is kind of larger than the art and the story we tried to create in the actual videogame, so I think it’s kind of perfect that it’s the LP.

Jim, the music really has a quirky personality all its own, and I think it’d be too easy to describe it aesthetically. How did you approach scoring the music, in finding a voice for this title?

Craig: Several of Jim’s songs pre-date the project, so they informed the aesthetic & concepts from the start. My role early on was to translate the music into artwork & narrative that would fit the general idea of the project. But yeah, beyond that I’ll let Jim fill in the blanks here!

What’s the production process like for the music itself?

Jim: I captured all of the music either on a PlayStation using MTV’s Music Generator and/or
[Apple] GarageBand. For example, on the song, ‘Lone Star,’ I drummed a beat onto a cassette four-track, burned that onto a CD, placed the CD into the PlayStation, sampled and looped in MTV Music Generator,
and then built a song around it using that software. THEN I brought it into GarageBand and added more layers and effects. I also used a [Casio] SK-1 peppered throughout. In terms of plug-ins and soft synths, I used a lot of the Arturia stuff, [Native Instruments] Kontakt, [XLN Audio] Addictive Drums, [Toontracks] Superior Drummer, and a [Universal Audio] UAD-2 card loaded with a bunch of their processing plug-ins.

Not all games are narrative, and I’ve never found conventional narrative to be a prerequisite to art (cough, Ebert). But there is a strong narrative aspect to this title, too. How do you go about telling a story and building a game mechanic at once? (And, for that matter, do you still scrawl things on index cards to get there?)

Craig: It’s funny, we are getting some positive responses to S:S&S EP’s narrative, but really, the narrative only exists to make sense of the player’s experience; it’s not exactly ‘the point.’ We started with the songs, then the art, then the mechanics that would bring it together. And while the broad narrative concepts were always there, it was only in the final stages that the script came together, and really it’s just a way for us to help communicate what’s supposed to be going on. I was on the line to write the script, and for a good long while, it kinda sucked while I was buried under art, sound & design tasks, but I kept iterating on it, editing it for brevity, clarity, and humor, with Jim and Kris and a few others kinda guiding the process.

So yeah, I guess we did some okay things with narrative, and I’m actually super-proud of the mind-fuck tear-jerker heart-breaker finale, but I think the only reason any of it comes across is because of Jim’s music wrapped up in paintings. And really, Jim’s songs are all the narrative I ever wanted.

Now that you’ve become gaming rockstars, what’s next?

Jim: A bottle of vodka?

Craig: Hahahaha… Jim’s already a rockstar, so this stuff is probably old news. I think we’re definitely enjoying our fifteen minutes of fame in this very specific niche, and I’ve been trying – maybe too hard – to keep that buzz going so the project stays visible as we gear up for the all-important iPhone & iPod Touch launch. Once all that’s out of the way, I’m really just looking forward to some quiet time: bike rides, swimming, hiking, and whatever else.

We’ll keep the Sword & Sworcery project rolling along in the background too. We have plans for a gala event here in Toronto in a few months and some other schemes related to the app itself that’ll last the year & maybe into next year. We’ve been given a real opportunity here & we want to continue to honor that.

What are you excited about in gaming – or, for that matter, audiovisual work – at the moment, beyond your own work? Anything you’re listening to, watching, playing (or all three) at the moment?

Jim: Honestly, I went into my iTunes to have a look at my ‘Recently Played’ list and for as far as the eye could see, it’s all stuff I’m working on. No time for art! Just work!

Craig: I’ve been too busy and too exhausted to be paying much attention to what’s happening out there in videogames, film or music. To be honest, what I’m most excited about right now is the prospect of getting some fresh air and some exercise, maybe getting away from electronic screens for a bit sometime, and then after a little break maybe starting on some new creative work.

I had the opportunity to see 2001: A Space Odyssey in theaters a few months ago. I’d seen it a few times before but only on VHS… so that was a real treat, it’s an entirely different film in the theaters, there’s so much more to enjoy. I’m also a huuuge fan of Kanye West’s “Runaway.” I think that’s a genuinely incredible piece of audiovisual work; Vanessa Beecroft’s art direction really shines. Banksy’s Exit Through The Gift Shop and James Cameron’s Avatar blew me away too, for entirely different reasons. I’ve just recently seen my friend Firas Momani’s Fantasia Festival award-winning short film The Adder’s Bite & it gave me all those groovy Cronenberg + Lynch + Kubrick feelings, very inspiring.

On the video game side I’m still intermittently playing Motorstorm: Pacific Rift for PS3, a 2008 effort from Liverpool’s Evolution Studios that I think is basically perfect, plus I’m digging in to Monster Hunter Tri on Wii. I’m playing Monster Hunter co-operatively with a couple friends every Sunday morning… we’re still just scratching the surface but it’s easily the most intricate and deep video game I’ve ever played, which takes me way outside of my comfort zone in an interesting way. I’m also cautiously optimistic about L.A. Noire, Uncharted 3, and The Last Guardian… we’ll see how they work out in the end.

On the music side, I’ve been listening to Jim’s Sword & Sworcery LP… even though I’ve heard these tunes so much in the last two years that my ears hurt, the record itself still comes across as beautiful & fresh, the songs still evoke all kinds of imaginings. That record aside I’ve got a heckuva lot of catching up to do… but first I have to give my ears a bit of a break. That said, I’m amped for the Beastie Boys record that’s hitting in the next little while.

All images courtesy Superbrothers and Jim Guthrie. Used with permission.

Do let us know what you think of the game, folks – or whatever audiovisual creations, in the form of games or otherwise, inspire you.

More on the art, the design, the coding – and why Superbrothers went iOS-only.

On our sister site:
Inside Handheld Game Art: The Art Style and Making of Swords & Sworcery, Superbrothers Pixel Cinema [Create Digital Motion]

And, oh yeah, don’t forget to get the game:
http://www.swordandsworcery.com/

Games really are reshaping music. Despite their relatively simple gameplay, the Guitar Hero and Rock Band franchises originated by developer Harmonix are stimulating interest in real music making. It’s no accident that you can walk into a Best Buy and, next to aisles of video games, find a growing selection of serious musical instruments and technology.

These titles are also stimulating interest in music and artists and producing a new distribution outlet, at a time when the distribution picture for music can seem bleak. But until now, that outlet has been limited to big acts, big tracks, and big deals with big labels. It has only promoted music you already know, not the discovery of new music. Rock Band Network could change all that.

We took a detailed look in August at how Rock Band Network worked technically, and how authoring a song for RBN could give you the same level of gameplay and choreographed graphics that the official Rock Band tracks get. But now here’s the big news: at long last, RBN is opening to the general public, starting with an open beta for artists and play-testers.

What it is: Rock Band Network is a new set of authoring tools (built around Reaper), a submission process (built around Microsoft’s Xbox 360 XNA Ceators Club), and an upcoming store to host indie tracks called the Rock Band Network Music Store.

What it costs: Rock Band Network membership is free, but you’ll need a $99/year XNA Creators’ Club Premium account to submit or test music.

What you’ll need: To author titles, you need an Xbox 360, a copy of the Reaper software, a set of free plug-ins for Reaper for RBN, the XNA account, and either a Windows PC or Mac. (You’ll need Windows, either virtualized or on another machine, in order to actually load the tracks for testing, but you can author on either; see below for more.)

What it gets you (as an artist): If you make it through the peer-reviewed submission process, you stand to set your own pricing and receive 30% royalties (retail, excluding tax) on everything you sell.

What it gets you (as a peer reviewer): With the XNA Creators’ Club membership, you can play as many tracks as you want without any additional charge, in exchange for your feedback. Tired: squeezing into sweaty, overcrowded bars at CMJ and South by Southwest to hear new acts. Wired: Scouting for new acts on your cough with your Xbox 360. And that could make a nice community of music, depending on how this evolves.

Where the tracks will be distributed: Anyone with a copy of Rock Band 2 (and presumably future versions of Rock Band) can play your tracks. Releases will initially debut on the Xbox 360 store for 30 days. A “selection” of tracks will also appear on the PS3 and Wii stores after that. (The approved songs will stay on the RBN Store on Xbox 360, regardless.)

When does all of this happen? The open beta launches today for peer reviewers and artists. The store is due, um, “real soon now.” (No specific date yet.) The game itself is ready to go, at least on Xbox 360: a patch introduced way back in September added the ability to play RBN tracks.

CDM Talks to Harmonix

John Drake, Program Manager for Rock Band Network, took some time out to answer my questions on the eve of launch.

CDM: What will the Rock Band Network Store look like? Where will you get access to it? Will it be a similar store on the PS3 and Wii?

John: The RBN store will run in parallel to the existing Harmonix DLC store, and will be in the same menu location within Rock Band 2. The RBN store has more info about each song than our existing DLC store does, and it has more ways to discover new music: you can search by subgenre, album, country of origin, record label, even the author of the song.

The PS3 store will be very similar to the Xbox 360 store. Details of the Wii RBN presence are still being worked out.

Ed. note: It’s especially nice to see the RBN store on equal footing. I had high hopes for the XNA-produced games on Xbox Live, but those titles aren’t displayed or listed in exactly the same way, which I think has hurt the initiative a bit.

CDM: In addition to the XNA Premium subscription, you still need Windows to support testing your own tracks, yes? Do you need a Windows PC to be a playtester?

John: You need to run Windows in order to transfer song files to the Xbox 360, because we use Games for Windows Live to manage the transfer. We have informally tested running Windows on a Mac on a number of virtual machines, as well as BootCamp, and most of them work perfectly for transferring files.

Ed.: I can add, a number of the Harmonix guys are Mac fans, so you can believe they tried the virtualization approach!

CDM: Since we last talked, there has been a private beta. Were there any additional improvements / changes since our August conversation? What kind of feedback have you gotten?

John: The closed beta has been absolutely invaluable to help us shape the experience for the new members just now joining the program. We’ve cleaned up and organized the documents section of the website, added a great deal of new information, clarified policies for submitting songs, and generally made sure that the pipeline is running smoothly. None of the major processes are any different than initially designed, but we have changed a million small details to make it better.

It’s important to note that the members that have been in the beta have been absolutely extraordinary: patient, intelligent, hard working, thoughtful, and helpful to each other as they worked through the inevitable issues that cropped up as we readied the site for launch.

CDM: Have any currently-available tracks come through the private beta process? (Jonathan Coulton’s?)

John: We currently have nearly 40 approved tracks, including tracks by the inimitable JoCo, and a bunch more up for playtesting and peer review. We’re expecting even more great content to go up for testing in the next few days, and we’re excited for people to join our playtesting ranks to get even more songs through the pipeline!

CDM: I see TuneCore is offering track preparation services. Have you seen similar offerings begin to appear? (For some of us, doing the authoring may actually be satisfying – we’re weird that way!)

John: There’s a great variety of services cropping up from authoring houses offering with different programs to create songs for bands. These range from straight, up-front fee structures to a $0 down, pay us out of your royalties deal. It’s really exciting to see how different groups are responding!

*PS, I’m with you on the satisfaction of authoring. I’ve been working with my band to put our whole last and current record (17 songs in total) up for RBN. It’s a lot of work, but it’s super rewarding to get involved in the process! And it’s really doable if you’re used to making music as a passion!

CDM: Outside RBN, are these tools beginning to be used on Harmonix’s own tracks? (I believe that was in the works when we last spoke.)

John: It was always the intention that the tools we developed for the Rock Band Network would be integrated internally at Harmonix and that has begun to happen. With the industry leading amount of content we produce (over 1000 songs and counting) anything that makes the job of our unparalleled Audio Team easier is welcome, and in most cases the Rock Band specific tools were built by members of the Audio Team themselves!

CDM: Okay, enough of the nit-picky details… what’s it mean for you that you finally get to take this to public beta? Now with a few months more perspective on it, what do you think this will mean for musicians to get on this platform, revenue aside?

John: As our Senior Producer Matthew Nordhaus said about Rock Band Network, “It completes me.” We’re already thrilled with the community working within RBN and we’re hopeful to see a lot more great content and enthusiastic playtesters signing up at Creators.RockBand.com now that we’re open!

Additionally, we’re really proud of our teams here at Harmonix and MTV Games, who have designed a really smart way of getting great music into the hands of fans. Empowering musical groups of all sizes and genres to be able to post their own content for sale is really a dream come true at Harmonix. Adding the great variety of music for our passionate fanbase only makes it that much sweeter. We’ll be even more excited when the store turns on and those first tracks sell!

Go Check it Out

I hope to help document both how artists are using RBN and the technical process for doing this yourself over the coming weeks. In the meantime, you can hop on the beta yourself if you’re interested:

How to submit a song: http://creators.rockband.com/docs/Website
Scroll down to “Adding a song to the pipeline.”

How to become a peer reviewer?
http://creators.rockband.com/docs/Playtest_Process

And yes, I still want to see an Amplitude/Frequency Network that’s friendly to electronic music, minus drums + guitar. I think Harmonix knows a few of us feel that way.

Jonathan Coulton on Rock Band Network, from the awesome PAX.

PAX ’09 Rock Band Network Panel #3 from Harmonix on Vimeo.

Everything you need:
http://Creators.RockBand.com

Video interview by G4:

It’s official: we had heard rumblings that game maker Harmonix was about to announce something, and it’s here. It’s a collaboration directly with the Beatles to make something that isn’t Rock Band or Guitar Hero — something completely new. And something completely new is exactly what’s needed.

Before Guitar Hero and Rock Band, before being purchased by MTV/Viacom, game developer Harmonix were a very different creative house. Co-founders Alex Rigopulos and Eran Egozy were MIT friends whose first project was an application that let you play guitar with a joystick. (Sounds like a research project you might read about here.) Their interactive music games were influenced by the explosion of Japanese titles like PaRappa the Rapper and Beatmania, to be sure. But part of what made FreQuency and Amplitude so important was that they offered more than just a simplified music experience. They were digitally-powered acid trips, with VJ-style video clips playing up buildings and surprisingly sophisticated interfaces that remixed the music as you played.

Make no mistake about it: Guitar Hero and Rock Band are brilliant titles with a fair dose of musical integrity in the way they abstract playing experiences for broader audiences. But there’s no question some of the original creativity — the sense that the game experience was unlike any other experience — is missing. And in this pumped-up HD age, in which surreal game experiences like intra-dimensional navigation in Portal or ambient floating cartoon paramecia in Spore, it’s hard to wonder if gamers who weren’t ready to snap up FreQuency a few years ago might be ready now.

So while rival Activision bakes a watered-down GarageBand-style app into another iteration of Guitar Hero, it’s intriguing, at least, that Harmonix is working with the Beatles. And they really are working with surviving Beatles and Beatles Significant Others: Sir Paul McCartney and Ringo Starr, Yoko Ono Lennon, and Olivia Harrison. (Okay, I’d like to see a special Yoko-inspired game on Xbox Live Arcade.) Most interesting, producer Giles Martin, heir to production legend Sir George Martin
and producer of the Love project with Cirque due Soleil, twice a Grammy winner, and the man behind The Beatles Anthology is involved, too. (See a great story on him in Sound on Sound.)

Let’s get straight to the point: for the band that made virtual acid trips mainstream decades ago, it’s time for a new, digital trip. (They do describe it as a “journey” through the Beatles’ work, after all.) I think the Beatles make a perfect choice. I can’t count the number of people I know in music composition who were addicted to Beatles records as kids — not the Beatles’ generation, but their offspring in the 80s and 90s.

And despite the intervening decades, Yellow Submarine still looks imaginative and bizarre. If gaming can do anything, it can take music we’ve heard a zillion times and make it new. It can make our regular experience, the reality around us feel a little different. Rock Band has proven to be a trojan horse: it’s literally driven up sales of real instruments. That’s proof that making something palatable to a mass market can help get them hooked on new kinds of experiences. Can a Beatles game feel less like interactive documentary or re-hashed Guitar Hero, and more like a groovy, retro journey into the strange imagination that turned a lot of us on to recording, music, visuals, and … uh … animations of strange creatures? I think so. Can’t wait to see what comes out.

PS — I want to play as George.

Kotaku gives us a video interview, here, while Music4Games gives us a written one here.

CDM recently had the opportunity to speak with some of the audio designers for Call of Duty: Roads to Victory for PSP. Mark Yeend is the Audio and Music Manager for Amaze Entertainment, Drew Cady was the lead sound designer, and composer Noel Gabriel, whose scores for Amaze have been recently compared to (confused with?) Hans Zimmer.
(Disclosure: Though I did not work on this title, these guys are my coworkers.)

Create Digital Music: Can you please familiarize our readers with the platform specifics of the PSP?

Mark Yeend: In terms of making games, it’s a lot like the PS2. You’ve got resident per-level-load RAM for sounds, and you also stream data off the UMD disc during runtime, which can be used for a music stream or for buffering sound data. The PSP speakers are bright and loud, and the steaming music format is Sony’s ATRAC3 codec, which sounds fantastic. Aside from gaming, the PSP can store, play and manage your music library, link to your PC, and play movies on the go (which look and sound great). I have really been impressed working on the PSP for the last few years.

Drew Cady: 32 channels of audio can be played at the same time, and believe me we use all of them in this game. Sampling rates can be relatively high, but at the cost of memory. The onboard speakers roll off below 500hz (-33db at this point), but I still try to keep the low end frequencies for headphone listeners.

CDM: Can you describe for our readers some of the primary constraints and limitations you had to work with in making this particular title for the PSP?

DC: RAM is the biggest constraint. The first thing I do on a project is lock down how much memory I will receive for audio. This helps me scale each section of audio. For example – level ambience, level effects, weapons, player character, non-player characters, global sounds, dialog and vehicles. For Call of Duty, the audio memory budget for each level was around 2.8MB. To help juggle memory I can downsample any file independently. Distant sounds can use a lower sample rate of 17k or 14k while first person gunshots are sampled higher at 28k. After creating a sound, I usually listen to it downsampled. For instance, a chain sound effect needs to be tweaked so they can be heard in game. If you downsample to 22k much of the high frequency detail is lost. If you raise the sample rate to 28k the high frequencies are not cut off and you can hear the jingle. Alternatively, you can also lower the pitch by two half steps and keep the sample rate at 22k. It takes a bit of experimentation to really find the sweet spot.

MY: As Drew noted, the two biggest limitations were the frequency response of the PSP speakers, and the audio RAM budget in each level. How do you create an earth-shattering explosion or the rumble of a Tiger tank on quarter-inch speakers? You can’t. We’re counting on people using headphones, but the deep sounds have to be audible without headphones too, because they often affect gameplay. For example, they’ll let you know when and from where the enemy tanks are coming. Our typical solution was to add a bit of mid-frequency content to the deep sounds; just enough to make them present on speakers without changing the character and realism of the sound. For RAM, despite our producers and programmers acknowledging how critical audio is for Call of Duty, we ended up with about 60% of the RAM originally allocated for audio. It was a really tight squeeze, and it’s a symptom of art and geometry and code taking up space to make a strong, robust game. This is just a reality of game development — setting team priorities and negotiating your RAM budget. We found some creative solutions in sharing sounds and using custom downsampling for just about every sound in the game, balancing size versus quality. Even though I’m an audio guy, I’m a gamer too, so I would rather play a fun game that sounds OK than a lame game that sounds great. Fortunately, I think this team did both.

CDM: What were some of the biggest challenges in working on this title in particular?

MY: The big challenge was re-creating the chaos and epic scale of WWII. We had a limited number of sounds loaded at any given time, and we had to get very creative about how we used and re-used the loaded sounds. Drew was especially innovative here. He made an elaborate template for ambient sound beds that stitches together all the gunshots and explosions we have loaded, playing them in random sequences, random volumes, and random pitches, so you never hear the same thing twice. He also made versions of this for inside and outside, so the ambience will dynamically change when the player goes in or out of a bunker. Additionally, he made varying intensity levels for these ambience templates, from which the level designers could choose, using just a number in a text file. It was a lot of work, but the results are realistic and exciting, and ultimately help make it feel like you’re in the war.

DC: The console and PC versions of Call of Duty have many non-player characters that add gunshots and explosions. These games can also use long, pre-mixed loops that stream from disk, and in most cases have larger memory budgets. For PSP, we were limited to 6 players on screen at one time, so I could not rely on a large sounding war from realtime game events. Instead, I had to create a dynamic ambient track using pre-existing and pre-loaded sounds. I did this by creating a preset engine that controlled a wide range of single shot and looped sounds and varied many parameters in real time. Several explosions, guns, rumble and vehicles are orchestrated on the fly. I can change how the sounds are triggered by changing trigger speed, probability, length, pan, volume and pitch. There are four presets per level (low, medium, high, mute) and they are selected according to the situation.

CDM: The Call of Duty series is known for its accurate representations of weapons and vehicles. Can you tell our readers a bit about how you started out with this project, in terms of acquiring your source material and defining the palette you’d use for sound design and music?

MY: For starters, we inherited a whole bunch of the awesome weapon and vehicle sounds that Activision used in previous Call of Duty games. That was a crucial starting point for us. These are real recordings of real weapons and vehicles we’re using. We designed the sound behavior of tanks and jeeps by working closely with developers, but honestly, vehicles are not a huge part of our game design, so the files were super small and re-used often. We bought a couple of CD libraries, but we really didn’t use them all that much. We did original recording for Foley sounds, which Ian* and Drew scripted for hundreds of animations. We also created some special effects from scratch, like tinnitus or heart beat when you have low health. *Sound Designer Ian Rodia also worked on Call of Duty: Roads to Victory, but was unavailable for this interview.

CDM: Were there any constraints on sound design, in terms of whether or not your title had to meet certain specifications or requirements to match the console titles?

MY: I played all of Call of Duty and Call of Duty 2 on the PC early in production, and I encouraged all the sound team to do the same. Besides what I’ve already mentioned, the constraints were just that the franchise set an incredibly high bar for us, which is a great problem to have. A great aesthetic precedent was a really pleasant constraint.

DC: Having the original sounds made the process smoother since they were already approved by the publisher. Whenever possible, I tried to stay true to the original games and source material, and Activision was pleased with what we created. In terms of authenticity, there were only a few issues popped up. For example, the firing of the M1 Garand, which needed a ‘ping’ when the gun ran out of bullets.

CDM: Were there any particular tools or applications that you found particularly useful in this title?

DC: I am a big fan of impulsing and convolution reverb. Impulsing allowed me to create convincing environments. Digital reverbs can be too perfect and since Call of Duty is in a real environment, impulses tend to work better. SIR (Super Impulse Reverb) is a handy impulse reverb plugin that allows you to slow down the impulse to exaggerate the reverb. When this happens the delays or slaps become more pronounced.

MY: We’re using proprietary tools for implemeting our audio. There’s a custom Lua script for dynamic music triggers and custom Lua scripts for animation sounds. Aside from those tools, being on good terms with your programmers is always a useful tool! And though it’s not sexy, we created some useful, custom Excel spreadsheets to track level memory budgets.

CDM: What level of communication and interaction do you have with the programmers and designers on a title like this?

DC: There are many audio-specific hurdles to get over for a game like this. The producer understood at the start of the project that audio was going to need more attention. As we created the game we discussed our options. I think it is the sound designer’s responsibility to push the envelope and aim high. Once we understood what we needed, the programmers were very helpful in creating the tools. You always need to leave time in the process for revisions – especially when you’re creating new toolsets. Fortunately we were able to revise and tune the tools with the programmers.

MY: At least one person from the audio team was in every team meeting from early on in pre-production, all the way through the project. We evangelized for robust audio in Call of Duty: Roads to Victory at every turn, and found that basically we were preaching to the choir. Everyone knew that the sound design would sell the realism, and the music and voice would sell the emotion. All the developers were extremely helpful, and the producers gave us the resources and support we needed to make it great. Sometimes we needed custom code hooks to support a special case, and we continually pressed for more memory in levels, but it never got really ugly with the programmers.

CDM: In composing the score for Call of Duty: Roads to Victory, were you working in any particular musical idiom, or did you have to follow the particular feel and flavor of previous titles?

Noel Gabriel: I suppose WWII music has become a musical idiom of sorts, with so many games and movies making use of the subject matter. I’d say the classic WWII score is derived from the music of Jerry Goldsmith (“Patton”, “MacArthur”) and Elmer Bernstein (“The Great Escape”, “Bridge at Remagen”) and you hear this influence in much of the score for Call of Duty 3. Without a doubt the most influential score in recent years was written by John Williams for “Saving Private Ryan”. Call of Duty: Roads to Victory pays homage to this in the music for the American campaign with its use of brass chorale and contrapuntal trumpet passages.

CDM: When you’re composing for a game such as Call of Duty: Roads to Victory, can you tell us a bit about how you ‘design’ the music, in terms of creating musical passages, transitions and segments?

NG: We didn’t rely too heavily on our dynamic music system for Call of Duty: Roads to Victory, but in terms of ensuring pieces joining together nicely, I made sure they all started in the same tonality. And I use the term tonality instead of key, because many of the combat pieces were chromatic enough that you couldn’t say they were in any set key.

CDM: What were some of your biggest challenges in composing the music for Call of Duty: Roads to Victory?

NG: Living up to the precedent established by Michael Giacchino and his use of an 80 piece orchestra recorded in a scoring stage in LA was daunting to say the least. But whether you have the luxury of premier players or have to create an entirely midi based score, the important part is to create an emotional connection between the game and the player. Some of the complex emotions evoked by the music (such as battle fatigue or a sense of loss over fallen comrades) were especially challenging with a midi-based orchestra. But I’m pleased with how it turned out. There was one point where I said “I’ve got to have a real trumpet player” to make the chorale I was writing work. Fortunately I was able to get approval to do so. The majority of the music is midi-based, however.

CDM: What tools do you use for composing?

NG: The sample libraries used for the music were a combination of East West Quantum Leap Symphonic Orchestra, East West Quantum Leap Symphonic Choir, and the Vienna Symphonic Library. The solo trumpet work was performed by Pat Murray.

MY: Noel really knocked it out of the park with his Call of Duty: Roads to Victory PSP score. And while I’m not quite the orchestral maestro like Noel, I do compose for our other games. I’m using Cubase SX with Gigastudio 3, Pluggo, some old E-MU hardware synths, and NI’s Battery, Reaktor and Absynth. My tangible instruments include an Epiphone electric guitar, an Alvarez acoustic guitar, Yamaha drums and dozens of percussion toys. When I write for GBA I use ModPlugTracker, and for DS we’re using Nintendo’s Nitro Composer. At home I’ve been using a ProTools LE rig for a few years now.

CDM: What’s next for the Amaze team?

MY: We are looking forward to some games for the XBox360 and the Wii, and I hope we make more games for the PSP. It’s a strong platform.

Being the C64, you get funny moments like this: “Some obstacles to be worked through include presentation of music given the 3 voice nature of the 6581 and 8580.” Indeed. The graphics are a bit more primitive, as well (ahem). But Guitar Hero with SID files? Priceless.

Just don’t let Timbaland anywhere near this.

Shredz64 Project
Via C64Music!, a superb source for round-the-clock updates on C64 music

(Good peoples of Harmonix, assuming Viacom still leaves you time to read this blog, curious what you think!)

Renowned Nintendo composer Koji Kondo also presented at this year’s GDC, and we’re on the lookout for reports. If you’ve got any, please pass them along.

The UK Sony PlayStation site has posted Josh Randall’s keynote at the Cybersonica arts fest in London. Josh Randall, a sometimes CDM reader (still out there, Josh?), is Creative Director of Harmonix, the groundbreaking game shop that created Guitar Hero and other interactive music games like Amplitude, Frequency (pictured), and (soon) Guitar Hero II.

Interview with Josh Randall [Video, PlayStation.com]

Cybersonica Keynote Excerpts [Video, PlayStation.com]

Via: Interactive Music for the Masses [Pixelsumo, blog of Cybersonica curator Chris O'Shea]

Josh talks in the interview about the upcoming PS3, Cybersonica, and the possibility of other instruments. (Keytar Hero? Accordion Hero? Ukelele Hero? I’m game, Josh!) In his keynote video, he charts the history and past games of Harmonix, talks about the creation of Guitar Hero and the potential of music games, and looks to the future — not only Guitar Hero II, but the future of musical games in general.

Of course, the convergence of music and gaming remains a story that’s dear to our hearts here. That makes it all the more fantastic seeing Guitar Hero hit the big time. It could be just the beginning.