It’s official: we had heard rumblings that game maker Harmonix was about to announce something, and it’s here. It’s a collaboration directly with the Beatles to make something that isn’t Rock Band or Guitar Hero — something completely new. And something completely new is exactly what’s needed.

Before Guitar Hero and Rock Band, before being purchased by MTV/Viacom, game developer Harmonix were a very different creative house. Co-founders Alex Rigopulos and Eran Egozy were MIT friends whose first project was an application that let you play guitar with a joystick. (Sounds like a research project you might read about here.) Their interactive music games were influenced by the explosion of Japanese titles like PaRappa the Rapper and Beatmania, to be sure. But part of what made FreQuency and Amplitude so important was that they offered more than just a simplified music experience. They were digitally-powered acid trips, with VJ-style video clips playing up buildings and surprisingly sophisticated interfaces that remixed the music as you played.

Make no mistake about it: Guitar Hero and Rock Band are brilliant titles with a fair dose of musical integrity in the way they abstract playing experiences for broader audiences. But there’s no question some of the original creativity — the sense that the game experience was unlike any other experience — is missing. And in this pumped-up HD age, in which surreal game experiences like intra-dimensional navigation in Portal or ambient floating cartoon paramecia in Spore, it’s hard to wonder if gamers who weren’t ready to snap up FreQuency a few years ago might be ready now.

So while rival Activision bakes a watered-down GarageBand-style app into another iteration of Guitar Hero, it’s intriguing, at least, that Harmonix is working with the Beatles. And they really are working with surviving Beatles and Beatles Significant Others: Sir Paul McCartney and Ringo Starr, Yoko Ono Lennon, and Olivia Harrison. (Okay, I’d like to see a special Yoko-inspired game on Xbox Live Arcade.) Most interesting, producer Giles Martin, heir to production legend Sir George Martin
and producer of the Love project with Cirque due Soleil, twice a Grammy winner, and the man behind The Beatles Anthology is involved, too. (See a great story on him in Sound on Sound.)

Let’s get straight to the point: for the band that made virtual acid trips mainstream decades ago, it’s time for a new, digital trip. (They do describe it as a “journey” through the Beatles’ work, after all.) I think the Beatles make a perfect choice. I can’t count the number of people I know in music composition who were addicted to Beatles records as kids — not the Beatles’ generation, but their offspring in the 80s and 90s.

And despite the intervening decades, Yellow Submarine still looks imaginative and bizarre. If gaming can do anything, it can take music we’ve heard a zillion times and make it new. It can make our regular experience, the reality around us feel a little different. Rock Band has proven to be a trojan horse: it’s literally driven up sales of real instruments. That’s proof that making something palatable to a mass market can help get them hooked on new kinds of experiences. Can a Beatles game feel less like interactive documentary or re-hashed Guitar Hero, and more like a groovy, retro journey into the strange imagination that turned a lot of us on to recording, music, visuals, and … uh … animations of strange creatures? I think so. Can’t wait to see what comes out.

PS — I want to play as George.

Kotaku gives us a video interview, here, while Music4Games gives us a written one here.

CDM recently had the opportunity to speak with some of the audio designers for Call of Duty: Roads to Victory for PSP. Mark Yeend is the Audio and Music Manager for Amaze Entertainment, Drew Cady was the lead sound designer, and composer Noel Gabriel, whose scores for Amaze have been recently compared to (confused with?) Hans Zimmer.
(Disclosure: Though I did not work on this title, these guys are my coworkers.)

Create Digital Music: Can you please familiarize our readers with the platform specifics of the PSP?

Mark Yeend: In terms of making games, it’s a lot like the PS2. You’ve got resident per-level-load RAM for sounds, and you also stream data off the UMD disc during runtime, which can be used for a music stream or for buffering sound data. The PSP speakers are bright and loud, and the steaming music format is Sony’s ATRAC3 codec, which sounds fantastic. Aside from gaming, the PSP can store, play and manage your music library, link to your PC, and play movies on the go (which look and sound great). I have really been impressed working on the PSP for the last few years.

Drew Cady: 32 channels of audio can be played at the same time, and believe me we use all of them in this game. Sampling rates can be relatively high, but at the cost of memory. The onboard speakers roll off below 500hz (-33db at this point), but I still try to keep the low end frequencies for headphone listeners.

CDM: Can you describe for our readers some of the primary constraints and limitations you had to work with in making this particular title for the PSP?

DC: RAM is the biggest constraint. The first thing I do on a project is lock down how much memory I will receive for audio. This helps me scale each section of audio. For example – level ambience, level effects, weapons, player character, non-player characters, global sounds, dialog and vehicles. For Call of Duty, the audio memory budget for each level was around 2.8MB. To help juggle memory I can downsample any file independently. Distant sounds can use a lower sample rate of 17k or 14k while first person gunshots are sampled higher at 28k. After creating a sound, I usually listen to it downsampled. For instance, a chain sound effect needs to be tweaked so they can be heard in game. If you downsample to 22k much of the high frequency detail is lost. If you raise the sample rate to 28k the high frequencies are not cut off and you can hear the jingle. Alternatively, you can also lower the pitch by two half steps and keep the sample rate at 22k. It takes a bit of experimentation to really find the sweet spot.

MY: As Drew noted, the two biggest limitations were the frequency response of the PSP speakers, and the audio RAM budget in each level. How do you create an earth-shattering explosion or the rumble of a Tiger tank on quarter-inch speakers? You can’t. We’re counting on people using headphones, but the deep sounds have to be audible without headphones too, because they often affect gameplay. For example, they’ll let you know when and from where the enemy tanks are coming. Our typical solution was to add a bit of mid-frequency content to the deep sounds; just enough to make them present on speakers without changing the character and realism of the sound. For RAM, despite our producers and programmers acknowledging how critical audio is for Call of Duty, we ended up with about 60% of the RAM originally allocated for audio. It was a really tight squeeze, and it’s a symptom of art and geometry and code taking up space to make a strong, robust game. This is just a reality of game development — setting team priorities and negotiating your RAM budget. We found some creative solutions in sharing sounds and using custom downsampling for just about every sound in the game, balancing size versus quality. Even though I’m an audio guy, I’m a gamer too, so I would rather play a fun game that sounds OK than a lame game that sounds great. Fortunately, I think this team did both.

CDM: What were some of the biggest challenges in working on this title in particular?

MY: The big challenge was re-creating the chaos and epic scale of WWII. We had a limited number of sounds loaded at any given time, and we had to get very creative about how we used and re-used the loaded sounds. Drew was especially innovative here. He made an elaborate template for ambient sound beds that stitches together all the gunshots and explosions we have loaded, playing them in random sequences, random volumes, and random pitches, so you never hear the same thing twice. He also made versions of this for inside and outside, so the ambience will dynamically change when the player goes in or out of a bunker. Additionally, he made varying intensity levels for these ambience templates, from which the level designers could choose, using just a number in a text file. It was a lot of work, but the results are realistic and exciting, and ultimately help make it feel like you’re in the war.

DC: The console and PC versions of Call of Duty have many non-player characters that add gunshots and explosions. These games can also use long, pre-mixed loops that stream from disk, and in most cases have larger memory budgets. For PSP, we were limited to 6 players on screen at one time, so I could not rely on a large sounding war from realtime game events. Instead, I had to create a dynamic ambient track using pre-existing and pre-loaded sounds. I did this by creating a preset engine that controlled a wide range of single shot and looped sounds and varied many parameters in real time. Several explosions, guns, rumble and vehicles are orchestrated on the fly. I can change how the sounds are triggered by changing trigger speed, probability, length, pan, volume and pitch. There are four presets per level (low, medium, high, mute) and they are selected according to the situation.

CDM: The Call of Duty series is known for its accurate representations of weapons and vehicles. Can you tell our readers a bit about how you started out with this project, in terms of acquiring your source material and defining the palette you’d use for sound design and music?

MY: For starters, we inherited a whole bunch of the awesome weapon and vehicle sounds that Activision used in previous Call of Duty games. That was a crucial starting point for us. These are real recordings of real weapons and vehicles we’re using. We designed the sound behavior of tanks and jeeps by working closely with developers, but honestly, vehicles are not a huge part of our game design, so the files were super small and re-used often. We bought a couple of CD libraries, but we really didn’t use them all that much. We did original recording for Foley sounds, which Ian* and Drew scripted for hundreds of animations. We also created some special effects from scratch, like tinnitus or heart beat when you have low health. *Sound Designer Ian Rodia also worked on Call of Duty: Roads to Victory, but was unavailable for this interview.

CDM: Were there any constraints on sound design, in terms of whether or not your title had to meet certain specifications or requirements to match the console titles?

MY: I played all of Call of Duty and Call of Duty 2 on the PC early in production, and I encouraged all the sound team to do the same. Besides what I’ve already mentioned, the constraints were just that the franchise set an incredibly high bar for us, which is a great problem to have. A great aesthetic precedent was a really pleasant constraint.

DC: Having the original sounds made the process smoother since they were already approved by the publisher. Whenever possible, I tried to stay true to the original games and source material, and Activision was pleased with what we created. In terms of authenticity, there were only a few issues popped up. For example, the firing of the M1 Garand, which needed a ‘ping’ when the gun ran out of bullets.

CDM: Were there any particular tools or applications that you found particularly useful in this title?

DC: I am a big fan of impulsing and convolution reverb. Impulsing allowed me to create convincing environments. Digital reverbs can be too perfect and since Call of Duty is in a real environment, impulses tend to work better. SIR (Super Impulse Reverb) is a handy impulse reverb plugin that allows you to slow down the impulse to exaggerate the reverb. When this happens the delays or slaps become more pronounced.

MY: We’re using proprietary tools for implemeting our audio. There’s a custom Lua script for dynamic music triggers and custom Lua scripts for animation sounds. Aside from those tools, being on good terms with your programmers is always a useful tool! And though it’s not sexy, we created some useful, custom Excel spreadsheets to track level memory budgets.

CDM: What level of communication and interaction do you have with the programmers and designers on a title like this?

DC: There are many audio-specific hurdles to get over for a game like this. The producer understood at the start of the project that audio was going to need more attention. As we created the game we discussed our options. I think it is the sound designer’s responsibility to push the envelope and aim high. Once we understood what we needed, the programmers were very helpful in creating the tools. You always need to leave time in the process for revisions – especially when you’re creating new toolsets. Fortunately we were able to revise and tune the tools with the programmers.

MY: At least one person from the audio team was in every team meeting from early on in pre-production, all the way through the project. We evangelized for robust audio in Call of Duty: Roads to Victory at every turn, and found that basically we were preaching to the choir. Everyone knew that the sound design would sell the realism, and the music and voice would sell the emotion. All the developers were extremely helpful, and the producers gave us the resources and support we needed to make it great. Sometimes we needed custom code hooks to support a special case, and we continually pressed for more memory in levels, but it never got really ugly with the programmers.

CDM: In composing the score for Call of Duty: Roads to Victory, were you working in any particular musical idiom, or did you have to follow the particular feel and flavor of previous titles?

Noel Gabriel: I suppose WWII music has become a musical idiom of sorts, with so many games and movies making use of the subject matter. I’d say the classic WWII score is derived from the music of Jerry Goldsmith (”Patton”, “MacArthur”) and Elmer Bernstein (”The Great Escape”, “Bridge at Remagen”) and you hear this influence in much of the score for Call of Duty 3. Without a doubt the most influential score in recent years was written by John Williams for “Saving Private Ryan”. Call of Duty: Roads to Victory pays homage to this in the music for the American campaign with its use of brass chorale and contrapuntal trumpet passages.

CDM: When you’re composing for a game such as Call of Duty: Roads to Victory, can you tell us a bit about how you ‘design’ the music, in terms of creating musical passages, transitions and segments?

NG: We didn’t rely too heavily on our dynamic music system for Call of Duty: Roads to Victory, but in terms of ensuring pieces joining together nicely, I made sure they all started in the same tonality. And I use the term tonality instead of key, because many of the combat pieces were chromatic enough that you couldn’t say they were in any set key.

CDM: What were some of your biggest challenges in composing the music for Call of Duty: Roads to Victory?

NG: Living up to the precedent established by Michael Giacchino and his use of an 80 piece orchestra recorded in a scoring stage in LA was daunting to say the least. But whether you have the luxury of premier players or have to create an entirely midi based score, the important part is to create an emotional connection between the game and the player. Some of the complex emotions evoked by the music (such as battle fatigue or a sense of loss over fallen comrades) were especially challenging with a midi-based orchestra. But I’m pleased with how it turned out. There was one point where I said “I’ve got to have a real trumpet player” to make the chorale I was writing work. Fortunately I was able to get approval to do so. The majority of the music is midi-based, however.

CDM: What tools do you use for composing?

NG: The sample libraries used for the music were a combination of East West Quantum Leap Symphonic Orchestra, East West Quantum Leap Symphonic Choir, and the Vienna Symphonic Library. The solo trumpet work was performed by Pat Murray.

MY: Noel really knocked it out of the park with his Call of Duty: Roads to Victory PSP score. And while I’m not quite the orchestral maestro like Noel, I do compose for our other games. I’m using Cubase SX with Gigastudio 3, Pluggo, some old E-MU hardware synths, and NI’s Battery, Reaktor and Absynth. My tangible instruments include an Epiphone electric guitar, an Alvarez acoustic guitar, Yamaha drums and dozens of percussion toys. When I write for GBA I use ModPlugTracker, and for DS we’re using Nintendo’s Nitro Composer. At home I’ve been using a ProTools LE rig for a few years now.

CDM: What’s next for the Amaze team?

MY: We are looking forward to some games for the XBox360 and the Wii, and I hope we make more games for the PSP. It’s a strong platform.

Being the C64, you get funny moments like this: “Some obstacles to be worked through include presentation of music given the 3 voice nature of the 6581 and 8580.” Indeed. The graphics are a bit more primitive, as well (ahem). But Guitar Hero with SID files? Priceless.

Just don’t let Timbaland anywhere near this.

Shredz64 Project
Via C64Music!, a superb source for round-the-clock updates on C64 music

(Good peoples of Harmonix, assuming Viacom still leaves you time to read this blog, curious what you think!)

Renowned Nintendo composer Koji Kondo also presented at this year’s GDC, and we’re on the lookout for reports. If you’ve got any, please pass them along.

The UK Sony PlayStation site has posted Josh Randall’s keynote at the Cybersonica arts fest in London. Josh Randall, a sometimes CDM reader (still out there, Josh?), is Creative Director of Harmonix, the groundbreaking game shop that created Guitar Hero and other interactive music games like Amplitude, Frequency (pictured), and (soon) Guitar Hero II.

Interview with Josh Randall [Video, PlayStation.com]

Cybersonica Keynote Excerpts [Video, PlayStation.com]

Via: Interactive Music for the Masses [Pixelsumo, blog of Cybersonica curator Chris O'Shea]

Josh talks in the interview about the upcoming PS3, Cybersonica, and the possibility of other instruments. (Keytar Hero? Accordion Hero? Ukelele Hero? I’m game, Josh!) In his keynote video, he charts the history and past games of Harmonix, talks about the creation of Guitar Hero and the potential of music games, and looks to the future — not only Guitar Hero II, but the future of musical games in general.

Of course, the convergence of music and gaming remains a story that’s dear to our hearts here. That makes it all the more fantastic seeing Guitar Hero hit the big time. It could be just the beginning.

It could be an isolated wonder, were it not for the entirely open source nature of the project. The code for Fijuu itself is open (though primarily as a model, as it’s built for this specific installation), and everything in it was creating with open source software, from Debian Linux to the phenomenal OGRE 3D engine that handles the 3D magic. Figuring out how music should translate to three dimensions, both in terms of visuals and interface, is no small problem. Open sourcing the results is essential if other artists are to take this idea and develop it further. In the meantime, Fijuu is incredibly gorgeous, which is why I’ve officially dubbed it “the hotness” on behalf of the CDM staff.

For more on Fijuu, see the official project page (with source), an interview with the creator in the Cybersonica video, and further discussion from Cybersonica’s curator, Chris O’Shea, at his blog, Pixelsumo.

Guitar Hero [Official Game Site]

As we reported previously, the game is the work of our friends in Boston at Harmonix, the same developers of other musical/interactive games like Frequency, Amplitude, Karaoke Revolution and EyeToy: AntiGrav. The hardware controller is the work of RedOctane, the favored maker of PS2 dance pads, and even Gibson Guitar has gotten in the act by licensing classic guitar models.

What makes Guitar Hero so much fun is the PS2 controller it ships with. Instead of mashing buttons on a game controller, you can strap this toy on your shoulder and “strum” a paddle where the strings would normally be. No real frets, just buttons, but it’s still strangely amusing thanks to clever game design — see Gizmodo’s recent hands-on. One controller ships with the US$70 game; another one will cost you $40.

Now, for the part you really care about, which doesn’t even require a PlayStation. PlayStation – to – USB adapters are easy to find these days, so you can plug this baby directly into your Mac or PC. (Make sure you don’t confuse that with a PS/2-to-USB adapter, as in IBM’s old PS/2. Look for PSX to USB.) Then you need some way of handling the data. The easiest is to use the “hi” (human interface) object in Cycling ’74’s Max/MSP, which is also built into their new Hipno plug-in library. In the spring, you could download a free executable Max patch called PSmaX that did the work for you with or without Max, but that link is now defunct. So, you could bug me to create a new one and share it. Let the bugging commence.

And soon we’ll all be jamming using a fake toy guitar. Excellent.