Composer/sound designer Ben Long has a resume of work on dozens of games. Here on CDM, he shares the topic on which he recently addressed GDC China: mobile. If mobile game audio is going to rise to people’s expectations, it’ll have to get past rushed developers and hardware obstacles, including revisiting the whole mono/stereo debate. Ben lets us know his insider take on that landscape, and shares with us the process for designing sounds for virtual slots. Everyone, drone in C with the slot machines… -Ed.

When I tell people “I make music and sounds for video games,” it usually brings responses ranging from people saying “neat!” to blank stares. This is often followed by the person asking “how did you get into that?”  It’s a long story, but I usually say something along the lines of this: “I have musical ADD and video games was a perfect fit!”

For years, I slept on the floor with some makeshift “studio” crammed in the corner.  Working IT jobs during the day and live gigs at night for 10 years paved the way to working on games. My very first ‘studio’ consisted of the Roland VS-880, a digital 8-track with a 2-inch, non-backlit LCD screen.  Mixing on this was thing like brushing your teeth with a tree branch, but hey, it was digital, baby — and this was back when a CD burner was $600.

I got used to being surrounded by bleeding-edge technology and noticed that the game industry was leading the pack.  It wasn’t long before I had my first gig, creating pirate music with MIDI.

So, having worked in the game industry, where does the future lie, and how can someone be prepared to meet it? The answer is clearly mobile.

Mobile Gaming: Challenges and Opportunities

Mobile is by far the fastest-growing sector of the game industry. Never before in history have so many people been exposed to video games. Mobile developers have the ability to take huge “leaps of faith’ of which other developers would not dream. These creative choices can go horribly wrong, but in the case of games like Angry Birds, they can be hugely successful.

The growth of that market is buoyed by an explosion in hardware. New mobile devices appear almost weekly, each promising heightened user experience. Each one has its own set of limitations and capabilities for audio. It’s no surprise that consumers are expecting better audio in their mobile games. The explosion of music-based games has created an interesting phenomenon: people are listening more.

No longer can mobile game developers rely on the visual canvas alone; there is an empty void around the device that can be occupied with captivating content. If a game is performing well in the marketplace, chances are it sounds great. Making audio for games is difficult, but factor in a tiny, mono speaker with extreme hardware/OS limitations and things get hairy. Luckily, mobile phones have rapidly evolved and the days of developers requesting that I create a 4K MIDI score have disappeared. Nevertheless, creating audio content for today’s mobile gaming devices requires a unique skill set.

Today’s mobile games are typically developed at breakneck speeds. As a result, audio often becomes the last ingredient, as developers scramble just to get their game out. To save time, they sometimes buy music tracks or sound effects from an online store. With the right set of ears, this can work, but audio is usually “thrown in” without much thought or testing. Such practices prevent a good game from climbing the charts. There is a reason why film producers hire a composer, sound designer and audio engineer rather than buying canned content – they want the final product to take on a life of its own.

In Living Mono: Starting with the Hardware

When Apple released the iPhone 3, they heralded the new features – including “enhanced mono”. Soon after release, this descriptive term was deleted from the specs. (Am I the only one that noticed the quick removal of that term from their description?) Technically, the iPhone/iPad speakers are not true stereo, but mono. The silky high frequencies make up for it, though – you can lay the device down and enjoy what you are hearing.

Of course, when using headphones you hear everything across the sonic spectrum. One challenge of creating audio content for mobile is taking into account the possibility of headphone usage. The iPad allows for a bit more mid-range frequency content, so I considered this when working with Backflip Studios on their new game “Backflip Slots.”

Developing Backflip Slots

Typically, a mobile game will have a looping background ambience, background music, dialogue and UI/gameplay sounds. For the menu ambience in Backflip Slots, I visited a casino here in Colorado and brought along my Zoom H4 handheld recorder to capture the casino soundscape. Luckily, security didn’t ask any questions as I entered the building with this strange little device. After finding my way to the rows of slots machines, I sat in the middle and started playing. This location gave me a nicely-balanced ambience, so I placed the recorder on the seat next to me and started experimenting with different configurations. All this was done without headphones, so I had to use my best judgement and just go for it.

The funny thing about casinos is that every slot machine plays their jingles in the key of C. The result is a hypnotizing cacophony that keeps the players hooked. I have actually played live gigs at casinos before and been instructed to keep every song in the key of C. This is surreal, much like playing alongside a choir of robots!

Since “Backflip Slots” was getting a more traditional look, we needed the sound to follow suit. For the reel spin, I went for a looping mechanical sound with a subtle friction texture. To achieve this, I combined the sound of a tractor engine with factory machinery samples from my own sound library. The two sounds were then mixed together and combined with elements from Native Instruments’ Sonic Fiction.

Seamless looping is commonplace in games but often brings technical challenges. That can detract from the creative focus, but it’s a necessary evil, given the technical limitations of the hardware platforms. The reel landing sounds needed more of a chunky ‘click’ feel that would not be too overbearing.

Each bonus spin-character icon has an animated sequence in which they come to life and jump off the screen. These actions received everything from an 8-bit flamethrower to the sound of a samurai sword being unsheathed. One of the keys to creating sonic appeal is subtlety and this can require extensive testing. Game sounds should not grate on the nerves even after being heard hundreds of times. This usually entails experimenting with volume, EQ and pitch shifting in the studio. The end result should always be a pleasant listening experience for the development team and ultimately, the player.

The win sequences were also in sharp contrast to the reel spins and button presses. Win animations explode on the screen with coins raining down on top of an animated logo. Add pulsating lasers and lightning strikes and you now have some serious eye candy! These received a good amount of ‘bling’ on the sonic end and went through extensive revisions before completion. Even when being piled on top of each other, each audio file must be audible and clean. Since the game featured all of their IP (Paper Toss, NinJump, Graffiti Ball, etc), I took the existing sounds from other games and remixed them to work within Slots. In the case of “NinJump”, I blended the hiyaahh with a gong cymbal for the combo win. Some of the combinations are buried deep in the game and will only be heard by the top players.

It’s always a blast working with Backflip on these games so stay tuned for some big suprises in 2011.

About the Author

Ben Long is a composer, sound designer and founder of NoiseBuffet. He recently spoke at GDC China on the subject of “creating audio content for mobile games” Ben’s sonic signature can be heard on over 40 games, including the forthcoming release of LodeRunner Mobile. In addition to games, his music is aired on every major TV network and was recently used in the Stevie Wonder Biography on A&E. Ben is a lifelong gamer and has been featured in three game design textbooks. Learn more about game audio and his new ebook at gameaudio101.com.

In your pocket, or perhaps orphaned in your closet, is a sophisticated piece of electronics going to waste. So, whether you’re suffering from iPhone envy or simply want to put toxic used electronics to useful musical applications, the cell phone noisemaking project at GetLoFi could help make a happier, noisier world.

The ingredients:

• Software running on the Java virtual machine (good, old-fashioned J2ME)
• A hack for proper input and output (from the various headset connectors)
• Sequenced MIDI files

The result: otherwise silent, ordinary cellphones become living musical creatures. Install files, plus information on how to make the circuit for I/O, went up on GetLoFi early in October, but thanks to Michael Una for pointing this my way over dinner recently.

Everything you need:
MIDI Loop Sequencer aka Cell Phone Noisemaker

Despite my complaints, you will find some useful music apps in the iTunes App Store – you can at least get some fine tuners. (Andy Ihnatko was excited on Twitter that one of them helps him tune his ukulele, thanks to four string support!) We do expect more hefty music tools in the coming months, and via the jailbroken platform.

But some of the real stars on the iPhone – or whatever your favorite smart mobile device may be – have to do with simply storing ideas and keeping your life together. That means one of the best downloads so far for the iPhone is Evernote. As Graham English writes in comments on CDM:

The app I’m most excited about for music is Evernote. You can record voice notes, text notes, and it even recognizes the text in iPhone pictures. So next time you write a killer hook on a bar napkin, snap a picture and sync it. Cool.

I’m a huge user of Evernote on my desktops and, via its web browser, on my Blackberry. The iPhone app looks especially great, though; I’m jealous.

Evernote for iPhone

My friend Francis Preve has written a whole set of useful tips for DJs that apply to any gigging musician / artist (which he’s been refining since the first iPod, in fact):

Top 10 iPhone tricks for DJs [Beatportal]

Some highlights:

• Keeping email templates for gig announcements on your device so you can send them quickly. (Recognize this scenario? “Hey, what’s new?” “I’m playing Friday.” “Oh, really?”)
• Mapping: Some providers require you to enable mapping capabilities on your device. Do it. Both the “real” GPS (via a dedicated radio) and the assisted GPS can be lifesavers if you travel at all. (The iPhone 2.0 update adds this feature, in the assisted form.)
• Rescue tracks: The iPhone is a capable music player, so it can, um, save you when your laptop dies or someone steals your hard drive. It even has video out capability, for you Create Digital Motion readers.

And the fact is, you can easily apply these ideas to whatever phone / smart device is your favorite. These could also put you over the top as far as cameras with phones – not because you’ll necessarily be taking attractive photos (dedicated cameras are a must for that) but because they can aid visual memory. (Airport parking spot? Check!)

I especially like Fran’s idea of logging creative time. Invoicing aside, I find that it’s so often a fight to get time to yourself that I really like the idea of keeping track of that time, whether it’s in the studio or just sitting in the park thinking about a new song. It could help give you some positive reinforcement for setting aside some working time or even badly-needed quiet time.

Going back to my first Palm (the PalmPilot Professional, no less), I’ve always found even simple mobile devices can help reduce stress, particularly on the road. And that to me is priceless.

Other ideas that boost your productivity, in terms of gigging, travel, and creativity? Let us know.

Previously:

iPod Touch/iPhone for Music Round-up (which, surprisingly, isn’t all that outdated by the App Store launch – we expect bigger announcements in the coming weeks)