Lured by the siren song of modular synthesis and DIY electronics, but not sure how to navigate the piles of requisite knowledge – or uncertain what the trip down this rabbit hole might have in store?

For years, Tom Whitwell’s Music Thing was a beloved daily read, as that site and this one were among the early blog-format destinations for music tech. Tom moved on – something about a major day-gig at a paper called The Times, perhaps named after the font? – but that makes us all the more delighted to get a dispatch from him. In this guest column for CDM, he introduces one project, a brilliant FM radio sequencer, but also helps us catch up on reading on modular synthesis and electronics dating back to the origins of the technology. And he has a realistic look at what this will do to your life – all inspired by “pure enthusiasm,” as he puts it, “this is fun, you should try it.”

Hey, isn’t that what the drug dealer said in those just-say-no instructional videos we watched in the 80s? Coincidence, I’m sure. -PK

Since buying a Eurorack modular synth a year ago, I’ve spent a lot of time building DIY synth modules and reading about synths and the people who build them. (See reading list, below, if you’d like to do the same.)

The hardest part of DIY electronics is starting out. My first step was building a few guitar pedal kits and learning by reading the Beavis Audio site. Other people start with noisemaker kits like the Atari Punk Console or circuit bending. They all lead in the same direction — down a very deep rabbit hole. 

There’s a lot to buy – a kind of infrastructure you need before doing anything – soldering kit, a multimeter, and a stock of components. None of it costs much, but it’s hard and disconcerting to buy. Online megastores like Farnell or Mouser will stock 50 versions of every component. Get the part number wrong, and you accidentally order capacitors as small as grains of sand, or as large as golfballs. Smaller stores – in the UK, I use http://www.bitsbox.co.uk/ - are easier because they only stock common hobby-friendly parts. 

After making a few guitar pedals, I moved onto synth modules. They’re a great DIY platform. The infrastructure is all there, in terms of power supply, case, inputs, and outputs. Parts are cheap, there’s a healthy and helpful community, and a nice learning curve, from basic utility modules to mind-bendingly complex frequency shifters and vocoders. 

In a year, I’ve built:

• a super-simple, chiptuney oscillator
• a tiny spring reverb driver
• a stupidly-complicated and blinkenlights-covered Arduino-powered Euclidean beat sequencer
• a very useful Arduino MIDI clock
• and a simple but handy 8-step sequencer (see video, below)

For this project, I was inspired by this quote from Don Buchla, the legend of west coast synthesis: 

“My studio at that time was ten feet wide. It was so crowded in there we hauled the workbench out on the sidewalk on good days and set up my oscilloscope and worked out there. [John] Cage came by and for voltage control I had hooked up my keyboard to an FM module that I’d built, a little module that was an FM receiver and I could play stations on it because I had one of the first varactor tuned FMs. Cage, as you can imagine, was just enormously interested in the fact that I could tune each key to a station and then proceeded to play the radio” ( Source [PDF] )

Thirty years later, Don released the 272e module (see Matrixsynth on the announcement), a $1250, four-channel polyphonic FM Tuner. There’s also the ADDAC102, a very fancy stereo €270 Eurorack module [see Synthtopia, with a video]. I wanted something quick, cheap and easy that would let me follow in Don and John’s footsteps. After a lot of searching and a few dead ends, I found the wonderful video demo, below, of a battery-powered FM sequencer based on a €15 radio kit from Germany. 

Projects like this follow a predictable curve. There’s a burst of experimental excitement at the start; receiving the crucial part, building the circuit on breadboard and realizing that — YES! — it’s going to work. 

Then comes a period of frustration and tedium. Re-buying a crucial part you blew up. Fiddling with the circuit so it responds just how you want it. Transferring the breadboard layout to a piece of perfboard, or designing a PCB and waiting for it to be made in China. If you’re using an Arduino or other programmable controller, there’s a long period of writing code, battling feature creep, debugging. 

During this period, you have to really, really want the thing you’re making, dreaming of how cool it will be, how much fun you’ll have playing it and telling everyone about it. 

Tom's FM radio-sequencing module project, in all its glory.

Building music gear is more multidisciplinary than you might imagine. The interface and the feel is as important as the functionality. My Euclidean sequencer is a cool-looking thing, with a big LED matrix. It’s really useful – turning trains of pulses into Afro-Latin rhythms. But it’s fiddly and annoying to use. The FM Radio module could be 50% smaller – and size is important in any modular synth – but this time I wanted good big knobs for fine tuning the signals and control voltages. 

So, as the project continues, you’ll spend time designing a front panel, deciding how many knobs you need, removing ones you’ll never use. And along the way, you’re learning. This time round, I wanted to get the control just right – precise, stable tuning so that stations would stay locked. That meant experimentation and [asking for help on the MuffWiggler forum]. I also spent ages reading ham radio sites, trying to work out how to make a voltage-controlled Shortwave radio (I gave up). 
Eventually, the lacquer is dry on the panel, the parts are all in, debugging is complete and the module is working. The result: either elation and fun, or almost immediate maker’s remorse. It’s bad enough spending money on a piece of music gear that you never love. It’s really annoying spending time building one that you can’t then flip on eBay. 

So far, this FM module is pure fun, an injection of random audio in the heart of the system. Every time I turn it on, something else comes out – pirate dubstep stations, Turkish music, news reports and Bryan Adams. You can filter it, sequence it, use it as a noise source, or let it modulate oscillators or open filters. Listen:

Radio sequencer 2 by MusicThing

Photos of the module:

Reading List

Great online resources for learning about modular synths and the first golden age of experimental electronic music include: 

Ubuweb’s electronic music resources section 
Also at Ubuweb, several editions of Electronic Music Review, a beautifully-designed but short-lived journal boasting Robert Moog as Technical Editor. 

The Red Bull Music Academy includes long, detailed interviews with Don Buchla, Tom Oberheim, Peter Zinovieff of EMS, Robert Moog and Morton Subotnik.

Synapse magazine was a mid-70s journal of electronic music, where you’d find DIY projects from people like Serge Tcherepnin

Vasulka is a huge and rather poorly-organised archive of documents, interviews and transcripts, containing some gems.

Source Magazine was, back in California in 1967, a plush avant-garde journal. Many editions came with 10″ vinyl records, pages printed on transparencies or fur. John Cage was a guest editor, and the magazine carried experimental scores from composers like Steve Reich. Original copies sell for $500+, but the articles and scores have been collected in a book: 
Source: Music of the Avant-garde, 1966-1973 [Amazon]

What’s Next?

Tom is already on to the next build since he finished up the radio sequencer. This time, it’s a shift register sequencer. A what?

A 16-step random sequencer, something between the Wiard Noise Ring, the CGS Gated Comparator and Nav’s BITSY.

It takes random noise to fill up 4 x 4 step 4015 shift registers, shifted by a clock input. The shift registers are looped – either after 8 or 16 steps. 8 of the steps are fed into a DAC0800 analog/digital converter, which produces a 0-8 volt output.

See also the prototype:

For the last few decades, generally speaking, we’ve had computers, and we’ve had physical, modular, analog gear. Computers are endlessly patchable, but not using physical cords. Modulars use physical cords, but they lack the flexibility (and affordability) of a computer.

Now, US$25 and an Arduino can change that.

rePatcher is a simple, tangible modular interface for computers. It could work with any software, but right out of the gate it already works with two popular (virtual) patching environments, Max/MSP and the free and open source Pure Data (Pd). You use physical patch cords to make connections, and those connections are reflected in the patch you see on the screen. The patch cords are coupled with requisite encoders for dialing in additional parameter changes. (Reason comes up as a possible candidate for additional compatibility, which would, of course, be really sweet.)

rePatcher is built as a shield for Arduino, so you’ll need one of those, but that still keeps the price low enough to say I absolutely have to have one of these right now.

It’s not the first attempt to do something like this, but it might be the most accessible and affordable – and interesting. And while those cute little patch cords are fun, there’s nothing stopping someone from building on this idea and going to bigger cords and something more extensive than this 6×6 matrix.

Best of all: the magic happens entirely over USB, so if you want to make this work with something else – say, your favorite VJ software – you can do so with anything that can communicate over serial.

More information:
http://www.openmusiclabs.com/projects/repatcher/

For fans of slicing, dicing, glitching, reversing, and shuffling incoming audio streams, this Max for Live Device is for you. Shuffler 2.0 is the latest in a series of “modular” Max for Live devices from developer Isotonik Studios. Mappable to MIDI, the suite of Devices focuses on simpler tasks in ways that can be combined. There are interactive Follow Actions, for instance — a feature I’ve long argued should be native to Ableton Live — plus tools for more easily mapping MIDI to envelopes. There’s a convenient Looper.

From last week, there’s a module called Smart, capable of mapping some eight macros to one knob.

The entire series is sold as a subscription for 17 quid:
http://isotonikstudios.com/modular/

Check out the Follow actions in the video below.

Truly generative musical scores in games have been few and far between, and “music games” has traditionally meant arcade-style rhythm games in which you repeat phrases or whole songs as accurately as possible. Pugs Luv Beats breaks those molds. Part of a vanguard of new gaming creations that generate dynamic music on the fly, it marries grid-based sequencing and resource-gathering gaming, as music making and gameplay blur together. The interactively-produced music could itself become a new way of delivering a musical signature with sound packs.

And beneath it all lurks a free and open source library, libpd – the embeddable version of tried-and-true free graphical music environment Pure Data. (That library is now on GitHub, and vastly updated, by the way, and we’re expecting a book soon from the library’s principle author Peter Brinkmann.)

Oh, yeah, and don’t forget about some seriously addictive gameplay and adorable pugs. I’m suddenly not concerned about the 15 hours Europe-to-North-America travel I’m doing tomorrow.

Here’s what the gameplay looks like, since it’s much easier to see:

Pugs Luv Beats was just approved on the iTunes App Store for iPhone and iPad.

Co-creator Yann Seznec (The Amazing Rolo) is a terrific musician; I just caught up with him in Edinburgh and Berlin and watched him play a homebrewed pig gut instrument with Matthew Herbert for the performance piece “One Pig,” on tour at Berghain. Working with Pd allowed Yann to focus on those musical impulses and not just engineering, and to let him try things he otherwise would never have imagined on a mobile title. So I asked Yann to walk us through how the project was built. He responded with an exhaustively-detailed examination of the evolution of this title, right down to the Pd patches. (Click through for high-res versions.) If your New Year’s Resolution is doing something with patching, you might want to hang onto these answers. Here’s Yann:

The origins of Pugs Luv Beats date back about two years. After making [musical iPhone game] Mujik, Jon (Jonathan Brodsky, aka jonbro) and I were trying to think of other approaches to music mobile app design, and we started thinking more and more about games. Music games, as a whole, are an oddly passive and traditionalist experience – you play along with a premade track, and you are judged on your accuracy and flair (which is strangely reminiscent of music conservatory mindset…). Obviously there are exceptions (RjDj’s Dimensions, Elektroplankton, etc.), but there you go.

Particularly interesting to me was the idea that game mechanics are often very similar to compositional techniques. So for example, when Sonic runs at a normal speed he collects rings at one rate. However when he powers up and goes super fast, he collects rings at a much higher rate. This could be compared to introducing a melody and then speeding it up – and when there are two players, doing this with two melodies. Instant fugue!

We started looking at how we could make a music game where the music and the game elements were fully intertwined and augmented by each other. So Jon prototyped a space shooter drum machine. It was awesome.

To make a (very very very long and boring) story short, our idea and prototype landed us some funding from Channel 4 and Creative Scotland to work on games that focus on musical creativity and composition.

For various reasons, we decided to put aside the space shooter drum machine for a while, and start from scratch. After going through several full prototyping iterations we eventually settled on a core game mechanic that turned out to be in many ways similar to a Tenori-on [Yamaha grid instrument]/Boiingg-style [monome hardware patch] music generation system – in our final prototype, you controlled a series of little dots that moved around the screen, creating loops. This is super fun from a musical perspective because it’s easy and rewarding within a few seconds, and when you have several loops going it can gain some pretty serious rhythmic and melodic depth.

The key from there for us was turning this into a game. We had been using free Internet graphics packs up until then (we hadn’t hired our artist Sean yet) which featured a ladybug, so we had been referring to the main characters as ‘bugs’. During some discussion one of us accidentally said ‘pugs’, and the game idea was born. We constructed a story about pugs and their love for beets (like the vegetables) which create beats (ha!), and how their love turned into greed and got out of control, destroying their world. The game, therefore, is about helping the pugs rebuild their lost civilization by guiding them to create beats. You grow your galaxy by collecting beats, which you do most efficiently when you dress your pugs up in costumes. What’s not to like?

To get to the part that I imagine CDM readers are most interested in, the app development was done by Jon using openFrameworks, [lightweight language] Lua, our own game engine called Blud, and the audio is all done in Pure Data using libpd (through ofxPd). In hindsight we started using libpd really late in the game, just at the very end of the prototyping stage, which was rather silly. Our adoption of libpd basically made our dev cycle about a million times more efficient. My background is as a musician and sound designer, and I have very little coding knowledge. I do, however, have lots of knowledge of Max/MSP, so picking up Pure Data was pretty easy. This allowed Jon to completely pass off all the audio processing (not to mention aesthetic sound design choices) to me, saving him loads of time, giving me direct control over the sound, and letting me test and prototype different approaches to audio within an environment that I knew would be recreated in the game. Also, as Jon mentioned to me recently, by using PD we are able to take advantage of 20 years of audio DSP research and development. Pretty amazing.

How it all works:

The entire audio engine is contained within this patch. Pardon the messiness.

The simplest part of the patch is the “sounds” section, which is used to playback simple sound effects, for the most part linked with interface actions in the game. I did this by creating a very simple patch which plays a sound when it receives a bang. Which sound it plays is dictated by the argument (in this case, the sound of discovering a new capsule). The process for adding a new sound, then, is as simple as adding the sound file to the /assets/sounds/ folder, and making a new instance of “sounds.pd” and naming it the same as the new sound. Jon, in the project code, created a list called “sounds” which is sent into Pure Data. When that list contains “capsule”, a bang is sent into that subpatch, and the sound is played.

A more complex version of what could be done with this type of data is seen in the voice of Mr Puggles, who helps you learn how to play the game. Mr Puggles pops on and off the screen to guide you through the first few worlds, and when he does he send Pure Data a “puggleShow” and “puggleHide” signal. I wanted to give Puggles a funny synthesizer voice that was different every time – dead simple in PD. To do that, I take the puggleShow bang and use it to trigger five more bangs, spaced out over a second. Each of these bangs triggers a random number which is translated into a MIDI note. This note controls the pitch of two oscillators (a sine and a sawtooth), one of which is slightly modified to make them slightly different pitches. These are played through a short volume envelope and a filter which is also controlled by a random number generator. Result? Hilarious beeping boopy Mr Puggles voice, all coming from one bang.

Every time a player buys or selects a planet, a short list is sent to Pure Data comprised of the planet BPM and a random number seed. The BPM is used to calculate delay times and such, and the random number seed is used to create a sort of musical identity for the planet. This is done by choosing a “beat library” and a musical mode.

The mode is created by building a lookup table that chooses the notes from a chromatic scale that would be used in a particular mode. For example, a major scale (ionian mode) uses notes 1, 3, 5, 6, 8, 10, and 12. Each melodic sound library I used is comprised of a full chromatic octave, and the notes that are played on any given planet are controlled by this table. This ensures not only that all of the different sound libraries being played on a planet will be in the same key, but also that a planet will have a strong melodic identity.

The sound libraries in the game are all controlled by the pugs on the planets. As they run around, each time they land they will trigger a sound. The type of sound is dependent on what terrain they are on – thus, if they run through the snow they play a toy piano, if they run through lava a distorted guitar, etc. There are two states of playing the sound, one if the player deliberately tells the pug to go to that tile, and the second if the pug is traveling over that tile to get somewhere else. It’s super easy to do that kind of thing in Pd; just set up two different ‘play sound’ envelopes, maybe a little extra delay or reverb, and you’re done!

The final piece of the puzzle for making the pugs running around into music is to make each tile be a different note. The terrain of each planet is created by making a sort of height map, where different heights correspond with different terrain types (grass, water, snow, etc). This also means that each tile has a unique number between 0 and 1. When the player buys or selects a planet, a giant random number table is generated in Pure Data which creates a number between 1 and 13 for each possible value between 0 and 1. That value is what is used to pick the note of the mode. This somewhat convoluted approach again lets us make sure that each planet will have a unique, but fully reproducible, musical character.

The actual playing of the sounds is probably the messiest part of the patch structure. Purists look away now.

I wanted to make sure this part of the patch was as flexible as possible, so I ended up using the soundfiler and tabread~ objects, rather than tabplay~, which is great in practice though does look rather uncouth. Additionally, I had some limitations imposed upon the structure of the patch – namely, I had to keep the number of tables down as much as possible, to save on memory. So each sound bank has two voice polyphony – there are many sound banks, and the beats and sound effects aren’t counted in this, so that limitation is not really heard in the final product at all. It did mean I had to work out a decent voice allocation system though!

I think my memory issues were probably my only problem with using PD in this project – though only indirectly. As I mentioned, they were hardly a problem artistically, however it took me a while to get used to the idea that not everything I patched on a computer would work on an iPhone. Similarly, I had to be very careful about things like relative volumes. In a generative music game like Pugs Luv Beats, the player could quite easily send 15 pugs running around making sound, which mounts up pretty quickly. It means that all of the patches and sound need to be designed to withstand lots of triggering without distorting. None of these things are problems, really, all they require is regular testing on devices and simulators – something that every mobile developer is already used to.

That’s the Pure Data audio engine in a nutshell. The end result is a flexible and powerful audio engine that sounds really great and is fully integrated into Pugs Luv Beats. The game is a great combination of music, silliness, and strategy – there’s a bit of something in there for everyone. You can definitely just play with the game to make beats, or you can try and collect all of the costumes, or you can try and make the most efficient planet ever. You can also explore the galaxies being made by your Game Center friends, to hear what they’re up to.

The background story:

And, just for fun, a silly promo featuring real pugs. Anyone traumatized by the sight of Pd patches, these should relax you.

More information at the developer site:
http://luckyframe.co.uk/pugsluvbeats/game.html

If you liked the generative, probability-based sequencing seen earlier this week, here’s another example – and it’s free and open source, so if you do want to pick it apart and you own a copy of Max/MSP or Max for Live, you can. Co-creator Giuseppe Sorce points us to the work:

This is a simple generative music synthesizer built in Max/MSP created by Diego Caponera, Nicolò Paternoster and Giuseppe Sorce. It involves 5 FM generators which play notes randomly based on a root key and intervals defined by the user. It’s an university project made for an exam for Sound’s Science degree ( Math Department of Roma Tor Vergata ).

The software is distributed “as is”, without any warranty, under a GPL license.

Not bad for an exam project. A heck of a lot more fun than a paper (and, believe me, I say that wholeheartedly having done a bit of university teaching).

Grab it:
http://code.google.com/p/smilesynth/

Graphics are good. Graphics are shiny. But when it comes to reality-bending, emotionally-immersive, perception-shifting power, look to sound and music.

At least that’s the feeling you could get after playing Dimensions. Following their reactive music tools and Inception dream states for iOS, RjDj have turned their mind-altering sonics to gameplay. As with previous releases, these tools are powered by the open source visual development environment Pure Data. Pd engineering wizardry here meetings the compositional and sound design prowess of Hans Zimmer.

You can see a bit of how the musical world works in the teaser video above, and the music sound design video below.

But we wanted quite a lot more information. So, CDM got RjDJ’s Rob, Joe, and Martin to share some detailed thoughts on how the game experience is put together and how it works.

The App

RjDj Team: Most games require your full attention when you play them. You either live your life or play the game. Dimensions is different. It’s designed to be played in parallel with your normal life.

Gameplay is intertwined deeply into your daily life. Some dimensions unlock if you are physically active and others unlock if you are quiet. The app automatically detects what you are doing and syncs the game to it making use of every possible sensor on the iPhone.

You stay immersed in the game by listening to augmented sound and the voice of Emily from Mission Control. She guides you through many exciting challenges like collecting Artifacts and avoiding the dreaded Nephilim.

With Dimensions we are very interested in creating a gameplay experience which is between the device based focus of a casual game and the passive use of listening to music. Its a game which you play by listening – a game that place in parallel to your everyday life.

The Tech: Reading Files

We built our own version of readsf, rj_readsf, in order to be able to read compressed audio and make the samples available for processing in Pd. One advantage of readsf is that possibly lengthy audio assets do not need to be loaded into memory. If memory is limited, especially when Pd may be running in the background, limiting exposure to system memory warnings helps keep the app running and the music playing. Given that compressed audio is roughly ten times smaller in size than uncompressed audio, and that audio assets make up the majority of the size of the entire app, it is a huge benefit to be able to deliver and read compressed audio assets directly, without the need to decompress in memory or onto disk. Dimensions requires that several dozen such players be open and viable at any time, and special consideration was given to concurrent behaviour. rj_readsf can loop a file when it gets to the end, and it indicates with a bang when a file has been loaded (an asynchronous operation) or the end as been reached (in the non-looping case). rj_readsf is built on iOS standard APIs and can read any file format that iOS can.

Ed.: I’m waiting to hear if rj_readsf will be open-sourced. The issue of reading files is one we’ve had around libpd recently. While their rj_readsf sounds great, my sense is the best long-term solution will be a similar object that is independent of the APIs of any one OS, so this same set of problems may need a different solution for the open source community more generally. (Building such a tool is absolutely possible, though it might require more effort.)

The Music, and How the Music Plays with You

The music of Dimensions uses various different techniques from straight sample playback to audio analysis and synthesis:

Realtime manipulation of audio input from the mic:

This is perhaps the most recognisable technique we use. We process audio from the iPhone microphone live in many different ways. It’s kinda like a feeling of being inside the music.

The key thing we do with effects is attempting to analyse the environment of the player / listener and then making appropriate things happen within the effect. For instance, the Flux Dimension features a filterbank on the mic input. We analyse the incoming audio from the players environment and make the filter frequencies change as events occur ( either due to pitch changes or onsets ) this gives the impression that objects and activity around the player is somehow “playing” the music.

In the Ghost Dimension there is an effect which records audio whenever it detects an event, then scrubs repeatedly forwards and backwards through the sample using granular techniques stretching it out in time. This manipulation accentuates the textural and pitch based qualities of the sample as it repeats and works well with the atmospheric music Hans Zimmer composed.

Dynamically-controlled stems:

All the Dimensions use stems and hits from a conventional sequencer in some way, re-arranged live on the device relative to how the player is interacting. These stems were mainly composed in Cubase and Logic.

For example, in the Kinetic Dimension we feed accelerometer data from the device into Pd and drive the music from that. The player hears more energetic beats when they go for a run, but if they stop at the lights to cross the road, the drums immediately drop away. This was achieved with a large number of hits with all the rhythmic sequencing happening in a hybrid reactive / generative way live on the device.

In the Tranquil Dimension, the music introduces more stems the longer the player is quiet. If they make too much noise the music “shrinks away” from them and becomes quieter. If they stay in a Zen like peaceful state, the music grows into a kind of crescendo of serenity.

Reactive synthesis:

We often control parts of the music by doing a frequency analysis of incoming microphone audio from the device and then using those frequencies to determine the notes synths will play within the music. The Travelling Dream in Inception the App uses this extensively. Tranquil Dimension in Dimensions also uses onset and frequency changes to trigger synth melodies in the music.

The synths we use range in complexity from very simple additive synthesis to some great synth patches from the rjlib by Frank Barknecht and Andy Farnell.

Generative approaches:

There are some sections within Dimensions which are generative. These play back prepared samples as well as triggering onboard synthesis. They also feed the results of this through various live sampling and glitching patches. They are governed by various sets of rules which have various long term parameters, like adjusting to the intensity of the audio environment of the listener, or how dense areas of music have been around the present time.

Sample triggering:

Ghost Dimension uses a simple but effective technique of triggering samples from the music on onsets in the environment. This can cause some real jump out of your skin moments. We combined this section with a randomised very short delay on the mic which acts almost like a resonator, turning the mic sounds into creepy atonal pitched noises.

Sound Design

The main hub section in Dimensions, called the Launch screen, acts as a entry point to your augmented adventures. It also displays all available Dimensions via the floating tile icons.

Visually, these represent a snapshot of your previous experience using your location at that time. Sonically we wanted them to have an aura or energy from the Dimensions themselves.

SoundCloud examples:
Dimensions Sound Design Example by rjdjme

Example of using mixture of synthesis and samples to create user feedback when interacting with Dimension icons in the game.

Map Tile Down: several recordings of a synth in Pure Data that is played when the tiles are touched. Each one is slightly different due using two detuned oscillators.

Map Tile Open Only: a sample from Logic Pro for the woosh sound when showing the information view.

Map Tile Click: a sample from Logic Pro for touch events.

Map Tile Open: recording of how it sounds when put together.

Sound is a mixture of samples and real-time synthesis. The energy sound is made using two oscillators (one detuned) to create some modulation for a glowing effect. Added to some harmonics to make it more of a beam sound and some chorus and reverb. The open tile is made in logic, when closed it’s the same sound but reversed and pitched down in Pd.

We wanted the tiles you tap on to feel like each Dimension has some sort of energy radiating out.

Here’s a sense of how the sound design works in the game:

It’s an ambitious app, and the whole cost is US$2.99. I guarantee it’ll change your world more than a latte. (Well … unless we’re talking a really crazy latte. And that might not be legal.) As sometimes-CDM contributor Jaymis Loveday notes, there are terrific choices in coloring Google Maps, and how modes change based on ambient sound and motion.

Requires an iPhone 3GS or better, or third-generation iPod Touch or better, or an iPad.

Dimensions @ iTunes Store

More reading:
Dimensions Augments Reality Purely Through Sound [Leigh Alexander, one of my favorite game writers, for Gamasutra
The Roundabout Tapes – RjDj now plans to game reality with sound [TCTV] [Techcrunch EU]

Labeled as ready for IDM or “Braindance” music, The Uncanny Sequencer could be something tasty for those tired of regular rhythms. The creation of Julien Bayle, The Uncanny Sequencer is a graphical, generative, multi-part sequencing Max for Live device built for Ableton Live. At its core, it creates polyrhythms and irregular rhythms by making the appearance of each beat probabilistic rather than determinate.

Thanks to Julien for sending this our way. (Matrixsynth and Synthtopia deserve credit for being faster.)

Features, as described by its creator:

• 8 channels (meaning 8 sequencers in one)
• from 1 to 32 steps in each channel (every number choosable)
• generates MIDI notes to any devices (inside or outside Ableton LIVE)
• based on probabilities
• unique sequences generation
• huge presets bank engine (including morphing between 2 presets)
• 10 pages documentation

€15,00, available for download.

http://designthemedia.com/products/file/13-the-uncanny-sequencer/

Perhaps you’ve seen the demo videos, as people do astounding things by moving their body around and using the Kinect camera to make music. Now, a set of Max for Live devices makes it reasonably easy to access your body as input inside Ableton Live.

FaceOSC Mapper

Pictured at top, this builds on Kinect superstar coder Kyle McDonald’s face-tracking tool and lets you use your face – position and even facial movements – to control Ableton Live parameters.

Kinect Camera

For use with the V-Module and vizzABLE systems, you can plug in one or more Kinect cameras, and get tilt, distance filtering (to remove backgrounds), depth maps, RGB and IR modes, and plug in your depth-sensing camera for more goodness.

Kinect – OSCeleton

The home run: look at skeletal tracking for extremely precise human control of parameters, as seen in the video. It only gives your left and right hand, but stay tuned for further developments. See also this example patch.

Example video shows how to “track hand positions and translate to volume and send levels.” Not impressed? Remember, it’s a proof of concept: you can assign to other parameters, practice your movements, change the musical content, and even modify the patch to make it work better.

It’s Electronic Body Music! (Sorry, couldn’t resist. Kids, ask your … parents jeez we’re getting old, aren’t we?)

What happens when you meld the most futuristic Microsoft technology with the most futuristic Apple technology with the most ColecoVision-esque graphics as built in Jitter? Or you create gameplay that couples physical human contortion with the step sequencing rhythms of music? A different take on music games, that’s what.

Developers Matt (“M@tt”) Boch and Ryan Challinor work, in their day jobs, on the music game as most people know it, at Harmonix. Harmonix’s roots remain in the rhythm game, so that music play, even at its most serious, is still about musical timing accuracy. Pxl Pusher is a very different mechanic: imagine a step sequencer grid on an iPad, presenting blocks that, true to the classic game Twister, require another player to balance and stretch their bodies to match.

I caught up with Matt and Ryan over the summer at New York’s Museum of Modern Art. Before throngs of crowds swarmed the game – easily one of the most popular of the night – I managed to get some quick footage of the creators doing last-minute patching and trying out gameplay. (The quiet there is atypical; we got to shoot before the doors opened.) An insane travel schedule kept me from publishing sooner, but here, Matt and Ryan share their process.

Another interesting twist: Max/MSP and Jitter allowed extremely rapid prototyping with Kinect, something of interest to anyone doing this sort of work. (And dig those “3D” images … if they don’t blind you.)

CDM: How did you work together on this project?

Matt: Jamin from Kill Screen had asked me to make a project for the Pop Rally, and I was looking for someone to team up with to make it happen. Around the same time, Ryan was developing Synapse, and I got super excited about that tech. Having worked with him closely on Dance Central 2, I was positive we’d make a killer team, so I proposed we team up and make it happen. He got the ball rolling immediately.

Max/MSP is a pretty ideal environment for the both of us to work in, as it lets us evaluate ideas really quickly, and completely obliterate the lines between disciplines that we’re used to in traditional game development. Each of us designed, coded, sound designed, and made art for the game. It was awesome to be able to share in the creation of all parts of the game.

Ryan: We riffed on ideas with each other until it snowballed into the final product. Max allowed us both to work in the same space, as opposed to the traditional programmer/designer arrangement, where I would work in code and Matt would tweak values that I expose. This allowed us to work much faster and blurred the line between programmer and designer.

Any roadblocks, discoveries along the way?

Ryan: The whole project was driven by discovery: discover a fun interaction, then figure out how to leverage it. There were no real roadblocks, other than Matt being on painkillers for the entirety of the project!

Matt: I fell, knocked out four teeth, and fractured my jaw, so I was on heavy doses of painkillers for much of the development. I’m doing much better now, almost fully recovered.

The greatest discovery for me was more of a rediscovery. I remembered how freeing it is to design a game for a very specific context. Before working at Harmonix, I made arcade games in a contemporary art context. Getting back into that mindset was a blast.

Tell us a bit how you work with Max for rapid prototyping. How did you set it up? Any specific tools you made use of? Any tips, either related to Max or Kinect specifically?

Ryan: No real tips for Max, other than do a lot of commenting when you’re working on a file with someone else. As far as Kinect tips, my main advice would be to explore and discover what fun you can have with he input, other than starting with a specific goal. The only tools we used outside of Max were Synapse for the Kinect data, Adobe Premiere for the background movie, and some web app Matt used to generate some of the sound effects.

Matt: We ended up using Beanstalkapp & Versions for version control, which was huge for us in the late stages of development when we both needed to be working simultaneously. It was great that we structured our patch in such a way that we could easily turn elements of the patch into standalone patches, so simultaneous work could happen. You lose some flexibility, but it’s worth it to be able to work in tandem.

How did you conceive the idea for this game?

Matt: Largely through iteration and idea exchange. I VJ around Boston a fair amount and often used TouchOSC to VJ with my iPhone, so I’m not stuck behind my laptop for hours at a time, so when Ryan brought up the idea of using TouchOSC with MAX and Synapse, I was totally sold. We quickly arrived at the mashup between Twister and Step Sequencer, and every after that was polishing mechanics, arriving at a unified aesthetic, and then finding the best way to ramp difficulty.

Ryan: Matt approached me to work on a game for the PopRally right around when I just finished up Synapse, so I wanted to leverage that in the project. I had also just learned about TouchOSC, so I thought it could be fun to incorporate that. So, the initial concept was born out of experimenting with interactions that combined the Kinect and iPad.

How did things go at MOMA? Any surprises in terms of reception? It was effectively rapid playtesting — anything that you would now incorporate into the game? Or things that worked well, for that matter?

Matt: It was a phenomenal success. I never imagined we’d have a line of people waiting to play the game all night, never mind a line full of people cheering, laughing, and clapping as they watched others play. We’ve talked a bit about ways to modify the existing game to make it a bit deeper and more broadly compatible, but I’m also wary of upsetting the balance we arrived at.

Ryan: I was pleased with the reception at MoMA, it was very satisfying to work on a project and then immediately have people playing it, literally 10 minutes after it was finished (thanks to a last-minute feature request from Matt’s friend Christina)

What does it mean to you to have this game in this landmark art museum?

Ryan: I’m really grateful that we were able to show it off in such an amazing space. It feels a little unreal to me, my brain hasn’t really processed it yet.

Matt: It was hugely validating, especially because my Dad was in town and was able to make it to the event. He’s never seen crowds of people play the games I’ve worked on so to have him there, seeing how much fun everyone was having, was awesome in and of itself. That it was at the MoMA? Still haven’t totally processed it. It was the stuff of dreams when I was getting my art degree; it’s going to take a bit for me to internalize that that dream was realized. Not only was I able to show work at the MoMA, but it was shown in the context of so many great games: Bit.Trip Beat, Limbo, B.U.T.T.O.N., QWOP, the list goes on…

How about the connection to Kill Screen Magazine?

Matt: I think Kill Screen is far and away the best game writing happening right now. Both their daily updates on the web and their themed print issues are staples in my life. To be fair, I might be biased as I’m good friends with [editor-in-chief] Jamin, but nevertheless, I think what they’ve accomplished is undeniably phenomenal. What other publication could convince the MoMA to have a video game night?

Can you tell us a bit about your roles at Harmonix, and how you came to be there?

Ryan: I’m a programmer at Harmonix, I’ve worked on The Beatles: Rock Band and the Dance Central series. I was a CS major at the University of Texas, I got an internship at a game studio in Austin that turned into a job, then a few years later a coworker talked me up to someone he met from Harmonix, and I got recruited to work there.

Matt: I am a Project Lead at Harmonix, though I’ve occupied many roles previously, most recently Senior Designer. I started at Harmonix as a Production Assistant while I was finishing my thesis at Harvard. My friend and VJ partner, Josh Randall, is the Creative Director of Harmonix. We’d been VJing together for a bit while I was in school. He gave me a call one day, suggesting I apply for a Production Assistant role. One of my jobs was to track all the prototype hardware, but I ended up spending most of my time repairing it, as my art practice of building arcades gave me a fair amount of experience about mechanical engineering, electronic engineering, and coding. That landed me a role as hardware designer, and, many Rock Band iterations and trips to China later, I was tasked with investigating tech for a potential dance game. That lead to me heading up the development of a game prototype that would become Dance Central, and I’ve been designing for the franchise ever since. A circuitous path for sure, but I’m really happy with where I ended up, and deeply grateful to Harmonix for allowing me to occupy such varied roles throughout my 4 years tenure.

Synapse is something other folks can use, too, correct? What does it do, and where might they begin?

Ryan: Yep, Synapse is a freely released toolset. The primary focus of the project was for controlling Ableton Live with Kinect, but it can be used with anything that receives OSC input. You can find more information at http://synapsekinect.tumblr.com/

Any future plans for this game, or other Synapse-powered stuff?

Ryan: In the immediate future, we’ll be showing off PXL PUSHR at Indiecade in LA on October 8th. Beyond that, I’m not sure what the future is. At the very least, we’re going to keep the ideas of the gameplay mechanic in mind to be incorporated into future projects. As far as Synapse, I’m currently exploring other ideas so I don’t have any Synapse stuff in the works at the moment, but I know of a few other people working on Synapse-powered projects currently.

Matt: As Ryan stated, we’re showing it at Indiecade, and are hoping we’ll get more opportunities to show the game publicly. I deeply enjoyed working on the game with Ryan and I hope that TeamPXL, the moniker we’ve adopted, will develop additional games with the same working methodology. For now, given that we both just wrapped development on Dance Central 2, we’re taking a much needed break to recharge. I know Ryan’s got some awesome audio manipulation patches in the works, and I’m excited to see what comes of those. In time, I’m sure an awesome idea for a game will cross one of our minds and we’ll have no choice but to dig in and make something new!

http://teampxl.tumblr.com/

In a twist on generating music from dance, artist and coder João Pais sends us Pd-generated scores, transformed into twinkling sounds by music boxes. As it happens, the music boxes are the same featured in Friday’s story on Ritornell.

The piece is RISS, a “performative installation/concert” produced outdoors in Berlin, with a 30-meter score generated by Pd from movement.

João describes his work: “The patch was simple, it analysed the sounds, and made a score using data structures. then I printed the score, and spent lots of hours puncturing all the holes.”

The video documents those sounds, in a project produced by the collaboration of composer and artist Christian Graupner, dancer/choreographer Phuong Nguyen, and João Pais. (For his part, João has been utterly invaluable in helping me with some Pd patches – I need to post some abstractions based on that work soon for everyone.)

More info:
http://www.humatic.net/art/p/riss/index.html

And I know this is just the tip of the … music box iceberg. Sorry, that’s a really horrible mixed metaphor. There are lots of people using music boxes, yes.