Digital: disposable, identical, infinitely reproducible. Recordings: static, unchanging.

Or … are they?

Icarus’ Fake Fish Distribution (FFD), a self-described “album in 1000 variations,” generates a one-of-a-kind download for each purchaser. Generative, parametric software takes the composition, by London-based musicians-slash-software engineers Ollie Bown and Sam Britton, and tailors the output so that each file is distinct.

If you’re the 437th purchaser of the limited-run of 1000, in other words, you get a composition that is different from 436 before you and 438 after you. The process breaks two commonly-understood notions about recordings: one, that digital files can’t be released as a “limited edition” in the way a tangible object can, and two, that recordings are identical copies of a fixed, pre-composed structure.

Happily, the music is evocative and adventurous, a meandering path through a soundworld of warm hums and clockwork-like buzzes and rattles, insistent rhythms and jazz-like flourishes of timbre and melody. It’s in turns moody and whimsical. The structure trickles over the surface like water, perfectly suited to the generative outline. At moments – particularly with the echoes of spoken word drifting through cracks in the texture – it recalls the work of Brian Eno. Eno’s shadow is certainly seen here, conceptually; his Generative Music release (and notions of so-called “ambient music” in general) easily predicted today’s generative experiments. But Eno was ahead of his time technically: software and digital distribution – both of files and apps – now makes what was once impractical almost obvious. (See also: Xenakis, whom the composers talk about below.)

You can listen to some samples, though it’s just a taste of the larger musical environment.

Fake Fish Distribution – version 500 sampler by Icarus…

12 GBP buys you your very own MP3 (320 kbps). Details:
http://www.icarus.nu/FFD/

The creators weigh in on the project for Q Magazine:
Guest column – Electronic band Icarus on whether algorithms can be artists?

The conceptual experiment is all-encompassing. Just to prove the file is “yours,” you can even use it to earn royalties – in theory. As David Abravanel, Ableton community/social manager by day and tipster on this story, writes:

“As a sort-of justification for the price, all Fake Fish Distribution owners are entitled to 50% of the royalties should the music on that specific version ever be licensed. A very unlikely outcome, but at least it’s sticking to concept.

I spoke with Ollie and Sam to share a bit about how the mechanism of this musical machine operates. Using Ableton Live and Max for Live, each rendition is “conducted” from threads and variables into a sibling of the others. The pair talk about what that means compositionally, but also how it fits into a larger landscape of music and thought. Of course, you can also go and just experience your own download (first, or exclusively) to let the music wash over you, an experience I also find successful. But if you want to dive into the deep end as far as the theory, here we go.

CDM: How is the generative software put together? What sorts of parameters are manipulated?

Ollie: The basic plan to do the album came before any decision about how to actually realise it, and we initially thought we’d approach the whole thing from a very low level, such as scripting it all in the Beads Java library that has been a pet project of mine for some time. But although we love the creative power of working at a low level, the thought of making an entire album in this way was pretty unappealing. We looked at some of the scripting APIs that are emerging in what you might call the hacker-friendy generation of audio tools like Ardour, Audacity, and Reaper, but these also seemed like a too-convoluted way to go about it.

Even though Max for Live was in hindsight the obvious choice, it wasn’t so obvious at the time, because we weren’t sure how much top-down control it provided. (As a matter of fact, one of the hardest things turned out to be managing the most top-level part of the process: setting up a process that would continuously render out all 1000 versions of each track.) Although it was quite elementary and unstable (at the time), [Max for Live] did everything we wanted to do: control the transport, control clips, device parameters, mix parameters, the tempo … you could even select and manipulate things like MIDI elements, although we didn’t attempt that. 

So we made our tracks as Live project files, as you might do for a live set (i.e., without arranging the tracks on the timeline), then set up a number of parametric controls to manipulate things in the tracks. Many of these were just effects and synth parameters, which we grouped through mappings so that one parameter might turn up the attack on a synth whilst turning down the compression attack in a compensatory way. So the parameter space was quite carefully controlled, a kind of composed object in its own right.

We also separated single tracks out into component parts so that they could be parametrically blended. For example, a kick drum pattern could be spilt into the 1 and 3 beats on the one hand, and a bunch of finer detail patterning on the other, so that you could glide between a slow steady pattern and a fast more syncopated one. So loads of the actual parameterisation of the music could actually be achieved in Live without doing any programming. Likewise, for many of the parts on the track, we made many clip variations, say about 30, such as different loops of a breakbeat. The progression through those clips — quantised in Live, of course — could also be mapped to parameters.

Finally, by parameterising track volumes and using diverse source material in our clips, we could ultimately parameterise the movement through high-level structures in the tracks. So we could do things like have a track start with completely different beginnings but end up in the same place. We did this in Two Mbiras, which is probably the track where we felt most like we were just naturally composing a single piece of music which just happened to be manifest it a multiplicity of forms. In that sense, this was the most successful track. Some of the other tracks involved more of an iterative approach where we didn’t have a clear plan for how to parameterise the track to begin with, but that fits with our natural approach to making tracks. At one point, we wondered if we could just drop a bank of 1000 different sound effects files into an Ableton track, to load as clips. To our glee, Live just crunched for a couple of seconds and then they were there, ready to be parametrically triggered. So each version of the track MD Skillz could end on a different sound effect.

The Max software consisted of a generic parametric music manager and track-specific patches that farmed out parametric control to the elements that we’d defined in Live. The manager device centred around a master “version dial”, a kind of second dimension (along with time), so you could think of the compositional process as one of composing each track in time-version space.

We used Emanuel Jourdan’s ej.function object, which is a powerful JavaScript alternative to the built-in Max breakpoint function object, and wrote some of our own custom function generators and function interpolation tools to interact with it. Using the ej.function object, we composed many alternative timelines to control the parameters, and then used the version dial to interpolate smoothly between these timelines, resulting in a very gentle transition between versions. I.e., version 245 and 246 are going to be imperceptibly different, but version 124 and 875 will be notably different (we quickly broke from our own rule and started to introduce non-smooth number sequences into some of the tracks, so for example in Colour Field two adjacent versions will actually have quite different structures). We spent some time making it well integrated into Live so that once we really got into the compositional process it would work smoothly and be generally applicable to all of the different ideas we wanted to throw at it. That said, it’s a few steps of refinement from being releasable software. 

Pictured: the master controller device, very minimal, just a version dial and a few debug controls. Double clicking on bp_gui leads to the other figure, a multitrack timeline editor, with generative tools for automatically generating timeline data using different probability distributions.

How did you approach this piece compositionally, both in terms of those elements that do get generated, and the musical conception as a whole?

Sam: Since 2005, we had been working a lot in the context of performance, not only as Icarus, but with improvising musicians through our label / collective Not Applicable. This is reflected in the records we put out both as Icarus and individually during that time, which increasingly used generative and algorithmic compositional techniques as structural catalysts for live improvisations. (As Icarus: Carnivalesque, Sylt and All Is For The Best In The Best Of All Possible World. Individually: Rubik Compression Vero, Five Loose Plans, Nowhere, Erase, Chaleur and The Resurfacing Of An Atavistic Trait). Our performance software was made using Max/MSP and Beads and we started by crafting various low level tools that would loop and sequence audio files in various different ways, giving us control parameters that were devised around musical seeds we were interested in exploring.

In many respects, our approach was very similar and partly inspired by Xenakis’ writings in Formalised Music, although the context is obviously very different. These low-level tools were augmented by various hand-crafted MSP processing tools which used generated trajectories and audio analysis as a method of automating the various parameters that effected the sounds themselves, the logic being that an FX unit as a manipulator of sound is in some way loosely coupled to the musical scenario it is contextualised in. In both cases above, the idea was to step back from performance ‘knob twiddling’ by using the computer to simulate specific types of behaviour that would control these processes directly (hence the reason why we have never used controllers in performance).

Our search for different methods of coupling our increasing parameter space led us to develop various higher-level control strategies at Goldsmiths and IRCAM respectively, culminating in autonomous performance systems built in the context of the Live Algorithms for Music Group at Goldsmiths College. The autonomous systems we developed used a battery of different techniques to effect control, from CTRNNs and RBNs to analysis-based sound mosaicing, psychoacoustic mapping and pattern recognition. This work resulted in us being commissioned to put together a suite of pieces for autonomous software in collaboration with improvising musicians Tom Arthurs and Lothar Ohlmeier called “Long Division” for the North Sea Jazz Festival in 2010. The challenge of putting together a 45-minute programme of autonomous music really forced us to think more strategically about how it was possible to structure musical elements within a defined software framework and how they could vary not only within each individual piece, but also from piece to piece.

The most obvious inspiration for how we might do this ultimately came from reflecting on what it is we do when we perform live as Icarus. The experience of working up entirely new live material and touring it without formulating it as specific tracks or compositions proved to be an ideal prototype not only for Long Division, but also ultimately for FFD. Here, in a similar sense to the work of John Cage, large-scale structure and form became a contextually-flexible entity, which meant that for us it became to a far greater extent derived from the idiosyncrasies of the performance software we developed and keyed in by our own specific way of listening out for certain musical structures and responding to them in either a complementary or deliberately obstructive fashion (or perhaps even not at all). Creating these two pieces (‘Long Division’ and ‘All Is For The Best In The Best Of All Possible Worlds’) gave us the conviction that we could devise musical structures that were both detailed enough and robust enough to benefit positively from some level of automated control.

Therefore, when we came to start working on FFD, the main question we had to ask ourselves was; within the music making practices we had already been working with, what were the tolerances for automation within which we were still ultimately in control of and ultimately composing the music we were creating? In the end, the framework we set up was comparatively restrained; the generative aspect of each track was always notated as a performance via a breakpoint function and therefore able to be rationalised by us, the variation between different versions of the same track was done using interpolation and is completely predictable and incremental and finally, the entire space of variation is bounded to 1000 versions, meaning that the trajectories of the variation never extend into some extreme and unrealisable space.

More notes on the album:

Web: http://www.icarus.nu
RSS: feed://www.icarus.nu/wp/feed/

Last.FM: http://www.last.fm/music/Icarus
Discogs: http://www.discogs.com/artist/Icarus+(2)

SoundCloud: http://soundcloud.com/icaruselectronic
Twitter: http://twitter.com/#!/birdy_electric

Myspace: http://www.myspace.com/icaruselectronic
Facebook: http://www.facebook.com/pages/Icarus/132324596558

CREDITS

Music, Software, Scripting – Icarus (Ollie Bown and Sam Britton)
Mastering – Will Worsley, Trouble Studios
Artwork – Harrison Graphic Design

Icarus gratefully thank the following for their support of the FFD project

The PRSF Foundation (UK)
STEIM (Netherlands)
Ableton (Germany)
The University of Sydney (Australia)
Emmanuel Jourdan (France)

Truly generative musical scores in games have been few and far between, and “music games” has traditionally meant arcade-style rhythm games in which you repeat phrases or whole songs as accurately as possible. Pugs Luv Beats breaks those molds. Part of a vanguard of new gaming creations that generate dynamic music on the fly, it marries grid-based sequencing and resource-gathering gaming, as music making and gameplay blur together. The interactively-produced music could itself become a new way of delivering a musical signature with sound packs.

And beneath it all lurks a free and open source library, libpd – the embeddable version of tried-and-true free graphical music environment Pure Data. (That library is now on GitHub, and vastly updated, by the way, and we’re expecting a book soon from the library’s principle author Peter Brinkmann.)

Oh, yeah, and don’t forget about some seriously addictive gameplay and adorable pugs. I’m suddenly not concerned about the 15 hours Europe-to-North-America travel I’m doing tomorrow.

Here’s what the gameplay looks like, since it’s much easier to see:

Pugs Luv Beats was just approved on the iTunes App Store for iPhone and iPad.

Co-creator Yann Seznec (The Amazing Rolo) is a terrific musician; I just caught up with him in Edinburgh and Berlin and watched him play a homebrewed pig gut instrument with Matthew Herbert for the performance piece “One Pig,” on tour at Berghain. Working with Pd allowed Yann to focus on those musical impulses and not just engineering, and to let him try things he otherwise would never have imagined on a mobile title. So I asked Yann to walk us through how the project was built. He responded with an exhaustively-detailed examination of the evolution of this title, right down to the Pd patches. (Click through for high-res versions.) If your New Year’s Resolution is doing something with patching, you might want to hang onto these answers. Here’s Yann:

The origins of Pugs Luv Beats date back about two years. After making [musical iPhone game] Mujik, Jon (Jonathan Brodsky, aka jonbro) and I were trying to think of other approaches to music mobile app design, and we started thinking more and more about games. Music games, as a whole, are an oddly passive and traditionalist experience – you play along with a premade track, and you are judged on your accuracy and flair (which is strangely reminiscent of music conservatory mindset…). Obviously there are exceptions (RjDj’s Dimensions, Elektroplankton, etc.), but there you go.

Particularly interesting to me was the idea that game mechanics are often very similar to compositional techniques. So for example, when Sonic runs at a normal speed he collects rings at one rate. However when he powers up and goes super fast, he collects rings at a much higher rate. This could be compared to introducing a melody and then speeding it up – and when there are two players, doing this with two melodies. Instant fugue!

We started looking at how we could make a music game where the music and the game elements were fully intertwined and augmented by each other. So Jon prototyped a space shooter drum machine. It was awesome.

To make a (very very very long and boring) story short, our idea and prototype landed us some funding from Channel 4 and Creative Scotland to work on games that focus on musical creativity and composition.

For various reasons, we decided to put aside the space shooter drum machine for a while, and start from scratch. After going through several full prototyping iterations we eventually settled on a core game mechanic that turned out to be in many ways similar to a Tenori-on [Yamaha grid instrument]/Boiingg-style [monome hardware patch] music generation system – in our final prototype, you controlled a series of little dots that moved around the screen, creating loops. This is super fun from a musical perspective because it’s easy and rewarding within a few seconds, and when you have several loops going it can gain some pretty serious rhythmic and melodic depth.

The key from there for us was turning this into a game. We had been using free Internet graphics packs up until then (we hadn’t hired our artist Sean yet) which featured a ladybug, so we had been referring to the main characters as ‘bugs’. During some discussion one of us accidentally said ‘pugs’, and the game idea was born. We constructed a story about pugs and their love for beets (like the vegetables) which create beats (ha!), and how their love turned into greed and got out of control, destroying their world. The game, therefore, is about helping the pugs rebuild their lost civilization by guiding them to create beats. You grow your galaxy by collecting beats, which you do most efficiently when you dress your pugs up in costumes. What’s not to like?

To get to the part that I imagine CDM readers are most interested in, the app development was done by Jon using openFrameworks, [lightweight language] Lua, our own game engine called Blud, and the audio is all done in Pure Data using libpd (through ofxPd). In hindsight we started using libpd really late in the game, just at the very end of the prototyping stage, which was rather silly. Our adoption of libpd basically made our dev cycle about a million times more efficient. My background is as a musician and sound designer, and I have very little coding knowledge. I do, however, have lots of knowledge of Max/MSP, so picking up Pure Data was pretty easy. This allowed Jon to completely pass off all the audio processing (not to mention aesthetic sound design choices) to me, saving him loads of time, giving me direct control over the sound, and letting me test and prototype different approaches to audio within an environment that I knew would be recreated in the game. Also, as Jon mentioned to me recently, by using PD we are able to take advantage of 20 years of audio DSP research and development. Pretty amazing.

How it all works:

The entire audio engine is contained within this patch. Pardon the messiness.

The simplest part of the patch is the “sounds” section, which is used to playback simple sound effects, for the most part linked with interface actions in the game. I did this by creating a very simple patch which plays a sound when it receives a bang. Which sound it plays is dictated by the argument (in this case, the sound of discovering a new capsule). The process for adding a new sound, then, is as simple as adding the sound file to the /assets/sounds/ folder, and making a new instance of “sounds.pd” and naming it the same as the new sound. Jon, in the project code, created a list called “sounds” which is sent into Pure Data. When that list contains “capsule”, a bang is sent into that subpatch, and the sound is played.

A more complex version of what could be done with this type of data is seen in the voice of Mr Puggles, who helps you learn how to play the game. Mr Puggles pops on and off the screen to guide you through the first few worlds, and when he does he send Pure Data a “puggleShow” and “puggleHide” signal. I wanted to give Puggles a funny synthesizer voice that was different every time – dead simple in PD. To do that, I take the puggleShow bang and use it to trigger five more bangs, spaced out over a second. Each of these bangs triggers a random number which is translated into a MIDI note. This note controls the pitch of two oscillators (a sine and a sawtooth), one of which is slightly modified to make them slightly different pitches. These are played through a short volume envelope and a filter which is also controlled by a random number generator. Result? Hilarious beeping boopy Mr Puggles voice, all coming from one bang.

Every time a player buys or selects a planet, a short list is sent to Pure Data comprised of the planet BPM and a random number seed. The BPM is used to calculate delay times and such, and the random number seed is used to create a sort of musical identity for the planet. This is done by choosing a “beat library” and a musical mode.

The mode is created by building a lookup table that chooses the notes from a chromatic scale that would be used in a particular mode. For example, a major scale (ionian mode) uses notes 1, 3, 5, 6, 8, 10, and 12. Each melodic sound library I used is comprised of a full chromatic octave, and the notes that are played on any given planet are controlled by this table. This ensures not only that all of the different sound libraries being played on a planet will be in the same key, but also that a planet will have a strong melodic identity.

The sound libraries in the game are all controlled by the pugs on the planets. As they run around, each time they land they will trigger a sound. The type of sound is dependent on what terrain they are on – thus, if they run through the snow they play a toy piano, if they run through lava a distorted guitar, etc. There are two states of playing the sound, one if the player deliberately tells the pug to go to that tile, and the second if the pug is traveling over that tile to get somewhere else. It’s super easy to do that kind of thing in Pd; just set up two different ‘play sound’ envelopes, maybe a little extra delay or reverb, and you’re done!

The final piece of the puzzle for making the pugs running around into music is to make each tile be a different note. The terrain of each planet is created by making a sort of height map, where different heights correspond with different terrain types (grass, water, snow, etc). This also means that each tile has a unique number between 0 and 1. When the player buys or selects a planet, a giant random number table is generated in Pure Data which creates a number between 1 and 13 for each possible value between 0 and 1. That value is what is used to pick the note of the mode. This somewhat convoluted approach again lets us make sure that each planet will have a unique, but fully reproducible, musical character.

The actual playing of the sounds is probably the messiest part of the patch structure. Purists look away now.

I wanted to make sure this part of the patch was as flexible as possible, so I ended up using the soundfiler and tabread~ objects, rather than tabplay~, which is great in practice though does look rather uncouth. Additionally, I had some limitations imposed upon the structure of the patch – namely, I had to keep the number of tables down as much as possible, to save on memory. So each sound bank has two voice polyphony – there are many sound banks, and the beats and sound effects aren’t counted in this, so that limitation is not really heard in the final product at all. It did mean I had to work out a decent voice allocation system though!

I think my memory issues were probably my only problem with using PD in this project – though only indirectly. As I mentioned, they were hardly a problem artistically, however it took me a while to get used to the idea that not everything I patched on a computer would work on an iPhone. Similarly, I had to be very careful about things like relative volumes. In a generative music game like Pugs Luv Beats, the player could quite easily send 15 pugs running around making sound, which mounts up pretty quickly. It means that all of the patches and sound need to be designed to withstand lots of triggering without distorting. None of these things are problems, really, all they require is regular testing on devices and simulators – something that every mobile developer is already used to.

That’s the Pure Data audio engine in a nutshell. The end result is a flexible and powerful audio engine that sounds really great and is fully integrated into Pugs Luv Beats. The game is a great combination of music, silliness, and strategy – there’s a bit of something in there for everyone. You can definitely just play with the game to make beats, or you can try and collect all of the costumes, or you can try and make the most efficient planet ever. You can also explore the galaxies being made by your Game Center friends, to hear what they’re up to.

The background story:

And, just for fun, a silly promo featuring real pugs. Anyone traumatized by the sight of Pd patches, these should relax you.

More information at the developer site:
http://luckyframe.co.uk/pugsluvbeats/game.html

Graphics are good. Graphics are shiny. But when it comes to reality-bending, emotionally-immersive, perception-shifting power, look to sound and music.

At least that’s the feeling you could get after playing Dimensions. Following their reactive music tools and Inception dream states for iOS, RjDj have turned their mind-altering sonics to gameplay. As with previous releases, these tools are powered by the open source visual development environment Pure Data. Pd engineering wizardry here meetings the compositional and sound design prowess of Hans Zimmer.

You can see a bit of how the musical world works in the teaser video above, and the music sound design video below.

But we wanted quite a lot more information. So, CDM got RjDJ’s Rob, Joe, and Martin to share some detailed thoughts on how the game experience is put together and how it works.

The App

RjDj Team: Most games require your full attention when you play them. You either live your life or play the game. Dimensions is different. It’s designed to be played in parallel with your normal life.

Gameplay is intertwined deeply into your daily life. Some dimensions unlock if you are physically active and others unlock if you are quiet. The app automatically detects what you are doing and syncs the game to it making use of every possible sensor on the iPhone.

You stay immersed in the game by listening to augmented sound and the voice of Emily from Mission Control. She guides you through many exciting challenges like collecting Artifacts and avoiding the dreaded Nephilim.

With Dimensions we are very interested in creating a gameplay experience which is between the device based focus of a casual game and the passive use of listening to music. Its a game which you play by listening – a game that place in parallel to your everyday life.

The Tech: Reading Files

We built our own version of readsf, rj_readsf, in order to be able to read compressed audio and make the samples available for processing in Pd. One advantage of readsf is that possibly lengthy audio assets do not need to be loaded into memory. If memory is limited, especially when Pd may be running in the background, limiting exposure to system memory warnings helps keep the app running and the music playing. Given that compressed audio is roughly ten times smaller in size than uncompressed audio, and that audio assets make up the majority of the size of the entire app, it is a huge benefit to be able to deliver and read compressed audio assets directly, without the need to decompress in memory or onto disk. Dimensions requires that several dozen such players be open and viable at any time, and special consideration was given to concurrent behaviour. rj_readsf can loop a file when it gets to the end, and it indicates with a bang when a file has been loaded (an asynchronous operation) or the end as been reached (in the non-looping case). rj_readsf is built on iOS standard APIs and can read any file format that iOS can.

Ed.: I’m waiting to hear if rj_readsf will be open-sourced. The issue of reading files is one we’ve had around libpd recently. While their rj_readsf sounds great, my sense is the best long-term solution will be a similar object that is independent of the APIs of any one OS, so this same set of problems may need a different solution for the open source community more generally. (Building such a tool is absolutely possible, though it might require more effort.)

The Music, and How the Music Plays with You

The music of Dimensions uses various different techniques from straight sample playback to audio analysis and synthesis:

Realtime manipulation of audio input from the mic:

This is perhaps the most recognisable technique we use. We process audio from the iPhone microphone live in many different ways. It’s kinda like a feeling of being inside the music.

The key thing we do with effects is attempting to analyse the environment of the player / listener and then making appropriate things happen within the effect. For instance, the Flux Dimension features a filterbank on the mic input. We analyse the incoming audio from the players environment and make the filter frequencies change as events occur ( either due to pitch changes or onsets ) this gives the impression that objects and activity around the player is somehow “playing” the music.

In the Ghost Dimension there is an effect which records audio whenever it detects an event, then scrubs repeatedly forwards and backwards through the sample using granular techniques stretching it out in time. This manipulation accentuates the textural and pitch based qualities of the sample as it repeats and works well with the atmospheric music Hans Zimmer composed.

Dynamically-controlled stems:

All the Dimensions use stems and hits from a conventional sequencer in some way, re-arranged live on the device relative to how the player is interacting. These stems were mainly composed in Cubase and Logic.

For example, in the Kinetic Dimension we feed accelerometer data from the device into Pd and drive the music from that. The player hears more energetic beats when they go for a run, but if they stop at the lights to cross the road, the drums immediately drop away. This was achieved with a large number of hits with all the rhythmic sequencing happening in a hybrid reactive / generative way live on the device.

In the Tranquil Dimension, the music introduces more stems the longer the player is quiet. If they make too much noise the music “shrinks away” from them and becomes quieter. If they stay in a Zen like peaceful state, the music grows into a kind of crescendo of serenity.

Reactive synthesis:

We often control parts of the music by doing a frequency analysis of incoming microphone audio from the device and then using those frequencies to determine the notes synths will play within the music. The Travelling Dream in Inception the App uses this extensively. Tranquil Dimension in Dimensions also uses onset and frequency changes to trigger synth melodies in the music.

The synths we use range in complexity from very simple additive synthesis to some great synth patches from the rjlib by Frank Barknecht and Andy Farnell.

Generative approaches:

There are some sections within Dimensions which are generative. These play back prepared samples as well as triggering onboard synthesis. They also feed the results of this through various live sampling and glitching patches. They are governed by various sets of rules which have various long term parameters, like adjusting to the intensity of the audio environment of the listener, or how dense areas of music have been around the present time.

Sample triggering:

Ghost Dimension uses a simple but effective technique of triggering samples from the music on onsets in the environment. This can cause some real jump out of your skin moments. We combined this section with a randomised very short delay on the mic which acts almost like a resonator, turning the mic sounds into creepy atonal pitched noises.

Sound Design

The main hub section in Dimensions, called the Launch screen, acts as a entry point to your augmented adventures. It also displays all available Dimensions via the floating tile icons.

Visually, these represent a snapshot of your previous experience using your location at that time. Sonically we wanted them to have an aura or energy from the Dimensions themselves.

SoundCloud examples:
Dimensions Sound Design Example by rjdjme

Example of using mixture of synthesis and samples to create user feedback when interacting with Dimension icons in the game.

Map Tile Down: several recordings of a synth in Pure Data that is played when the tiles are touched. Each one is slightly different due using two detuned oscillators.

Map Tile Open Only: a sample from Logic Pro for the woosh sound when showing the information view.

Map Tile Click: a sample from Logic Pro for touch events.

Map Tile Open: recording of how it sounds when put together.

Sound is a mixture of samples and real-time synthesis. The energy sound is made using two oscillators (one detuned) to create some modulation for a glowing effect. Added to some harmonics to make it more of a beam sound and some chorus and reverb. The open tile is made in logic, when closed it’s the same sound but reversed and pitched down in Pd.

We wanted the tiles you tap on to feel like each Dimension has some sort of energy radiating out.

Here’s a sense of how the sound design works in the game:

It’s an ambitious app, and the whole cost is US$2.99. I guarantee it’ll change your world more than a latte. (Well … unless we’re talking a really crazy latte. And that might not be legal.) As sometimes-CDM contributor Jaymis Loveday notes, there are terrific choices in coloring Google Maps, and how modes change based on ambient sound and motion.

Requires an iPhone 3GS or better, or third-generation iPod Touch or better, or an iPad.

Dimensions @ iTunes Store

More reading:
Dimensions Augments Reality Purely Through Sound [Leigh Alexander, one of my favorite game writers, for Gamasutra
The Roundabout Tapes – RjDj now plans to game reality with sound [TCTV] [Techcrunch EU]

First revealed last month, PreenFM is an open source hardware synth. As the name implies, it’s an FM synth, with some very serious specs: up to six-operator FM synthesis with some nine algorithms, up to 4-voice polyphony (depending on algorithm), glide, selectable LFOs, modulation matrix, and preset banks with SysEx support. It’s all usable via a display and MIDI support.

It’s also fully open source hardware; whereas early efforts often had commercial restrictions attached, PreenFM is free for use under the GPLv3 and Creative Commons. And it’s got a unique platform under the hood: the open source LeafLabs 32-bit development platform gives this some serious horsepower. It’s very much in contrast to the ultra-inexpensive 8-bit brain of our own MeeBlip synth; think of the MeeBlip as an exercise in what you can do with a little two-stroke engine versus the V8 muscle in this. (The creator says the MeeBlip helped inspire his creation – yes, synths are multiplying!)

You may have glimpsed the PreenFM making the rounds online, but I got creator Xavier Hosxe to tell us more of the gory details and share some sounds.

CDM: So this is all based on the Leaf platform?

Xavier: Yes it’s built around a LeafLab board.
I coded a first version on the Maple [development board]; then when they announced their “Maple Mini,” I realized it was going to be very easy to plug it into a PCB.
I’m not directly connected to [LeafLabs]; I participated in the forum and learnt many things from the team.

They are very friendly and helpful.

What was it like working with the Leaf?

The LeafLabs boards uses an ARM Cortex-M3 microcontroller.

It’s a 32-bit chip runing at 72Mhz that can do 32-bit multiplication in 1 clock cycle, has 128Kb of flash [memory] and 20Kb of RAM. That seems very few but it’s not, PreenFM software uses 92Kb for the moment.

LeafLabs provide a Linux/gcc toolchain that allows to develop in your IDE of choice… Eclipse in my case, which is very confortable.

They also provide a strong bootloader and some libraries that worked perferfeclty for my needs : Usart (Midi), I2C (EEPROM), LiquidCrystal (LCD).

What will you get in the PreenFM kit?

All you need to build yourself a complete synth: PCB, screws, resistors, ICs, audio/midi jack, box, 20×4 LCD, encoders, knobs, buttons… even an USB cable [for power].

You’ll also get a Maple Mini board with PreenFM soft preloaded. The Maple Mini is easily updatable, and you can experiment lots of different things with it.

PreenFM C++ source code is available on GitHub. It’s easy to read and modifiable. If you want to see your name to welcome you on the boot screen, go ahead

To build the kit, you only need a soldering iron and some solder.

There will be 2 differences with the photos you can see on the site: the final PCB will be blue (I should receive them next week). [Ed.: See the photo in blue at top; the orange one is included here for variety! -PK]

Xavier also sends along some welcome news:

Here is a sound I can get with my soon-to-be-released StepSequencer feature in PreenFM.

This is a single voice of a simple 3-oscillator voice. 1 very slow LFO + 2 * step sequencer routed to the modulation indices.

PreenFMStepSeq by cdm

Here’s a sound sample:
PreenFM 1 by cdm

http://www.preenfm.net

Artist Zack Wright, for a handful of followers of what we used to call IDM, will be a blast from the past. Recording as Deceptikon on labels like Merck and Daly City Records, Zack is back. His name is now Dkon, and the story is more than just him: in the absence of a Merck to release adventurous music, Dkon is helping launch a new label entitled Tokyo Ghost Island, with an EP to be followed soon by new records from Jemapur, Secret Palindromes, and an EP from Stockton & Malone, among other things.

Swimming upstream against gear fetishism, the 800 EP is proud to be cheap. The Korg Poly 800 on which the release is focused is a dirt-cheap eBay score, but as Dkon puts it, it’s also “one of the most underrated analog polysynths out there.” I’d be nervous about CDM driving up its value before I can get one – it’s been on my wish list – except that there are a lot of them. It was the first synth for many players.

With that spirit, Dkon sends along a manifesto of sorts about music making. He’s been coupling the Poly 800 with a production workflow entirely centered on Renoise, the modern tracker, for recording and sequencing. But tools aside, there’s a minimal philosophy here I think a lot will like.

Oh, and about the album: it’s raw, unaffected, with the sweet spare sounds of the Korg set to good-natured beats, as clean as your local Poly 800 in a garage sale probably isn’t. It’s not retro; it’s just … well, good. The synthesis is unabashedly front and center, everything perfectly machined in pure economy. Less is more, indeed. Have a listen: the full tracks are on SoundCloud:

Dkon – 800 EP by Dkon / Deceptikon

Grab the EP on iTunes

Facebook fan page

(I love this sound — but for a radically different side of the artist, be sure to hear some of his past work and remixes below; he’s got quite a range.)

For his part, Dkon is based in San Francisco, by way of Tokyo, Seattle, Washington, Eugene, Oregon, and Portland, Oregon, except I ran into him in Brooklyn at Percussion Lab.

Bonus points if you remember Deceptikon. And if you don’t, you know we’re not music snobs here; I think you’ll be pleasantly surprised to discover him through the new Dkon music. (See bottom for some Deceptikon music, too.)

But let’s see if you agree with Dkon’s philosophy, behind this record and DIY, economy-be-damned, do-it-on-the-cheap, make-it-great spirit. He shares those thoughts with CDM:

DKON’S TIPS FOR CREATIVE SUCCESS

1. Less is more.
If you read nothing else in this article, read this. Having more options is not good for your creativity. Learn what you have, use what you have. Having a limited set of options forces you to focus.

2. You don’t need expensive stuff.
There are a lot of people who think you need to keep improving your studio, and getting the latest, most expensive gear in order to have the ability to be able to make something good. This is nonsense. From an economic point of view, the 800 EP cost me about $125 to make. (Renoise license of about ~$75, and I bought the 800 on Craigslist for $40). I made my first several albums (*Lost Subject*, *Greater Cascadia*, and *Mythology of the Metropolis*) with very limited means and equipment. Make do with what you have. Buy gear secondhand, but only what you will actually use. Use free or cheap software. Use free or cheap plugins.

3. It doesn’t matter what software you use.
There are so many DAW options now, but they all do basically the same thing. The only real difference is workflow. Pick one that appeals to you, learn it as you go along, and you will succeed. I have been using mostly Renoise for the past few years because I like the workflow and relatively simple interface. It may look confusing if you’ve never used a tracker before, but once you get the hang of it, it’s incredibly fast to get your ideas down, which is a major advantage. When inspiration hits you, the faster you can start working, the better.

4. Work around the limitations of what you have.
If something is limited in some way, use it to your advantage. Why do you think things like the 303 and 808 are still universally adored? They are both incredibly limited instruments, but what they do, they do very well. Using a more concrete example in my case, the Poly 800. It’s horribly tedious to program, but has a great sound and a lot of character. If it was covered in knobs and sliders, I don’t think it would be as appealing in a bizarre kind of way. The limited nature of the instrument encourages creativity.

5. Treat everything as a sample.
Especially in regards to software like Renoise. Find a sound on an instrument you like. Record yourself playing a few chords or a sequence of notes. Chop it up, sequence it, and rearrange it. Usually, if I do this, the sequence that ends up being used is different than the one that I originally played. Move things around, play with the pitch, change the envelopes. Being imprecise with your editing gives it a more humanized feel, without resorting to adding “humanization” after the fact.

6. Fidelity is highly overrated.
Do you think anyone is going to care if your snares are amazingly compressed and EQ’ed if your song is terrible? No. Making your music sound “nice” should be an afterthought. Focus on content, not gloss.

7. If you’re not having fun, you’re doing it wrong.
Making music, or art of any kind, should be fun. Treat it as play, not as work. Don’t think of what you want to make before you start – let the finished product reveal itself through your work. Dive in and explore without conscious thought.

http://www.deceptikon.net/
http://soundcloud.com/dkon
http://www.renoise.com/

Inside the Studio: Gear and Renoise Session Screenshots

Click the images for a closer look; all images courtesy the artist and used by permission.

More Music

Remixes by Dkon / Deceptikon

Mythology of the Metropolis 12″ by Dkon / Deceptikon

Artwork for the Mythology of the Metropolis album is, I think, really beautiful:

The painting is the work of Philadelphia-based Richard Bailey, aka artist proem, who also did my album cover as well as the CSS work on CDM. This isn’t some sort of cabal we’ve put together; I keep running into these lads and the connections between them by pure accident. There’s a sort of diffuse, scattered community of people who are expatriated from a forgotten IDM nation. If IDM dies, CDM lives, at least.

And for good measure, the music video for “Broken Synthesizers,” via reader mikrosopht in comments, who worked on it.

Brilliant idea – hacking YouTube timelines to make an interactive 909 – though I can’t get it to work for me at the moment.

Thanks to Dkon for all these ideas.

Care to debate – or echo – his creative tips? Sound off in comments.

With laidback, exotic grooves and richly-coordinated interlaced cello, saxophone, turntable, and electronics, Archie Pelago’s music relies on some serious technological savvy. To be sure, all you really need to play instruments and computers and turntables together is to get into a room and start jamming. But to realize their specific musical vision, the trio of Hirshi, Cosmo D and Kroba have turned to an advanced Ableton Live rig, centered around The Bridge to couple Serato and Live. Here’s a look at their music – and all the gory details that combine to make their setup tick.

Grab the free EP for some music:

END004: Shrinin EP by Archie Pelago by end fence

To be honest, after a lot of launch hype, it hasn’t always been easy anecdotally speaking to find a lot of people using The Bridge. The software, combining Ableton’s clip-launching facilities with Serato’s digital DJ setup, perhaps demands a lot conceptually and musically of its users. But boy, are these three using it – and pushing its envelope to the breaking point. I caught up with virtuoso cellist and technologist Greg Heffernan (Cosmo D) at the lovely Percussion Lab party in New York. Greg sends a full description on how the setup works technically, as he originally wrote for the folks at Ableton (who I imagine were quite interested). There’s a lot to follow, so happily, there’s a gear diagram, as well:

To start, there are three of us. I play cello, Zach ‘Kroba’ Koeber plays saxophone and Dan ‘Hirshi’ Hirshorn is on two turntables + mixer. We play our instruments into and alongside Ableton, recording, manipulating and effecting our sounds on-the-fly. Dan provides the rhythmic foundation for our music and because of The Bridge, all of us are in sync with each other.

I’m running my cello into a MOTU Ultralite Mk3 Firewire interface, which is connected to my MacBook Pro running Ableton and Serato simultaneously. I use a Behringer FCB1010 foot pedal (connected via an M-Audio UNO [MIDI] interface into my computer) to record clips of my cello playing, cue effects and generally navigate around the Session View of Ableton. I record my cello live into the Session View as clips, then run these clips through an effects chain on an effects rack. I then use the two expression pedals on the FCB1010 to crossfade between my ‘dry’ cello sound into an effected sound. The effects are controlled with two Korg Nano Kontrols situated below my laptop. I also use a Korg nanoPAD to play sampled clips of various found audio.

Zach’s setup mirrors mine, to an extent. Using a mic to capture his live sax sound, he runs his signal into a Tascam US100 [audio] interface which goes his computer running Live. He uses his FCB1010 to capture and record clips of his own, alongside a Korg nanoKONTROL to control effects. The reason why we chose the FCB1010 and the Korg nanoKONTROL is because they’re relatively inexpensive, easy to carry around the city and on the subway, and have a lot of buttons, knobs and faders for their size. Zach’s computer is connected to mine via an Ethernet cable and his Live set is slaved to mine via Midi Sync, so we’re always locked in the groove together.

The linchpin of this whole setup, however, is Serato and The Bridge. In addition to my running Ableton, I have Serato running via a Rane SL1 [mixer] also connected to my laptop. Coming out of my laptop is a cable going into an external monitor. Dan uses this monitor to display Serato, enabling him to do what he does with his two turn tables, Serato control vinyls, and his Behringer DJ Mixer. Because of the Bridge, my Ableton rig is locked in with whatever he’s spinning, whether it be our original dubs or tunes that inspire us. With everything sync’d up, Zach and I, through our instruments and software, react musically and rhythmically to Dan’s DJing. Dan, via the effects on his mixer and control of his vinyl, reacts musically to us as well.

In terms of audio routing, my audio and Zach’s audio are running into Dan’s mixer, so he’s mixing our sounds as much as he’s controlling Serato.

The Gear

Cosmo D:
1 cello
1 MacBook Pro 13″
MOTU UltraLive Mk3 Firewire Interface
2 Korg nanoKONTROLs
1 Korg nanoPAD
1 Behringer FCB1010

Zach:

1 saxophone
1 MacBook Pro 15″
1 Tascam US100 interface
1 Korg nanoKONTROL
1 Behringer FCB1010

Dan:

1 Rane SL1 DJ Interface
1 Behringer DJX750 DJ Mixer
1 Dell 17″ Flatscreen Computer Monitor
2 Turntables (Technics SL 12000)
2 Custom needles.

Lots of cables.

In Videos

New York public radio station WNYU hosted the trio on their program Table Tennis. Three highlight excerpts below, followed by the full program for those of you who want it:

For those of you who aren’t quite ready to leap into The Bridge yet, but do want to loop your instrument, here’s a great place to start. Cosmo D talks about his live looping process in Ableton Live, at the site Bangbang.

And in an interesting way of visually interpreting their music, dancer Genna Baroni choreographs a dance to a track from the trio’s Chocolate Waveplates EP in a music video:

There – now no one has any excuse for not dancing at an Archie Pelago jam. The setting and videography is pretty informal, but it’s nice to see movement as a way of interpreting music.

Music:
Chocolate Waveplates EP Sampler [Slime Recordings] by Archie Pelago

And lastly, a live show from earlier this year at Glasslands in Brooklyn.

Official site:
http://archiepelago.com/

The question of how to build controls to fly today’s live laptop music sets – whether “DJ,” “live PA,” or something else – remains open. We’ve got an inside look at the newest entry, the most recent device to explore just what should be mapped and tactile, and what should be left behind. Amidst various look-alike mixer-and-deck controllers, it fits in with those pushing to make computer control a hybrid of traditional DJ metaphors and new computer ones.

Meet the Twitch. The result of a collaboration between New Zealand DJ software developer Serato and UK hardware maker Novation, Twitch deviates from a number of norms.

First, while made with Serato, it’s explicitly designed to support Ableton Live and rival NI Traktor via upcoming updates. That’s something we’ve seen among general-purpose controllers, but usually the main selling point of DJ-specific gear is some sort of tight integration. (That said, you will see in the specs that they promise ITCH support for Serato will be “one-to-one.”)

Second, the developers tell us they’re heavily influenced by the monome and controller hacking communities.

What you get is a do-everything controller and audio interface, all in one box. There are beat slicing features, mixing and effects, a replaceable crossfader, and touchstrips that can be used for various purposes, which promises to be flexible enough to suit a range of applications. And unlike some rivals, the inclusion of an audio interface and USB bus power could make this more practical in the typical plug-and-play gig situation. More on how the applications work in the video:

The resulting tool even carries the “controllerist” moniker promoted in the past by artist/technologists Moldover and Ean Golden.

The Twitch has touchstrips, faders, knobs, and buttons, a strange hybrid of a lot of different ideas. I’m still wrapping my head around it – and whether this will feel like a genius cockpit for music or a Frankenstein combo of other controllers. But the Twitch at the very least looks less like the result of a product planning meeting and more like exactly the sort of thing digital musicians, DJs, and readers of this site might design for their own purposes.

It also promises to be more compact. (Akai’s APC made one misstep there – unfortunately, DJ booths are too often measured in handfuls of square inches, and even those atop the flimsy plastic cases that cover the turntables.) Straight from the source:
Weight – 1.8kg (just under 4 pounds)
Dimensions – 350mm W x 275mm D x 65mm height (13.8″ x 10.8″ x 2.6″)
– not bad.

Product Manager Dylan Wood of Serato writes us from Auckland to explain how the design came about, and how it came to fruition. It’s a great glimpse both of their design thinking and how you translate a product like this to the market. Dylan says:

The collaborative design process kicked off a couple of years ago when we first started to talk to Novation. We didn’t have to have too many conversations before we realized we had quite similar ideas about where live performance and controller technology was heading. The very first time we exchanged concept drawings at a Messe show they turned out to be really similar in terms of form factor and control set, which is always a good sign. On my way back to New Zealand after the show I visited the Novation HQ in the UK and spent time with Lars and Matt from the their team to progress the idea of a new style of DJ controller forward. We had this common idea to combine the realtime live feel of Serato DJ software with the Controllerist elements that Novation are known for.

A tip of the hat has to be given to MLR and the Monome community at this point. We’d all be using community apps on the lemur (I was going through a Monome obsession at the time and was mid Arduinome 128 build). After watching endless youtube videos of performers doing amazing things with buttons we came up with the concept of the Slicer. The Slicer is like a rolling window of cue points that moves through a song in time with the music. It does MLR style loop chopping and mashing but over a whole track instead of just short loops. It gives a DJ or performer a way to cut up and re-perform their music without having to edit or sample it manually and was something that no other software is doing in the DJ context. After we’d struck on this as being a core feature for the product, we exchanged a few updated concept drawings and I headed back to New Zealand.

Being at literally opposite ends of the world made for an interesting challenge. As Lars and I worked further on the concept, someone always had to stay up to the middle of the night so we could talk on the phone and keep developing the ideas. We got Lars down to New Zealand and spent a week or so locked in an office around a table listening to tunes and bashing the design into shape.

With the software and hardware engineering talent that we’ve got at both ends we were able to dig deep into some of the new features we were adding, like the Touchstrip, ensuring that we were implementing a solid method of control. The firmware and software interaction our engineers have come up with makes the strips feel really responsive in all the different modes.

Several rounds of hardware prototypes and lots of testing with our in house DJ’s means we’ve ended up with something that feels great, even to someone used to more traditional control like turntables or CDJs. The most arduous of the real world tests devised during this process was to ensure that the Touchstrips would work really well in sweaty, live club conditions. It involved a bare touch strip PCB with a live USB connection, and a jar full of marmalade. A thoroughly English test for the robustness of the product!

The beta process has been a passionate one – as there are a lot of keen musicians and DJs at both companies there have been some fairly epic conversations around exactly how a feature should or shouldn’t work. As well as in house DJ’s we’ve also got a crack team of external beta testers that work with us on various different projects. We heavily utilize private areas of our forum for beta communication and it makes working on projects a little more community based in a lot of ways, as it’s all about conversations with real people that are actually using our products. This kind of iterative user lead feature development process can sometimes take a long time, especially when you’re working on something that is a bit adventurous feature wise, but the results are worth it as you end up with something that feels really good to use, which has been thoroughly sanity checked by real end users.

So lots of software development and plenty of hardware prototypes later, here we are. We’re really excited for Twitch to see the light of day. I for one can’t wait to see the kind of crazy button smashing videos that users are going to post up on youtube of themselves using this thing. It’ll have come full circle at that point.

Thanks, Dylan.

Dylan also sends CDM exclusive video of a stress test of the touch strip … combined with marmalade.

Full specs:

Availability: Early July 2011
Pricing: US$599.99 MSRP / $499.99 street (UK GBP 399.99 including VAT est. street)

Bundled Serato Professional – so not a limited version of the software
Tactile multi-function touch strip
“Slicer mode” for chopping up beats
Compact and portable (they do claim it’ll fit in your bag and the DJ booth – we’ll test that!)
“One-to-one” ITCH control for Serato
Aluminum top plate
2-in, 4-out audio interface
“High-quality” replaceable cross-fader
Software effect control
USB bus-powered
Mic/aux input
Switchable booth outputs (master + cue feeds)
MIDI compatible

Full details: Novation Twitch

What Harmonix has achieved with Rock Band, and their original Guitar Hero, is remarkable. At their core, these games are descended from arcade rhythm games, reducing music to simple coordination of a few buttons. Yet numerous studies and anecdotal evidence suggest that, in an age in which recording has made musical experience passive for many, the fantasy of holding a plastic instrument is enough to convince people to explore music making again. Rock Band’s collaborative gameplay has people singing and playing again, karaoke style, and more than a few gamers have decided to graduate to real instruments and lessons. Don’t be surprised to walk into a Best Buy and see instruments and pro audio tech in the aisle next to Xbox 360 and PlayStation 3. The genius of Harmonix is that music is again entertainment, not specialization.

What people may not realize is that designing these games is hard. The illusion of simplicity, the experience of fun – these are some of the most daunting challenges in design, period.

So what happens when Rock Band evolves beyond mere rhythm game?

John Drake of Harmonix colorfully sums up the spirit of the new, real-transcription Pro game: “Good luck on that solo, asshole.”

Ramping up the difficulty of a game to real music was a transformative design challenge. We go behind the scenes to hear how Harmonix approached it, what it means for how music works, and what it can mean for your music — or the next time you want to use a game with friends to hone your musical chops.

A New Game

Rock Band 3, released today, introduces new instruments and new play modes that blur the line between rhythm game and musical exercise. The hardware inputs are now actual MIDI controllers. A new guitar, the US$150 Fender Mustang Pro, uses around 100 buttons to allow real chord fingering positions – minus the callouses. A new keyboard features two octaves, velocity sensitivity, and touch controls. The guitar and keyboard each have standard MIDI DIN output feature extensive mappings of even the Xbox buttons onboard to MIDI control changes and custom MIDI assignments. A MIDI adapter lets you use your own MIDI hardware. (Ironically, this puts the Mad Catz-built hardware ahead of many supposedly “pro” sub-$100 devices, which now have only USB connections. CDM will have a detailed hands-on with information on how to make use of that MIDI controller in a separate article.)

Accordingly, “Pro Mode” songs feature more extensive transcriptions; learning them is tantamount to simply learning the music. Chords are real chords, and, while reduced to an octave or so, the keyboard parts really are what’s in the song. Needless to say, the presence of a keyboard also opens the floodgates to properly providing keyboard music in the game, from Elton John to John Lennon.

To understand how these changes came about, we have an epic interview with some of the folks at Harmonix. Even if you’re not a music gamer, there’s plenty of reason to pay attention: what they have to say could be relevant to getting your music to a wider audience, and many of the design considerations reveal insights into how people process musical information visually.

And if you are a musician and gamer, you may finally have found a music game you can share with non-musicians without dumbing down your playing.

Speaking to CDM: Daniel Sussman, Rock Band 3 project leader, John Drake, program manager of the Rock Band Network, and Matt “Nord” Nordhaus, senior producer for Rock Band Network. (RBN allows musicians to author their own content for the game and distribute it to players.)

Beyond Rhythm Games

CDM: We know already that your games have turned people on to music, and now it seems Rock Band 3 bridges some of the gulf between game and music. What does that mean for the evolution of Rock Band?

John: From my perspective, it’s certainly opened up a whole new angle to approach people who aren’t rhythm gamers. And as rhythm gamers, people who laugh at Expert guitar charts and say, I can five-star this on a five-button guitar, no problem, like Harmonix are wusses. And I’m like, okay, now you have [Ozzy Osborne's] Crazy Train on expert, so, good luck on that solo, asshole.

Daniel: Rock Band 3 is welcoming and doesn’t take into consideration any kind of musical background or education. As it relates to a feature set, we really look at making our game musical, but also fun and crazy interactive. What’s unique about the Rock Band Network aspect is that the RBN experience is not really so much a game – it’s more of a pipeline that musicians can use to get their content, their songs into the Rock Band world. If you think about the musical community out there, the ecosystem is really a way into that musical community. You have a game that appeals to a non-musician, they play the game, they have fun, maybe they take a greater interest in music. They use the game to learn how to play guitar or drums, to appreciate music. They start writing their own music, and then put that into the game world for other people to play and interact with.

It’s a kind of funny thing. I don’t think it was totally intentional – the idea that Rock Band would get to the point where you’re able to play the game on controllers that then you can plug into your laptop to make music and then use that same laptop to do all the game authoring and then put that back into Rock Band. That was sort of where we ended up, and it’s great because it all works really well together, but it speaks to the ambition of the studio.

Do you have a sense of what will happen as these new hardware inputs make their way into gamers hands? I guess you have to wait and see.

Daniel: We’re very excited to see what happens. If you read a lot of the statements we’ve made about the ambition for Rock Band Pro, part of it was to draw a deeper connection to the music. Another part of it, really, was to provide a new gameplay experience to an audience of gamers that had been playing the same game for five years on the guitar, or three years on the drums. Really the problem we had to solve was, how can we reinvigorate the category? How can we give these gamers something new to play? And how can we continue to challenge the music gamer in a way that doesn’t just involve the content? Can we build gameplay around something new and unique, and then use that to drive the progression of the franchise?

What’s been interesting is that we get these hardcore gamers who are the cream of the crop in the Rock Band world, they can beat every song on Expert the day it comes out. And then we sit them down with the Pro guitar, and we say, alright hotshot, you probably want to start on easy. I know that’s a novel concept to you know, but try it. And they do, and what happens is, they’re getting like three stars and 60-65% of the notes. And it’s kind of the same experience they had when they first played Guitar Hero 1, or they first tried the drums in Rock Band 1. And that’s really the phenomenal thing here, is that we’ve found a way to reconnect people to all of this great music through the gameplay. And almost as a bonus, the gameplay is totally steeped in actual musical ability, so by playing the game you develop skill that can be applied to things outside the game. But really, that’s secondary, from our standpoint.

What does that mean for play testing and authoring, then, to have these new tiers of difficulty? I know in the past, the first step in authoring was to just do a full transcription of a song, and then try to reduce it to what’s playable on the game controls at different difficulty levels.

Daniel: Well, I think we still start at the Expert authoring level, which is the basic note-for-note transcription of the song. And then as you pare down from there to get to hard, medium, and easy, consider that it’s more like the “campfire” version of the song. On Easy, it’s really root notes of chords on downbeats. It’s more like you’re playing along with the song; you’re not playing exactly what the song is. And then on Medium, we introduce power chords, so you’re playing that root-fifth combination on downbeats. And then on hard, you start playing the full chords, major-minor chords, open chords, and riffs. Expert is everything. We use a lot of the same design strategy as we pare down from expert to easy that we do in the core game, in the core five-lane game.

Rock Band’s Place in the Music World

It seems like there’s a strong awareness of what RB3 is doing in the game community, that it is at this new level of musicianship, but maybe not in the music community yet.

Daniel: Our focus really has been on the gamer out there, because we want to make sure the experience is accessible and not intimidating. But we see a lot of potential for an intermediate- to pro-level guitar player or keyboard player who wants to use Rock Band 3 as a way to learn new music, to learn new songs. And I suspect that we’ll get there as the game gets out and people realize what’s going on, the guitars get out and people put it together.

I spoke at a panel earlier this week; it was a panel that was sponsored by NARAS. [That's the National Academy of Recording Arts and Sciences - yes, the folks who give out the Grammy.] We were talking about digital distribution in this day and age, and how musicians can take advantage of some of the tools. And I was stunned at how few people were aware of RBN, and Rock Band 3. And I still think that in the musical community, music games have this stigma as a game, as a toy. And I don’t think enough musicians out there are as aware of the powerful distribution that’s offered through the Rock Band franchise, and then the actual musical benefits that our game has afforded all the way back. I’m looking for the tide to change within the musical community. I’m in two bands, a lot of people here are very musical people, and deeply, we feel that we want to use the Rock Band platform to promote music and to promote musical experiences, and to encourage people to be musicians. We want to be a part of the musical community, not competing with the musical community in any way.

Have you gotten feedback from musicians as you worked on Rock Band 3, apart from, obviously, the numerous musicians who work for Harmonix?

Daniel: Some. We worked pretty closely with a couple of folks at Berklee College of Music. We’re building this game, we want to make sure that we don’t want to teach people any horrible habits, and all of our chord language is correct, and our fingering is correct, and our ramp is from easy to medium and medium to hard, following loosely with stable, academic doctrine. And so we had a couple of people come in on a weekly basis to play the game. And people were very excited. I think a lot of people have seen the potential of this within the music community. So there are certainly people that are very impressed with what we’ve done and are looking forward to applying it as a tool in the music community.

Do you have hard numbers on the relationship of the game and this game genre to people going out and learning instruments? It seems Rock Band 3 has the potential to make that happen even more.

Daniel: Obviously, it’s too soon to tell what the impact of RB3 will be on, you know, the society that we live in. [laughs] There have been studies, vendors have done studies, Cornell did a study, just on the attach rate between music gamer and how many people play Guitar Hero and then go buy a guitar, how many people play Rock Band and take musical instrument lessons. I know that it’s an interesting topic in the musical academic world. And I think RB3 definitely changes the game. The connection between the game and the actual musical ability is way less tenuous than it has been in previous games. It’s pretty real. I think you’ll see a higher attach rate, but you know, we’ll see.

Some of those hard number… Harmonix provided us with a study, announced in January of 2009, by Fender and non-profit music education organization Little Kids Rock, looking at schools around the United States. The results: educators widely attribute a renewed interest in music education to the games. The study looked at teachers with students in the 8-13-year-old age group.

Results:
67% said guitar enrollment increased as a result of Rock Band and Guitar Hero; 46% bass, and 52% drums. A tiny fraction thought it decreased.

78% said they felt these games had a positive effect.

88% said it had increased interest in classic, guitar-based rock, and a whopping 95% said the two games would help attract new students.

Rock Band Network, Meet Pro Mode, Keys

How are the tools being received in the Rock Band Network community?

Matt: We’re working on the toolset now to add keyboards and harmony to Rock Band Network. The creative community has been incredibly excited about it. I know a lot of them are both holding songs back that have keyboard parts in it, and sort of going after artists who they know might be interested in it. We’ve already had interest from a few major-label artists who have already gotten in touch with us to try to get their stuff in. So I think it’s going to certainly expand the RBN stuff into the keyboard-centric area.

What made the difference for them — is it, okay, now I have an engine that can represent my music, is it that now it’s something that I can take more seriously, or a combination?

Matt: I think it’s more the former. I think these are people who are very keyboard-centric. Billy Joel’s a great example of someone who was added. We’ve had people who said, oh, cool, you have keyboards? I really want to get my songs in there, people who are known for playing keyboards.

John: I think it’s people who make music on that instrument want to see even the representational gameplay. Even if you’re playing keys mode, you don’t have to play the two octaves, we still have that five-button-style gameplay even on the keyboard, I think we’ve always felt a little weird about it when it’s been like, play the organ solo to Smokin’ by Boston on a guitar controller. We did it because it’s an amazing song and we didn’t want to hold it back, but it makes so much more sense, and it’s so much more fun for them to see their music expressed on an instrument, even in a simulation format that’s that much closer to reality. I think it opens the door for them to get excited about it. And then once they’re excited about it, it’s kind of a no-brainer to get their music in.

Of course, as a keyboardist, I’m pleased to see the addition of keyboards; what does that change mean for Harmonix, especially coming from only the guitar and drums?

John: The exciting thing for us is, keyboard is so fundamental in terms of the way that people process and understand music. It’s probably the most linear layout of notes that you can have. It makes a lot of sense to be able to look at a keyboard and be able to understand what that harmonic structure is, versus a guitar. When you see people trying to learn theory from a guitar, they’re able to do it, but it’s a very disconnected and disjointed thing in their minds a lot of the time, if they’re not musicians.

I come from a jazz background. You have great pianists and great piano solos in the context of jazz, but at laest half the time you’re comping. That’s always part of it. It’s such a versatile instrument that can do all those things. I think with Rock Band Network and with Rock Band DLC, what we’re seeing a lot of is the ability to highlight those songs. We’ve got music like Billy Joel, like Imagine by John Lennon, like Bohemian Rhapsody, where the piano has those stand-out moments, where we’re really looking at the keyboardist as featured player. But also we have have songs like Roundabout by Yes, or Freebird by Leonard Skynard, where the keyboard has a standout section or there’s a crazy keyboard solo or sort of insane part, but really it does work as an instrument that’s part of a collaborative effort, which is what Rock Band’s all about. Rock Band’s not about Guitar Heroes, and it’s not about drum solos. It’s about your band playing together. So the keyboard will have its stand-out moments, it’ll cast more light on an instrument that’s often overshadowed.

Matt: The thing about RBN is that you typically see the more the fringe-y, niche-y stuff. It seems likely to be that we’re going to get some of those, like maybe we’ll get a jazz tune where the keyboard comps the whole time. And that would be cool. It’s the sort of thing that we probably wouldn’t release as DLC, because we have more high-profile bands in front of it. But it’s exactly the sort of thing that people put into RBN because they’re passionate about the bands they love and the kind of music they like.

What will the impact of these new levels be on the authoring process? I know you’re working on releasing new tools to work with the new implementations for Rock Band Network; what are you changing?

Matt: We’ve improved a whole bunch of the audition tools to make auditioning much, much quicker and easier — like adding rewind and skip forward, vocal guide pitches and keyboard guide pitches. We actually brought a group of people in, some of the more advanced authors, to check out the MIDI spec, and one of the things they reacted to is that the keyboards are really, really complicated. The difference between five lanes and two octaves doesn’t seem that big, but when you figure out that you’ve got chords and it’s real stuff in real time, it’s a very significant increase in difficulty.

John: Both to author and to play test.

Matt: — and to play. So we’re trying to add some tools to allow them to make sure what they’ve done is correct according to what’s actually in the song.

John: The thing now is that it’s pitch-accurate. So it’s not just charting what feels right; it’s charting what is right.

There’s still compression going on. It’s not the same compression that we do to take all the sonic stuff that you’re hearing down to five colors, but we’re still talking about, if you have a full keyboard part, taking it down to about an octave and a half — you have to think about both the theoretical, what is the best way to do that to make it feel right from a gameplay standpoint, but also how to communicate information so that it’s still pitch-accurate, and it still plays well, with jumping around. When do you leave the bass note in, when do you take the bass note out? It’s a lot of thinking.

It’s not like when we launched originally, when we had four years of guitar authoring under our belts.

So it’s like doing a transcription, in other words — it’s like doing an orchestral transcription for piano, or in this case a piano transcription for toy piano.

John: Yeah, exactly. That’s sort of a funny way to refer to it, but it’s pretty accurate. It’s more an art than a craft.

Learning Music with Rock Band

Last week we got to see GarageBand ’11, with additional lessons, and metrics behind those lessons. In a way, the games and the music tools are converging. It seems like what the games are doing can help with learning music.

John: You know, I grew up playing piano. I’m a classically-trained percussionist, and learned piano when I was six years old, and all that good stuff. And the thing that sucks about learning an instrument in isolation is that you play scales, and you play Mary Had a Little Lamb, and you play your A harmonic minor scale. And when you mess up, it sounds bad. And when you do it right, it doesn’t sound that great either. It sounds like a scale.

The fun thing with Rock Band 3 is that the stories we have, we back you with a full band of music all the time. Even when you’re playing your C major scale, or your C major triad, you’re playing it on top of a bluegrass band, or a metal riff. And you feel like you’re accomplishing something. And it makes you want to get over that hump and get over that musical boredom and inertia that holds people down, where they’re not getting engaged on a real instrument. I’m hopeful we get people to feel like they’re learning, to feel like they’re engaging with a game and playing something, and they actually develop some chops and maybe some habits that can get them thinking about musical theory and maybe taking a piano lesson, or reading Keyboard Magazine, or downloading some GarageBand lessons. That’s all good.

It’s like the Jamey Aebersold tapes, where you have a backing track.

John: Exactly. Oh, God, those tapes… I was trained on those tapes. I was trained on… oh, God. They’re good. They’re torture.

So, now we have these full guitars and real two-octave keyboards — was there some iteration to arrive at that solution? I got to poke a little fun at the process when the announcement was made, mocking up a whole staff coming at you in the interface. Maybe at one point there was even something a little like that.

John: God, there was tons of iteration on that. I think this is sort of the genius of Harmonix games. To be very clear. I’m not a UI designer, that’s not my job or my baby, but having seen the iteration process it went through, when you see the end result, when you see the polish, you think, “oh, it must have been a really natural process to come up with this.” But what you’re looking at is hundreds of hours of painstaking thought and work that went into it.

With keyboards specifically, I think the biggest challenge was really getting people who are not used to reading piano roll, and not used to understanding the difference between a fourth and a fifth onscreen, being able to identify that quickly through a seamless UI and jump right in. And I think that our team did a killer job of what’s coming down the display on the screen. I think it’s pretty much the only way you can do it. The idea of having one octave of gameplay at a time, because we tried it with more than one octave.

Matt: And you know, adding the colors. I sat in on all those meetings. I go to all the design meetings. It was fascinating to watch us work through it. We tried not having black and white notes and coloring all the notes, and eventually we ended up on black and white because people are used to that. We struggled with, like, do you light up the lanes when you press them? How thick are the dividers between the lanes? We were tweaking stuff literally until the last day. And it was a nonstop process of iteration through the whole cycle to get it right. It’s very, very hard.

John: And it’s because people are so deep into music here, and want so badly to give people that experience and make it accessible. We can get my mom to play keyboards, and then get a great keyboard player, both being able to sightread something on easy and having a good time, not feeling like they’re reading music and suffering through a rehearsal process.

Matt I think it was a huge debate whether we had a key signature list at the beginning of each song. The keyboard people wanted it, and the design and UI people didn’t want it because they felt it was clutter. We went back and forth on that a bunch of times.

John: At the end of the day, that’s what designing games really is for something like this. If we’re translating a real-world experience, making decisions to give the most amount of information to people without overwhelming them. I think Rock Band Pro really walks that line of filling that screen with all the notes in the keyboard solo in Roundabout, without making you feel like you’re going to have an aneurysm from trying to play them.

My favorite thing is what people said about the pro guitar — that’s been in development as long, if not longer, than keyboards. We’ve been futzing around with that for two-plus years. And playtesting it, people were saying, oh, sure, you can play on it, and it makes sense to you, but you’re a guitar player. And to bring people in who never played guitar before, and give them our tutorial system, to see them like an hour later playing power chords? Playing I Love Rock and Roll? I mean, they’re not mastering it and no one’s playing crazy guitar solos, but to be able to fret three or four chords based on learning it through the game in an hour or two hours. Having them have to stop because their hands hurt — they were grinning from ear to ear, and they were doing it and loving it.

That’s the power of someone learning an instrument and getting excited about it. That’s what we had hoped would happen. The idea that anyone could pick up a guitar and spend two or three hours in our game and walk away knowing two or three chords and how open notes work and how they can move their way around a fretboard — that’s pretty crazy.

That’s the biggest surprise to me — that it fucking works.

It’s why people have been playing music for thousands of years.

All photos: Kyle Mercury. Images courtesy Harmonix.

Pianoteq is an effort to model, rather than sample, acoustic pianos and other instruments on the computer. Now in its third major release, its interface and sound generation have each matured. Using mathematical models in place of recorded sounds, an entire grand piano fits in just a few megs of space, rather than requiring several DVDs, and the software maker claims the results can be more natural and playable.

Pianoteq, which runs as do its rivals on Mac and Windows, is also unusual in providing support for the Linux operating system – something some developers have claimed isn’t practical with commercial music software. And a new “Player” addition, announced this month, makes it more affordable. In addition to software development, the team has even launched an extensive piano restoration effort:
http://www.pianoteq.com/kivir

I’ve been impressed with my time playing with Pianoteq’s software. That’s especially meaningful to me, as my background in music has been on acoustic pianos, back to when I was literally old enough to reach the keys for the first time. But I also wanted to know more about how this software developed. Its rigorous approach to modeling has attracted a lot of attention in the virtual instrument world, and the fact that it targets Linux alongside Mac and Windows challenges notions that commercial software can’t make it on the free operating system.

Pianoteq sent along some extensive answers, which I’m pleased to be able to share. Naturally, they’re proud of their software, so there is a bit of expected boasting here. (I’ll discuss more of the experience of using the tool, and the new Player version, shortly). But they also have some fascinating commentary on sound design, modeling, and the development process. In the “geeky as we want to be” spirit of this site, here’s the full scoop.

CDM: Can you talk about the background of the company? How does one make a shift from dealing with the physical instruments and tuning to thinking about mathematical models, let alone translate that into actual software?

Philippe: the Pianoteq history is strongly connected to my first job as a piano tuner. Then at the age of 31, I started a new life with basic studies in mathematics at the University of Toulouse, France. After what I prepared my PhD thesis on the parametrization of vibrating phenomena, without imagining that it would be the basis of my third life with Pianoteq. Thanks to these two skills and to an exceptional scientific environment in Toulouse, I succeeded in identifying important phenomena responsible for the generation of the piano sound and proposed a model which describes the whole interaction of the soundboard, strings, bridge and air.

Julien: I was working as an engineer at Institute of Mathematics of Toulouse, with Philippe (who was also my teacher when I was a student). My focus was on an open-source finite element package ( http://home.gna.org/getfem ) when Philippe told me about his project of piano sound synthesis. I took charge of the development of the real-time engine, and we quickly decided that we wanted to turn this research project into a commercial product. Thanks to the French law on innovation and research (1999), the support of INSA Toulouse, and the Institute of Mathematics, the start-up MODARTT was created in 2006 to sell Pianoteq, which was at that time the first fully modeled piano instrument.

Niclas: I represented many of Pianoteq’s customers of today, being a part time piano composer with a love for the piano instrument. I was updating an extensive article on digital piano technology in 2004 when someone advised me to have a look at Philippe’s research. I immediately understood its potential, which is why I suggested Philippe to assist in marketing and product development. Since then, I have participated in the product development and testing and am also in charge of sales and customer service.

Modeling the Piano with Math

I remember trying previous attempts at piano modeling and finding them interesting but ultimately unsatisfying. I think people who pick up Pianoteq have, immediately, a different experience. What’s different about this modeling approach than those that have come before – and, for that matter, why did it take until now?

Philippe: The idea of modeling musical instruments is very old and has always faced great difficulties: the complexity of physical phenomena, the sensitivity of the human ear to the slightest imperfection, and the difficulty of running a complex model in real-time. The latency needs to be so small that it gives to the musician the impression of playing a real acoustic instrument. Until now, attempts have only confirmed that the task was not easy. The state of the art of digital pianos is based on sampling technology. Each note is a recording of how it sounded at a specific moment, without taking into account the complexity of the instrument. The huge data generated by sampling can reach 40 Gbytes for a single piano. The flow rate of data transmitted from the hard drive to the sound device is too high for the current hardware capacity and it can happen that one hears crackles. [Ed.: I would say fast hard drives, optimized software streaming, and other intelligent configuration can certainly avoid crackles, but the fact that, say, a low-end hard drive might choke means that Philippe's point about data intensity here is nonetheless well taken. -PK]

Moreover, the reproduced sound lacks vividness. Hence, creating a piano model which takes into account the interaction between hammer and strings, the interaction between strings and soundboard via the bridge and the interaction of the soundboard with the air is of great interest. Based on mathematical models, Pianoteq allows parameters to be stretched as long as the model permits, resulting not only in new performance styles but also in new piano sounds. Pianoteq is thus also an innovating tool for music creation and can be useful not only to musicians but also to piano manufacturers and piano tuners for simulation and training purposes. Pianoteq makes excellence in piano available to all. Among Pianoteq users, composers and professionals in music creation are certainly the most excited with our innovation. Pianoteq offers what acoustic and sampled pianos cannot offer: new opportunities for music creation and a pure piano sound that is not altered by its environment (reverberation) or by recording devices.

Can you talk more about the model itself? I can see the components that are modeled, but – realizing we have fairly technical people among our audience – what are the basic modeling techniques?

Philippe: the modeling technique is based on various standard techniques issued from mechanics theory including modal analysis (calculation of vibration frequencies and the corresponding modes) and parametrization techniques that we developed at the university, as well as from a precise knowledge issued from my previous job as piano tuner/restorer of what is going on in a piano and what is important.

How did that approach to modeling evolve? Obviously, there’s this strong mathematical research background. But what’s the process like of translating that theory into something that’s usable? Were there mistakes or adjustments along the way?

Philippe: I don’t think there were mistakes along our model evolution, but more a constant improvement in the details taken into account by the model, looking closer to the physics of the piano and finding suddenly some simplification in the algorithms that allow to take include more details for the same computational cost or being more precise in the simulation.

Developing for Linux

How did you make the decision to support Linux in addition to Windows and Mac?

Julien: The initial prototypes for pianoteq were developed on Linux, using [audio system] JACK, with no GUI. Later, when we added a graphical interface and turned it into a VST plugin, we used VSTGUI for its interface, which is not available on Linux. However we had quite a few requests from Linux users, and we did make sure that pianoteq was running fine in WINE [an open-source implementation of Windows' APIs, allowing Windows programs to run in Linux]. During the development of Pianoteq 3, we switched to the JUCE toolkit, which is a great piece of cross-platform software. Thanks to JUCE, the Linux port was really easy to do, so we decided to give it a try and see what happens.

One complaint I hear from developers about Linux is that it’s “impossible” to do commercial development, because you “can’t” distribute binaries. Obviously, that didn’t stop you. I’ve tried Pianoteq on Fedora and Ubuntu, though, under both the real-time and default kernels, and had immediate success. Now, I imagine there’s a good bit of work that goes into making that happen. What was your experience like as a developer? Do you feel that the result is successful, that it is a usable solution for users?

Julien: This was in fact something that we also feared, that the Linux port would turn into a support nightmare. However a good example of a successful application that is distributed in binary form is Renoise. That showed us that it was possible to do. In fact Renoise also uses JUCE, but I was not aware of that fact at that time. What helps here for binary portability is that we have very few dependencies. JUCE is statically linked, so pianoteq depends on very few dynamic libraries: ALSA, X11, libc (even old versions), and basically that’s all. We had to hack some sort of weak linking for JACK in order to allow pianoteq to run even when libjack.so is not available. Of course, if you want better integration in the desktop, things get much more complicated.

What’s your own Linux testing setup like? (distro? kernel?)

Julien: Pianoteq is built on a Debian Sarge box, otherwise we generally use Ubuntu for the desktop, with the default kernel. [Ed.: The distribution Ubuntu is itself built on packages from Debian; 10.04 LTS uses Debian Testing.]

I’ve likewise been impressed with the vanilla kernel (as opposed to the “real-time” branch), which can save some setup time and configuration work. (My audio interface is a Native Instruments Audio Kontrol 1). Any thoughts on what setups may be most advisable? (You document some of this in the readme.)

Julien: I’ve never been lucky with the kernels labelled “rt” , and I really hate when the computer randomly hangs so I prefer to stick with default kernels. We don’t have issues with them, as long as your user account has been granted real-time priviledges. I believe that for now, the most overlooked setting for realtime audio is the CPU frequency throttling, which is a real audio performance killer, especially on the less powerful machines such as netbooks. You really need to have your cpu running at full speed 100% of the time, especially with a software like Pianoteq which needs quite a bit of CPU power.

Ed.: Before we give the realtime kernel a bad name, some of those “random hangs” were not necessarily the kernel’s fault – a bug in Ubuntu’s implementation caused the system to crash when combining the RT kernel with proprietary NVIDIA drivers, for instance. But if this sort of thing scares you, the vanilla kernel remains a strong option – it’s the default for a reason. The larger discussion is best saved for another article, but suffic,e to say, if latency-sensitive piano instrument developers are okay with the vanilla kernel, you shouldn’t feel you have to install a realtime kernel just to make music. If you want to test it, projects like Fedora’s Planet CCRMA can make it easier to use.

Are you finding that there is some positive response to the Linux version?

Julien: Yes, very positive response. In fact, a bit more than what we expected at the beginning. Approximately 4% of our customers are using the Linux version.

I could even imagine it working on netbooks. Based on load, it appears perfectly workable, which means a really cheap ultra-portable piano you can take anywhere.

We spent some time to make sure that the latest versions could run on netbooks, altough with very high CPU load (80% or more). However I’d recommend to use a more powerful laptop in order to have more room for the cpu. Ed.: Given the range of Atom netbooks out there now, I may have to test some of the newer models on this.]

There’s not currently a Linux plug-in version, correct? I’ve been just as happy using JACK [a standard for routing audio between applications], but what went into that decision?

Julien: Right, no plug-in version on Linux for now. The problem is that “plug-in” may mean any combination of VST, DSSI, and LV2. VST would be the easiest for us, but very few hosts support it ( basically only proprietary hosts such as renoise and energyxt, and also jost). DSSI is said to be obsolete, while being not to hard to support (except the GUI has to run in a separate process..). LV2 is said to be the future, but
it seems to be quite complicated to fit a “vst-like” plugin into an lv2 plug-in. We have not yet taken a decision. It is already enough of a pain to support the numerous plug-in formats on Mac and Windows. We will probably add support for JACK sessions quite soon.

In Use

Initially, having so much power over sound could be overwhelming – looking at the number of parameters you can adjust in the real-time mathematical model. Aside from the presets (which themselves sound pretty terrific), how would you suggest someone go about beginning to explore the options? Is there a workflow that makes sense for approaching adjusting the sound?

Answer from Pianoteq:
http://www.pianoteq.com/faq?category=pianoteq_working

Ed.: So I should have read the *** manual! Here’s their advice:

If you need to adapt the piano sound you could for example try adjusting the hammer hardness (2) to achieve a different brightness of the hammer strokes. Increasing unison width (3) makes it a bit out of tune (resembling certain acoustic pianos). The new powerful sound recording feature (4) lets you place up to 5 virtual microphones anywhere around the piano to achieve ultimate ambience and tone colour. The dynamics and velocity curve (5) will most likely need to be adjusted to the keyboard used in a MIDI file performance.

Indeed, this commentary makes sense. Hammer hardness is something that could be adjusted in the maintenance of an actual piano. Since you listen to a software piano model as though it is amplified, adjusting mic placement (as on a number of piano software emulations) is a no-brainer. And dynamics and velocity curve are essential not only for MIDI files, but if your keyboard controller lacks these controls onboard.

As my friend Jim Aikin noted in his review of Pianoteq – why would you adjust the speed of sound? (I suppose you could account for different altitudes; I could replicate the Aspen Music Festival!)

Julien: Well, why not ! Being in a virtual world gives you access to some parameters that cannot be easily modified in the real world, if they give interesting variations of the sound, then they are worth being adjusted!

One small note – it seems the metronome is not connected to the playback and recording, which means that MIDI sequences won’t export to SMF correctly? (Or is this a Linux bug?)

Julien: Yes, it’s not a bug, the MIDI recording and playback abilities of the standalone application are very minimalistic. It is best to use a real sequencer for serious work.

Where can people read more about the featured historical instruments?

http://www.pianoteq.com/cimbalom
http://www.pianoteq.com/grimaldi
http://www.pianoteq.com/blanchet
http://www.pianoteq.com/schmidt
http://www.pianoteq.com/graf
http://www.pianoteq.com/cp-80

There are some fascinating add-ons that aren’t pianos. Is it possible at some point that other sound designers might be able to use the sound engine to design their own instruments?

Julien: We believe that Pianoteq PRO is already a first step in that direction, with its ability to edit each parameter note by note.

How are users using this onstage and in the studio? What sorts of users have you found using the product?

Niclas: There are many composers and keyboardists that use Pianoteq, from amateurs to professionals, on stage as well as in the recording studios. We have presented a few reference users here: http://www.pianoteq.com/references

Thanks to the Pianoteq guys for being thorough in the answers. I know not everyone responds in the same way in regards to the perceived quality of the model, but my own feeling is that the effort makes the instrument terrifically playable and responsive. The best way to see for yourself is to give the demo a go, and listen to the results. I’ll follow up more on this instrument, and how it’s become a central part of my Linux music workstation, soon. Let us know if you have questions for the developers I missed.

Shoes are the new turnables.

Or at least that’s the conclusion you might reach after watching a new Japanese campaign for Nike’s Free Run+ running shoes. Apparently wishing to tout the bendable qualities of its new footwear, Nike enlisted sound artists to transform its product into a musical instrument. The shoes get plugged in, switched on, and mixed up, battle-style, as they sense when the shoe is flexed or moved in space. And yes, everything you see in the video is real: the shoes really are controlling digital sound live. We even have the Max patch to prove it.

Lovers of experimental sound art will immediately recognize one of the Shoe-Js: it’s Daito Manabe, a bleeding-edge sound artist and alternative interface guru with a background in turntablism. I spoke to Daito, and convinced him to share the software that makes the project tick. Daito says he used flex sensors (see examples) and accelerometers to make the shoes interactive. He then processed the control signal and converted it to sound using the modular visual programming environment Max/MSP and Ableton’s Max for Live. (For another example and other resources, you can check out the article I wrote for Make Magazine issue 8, in which I stuffed flex sensors into a sock monkey and connected it via MIDI.)

What’s striking to me about the Max patch is its elegance. For all the power of these interactive environments, sometimes they’re at the best when you do something really simple. In this case, that frees up someone like Daito to focus on the performance aspect.

Here’s what Daito had to say about the project. It doesn’t hurt that the whole team does such nice work:

The agency is W+K Tokyo.
Hardware programming is by Tomoaki Yanagisawa (4nchor5 La6)
and software and sound programming is by me.

My patch is not interesting at all..
Ed.: I respectfully disagree; see above comment – sometimes performing a simple task is the strength of a tool like Max. -PK

I used max for serial communication between the shoes(arduino) and a macbookpro,
and max for live sound.
The serial part crashed many times,
so I separated serial part and sound part.
I use OSC and midi for communication between max and maxforlive.

For making and triggering sound,
I used simple msp patches and Ableton’s sampler
and I used some effects in Ableton live.
The effects are also controlled by the shoes.

The sound settings are a bit strange.
We didn’t need to use a loop machine
because we used Ableton live, but
everything was decided at the last minute,
so we used the loop machine for sampling and looping (i think it was roland machine)

I hope people think it is not fake

I actually like the impromptu feel. That’s usually the sort of thing the advertisers want. (Oh, look! An abandoned tunnel! Open the vodka! Text your friends! Let’s have a disco! Wow, everyone we know is a model!) Of course, in this case, some of the sense of “let’s set up some shoes and make digital music” is just as improvised as it looks. And this clip is making the rounds, because my Dad sent it to me!

Somewhere, Charlie Chaplin smiles.

See also (for coverage of this and many other wonderful things):
CreativeApplications.net

Be sure to check out Daito’s other work; he’s done some really beautiful sound art and interactive pieces, and his site is full of inspiring ideas:
http://www.daito.ws/#5

He’s also known for body hacks, like making music with parts of his face.