I’m not sure how everyone who owns an iPad uses it for music, but I find myself strangely drawn, more than anything else, to analog step sequencers. With MIDI connections – via a special interface or a standard USB MIDI interface connected via adapter to the tablet – you can even drive hardware. For me, the app of choice has been Little MIDI Machine. Developer Chris Randall has a new application in the analog-style sequencing category, though, called Phaedra.

If you haven’t grabbed it already, you have until the New Year to get it for US$4.99 before the price jumps to ten bucks. And you get an impressive array of features:

• Multiple buses, with 32 steps max each
• Programmable note, velocity, gate time, and two MIDI CC outs for each step
• Send or receive MIDI Clock for sync
• Use MIDI hardware (via Core MIDI), other apps (using “background MIDI” or OMAC), or your computer (networking via a MIDI Network Session

Phaedra for iPad [iTunes Store Link; you'll need iOS 5.0]

I wanted to know more about the creation of Phaedra. Developer and musician Chris Randall, known for his work with boutique plug-in maker Audio Damage, released this under a new moniker, Naughty Panther, which does iOS and MIDI development. Chris has been known to mix old and new, as with his musical use of the Apple II. Here, he gives us some insight into how he went through the design process on this new tool.

CDM: How do you sketch out the UI on a project like this? Paper and pencil?

Chris: Normally I just make notes in a little Field Notes notebook as I think about them — I carry one everywhere — then once I have a clear mental image of what I want to make, I just bust it straight out in Photoshop (or, for skeuomorphic interfaces like this one, a combination of Photoshop for the panel and 3D Studio Max for the knobs and buttons).

What inspiration did you consider as far as hardware?

Adam sent me the page for the Moon Modular 568 “Sequential Trigger Source” and quipped that it would make a fun iPad app, and I kind of ran with it. (Their site is a frame-a-thon, but ’tis here: http://www.lunar-experience.com/home.html) As you can see, the UI for Phaedra closely follows the Moon Modular design. The functionality diverges quite a bit, though. Once I had the initial look, I just started adding features I personally thought would be nice in a step sequencer. Then the beta testers had some more input. And now that it is out, I’m getting a ton of great feature requests from the users. The next update will have some cool tricks in it, provided I can figure out how to code them.

What hardware would you recommend for connecting MIDI?

The best MIDI I/O situation for Phaedra is an Alesis iO Dock. Hands down. I have tested it with a half dozen MIDI interfaces via the [Apple] Camera Connection Kit [for connecting driverless USB MIDI interfaces], and they all work fine except that $5 cheap one that people buy from Amazon, which unsurprisingly, has problems. Phaedra also automatically creates a virtual port and connects to all virtual endpoints running on the iPad, so it can drive NLog Pro, Animoog, whatever. And it also is [Apple MIDI] Network Session aware, so it can just work over wi-fi.

I’m going to add OSC output as an option in the next update, I think. I’m still thinking about how to implement that, but it seems like it would be handy, and then Phaedra’s abilities would be greatly increased.

I’m also adding trigger outputs via the audio ports, so you can use Phaedra to clock an external modular sequencer like Makenoise René, or a pre-MIDI drum machine. I’m giving it 48-ppq clock for Korg drum machines, 24-ppq clock for most every other drum machine, and arbitrary rates from 16-ppq on up to 1-ppq for driving modular sequencers and your Monotribe. This should be handy, and will be sample-accurate in line with the MIDI clock output.

See also Chris’ blog entry on the app at Analog Industries:
Some Cool Shit

Music lovers are hacking Apple’s Siri voice recognition technology. By connecting to some of the “intelligence” of the cloud, these tools can make your phone rap or send music files to a player piano for instant musical playback.

First up: Yamaha’s piano taking requests, thanks to music grabbed online. A Yamaha rep explains:

Yamaha consultant Craig Knudsen demonstrates a unique implementation of Apple’s incredible Airplay technology in an exciting new way.
Here’s how it works:
Take a standard MIDI songfile and convert it to an audio file (while maintaining the MIDI data). The songfile is then sent wirelessly via WiFi to an Apple Airport Express (which is mounted underneath a Yamaha Disklavier reproducing piano. The audio output of the Airport Express is then connected to the analog MIDI inputs of the Disklavier, using a standard audio cable.
Then, you simply ask Siri to play your favorite song from your iTunes library, and Siri responds immediately, by making the Disklavier’s keys and pedal move up and down, recreating the performance, including full orchestration.
The result is nothing short of magical.

Of course, the actual “playing” is thanks to the capabilities of the Yamaha Disklavier. I’m actually a bit puzzled as to how the online conversion works, exactly, and I was curious for any Disklavier-owning CDM readers whether this is something publicly available. I’m waiting to hear back from Yamaha.

And now, for something completely different: Siri rapping. (Somewhat … erm … badly, if amusingly. It is a hack.)

My friend Robert “Robb” Böhnke had a lot of fun combining Siri’s voice synthesis and the lyrics of Notorious B.I.G.

My hack for the Music Hack Day 2011 in London, a 24h Hackathon for all things music.

SiriProxy is used to intercept the communication with Apple’s servers. Notorious Siri then sends Notorious B.I.G.’s Hypnotize to the device.

Siri’s speech synthesis is synced to the beat using the timestamps obtained from the Echonest API which were then manually tweaked, to smooth out delays in the text-to-speech engine.

Thanks to Universal Music for awarding me a nice pair of Dr. Dre headphones

The Echonest API, by the way, is an amazing do-everything “API for music,” one that analyzes musical files and connects to a vast storehouse of musical intelligence.

Warning: this video is most definitely Not Safe For Work. (Heck, even the thumbnail isn’t, exactly.) If that concerns you, just go listen to the Yamaha video again, okay?

Siri FAQ [Apple.com]

Previously: Google Translate Beatboxing

…and, of course, that means: Android, your move.

Truly generative musical scores in games have been few and far between, and “music games” has traditionally meant arcade-style rhythm games in which you repeat phrases or whole songs as accurately as possible. Pugs Luv Beats breaks those molds. Part of a vanguard of new gaming creations that generate dynamic music on the fly, it marries grid-based sequencing and resource-gathering gaming, as music making and gameplay blur together. The interactively-produced music could itself become a new way of delivering a musical signature with sound packs.

And beneath it all lurks a free and open source library, libpd – the embeddable version of tried-and-true free graphical music environment Pure Data. (That library is now on GitHub, and vastly updated, by the way, and we’re expecting a book soon from the library’s principle author Peter Brinkmann.)

Oh, yeah, and don’t forget about some seriously addictive gameplay and adorable pugs. I’m suddenly not concerned about the 15 hours Europe-to-North-America travel I’m doing tomorrow.

Here’s what the gameplay looks like, since it’s much easier to see:

Pugs Luv Beats was just approved on the iTunes App Store for iPhone and iPad.

Co-creator Yann Seznec (The Amazing Rolo) is a terrific musician; I just caught up with him in Edinburgh and Berlin and watched him play a homebrewed pig gut instrument with Matthew Herbert for the performance piece “One Pig,” on tour at Berghain. Working with Pd allowed Yann to focus on those musical impulses and not just engineering, and to let him try things he otherwise would never have imagined on a mobile title. So I asked Yann to walk us through how the project was built. He responded with an exhaustively-detailed examination of the evolution of this title, right down to the Pd patches. (Click through for high-res versions.) If your New Year’s Resolution is doing something with patching, you might want to hang onto these answers. Here’s Yann:

The origins of Pugs Luv Beats date back about two years. After making [musical iPhone game] Mujik, Jon (Jonathan Brodsky, aka jonbro) and I were trying to think of other approaches to music mobile app design, and we started thinking more and more about games. Music games, as a whole, are an oddly passive and traditionalist experience – you play along with a premade track, and you are judged on your accuracy and flair (which is strangely reminiscent of music conservatory mindset…). Obviously there are exceptions (RjDj’s Dimensions, Elektroplankton, etc.), but there you go.

Particularly interesting to me was the idea that game mechanics are often very similar to compositional techniques. So for example, when Sonic runs at a normal speed he collects rings at one rate. However when he powers up and goes super fast, he collects rings at a much higher rate. This could be compared to introducing a melody and then speeding it up – and when there are two players, doing this with two melodies. Instant fugue!

We started looking at how we could make a music game where the music and the game elements were fully intertwined and augmented by each other. So Jon prototyped a space shooter drum machine. It was awesome.

To make a (very very very long and boring) story short, our idea and prototype landed us some funding from Channel 4 and Creative Scotland to work on games that focus on musical creativity and composition.

For various reasons, we decided to put aside the space shooter drum machine for a while, and start from scratch. After going through several full prototyping iterations we eventually settled on a core game mechanic that turned out to be in many ways similar to a Tenori-on [Yamaha grid instrument]/Boiingg-style [monome hardware patch] music generation system – in our final prototype, you controlled a series of little dots that moved around the screen, creating loops. This is super fun from a musical perspective because it’s easy and rewarding within a few seconds, and when you have several loops going it can gain some pretty serious rhythmic and melodic depth.

The key from there for us was turning this into a game. We had been using free Internet graphics packs up until then (we hadn’t hired our artist Sean yet) which featured a ladybug, so we had been referring to the main characters as ‘bugs’. During some discussion one of us accidentally said ‘pugs’, and the game idea was born. We constructed a story about pugs and their love for beets (like the vegetables) which create beats (ha!), and how their love turned into greed and got out of control, destroying their world. The game, therefore, is about helping the pugs rebuild their lost civilization by guiding them to create beats. You grow your galaxy by collecting beats, which you do most efficiently when you dress your pugs up in costumes. What’s not to like?

To get to the part that I imagine CDM readers are most interested in, the app development was done by Jon using openFrameworks, [lightweight language] Lua, our own game engine called Blud, and the audio is all done in Pure Data using libpd (through ofxPd). In hindsight we started using libpd really late in the game, just at the very end of the prototyping stage, which was rather silly. Our adoption of libpd basically made our dev cycle about a million times more efficient. My background is as a musician and sound designer, and I have very little coding knowledge. I do, however, have lots of knowledge of Max/MSP, so picking up Pure Data was pretty easy. This allowed Jon to completely pass off all the audio processing (not to mention aesthetic sound design choices) to me, saving him loads of time, giving me direct control over the sound, and letting me test and prototype different approaches to audio within an environment that I knew would be recreated in the game. Also, as Jon mentioned to me recently, by using PD we are able to take advantage of 20 years of audio DSP research and development. Pretty amazing.

How it all works:

The entire audio engine is contained within this patch. Pardon the messiness.

The simplest part of the patch is the “sounds” section, which is used to playback simple sound effects, for the most part linked with interface actions in the game. I did this by creating a very simple patch which plays a sound when it receives a bang. Which sound it plays is dictated by the argument (in this case, the sound of discovering a new capsule). The process for adding a new sound, then, is as simple as adding the sound file to the /assets/sounds/ folder, and making a new instance of “sounds.pd” and naming it the same as the new sound. Jon, in the project code, created a list called “sounds” which is sent into Pure Data. When that list contains “capsule”, a bang is sent into that subpatch, and the sound is played.

A more complex version of what could be done with this type of data is seen in the voice of Mr Puggles, who helps you learn how to play the game. Mr Puggles pops on and off the screen to guide you through the first few worlds, and when he does he send Pure Data a “puggleShow” and “puggleHide” signal. I wanted to give Puggles a funny synthesizer voice that was different every time – dead simple in PD. To do that, I take the puggleShow bang and use it to trigger five more bangs, spaced out over a second. Each of these bangs triggers a random number which is translated into a MIDI note. This note controls the pitch of two oscillators (a sine and a sawtooth), one of which is slightly modified to make them slightly different pitches. These are played through a short volume envelope and a filter which is also controlled by a random number generator. Result? Hilarious beeping boopy Mr Puggles voice, all coming from one bang.

Every time a player buys or selects a planet, a short list is sent to Pure Data comprised of the planet BPM and a random number seed. The BPM is used to calculate delay times and such, and the random number seed is used to create a sort of musical identity for the planet. This is done by choosing a “beat library” and a musical mode.

The mode is created by building a lookup table that chooses the notes from a chromatic scale that would be used in a particular mode. For example, a major scale (ionian mode) uses notes 1, 3, 5, 6, 8, 10, and 12. Each melodic sound library I used is comprised of a full chromatic octave, and the notes that are played on any given planet are controlled by this table. This ensures not only that all of the different sound libraries being played on a planet will be in the same key, but also that a planet will have a strong melodic identity.

The sound libraries in the game are all controlled by the pugs on the planets. As they run around, each time they land they will trigger a sound. The type of sound is dependent on what terrain they are on – thus, if they run through the snow they play a toy piano, if they run through lava a distorted guitar, etc. There are two states of playing the sound, one if the player deliberately tells the pug to go to that tile, and the second if the pug is traveling over that tile to get somewhere else. It’s super easy to do that kind of thing in Pd; just set up two different ‘play sound’ envelopes, maybe a little extra delay or reverb, and you’re done!

The final piece of the puzzle for making the pugs running around into music is to make each tile be a different note. The terrain of each planet is created by making a sort of height map, where different heights correspond with different terrain types (grass, water, snow, etc). This also means that each tile has a unique number between 0 and 1. When the player buys or selects a planet, a giant random number table is generated in Pure Data which creates a number between 1 and 13 for each possible value between 0 and 1. That value is what is used to pick the note of the mode. This somewhat convoluted approach again lets us make sure that each planet will have a unique, but fully reproducible, musical character.

The actual playing of the sounds is probably the messiest part of the patch structure. Purists look away now.

I wanted to make sure this part of the patch was as flexible as possible, so I ended up using the soundfiler and tabread~ objects, rather than tabplay~, which is great in practice though does look rather uncouth. Additionally, I had some limitations imposed upon the structure of the patch – namely, I had to keep the number of tables down as much as possible, to save on memory. So each sound bank has two voice polyphony – there are many sound banks, and the beats and sound effects aren’t counted in this, so that limitation is not really heard in the final product at all. It did mean I had to work out a decent voice allocation system though!

I think my memory issues were probably my only problem with using PD in this project – though only indirectly. As I mentioned, they were hardly a problem artistically, however it took me a while to get used to the idea that not everything I patched on a computer would work on an iPhone. Similarly, I had to be very careful about things like relative volumes. In a generative music game like Pugs Luv Beats, the player could quite easily send 15 pugs running around making sound, which mounts up pretty quickly. It means that all of the patches and sound need to be designed to withstand lots of triggering without distorting. None of these things are problems, really, all they require is regular testing on devices and simulators – something that every mobile developer is already used to.

That’s the Pure Data audio engine in a nutshell. The end result is a flexible and powerful audio engine that sounds really great and is fully integrated into Pugs Luv Beats. The game is a great combination of music, silliness, and strategy – there’s a bit of something in there for everyone. You can definitely just play with the game to make beats, or you can try and collect all of the costumes, or you can try and make the most efficient planet ever. You can also explore the galaxies being made by your Game Center friends, to hear what they’re up to.

The background story:

And, just for fun, a silly promo featuring real pugs. Anyone traumatized by the sight of Pd patches, these should relax you.

More information at the developer site:
http://luckyframe.co.uk/pugsluvbeats/game.html

Graphics are good. Graphics are shiny. But when it comes to reality-bending, emotionally-immersive, perception-shifting power, look to sound and music.

At least that’s the feeling you could get after playing Dimensions. Following their reactive music tools and Inception dream states for iOS, RjDj have turned their mind-altering sonics to gameplay. As with previous releases, these tools are powered by the open source visual development environment Pure Data. Pd engineering wizardry here meetings the compositional and sound design prowess of Hans Zimmer.

You can see a bit of how the musical world works in the teaser video above, and the music sound design video below.

But we wanted quite a lot more information. So, CDM got RjDJ’s Rob, Joe, and Martin to share some detailed thoughts on how the game experience is put together and how it works.

The App

RjDj Team: Most games require your full attention when you play them. You either live your life or play the game. Dimensions is different. It’s designed to be played in parallel with your normal life.

Gameplay is intertwined deeply into your daily life. Some dimensions unlock if you are physically active and others unlock if you are quiet. The app automatically detects what you are doing and syncs the game to it making use of every possible sensor on the iPhone.

You stay immersed in the game by listening to augmented sound and the voice of Emily from Mission Control. She guides you through many exciting challenges like collecting Artifacts and avoiding the dreaded Nephilim.

With Dimensions we are very interested in creating a gameplay experience which is between the device based focus of a casual game and the passive use of listening to music. Its a game which you play by listening – a game that place in parallel to your everyday life.

The Tech: Reading Files

We built our own version of readsf, rj_readsf, in order to be able to read compressed audio and make the samples available for processing in Pd. One advantage of readsf is that possibly lengthy audio assets do not need to be loaded into memory. If memory is limited, especially when Pd may be running in the background, limiting exposure to system memory warnings helps keep the app running and the music playing. Given that compressed audio is roughly ten times smaller in size than uncompressed audio, and that audio assets make up the majority of the size of the entire app, it is a huge benefit to be able to deliver and read compressed audio assets directly, without the need to decompress in memory or onto disk. Dimensions requires that several dozen such players be open and viable at any time, and special consideration was given to concurrent behaviour. rj_readsf can loop a file when it gets to the end, and it indicates with a bang when a file has been loaded (an asynchronous operation) or the end as been reached (in the non-looping case). rj_readsf is built on iOS standard APIs and can read any file format that iOS can.

Ed.: I’m waiting to hear if rj_readsf will be open-sourced. The issue of reading files is one we’ve had around libpd recently. While their rj_readsf sounds great, my sense is the best long-term solution will be a similar object that is independent of the APIs of any one OS, so this same set of problems may need a different solution for the open source community more generally. (Building such a tool is absolutely possible, though it might require more effort.)

The Music, and How the Music Plays with You

The music of Dimensions uses various different techniques from straight sample playback to audio analysis and synthesis:

Realtime manipulation of audio input from the mic:

This is perhaps the most recognisable technique we use. We process audio from the iPhone microphone live in many different ways. It’s kinda like a feeling of being inside the music.

The key thing we do with effects is attempting to analyse the environment of the player / listener and then making appropriate things happen within the effect. For instance, the Flux Dimension features a filterbank on the mic input. We analyse the incoming audio from the players environment and make the filter frequencies change as events occur ( either due to pitch changes or onsets ) this gives the impression that objects and activity around the player is somehow “playing” the music.

In the Ghost Dimension there is an effect which records audio whenever it detects an event, then scrubs repeatedly forwards and backwards through the sample using granular techniques stretching it out in time. This manipulation accentuates the textural and pitch based qualities of the sample as it repeats and works well with the atmospheric music Hans Zimmer composed.

Dynamically-controlled stems:

All the Dimensions use stems and hits from a conventional sequencer in some way, re-arranged live on the device relative to how the player is interacting. These stems were mainly composed in Cubase and Logic.

For example, in the Kinetic Dimension we feed accelerometer data from the device into Pd and drive the music from that. The player hears more energetic beats when they go for a run, but if they stop at the lights to cross the road, the drums immediately drop away. This was achieved with a large number of hits with all the rhythmic sequencing happening in a hybrid reactive / generative way live on the device.

In the Tranquil Dimension, the music introduces more stems the longer the player is quiet. If they make too much noise the music “shrinks away” from them and becomes quieter. If they stay in a Zen like peaceful state, the music grows into a kind of crescendo of serenity.

Reactive synthesis:

We often control parts of the music by doing a frequency analysis of incoming microphone audio from the device and then using those frequencies to determine the notes synths will play within the music. The Travelling Dream in Inception the App uses this extensively. Tranquil Dimension in Dimensions also uses onset and frequency changes to trigger synth melodies in the music.

The synths we use range in complexity from very simple additive synthesis to some great synth patches from the rjlib by Frank Barknecht and Andy Farnell.

Generative approaches:

There are some sections within Dimensions which are generative. These play back prepared samples as well as triggering onboard synthesis. They also feed the results of this through various live sampling and glitching patches. They are governed by various sets of rules which have various long term parameters, like adjusting to the intensity of the audio environment of the listener, or how dense areas of music have been around the present time.

Sample triggering:

Ghost Dimension uses a simple but effective technique of triggering samples from the music on onsets in the environment. This can cause some real jump out of your skin moments. We combined this section with a randomised very short delay on the mic which acts almost like a resonator, turning the mic sounds into creepy atonal pitched noises.

Sound Design

The main hub section in Dimensions, called the Launch screen, acts as a entry point to your augmented adventures. It also displays all available Dimensions via the floating tile icons.

Visually, these represent a snapshot of your previous experience using your location at that time. Sonically we wanted them to have an aura or energy from the Dimensions themselves.

SoundCloud examples:
Dimensions Sound Design Example by rjdjme

Example of using mixture of synthesis and samples to create user feedback when interacting with Dimension icons in the game.

Map Tile Down: several recordings of a synth in Pure Data that is played when the tiles are touched. Each one is slightly different due using two detuned oscillators.

Map Tile Open Only: a sample from Logic Pro for the woosh sound when showing the information view.

Map Tile Click: a sample from Logic Pro for touch events.

Map Tile Open: recording of how it sounds when put together.

Sound is a mixture of samples and real-time synthesis. The energy sound is made using two oscillators (one detuned) to create some modulation for a glowing effect. Added to some harmonics to make it more of a beam sound and some chorus and reverb. The open tile is made in logic, when closed it’s the same sound but reversed and pitched down in Pd.

We wanted the tiles you tap on to feel like each Dimension has some sort of energy radiating out.

Here’s a sense of how the sound design works in the game:

It’s an ambitious app, and the whole cost is US$2.99. I guarantee it’ll change your world more than a latte. (Well … unless we’re talking a really crazy latte. And that might not be legal.) As sometimes-CDM contributor Jaymis Loveday notes, there are terrific choices in coloring Google Maps, and how modes change based on ambient sound and motion.

Requires an iPhone 3GS or better, or third-generation iPod Touch or better, or an iPad.

Dimensions @ iTunes Store

More reading:
Dimensions Augments Reality Purely Through Sound [Leigh Alexander, one of my favorite game writers, for Gamasutra
The Roundabout Tapes – RjDj now plans to game reality with sound [TCTV] [Techcrunch EU]

First revealed last month, PreenFM is an open source hardware synth. As the name implies, it’s an FM synth, with some very serious specs: up to six-operator FM synthesis with some nine algorithms, up to 4-voice polyphony (depending on algorithm), glide, selectable LFOs, modulation matrix, and preset banks with SysEx support. It’s all usable via a display and MIDI support.

It’s also fully open source hardware; whereas early efforts often had commercial restrictions attached, PreenFM is free for use under the GPLv3 and Creative Commons. And it’s got a unique platform under the hood: the open source LeafLabs 32-bit development platform gives this some serious horsepower. It’s very much in contrast to the ultra-inexpensive 8-bit brain of our own MeeBlip synth; think of the MeeBlip as an exercise in what you can do with a little two-stroke engine versus the V8 muscle in this. (The creator says the MeeBlip helped inspire his creation – yes, synths are multiplying!)

You may have glimpsed the PreenFM making the rounds online, but I got creator Xavier Hosxe to tell us more of the gory details and share some sounds.

CDM: So this is all based on the Leaf platform?

Xavier: Yes it’s built around a LeafLab board.
I coded a first version on the Maple [development board]; then when they announced their “Maple Mini,” I realized it was going to be very easy to plug it into a PCB.
I’m not directly connected to [LeafLabs]; I participated in the forum and learnt many things from the team.

They are very friendly and helpful.

What was it like working with the Leaf?

The LeafLabs boards uses an ARM Cortex-M3 microcontroller.

It’s a 32-bit chip runing at 72Mhz that can do 32-bit multiplication in 1 clock cycle, has 128Kb of flash [memory] and 20Kb of RAM. That seems very few but it’s not, PreenFM software uses 92Kb for the moment.

LeafLabs provide a Linux/gcc toolchain that allows to develop in your IDE of choice… Eclipse in my case, which is very confortable.

They also provide a strong bootloader and some libraries that worked perferfeclty for my needs : Usart (Midi), I2C (EEPROM), LiquidCrystal (LCD).

What will you get in the PreenFM kit?

All you need to build yourself a complete synth: PCB, screws, resistors, ICs, audio/midi jack, box, 20×4 LCD, encoders, knobs, buttons… even an USB cable [for power].

You’ll also get a Maple Mini board with PreenFM soft preloaded. The Maple Mini is easily updatable, and you can experiment lots of different things with it.

PreenFM C++ source code is available on GitHub. It’s easy to read and modifiable. If you want to see your name to welcome you on the boot screen, go ahead

To build the kit, you only need a soldering iron and some solder.

There will be 2 differences with the photos you can see on the site: the final PCB will be blue (I should receive them next week). [Ed.: See the photo in blue at top; the orange one is included here for variety! -PK]

Xavier also sends along some welcome news:

Here is a sound I can get with my soon-to-be-released StepSequencer feature in PreenFM.

This is a single voice of a simple 3-oscillator voice. 1 very slow LFO + 2 * step sequencer routed to the modulation indices.

PreenFMStepSeq by cdm

Here’s a sound sample:
PreenFM 1 by cdm

http://www.preenfm.net

Alongside its Pro Tools 10 and HDX unveiling, Avid turned some heads by recently announcing it was replacing its RTAS and TDM formats to a new format called AAX, “Avid Audio eXtension.” Now, your first reaction may not be unbridled enthusiasm, exactly: it seems the last thing users are likely saying is, “yes, please, I’d like a new plug-in format to worry about.” But I wanted to give the engineers at Avid a chance to tell us what they were thinking and why they made the move.

Avid’s product announcements have unfortunately coincided, presumably because of the financial calendar, with unpleasant restructuring and downsizing news, a topic NYC-based audio engineering site SonicScoop takes up. However, I prefer to focus here on the engineering side of what’s happening; we can look at Avid’s business and the changing business landscape another day. (For what it’s worth, I’m not as bleak as SonicScoop about the industry at large – least of all because I think the larger audio market remains healthy, even if Avid has been caught adapting to a new marketplace.)

The picture painted by Avid is one of a smooth transition to AAX. Now, of course, you’d expect them to say that, but I think they do have some specific technical reasons that, even with the change of name, the shift should be friendly to Avid developers. I’ll let them explain, though.

Bobby Lombardi, Senior Pro Tools Product Manager goes into the technical details of what AAX, and what it means for Pro Tools developers and users.

CDM: The main draw appears to be the ability to switch between native and DSP-based processing more easily, correct? From the end user perspective, can you get into specifics on what a user will see and how this will differ from RTAS/TDM?

Bobby: Visually, a user will see that the RTAS/TDM pop-up on the Pro Tools 10 Plug-in header has changed to Native/DSP. The exciting part is what they will hear. In the past, with HD Accel systems using a 24-bit fixed point processing environment, and host-based systems using 32-bit floating point processing, the gain staging could be quite different and produce significantly different results. With the introduction of AAX, sessions that migrate between host-based and DSP-accelerated HDX Pro Tools systems will sound identical.

How much work will it be for developers to migrate from RTAS/TDM to AAX?

Moving from an existing RTAS plug-in to AAX Native is relatively simple. Plus, once a developer has an AAX Native plug-in running, it will take a small amount of development effort to support AAX DSP. In comparison to TDM 56k used with the legacy HD hardware, AAX DSP is much easier to support and developers do not require specialized skills in writing 56k assembly code, so it opens up the opportunity for many developers to create DSP accelerated versions of their plug-ins.

We’ll still see parallel, separate versions of plug-ins for AAX Native and AAX DSP, correct? And some will, as with RTAS, presumably be native-only?

This is really up to the individual developer. Some developers may find it strategic to support one or the other, however Avid’s goal with this new format was to simplify plug-in development and reduce the complexity to support accelerated hardware.

RTAS and TDM are listed as “legacy” formats. Is Avid making any commitment to how long they’ll last?

The RTAS and TDM formats will continue to be supported in the 32-bit versions of Pro Tools but will not be supported once Pro Tools is released as a 64-bit application. The new AAX plug-in format is the bridge to 64-bit plug-ins for the Pro Tools and Media Composer platform.

Avid has expressed a desire to embrace open formats, but why is there still no common, industry-wide plug-in format? (Actually, there may be no good answer to that question, but I feel obligated to ask!)

We absolutely do embrace open platforms because they can open workflows that enhance the user experience. Part of the experience we need to ensure is that it’s stable, integrates well on our control surfaces, and provides a long-term commitment to the customer. For these reasons it’s important that we can design the plug-in architecture. For example, without designing AAX we could never give customers a plug-in environment that supports both DSP accelerated and native Pro Tools systems and ensure they delivered 100% sound parity. We hope that by providing a modernized AAX SDK for plug-in development, developers will find it easier not only to support Avid’s products, but also finding it easier to develop for non-Avid plug-in formats.

Any word on when we’ll see third-party plugs with AAX support?

More developers are coming online each week with their AAX offerings. We had over 25 developers showing over 60 individual AAX plug-ins at the AES tradeshow in NYC last month, and expect to see many more at the upcoming NAMM tradeshow. With the ability to provide all Avid third party developers a optimized development path to DSP-accelerated plug-ins, we do expect to see more DSP-accelerated plug-ins on the new AAX platform than the legacy TDM platform.

Thanks, Bobby. And for one other take, I happened to get to talk to Universal Audio about their transition. UA, given that they have their own DSP platform and support Avid’s rival packages, certainly aren’t dependent in their business on the Avid ecosystem (though you can be sure it makes a big part of their market). Anyway, here’s what they say; I’m guessing other third parties would say something similar, but if you’re a third party reading and wish to comment, please do so, and don’t let the fact that I only have UA here dissuade you.

Lev Perrey, Universal Audio Director of Product Development, responds to CDM:

Universal Audio intends to support AAX Native in conjunction with UAD-2 DSP accelerators ­ exactly like we have just completed with RTAS support in UAD Powered Plug-ins v6. There is no announcement as of yet as to when the transition to AAX will be complete but we are actively developing and committed to the Pro Tools platform. Pro Tools 10 does support RTAS and initial testing with UAD plug-ins shows it to work just like Pro Tools 9.

As for the significance question, for UA moving to AAX Native should be similar to our recent migration to RTAS ­ although it will be easier for us now moving to AAX since we have fully invested in direct Pro Tools development and better understand the Avid SDK.

We’ll continue to follow this story. Thanks to Avid for getting us more details; I know it’s appreciated.

More info:
AAX Audio Plug-ins @ Avid

Integrated Melodyne pitch correction in PreSonus’ Studio One is made more interesting by the technology behind it. Celemony this week describes a new technology they call ARA, or “Audio Random Access.” The notion is this: rather than just receiving or generating audio signal, the plug-in gets access to audio data. That means you can actually write a plug-in that rewrites the audio content in a recorded DAW track, as Melodyne does in Studio One.

As developer Celemony describes it, “ARA opens an additional channel of communication through which the DAW and plug-in can exchange information about the audio file, tempo, pitch, rhythm and much more, which allows them to work together considerably more closely.”

It’s the ability to exchange audio data information that seems the most compelling. Previously, audio processing plug-ins simply took buffers of audio signal from the DAW. You could “look ahead” further into that signal by increasing the buffer (and thus latency with it), but generally speaking, you’re doing the processing in something that approximates real-time. ARA in the example of Melodyne gives you access to an entire recorded track without having to transfer the audio file to and from the plug-in.

Celemony says this is “an extension of the existing plug-in interfaces,” not a new plug-in format. (If it were the latter, I’d have to point to this xkcd cartoon.) I’m still obligated to express some skepticism about how widely this will be adopted, or if it can be considered a “standard” extension, though they do promise additional vendors soon. (Implementation would seem to be by necessity on a host by host basis – and then once you have the host, a plug-in creator might add support.) It’s a proprietary technology, but then, so are the plug-in formats currently in wide use (AU controlled by Apple for Mac OS, VST by Steinberg, and RTAS by Digidesign, unless we see more of LV2). For now, though, we’ll have to see if the idea itself can extend what a plug-in can do. Check out the videos for more.

celemony.com/ara (no documentation for developers, but there is an email address to use if you’re interested)

Max 6 in Public Beta; For Home-brewing Music Tools Graphically, Perhaps the Biggest Single Update Yet

Just because a music tool fills your screen with tools and options doesn’t necessarily make it easier to realize your ideas. From the beginning, the appeal of Max – as with other tools that let you roll your own musical tools from a set of pre-built building blocks – has been the blank canvas.

Max 6 would appear to aim to make the gap between your ideas and those tools still narrower, and to make the results more sonically-pleasing. The reveal: it could also change how you work with patches in performance and production. I was surprised when early teasers failed to impress some users, perhaps owing to scant information. Now, Max 6 is available in public beta, and the details are far clearer. Even if Max 5 was the biggest user interface overhaul in many years, Max 6 appears to be the biggest leap in actual functionality.

It’s what I’d describe as a kitchen-sink approach, adding to every aspect of the tool, so there’s almost certain to be some things here you won’t use. What could appeal to new users, though, are I think two major changes.

More visual patching feedback and discoverability. First, building upon what we saw in Max 5, Max’s approach is to provide as much visual information as possible about what you’re doing. It’s probably the polar opposite of what we saw earlier this week in something like the live-coding environment Overtone: Max’s UI is actively involved with you as you patch. There are visual tools for finding the objects you want, then visual feedback to tell you what those objects do, plus an always-visible reference bar and rewritten help. This more-active UI should make Max more accessible to people who like this sort of visual reference as they work. No approach will appeal to everyone – some people will find all that UI a bit more than they like – but Max’s developers appear to be exploiting as much as they can with interactive visual patching.

Multiple patches at once. New objects for filters and data, a 64-bit audio engine, and low-level programming are all well and good. But the change that may more profoundly impact users and workflow is be the way Max 6 handles multiple patches. Max – and by extension Pd – have in the past made each patch operate independently. Sound may stop when you open a patch, and there’s no easy or fully reliable way to use multiple patches at once. (Compare, for example, SuperCollider, which uses a server/client model that lacks this limitation.) That changes with Max 6: you can now operate multiple patches at the same time, mix them together with independent volume, mute, and solo controls, and open and close them without interrupting your audio flow. (At least one reader notes via Twitter that you can open more than one patch at once – I’d just say this makes it better, with more reliable sound and essential mixing capabilities.) Update: since I mentioned Pd, Seppo notes that the pd~ object provides similar functionality in regards to multiple patches and multi-core operation. This has been an ongoing discussion in the libpd group, so I think we’ll revisit that separately!

One upshot of this change: some users have turned to Ableton Live just to host multiple patches. For users whose live performance set involves Ableton, that’s a good thing. But it could be overkill if all you want to do is bring up a few nifty patches and play with them. Now, I think we’ll start to see more people onstage with only Max again. (Check back in a few months to see if I’m right.)

Here’s an overview of what’s new:

• Discoverability: A “wheel” makes the mysterious functions of different objects immediately visible; Object Explorer makes them easier to find, and new help and reference sidebar keep documentation close at hand.

• 64-bit audio engine

• Open multiple patches, solo and mute them, open and close them without stopping audio, mix audio between them with independent volume, and take advantage of multiple processors with multiple patches.

• Low level building blocks: You don’t get new synth objects, but you could build them yourself. New low-level data-crunching goodness work with MSP audio, Jitter Matrix, and OpenGL textures

• More JavaScript: An overhauled JavaScript engine makes JS scripting faster and more flexible, and there’s a proper text editor with syntax highlighting (though, of course, you may still prefer your own).

• New visuals: Vector graphics and “HTML5 Canvas-like” UI scripting (though to me it’s a shame this isn’t just the HTML5 Canvas). There are also massively-expanded Jitter powers, but those are best left to our sister site Create Digital Motion.

• Filters: New filter-visualizing tools for audio filter construction and manipulation.

• Dictionary data type and associated objects let you describe information in a more structured way (all kinds of potential here from control to composition)

• Projects now let you organize data, media, and scripts in the manner more associated with conventional development environments

• What about Ableton? No news on that front, but I expect more soon. Max for Live users will at the very least get the advantages above, since Max for Live is really Max inside Live.

Looking over all that Max does, I have to say, I’m really amazed. I wonder if computer musicians ever pause to consider how fortunate we are. Even if this isn’t the tool for you, its availability – compounded by the availability of a range of other tools – is itself worth reflection.

Max is a program that shouldn’t exist, doing a number of things it shouldn’t do, for a user base that shouldn’t exist, doing things they shouldn’t be doing.

It doesn’t make sense that you could maintain a commercial project for this kind of audience, that you’d wind up with something this mature and powerful that had a continuous lineage stretching back to the 1980s. It doesn’t make sense that musicians would embrace such a tool and produce invention. The only explanation is sheer love.

Then, even as Max reaches new heights, some of the alternatives you have for making your own music tools are simultaneously growing by leaps and bounds. They provide very different approaches to music making (compare Overtone and SuperCollider, or Pd and libpd, or AudioMulch, or new Web audio tools). There really aren’t many fields that have this kind of choice, free and commercial, in their medium. In science and engineering, there’s private and public funding, producing some amazing tools but nothing with this kind of meeting of power and accesibility. There’s just something about music.

The fact that Cycling ‘74 can maintain a business model – just as open source projects maintain volunteer contributions – is a testament to sheer passion and love for music, and a commitment to perpetually re-imagining how that music is made from an atomic level up. There was a wonderful piece on C creator and UNIX co-creator Dennis Ritchie, whom I remembered yesterday, that observed that what he did was to do what others said couldn’t be done. From Max itself to what people make with it, I think that fits nicely.

So, have a look at the public beta, and let us know what you think. The release of Max 6 has caused more people to ask what this means for Pd and other tools, or even whether to patch things from scratch at all, but I’ll leave that question to a bit later. (I do have my own opinion about which tool fits which circumstance and user, but that’s best left to a separate discussion.) For now, you can try Max yourself and see what the fuss is about. If it doesn’t fit your means of music-making, know that you have a wide array of other options – pre-built to low-level code to old-fashioned tape-and-mic approaches, and everything in between. Go out and listen and see what you discover.

http://cycling74.com/downloads/max-6-public-beta/

Cyril Stoller shares this project, for a variety of multitouch devices (Mac / Windows / Linux / Android, screens and projection) through the also-free-and-open-source Kivy framework. It’s inspired by the brilliant Haken Continuum fingerboard, but whereas that more tactile controller is hard to get, this runs cheaply all over the place. (It could also be a way to practice and save up for a Haken, it occurs to me.)

Thanks to Graham Comerford for the tip. I wonder what other ideas might apply to playing multitouch as an instrument.

More discussion on the excellent NUI forum – a great spot for alternative and gestural interaction chatter and resources:
http://nuigroup.com/forums/viewthread/13227/

The tablet – or at least the iPad – is beginning to look like a terrific accessory for lovers of MIDI and hardware. With its compact form factor, it coexists nicely with your MIDI gear and lets you focus on sequencing, perhaps moving to the traditional computer to finish up your track, mixing, and the like.

And it’s spawning MIDI sequencer apps that imaginatively explore ideas for how to create sequencing, all with an immediate touchable interface.

The latest entry: Genome MIDI Sequencer claims to be the “first true pattern-based MIDI sequencer for iPad.” The word “true” might be debateable, but it looks thoroughly full-featured, with clock sync, MIDI import / export (ideal for working with your desktop Mac or PC), and other must-have portable sequencing features:

• Hard-line MIDI support: Core MIDI, Camera Connection Kit, Line 6 Mobilizer Mk I and Mk II compatibility
• Wireless MIDI: Network MIDI support (works with Mac and, via other support, Linux and Windows)
• MIDI Clock: Start, stop / send, receive
• Lots o’ messages: Note On, Note Off, Pitch Bend, Aftertouch, Channel Pressure and CC’s
• Pattern-based; pattern changes occur on bar boundaries so song stays in sync Ed., oh, come on, where’s the fun in that?
• 16 simultaneous tracks, unlimited patterns

There’s also an on-screen keyboard interface (in case you don’t have a MIDI keyboard handy), song and live modes, incoming CC and note recording, and undo/redo for “most” actions.

Having the multichannel capability alone is a must for people with big studios.

Developer Dave Wallin says he’s been working on this for some time. An experienced dev with tools like bleep!BOX and bleep!Synth under his belt for iOS (as well as Additive and Zero Vector for desktop), he’s got a good perspective on the bigger picture.

Since Dave is a user as well as a developer (as is often the case with music tech), I asked him to tell us a bit more about the app, his thoughts on development, and how you might use this.

He even has some good thoughts, in case you’re wondering, about how this compares with and fits in with desktop studios.

CDM: This looks to me to be the most full-featured MIDI sequencer we’ve seen yet, more than just a pattern or step sequencer (though I like those sorts of things, too). That said, what other applications do you like in terms of MIDI control?

Dave: There are a couple other apps that look close in terms of functionality like Step Poly Arp, Polychord and Brainwave Sequencer. The first two are more focused on ‘performance’ imo with Brainwave being the closer. I wanted to provide some of the functionality you’d expect from a desktop sequencing app but redesign the core interface around touch and just make it simple and fun to use. My inspirations for how the piano roll works are 50% monome / [Yamaha] Tenori-on (in terms of a familiar grid and easy one-tap note entry) and 50% iPad paint application (in terms of using gestures to navigate). It also takes some inspiration from [Nintendo] Game Boy tracking apps in that I keep the octave / bar navigation locked to a grid. GMS isn’t locked into 16 step patterns and you can adjust the grid and bar length more freely than some other apps.

Ed.: Of course, at iPad app prices, you can easily pick up all the apps above, and they all have some appeal – yes, I know it’s time for another iPad app round-up. A MIDI-specific one could be fun; I’ll work on it! -PK

How are you using this in your studio – what gear are you using?

Right now I am using it with my [Access] Virus b (desktop [synth]) and a drum machine primarily. It’s great with the Virus since it can do up to 16 parts and I can save these setups and easily recall them later without having to worry about program changes or anything like that. The drum machine saves me from having to waste Virus parts on drums. With just those two pieces of gear and not much else I can make some pretty decent sounding songs. My current work flow is to jam around a bit with Genome and the gear and when I come up with something good, record some loops or tracks it and bring it over to the desktop for additional processing and arrangement. My actual studio doesn’t even have a computer in it at the moment – it’s in a totally seperate room.

At what point do you imagine people would use the iPad sequencer versus, say, the sequencer on their desktop machine?

What I have learned from my own experience and from talking to other people is that the iPad is not a total replacement for the desktop or even for a laptop, unless you’re doing some very minimal stuff. The iPad is more of a convenience and leisure device at this point. I have limited time to relax and if I can spend 30 minutes chilling on the couch, making a tune on my iPad, it’s a win for me. It doesn’t have to be a whole song – it could be some musical ideas, sounds or loops that I end up using later on the desktop or in another app.

With Genome, I think the big wins are portability and ease of use. If I want to jam with a friend, I can throw my iPad in a bag and bring it with me. Sometimes I don’t want to have to deal with all the overhead that comes with a desktop sequencing application like hardware setup, configuring plugin directories, or GUI’s that are packed to the brim with controls. GMS is very easy to plug in and get started within a few seconds.

I’m curious about your reaction to the Open Music App Collaboration Manifesto posted recently. [See Synthtopia coverage.] Any of these issues relevant to your work? What would you like to see in terms of inter-app integration?

I actually just posted some thoughts on this to their Google Group this morning. My response is here:

http://groups.google.com/group/open-music-app-collaboration/browse_thread/thread/75cdd385048fa3ce

Communicating with other apps on the same device is something that I think Genome will be very good for (after some minor updates). Right now, not a lot of apps support Network MIDI, let alone regular CoreMIDI. How well this works will depend on a lot of factors and we’ll just have to wait and see how extensively it’s adopted, both by developers and by users. I could imagine an ecosystem where you can string a bunch of apps together and make songs. The first step right now is to come up with a spec for developers to implement and maybe a way to certify that apps have implemented it properly. If done well, I think there could be some good cross-promotion opportunities to entice developers to do it and it could catch on, much like with Audio Copy did.

Had you evaluated Android at all, or other platforms?

Some work is being done to port libNUI [C++ framework] (which I use for all my apps currently) to Android, however it seems the state of MIDI on Android is nowhere near what it is on iOS. If some well known hardware makers make an accessory or if a good SDK becomes available, I would definitely look into it. GMS is being ported to a Mac desktop app and will be available in the Mac App Store at some point. The desktop version will be almost identical, except for some changes made to adapt it to work with a mouse, instead of multitouch. I think a simple, cheap MIDI sequencer will be attractive to some people and the amount of time to port it is minimal for me.

Ed.: Hopefully more to say about the state of Android MIDI soon; while wired accessories may not be as convenient, wireless MIDI, especially via Bluetooth, holds some promise. I can certainly make a good argument *against* a developer immediately rushing to support Android, but it remains something we’re watching. I also hear these computer things can do MIDI. -PK

US$12.99 in the iTunes App Store, for iPad (iOS 4.2+)

http://www.whitenoiseaudio.com/apps/genome-midi-sequencer/

http://www.whitenoiseaudio.com/