Sword & Sworcery, the iPad album-as-game, has gotten plenty of love from this site before, and recognition for friend-of-the-site composer Jim Guthrie. (See Jim open up about what happened behind the scenes.)

Now, it seems the Canadian songwriter and soundmaker will meet up with some of the biggest game composers from Japan in a unique remix album. In the lineup:

Michiru Yamane (Castlevania: Symphony of the Night)
Akira Yamaoka (Silent Hill)
Baiyon (PixelJunk 4am)
Mitsuto Suzuki (Final Fantasy XIII-2)
macotom3 (JP chip music artist)
Decasségui Hip

The release will be on iTunes and Bandcamp – the new havens for the indie artist (and certainly for game music).

It’s terrific to see this cross-cultural, international fusion. And the preview clips released, while brief, should whet the appetite of fans of musical invention in the game medium. Have a listen below.

Preview tracks:

00:00 Little Furnace (00:15 PostProduction Mitsuto Suzuki Mix) Mitsuto Suzuki
00:37 The Ballad of the Space Babies (00:52 Supernova Babies Mix) macotom3
01:14 Lone Star (01:30 Baiyon Just Walking Remix) Baiyon
01:54 The Maelstrom (02:09 Radiant Darkness Mix) デカセギ・ヒップ
02:32 The Prettiest Weed (02:48 *’s Ambition Remix) デカセギ・ヒップ
03:10 Unknowable Geometry (03:27 Symphony Mix) 山根ミチル
03:48 Bones McCoy (04:04 Akira Yamaoka Mix) 山岡晃

The remix project coincides with the Japan release for Sword & Sworcery. Remember, kids, localization is the future of everything. Teaser for that:

More:
http://sworcery.jp/remix/

We hear you: amidst the various music doodling tools calibrated for casual taps on a phone screen, you want something you can actually make your own with your own sounds. Here’s one example of that: triqtraq. It’s got the “tap it on the bus to improvise a pattern” feel of some of the apps we’ve seen lately, but without sacrificing the sort of control that might keep you coming back to it. And because it lets you include your own samples, there is some depth. (No Audio Copy/Paste, though, before some reader chimes in and points that out.)

Creators Sebastian Schatz, Olaf van Zon, and Joerg Peschel say that they wanted to scratch an itch similar to the one we’ve heard from readers. They write in the press release: “With this app, we (a bunch of friends that are crazy about electronic music) are trying to take music jamming on the iPhone to the next level. Instead of creating an app that simplifies the creation of music on an iPhone so much that you are bored with the possibilities within a week, triqtraq gives you a lot of flexibility.”

Features:

create musical patterns fast and intuitively, by programming live or by using the ‘step edit’-feature
automate parameters like pitch, filter, delay, decay and level in real-time
change samples or sample kits on-the-fly while jamming
edit multiple tracks simultaneously
store up to 16 patterns and 32 samples per session
specify the length of a sequence per track, or set the automation length for each parameter individually
switch seamlessly between patterns
use the loop range feature to create poly-rhythmic sequences
use sounds from the 350+ factory sample library
import your own sounds via iTunes file sharing

Find it in the CDM apps section:
http://apps.createdigitalmusic.com/apps/triqtraq-jam-sequencer

Among other information, the developers have loads of tutorials on their site:
http://www.triqtraq.com/index.php/tutorials

You’re not, in fact, a lonely bedroom musician with no friends. You’ve got friends. You’ve got collaborators. You’ve got audio and visual artists who want to play with your laptop with sync piped between you, and yet wireless synchronization for laptop performance has often been an elusive ideal. Here, in spectacular fashion, you can see it all come together over WiFi with Ableton Live, a projection-mapped coffeemaker, and everything from percolation sounds to the keypad sound effect from The Hitchhiker’s Guide to the Galaxy.

This just in from Barcelona – a new Max for Live plug-in is making laptops and coffee pots jam in harmony. A proper review is in order from CDM. (Well, once we find a few friends… by astounding coincidence, I’ll be surrounded by Ableton users – visual and audio – next week in Rome. I’ll make it happen.) But that’s no reason to deny you the pleasure of the video now. It’s the best part of waking up.

More: http://www.ooeevv.com/

Some of you are probably already sitting on top of a Max for Live license for your copy of Ableton Live. It’s there, just waiting to do … something. Maybe you’ve loaded one of the many extraordinary patches out there – good move. But as for building your own patches, you may easily have become overwhelmed by choice. Max is a blank slate, and a blank slate that can do everything can make it hard to start with anything.

It’s easy to overlook simple first steps. Max was originally built just to do simple math on messages, before it even had audio capabilities. So that means simple message processing is a great place to start. The Ableton Cookbook’s Anthony Arroyo introduces Max for Live in just that fashion, by starting you out building an arpeggiator. No fancy granular audio processing, no mind-bending processing of the event engine in Live – just some simple, old-fashioned arithmetic. You’ll learn MIDI in, MIDI out, monitoring what’s going on, basic math, and sliders. You can always go deeper after that.

This is the first of more videos to come, all promising to focus on simple devices; I’m curious to see where they go.

Not quite your speed? Here are two more intro tutorials – and one advanced tutorial – to get you going.

Ready to get a little advanced? It’s an older video, but still relevant to new versions of Live – don’t let the date stop you. Here, a serious Max for Live guru goes deep into spectral mixing. It’s not at all the simple, step-by-step approach I’ve just endorsed, but … hey, you’re still with me, and this is fun. Description:

In this video new addition to the Dubspot team Dave Linnenbank, creator of Puremagnetik’s Max Fuel collection of patches for Ableton and Cycling 74′s Max For Live walks us through his Spectral Mixer patch. It allows you to adjust the volume of the loud, medium and quiet parts of a sound and create some very interesting sounds.

Blog post and downloads: Max for Live Tutorial :: ‘Spectral Mixer’ [Dubspot Blog]

Speaking of the iPad and music making, Japan’s Human Boot Project draw a music video release for their latest track directly from screen as they play, working in the app Tabletop. I really love the personality of the resulting sounds. There’s a rawness to their music, the dense downtempo jazz-inspired electronics, which you can enjoy on their new EP “Picture Over Picture.” It’s a Beatport exclusive from Greece’s V.I.M. Records:

http://www.beatport.com/release/picture-over-picture-ep/887326

The artists say it’s “Downtempo beauty to Jazz Electronica futurism !!” (Exclamation points theirs, but … I agree.)

Human Boot Project are a Japanese underground music unit. This is a genre breaker fusing Jazz with Rock and Drum and Bass and one to get the synapses twinging.

http://www.humanboot.com
http://www.facebook.com/humanboot
http://soundcloud.com/hbp

Tabletop is on sale now for 99 cents currently free, though note that a lot of what you’re seeing here requires in-app purchases of add-ons:
http://apps.createdigitalmusic.com/apps/tabletop

Artist Paul Salva consulted on the app, and in a video with the LA Times walks through the features in detail, if you want a closer look at the app. (Interestingly, he notes many in the LA scene are going iPad live.)

Could a piece of software make you think differently about sound? Could it reflect ideas, the culture of listening?

The developers of the SUFI series of plug-ins seem to think so. In place of screencasts showing which knob to turn which way, they head with a video crew to Morocco. The “instruction” might be about the value of reflection or call to prayer, about living as much as how to use a tool. You can see the first two examples: a meditation on the idea of daily interruptions in the soundscape coming from God, and a collection of electronic drones set to a beautifully-shot backdrop. The interfaces are rendered in graphics and (for the vast majority of us) a foreign language, and instead of reverting to the conventions of plug-in design, they assimilate ideas from another culture about tonality and function.

The plug-ins will be released for Max for Live on the 8th of May, and VST plug-ins later on. (Some version of the Max for Live plug-ins are available now – links at bottom.) The collection includes:

• DEVOTION, lowering your volume five times a day at the time of call to prayer
• A drone machine (in the second video, sounding quite nice)
• Four soft synths tuned to Arabic maqam scales. (They describe these as “North African maqams,” but I believe the tuning should be consistent with the use of maqam elsewhere around the Mediterranean and Arabic world.
• One drum machine amidst the synths, Palmas, with a hand-clapping UI (see screenshot).

You have a week to practice learning to read neo-Tifinaght Amazigh script.

Updated: There are in fact no references in the videos here to Sufism, but the creators respond to questions about why they selected this name on their FAQ. As with the videos above, collaborations and friendship inspired their thinking. They write:

The title is an homage to several Moroccan Sufi musicians we’ve worked with over the years who influenced our thinking about musicianship & sound itself, as well as a way of foregrounding the complex but largely unremarked relationship between faith and technology. We’re fascinated with how software and digital environments encode cultural values and beliefs by conditioning choices and framing possibilities. For example, If Apple is a secular religion, selling contemporary magic, then should that change the way we feel about – and engage with – its operating system? The spirit of Sufi aphorisms, we hope, is manifest in these plug-ins. At a literal level, many of the roll-over infotexts come from Sufi verse.

Apart from being an interesting “cross-cultural” exercise, though, these plug-ins can serve as a reminder of two things. First, design choices are constrained only by your imagination. Aside from any perceived cultural values, you can really make software do, theoretically, anything – and make any sound. Convention can be a useful tool, but it can also become a prison. Second, the creators consider VST compatibility as a way to reach users in the Middle East and Africa. Whether this particular effort is successful or not, those are massive and growing audiences. (To anyone reading there, by the way, hello from way up at this end of the Northern Hemisphere!) Of course, these plug-ins will be just as foreign to nearly all of that audience as it is to, say, producers in Melbourne or London, but as we watch the videos from Morocco, it’s worth considering just how small our Internet-connected planet is – and how wonderfully-vast the spaces between us, and the possibility contained there, remains.

Software can serve for a medium for collaboration, as in this case, which ties together a variety of backgrounds from traditional producer to Amazigh musician. The Amazigh people, tying together modern Arabic culture and language with Phoenician roots (much like my own Lebanese ancestry), represent a rich practice of music. Just as the remote, historical world of J.S. Bach might direct a modern software plug-in, these can, too – and in living fashion.

The work is led by Jace Clayton (DJ Rupture), with programmer Bill Bowen, designer Rosten Woo, Amazigh musician Hassan Wargui , and videographers Maggie Schmitt and Juan Alcón Durán. The creators report that “a physical Sufi Plug Ins Forever Box is expected for late 2012, and Clayton is currently preparing an installation version of the Sufi Plug Ins.”

Mark your calendar for next Tuesday, or join the mailing list at the site. More information:

http://www.beyond-digital.org/sufiplugins/

Thanks, Jesse Engel!

As seen on maxforlive.com (thanks, David):

Devotion: http://www.maxforlive.com/library/device/1140/devotion
Drone: http://www.maxforlive.com/library/device/1139/drone
Palmas: http://www.maxforlive.com/library/device/1138/palmas
Hijaz: http://www.maxforlive.com/library/device/1137/hijaz
Bayati: http://www.maxforlive.com/library/device/1136/bayati
Saba: http://www.maxforlive.com/library/device/1134/saba
Khomasi: http://www.maxforlive.com/library/device/1133/khomasi

It starts as just another toy to play around with in a few minutes of distraction in your Web browser – as if the Web were short on distraction. But then, something amazing can happen. Like a musical Turing Test, you start to get a feeling for what’s happening on the other side. Someone’s stream of colored dots starts to jam with your stream of colored dots. You get a little rhythm, a little interplay going. And instead of being a barrier, the fact that you’re looking at simple animations and made-up names and playing a pretty little tune with complete strangers starts to feel oddly special. The absence of normal interpersonal cues makes you focus on communicating with someone, completely anonymously, using music alone.

Dinah Moe’s “Plink” is the latest glimpse of what Web browser music might be, and why it might be different than (and a compliment to) other music creation technology. You can now create private rooms to blow off steam with a faraway friend, or find new players online. It’s all powered with the Web Audio API, the browser-native, JavaScript-based tools championed by Mozilla. That means you’ll need a recent Chrome or Firefox (Chrome only at the moment; this is a Chrome Experiment), and mobile browsers won’t be able to keep up. But still, give it a try – I think you may be pleasantly surprised. (Actually, do it right now, as you’ll probably be doing it with other CDM readers. I expect greater things!)

http://labs.dinahmoe.com/plink/

Thanks to Robin Hunicke, who worked with multiplayer design and play at That Game Company’s Journey on PS3 and now on the browser MMO Glitch. I think her friends were more musical than most, because the place came alive after she linked from Facebook.

The browser is becoming a laboratory, a place to quickly try out ideas for music interaction, and for the code and structure that describe music in a language all their own. As in Plink, it can also benefit from being defined by the network and collaboration.

Dinah Moe’s experiments go in other directions, as well. In Tonecraft, inspired by the 3D construction metaphor of Minecraft, three-dimensional blocks become an alternative sequencer.

http://labs.dinahmoe.com/ToneCraft/

There are many reasons not to use Web tools. The Web Audio API still isn’t universal, and native options (like Google’s Native Client) have their own compatibility issues, stability concerns, and – because of security – they don’t do all the things a desktop application will. Desktop music tools are still more numerous, more powerful, and easier to use, so if you’re a reader out there finishing a thesis project, you might look elsewhere. (Actually, you’re probably in trouble, anyway, by any nation’s academic calendar, given it’s the First of May, but I digress.)

But think instead of this as another canvas, and the essential building blocks of interface design, code, and networking as shared across browsers and desktop apps. Somehow, in the light of the Internet, its new connectedness, and its new, more lightweight, more portable code and design options, software is changing. That transformation could happen everywhere.

If you need something to help you meditate on that and wait for a revelation to occur to you, I highly recommend watching a soothing stream of dots and some pleasing music as you jam with your mouse.

Of course, in the end, like a digital mirror, it might inspire you to go out to the park with a couple of glockenspiels and jam the old-fashioned way. But maybe that’s another reason to make software.

(Here’s a video, in case you’re not near a browser that supports the app!)

More, plus reflections on adaptive music:
http://labs.dinahmoe.com/

Equipped with MIDI, a phone or tablet can communicate with a vast range of standalone hardware and computer software for music. So, what to do with that power? Two recent applications show just some of the breadth of possibility, one from Japan, one from Ukraine. One provides an array of powerful tools, combining into one application a lot of functions that have otherwise been available only in separate apps. One takes a more novel approach. Each demonstrates Apple’s increasingly-ubiquitous iPhone and iPad to be a surprisingly-indispensible musical tool.

Let’s look at the more whimsical of the two first.

From Japanese media artist and developer Masayuki Akamatsu (known as aka), MIDIWriter is a bit unlike MIDI tools you’ve likely seen before. It sends MIDI notes not from a piano keyboard or more familiar musical interface, but from the key entry you’d usually use to type in messages. That means the on-screen keyboard – even, as pictured, in another language – can become a musical instrument instead of input method for SMS and the like.

Where things get even more interesting is when you add a Bluetooth keyboard or keyboard dock, for iPad or iPhone. Then, those gadgets become physical input devices. (In the oddest example, a projected keyboard even works.)

http://akamatsu.org/aka/ios/apps/midiwriter/ [lots of great documentation, in both English and Japanese]
MIDIWriter @ apps.createdigitalmusic.com [View, install; US$0.99]

It’s all quite peculiar, but I can think of one particular advantage to doing something a bit unusual: sometimes, the best way to break out of tired musical habits is to face an unfamiliar musical interface.

In the more conventional and practical end of the pool, we have Wiksnet’s MIDI Studio. With rather lovely, refined-looking interface design, the Ukrainian app treads in the competitive waters of iOS MIDI controllers. What it does that those apps don’t necessarily do, though:
1. It adds velocity senstivity, via vibration, as seen in Apple’s own GarageBand but less-commonly in MIDI tools.
2. It combines layouts into convenient configurations, coupling, for instance, knobs with MIDI keys.

And it looks nice. A future version promises editable templates, but for many, having some nice stuff up and running without any additional effort could be a draw. From the developers, a feature list:

• Essential MIDI compatibility (Core MIDI, over WiFi and USB)
• Drum pads with modulation across X/Y axes, velocity
• Two keyboards, each with a different key size, and customizable key/tuning mappings
• Built-in arpeggiator
• Easy mapping of ADSR, etc. to knobs on the keyboard layout

http://www.wiksnet.com/
MIDI Studio @ apps.createdigitalmusic.com [View, install; US$10.99]

There’s no official view, but here’s one fan-made entry:

The Ukrainian developer has done other commercial work, they say, but this is their first unique iOS music app. The next release will have, alongside editable templates, velocity sensitivity, and will fill other user requests.

In the case of MIDIWriter, there’s not a lot of comparison to be made – desktop software has often mapped standard input and keyboards to music, but not necessarily iOS apps. With MIDI Studio, of course, we’re overdue for an overview of MIDI apps.

Let us know how you use these – or other tools, including things that don’t begin with a lowercase “i” – to produce MIDI events in your workflow.

Pure Data is a wonder: a free and open source environment for creating your own musical and multimedia creations with graphical programming, from Miller Puckette, the original creator of Max. You can produce everything from interactive sequencers and drum machines to synths to video performance tools by connecting patch cables visually, and you can run on virtually any platform, from BeagleBoards and Rasberry Pi to Mac, Windows, and Linux desktop. Via libpd, you can target other development languages and environments, embed engines in games, or work with Android and iOS.

It’s so important, in fact, that CDM invites Hans-Christoph Steiner, one of the key developers of Pd-extended, to give us a tour of what’s new. (Note: because Pd-extended includes various additional objects or “externals” that Pd Vanilla lacks, you should be careful when building patches for libpd. What I like to do is use Pd-extended as my editing environment, then double-check patches by opening them in Vanilla to make sure I haven’t accidentally used an object that’s not part of the bare-bones version. I can then substitute an object, copy an abstraction, or if necessary build that external.) -Ed.

The Pd-extended 0.43 release has been brewing an extra long time, about 18 months now, mostly because there are lots of big improvements. We wanted to make sure we got it right, so your patches all work, but the improvements all shine, so its taken a while. It’s now solidly beta, so we’re looking for testers. Download a beta build to try here:

http://puredata.info/downloads/pd-extended/releases/0.43.1

First off, the pd-gui side of Pd has been rewritten from scratch. The focus for most of the recent work has been on the editing experience, making your patching experience as productive and flexible as possible. To give some background, Pd has always been made up of two programs: pd is the core engine and pd-gui is the GUI. Since basically all computers now come with multiple CPU cores, this means that pd-gui will usually run on a separate CPU core than pd, so they don’t step on each other’s toes. pd can entirely take over its own core. If you want to make your patch use more CPU cores, then check out the [pd~] object introduced in the last release, but fine-tuned in this one.

There are so many ideas for making a better editing experience in Pd; this release makes big strides to address the editing experience. There are new features like Magic Glass, Autotips, Autopatch and Perf Mode, all available on the Edit menu.

• Magic Glass lets you magically see the messages as they pass through the cords. Just turn it on and hover above a cord, and you’ll see the messages as they go by. You can even look at signal/audio cords.
• Autotips gives you tips about what an object does, what its inlet expects, and what comes out of the outlets.
• Autopatch mode automatically connects objects as you create them.
• Perf Mode, is a mode for performance that makes it harder to accidentally close windows that are part of your performance.

A whole new Pd Window

The Pd Window is also majorly overhauled. First of all, it’s fast. Much much faster than the old one. You can now print thousands of messages per second to the Pd Window and still edit your patch. No more will an accidental dump of info cause the GUI to freeze up (well, okay, maybe if you send 10,000 messages/second, but that is way too many). There are also five levels of printing messages to the Pd Window: fatal, error, normal, debug, all. If you are only interested in fatal errors, switch the Pd Window to 0 – fatal, and you’ll only see the worst problems. You want to see every single message to debug? Switch to 4 – all, and you’ll drink from the firehose.

There is also the new log library, which lets you easily send messages for those different levels. And all messages logged with the objects from the log library are clickable: when you Ctrl-Click or Cmd-click (Mac OS X) on the line in the Pd Window, it’ll pop up the patch where the message came from, and highlight the specific object that printed it. That even works for many messages from other objects, as well.

The Pd Window also includes very basic level meters for monitoring the input and output levels. And for those who want to play with the GUI in realtime, you can type Tcl code in the Tcl entry field, and directly modify and probe the running GUI.

Customize the GUI with Plugins

One thing that you can do now is customize the GUI using GUI plugins. You can change all sorts of colors, some fonts, and many behaviors. Want to create a new object when you triple-click? Try the tripleclick example plugin Want to make the patch cords disappear when you leave Edit Mode? Check out the “only show cords in edit mode” example. Those are the simple ones. There is also Tab Completion, a search engine for the docs, a category browser for the right-click menu, a buttonbar for creating objects, and more.

You can find many GUI plugins in the new section of the downloads page as well as documentation for making your own. (What kind of GUI plugin will you write?)

Write Pd objects in more languages

Traditionally, Pd objects are written in Pd (abstractions), C and some in C++. This new release includes two “loaders”, Lua and Tcl, which allow you to write regular Pd objects in either Lua or Tcl. Pd is not the best for processing strings, so if you need to do that, you can now easily use Lua or Tcl, both very easy scripting languages for working with strings. Lua is often used for OpenGL work, so you can also run Lua objects to work in conjunction with Gem. Also, the Tcl loader lets you write GUI objects in pure Tcl, no C needed.

Multi-processing, Pd-style!

The [pd~] object now works out of box. In case you missed the introduction of the [pd~] object in the last release, we’ll introduce you now. [pd~] is Pd itself incapsulated into an object. You can run any patch inside that instance of Pd, the difference is that the Pd in the [pd~] object runs in a totally separate process. So if your computer has multiple CPU cores, which basically all computers do these days, then the Pd process inside the [pd~] object will run on a separate core. This means you can have a heavy Pd patch spread across multiple cores or CPUs. Or for people who work with video and audio together, you can have one instance for video running at a normal priority, then another instance for audio running at a high priority to make sure there aren’t clicks in the audio caused by heavy video processing.

Autotips, generated from help patches

This release also provides a new “autotips” feature to provide instant information about objects and their inlets and outlets. It is one of the first new developments to showcase all of the meta data that is now included in all of the help patches. (Check out the [pd META] subpatches.) When you hover above an inlet or the object itself in Edit Mode, you’ll see a short text description pop up on the lower left corner. But, of course, using a GUI plugin, you could customize how they are displayed to make it how you want to see it. If you want to add autotips to your object, then just add a [pd META] subpatch to your objects’ help patches, and fill out the description, etc. Voila! They’ll have instant information.

What’s next?

The core pd process still handles a lot of the GUI stuff, but we are working on splitting that out for the 0.44 release. That is a big chunk of work, but it will also bring big gains. In particular, it means that it will be possible for people to write their own GUIs for Pd, covering not just the display of the patch, but also the editing, and everything else. You like OpenFrameworks, Python, iOS, JUCE, Qt, etc.? Write your own pd-gui using the toolkit of your choice. That’s the idea at least. That will take a solid chunk of work, so we are looking for people to join that effort.

Try it yourself:

http://puredata.info/downloads/pd-extended/releases/0.43.1
http://puredata.info/downloads/pd-extended

Where to learn Pd:

Resources to start learning

-Hans-Christoph Steiner for CDM

How do you visualize the invisible? How do expose a process with multiple parameters in a way that’s straightforward and musically intuitive? Can messing about with granular sound feel like touching that sound – something untouchable?

Music’s ephemeral, unseeable quality, and the ways we approach sound in computer music in similarly abstract ways, are part of the pleasure of making noise. But working out how to then design around that can be equally satisfying. That’s why it’s wonderful to see work like the upcoming Borderlands for iPad and desktop. It solves a problem familiar to computer users – designing an interface for a granular playback instrument – but does so in a way that’s uncommonly clear. And with free code and research sharing, it could help inspire other projects, too.

Its creator also reminds, us, though, that the impetus for all of this can be the quest for beautiful sound.

Creator Chris Carlson is publishing source code and a presentation for the NIME [New Interfaces for Musical Expression] conference. But this isn’t just an academic problem or a fun design exercise: he also uses this tool in performance, so the design is informed by those needs. (I’m especially attuned to this particular problem, as I was recently mucking about with a Pd patch of mine that did similar things, working out how to perform with it and what the interface should look like. I know I’m not alone, either.)

The basic function of the app: load up a selection of audio clips, and the software distributes them graphically in the interface. Next:

A “grain cloud” may be added to the screen under the current mouse position with the press of a key. This cloud has an internal timing system that triggers individual grain voices in sequence. The user has control over the number of grain voices in a cloud, the overlap of these grains, the duration, the pitch, the window/envelope, and the extent of random motion in the XY plane. By selecting a cloud and moving it over a rectangle, the sound contained in the rectangle will be sampled at the relative position of each grain voice as it is triggered. By moving the cloud in along the dimension of the rectangle that is orthogonal to the time dimension, the amplitude of the resulting grain bursts changes.

You can see how Chris is imagining this conceptually in a sketch he shares on his site:

An extended demo shows in greater detail how this all works:

Chris is a second-year Master’s student at Stanford University’s Center for Computer Research in Music and Acoustics [CCRMA] in California. The iPad version is coming soon, but you can get started with the Linux and Mac versions right away, and even join a SoundCloud group to share what you’re making. You’ll find all the details, and links to source code, on the CCRMA site. (And if someone feels like building this on Windows, you can save Chris the trouble.)

https://ccrma.stanford.edu/~carlsonc/256a/Borderlands/index.html

I also love this Max Mathews quote Chris shares as inspiration:

Max Mathews, in a lecture delivered at Stanford in the fall of 2010
“Any sound that the human ear can hear can be made by a sequence of digits. And that’s a true theorem. Most of the sounds that you make, shall we say randomly are either uninteresting, or horrible, or downright dangerous to your hearing. There’s an awful lot to be learned on how to make sounds that are beautiful.”

Beyond the technology, beyond this design I admire, anything that sends you on the path to making beautiful sound seems to be a worthy exercise. It’s a challenge you can face every day and never grow tired.

http://modulationindex.com/ [Chris' site, with more information]

Thanks to Ingmar Koch (Dr. Walker) for the tip!