How do you visualize the invisible? How do expose a process with multiple parameters in a way that’s straightforward and musically intuitive? Can messing about with granular sound feel like touching that sound – something untouchable?

Music’s ephemeral, unseeable quality, and the ways we approach sound in computer music in similarly abstract ways, are part of the pleasure of making noise. But working out how to then design around that can be equally satisfying. That’s why it’s wonderful to see work like the upcoming Borderlands for iPad and desktop. It solves a problem familiar to computer users – designing an interface for a granular playback instrument – but does so in a way that’s uncommonly clear. And with free code and research sharing, it could help inspire other projects, too.

Its creator also reminds, us, though, that the impetus for all of this can be the quest for beautiful sound.

Creator Chris Carlson is publishing source code and a presentation for the NIME [New Interfaces for Musical Expression] conference. But this isn’t just an academic problem or a fun design exercise: he also uses this tool in performance, so the design is informed by those needs. (I’m especially attuned to this particular problem, as I was recently mucking about with a Pd patch of mine that did similar things, working out how to perform with it and what the interface should look like. I know I’m not alone, either.)

The basic function of the app: load up a selection of audio clips, and the software distributes them graphically in the interface. Next:

A “grain cloud” may be added to the screen under the current mouse position with the press of a key. This cloud has an internal timing system that triggers individual grain voices in sequence. The user has control over the number of grain voices in a cloud, the overlap of these grains, the duration, the pitch, the window/envelope, and the extent of random motion in the XY plane. By selecting a cloud and moving it over a rectangle, the sound contained in the rectangle will be sampled at the relative position of each grain voice as it is triggered. By moving the cloud in along the dimension of the rectangle that is orthogonal to the time dimension, the amplitude of the resulting grain bursts changes.

You can see how Chris is imagining this conceptually in a sketch he shares on his site:

An extended demo shows in greater detail how this all works:

Chris is a second-year Master’s student at Stanford University’s Center for Computer Research in Music and Acoustics [CCRMA] in California. The iPad version is coming soon, but you can get started with the Linux and Mac versions right away, and even join a SoundCloud group to share what you’re making. You’ll find all the details, and links to source code, on the CCRMA site. (And if someone feels like building this on Windows, you can save Chris the trouble.)

I also love this Max Mathews quote Chris shares as inspiration:

Max Mathews, in a lecture delivered at Stanford in the fall of 2010
“Any sound that the human ear can hear can be made by a sequence of digits. And that’s a true theorem. Most of the sounds that you make, shall we say randomly are either uninteresting, or horrible, or downright dangerous to your hearing. There’s an awful lot to be learned on how to make sounds that are beautiful.”

Beyond the technology, beyond this design I admire, anything that sends you on the path to making beautiful sound seems to be a worthy exercise. It’s a challenge you can face every day and never grow tired. [Chris’ site, with more information]

Thanks to Ingmar Koch (Dr. Walker) for the tip!

  • Oootini

    i think i woiuld buy an ipad just for this.

    • Alwyn

      cant wait for this, a AU component would be nice i really think thats the way with iPad audio apps they need a complimentary AU for daw integration so sketch on the iPad get back to your daw to work on it further

    • adam

      i also thought we’d need an iPad daw integration standard, but then i started to use my iPad as external standalone gear recording the (wired) audio output into my daw. just like a real instrument. if you don’t like the recording try again. no further treatment. this made things a lot easier and more intuitive for me…

    • starving student

      this is looking good, I have a question, it’s a simple one but I’m having some difficulty finding good info i hope it’s ok to ask you here peter and anyone else knowledgeable about this kind of stuff. I’m looking for an ipad app or apps that really focus on mangling samples. I’m a heavy re-samplist and I would like to find apps that let you simply load up a sample onto the screen, do all manner of organic mangling (in other words I’m not interested in focusing on distortion, or bitcrushing) but organic mangling to the loop, making it possible to completely transform the sample into anything and everything, then take this transformation and save/export it. Ideally this app would also have sample slicing so that I could make loops out of whatever is being worked on as well, instead of having to make loops in another application.

      sample whiz is a great app but doesn’t have a slicing feature. the experience I’m trying to get out of this is to be able to sit on my couch on a sunday afternoon and edit, mangle, and create a sample library all on my ipad. of which to use in other apps or with my hardware samplers.

      and thanks in advance for any insight anyone can give.

    • Guaranath

       reply for starving student:

      Grain Science by Wooji juice is really excellent and organic sounding grain unit  for iPad. It allows loading of one or two samples to grain, ribbon-style modultion keys, 4 assignable x-y pads, a host of effects units and assignables in a patch-bay… and much more! See their website for details regarding editing. And incidentally,  they also make Hokusai an audio editor that many use as a ‘go-between’ for getting audio copy/paste files to your desktop DAW in a 32-bit WAV format (via iTunes filesharing).

  • digid

    Looks VERY nice! The way the various circles interact seems reminiscent of how the excellent, but not-being-developed-anymore, Jasuto Pro works.

    Btw, seems Mac and PC users need to find their own way to compile Borderlands for OSX or Windows. No binaries available …

    • Peter Kirn

      Right, it’s just source, not binaries. 

  • Chris Stack

    Can’t wait until this is available.  Any way to get a notification?

    • Peter Kirn

      Stay tuned here. 😉

  • Ronnie

    This is absolutely lovely. The interaction is so nice,  I can see myself spending a lot of time with an app like this.

  • Marc Lorenz

    This looks amazing. Personally I don’t care about the academics stuff, source code etc.
    But the iPad app…please bring it on!

  • Magnus

    Love the interface. Makes me want to study at CCRMA.

  • Eric Sheffield

    This looks fantastic and is one of the few iPad instruments I’ve seen that truly looks “performable”.
    Not just a technical and/or design achievement, either. Aesthetically (aural and visual) pleasing and versatile.
    Makes me feel less and less like I’m smart or talented enough to apply to CCRMA…

  • YETI

    Can’t wait for this to become a windows program or a VST.

  • Chad

    Impressed with how fluid this seems.  Pushes the “math” component away and draws the “expression” component nearer.

    – c

  • Chris Carlson

    peter, thanks so much for the beautiful post!  commenters, thanks for your enthusiasm! will do my best to keep everyone informed of the eventual release date.  until then, i’ll be coding…

    also – anybody interested in studying at CCRMA should DEFINITELY apply. it is a wonderful place filled with people with completely varied backgrounds. if you’re in the area, come by for a visit!

    • Sinapsya

      Hi….I’m weating for iPad version…but I cant wait and want install osX version..can you guide me about Terminal use….I’m not genius like you : )

  • CoreyP.

    A little hesitant about using Terminal – – yet another initiation beyond simple geekhood – – any good tutorials out there (like how to back out of it, undo what’s done, restart in the previous state?

    Otherwise, looks really awesome!

  • jhhl

    Very pretty – I like hands on Granular. That’s why I wrote one myself. Coming Soon!

  • Olav Slonovski

    I know you shouldn’t run executables from strangers.
    But if anyone’s interested, compiled binary for OS X is here:
    Execute ./ from the terminal.

    • uh

      Thanks! But the binary still needs the three libraries to be installed, right? I didn’t manage to install those… :(

    • Michiel de Lange

      I made a package that includes the libraries (osx 10.6 64 bit): See the README for instructions.

    • uh

      got it running! thank you very much! :-)
      great software, i’m impressed.

    • Olav Slonovski

      libstk and librtaudio are compile time dependencies, you don’t need them to run
      the binary itself. You only need libsndfile. One way is to put libsndfile.1.dylib where it expects to find it, like Michiel did. Other way is to use a launcher script with DYLD_LIBRARY_PATH environment variable to preload it from extracted archive like I did.

  • Polite Society

    I want to touch… !!

    This is the best representation of granular synthesis I’ve ever seen. Brilliantly done!

  • zeroref

    awesome. beautiful interface. makes me feel that touch has really ‘arrived’ now, though i don’t know why. seems like there have been other good touch apps previous to this.

  • Alf

    While this looks amazing I’m a complete novice and was unable to get it working after a good 20 minutes of trying. I think a .dmg is more my skill level. 

  • starving student

    thank you Guaranath  I will definitely check that out and if anyone knows about anything else like this or even something that isn’t focused on grains I’d love to hear about it.

  • gesslr

    Truly visionary. I love applications that re-think user interaction, turns it around, and offers something different that both expands what I can do and makes it easier to get to these new places so I can do it. Truly beautiful design. Another “breakthrough” design I’ve seen recently is for AudioGL a new take on an integrated sequencer environment. Chris, you might find this interesting and, like you, the developer/author (Jonathon Heppner) is doing this breakthrough work solo. Check it out.

  • Chad Eby

    So this motivated me to finally download and install Xcode; I have never compiled anything in OS X, and it took me about an hour of flailing and Googling and system updating and reading READMEs (and quietly swearing) to get a working binary together…the last obstacle was overcome by making sure that I was “super user” (sudo) when installing libsndfile. It was definitely worth the effort! Thanks, Chris for a beautiful and thought-provoking tool. 

    • PaulDavisTheFirst

      welcome to the hell of building more or less any cross platform software on OS X. its a super nice platform for building OS X-only tools, but the worst *nix by far for building cross-platform stuff. sad. 

    • Peter Kirn

      Yes, true … though that’s before we get into the fresh hell that is doing this stuff on Windows.

  • Andrew Stone

    Compiled all the libraries and Borderline a few hours ago. I am stunned by how simple and elegant the application is. Gives you a face first feel for granular synthesis.

    To those struggling with the code installs… All the apps simply need a ./compile and then a “make”. If you have done linux or unix in your past it’s pretty straight forward. There are a few caveats in the install process but if you read the readme/install files carefully most should be good to go and make sure you read up about the switches for the ./compile in the dependent libraries. You will probably only have to use one switch depending on your OS.

    Chris you might want to even write up your own install instructions for the various dependent stuff and Borderline so it is clearer.

    And thanks for your efforts Chris. Wildly exciting work you’ve done.

  • Ramon Solorio

    I think Chris is about to make a lot of money 😉 He’s got some of mine as soon as this app is released. Inspiring.

  • j-

    Looks very cool indeed!

  • umma08

    took me a while to figure out the necessity of shift + g. ffs

    and thankyou for simplifying the dependency process Michel de Lange

  • edu

    Anyone hat it working on OS 10.7.3??? I try and the screen go black and it crash….

  • Frederik Tollund Juutilainen

    I’m way too inexperienced in compiling to make this work on OSX, even though I’d love to try it out. Can someone, with a bit of time on their hands, please post a short guide using xcode?

    Best regards

  • casey

    As a GPL 3.0 license, how is this app able to run on the iPad? I was under the impression that no GPL software was legally allowed? Is it only for jailbroken devices?

  • Cly Suva

    I am quite sure Borderlands is a registered trademark.