Swiping through clouds of timbre and melody, using Gestrument is an experience of a different musical animal. Jesper Nordin and Kymatica / Jonatan Liljedahl created the iPad version, available now for ten bucks. But it’s worth looking at that side by side with Gestrument for Kinect, as the same metaphors can translate across input methods. Both employ a kind of meta-composition – instrument as conductor of musical structures, more than an object for exclusively solo expression.

What’s especially nice is hearing these digital, synthesized environments meld with acoustic instruments. The solution may not convince everyone yet, but there’s clearly potential for these digital instruments to join up with conventional chamber ensembles and acoustic instruments. (See the video below, a live concert with Trio Trespassing. Christophe Lebreton from GRAME in Lyon, France contributed to the development.)

And at the very least, Jesper Nordin’s development work and compositional experiments have earned him attention; the City of Stockholm has even recognized the app.

First, the iOS app:

And here’s a performance of the Kinect version, combined with an ensemble. (I’m curious, readers: sold one one more than the other? Neither?)

All reviews can be read on our Facebook page –

Feature set of the iOS app include various options for control, sharing and collaboration, and support for file import/export, AudioCopy, MIDI, and now AudioBus:

• Play and compose music with the swipe of your finger
• Generate music within defined scales and rhythms
• Improvise freely within a fully controlled musical framework
• Use the internal GM sound bank or your favorite MIDI synthesizer, or load your own sf2 soundfonts.
• Use Gestrument to control other iOS apps or external MIDI programs through a MIDI interface ([Apple] Core MIDI enabled) or Network MIDI
• Virtual MIDI with MIDI clock in and out lets you sync with sequencers and DAWs
• Create and save your own presets and share with your friends or download new presets from
• Record what you play to audio and MIDI file
• Export recordings to AudioShare – audio document manager
• Copy recordings with AudioCopy
• Play back previous recordings, optionally looping
• Use Gestrument as a source synth in Audiobus
• Allow mixing with other apps, for example play a track in the
Music app and play along with it in Gestrument.

The idea is, you surf around scales and rhythms that are pre-mapped to the layout. Korg’s Kaossilator popularized some of these sorts of ideas in hardware; here, you have various other ways of setting up those kinds of layouts. The idea isn’t new: the traditional Autoharp has long allowed folk musicians easy access to harmonies simply by strumming. (Frets are a more complex solution to the same idea.)

You are a bit restricted in sound set, but with MIDI out, there’s room for growth.

More on the app:

  • Nikola

    Good news. It sounds like it was programmed by someone who actually knows something about harmony and composition, and unlike other half-baked attempts this implementation could be very useful.

    • Jesper Nordin

      Thanks! Yes, I am a classical composer (studied in Sweden at the Royal Academy, in France at IRCAM and in the US at Stanford) and Jonatan Liljedahl is an electronica artist with a classical training. The app is still a scaled down version of my original tool that only exists as a Max/MSP patch right now, but we are working hard on implementing more features as we go! I hope you like it. /Jesper Nordin

    • Øivind Idsø

      Ahem … is that MaxMSP patch available by any chance? :)

    • Jesper Nordin

      Sorry, it’s a bit of a mess right now… I will try to fix it up in the future, but for now I don’t have the time – fully absorbed in three large orchestral pieces for the upcoming year, as well as trying to continue to develop the app… But I will try to fix it eventually, and when I do I’ll post it on the gestrument website.
      Thanks for the interest!

    • Øivind Idsø

      No problem! I’ll be buying the iPad app anyways, just thought I’d ask!

  • Dave O Mahony

    Absolutely fascinating composition potential! The sound examples sound very good on their own but with MIDI out, I think this will be a must buy app! Many congratulations Jesper & huge thanks to CDM for the heads up!

    • Jesper Nordin

      Thank you! I’ve been using my earlier version of this tool for many years now to compose both electronic and acoustic music, including several orchestral pieces. We will continue to add to the app to get more depth in the theoretical control, but it’s already quite powerful as it is today.

  • Tim Thompson

    It’s very nice to see this effective use of the Kinect’s raw depth information, I think it has much higher potential for expressive interfaces than the skeleton tracking that most people use.

  • David Bagshaw

    I would very much like to try the Kinect version, either via Max or an executable. I guess it will be a while before either are available…Please keep us posted, Peter.

    • Jesper Nordin

      Yes, as of now we are only using it for our own artistic and pedagogical projects. But hopefully we will be able to release it one way or another in the future. Thanks for you interest! /Jesper Nordin of Gestrument