The tablet – or at least the iPad – is beginning to look like a terrific accessory for lovers of MIDI and hardware. With its compact form factor, it coexists nicely with your MIDI gear and lets you focus on sequencing, perhaps moving to the traditional computer to finish up your track, mixing, and the like.

And it’s spawning MIDI sequencer apps that imaginatively explore ideas for how to create sequencing, all with an immediate touchable interface.

The latest entry: Genome MIDI Sequencer claims to be the “first true pattern-based MIDI sequencer for iPad.” The word “true” might be debateable, but it looks thoroughly full-featured, with clock sync, MIDI import / export (ideal for working with your desktop Mac or PC), and other must-have portable sequencing features:

• Hard-line MIDI support: Core MIDI, Camera Connection Kit, Line 6 Mobilizer Mk I and Mk II compatibility
• Wireless MIDI: Network MIDI support (works with Mac and, via other support, Linux and Windows)
• MIDI Clock: Start, stop / send, receive
• Lots o’ messages: Note On, Note Off, Pitch Bend, Aftertouch, Channel Pressure and CC’s
• Pattern-based; pattern changes occur on bar boundaries so song stays in sync Ed., oh, come on, where’s the fun in that?
• 16 simultaneous tracks, unlimited patterns

There’s also an on-screen keyboard interface (in case you don’t have a MIDI keyboard handy), song and live modes, incoming CC and note recording, and undo/redo for “most” actions.

Having the multichannel capability alone is a must for people with big studios.

Developer Dave Wallin says he’s been working on this for some time. An experienced dev with tools like bleep!BOX and bleep!Synth under his belt for iOS (as well as Additive and Zero Vector for desktop), he’s got a good perspective on the bigger picture.

Since Dave is a user as well as a developer (as is often the case with music tech), I asked him to tell us a bit more about the app, his thoughts on development, and how you might use this.

He even has some good thoughts, in case you’re wondering, about how this compares with and fits in with desktop studios.

CDM: This looks to me to be the most full-featured MIDI sequencer we’ve seen yet, more than just a pattern or step sequencer (though I like those sorts of things, too). That said, what other applications do you like in terms of MIDI control?

Dave: There are a couple other apps that look close in terms of functionality like Step Poly Arp, Polychord and Brainwave Sequencer. The first two are more focused on ‘performance’ imo with Brainwave being the closer. I wanted to provide some of the functionality you’d expect from a desktop sequencing app but redesign the core interface around touch and just make it simple and fun to use. My inspirations for how the piano roll works are 50% monome / [Yamaha] Tenori-on (in terms of a familiar grid and easy one-tap note entry) and 50% iPad paint application (in terms of using gestures to navigate). It also takes some inspiration from [Nintendo] Game Boy tracking apps in that I keep the octave / bar navigation locked to a grid. GMS isn’t locked into 16 step patterns and you can adjust the grid and bar length more freely than some other apps.

Ed.: Of course, at iPad app prices, you can easily pick up all the apps above, and they all have some appeal – yes, I know it’s time for another iPad app round-up. A MIDI-specific one could be fun; I’ll work on it! -PK

How are you using this in your studio – what gear are you using?

Right now I am using it with my [Access] Virus b (desktop [synth]) and a drum machine primarily. It’s great with the Virus since it can do up to 16 parts and I can save these setups and easily recall them later without having to worry about program changes or anything like that. The drum machine saves me from having to waste Virus parts on drums. With just those two pieces of gear and not much else I can make some pretty decent sounding songs. My current work flow is to jam around a bit with Genome and the gear and when I come up with something good, record some loops or tracks it and bring it over to the desktop for additional processing and arrangement. My actual studio doesn’t even have a computer in it at the moment – it’s in a totally seperate room.

At what point do you imagine people would use the iPad sequencer versus, say, the sequencer on their desktop machine?

What I have learned from my own experience and from talking to other people is that the iPad is not a total replacement for the desktop or even for a laptop, unless you’re doing some very minimal stuff. The iPad is more of a convenience and leisure device at this point. I have limited time to relax and if I can spend 30 minutes chilling on the couch, making a tune on my iPad, it’s a win for me. It doesn’t have to be a whole song – it could be some musical ideas, sounds or loops that I end up using later on the desktop or in another app.

With Genome, I think the big wins are portability and ease of use. If I want to jam with a friend, I can throw my iPad in a bag and bring it with me. Sometimes I don’t want to have to deal with all the overhead that comes with a desktop sequencing application like hardware setup, configuring plugin directories, or GUI’s that are packed to the brim with controls. GMS is very easy to plug in and get started within a few seconds.

I’m curious about your reaction to the Open Music App Collaboration Manifesto posted recently. [See Synthtopia coverage.] Any of these issues relevant to your work? What would you like to see in terms of inter-app integration?

I actually just posted some thoughts on this to their Google Group this morning. My response is here:

http://groups.google.com/group/open-music-app-collaboration/browse_thread/thread/75cdd385048fa3ce

Communicating with other apps on the same device is something that I think Genome will be very good for (after some minor updates). Right now, not a lot of apps support Network MIDI, let alone regular CoreMIDI. How well this works will depend on a lot of factors and we’ll just have to wait and see how extensively it’s adopted, both by developers and by users. I could imagine an ecosystem where you can string a bunch of apps together and make songs. The first step right now is to come up with a spec for developers to implement and maybe a way to certify that apps have implemented it properly. If done well, I think there could be some good cross-promotion opportunities to entice developers to do it and it could catch on, much like with Audio Copy did.

Had you evaluated Android at all, or other platforms?

Some work is being done to port libNUI [C++ framework] (which I use for all my apps currently) to Android, however it seems the state of MIDI on Android is nowhere near what it is on iOS. If some well known hardware makers make an accessory or if a good SDK becomes available, I would definitely look into it. GMS is being ported to a Mac desktop app and will be available in the Mac App Store at some point. The desktop version will be almost identical, except for some changes made to adapt it to work with a mouse, instead of multitouch. I think a simple, cheap MIDI sequencer will be attractive to some people and the amount of time to port it is minimal for me.

Ed.: Hopefully more to say about the state of Android MIDI soon; while wired accessories may not be as convenient, wireless MIDI, especially via Bluetooth, holds some promise. I can certainly make a good argument *against* a developer immediately rushing to support Android, but it remains something we’re watching. I also hear these computer things can do MIDI. -PK

US$12.99 in the iTunes App Store, for iPad (iOS 4.2+)

http://www.whitenoiseaudio.com/apps/genome-midi-sequencer/

http://www.whitenoiseaudio.com/

On tablets, on displays, multi-touch control these days is calibrated largely as a software interface – more Starship Enterprise panel than violin. As such, it works well for production tools and exploring compositional ideas. But it falls far short of being an instrument: even on the much-hyped iPad, touch timing and sensitivity is too imprecise, and the absence of tactile feedback and real, kinetic resistance makes you feel like an operator rather than a musician.

Several projects in experimental instrument research seek to change that. But of all of them, the one that has generated the most enthusiasm is Randy Jones’ Soundplane, co-developed with hardware designer Brian Willoughby. CDM shares a conversation today with Randy about his brilliant Aalto synth, and I’m working on a review soon. But wonderful as Aalto is, many of us are still eager to hear more of the Soundplane controller. I chose to wax poetic and optimistic back in December of 2008:
Intimate Control: Multi-Touch, New Models, and What 2009 is Really About

I shouldn’t have put a year on my predictions, though – good things take time. (If I could clearly recall what happened in 2009, maybe my general prediction was correct. The past tends to blur together for me into a continuum in the manner of the modern technologist, a vague assemblage of stuff that happened in the 60s with things that are actually still in the future.)

The good news: Randy continues working on the Soundplane, and Aalto will help.

Continuing our interview, here are the thoughts most relevant to Soundplane — and a glimpse of what it’s looking like as he works on it in the lab.

First, Randy explains his ideas about running a small business, continuing what he had to say in our Aalto story. The basic idea: Aalto’s software will bootstrap Soundplane’s hardware.

I think the whole idea of venture capital is sort of a poisonous one. It’s a little like bands wanting to get signed right away. The first thing you want to focus on is giving up your autonomy, really?

Instead, why not scrape together whatever you can from friends or family and just make something that you can sell right away, however small. I didn’t have enough saved to finish the Soundplane project so halfway through I switched to putting out Aalto as a plan B for paying the rent. Now it’s out and it’s a product I’m proud of that I think reflects where we’re coming from, and it’s going to fund Soundplane development, and it’s letting tons of people know we exist. Just get a foot in the door, do something useful.

He also shares his feelings about patents:

Some people won’t like to hear this, but I applied for a patent on the sensor used in the Soundplane. I know, the patent system is totally broken, and often, if not usually, used in stupid ways. But if there’s one thing I think it is actually good for, it’s to protect small companies like ours that innovate against a bigger entity simply stealing their R&D. This is why it was designed, right? I don’t know if our patent will save the day if such a thing ever happens, but if it does I’d much rather have one than not. It’s a pain to write one but it’s not impossible, you just need a lot of patience. “Patent it Yourself“, Nolo Press, is a good reference.

The patent question raises some additional questions for me – in fact, I’d love to see open source hardware that’s also backed by patent protection, in the same way that the GPL license is made tenable largely through the existence of traditional copyright laws.

But I do tend to agree that in the case of a truly novel technology, which this is, patent protection may be necessary. The question for projects like this will be whether to operate as a conventional, patent-protected design, or whether some sort of open source model with a patent covenant and a copyleft license like GPL will make sense — both preventing exploitation and allowing free experimentation. If there are any IP lawyers lurking around out there, let us know (I have some contacts, too); and definitely let us know if that’s a conversation you’d like us to continue.

In the meantime, the important thing is that Soundplane lives, and using Aalto could help it come to fruition. We’ll absolutely keep you posted.

As proof, though, more shots from the lab:

Also, must-read article from shortly after Jones’ NIME presentation:
Why Soundplane?

The whole article is worth reading, but Jones argues that not only is it likely many people will try to do tactile multi-touch, but it may be necessary. For those of you not all that good at hardware design, you could be just as essential as well to there being any future for these curiosities. The designers need other designers. The hardware needs software creators – lots of them. The software creators need to try lots of ideas. And everybody needs players, composers … users.

It’s all-too-tempting to sit back on the Web and marvel at what everyone else is doing, to take their genius and novelty as an engraved invitation to give up on your own work. “It’s been done before.” “Someone else is already doing this.” It’s probably a topic for a dedicated article, but it’s simply the wrong reaction. “It’s been done before — maybe it’s worth doing. Or doing again. Or doing better. Or doing over and over again.” “Other people are doing this — that means I have someone else to do it with.”

Historically, revolutions aren’t solo pieces. They’re ensembles.

Updated: speaking of work being ensembles, while Randy’s name is most associated with the Soundplane project, credit is due to hardware designer Brian Willoughby, who did the hardware design for the instrument. As he wrote in comments on CDM in 2010, when we covered Roger Linn’s Linnstrument: “For my part, I’ve been deep into the process of designing the analog circuits, DSP hardware and firmware necessary for the product, so it’s nice to poke my head up for a moment and see interest on this site, as well as to hear about other engineers trying new things and inspiring ideas.”

Software can easily enough emulate, down to each knob and patch cord, a vintage synthesizer. But can a genuinely new software synth incorporate the ideas about instrument design beloved in a classic synth like the Buchla modular? How do you balance open-ended sound design with the sorts of limitations that give an instrument personality, limitations that inspire?

And could all of this be meaningful even for someone first discovering synthesis, who may never have seen or heard of a Buchla? (If that’s you, please don’t bail on us just yet!)

It’d be pretty pointless to celebrate the legacy of designers like Don Buchla or Bob Moog if you didn’t think that legacy would be carried on. Randy Jones, of Madrona Labs, is just the kind of person to watch. He’s the creator of the innovative multi-touch, tactile Soundplane hardware, as well as the semi-modular Aalto soft synth, updated this month. Madrona has just released Aalto 1.2. The banner feature: Windows support, meaning you now have full 64-bit (or 32-bit) support on Mac and Windows alike. There’s now really no reason not to try out the instrument, with free demos on each.

What makes Aalto special? It has a unique interface that focuses on sound, and semi-modular design that allows you to produce the sort of sounds a Buchla modular would – without trying slavishly to emulate that hardware in software. It’s something new. As Randy puts it:

The main UI element is a combination oscilloscope + dial that is totally new. It’s inspired by the Max/MSP multislider, as well as the idea “computers are so fast now, and promise so many new possibilities for visualizing sonic systems, why are we still making virtual knobs?”

Randy is reflective as well as inventive, so here are his thoughts on designing the instrument – as well as the most unique approach to copy protection (and cookie-cutter soft synths) I’ve ever heard. (And I do mean heard.)

Aalto, a Tool for Sound Designers; No Two Are Alike

I had two basic ideas: one was to build a softsynth that really encouraged people to make their own sounds, made it fun and easy. Also I wanted to make software that could duplicate some of the Buchla sounds programmed by Morton Subotnik– those wonderful Vactrol / LPG plunks, because I’d never heard that done in software before.

So I picked the Music Easel as an inspiration, not just in sound but in UI, making a small set of modules that would lead to a surprising variety of results.

Limitations are really key, and while Max/MSP and other general environments that try to do everything are very useful, I think musically there’s a lot to be said for making fixed instruments that you learn and don’t mess with. That way you don’t have this situation where everyone ends up with their own instrument. This idea goes for hardware as well as software…

I think good instruments are style-agnostic… hardware as well as software. Making something for a particular genre is just reacting to fashion, which is the enemy of expression.

It’s depressing how popular sample sets are. There are so many possibilities for making your own sounds—recording sounds well, processing and synthesizing them are all easier and cheaper than ever before. If you do electronic music, timbre is a crucial element– how can you buy a sound from someone else? That’s like a poet buying words from someone else.

Aalto is designed to be unhyped. Sit well in a mix. Good music makers have their own compressors, they don’t want a synth to decide to compress itself. Good music makers have their own effects, they don’t want a ton of reverb and delay slathered over every preset.

A precise tool that’s good for learning is also good for using. Knobs are marked with actual time and pitch units. It’s harder to learn synthesis if your knobs all just go from 0-1.

Nothing that affects the sound is hidden. Everything is one click away. Aalto has no menus. I think this makes it a softsynth actually good enough to perform with, not just playing notes but moving through patch space.

It’s easy to go off the rails. Enough rope to hang yourself. A whole world of bad sounds is available at the touch of a dial. Because without them, the surprising good ones wouldn’t be there, only the boring good ones.

The signal scaling to oscillator pitch is a handy knob– one turn of it totally messes up the musical intervals if you want. It’s also easy to get back to a default.

As a composer and performer I’ve been burned time and time again by copy protection, so Aalto doesn’t have any. You can make as many copies as you want, but only run one of them at a time. The registration is embedded in the plugin itself, so there’s no keyfile to worry about. Want to use Aalto at a friend’s studio? Copy it to your zip drive and use it.

Because of the embedded license info being available, I went and did something a little weird: each copy of Aalto makes a slightly different sound. No more than different units of a Prophet-5, but the filter cutoff and oscillator detune and some other parameters are slightly affected by your unique user data. So the sound of the synth is in a way made very personal to the license holder.

Though it’s not so easy to make a good one, it’s a lot easier to make a new softsynth than a new hardware synth. So why are softsynths mostly so boring? Why so little experimentation? I think more interesting things are happening on tablets, with weird audio-visual toys people wouldn’t have made without the interface being available. But the sound quality is not there yet with the tablets. We still have these desktop and laptop computers of amazing power that everyone is using for production… but where is the spirit of innovation that we see in the tablet space?

Being a Small Company

Little companies can do stuff unimaginable a decade ago, thanks to the internet for research and promotion, and the open source community for access to tools. There’s this big area of possibilities with companies like ours that have more of a craftsman than a startup vibe, are not in it to get bought out but to make things of value and beauty over the long haul. Certainly the monome folks are a huge inspiration to me here.

You don’t need a big team to make and promote cutting edge tech anymore, you just need a few really good people. What that means is that companies like ours can innovate and make a product for an audience of a few thousand, something that a Roland or Korg would never bother with, and have a sustainable business model doing that.

Hopefully, this gets the conversation started. Randy’s very open with his ideas, so we’ll be in touch; if you have other questions for him, let us know. And, of course, we should look at Aalto very soon.

http://madronalabs.com/aalto

What can a new digital synth be in 2011? How will it work and sound? And given access to so many excellent tools, how can it stand apart? In place of a press release and some marketing-speak, developer Christopher Penrose (Leisuresonic, Cosmovox) sent us an extended essay explaining his thinking behind his just-released SynthTronica synth for the iPad. Aside from getting into the nitty-gritty technical details, it cuts to the crux of the issue: how to make something personal and new that nonetheless can work for other people, and how that idea can be tailored to a tablet.

As the geeks are glued to the latest iPad announcement, let’s look for a moment beyond platforms. Great ideas in synthesis endure long past platforms. The specific medium is wonderful in that it gives designers, engineers, and musicians the opportunity to realize those ideas, while presenting certain conveniences for developer and user alike.

All of this is worth reading in this case as it sounds like Christopher has a synth that isn’t like everything else out there – not at all. With audio files of your choosing transforming the timbres of synthesized sound in a graphical, spectrally-displayed filter, it looks like it could be a brilliant canvas for producing unusual sounds. That might help it find a place wired into your desktop PC or Mac for production.

In fact, it reminds me of the kind of creative synth we’ve seen all too rarely. The design feels heavily reminiscent of the ground-breaking U+I MetaSynth conceived by Eric Wenger (of Bryce fame). I was always disappointed other software didn’t run with some of those ideas; seeing it with some new twists, the take of a different artist, and touch input on the iPad looks terrific.

Christopher’s notes are quite long, but worth including in their entirety, especially knowing we have other developers in the crowd.

And, oh yeah, we could ramble on about this all day, but I think most of you will get the idea from this video below. It sounds great, and since you can input different audio files to get different filter content, you may be able to escape both overt recognizability and the disposability of many mobile and tablet instruments.

Here’s Christopher:

Notes on SynthTronica’s Development

Back in January of last year, I had been following the Apple tablet rumors with great interest. When the iPad was announced, I was surprised by both its name and its operating system. But it took me only a few hours to decide that I was going to design a synth for the new tablet.

Context

I have developed music software, with varying levels of attention, since 1988, and much of that effort has been spent developing idiosyncratic DSP algorithms for sound exploration. In particular, I focussed on spectral techniques for mating sounds — taking the characteristics of two (and sometimes more) sounds to create a new one. These efforts haven’t made it out of the Max/MSP, Pd, and Unix shell software ecosystems largely due to the limitations of audio plug-in hosts. “Side-chain” processing implementations are obscure and clumsy.

I am also a composer, and, until recently, my software was largely designed to aid my personal music-making. I can honestly say without pretense that my music is idiosyncratic; even Illegal Art, a label which has released one of my albums, regularly characterizes my music as being on the “challenging” end of the spectrum of their musical offerings. I think that SynthTronica has been a good project for me. While developing it, I have been challenged to corral and focus my motley DSP technologies into a broadly-accessible musical instrument. I took a music making process that combines synthesis and sampling, which I used often in my music making in the last decade, and put it at the core of a keyboard synthesizer. Hopefully, I have been able to distill an elegant instrument design from my personal composition practices.

Synth Architecture

SynthTronica is a hybrid instrument; its synth engine combines characteristics of virtual analog synthesis and digital sampling. For most “traditional” synthesizers, sound evolution is controlled by parametric filters — combinations of VCFs and LFOs. Instead, SynthTronica uses time-varying filter data to provide spectral evolution; an instance of such data is called a “formant”. Formants can be created in several ways, through the iPad’s microphone, importing audio files, and capturing performances of SynthTronica’s multitouch filter. While formants are currently played in strict loops, the maximum formant duration is fairly large — just over 60 seconds — providing potential for significant, albeit prerecorded, variation. The benefit of formants lies with their generality. A formant can be made from sources as disparate and varied as Nord percussion, cellos, choirs, braying donkeys, or the chorus of Katy Perry’s latest single. The latter example is an interesting consideration: a formant can reflect much of the rhythmic and sometimes vocal characteristics of its source sound, while effectively obliterating its pitch. Pitch is instead provided by SynthTronica’s synthesizer front-end. When readily-discernible formant sources are used, SynthTronica provides a unique musical space that lies between the boundaries of pure sound synthesis and referential sampling. Formant synthesis is not explicit like sampling; you play through the Katy Perry groove as if it was your avatar. With SynthTronica, a performer needs to make pitch choices for any sound to be heard.

Multitouch Filter

The iPad’s large touch surface was a serious attraction for this project. The idea for the Dynamic Multitouch Filter immediately came to mind. SynthTronica’s multitouch filter serves as a live and expressive counterbalance to the static character of formants by providing fluid gesture-controlled filtering of the synthesizer’s output. Given the spectral architecture of the synth engine, adding up to eleven touch-triggered filters (eleven per voice, technically, though they currently are used synchronously) does not overwhelm the processor resources of the iPad.

Hold Mode

Perhaps an ambient musician’s dream, hold mode simply allows notes to sustain by a single touch. They can be released singly by an additional touch, or released en masse via the “all notes off” button. Hold mode is an excellent counterpart for the multitouch filter, as the filter can easily be the focus of both hands when notes are sustained automatically.

Multiple Keyboards and Scale Patterns

While there are several ways keyboard control could be further developed in SynthTronica, I took advantage of several possibilities made available by the tablet design of the iPad. It is clear that a touchscreen does not offer tactile feedback, so I sought to implement interface dynamics that would make up for this lack in several ways. I choose to support multiple keyboard designs providing two piano style formats and a unique grid-based design.

From my iPhone app Cosmovox, I had a large database of musical scales available. I repurposed these in SynthTronica by providing selectable scale mappings for the keyboards. The piano keyboard has a particular design pattern which is accentuated by the contrast of white and black keys. Being a mallet percussionist in a former life, it was clear to me that this color contrast is optional and the key arrangement itself provides enough information to discriminate notes on a keyboard. Thus musical scales can be represented on a keyboard by changing this color contrast pattern. I often desire to escape my habits when creating music, and altering the keyboard scale pattern can be revealing for me. Further, the scale pattern facility allows for the use of a more radical keyboard design: the grid keyboard. The grid keyboard alters note relationships in interesting ways. The keyboard is compact, allowing one hand to access a two-octave range. Large intervals are no longer as physically distant from a given pitch. The keyboard can be bewildering to play if you play by note (which is a positive feature for me particularly as the keyboard is optional), and can reveal fascinating characteristics of scale architecture.

Design

I went with a modernist design aesthetic as I believe that SynthTronica does not have any appropriate analogs in gear. I find that creating interfaces for software that imitate gear introduces problematic usability issues. While I can understand the desire to have every useful performance control accessible on a single screen, there are practical limits to the number of interface elements that can coexist and still remain effective. Virtual knobs use less screen real estate, make a reference to audio gear, but are more difficult to use than sliders. I chose a slider-only interface using color and orientation for contrast. SynthTronica’s multi-screen design may reduce the accessibility of parameters during performance, but I think the architecture of the synth favors pre-performance sound design and emphasizes use of the Dynamic Multitouch Filter for expressive control in live performance.

Christopher also includes some frank thoughts on limitations of the synth for the time being, including some of his concerns about third-party audio interface support generally. This is beyond my area of expertise, so I’ll leave others to reach what conclusions they will – and I suspect we’ll hear some other developer views.

Limitations

While the preset architecture is robust from a database perspective, it can be frustrating for performance in its current state. The design of the reverb processor is one of the culprits. If reverberation time is different between two presets, changing from one to the other while the synth is sounding can produce awkward glitches. It is possible to ignore reverberation settings from presets by adjusting a SynthTronica parameter in the iPad’s settings application. I would like to improve preset change behavior in a future update.

Some goodies that the electronic music literati would desire — MIDI, audio interface support — have yet to be developed. MIDI is actually very high on the list now, as Apple has provided SDK support [Core MIDI] and I have purchased two Akai LPK25s and an Emu XMidi 1×1 for testing. OSC support is minimal at the moment: there are no supported in-bound messages yet, but a few outbound messages are implemented. Full class-compliant audio interface support will not be added until SynthTronica migrates to an iOS 4.x-only architecture, and even then there may be a performance reduction for many interfaces. A rant could be placed here which would be directed at audio interface manufacturers.

What I will say is that SynthTronica is less flexible with respect to audio buffer sizes as it is a spectral synthesizer; it uses power-of-two FFTs. But this is not unheard of in the least for audio processing; there probably isn’t an MP3 player that does not use them. To support audio buffer sizes that are not powers of two would cause a significant reduction in performance for SynthTronica (namely, a 50% reduction in usable polyphony due to CPU spikes). While there may be an audio interface that works out of the box with SynthTronica, I can’t name one at the moment. The class-compliant audio interfaces I have tested refuse to provide a power-of-two buffer when requested. While I am sure their engineers can come up with an excuse, they really should understand that power of two constraints are ubiquitous in computing, particularly for digital audio signal processing; it is bizarre that their hardware forces applications to perform additional buffering to support powers of two.

Anyway, let’s wrap up not with words, but with…

Sound Samples

PlanetMaster by Leisuresonic

HypnoticCaressing by Leisuresonic

PariahEmoting by Leisuresonic

PulseTherapy by Leisuresonic

More

Check out the sites for more. It was a bit unorthodox to include all these thoughts, but I enjoyed reading it and it made me want to spend some time with the synth. Let us know what you think.

http://synthtronica.com/
http://leisuresonic.com/

on the iTunes App Store

Ever wish your music software could do something your way, something it can’t do now? Wish you could just get in there and change it yourself?

That’s some of the ambition of Renoise 2.6, the multi-platform music creation tool. By opening up the entire music tracker to scripting, users can create custom functionality and control surface. But scripting – while it sounds like the domain of hard-core geeks – doesn’t have to be daunting. That’s important, as presumably you want to spend some time making music. Scripting should save you time and let you express ideas more directly, not act as an impediment. So, the design of the Duplex feature in Renoise does work to make this customization accessible.

Renoise 2.6 has just gone gold master, meaning you can add it to your stable music setup. New in this release:

• Lua scripting. Customize the app using an elegant, clean, friendly language.
• OSC, MIDI support. Integrated control with Duplex (MIDI/OSC), native Open Sound Control support.
• Extensive hardware support. Maybe you don’t want to write a line of code, ever. You can let someone else do it for you, and reap the rewards. Already, Renoise has native, fully-integrated support for the AlphaTrack, BCF-2000, BCR-2000, KONTROL49, FaderPort, microKONTROL, nanoKONTROL, Launchpad, Remote SL-MKII, Nocturn, Monome, Ohm64, iPad via TouchOSC… all thanks to community support for the new scripting engine.
• Sample autoseek. Absolutely essential to making audio behave in the way it does in linear arrangement tools, the sample will play back from the position in the timeline, rather than from the beginning each time you hit play. (Seems obvious, but it’s part of making Renoise bridge tracker-style apps and more conventional, linear ones.)
• Better performance, compatibility. Tweaked performance on Linux and Mac, expanded file format compatibility, plus 64-bit Linux, DSSI Linux support. Renoise is a reason to run Linux, and Linux a reason to run Renoise, if you hadn’t guessed that yet. No, seriously, you’ll enjoy it. (I always feel like it’s telling someone to go vegan. Linux can actually be fun. And you still get to eat bacon.)

The release date seems the perfect time to really explain what Duplex is about, and what it means to you. First, it’s best to see it in action in this Duplex video. What you see is fully integrated hardware and software, but in a way that doesn’t necessarily require specific hardware. (There’s no “Renoise” logo on the controller – and you could substitute something very different and get the same impact.)

More information on the Renoise forum from the video’s creator, Danoise: Duplex – Playing With Loops

Basically, I’m arranging a small song on-the-fly, using a Launchpad + monome. Since the song was basically written using the StepSequencer, the vertical resolution of each pattern is just 8 lines. I then use the new loop feature in the Matrix to “pair” patterns into longer sequences.

This is just one possible workflow among many, but it’s one that’s I’ve found to be immensely rewarding when you’re sketching a tune out.

Bjørn Nesby, Duplex’s lead developer, explains his creation to CDM:

Duplex is aimed at both people who are willing to create their own scripts, and those who just want a nice way to interact with their music using Renoise. Many of the scripts (called ‘applications’ in Duplex) are pretty generic in nature, and will simply take control of a specific part of Renoise, like the Mixer or Pattern Matrix. This is something everybody can use, so this is where I focused my efforts to begin with. More exotic applications are also planned, but we needed to get the fundamentals in place first.

A thing that was clear from the beginning was, that the whole setup and configuration process needed to be as simple as possible. I think we succeeded in that, as my personal copy of Renoise will automatically launch the applications I need when the program starts, on three separate controllers. And I’ve heard from many people that they love this aspect of Duplex, as it reduces a potentially tedious startup process to an absolute minimum. Of course you can have an initial device setup process that you need to go through (like selecting the input and output ports for your device, which might vary from system to system), but in most cases you’d only need to go through this once, after which the device is ready to use.

And I believe this is not just about ‘convenience’, because sometimes you need to be absolutely focused on the music and not the order of which you launch various programs – especially true when you bring your music to the stage.

However, I have to point out that the configuration process is not perfect yet. There’s still room for improvement when customizing application mappings – this is currently done by editing some of the accompanying configuration files by hand, and although that might sound scary, it’s actually a pretty straightforward thing to do (and if not, the Renoise forum is there to help people out). Also, finetuning a setup like this is hardly part of the music-making process itself, so I hope it’s something people can live with for a little while longer.

From a developer point of view, the Duplex framework might be technically interesting as it attempts to follow the ‘write once, run everywhere’ model, as known from the mobile computing world, but instead applied to musical gear. For example, the Mixer application is able to run on all devices, from the Novation Nocturn to the monome128. Physically speaking, those are two very different devices, but everything in the Duplex API is abstracted to the point where a standard user-interface element like a slider can be a rotary dial (Nocturn), or an array of buttons (monome). In the application code, you simply create a slider, and base your logic around that. The framework will do all the dirty work of translating that into *actual* controls. This is possible because everything in Duplex is based around a descriptive XML file, the control-map. Unlike a traditional MIDI implementation chart, the control-map will not only describe the parameters and their ranges, but rather the complete physical layout of the device. Once a proper description has been made (and they are not hard to make, several of Duplex’ control-maps are user-contributed) you can launch an application on e.g. the monome that creates virtual sliders from individual buttons, because each button “knows” where it’s located in a X/Y coordinate space.

I’ve also tried to keep the syntax as familiar as possible. Many people who’ve done a bit of actionscript will probably recognize many of the concepts in this framework, hopefully making the whole experience a little less daunting for budding scripters.

One unique aspect of Duplex: the virtual control surface. When Duplex is installed, you can try out all the various supported devices, even if you don’t own them. Again, it’s the control-map structure that makes this possible, as you can define things like button size, color etc. Of course, this is not the same as the real thing (try hitting two buttons simultaneously using a mouse?), but it’s still interesting to play with, a huge advantage for developers as you can design a control-map that device owners can then try out and test, and makes for self-documenting applications, as you can assign tool-tips to the control surface that display exactly what each button does.

More information:
What’s New in Renoise 2.6 – Renoise Geek Edition.

Renoise Lua Scripting

2.6 Forum Discussion

And, of course, you can discuss Renoise and other trackers on our own Noisepages community. Specifically, we’re looking at how to use trackers in live performance.

Trackers for Live Performance @ Noisepages

Music software development includes some of the most sophisticated, expressive software out there. But it has long faced serious challenges in sales – audio software still appeals, generally, to a small slice of people, made smaller by factors ranging from piracy to the sheer complexity of available audio tools. As computing’s distribution model for software shifts, audio developers are undoubtedly watching.

Love it or hate it, what’s unique about Apple’s App Store for iOS is that it’s a one-stop shop for everything. With App Store fever spreading – new stores for mobile and desktop are either available or planned from the likes of Apple, Intel, Microsoft, and Linux vendor Canonical – we’re likely to see a new kind of store model. On desktops, Android devices, and others, multiple stores will compete with one another in overlapping arenas. They’ll do it without lock-in, too – unlike on Apple’s stores for iOS, you’ll have a choice of where to get your software.

Last week, of course, that list expanded to include Apple’s Mac App Store, coming to Snow Leopard and then the just-announced Lion.

Music creation and pro audio apps may be a specific niche, but creators of everything from plug-ins to audio software are at least interested. Little wonder: desktop music making software has always faced an uphill climb, but recently, iPhone creations have become breakout hits.

Just don’t get too excited yet. An early look at Apple’s guidelines for the store suggest restrictions will rule out a great deal of current Mac software, particularly audio software that relies on plug-in models. I’ve asked some independent developers to comment on what the store means to them, and take a look at some of those restrictions.

Several developers responded to my questions. Now, a disclaimer: clearly, the Mac App Store is not aimed at creators of strange synthesizers and effects. Nor is it possible to represent the full gamut of developers making software for musicians. TUAW has a nice round-up of more typical Mac developers, who are, unsurprisingly, more upbeat. I likewise expect that anyone who now has some success on the iOS platform – vendors like IK Multimedia or Smule — will be optimistic about the Mac App Store.

So, instead, consider this as a sampling of developers for whom the App Store may not actually change that much. I was, frankly, surprised to see plug-in creators and pro audio users assuming that the Mac App Store would be a natural marketplace for the software they care about. Early evidence is that it isn’t. But with app stores spreading across devices, the responses from developers provide some insight into longer-range challenges that transcend even Apple’s latest offering.

Gallery: sample applications and stores.

Could the Apple App Store be a viable option for creative music developers?

Angus Hewlett, FXpansion:

I’d say it’s a viable option for entry-level and somewhat novelty apps, and as a launch-assistance platform for brand new developers (allowing unknowns with no established reputation to get started in the world of ecommerce). It’s not of great appeal to FXpansion – we’ve been around long enough that I’d hope commercial trust isn’t a massive barrier to people buying from our web-store – but as a get-yourself-started platform, it is not completely without merit.

Of course, because these app stores are usually tied to a specific platform, for those developers who are on more than one platform, it does just add additional complexity, cost, and hassle. Admittedly it improves convenience for end users a certain amount (a good thing in my book), but the effort/overhead of getting out your credit card and typing in the number looks completely different for a $0.99 game you’re going to play for 20 minutes, compared to a $249 plug-in that you’ll spend hours/days just learning and (we hope) use several times a week for years to come.

Christopher Randall, Audio Damage:

The guidelines preclude selling plug-ins, so that rules out the segment of the industry I’m most familiar with. This will probably change, but my general feeling is that people that make things like Numerology will be well served, but for the majority of our business, our needs are a bit too particular to really benefit from something as broadly-focused as the App Store. And there’s no real financial incentive on Apple’s part to cater specifically to us, because we’re such a small segment of the overall software market.

David Viens, Plogue Art et Technologie:

As much as we like having our code base tested on as much compilers and platforms possible for correctness and efficiency, constant platform changes are quite boring, and usually dont spark any innovative ideas from us. Innovation not only is what drives us in the morning, but it’s also what users want, hopefully more than the typical will-it-run-on-my-toaster? kind. Also innovation is highly regarded by various tax break programs in many countries Also app stores make it impossible for us to do quick fixes, we could be committing code to Nintendo ROM carts that it couldn’t be different. So there is a need to raise QA and testing budget by a very significant amount, before release….

For standalone software, the kinds of things we’ve seen for iOS seem a likely candidate – particularly general-consumption audio “toys” (in the sense of stuff anyone can open up and use to make sound)?

Chris Randall:

That was my my general thinking. I was pondering it at length last night, and the smaller single-use app seems more likely to benefit from it, assuming it is a parallel environment to the existing App Store, with the same sort of customers. The chief difference between this App Store and the iOS one is that this isn’t the only option for purchasing software for your Mac. It has to compete with other channels, which is an important distinction, especially if most of its offerings are simple “casual” apps.

What about the app store landscape in general, as other players get into the business of doing their own stores? How does Apple fit in?

Gavin Burke, Future Audio Workshop:

The main issue is that the app store model just one piece of a bigger jigsaw and is tied in closely to the hardware, software frameworks and what this means to an independent developer and his/her ability to compete on a level playing field with established brands.

App store success is just one part in a bigger picture. The other players need to get the various parts right and not just create an app store and think it will work.

A major part is the price and ease of purchase. It’s easier to buy the software for $1.00 with one-click purchase than look on rapidshare.com for a crack. So price, ease of purchase, and last but not least, [making it] difficult to get the cracked version. Looking at it, it may only work if there is a single distribution channel and not multiple ones ( including rapidshare as a channel )

One flaw, though, in the Apple App Store is the charts. At the moment it is based on sales volume. Allowing people to view by highest user rating, etc., might help level things. Also, Apple can act as king maker with their ability to dish out the free advert slots on the device.

Angus Hewlett:

It’s a new channel that will no doubt get a lot of coverage – a few developers with the right products and first mover advantage will make some fast bucks for sure. After that, I don’t know. I suspect phones (and consoles – myself, I spend way more on XBox Live Arcade than on the iPhone or Android stores) are a better and more natural platform for cheap, one-shot novelty apps than desktops/laptops, simply because of how & where they fit in to peoples’ lives, but I’m ready to be proven wrong on that.

Are you concerned about Apple’s 30% cut of revenue?

Chris Randall:

Not at all. The app store runs on volume; that is its main attraction from a commerce standpoint. The trick is to take advantage of that potential volume, and the way to do that is through lowest-common-denomenator (e.g. “I Am T-Pain”) products.

Angus Hewlett:

Yes. It’s a lot more than the original generation of “app stores” (shareware ecommerce middleman sites like ShareIt, DigitalRiver, Kagi, NorthStar etc.) typically charged. We used to sell through ShareIt back in the day, they took about 10%, but once your turnover hits $10-15k a month,
it’s more economical to have a proper merchant account based shopping cart system (the hassle that entails costs a few hundred dollars a month in terms of overheads, paperwork, other bank-related BS, but it brings the average cost per transaction down to 3-5%). Also, at 10-15k a month turnover, your brand is probably well enough known that potential customers are likely to trust you somewhat as an online vendor.

Having said that – the terms-and-conditions aspect of being in an app store, especially when the operator is particular, capricious, anally retentive or all three at once (naming no names), is far more toxic than the 30% cut. Losing a predictable amount of money per sale is one thing, but failing to sell a single copy of your app – after you’ve spent months and $thousands developing it – because the store owner rejected it for reasons outside of your control is quite another.

David Viens:

30% is ridiculous. What enrages me is that users seem to think it’s normal and much less than ‘retail’ .. wuht whut??? We have never done retail ever and been using Share-It (which costs us less than 10%) for 6 years. That’s the price of a payment processor.

Sure, it doesn’t give you ‘visibility’ but what is that visibility on the 15th page of music software selection in a store? Can’t we just be as creative with our viral marketing, social network tricks as we are with the software itself?

Bandwidth price on Amazon S3 is microscopic (10 cents a GB), so not an issue, even with 100-megabyte demo downloads.

Share-it don’t care about the content, they never put their noses in our practices, suggest guidelines, or anything.

People are just getting to enjoy their new-found freedom with independent online music and fair-trade and local grown foods, however. they will let the inverse happen to software?
Will we see the movement to Fair-Trade software in 15 years?

Do you think it’s a model that could work, from a business perspective?

Angus Hewlett:

It’s a new channel, a few developers with the right products and first mover advantage will make some fast bucks. After that, I don’t know.

How well versed are you on acoustic physics in relation to loudspeakers, impedance etc.? There are some interesting parallels here with app stores – basically they are a good platform for allowing very small developers to cast a very wide net. Those of us who have a more specialist, focused audience can probably build trust with our audience via more efficient, focused channel….

It appears that plug-ins are ruled out by several of the guidelines issued by Apple. Care to comment?

Gavin Burke:

I can’t see the app store concept working for plug-ins; it’s pretty much already there with the downloads page on the Apple site.

If the app store is the only channel to purchase applications for a device that cracked software is not easily available for, then yes, it works. Otherwise, not so sure. We already have app stores for music software, like Don’t Crac[k], etc., with somewhat limited success. Also for complex niche software, it’s hard to beat the personal connections distributors have with stores and in turn with their customers. We find this especially true for Japan.

Artistic freedom and censorship

Artist and developer Kassen Oud offered some compelling thoughts on Apple’s developer “guidelines” and rules via Facebook. To him, the restrictions on what goes in the store conflict with making software art. I think it’s a reasonable and challenging point to make – just as Apple has the right to conduct their store in the way you wish, developers and artists presumably have just as much right to opt out.

To me, the big appeal to developing software for music is the chance to do something unique and individual. External limitations (like arbitrary moral guidelines or limits on the language to be used) conflict with that, to me. As the process used is important to me I need to be able to express myself about that publicly as well. This rules out Apple’s app store. With regard to code/ application distribution those are more important factors to me than the need to create applications for Apple’s app store on a Apple computer though that in and of itself would also be a prohibitive factor to me.

I don’t mean to imply that software on Apple’s platform would inherently be less “unique” or “individual”; I certainly do not wish to take anything away from my friends whose creative process wasn’t (apparently) affected by these factors and who did create very interesting works released on it, taking nothing away from their FOSS work.

Apple Developer Guidelines – Plug-ins Need Not Apply

Apple’s draft review guidelines for the Mac App Store have been widely posted, including full text. Here are the excerpts most relevant to the above discussion.

Plug-ins will almost certainly be rejected.

2.15
Apps must be self-contained, single application installation bundles, and cannot install code or resources in shared locations

(Plug-ins, by definition, install to shared Library locations, as per Apple’s own guidelines.)

Interestingly, though, hosts appear to be okay, just not the plug-ins themselves:

7.1
Apps that unlock or enable additional features or functionality with mechanisms other than the App Store will be rejected, except in cases where the application hosts plug-ins or extensions

Demos aren’t allowed.

2.6
Apps that are “beta”, “demo”, “trial”, or “test” versions will be rejected

Other guidelines are worth watching.

Various other guidelines provide fairly restrictive policies that developers will have to balance against their business interests. These aren’t unprecedented – see the strict review policies of venues like the Steam store for games. But those stores have seen their own share of developer complaints, and they’re specific to an audience (like gamers); here, it may be tougher for niche developers to justify. (That’s, at least, the feedback I’ve been hearing from music developers. For mainstream developers, the equation can be different.)

2.18
Apps that install kexts will be rejected

2.19
Apps that require license keys or implement their own copy protection will be rejected

2.20
Apps that present a license screen at launch will be rejected

2.21
Apps may not use update mechanisms outside of the App Store

2.7
Apps that duplicate apps already in the App Store may be rejected, particularly if there are many of them

Of course, some of the challenges of audio software are … unique. How many pro audio applications would meet the following guidelines?

6.3
Apps that do not use system provided items, such as buttons and icons, correctly and as described in the Apple Macintosh Human Interface Guidelines will be rejected

6.4
Apple and our customers place a high value on simple, refined, creative, well thought through interfaces. They take more work but are worth it. Apple sets a high bar. If your user interface is complex or less than very good it may be rejected

Comments from other developers are welcome. We’ll keep you posted.

Remember the music industry? We used to talk about radio play and record deals. Now, we’re talking developers, APIs, and analytics. Of course, the test, now as then, is whether there’s actually substance for music listeners and artists. On Friday, we looked at Apple’s Ping and how, via TuneCore, artists who aren’t Lady Gaga can get their own pages. We also saw some vigorous discussion of TuneCore, which helps you get your music into “big bucket” sites like Amazon and iTunes, and SoundCloud, who together offer integrated sharing and distribution.

The Echo Nest is an unusual animal in this landscape, an “intelligent platform” for other tools built on machine listening. Via algorithms that analyze music and try to absorb human knowledge about music, Echo Nest powers apps that do everything from matching beats to workouts to tracking data for artists and labels. They even apply natural language algorithms to try to absorb text about music, meaning a robot may be … listening to … everything I say. Okay, moving on… (This leads to some interesting sci fi scenarios… No! Robot! Obey your master! You will NOT harm Katy Perry!) Check out how it works.

The latest news from Echo Nest: they’re partnering with online store 7Digital to connect all that data with some actual music sales opportunities. The result: a DIY online music store platform. Developers can create their own applications and music sales websites, drawing on all of the intelligent data on music from Echo Nest – from machine-analyzed news feeds to musical attributes of individual tracks – with the 7Digital store. 7Digital is now up to 10 million songs, with art, previews, and sales. And whereas a lot of deals (like Ping) are closed, in this case the “product” is actually a toolkit that relies on developers to do interesting things. Whether or not this particular idea gets traction, that’s an interesting shift in models.

If you’re skeptical about those machine algorithms, you’ll be gratified to know even Echo Nest’s own Brian Whitman conceded around the Ping launch that he has “a strong aversion to music recommenders and music similarity services.” And Whitman knows his stuff – he’s an MIT Media Lab PhD and co-founder of the company. But there’s no saying this has to be limited to recommendations – as those jogging apps that analyze tempo illustrate.

I spoke to Echo Nest to find out more about the new offering, and to see what they think about other trends, like Apple’s much-hyped – then much-criticized – Ping.

Jim Lucchese of Echo Nest and developer Paul Lamere respond.

CDM: How do the two APIs fit together? That is, given that The Echo Nest and 7digital each have their own open APIs to begin with, what’s unique here about the joint offering?

Paul: We’ve incorporated 7Digital Artist and song IDs into our Rosetta stone system. [Ed. As the name implies, that's a mechanism for translating all the different ID numbers used by music databases for tracking songs, since there's no universal numbering system for music.]

This means that developers can use 7Digital Artist and Song IDs with our APIs, and can instruct our APIs to return 7Digital IDs.

In addition, all Echo Nest results can be limited to the 7Digital ID space. This is useful for tasks such as search and artist similarity. For instance, we can limit the results of an artist similarity query to include just artists that are in the 7Digital catalog.

We have also applied the Echo Nest analyzer to each of the 10 million or so 7Digital tracks. This means that we have a very detailed understanding of what every song in the 7Digital catalog sounds like. We know the tempo, mode, key, time signature. We have a detailed understanding of the loudness profile of the song and of the rhythm structure of the song, we know where all the bars, beats and tatums are. We have a detailed understanding of the harmonic and timbral content of each of the songs. With the Echo Nest / 7Digital partnership, developers can get access to this detailed analysis for any of the 7Digital tracks and use this analysis for all sorts of apps.

The Analysis data allows developers to create interesting playlists using the new Echo Nest Playlist API based on the 7Digital catalog. For instance, a developer could create a playlist of songs by artists similar to Lady Gaga with a tempo between 120 and 135 BPMs. You can read more about the playlisting API on my blog [Music Machinery]:

Is that a million songs in your pocket, or are you just glad to see me?
Echo Nest Playlist API Methods

Is this something that might become relevant to other stores, too — provided, of course, they were as open as 7digital is with their data?

Jim: We expect so, though we’ve found that 7Digital is definitely the most forward-thinking around open API access and developer offerings.

You mentioned that monetization for developers could be a big draw. How does that work — is it simply a 7digital affiliate program, essentially?

Jim: Yes. It’s like an uber affiliate network of developers. Sell tracks through your app and get a cut of each sale.

What sorts of applications do you imagine this might encourage?

Paul: I see a wide range of possibilities here:

• Music discovery apps – given the deep data that Echo Nest has about music coupled with the content that 7Digital can provide will make it much easier for developers to create new music discovery and exploration apps – think of a next generation of Ishkur’s guide to electronic music
  [4], or musicovery [mood-based radio search]
• Music listening apps – Applications that help people listen to music – for instance I can imagine a pandora-style app that gives the user more control over the listening experience – ‘more music like this but with more energy’ or ‘give me music by hair metal bands from the 70s that are from the UK’
• Music Interaction apps – since developers have access to a detailed analysis of each track, it would be possible for developers to create interesting music visualizers that respond to the music
• Apps that we can’t even think of – never underestimate the creativity of a developer – someone will create something that we can’t think of around music that we can’t even imagine now, but will become a big part of our music experience.

Artists are becoming increasingly excited about engaging these platforms themselves, particularly as many of them get more savvy about development and use independent distribution to get their work out there. Is there a place for music makers to do something with this kind of technology?

Paul: That’s a big question – some areas where platforms like 7Digital/Echo Nest platform will be relevant to artists are:

• Interactive media – the recent Arcade Fire interactive video ‘The Wilderness Downtown‘ is an excellent example of the type of interactive app that artists will be able to create on top of platforms like 7Digital/Echo Nest/
• Analytics about fans – artists are spending more time figuring out how to market their music. APIs around data about who is listening to, going to shows, or writing about an artist will be very useful for artists

What about the end user (that is, the listener) – what will this allow a music lover to do that they couldn’t before?

Paul: As the music app space developers, we will see better applications for music discovery and exploration. A music lover will be able to find more interesting music that they will like.

Music listening apps can be greatly improved with access to all of the contextual information about the artist and the song, along with recommendations for other artists And you can see context-dependent playlisting – a user could create a ‘jogging’ playlist that has songs at just 135 BPM, or a high school reunion playlist of pop songs from 1985.

One of the things that struck me about Apple’s Ping announcement was that – while we’re waiting on information about any potential API – the general sense was of something pretty walled-off, specific to just their store, in the client software, and not connected to the Web and across platforms. Do you see a contrast in philosophy here?

Also, the initial reaction of the press surprised me; the first response was that Apple’s installed user base would just wipe everyone else out. I disagree, but how do you see open offerings like this fitting into (or providing alternatives to, or both) that Apple ecosystem?

Paul: There’s no question that Apple’s installed base makes their foray into social music a significant event. Millions of people who would never think of visiting a social music site like Last.fm will now become part of a social music network. That will be good. Music is very social, and one of the best ways to discovery new music is through one’s friends. However, social music systems can have some problems: Feedback loops, decreasing diversity in listening, susceptibility to hacking, schilling and spamming, and the cold start problem (where new artists have no way to break into the listening rotation) are all issues that plague social recommender systems.

Systems that can draw on multiple sources of data – social data, plus other data like curated reviews, blog posts and the content of the music itself can provide a way to avoid the problems inherent in social-only systems. I think that music discovery applications that are built on broad sets of data (like we are building here at the Echo Nest) will ultimately do a better job in helping people finding new music.

The most interesting possibility I see with the opening up of all the different music-oriented APIs is the emergence of a thriving music application developer community – (like we are starting to see coming out of the Music Hack Days). This developer community is ultra creative and they love music. They love to listen to it, and they love to create it. I am convinced that these developers are going to have as much of an impact on the future of music as the iPod did. These developers will be creating the applications that will help all of us to create, to discover, to interact with and to listen to music.

The iPad may be the catalyst, but look beyond the platform, and you’ll see a reinvigorated examination of how to think about musical interfaces. If you’re looking to relax and make some musical noises on your iPad, check out the videos above. It isn’t actually necessary to fully understand the design work behind the interface to play and make music. But if you are interested in reflecting on the way design issues impact musical expression, and you’re a bit of a music theory nerd, read on.

Using an array of rectangles arranged in a harmonically useful way, and color coding for pitch, SoundPrism is a glimpse of a more graphical future for music software design. (Nor is this necessarily limited to the iPad in the long term – in addition to Windows 7, Ubuntu 10.10 is getting official multitouch support, which I think both validates Apple’s work and suggests we’ll see more platforms for this kind of interface.)

And, bonus, it all demonstrates why arranging pitch by the Circle of Thirds can be ideal. I got a chance to talk to the developers of SoundPrism about the thinking behind the software.

We will get a mention of Jean-Philippe Rameau. I just hope it doesn’t start a debate between Rameau and Lully fanboys again.

Sebastian Dittmann, CEO of developer Audanika, talks to us about the software.

CDM: What’s your background as a developer / musician?

Sebastian: I wouldn’t call myself developer although I’ve been working in IT and digital media creation/marketing before. I come from a very-music oriented family – both parents opera singers, so I had to learn to play the piano, but can’t play it very well.

I met Gabriel Gatzsche – the inventor of the technology SoundPrism is based upon – during a colloqium for music in digital games at the Fraunhofer Institue for Digital Media Technology when he was showcasing an earlier prototype of it back then. His demo of 5 minutes taught me more about harmonic theory than years of piano lessons before. That was 10 months ago and he is now the CTO of our company that we decided to found.

Can you talk about what inspired the pitch layout on SoundPrism?

The idea for SoundPrism is quite old actually. About 22 years ago Gabriel’s father found that organizing tones of keys in a circle of thirds makes it very easy to understand basic harmonic theory because the visualization ‘just makes sense’. Back then he taught it to his two sons, Gabriel and David, and from then on they passed all music school theory exams with flying colors. He tried to convince their teachers that this method was far better for teaching children but they didn’t listen to him at all.

David studied music to be a guitar/music teacher, writing his final thesis about the didactic concept of SoundPrism. Gabriel studied media technology and wrote his dissertation about it and came up with software that made it possible to not just use the theory to explain music but also to play music with an interface that is set up like the theory.

During his dissertation he found out that lots of other scientists came up with similar concepts. He quoted them in his thank you note to the team when we submitted SoundPrism to Apple: Jean-Philippe Rameau, Leonard Euler, Moritz Wilhelm Drobisch, Hugo Riemann, Moritz Hauptmann, Fred Lehrdahl, Carol Krumhansl and Werner Pöhlert just to name a few.

The reason we are able to actually use these concepts now is that we finally have interfaces that can change visually and aren’t static. I see SoundPrism as knowledge poured into a dynamic interface that enables users to just use that knowledge without having to acquire it first.

Could you describe – for people comfortable with music theory – how it’s laid out and why?

SoundPrism is based on Hauptman’s array of thirds in its vertical dimension and horizontally on a pitch class/pitch height model by Moritz Wilhelm Drobisch. We found the combination of these concepts allows musical interfaces which enable the player to control both harmony and melody at the same time in an optimal way.

This is just the very first version in the first configuration on the iPad though. There have been prototypes using Space Navigator technology, a WiiMote, Jazzmutant Lemur and also multi user augmented reality installations with camera based motion tracking.

You’d have to talk with Gabriel about this since there’s a lot more to it. The interface right now is tailored for tablets like the iPad. We’ve evaluated other representations with field tests and the current form lets users express themselves freely while at the same time guiding them gently to ‘do the right thing’ from a harmonic standpoint.

Gabriel alone has spent 5 years of his dissertation working on the concepts and there’s lots of music psychology and other really complicated concepts involved in it that I’m not the right person to explain.

Ed.: I wanted to go ahead and get this out there, but if you’re interested, readers, I’m happy to pursue.

Have you considered providing control data to other applications (i.e., routing over WiFi, Bluetooth, etc. to a computer for people wanting to use this as a controller)?

Yes.

Obviously, at this point, you’re just waiting on Apple approval, but any plans generally for the future?

SoundPrism is really just the beginning. We think we roughly know what musicians want because our team consists of musicians. At the same time we already have other prototypes and ideas that do very neat stuff beyond the scope of the current implementation of SoundPrism for the iPad.

But we hope for lots of constructive feedback from all over the world so we can add the features people want that we haven’t though of yet. I have to say I’m a bit nervous because I didn’t think SoundPrism 1.0 would get that much attention already before we’re live.

One of our dreams is to establish the SoundPrism technology as a widely used tool. Not just for musicians but also as a door to the world of creating music for people who don’t think they can play an instrument.

What will pricing be?

SoundPrism will be at around 4 US$ initially. That might change later on and we’re also planning to include an InApp store for people to add more features, sounds and other really neat stuff.

Now just waiting on approval…

Follow CDM on Twitter and I’ll post when the app is up.

In the meantime, here’s one more video. I love the way this sounds, here paired with a Korg KAOSS Pad KP3 in what appears to be a really practical live music rig:

More at the developer site:
http://audanika.com/

Thanks to Chris Mousdale for connecting us with the developers!

Developers for iPhone, iPod touch, and now the iPad have long complained about lack of access to the iTunes library, the file store and metadata for uploaded files. While version 3.0 of the OS provided limited playback capabilities (play/pause/stop), it was a far cry from what you’d need to build a DJ or other music app that made use of a user’s content. You couldn’t, for instance, adjust volume control, cueing, no mixing, or scratching.

Update/clarification: I may have exaggerated how much access is available. Obviously, Flare is working, as you can see, but the question of the exact mechanism by which it works and the documentation of the updated APIs is something that remains to be confirmed, beyond Asyn’c experience. I’m actively researching this issue. I’ve changed some wording to reflect the fact that what we do know about access to this functionality in the SDK, and how it works in the real world, is based on one shipping application for iOS 4 and the experiences of the developer of that app. The other issue is what mediates between the file in the database and playback; that is, unlike on desktop, you don’t necessarily get direct buffer access to files without an additional step. But the upshot is still that you can scratch tunes from iTunes, at least! -PK

Executive summary: It appears what has opened up is the ability to export assets from the library on the device, and following that extra step, use a local copy for access to buffers. See comments for more discussion. This would fall well short of being able to manipulate buffers from the file directly, but it’s a step. Stay tuned for further info next week as I confirm with developer sources.

With iOS 4.0, currently available for the iPhone and iPod touch, that changes, at least according to what we’re hearing from one developer – and seeing demonstrated in a shipping app. CDM has confirmed with developer Async Games that a new public API (meaning a fully supported developer tool) provides something closer to full-blown access to the iTunes library, in terms of not only metadata about tracks but also manipulation of audio data during playback. Result: you can now, for instance, scratch audio from songs uploaded from iTunes on a Mac or PC to the device. If correct, that would be likely to mean a coming flood of DJing on the iPhone and iPod touch, soon to be followed by the iPad whenever the 4.0 OS becomes available for Apple’s tablet. (Nor is this even limited in applicability to DJ apps – iTunes could become a more convenient way for syncing your own tracks and samples, or loading a playlist of backing tracks or other musical content onto your Apple mobile.)

The caveat appears to be that you need to “import” tracks from the library first; I’m confirming what that means and how long it takes. That would be shy of having direct, buffer-level access to tracks as seen in desktop DJ apps.

So far, the first app to actually ship with support for the feature is Flare Scratch, a simple simulation of a turntable with touch scratch support. You can scratch any song in your iTunes library, meaning this app developer has figured out a way to access audio buffers as they’re played. Apparently, the API isn’t entirely perfect, but it sounds like a step in the right direction, based on the impressions we hear from this developer. The developers of Flare Scratch also make a more full-featured DJ tool called Baby Decks DJ for the iPad, which could mean that, with iPad support, the tablet could become an all-in-one DJ solution. (Of course, a MacBook still has one significant edge – far, far greater storage capabilities.)

Flare Scratch @iTunes
Flare site

Incidentally, this isn’t an issue with Google’s Android platform, which provides open access to any file stored on the user’s SD card, and complete buffer access to that media (video, audio, the lot). Note that there may be some issues even on Android with using compressed assets; I’m writing some tests. Android developers, however, face a different set of challenges, like inconsistent handset audio drivers that can interfere with crackle-free, low-latency audio performance. (That situation looks brighter in the future, but it’s a whole other can of worms.)

Anyway, if you’re all alone on a park bench and want to scratch softly to yourself, you now have a solution. (Other videos, ranging from the useful tutorial to the somewhat frightening demo, at iflarescratch’s YouTube channel.)

I hereby open commenting to a bunch of people complaining about how this will ruin DJing and continue the slow decay of civilization itself. (Hey, I’m just the messenger. Leave me out of it.)

Pianoteq is an effort to model, rather than sample, acoustic pianos and other instruments on the computer. Now in its third major release, its interface and sound generation have each matured. Using mathematical models in place of recorded sounds, an entire grand piano fits in just a few megs of space, rather than requiring several DVDs, and the software maker claims the results can be more natural and playable.

Pianoteq, which runs as do its rivals on Mac and Windows, is also unusual in providing support for the Linux operating system – something some developers have claimed isn’t practical with commercial music software. And a new “Player” addition, announced this month, makes it more affordable. In addition to software development, the team has even launched an extensive piano restoration effort:
http://www.pianoteq.com/kivir

I’ve been impressed with my time playing with Pianoteq’s software. That’s especially meaningful to me, as my background in music has been on acoustic pianos, back to when I was literally old enough to reach the keys for the first time. But I also wanted to know more about how this software developed. Its rigorous approach to modeling has attracted a lot of attention in the virtual instrument world, and the fact that it targets Linux alongside Mac and Windows challenges notions that commercial software can’t make it on the free operating system.

Pianoteq sent along some extensive answers, which I’m pleased to be able to share. Naturally, they’re proud of their software, so there is a bit of expected boasting here. (I’ll discuss more of the experience of using the tool, and the new Player version, shortly). But they also have some fascinating commentary on sound design, modeling, and the development process. In the “geeky as we want to be” spirit of this site, here’s the full scoop.

CDM: Can you talk about the background of the company? How does one make a shift from dealing with the physical instruments and tuning to thinking about mathematical models, let alone translate that into actual software?

Philippe: the Pianoteq history is strongly connected to my first job as a piano tuner. Then at the age of 31, I started a new life with basic studies in mathematics at the University of Toulouse, France. After what I prepared my PhD thesis on the parametrization of vibrating phenomena, without imagining that it would be the basis of my third life with Pianoteq. Thanks to these two skills and to an exceptional scientific environment in Toulouse, I succeeded in identifying important phenomena responsible for the generation of the piano sound and proposed a model which describes the whole interaction of the soundboard, strings, bridge and air.

Julien: I was working as an engineer at Institute of Mathematics of Toulouse, with Philippe (who was also my teacher when I was a student). My focus was on an open-source finite element package ( http://home.gna.org/getfem ) when Philippe told me about his project of piano sound synthesis. I took charge of the development of the real-time engine, and we quickly decided that we wanted to turn this research project into a commercial product. Thanks to the French law on innovation and research (1999), the support of INSA Toulouse, and the Institute of Mathematics, the start-up MODARTT was created in 2006 to sell Pianoteq, which was at that time the first fully modeled piano instrument.

Niclas: I represented many of Pianoteq’s customers of today, being a part time piano composer with a love for the piano instrument. I was updating an extensive article on digital piano technology in 2004 when someone advised me to have a look at Philippe’s research. I immediately understood its potential, which is why I suggested Philippe to assist in marketing and product development. Since then, I have participated in the product development and testing and am also in charge of sales and customer service.

Modeling the Piano with Math

I remember trying previous attempts at piano modeling and finding them interesting but ultimately unsatisfying. I think people who pick up Pianoteq have, immediately, a different experience. What’s different about this modeling approach than those that have come before – and, for that matter, why did it take until now?

Philippe: The idea of modeling musical instruments is very old and has always faced great difficulties: the complexity of physical phenomena, the sensitivity of the human ear to the slightest imperfection, and the difficulty of running a complex model in real-time. The latency needs to be so small that it gives to the musician the impression of playing a real acoustic instrument. Until now, attempts have only confirmed that the task was not easy. The state of the art of digital pianos is based on sampling technology. Each note is a recording of how it sounded at a specific moment, without taking into account the complexity of the instrument. The huge data generated by sampling can reach 40 Gbytes for a single piano. The flow rate of data transmitted from the hard drive to the sound device is too high for the current hardware capacity and it can happen that one hears crackles. [Ed.: I would say fast hard drives, optimized software streaming, and other intelligent configuration can certainly avoid crackles, but the fact that, say, a low-end hard drive might choke means that Philippe's point about data intensity here is nonetheless well taken. -PK]

Moreover, the reproduced sound lacks vividness. Hence, creating a piano model which takes into account the interaction between hammer and strings, the interaction between strings and soundboard via the bridge and the interaction of the soundboard with the air is of great interest. Based on mathematical models, Pianoteq allows parameters to be stretched as long as the model permits, resulting not only in new performance styles but also in new piano sounds. Pianoteq is thus also an innovating tool for music creation and can be useful not only to musicians but also to piano manufacturers and piano tuners for simulation and training purposes. Pianoteq makes excellence in piano available to all. Among Pianoteq users, composers and professionals in music creation are certainly the most excited with our innovation. Pianoteq offers what acoustic and sampled pianos cannot offer: new opportunities for music creation and a pure piano sound that is not altered by its environment (reverberation) or by recording devices.

Can you talk more about the model itself? I can see the components that are modeled, but – realizing we have fairly technical people among our audience – what are the basic modeling techniques?

Philippe: the modeling technique is based on various standard techniques issued from mechanics theory including modal analysis (calculation of vibration frequencies and the corresponding modes) and parametrization techniques that we developed at the university, as well as from a precise knowledge issued from my previous job as piano tuner/restorer of what is going on in a piano and what is important.

How did that approach to modeling evolve? Obviously, there’s this strong mathematical research background. But what’s the process like of translating that theory into something that’s usable? Were there mistakes or adjustments along the way?

Philippe: I don’t think there were mistakes along our model evolution, but more a constant improvement in the details taken into account by the model, looking closer to the physics of the piano and finding suddenly some simplification in the algorithms that allow to take include more details for the same computational cost or being more precise in the simulation.

Developing for Linux

How did you make the decision to support Linux in addition to Windows and Mac?

Julien: The initial prototypes for pianoteq were developed on Linux, using [audio system] JACK, with no GUI. Later, when we added a graphical interface and turned it into a VST plugin, we used VSTGUI for its interface, which is not available on Linux. However we had quite a few requests from Linux users, and we did make sure that pianoteq was running fine in WINE [an open-source implementation of Windows' APIs, allowing Windows programs to run in Linux]. During the development of Pianoteq 3, we switched to the JUCE toolkit, which is a great piece of cross-platform software. Thanks to JUCE, the Linux port was really easy to do, so we decided to give it a try and see what happens.

One complaint I hear from developers about Linux is that it’s “impossible” to do commercial development, because you “can’t” distribute binaries. Obviously, that didn’t stop you. I’ve tried Pianoteq on Fedora and Ubuntu, though, under both the real-time and default kernels, and had immediate success. Now, I imagine there’s a good bit of work that goes into making that happen. What was your experience like as a developer? Do you feel that the result is successful, that it is a usable solution for users?

Julien: This was in fact something that we also feared, that the Linux port would turn into a support nightmare. However a good example of a successful application that is distributed in binary form is Renoise. That showed us that it was possible to do. In fact Renoise also uses JUCE, but I was not aware of that fact at that time. What helps here for binary portability is that we have very few dependencies. JUCE is statically linked, so pianoteq depends on very few dynamic libraries: ALSA, X11, libc (even old versions), and basically that’s all. We had to hack some sort of weak linking for JACK in order to allow pianoteq to run even when libjack.so is not available. Of course, if you want better integration in the desktop, things get much more complicated.

What’s your own Linux testing setup like? (distro? kernel?)

Julien: Pianoteq is built on a Debian Sarge box, otherwise we generally use Ubuntu for the desktop, with the default kernel. [Ed.: The distribution Ubuntu is itself built on packages from Debian; 10.04 LTS uses Debian Testing.]

I’ve likewise been impressed with the vanilla kernel (as opposed to the “real-time” branch), which can save some setup time and configuration work. (My audio interface is a Native Instruments Audio Kontrol 1). Any thoughts on what setups may be most advisable? (You document some of this in the readme.)

Julien: I’ve never been lucky with the kernels labelled “rt” , and I really hate when the computer randomly hangs so I prefer to stick with default kernels. We don’t have issues with them, as long as your user account has been granted real-time priviledges. I believe that for now, the most overlooked setting for realtime audio is the CPU frequency throttling, which is a real audio performance killer, especially on the less powerful machines such as netbooks. You really need to have your cpu running at full speed 100% of the time, especially with a software like Pianoteq which needs quite a bit of CPU power.

Ed.: Before we give the realtime kernel a bad name, some of those “random hangs” were not necessarily the kernel’s fault – a bug in Ubuntu’s implementation caused the system to crash when combining the RT kernel with proprietary NVIDIA drivers, for instance. But if this sort of thing scares you, the vanilla kernel remains a strong option – it’s the default for a reason. The larger discussion is best saved for another article, but suffic,e to say, if latency-sensitive piano instrument developers are okay with the vanilla kernel, you shouldn’t feel you have to install a realtime kernel just to make music. If you want to test it, projects like Fedora’s Planet CCRMA can make it easier to use.

Are you finding that there is some positive response to the Linux version?

Julien: Yes, very positive response. In fact, a bit more than what we expected at the beginning. Approximately 4% of our customers are using the Linux version.

I could even imagine it working on netbooks. Based on load, it appears perfectly workable, which means a really cheap ultra-portable piano you can take anywhere.

We spent some time to make sure that the latest versions could run on netbooks, altough with very high CPU load (80% or more). However I’d recommend to use a more powerful laptop in order to have more room for the cpu. Ed.: Given the range of Atom netbooks out there now, I may have to test some of the newer models on this.]

There’s not currently a Linux plug-in version, correct? I’ve been just as happy using JACK [a standard for routing audio between applications], but what went into that decision?

Julien: Right, no plug-in version on Linux for now. The problem is that “plug-in” may mean any combination of VST, DSSI, and LV2. VST would be the easiest for us, but very few hosts support it ( basically only proprietary hosts such as renoise and energyxt, and also jost). DSSI is said to be obsolete, while being not to hard to support (except the GUI has to run in a separate process..). LV2 is said to be the future, but
it seems to be quite complicated to fit a “vst-like” plugin into an lv2 plug-in. We have not yet taken a decision. It is already enough of a pain to support the numerous plug-in formats on Mac and Windows. We will probably add support for JACK sessions quite soon.

In Use

Initially, having so much power over sound could be overwhelming – looking at the number of parameters you can adjust in the real-time mathematical model. Aside from the presets (which themselves sound pretty terrific), how would you suggest someone go about beginning to explore the options? Is there a workflow that makes sense for approaching adjusting the sound?

Answer from Pianoteq:
http://www.pianoteq.com/faq?category=pianoteq_working

Ed.: So I should have read the *** manual! Here’s their advice:

If you need to adapt the piano sound you could for example try adjusting the hammer hardness (2) to achieve a different brightness of the hammer strokes. Increasing unison width (3) makes it a bit out of tune (resembling certain acoustic pianos). The new powerful sound recording feature (4) lets you place up to 5 virtual microphones anywhere around the piano to achieve ultimate ambience and tone colour. The dynamics and velocity curve (5) will most likely need to be adjusted to the keyboard used in a MIDI file performance.

Indeed, this commentary makes sense. Hammer hardness is something that could be adjusted in the maintenance of an actual piano. Since you listen to a software piano model as though it is amplified, adjusting mic placement (as on a number of piano software emulations) is a no-brainer. And dynamics and velocity curve are essential not only for MIDI files, but if your keyboard controller lacks these controls onboard.

As my friend Jim Aikin noted in his review of Pianoteq – why would you adjust the speed of sound? (I suppose you could account for different altitudes; I could replicate the Aspen Music Festival!)

Julien: Well, why not ! Being in a virtual world gives you access to some parameters that cannot be easily modified in the real world, if they give interesting variations of the sound, then they are worth being adjusted!

One small note – it seems the metronome is not connected to the playback and recording, which means that MIDI sequences won’t export to SMF correctly? (Or is this a Linux bug?)

Julien: Yes, it’s not a bug, the MIDI recording and playback abilities of the standalone application are very minimalistic. It is best to use a real sequencer for serious work.

Where can people read more about the featured historical instruments?

http://www.pianoteq.com/cimbalom
http://www.pianoteq.com/grimaldi
http://www.pianoteq.com/blanchet
http://www.pianoteq.com/schmidt
http://www.pianoteq.com/graf
http://www.pianoteq.com/cp-80

There are some fascinating add-ons that aren’t pianos. Is it possible at some point that other sound designers might be able to use the sound engine to design their own instruments?

Julien: We believe that Pianoteq PRO is already a first step in that direction, with its ability to edit each parameter note by note.

How are users using this onstage and in the studio? What sorts of users have you found using the product?

Niclas: There are many composers and keyboardists that use Pianoteq, from amateurs to professionals, on stage as well as in the recording studios. We have presented a few reference users here: http://www.pianoteq.com/references

Thanks to the Pianoteq guys for being thorough in the answers. I know not everyone responds in the same way in regards to the perceived quality of the model, but my own feeling is that the effort makes the instrument terrifically playable and responsive. The best way to see for yourself is to give the demo a go, and listen to the results. I’ll follow up more on this instrument, and how it’s become a central part of my Linux music workstation, soon. Let us know if you have questions for the developers I missed.