By the 1990s, the notion that computer software could be a means of delivering interactive digital art more personally was enjoying a Renaissance. This was the age of the Voyager CD-ROM, which catered to new multimedia PCs and Macs with titles from the likes of Laurie Anderson and Morton Subotnick, the decade in which Brian Eno released Generative Music as software and Monolake – before Ableton – included a Max/MSP patch with an album. But the reach of these experiments was doomed to be relatively limited.

Now, of course, things are different. First, we saw some widely-available audiovisual toys, coinciding in particular with the debut of the iTunes App Store. But now, those fairly one-dimensional experiments are beginning to blossom into something else. When these particular gadgets and app stores are forgotten, the question is whether those aesthetic adventures, the personalization of the digital art experience, will endure.

Joshue Ott, co-creator of Thicket for iOS, points to a review of that application on Apple’s App Store. “I always want to touch the masterpieces in museums,” a user says in that review. “I’ll use Thicket instead of getting arrested!”

“It’s the democratization of our own performance works,” muses Ott. “It’s a way people can play along with us,” he says. “We’re constantly creating processes to create sound and music; it’s what we’ve done for ten years or so,” chimes in Ott’s creative partner, Morgan Packard. “Now people can own the processes, not just the results.”

Ott and creative partner Packard have long each been visual and music performers, respectively. That meant what it has traditionally meant: the artist gets up in front of an audience, the real work hidden behind an onstage laptop. With Thicket, by contrast, the raw materials of that performance became embodied in the software itself, and thus in the hands of the audience, who can double as performer. At first, this software included only a simple mode or two, each with a specific sound, musical ambience, and visual look. Even in those versions, Thicket made some appearances in an occasional gallery show or performance – the app you download could also be the art.

As Thicket has added modes, though, it has evolved in a kind of platform of its own. Ott and Packard produce new works that can be distributed as in-app purchases (more on how they contend with that in a bit). The sum total of those modes has created a massive audiovisual playground, a compendium of ideas and aesthetic.

A new version released this week adds three new modes, seen in the video at top here, building atop modes added in late December. For the first time, you can use Thicket on an iPhone and not just an iPad; it’s a Universal app. Screenshot sharing is available, too. But the addition of all these modes, unveiled with a “reboot” of the app at the end of last year, represents a shift in thinking as these artists and developers reevaluated what it was they were doing.

“We found that the modes were becoming so different, so much deeper,” says Ott:

We were having such fun using it as a big sketchbook that we decided to ditch the ‘rotate to change modes’ system so that we could handle lots of modes, rather than just four or five. The modes in Thicket reboot are completely new, and each one is a lot more complex than the older modes. They’re all very different, and each have separate methodologies behind how you control them. We’re playing with different concepts in user interaction design, searching for the right intuitive feel to make a true audiovisual instrument (among quite a few other things).

In other words, if you haven’t played with Thicket lately, it’s a different animal. It’s a Long Play album to the first version’s single cut. The work is immersive, too; you can transmit video output via HDMI or VGA on the iPad, and get up to 1920×1200 HD output, with no menu intervening. (One of the many significant current drawbacks of Android for the moment for artists: the move to a soft menu on Android tablets means menu detritus that never goes away. Artists were intensely relieved this week when Apple’s new iPad kept its signature, dedicated hardware menu button.)

Morgan Packard says he has some strong feelings about why this kind of experience has value in his work:

I’d say where we both overlap is our shared interest in how abstract sound and picture, plus interactivity, all can work together. Thicket is a bit of a research sketchbook for us. There’s something very magical about just twiddling your fingers and having sound and visuals spring to life. Frankly, we don’t entirely understand this medium yet. But we like not knowing, trying to understand it in different ways.

The gestural thing is huge with us, and is at the core of what thicket is. It’s partly why I’m a bit resistant to the idea of layering features on to Thicket. Of all the different people who give us feedback, I get the most gratification from parents of special needs kids.The non-fiddly, large-motor interaction style is very accessible to a huge range of minds and hands. I want to explore this more, to give people new ways of feeling expressive and creative with movement and gesture. In my mind, that’s what’s really special about what we’re doing.

The duo did get a chance to try the app with people with different user needs. Ott explains:

Morgan and I actually toured a special needs school earlier this year and observed autistic kids using Thicket. A very special music teacher is using Thicket (among a couple of other technologies) to teach the kids music, and had found that it seemed to really empower them. He offered to let us visit and we happily agreed… really really amazing experience.

As Subotnick hoped years ago in “All My Hummingbirds Have Alibis” for Voyager, the distribution of art as software can create a new kind of “chamber” art, in which the work is personal, enjoyed by a few people. It can be a family or a couple of friends on a couch.

Of course, somewhere in all of this, these artists are looking for revenue in order to be able to devote the massive amounts of development and testing time the application demands. (Neither has quit day jobs, which means finding a way to devote resources to development.) Thicket easily climbed in download counts, but only after the application was made free. In-app purchases have been a tough mountain to climb, but have at least allowed some revenue to trickle in; the challenge was finding a way to make them appealing to users, says Ott:

I think in general people hate In-App Purchasing (IAP), because, in general, I think IAP is usually not handled so well. We have thought a lot about how to show people exactly what they are buying before they buy it, and I’m really pleased with what we’ve come up with. Every mode in the new Thicket has a pre-recorded “demo” of one of us playing the mode. Before you buy a mode you can watch this demo, learn what the mode can do, watch someone use it in an interesting way, and decide if that’s something you’re interested in or not. You can of course watch the demos even after you’ve purchased the mode (and the free Sinemorph mode also includes a demo as well). The demos are a great way for us to show users different tricks and techniques.

So the reboot is really about making Thicket a platform rather than just a single art piece. Something that we can keep adding to (with a financial structure that makes sense for us to keep adding to). Something that we can start collaborating with other artists on – we are talking to a couple of different people about releasing modes within the Thicket system. So yeah, that’s what the platform part is. We’re really excited about it, and what it will become in the future.

But these concerns aside, the developers aren’t just creating Thicket for users; they’re building something they use themselves. As Josh explains:

I’ve performed with Thicket now a couple of times, once at the excellent SONiC festival, and another at Issue Project Room in a program curated by Ryan Lott (AKA Son Lux), and have started to really feel like it has the potential to be a new form of audiovisual instrument. I want to see more stuff like it- things that generate graphics and audio intertwined, and I want to continue to explore these relationships in different ways. I’m actually pretty excited about performing with Thicket more, and I think doing so will push it even further in that direction.

“That’s really what an audiovisual instrument is to me,” says Ott. “It’s something that you can bang on and make something interesting, but you can touch it subtly, as well, to shape it, to express with it. That’s what I want to make. We’re right at the beginning of that exploration, and I think we have something that is a promising vehicle for it.”

You can try out the new Thicket now, as seen in CDM Apps:

Thicket @ CDM Apps
[Says iPad, is actually now Universal. PS - music and beauty flow from my fingers all the time - no app needed - but I'm glad now the rest of you get the chance.]

Apple yesterday described their iPad as “this magical pane of glass that can become anything you want it to be.” So – how about making mobile devices into what you want it to be?

With the help of author Peter Brinkmann and publisher O’Reilly, we’d like to give you a taste of Peter’s new book, Making Musical Apps: Real-time audio synthesis on Android and iOS. Imagining that a lot of you are especially curious about iOS, we’ll include the chapter on how to get started with development. It really gives you a sense of how easy this can be; the challenge is, as it should be, coming up with musical ideas. And Apple did say that they thought that technology was at its best when it was “invisible,” not when it was “inaudible.” So let’s make it make some noise.

(Android developers, libpd actually got its start on Android and runs quite well even on very primitive Android handsets, so consider this a sample; the full book – and the libpd site – include loads of examples on the Android side, too. In fact, because libpd works basically identically on the two platforms, it’s a great choice for making cross-platform development easier.)

In this excerpt, Peter covers:

• How to set up your development environment
• Starting a project with Xcode, and including Pd
• How to make a Pd patch run in your app
• Making the Pd patch and your UI connect with each other (here, from the app’s UI to Pd; the book covers both directions)

In fact, in just a few pages, you’ll have a working guitar tuner for iOS. Have a look:

Read it on CDM’s Scribd page:

Making Musical Apps (Excerpt: How to Build a Music App for iOS)

Direct PDF download link, hosted by CDM (please don’t link to this file directly):
Making Musical Apps (Excerpt) [PDF]

I’ve read an advance copy of the whole book, and my review is simple: if you’re curious about this stuff, get this book. Peter’s style is friendly and precise; no technical detail is left out, and yet those details aren’t overwhelming. The book can be accessible to those new to development, which is essential for a title that’s likely to be read by people who are used to Pd, but dipping their toes into Java and Objective-C for the first time in order to get their patches running on a device.

Ready for the full book?

Get a printed copy on Amazon:

Or read the Kindle edition:

For multi-platform epub, mobi, and PDF formats, head straight to the O’Reilly page:
Making Musical Apps [shop.oreilly.com]

http://libpd.cc/read-the-book/

The tablet – or at least the iPad – is beginning to look like a terrific accessory for lovers of MIDI and hardware. With its compact form factor, it coexists nicely with your MIDI gear and lets you focus on sequencing, perhaps moving to the traditional computer to finish up your track, mixing, and the like.

And it’s spawning MIDI sequencer apps that imaginatively explore ideas for how to create sequencing, all with an immediate touchable interface.

The latest entry: Genome MIDI Sequencer claims to be the “first true pattern-based MIDI sequencer for iPad.” The word “true” might be debateable, but it looks thoroughly full-featured, with clock sync, MIDI import / export (ideal for working with your desktop Mac or PC), and other must-have portable sequencing features:

• Hard-line MIDI support: Core MIDI, Camera Connection Kit, Line 6 Mobilizer Mk I and Mk II compatibility
• Wireless MIDI: Network MIDI support (works with Mac and, via other support, Linux and Windows)
• MIDI Clock: Start, stop / send, receive
• Lots o’ messages: Note On, Note Off, Pitch Bend, Aftertouch, Channel Pressure and CC’s
• Pattern-based; pattern changes occur on bar boundaries so song stays in sync Ed., oh, come on, where’s the fun in that?
• 16 simultaneous tracks, unlimited patterns

There’s also an on-screen keyboard interface (in case you don’t have a MIDI keyboard handy), song and live modes, incoming CC and note recording, and undo/redo for “most” actions.

Having the multichannel capability alone is a must for people with big studios.

Developer Dave Wallin says he’s been working on this for some time. An experienced dev with tools like bleep!BOX and bleep!Synth under his belt for iOS (as well as Additive and Zero Vector for desktop), he’s got a good perspective on the bigger picture.

Since Dave is a user as well as a developer (as is often the case with music tech), I asked him to tell us a bit more about the app, his thoughts on development, and how you might use this.

He even has some good thoughts, in case you’re wondering, about how this compares with and fits in with desktop studios.

CDM: This looks to me to be the most full-featured MIDI sequencer we’ve seen yet, more than just a pattern or step sequencer (though I like those sorts of things, too). That said, what other applications do you like in terms of MIDI control?

Dave: There are a couple other apps that look close in terms of functionality like Step Poly Arp, Polychord and Brainwave Sequencer. The first two are more focused on ‘performance’ imo with Brainwave being the closer. I wanted to provide some of the functionality you’d expect from a desktop sequencing app but redesign the core interface around touch and just make it simple and fun to use. My inspirations for how the piano roll works are 50% monome / [Yamaha] Tenori-on (in terms of a familiar grid and easy one-tap note entry) and 50% iPad paint application (in terms of using gestures to navigate). It also takes some inspiration from [Nintendo] Game Boy tracking apps in that I keep the octave / bar navigation locked to a grid. GMS isn’t locked into 16 step patterns and you can adjust the grid and bar length more freely than some other apps.

Ed.: Of course, at iPad app prices, you can easily pick up all the apps above, and they all have some appeal – yes, I know it’s time for another iPad app round-up. A MIDI-specific one could be fun; I’ll work on it! -PK

How are you using this in your studio – what gear are you using?

Right now I am using it with my [Access] Virus b (desktop [synth]) and a drum machine primarily. It’s great with the Virus since it can do up to 16 parts and I can save these setups and easily recall them later without having to worry about program changes or anything like that. The drum machine saves me from having to waste Virus parts on drums. With just those two pieces of gear and not much else I can make some pretty decent sounding songs. My current work flow is to jam around a bit with Genome and the gear and when I come up with something good, record some loops or tracks it and bring it over to the desktop for additional processing and arrangement. My actual studio doesn’t even have a computer in it at the moment – it’s in a totally seperate room.

At what point do you imagine people would use the iPad sequencer versus, say, the sequencer on their desktop machine?

What I have learned from my own experience and from talking to other people is that the iPad is not a total replacement for the desktop or even for a laptop, unless you’re doing some very minimal stuff. The iPad is more of a convenience and leisure device at this point. I have limited time to relax and if I can spend 30 minutes chilling on the couch, making a tune on my iPad, it’s a win for me. It doesn’t have to be a whole song – it could be some musical ideas, sounds or loops that I end up using later on the desktop or in another app.

With Genome, I think the big wins are portability and ease of use. If I want to jam with a friend, I can throw my iPad in a bag and bring it with me. Sometimes I don’t want to have to deal with all the overhead that comes with a desktop sequencing application like hardware setup, configuring plugin directories, or GUI’s that are packed to the brim with controls. GMS is very easy to plug in and get started within a few seconds.

I’m curious about your reaction to the Open Music App Collaboration Manifesto posted recently. [See Synthtopia coverage.] Any of these issues relevant to your work? What would you like to see in terms of inter-app integration?

I actually just posted some thoughts on this to their Google Group this morning. My response is here:

http://groups.google.com/group/open-music-app-collaboration/browse_thread/thread/75cdd385048fa3ce

Communicating with other apps on the same device is something that I think Genome will be very good for (after some minor updates). Right now, not a lot of apps support Network MIDI, let alone regular CoreMIDI. How well this works will depend on a lot of factors and we’ll just have to wait and see how extensively it’s adopted, both by developers and by users. I could imagine an ecosystem where you can string a bunch of apps together and make songs. The first step right now is to come up with a spec for developers to implement and maybe a way to certify that apps have implemented it properly. If done well, I think there could be some good cross-promotion opportunities to entice developers to do it and it could catch on, much like with Audio Copy did.

Had you evaluated Android at all, or other platforms?

Some work is being done to port libNUI [C++ framework] (which I use for all my apps currently) to Android, however it seems the state of MIDI on Android is nowhere near what it is on iOS. If some well known hardware makers make an accessory or if a good SDK becomes available, I would definitely look into it. GMS is being ported to a Mac desktop app and will be available in the Mac App Store at some point. The desktop version will be almost identical, except for some changes made to adapt it to work with a mouse, instead of multitouch. I think a simple, cheap MIDI sequencer will be attractive to some people and the amount of time to port it is minimal for me.

Ed.: Hopefully more to say about the state of Android MIDI soon; while wired accessories may not be as convenient, wireless MIDI, especially via Bluetooth, holds some promise. I can certainly make a good argument *against* a developer immediately rushing to support Android, but it remains something we’re watching. I also hear these computer things can do MIDI. -PK

US$12.99 in the iTunes App Store, for iPad (iOS 4.2+)

http://www.whitenoiseaudio.com/apps/genome-midi-sequencer/

http://www.whitenoiseaudio.com/

On tablets, on displays, multi-touch control these days is calibrated largely as a software interface – more Starship Enterprise panel than violin. As such, it works well for production tools and exploring compositional ideas. But it falls far short of being an instrument: even on the much-hyped iPad, touch timing and sensitivity is too imprecise, and the absence of tactile feedback and real, kinetic resistance makes you feel like an operator rather than a musician.

Several projects in experimental instrument research seek to change that. But of all of them, the one that has generated the most enthusiasm is Randy Jones’ Soundplane, co-developed with hardware designer Brian Willoughby. CDM shares a conversation today with Randy about his brilliant Aalto synth, and I’m working on a review soon. But wonderful as Aalto is, many of us are still eager to hear more of the Soundplane controller. I chose to wax poetic and optimistic back in December of 2008:
Intimate Control: Multi-Touch, New Models, and What 2009 is Really About

I shouldn’t have put a year on my predictions, though – good things take time. (If I could clearly recall what happened in 2009, maybe my general prediction was correct. The past tends to blur together for me into a continuum in the manner of the modern technologist, a vague assemblage of stuff that happened in the 60s with things that are actually still in the future.)

The good news: Randy continues working on the Soundplane, and Aalto will help.

Continuing our interview, here are the thoughts most relevant to Soundplane — and a glimpse of what it’s looking like as he works on it in the lab.

First, Randy explains his ideas about running a small business, continuing what he had to say in our Aalto story. The basic idea: Aalto’s software will bootstrap Soundplane’s hardware.

I think the whole idea of venture capital is sort of a poisonous one. It’s a little like bands wanting to get signed right away. The first thing you want to focus on is giving up your autonomy, really?

Instead, why not scrape together whatever you can from friends or family and just make something that you can sell right away, however small. I didn’t have enough saved to finish the Soundplane project so halfway through I switched to putting out Aalto as a plan B for paying the rent. Now it’s out and it’s a product I’m proud of that I think reflects where we’re coming from, and it’s going to fund Soundplane development, and it’s letting tons of people know we exist. Just get a foot in the door, do something useful.

He also shares his feelings about patents:

Some people won’t like to hear this, but I applied for a patent on the sensor used in the Soundplane. I know, the patent system is totally broken, and often, if not usually, used in stupid ways. But if there’s one thing I think it is actually good for, it’s to protect small companies like ours that innovate against a bigger entity simply stealing their R&D. This is why it was designed, right? I don’t know if our patent will save the day if such a thing ever happens, but if it does I’d much rather have one than not. It’s a pain to write one but it’s not impossible, you just need a lot of patience. “Patent it Yourself“, Nolo Press, is a good reference.

The patent question raises some additional questions for me – in fact, I’d love to see open source hardware that’s also backed by patent protection, in the same way that the GPL license is made tenable largely through the existence of traditional copyright laws.

But I do tend to agree that in the case of a truly novel technology, which this is, patent protection may be necessary. The question for projects like this will be whether to operate as a conventional, patent-protected design, or whether some sort of open source model with a patent covenant and a copyleft license like GPL will make sense — both preventing exploitation and allowing free experimentation. If there are any IP lawyers lurking around out there, let us know (I have some contacts, too); and definitely let us know if that’s a conversation you’d like us to continue.

In the meantime, the important thing is that Soundplane lives, and using Aalto could help it come to fruition. We’ll absolutely keep you posted.

As proof, though, more shots from the lab:

Also, must-read article from shortly after Jones’ NIME presentation:
Why Soundplane?

The whole article is worth reading, but Jones argues that not only is it likely many people will try to do tactile multi-touch, but it may be necessary. For those of you not all that good at hardware design, you could be just as essential as well to there being any future for these curiosities. The designers need other designers. The hardware needs software creators – lots of them. The software creators need to try lots of ideas. And everybody needs players, composers … users.

It’s all-too-tempting to sit back on the Web and marvel at what everyone else is doing, to take their genius and novelty as an engraved invitation to give up on your own work. “It’s been done before.” “Someone else is already doing this.” It’s probably a topic for a dedicated article, but it’s simply the wrong reaction. “It’s been done before — maybe it’s worth doing. Or doing again. Or doing better. Or doing over and over again.” “Other people are doing this — that means I have someone else to do it with.”

Historically, revolutions aren’t solo pieces. They’re ensembles.

Updated: speaking of work being ensembles, while Randy’s name is most associated with the Soundplane project, credit is due to hardware designer Brian Willoughby, who did the hardware design for the instrument. As he wrote in comments on CDM in 2010, when we covered Roger Linn’s Linnstrument: “For my part, I’ve been deep into the process of designing the analog circuits, DSP hardware and firmware necessary for the product, so it’s nice to poke my head up for a moment and see interest on this site, as well as to hear about other engineers trying new things and inspiring ideas.”

Software can easily enough emulate, down to each knob and patch cord, a vintage synthesizer. But can a genuinely new software synth incorporate the ideas about instrument design beloved in a classic synth like the Buchla modular? How do you balance open-ended sound design with the sorts of limitations that give an instrument personality, limitations that inspire?

And could all of this be meaningful even for someone first discovering synthesis, who may never have seen or heard of a Buchla? (If that’s you, please don’t bail on us just yet!)

It’d be pretty pointless to celebrate the legacy of designers like Don Buchla or Bob Moog if you didn’t think that legacy would be carried on. Randy Jones, of Madrona Labs, is just the kind of person to watch. He’s the creator of the innovative multi-touch, tactile Soundplane hardware, as well as the semi-modular Aalto soft synth, updated this month. Madrona has just released Aalto 1.2. The banner feature: Windows support, meaning you now have full 64-bit (or 32-bit) support on Mac and Windows alike. There’s now really no reason not to try out the instrument, with free demos on each.

What makes Aalto special? It has a unique interface that focuses on sound, and semi-modular design that allows you to produce the sort of sounds a Buchla modular would – without trying slavishly to emulate that hardware in software. It’s something new. As Randy puts it:

The main UI element is a combination oscilloscope + dial that is totally new. It’s inspired by the Max/MSP multislider, as well as the idea “computers are so fast now, and promise so many new possibilities for visualizing sonic systems, why are we still making virtual knobs?”

Randy is reflective as well as inventive, so here are his thoughts on designing the instrument – as well as the most unique approach to copy protection (and cookie-cutter soft synths) I’ve ever heard. (And I do mean heard.)

Aalto, a Tool for Sound Designers; No Two Are Alike

I had two basic ideas: one was to build a softsynth that really encouraged people to make their own sounds, made it fun and easy. Also I wanted to make software that could duplicate some of the Buchla sounds programmed by Morton Subotnik– those wonderful Vactrol / LPG plunks, because I’d never heard that done in software before.

So I picked the Music Easel as an inspiration, not just in sound but in UI, making a small set of modules that would lead to a surprising variety of results.

Limitations are really key, and while Max/MSP and other general environments that try to do everything are very useful, I think musically there’s a lot to be said for making fixed instruments that you learn and don’t mess with. That way you don’t have this situation where everyone ends up with their own instrument. This idea goes for hardware as well as software…

I think good instruments are style-agnostic… hardware as well as software. Making something for a particular genre is just reacting to fashion, which is the enemy of expression.

It’s depressing how popular sample sets are. There are so many possibilities for making your own sounds—recording sounds well, processing and synthesizing them are all easier and cheaper than ever before. If you do electronic music, timbre is a crucial element– how can you buy a sound from someone else? That’s like a poet buying words from someone else.

Aalto is designed to be unhyped. Sit well in a mix. Good music makers have their own compressors, they don’t want a synth to decide to compress itself. Good music makers have their own effects, they don’t want a ton of reverb and delay slathered over every preset.

A precise tool that’s good for learning is also good for using. Knobs are marked with actual time and pitch units. It’s harder to learn synthesis if your knobs all just go from 0-1.

Nothing that affects the sound is hidden. Everything is one click away. Aalto has no menus. I think this makes it a softsynth actually good enough to perform with, not just playing notes but moving through patch space.

It’s easy to go off the rails. Enough rope to hang yourself. A whole world of bad sounds is available at the touch of a dial. Because without them, the surprising good ones wouldn’t be there, only the boring good ones.

The signal scaling to oscillator pitch is a handy knob– one turn of it totally messes up the musical intervals if you want. It’s also easy to get back to a default.

As a composer and performer I’ve been burned time and time again by copy protection, so Aalto doesn’t have any. You can make as many copies as you want, but only run one of them at a time. The registration is embedded in the plugin itself, so there’s no keyfile to worry about. Want to use Aalto at a friend’s studio? Copy it to your zip drive and use it.

Because of the embedded license info being available, I went and did something a little weird: each copy of Aalto makes a slightly different sound. No more than different units of a Prophet-5, but the filter cutoff and oscillator detune and some other parameters are slightly affected by your unique user data. So the sound of the synth is in a way made very personal to the license holder.

Though it’s not so easy to make a good one, it’s a lot easier to make a new softsynth than a new hardware synth. So why are softsynths mostly so boring? Why so little experimentation? I think more interesting things are happening on tablets, with weird audio-visual toys people wouldn’t have made without the interface being available. But the sound quality is not there yet with the tablets. We still have these desktop and laptop computers of amazing power that everyone is using for production… but where is the spirit of innovation that we see in the tablet space?

Being a Small Company

Little companies can do stuff unimaginable a decade ago, thanks to the internet for research and promotion, and the open source community for access to tools. There’s this big area of possibilities with companies like ours that have more of a craftsman than a startup vibe, are not in it to get bought out but to make things of value and beauty over the long haul. Certainly the monome folks are a huge inspiration to me here.

You don’t need a big team to make and promote cutting edge tech anymore, you just need a few really good people. What that means is that companies like ours can innovate and make a product for an audience of a few thousand, something that a Roland or Korg would never bother with, and have a sustainable business model doing that.

Hopefully, this gets the conversation started. Randy’s very open with his ideas, so we’ll be in touch; if you have other questions for him, let us know. And, of course, we should look at Aalto very soon.

http://madronalabs.com/aalto

What can a new digital synth be in 2011? How will it work and sound? And given access to so many excellent tools, how can it stand apart? In place of a press release and some marketing-speak, developer Christopher Penrose (Leisuresonic, Cosmovox) sent us an extended essay explaining his thinking behind his just-released SynthTronica synth for the iPad. Aside from getting into the nitty-gritty technical details, it cuts to the crux of the issue: how to make something personal and new that nonetheless can work for other people, and how that idea can be tailored to a tablet.

As the geeks are glued to the latest iPad announcement, let’s look for a moment beyond platforms. Great ideas in synthesis endure long past platforms. The specific medium is wonderful in that it gives designers, engineers, and musicians the opportunity to realize those ideas, while presenting certain conveniences for developer and user alike.

All of this is worth reading in this case as it sounds like Christopher has a synth that isn’t like everything else out there – not at all. With audio files of your choosing transforming the timbres of synthesized sound in a graphical, spectrally-displayed filter, it looks like it could be a brilliant canvas for producing unusual sounds. That might help it find a place wired into your desktop PC or Mac for production.

In fact, it reminds me of the kind of creative synth we’ve seen all too rarely. The design feels heavily reminiscent of the ground-breaking U+I MetaSynth conceived by Eric Wenger (of Bryce fame). I was always disappointed other software didn’t run with some of those ideas; seeing it with some new twists, the take of a different artist, and touch input on the iPad looks terrific.

Christopher’s notes are quite long, but worth including in their entirety, especially knowing we have other developers in the crowd.

And, oh yeah, we could ramble on about this all day, but I think most of you will get the idea from this video below. It sounds great, and since you can input different audio files to get different filter content, you may be able to escape both overt recognizability and the disposability of many mobile and tablet instruments.

Here’s Christopher:

Notes on SynthTronica’s Development

Back in January of last year, I had been following the Apple tablet rumors with great interest. When the iPad was announced, I was surprised by both its name and its operating system. But it took me only a few hours to decide that I was going to design a synth for the new tablet.

Context

I have developed music software, with varying levels of attention, since 1988, and much of that effort has been spent developing idiosyncratic DSP algorithms for sound exploration. In particular, I focussed on spectral techniques for mating sounds — taking the characteristics of two (and sometimes more) sounds to create a new one. These efforts haven’t made it out of the Max/MSP, Pd, and Unix shell software ecosystems largely due to the limitations of audio plug-in hosts. “Side-chain” processing implementations are obscure and clumsy.

I am also a composer, and, until recently, my software was largely designed to aid my personal music-making. I can honestly say without pretense that my music is idiosyncratic; even Illegal Art, a label which has released one of my albums, regularly characterizes my music as being on the “challenging” end of the spectrum of their musical offerings. I think that SynthTronica has been a good project for me. While developing it, I have been challenged to corral and focus my motley DSP technologies into a broadly-accessible musical instrument. I took a music making process that combines synthesis and sampling, which I used often in my music making in the last decade, and put it at the core of a keyboard synthesizer. Hopefully, I have been able to distill an elegant instrument design from my personal composition practices.

Synth Architecture

SynthTronica is a hybrid instrument; its synth engine combines characteristics of virtual analog synthesis and digital sampling. For most “traditional” synthesizers, sound evolution is controlled by parametric filters — combinations of VCFs and LFOs. Instead, SynthTronica uses time-varying filter data to provide spectral evolution; an instance of such data is called a “formant”. Formants can be created in several ways, through the iPad’s microphone, importing audio files, and capturing performances of SynthTronica’s multitouch filter. While formants are currently played in strict loops, the maximum formant duration is fairly large — just over 60 seconds — providing potential for significant, albeit prerecorded, variation. The benefit of formants lies with their generality. A formant can be made from sources as disparate and varied as Nord percussion, cellos, choirs, braying donkeys, or the chorus of Katy Perry’s latest single. The latter example is an interesting consideration: a formant can reflect much of the rhythmic and sometimes vocal characteristics of its source sound, while effectively obliterating its pitch. Pitch is instead provided by SynthTronica’s synthesizer front-end. When readily-discernible formant sources are used, SynthTronica provides a unique musical space that lies between the boundaries of pure sound synthesis and referential sampling. Formant synthesis is not explicit like sampling; you play through the Katy Perry groove as if it was your avatar. With SynthTronica, a performer needs to make pitch choices for any sound to be heard.

Multitouch Filter

The iPad’s large touch surface was a serious attraction for this project. The idea for the Dynamic Multitouch Filter immediately came to mind. SynthTronica’s multitouch filter serves as a live and expressive counterbalance to the static character of formants by providing fluid gesture-controlled filtering of the synthesizer’s output. Given the spectral architecture of the synth engine, adding up to eleven touch-triggered filters (eleven per voice, technically, though they currently are used synchronously) does not overwhelm the processor resources of the iPad.

Hold Mode

Perhaps an ambient musician’s dream, hold mode simply allows notes to sustain by a single touch. They can be released singly by an additional touch, or released en masse via the “all notes off” button. Hold mode is an excellent counterpart for the multitouch filter, as the filter can easily be the focus of both hands when notes are sustained automatically.

Multiple Keyboards and Scale Patterns

While there are several ways keyboard control could be further developed in SynthTronica, I took advantage of several possibilities made available by the tablet design of the iPad. It is clear that a touchscreen does not offer tactile feedback, so I sought to implement interface dynamics that would make up for this lack in several ways. I choose to support multiple keyboard designs providing two piano style formats and a unique grid-based design.

From my iPhone app Cosmovox, I had a large database of musical scales available. I repurposed these in SynthTronica by providing selectable scale mappings for the keyboards. The piano keyboard has a particular design pattern which is accentuated by the contrast of white and black keys. Being a mallet percussionist in a former life, it was clear to me that this color contrast is optional and the key arrangement itself provides enough information to discriminate notes on a keyboard. Thus musical scales can be represented on a keyboard by changing this color contrast pattern. I often desire to escape my habits when creating music, and altering the keyboard scale pattern can be revealing for me. Further, the scale pattern facility allows for the use of a more radical keyboard design: the grid keyboard. The grid keyboard alters note relationships in interesting ways. The keyboard is compact, allowing one hand to access a two-octave range. Large intervals are no longer as physically distant from a given pitch. The keyboard can be bewildering to play if you play by note (which is a positive feature for me particularly as the keyboard is optional), and can reveal fascinating characteristics of scale architecture.

Design

I went with a modernist design aesthetic as I believe that SynthTronica does not have any appropriate analogs in gear. I find that creating interfaces for software that imitate gear introduces problematic usability issues. While I can understand the desire to have every useful performance control accessible on a single screen, there are practical limits to the number of interface elements that can coexist and still remain effective. Virtual knobs use less screen real estate, make a reference to audio gear, but are more difficult to use than sliders. I chose a slider-only interface using color and orientation for contrast. SynthTronica’s multi-screen design may reduce the accessibility of parameters during performance, but I think the architecture of the synth favors pre-performance sound design and emphasizes use of the Dynamic Multitouch Filter for expressive control in live performance.

Christopher also includes some frank thoughts on limitations of the synth for the time being, including some of his concerns about third-party audio interface support generally. This is beyond my area of expertise, so I’ll leave others to reach what conclusions they will – and I suspect we’ll hear some other developer views.

Limitations

While the preset architecture is robust from a database perspective, it can be frustrating for performance in its current state. The design of the reverb processor is one of the culprits. If reverberation time is different between two presets, changing from one to the other while the synth is sounding can produce awkward glitches. It is possible to ignore reverberation settings from presets by adjusting a SynthTronica parameter in the iPad’s settings application. I would like to improve preset change behavior in a future update.

Some goodies that the electronic music literati would desire — MIDI, audio interface support — have yet to be developed. MIDI is actually very high on the list now, as Apple has provided SDK support [Core MIDI] and I have purchased two Akai LPK25s and an Emu XMidi 1×1 for testing. OSC support is minimal at the moment: there are no supported in-bound messages yet, but a few outbound messages are implemented. Full class-compliant audio interface support will not be added until SynthTronica migrates to an iOS 4.x-only architecture, and even then there may be a performance reduction for many interfaces. A rant could be placed here which would be directed at audio interface manufacturers.

What I will say is that SynthTronica is less flexible with respect to audio buffer sizes as it is a spectral synthesizer; it uses power-of-two FFTs. But this is not unheard of in the least for audio processing; there probably isn’t an MP3 player that does not use them. To support audio buffer sizes that are not powers of two would cause a significant reduction in performance for SynthTronica (namely, a 50% reduction in usable polyphony due to CPU spikes). While there may be an audio interface that works out of the box with SynthTronica, I can’t name one at the moment. The class-compliant audio interfaces I have tested refuse to provide a power-of-two buffer when requested. While I am sure their engineers can come up with an excuse, they really should understand that power of two constraints are ubiquitous in computing, particularly for digital audio signal processing; it is bizarre that their hardware forces applications to perform additional buffering to support powers of two.

Anyway, let’s wrap up not with words, but with…

Sound Samples

PlanetMaster by Leisuresonic

HypnoticCaressing by Leisuresonic

PariahEmoting by Leisuresonic

PulseTherapy by Leisuresonic

More

Check out the sites for more. It was a bit unorthodox to include all these thoughts, but I enjoyed reading it and it made me want to spend some time with the synth. Let us know what you think.

http://synthtronica.com/
http://leisuresonic.com/

on the iTunes App Store

Ever wish your music software could do something your way, something it can’t do now? Wish you could just get in there and change it yourself?

That’s some of the ambition of Renoise 2.6, the multi-platform music creation tool. By opening up the entire music tracker to scripting, users can create custom functionality and control surface. But scripting – while it sounds like the domain of hard-core geeks – doesn’t have to be daunting. That’s important, as presumably you want to spend some time making music. Scripting should save you time and let you express ideas more directly, not act as an impediment. So, the design of the Duplex feature in Renoise does work to make this customization accessible.

Renoise 2.6 has just gone gold master, meaning you can add it to your stable music setup. New in this release:

• Lua scripting. Customize the app using an elegant, clean, friendly language.
• OSC, MIDI support. Integrated control with Duplex (MIDI/OSC), native Open Sound Control support.
• Extensive hardware support. Maybe you don’t want to write a line of code, ever. You can let someone else do it for you, and reap the rewards. Already, Renoise has native, fully-integrated support for the AlphaTrack, BCF-2000, BCR-2000, KONTROL49, FaderPort, microKONTROL, nanoKONTROL, Launchpad, Remote SL-MKII, Nocturn, Monome, Ohm64, iPad via TouchOSC… all thanks to community support for the new scripting engine.
• Sample autoseek. Absolutely essential to making audio behave in the way it does in linear arrangement tools, the sample will play back from the position in the timeline, rather than from the beginning each time you hit play. (Seems obvious, but it’s part of making Renoise bridge tracker-style apps and more conventional, linear ones.)
• Better performance, compatibility. Tweaked performance on Linux and Mac, expanded file format compatibility, plus 64-bit Linux, DSSI Linux support. Renoise is a reason to run Linux, and Linux a reason to run Renoise, if you hadn’t guessed that yet. No, seriously, you’ll enjoy it. (I always feel like it’s telling someone to go vegan. Linux can actually be fun. And you still get to eat bacon.)

The release date seems the perfect time to really explain what Duplex is about, and what it means to you. First, it’s best to see it in action in this Duplex video. What you see is fully integrated hardware and software, but in a way that doesn’t necessarily require specific hardware. (There’s no “Renoise” logo on the controller – and you could substitute something very different and get the same impact.)

More information on the Renoise forum from the video’s creator, Danoise: Duplex – Playing With Loops

Basically, I’m arranging a small song on-the-fly, using a Launchpad + monome. Since the song was basically written using the StepSequencer, the vertical resolution of each pattern is just 8 lines. I then use the new loop feature in the Matrix to “pair” patterns into longer sequences.

This is just one possible workflow among many, but it’s one that’s I’ve found to be immensely rewarding when you’re sketching a tune out.

Bjørn Nesby, Duplex’s lead developer, explains his creation to CDM:

Duplex is aimed at both people who are willing to create their own scripts, and those who just want a nice way to interact with their music using Renoise. Many of the scripts (called ‘applications’ in Duplex) are pretty generic in nature, and will simply take control of a specific part of Renoise, like the Mixer or Pattern Matrix. This is something everybody can use, so this is where I focused my efforts to begin with. More exotic applications are also planned, but we needed to get the fundamentals in place first.

A thing that was clear from the beginning was, that the whole setup and configuration process needed to be as simple as possible. I think we succeeded in that, as my personal copy of Renoise will automatically launch the applications I need when the program starts, on three separate controllers. And I’ve heard from many people that they love this aspect of Duplex, as it reduces a potentially tedious startup process to an absolute minimum. Of course you can have an initial device setup process that you need to go through (like selecting the input and output ports for your device, which might vary from system to system), but in most cases you’d only need to go through this once, after which the device is ready to use.

And I believe this is not just about ‘convenience’, because sometimes you need to be absolutely focused on the music and not the order of which you launch various programs – especially true when you bring your music to the stage.

However, I have to point out that the configuration process is not perfect yet. There’s still room for improvement when customizing application mappings – this is currently done by editing some of the accompanying configuration files by hand, and although that might sound scary, it’s actually a pretty straightforward thing to do (and if not, the Renoise forum is there to help people out). Also, finetuning a setup like this is hardly part of the music-making process itself, so I hope it’s something people can live with for a little while longer.

From a developer point of view, the Duplex framework might be technically interesting as it attempts to follow the ‘write once, run everywhere’ model, as known from the mobile computing world, but instead applied to musical gear. For example, the Mixer application is able to run on all devices, from the Novation Nocturn to the monome128. Physically speaking, those are two very different devices, but everything in the Duplex API is abstracted to the point where a standard user-interface element like a slider can be a rotary dial (Nocturn), or an array of buttons (monome). In the application code, you simply create a slider, and base your logic around that. The framework will do all the dirty work of translating that into *actual* controls. This is possible because everything in Duplex is based around a descriptive XML file, the control-map. Unlike a traditional MIDI implementation chart, the control-map will not only describe the parameters and their ranges, but rather the complete physical layout of the device. Once a proper description has been made (and they are not hard to make, several of Duplex’ control-maps are user-contributed) you can launch an application on e.g. the monome that creates virtual sliders from individual buttons, because each button “knows” where it’s located in a X/Y coordinate space.

I’ve also tried to keep the syntax as familiar as possible. Many people who’ve done a bit of actionscript will probably recognize many of the concepts in this framework, hopefully making the whole experience a little less daunting for budding scripters.

One unique aspect of Duplex: the virtual control surface. When Duplex is installed, you can try out all the various supported devices, even if you don’t own them. Again, it’s the control-map structure that makes this possible, as you can define things like button size, color etc. Of course, this is not the same as the real thing (try hitting two buttons simultaneously using a mouse?), but it’s still interesting to play with, a huge advantage for developers as you can design a control-map that device owners can then try out and test, and makes for self-documenting applications, as you can assign tool-tips to the control surface that display exactly what each button does.

More information:
What’s New in Renoise 2.6 – Renoise Geek Edition.

Renoise Lua Scripting

2.6 Forum Discussion

And, of course, you can discuss Renoise and other trackers on our own Noisepages community. Specifically, we’re looking at how to use trackers in live performance.

Trackers for Live Performance @ Noisepages

Music software development includes some of the most sophisticated, expressive software out there. But it has long faced serious challenges in sales – audio software still appeals, generally, to a small slice of people, made smaller by factors ranging from piracy to the sheer complexity of available audio tools. As computing’s distribution model for software shifts, audio developers are undoubtedly watching.

Love it or hate it, what’s unique about Apple’s App Store for iOS is that it’s a one-stop shop for everything. With App Store fever spreading – new stores for mobile and desktop are either available or planned from the likes of Apple, Intel, Microsoft, and Linux vendor Canonical – we’re likely to see a new kind of store model. On desktops, Android devices, and others, multiple stores will compete with one another in overlapping arenas. They’ll do it without lock-in, too – unlike on Apple’s stores for iOS, you’ll have a choice of where to get your software.

Last week, of course, that list expanded to include Apple’s Mac App Store, coming to Snow Leopard and then the just-announced Lion.

Music creation and pro audio apps may be a specific niche, but creators of everything from plug-ins to audio software are at least interested. Little wonder: desktop music making software has always faced an uphill climb, but recently, iPhone creations have become breakout hits.

Just don’t get too excited yet. An early look at Apple’s guidelines for the store suggest restrictions will rule out a great deal of current Mac software, particularly audio software that relies on plug-in models. I’ve asked some independent developers to comment on what the store means to them, and take a look at some of those restrictions.

Several developers responded to my questions. Now, a disclaimer: clearly, the Mac App Store is not aimed at creators of strange synthesizers and effects. Nor is it possible to represent the full gamut of developers making software for musicians. TUAW has a nice round-up of more typical Mac developers, who are, unsurprisingly, more upbeat. I likewise expect that anyone who now has some success on the iOS platform – vendors like IK Multimedia or Smule — will be optimistic about the Mac App Store.

So, instead, consider this as a sampling of developers for whom the App Store may not actually change that much. I was, frankly, surprised to see plug-in creators and pro audio users assuming that the Mac App Store would be a natural marketplace for the software they care about. Early evidence is that it isn’t. But with app stores spreading across devices, the responses from developers provide some insight into longer-range challenges that transcend even Apple’s latest offering.

Gallery: sample applications and stores.

Could the Apple App Store be a viable option for creative music developers?

Angus Hewlett, FXpansion:

I’d say it’s a viable option for entry-level and somewhat novelty apps, and as a launch-assistance platform for brand new developers (allowing unknowns with no established reputation to get started in the world of ecommerce). It’s not of great appeal to FXpansion – we’ve been around long enough that I’d hope commercial trust isn’t a massive barrier to people buying from our web-store – but as a get-yourself-started platform, it is not completely without merit.

Of course, because these app stores are usually tied to a specific platform, for those developers who are on more than one platform, it does just add additional complexity, cost, and hassle. Admittedly it improves convenience for end users a certain amount (a good thing in my book), but the effort/overhead of getting out your credit card and typing in the number looks completely different for a $0.99 game you’re going to play for 20 minutes, compared to a $249 plug-in that you’ll spend hours/days just learning and (we hope) use several times a week for years to come.

Christopher Randall, Audio Damage:

The guidelines preclude selling plug-ins, so that rules out the segment of the industry I’m most familiar with. This will probably change, but my general feeling is that people that make things like Numerology will be well served, but for the majority of our business, our needs are a bit too particular to really benefit from something as broadly-focused as the App Store. And there’s no real financial incentive on Apple’s part to cater specifically to us, because we’re such a small segment of the overall software market.

David Viens, Plogue Art et Technologie:

As much as we like having our code base tested on as much compilers and platforms possible for correctness and efficiency, constant platform changes are quite boring, and usually dont spark any innovative ideas from us. Innovation not only is what drives us in the morning, but it’s also what users want, hopefully more than the typical will-it-run-on-my-toaster? kind. Also innovation is highly regarded by various tax break programs in many countries Also app stores make it impossible for us to do quick fixes, we could be committing code to Nintendo ROM carts that it couldn’t be different. So there is a need to raise QA and testing budget by a very significant amount, before release….

For standalone software, the kinds of things we’ve seen for iOS seem a likely candidate – particularly general-consumption audio “toys” (in the sense of stuff anyone can open up and use to make sound)?

Chris Randall:

That was my my general thinking. I was pondering it at length last night, and the smaller single-use app seems more likely to benefit from it, assuming it is a parallel environment to the existing App Store, with the same sort of customers. The chief difference between this App Store and the iOS one is that this isn’t the only option for purchasing software for your Mac. It has to compete with other channels, which is an important distinction, especially if most of its offerings are simple “casual” apps.

What about the app store landscape in general, as other players get into the business of doing their own stores? How does Apple fit in?

Gavin Burke, Future Audio Workshop:

The main issue is that the app store model just one piece of a bigger jigsaw and is tied in closely to the hardware, software frameworks and what this means to an independent developer and his/her ability to compete on a level playing field with established brands.

App store success is just one part in a bigger picture. The other players need to get the various parts right and not just create an app store and think it will work.

A major part is the price and ease of purchase. It’s easier to buy the software for $1.00 with one-click purchase than look on rapidshare.com for a crack. So price, ease of purchase, and last but not least, [making it] difficult to get the cracked version. Looking at it, it may only work if there is a single distribution channel and not multiple ones ( including rapidshare as a channel )

One flaw, though, in the Apple App Store is the charts. At the moment it is based on sales volume. Allowing people to view by highest user rating, etc., might help level things. Also, Apple can act as king maker with their ability to dish out the free advert slots on the device.

Angus Hewlett:

It’s a new channel that will no doubt get a lot of coverage – a few developers with the right products and first mover advantage will make some fast bucks for sure. After that, I don’t know. I suspect phones (and consoles – myself, I spend way more on XBox Live Arcade than on the iPhone or Android stores) are a better and more natural platform for cheap, one-shot novelty apps than desktops/laptops, simply because of how & where they fit in to peoples’ lives, but I’m ready to be proven wrong on that.

Are you concerned about Apple’s 30% cut of revenue?

Chris Randall:

Not at all. The app store runs on volume; that is its main attraction from a commerce standpoint. The trick is to take advantage of that potential volume, and the way to do that is through lowest-common-denomenator (e.g. “I Am T-Pain”) products.

Angus Hewlett:

Yes. It’s a lot more than the original generation of “app stores” (shareware ecommerce middleman sites like ShareIt, DigitalRiver, Kagi, NorthStar etc.) typically charged. We used to sell through ShareIt back in the day, they took about 10%, but once your turnover hits $10-15k a month,
it’s more economical to have a proper merchant account based shopping cart system (the hassle that entails costs a few hundred dollars a month in terms of overheads, paperwork, other bank-related BS, but it brings the average cost per transaction down to 3-5%). Also, at 10-15k a month turnover, your brand is probably well enough known that potential customers are likely to trust you somewhat as an online vendor.

Having said that – the terms-and-conditions aspect of being in an app store, especially when the operator is particular, capricious, anally retentive or all three at once (naming no names), is far more toxic than the 30% cut. Losing a predictable amount of money per sale is one thing, but failing to sell a single copy of your app – after you’ve spent months and $thousands developing it – because the store owner rejected it for reasons outside of your control is quite another.

David Viens:

30% is ridiculous. What enrages me is that users seem to think it’s normal and much less than ‘retail’ .. wuht whut??? We have never done retail ever and been using Share-It (which costs us less than 10%) for 6 years. That’s the price of a payment processor.

Sure, it doesn’t give you ‘visibility’ but what is that visibility on the 15th page of music software selection in a store? Can’t we just be as creative with our viral marketing, social network tricks as we are with the software itself?

Bandwidth price on Amazon S3 is microscopic (10 cents a GB), so not an issue, even with 100-megabyte demo downloads.

Share-it don’t care about the content, they never put their noses in our practices, suggest guidelines, or anything.

People are just getting to enjoy their new-found freedom with independent online music and fair-trade and local grown foods, however. they will let the inverse happen to software?
Will we see the movement to Fair-Trade software in 15 years?

Do you think it’s a model that could work, from a business perspective?

Angus Hewlett:

It’s a new channel, a few developers with the right products and first mover advantage will make some fast bucks. After that, I don’t know.

How well versed are you on acoustic physics in relation to loudspeakers, impedance etc.? There are some interesting parallels here with app stores – basically they are a good platform for allowing very small developers to cast a very wide net. Those of us who have a more specialist, focused audience can probably build trust with our audience via more efficient, focused channel….

It appears that plug-ins are ruled out by several of the guidelines issued by Apple. Care to comment?

Gavin Burke:

I can’t see the app store concept working for plug-ins; it’s pretty much already there with the downloads page on the Apple site.

If the app store is the only channel to purchase applications for a device that cracked software is not easily available for, then yes, it works. Otherwise, not so sure. We already have app stores for music software, like Don’t Crac[k], etc., with somewhat limited success. Also for complex niche software, it’s hard to beat the personal connections distributors have with stores and in turn with their customers. We find this especially true for Japan.

Artistic freedom and censorship

Artist and developer Kassen Oud offered some compelling thoughts on Apple’s developer “guidelines” and rules via Facebook. To him, the restrictions on what goes in the store conflict with making software art. I think it’s a reasonable and challenging point to make – just as Apple has the right to conduct their store in the way you wish, developers and artists presumably have just as much right to opt out.

To me, the big appeal to developing software for music is the chance to do something unique and individual. External limitations (like arbitrary moral guidelines or limits on the language to be used) conflict with that, to me. As the process used is important to me I need to be able to express myself about that publicly as well. This rules out Apple’s app store. With regard to code/ application distribution those are more important factors to me than the need to create applications for Apple’s app store on a Apple computer though that in and of itself would also be a prohibitive factor to me.

I don’t mean to imply that software on Apple’s platform would inherently be less “unique” or “individual”; I certainly do not wish to take anything away from my friends whose creative process wasn’t (apparently) affected by these factors and who did create very interesting works released on it, taking nothing away from their FOSS work.

Apple Developer Guidelines – Plug-ins Need Not Apply

Apple’s draft review guidelines for the Mac App Store have been widely posted, including full text. Here are the excerpts most relevant to the above discussion.

Plug-ins will almost certainly be rejected.

2.15
Apps must be self-contained, single application installation bundles, and cannot install code or resources in shared locations

(Plug-ins, by definition, install to shared Library locations, as per Apple’s own guidelines.)

Interestingly, though, hosts appear to be okay, just not the plug-ins themselves:

7.1
Apps that unlock or enable additional features or functionality with mechanisms other than the App Store will be rejected, except in cases where the application hosts plug-ins or extensions

Demos aren’t allowed.

2.6
Apps that are “beta”, “demo”, “trial”, or “test” versions will be rejected

Other guidelines are worth watching.

Various other guidelines provide fairly restrictive policies that developers will have to balance against their business interests. These aren’t unprecedented – see the strict review policies of venues like the Steam store for games. But those stores have seen their own share of developer complaints, and they’re specific to an audience (like gamers); here, it may be tougher for niche developers to justify. (That’s, at least, the feedback I’ve been hearing from music developers. For mainstream developers, the equation can be different.)

2.18
Apps that install kexts will be rejected

2.19
Apps that require license keys or implement their own copy protection will be rejected

2.20
Apps that present a license screen at launch will be rejected

2.21
Apps may not use update mechanisms outside of the App Store

2.7
Apps that duplicate apps already in the App Store may be rejected, particularly if there are many of them

Of course, some of the challenges of audio software are … unique. How many pro audio applications would meet the following guidelines?

6.3
Apps that do not use system provided items, such as buttons and icons, correctly and as described in the Apple Macintosh Human Interface Guidelines will be rejected

6.4
Apple and our customers place a high value on simple, refined, creative, well thought through interfaces. They take more work but are worth it. Apple sets a high bar. If your user interface is complex or less than very good it may be rejected

Comments from other developers are welcome. We’ll keep you posted.

Remember the music industry? We used to talk about radio play and record deals. Now, we’re talking developers, APIs, and analytics. Of course, the test, now as then, is whether there’s actually substance for music listeners and artists. On Friday, we looked at Apple’s Ping and how, via TuneCore, artists who aren’t Lady Gaga can get their own pages. We also saw some vigorous discussion of TuneCore, which helps you get your music into “big bucket” sites like Amazon and iTunes, and SoundCloud, who together offer integrated sharing and distribution.

The Echo Nest is an unusual animal in this landscape, an “intelligent platform” for other tools built on machine listening. Via algorithms that analyze music and try to absorb human knowledge about music, Echo Nest powers apps that do everything from matching beats to workouts to tracking data for artists and labels. They even apply natural language algorithms to try to absorb text about music, meaning a robot may be … listening to … everything I say. Okay, moving on… (This leads to some interesting sci fi scenarios… No! Robot! Obey your master! You will NOT harm Katy Perry!) Check out how it works.

The latest news from Echo Nest: they’re partnering with online store 7Digital to connect all that data with some actual music sales opportunities. The result: a DIY online music store platform. Developers can create their own applications and music sales websites, drawing on all of the intelligent data on music from Echo Nest – from machine-analyzed news feeds to musical attributes of individual tracks – with the 7Digital store. 7Digital is now up to 10 million songs, with art, previews, and sales. And whereas a lot of deals (like Ping) are closed, in this case the “product” is actually a toolkit that relies on developers to do interesting things. Whether or not this particular idea gets traction, that’s an interesting shift in models.

If you’re skeptical about those machine algorithms, you’ll be gratified to know even Echo Nest’s own Brian Whitman conceded around the Ping launch that he has “a strong aversion to music recommenders and music similarity services.” And Whitman knows his stuff – he’s an MIT Media Lab PhD and co-founder of the company. But there’s no saying this has to be limited to recommendations – as those jogging apps that analyze tempo illustrate.

I spoke to Echo Nest to find out more about the new offering, and to see what they think about other trends, like Apple’s much-hyped – then much-criticized – Ping.

Jim Lucchese of Echo Nest and developer Paul Lamere respond.

CDM: How do the two APIs fit together? That is, given that The Echo Nest and 7digital each have their own open APIs to begin with, what’s unique here about the joint offering?

Paul: We’ve incorporated 7Digital Artist and song IDs into our Rosetta stone system. [Ed. As the name implies, that's a mechanism for translating all the different ID numbers used by music databases for tracking songs, since there's no universal numbering system for music.]

This means that developers can use 7Digital Artist and Song IDs with our APIs, and can instruct our APIs to return 7Digital IDs.

In addition, all Echo Nest results can be limited to the 7Digital ID space. This is useful for tasks such as search and artist similarity. For instance, we can limit the results of an artist similarity query to include just artists that are in the 7Digital catalog.

We have also applied the Echo Nest analyzer to each of the 10 million or so 7Digital tracks. This means that we have a very detailed understanding of what every song in the 7Digital catalog sounds like. We know the tempo, mode, key, time signature. We have a detailed understanding of the loudness profile of the song and of the rhythm structure of the song, we know where all the bars, beats and tatums are. We have a detailed understanding of the harmonic and timbral content of each of the songs. With the Echo Nest / 7Digital partnership, developers can get access to this detailed analysis for any of the 7Digital tracks and use this analysis for all sorts of apps.

The Analysis data allows developers to create interesting playlists using the new Echo Nest Playlist API based on the 7Digital catalog. For instance, a developer could create a playlist of songs by artists similar to Lady Gaga with a tempo between 120 and 135 BPMs. You can read more about the playlisting API on my blog [Music Machinery]:

Is that a million songs in your pocket, or are you just glad to see me?
Echo Nest Playlist API Methods

Is this something that might become relevant to other stores, too — provided, of course, they were as open as 7digital is with their data?

Jim: We expect so, though we’ve found that 7Digital is definitely the most forward-thinking around open API access and developer offerings.

You mentioned that monetization for developers could be a big draw. How does that work — is it simply a 7digital affiliate program, essentially?

Jim: Yes. It’s like an uber affiliate network of developers. Sell tracks through your app and get a cut of each sale.

What sorts of applications do you imagine this might encourage?

Paul: I see a wide range of possibilities here:

• Music discovery apps – given the deep data that Echo Nest has about music coupled with the content that 7Digital can provide will make it much easier for developers to create new music discovery and exploration apps – think of a next generation of Ishkur’s guide to electronic music
  [4], or musicovery [mood-based radio search]
• Music listening apps – Applications that help people listen to music – for instance I can imagine a pandora-style app that gives the user more control over the listening experience – ‘more music like this but with more energy’ or ‘give me music by hair metal bands from the 70s that are from the UK’
• Music Interaction apps – since developers have access to a detailed analysis of each track, it would be possible for developers to create interesting music visualizers that respond to the music
• Apps that we can’t even think of – never underestimate the creativity of a developer – someone will create something that we can’t think of around music that we can’t even imagine now, but will become a big part of our music experience.

Artists are becoming increasingly excited about engaging these platforms themselves, particularly as many of them get more savvy about development and use independent distribution to get their work out there. Is there a place for music makers to do something with this kind of technology?

Paul: That’s a big question – some areas where platforms like 7Digital/Echo Nest platform will be relevant to artists are:

• Interactive media – the recent Arcade Fire interactive video ‘The Wilderness Downtown‘ is an excellent example of the type of interactive app that artists will be able to create on top of platforms like 7Digital/Echo Nest/
• Analytics about fans – artists are spending more time figuring out how to market their music. APIs around data about who is listening to, going to shows, or writing about an artist will be very useful for artists

What about the end user (that is, the listener) – what will this allow a music lover to do that they couldn’t before?

Paul: As the music app space developers, we will see better applications for music discovery and exploration. A music lover will be able to find more interesting music that they will like.

Music listening apps can be greatly improved with access to all of the contextual information about the artist and the song, along with recommendations for other artists And you can see context-dependent playlisting – a user could create a ‘jogging’ playlist that has songs at just 135 BPM, or a high school reunion playlist of pop songs from 1985.

One of the things that struck me about Apple’s Ping announcement was that – while we’re waiting on information about any potential API – the general sense was of something pretty walled-off, specific to just their store, in the client software, and not connected to the Web and across platforms. Do you see a contrast in philosophy here?

Also, the initial reaction of the press surprised me; the first response was that Apple’s installed user base would just wipe everyone else out. I disagree, but how do you see open offerings like this fitting into (or providing alternatives to, or both) that Apple ecosystem?

Paul: There’s no question that Apple’s installed base makes their foray into social music a significant event. Millions of people who would never think of visiting a social music site like Last.fm will now become part of a social music network. That will be good. Music is very social, and one of the best ways to discovery new music is through one’s friends. However, social music systems can have some problems: Feedback loops, decreasing diversity in listening, susceptibility to hacking, schilling and spamming, and the cold start problem (where new artists have no way to break into the listening rotation) are all issues that plague social recommender systems.

Systems that can draw on multiple sources of data – social data, plus other data like curated reviews, blog posts and the content of the music itself can provide a way to avoid the problems inherent in social-only systems. I think that music discovery applications that are built on broad sets of data (like we are building here at the Echo Nest) will ultimately do a better job in helping people finding new music.

The most interesting possibility I see with the opening up of all the different music-oriented APIs is the emergence of a thriving music application developer community – (like we are starting to see coming out of the Music Hack Days). This developer community is ultra creative and they love music. They love to listen to it, and they love to create it. I am convinced that these developers are going to have as much of an impact on the future of music as the iPod did. These developers will be creating the applications that will help all of us to create, to discover, to interact with and to listen to music.

The iPad may be the catalyst, but look beyond the platform, and you’ll see a reinvigorated examination of how to think about musical interfaces. If you’re looking to relax and make some musical noises on your iPad, check out the videos above. It isn’t actually necessary to fully understand the design work behind the interface to play and make music. But if you are interested in reflecting on the way design issues impact musical expression, and you’re a bit of a music theory nerd, read on.

Using an array of rectangles arranged in a harmonically useful way, and color coding for pitch, SoundPrism is a glimpse of a more graphical future for music software design. (Nor is this necessarily limited to the iPad in the long term – in addition to Windows 7, Ubuntu 10.10 is getting official multitouch support, which I think both validates Apple’s work and suggests we’ll see more platforms for this kind of interface.)

And, bonus, it all demonstrates why arranging pitch by the Circle of Thirds can be ideal. I got a chance to talk to the developers of SoundPrism about the thinking behind the software.

We will get a mention of Jean-Philippe Rameau. I just hope it doesn’t start a debate between Rameau and Lully fanboys again.

Sebastian Dittmann, CEO of developer Audanika, talks to us about the software.

CDM: What’s your background as a developer / musician?

Sebastian: I wouldn’t call myself developer although I’ve been working in IT and digital media creation/marketing before. I come from a very-music oriented family – both parents opera singers, so I had to learn to play the piano, but can’t play it very well.

I met Gabriel Gatzsche – the inventor of the technology SoundPrism is based upon – during a colloqium for music in digital games at the Fraunhofer Institue for Digital Media Technology when he was showcasing an earlier prototype of it back then. His demo of 5 minutes taught me more about harmonic theory than years of piano lessons before. That was 10 months ago and he is now the CTO of our company that we decided to found.

Can you talk about what inspired the pitch layout on SoundPrism?

The idea for SoundPrism is quite old actually. About 22 years ago Gabriel’s father found that organizing tones of keys in a circle of thirds makes it very easy to understand basic harmonic theory because the visualization ‘just makes sense’. Back then he taught it to his two sons, Gabriel and David, and from then on they passed all music school theory exams with flying colors. He tried to convince their teachers that this method was far better for teaching children but they didn’t listen to him at all.

David studied music to be a guitar/music teacher, writing his final thesis about the didactic concept of SoundPrism. Gabriel studied media technology and wrote his dissertation about it and came up with software that made it possible to not just use the theory to explain music but also to play music with an interface that is set up like the theory.

During his dissertation he found out that lots of other scientists came up with similar concepts. He quoted them in his thank you note to the team when we submitted SoundPrism to Apple: Jean-Philippe Rameau, Leonard Euler, Moritz Wilhelm Drobisch, Hugo Riemann, Moritz Hauptmann, Fred Lehrdahl, Carol Krumhansl and Werner Pöhlert just to name a few.

The reason we are able to actually use these concepts now is that we finally have interfaces that can change visually and aren’t static. I see SoundPrism as knowledge poured into a dynamic interface that enables users to just use that knowledge without having to acquire it first.

Could you describe – for people comfortable with music theory – how it’s laid out and why?

SoundPrism is based on Hauptman’s array of thirds in its vertical dimension and horizontally on a pitch class/pitch height model by Moritz Wilhelm Drobisch. We found the combination of these concepts allows musical interfaces which enable the player to control both harmony and melody at the same time in an optimal way.

This is just the very first version in the first configuration on the iPad though. There have been prototypes using Space Navigator technology, a WiiMote, Jazzmutant Lemur and also multi user augmented reality installations with camera based motion tracking.

You’d have to talk with Gabriel about this since there’s a lot more to it. The interface right now is tailored for tablets like the iPad. We’ve evaluated other representations with field tests and the current form lets users express themselves freely while at the same time guiding them gently to ‘do the right thing’ from a harmonic standpoint.

Gabriel alone has spent 5 years of his dissertation working on the concepts and there’s lots of music psychology and other really complicated concepts involved in it that I’m not the right person to explain.

Ed.: I wanted to go ahead and get this out there, but if you’re interested, readers, I’m happy to pursue.

Have you considered providing control data to other applications (i.e., routing over WiFi, Bluetooth, etc. to a computer for people wanting to use this as a controller)?

Yes.

Obviously, at this point, you’re just waiting on Apple approval, but any plans generally for the future?

SoundPrism is really just the beginning. We think we roughly know what musicians want because our team consists of musicians. At the same time we already have other prototypes and ideas that do very neat stuff beyond the scope of the current implementation of SoundPrism for the iPad.

But we hope for lots of constructive feedback from all over the world so we can add the features people want that we haven’t though of yet. I have to say I’m a bit nervous because I didn’t think SoundPrism 1.0 would get that much attention already before we’re live.

One of our dreams is to establish the SoundPrism technology as a widely used tool. Not just for musicians but also as a door to the world of creating music for people who don’t think they can play an instrument.

What will pricing be?

SoundPrism will be at around 4 US$ initially. That might change later on and we’re also planning to include an InApp store for people to add more features, sounds and other really neat stuff.

Now just waiting on approval…

Follow CDM on Twitter and I’ll post when the app is up.

In the meantime, here’s one more video. I love the way this sounds, here paired with a Korg KAOSS Pad KP3 in what appears to be a really practical live music rig:

More at the developer site:
http://audanika.com/

Thanks to Chris Mousdale for connecting us with the developers!