Pictured: Loopseque, in final form (top) and sketched on paper (bottom). Images courtesy the developers; visit them on Flickr.

Saturday afternoon in Toronto, I’m giving a talk to the North by Northeast festival on music software and tablets. I’ll explain a bit about what tablets are about, and some of the software that’s out there on the landscape (principally, of course, on the iPad). But I hope to emphasize a deeper issue: how you design software for the tablet, and what’s unique about this convergence of form factor and touch interface. I mean this generically for a reason: on CDM, we covered some of these ideas before even the announcement of the iPhone, and I was an early (and skeptical, I might add) reviewer of the JazzMutant Lemur.

Even looking beyond that, I hope to talk a bit about how representing music graphically has been an essential part of human practice, not only beyond the iPad, but beyond even the current notational system as derived from the Western church. Talk about early tablets: the first known music notation appeared in ancient stone Greek and Byzantine tablets. (On weight and thinness, I don’t think they compete with the iPad.)

That sounds lofty, especially for a potentially-hungover crowd of musicians and designers on a Saturday, so here’s the executive summary: you don’t have to make a bunch of fake knobs.

I’m really mostly curious to start a conversation about design; ideally, I’ll get some designers showing up here in Toronto, but it’s time to make that conversation happen on the Web, too.

With that in mind, I’m curious:

What software designs – iPad or otherwise – have you seen that have most inspired you, in terms of the way the interface was designed?

Fair game: sound toys, music notation (really), art pieces, games, control surfaces … whatever you like.

I’ll post notes from my presentation by early next week, because I’ll probably be assembling it at the last minute it’s already totally done and perfect and rehearsed and I just wouldn’t want to spoil it.

Pictured: Loopseque; previously on CDM

Also, because I’m a huge fanboy of circles in general (as readers of this site know), I love this image and blog post from Loopseque. They didn’t exactly invent the idea of visualizing loops as circles, but let’s join this revolution.
Another step in the evolution of music interface [Loopseque Blog]

Honestly, if tablets are nothing other than an excuse to ask these questions again, all the better – and there’s no reason not to then apply what you’ve learned to computers, embedded hardware, analog hardware, paper notation – anything.

If anyone would like to start a circles versus rectangles fanboy platform war, troll away! I’ll start:

stupd circle &*(&$s you losers got not edges. serious muzos have right angles. go play with your dumba** frisbee shaped toys that dont have even no sides on them and see if you can even figure out PI LOLZ pie like something youd eat its not even a rational number whatevss
real pros use polygons

whaaaa??? ow did someone just hurt on their foursided pointy pointy pointy edge? shoulda used a circle, youd be happier :P :P r4d1us 4 l1f3

Seriously, definitely let me know what new interfaces you’ve found inspiring lately, and I’ll be sure to credit you in my talk!

  • Randy

    You're going to be in Toronto!? I live in Toronto, wish I'd known in advance, I would have tried to make arrangements to attend the talk.

  • http://www.midipixel.com Midipixel

    Hey Peter, do you plan to have this talk on tape later on? I'm all about interaction design and music, so I'd love to watch this presentation.

  • http://www.nofi.org Jeffrey Melton

    I'm a big fan of the interfaces for synthPond, Reactable, chipPad. I also like Soundrop, Beatwave, Sound Scope Space and Curtis Granular Synth, though a lot of iOS musicmaking apps seem like one-trick ponies to me rather than acting as a complete environment for making music.

  • http://jonathanreus.com Jonathan Reus

    To be fair, I don't think the tablet interface is such a big step up for musicians from PCs or laptops. While the design challenges are different, it's more or less the same problem.. we're trying to use a platform that was not made for music in a musical way. 

    Yes we, as designers, need to get away from the old anachronistic interface paradigms like knobs and faders.. but I'm far more interested in single-purpose physical interfaces and an open platform for developing and connecting them to mobile/ubicomp devices. 

    Android is way ahead of the curve here. Google recently announced an open accessory standard for Android devices based on Arduino.

    http://www.engadget.com/2011/05/10/google-announc

  • ArmandoC

    Hey Peter,

    Look at what the new emulator ipad app is supposed to do. I feel like it finally closes the gap between Lemur and iPad users, but still like many, I feel left out when I don't have a tangible interface to interact with my ipad. 

    Don't get me wrong, the concept of a touch screen is great, but there is an article from Sasha from a few years ago who even said mixing on tablets is not the same as having a traditional interface. Like myself, he feels like he looses touch with the art of making and tweaking music. 

    Now going back to a previous post I believe on Synthtopia over the Waves One knob plugs (everyone getting in an uproar over one knob mixing). I hate to say it but as technology makes it easier to make music, there's always going to be a path dug for new innovative ways for creating whatever is the hottest sound at the moment… wub wub wub wub. Whether its an ipad, or some magical in-air interface to interact with, people will make pretty fucking fantastic music no matter what device they use.

    My .02, learn and make use by what you have and utilize the limitations to your best advantage. In the end,  no two people think alike, and that's something beautiful to say about music. 

  • http://www.jhhl.net/iPhone jhhl

    I'm in the circle camp – and they were mostly flat before because of manufacturing constraints. Tondo's loop is a circle, and Droneo has a circular spiral which is akin to a circular slide rule (which is better than a flat one). Circles tend to have less resolution near the center, so a detail view or zoom is sometimes necessary.

    Rectangular interfaces which loop can be made a lot better by showing the overlap at the looping ends.  That way, you get some of the advantages of both systems.

     What really bugs me is the inflexibility of the grid systems in music interfaces, but that's part of my prejudice against beats in general and 4/4 in particular. 

  • Peter Kirn

    Good comments; keep them coming. 

    Toronto: don't worry, plans to come back and at a public event! I figured folks interested in NXNE would be already doing NXNE. ;)

    Limitations of tablets: I intended to cover this, as well; I'm not there to cheerlead for tablet platforms, either any specific one or talking about them in general. To me, they're another canvas for computing. And I think the software we've seen speaks for itself; software designers have responded to new challenges with some new tools.

    Augmentation with hardware: definitely intend to talk about that, as it's personally very interesting to me! :) I don't necessarily agree that Google is ahead of the curve with the Open Accessory standard, as it's a fairly poor substitute for doing across-the-board USB class implementations. In other words, the computer remains ahead of the curve, and tablets (audio and MIDI class on iOS, Google's faux host mode on phones and proper host mode on tablets) are gradually catching up. Don't forget wireless connectivity, either.

  • http://www.holotropik.com Holotropik

    I too prefer the circular design for music too, just makes more sense in my mind.

    Although recently I found Artikulator (iPad) which is linear but allows you to draw sounds which adds a new feel to sound creation and takes advantage of the touch surface in a unique way. It's like Bebot but you can see the sounds you have layered.

  • JCB

    I think that one of the mistakes often made when designing the interface of a music app for a tablet (or a music software for a computer for that matter) is using visuals that are a metaphor of the studio hardware side of things and not starting to think about the actual hardware in use (the tablet or PC) and what can be the best UI for that hardware. For example, take a look at the countless softwares representing parameters with a knob shape, which to me is not the best shape to interact with using a mouse or a finger (unless, like it's the case in TouchOSC, the knob can be huge and that the position of the parameter is absolute). The knob makes sense in the physical world, but not so much in the virtual world, aside from saving a little space. A fader is at least both clearer and easier to use with a mouse or a finger.

  • JCB

    …So I guess that leaves me out of the circle camp!

  • http://sustainedsound.com davidestevens

    My favourite iPad interface app is Mugician – I really like that you can change the degree of "frettedness" from fixed to none at all. Definitely the most expressive interface I've come across. Which I suppose means that I'd better start saving for a Soundplane :-)

    I just discovered Droneo (which I love) and the note spiral in there is a great way of thinking about note relationships and "chords".

    I think the value of the circle approach for me is that it forces me to think in a different way about what I'm doing, which helps me break out of my ruts. Loopesque is an interesting beginning, but for me it's still too locked to the beat grid, 4 in a bar approach. 

  • bitbitly

    this is easy: SynthX.

    without a doubt the best iOS music app i've ever used. after i launched it the first time it wasn't until 2 hours later that i was able to put it down. the keyboard/grid interface with its y axis & 'snap to scale' feature allows for some ridiculously expressive playing. i don't even use my physical midi keyboard anymore.

  • http://music-interface.com mat

    Hey, I build interfaces on the Lemur and I am an ergonomic researcher (main job), so a few of my thoughts:

    - main advantage of touchscreens is their interactive layout. press one button and you controller changes. that enables to give the user that control he needs for a special function and another control for another function…not possible on hardware.

    - hardware and haptic feedback will never be replaced by touchscreen, but it can compement control

    - Instrument designers are often limited by the controls they can use for their ideas (TouchOSC and Lemur have only some graphical elements, Monome only a button grid) – it is great what already come out of this, but we need more objects for our sandbox tools!

    - I miss the opportunity to "paint" my own control objects (like a curved fader, fitting to hand position and angle) on these sandbox touchscreen control apps

    - circular sequencing is a nice idea, but it misses the point as long as it fits to linear grid, same speed on each ring and same resolution

    - with the new freedom in interface design on touchscreens a detailed task analysis for the function you want to represent becomes more necessary. Like it is done in complex work enviroments – what are the needs for e.g. sequencing? (Till now- designers (me included) use their own needs as frame of reference, but so you miss/oversee many chances)

    - the freedom in interface design is based on the freedom of routing the signal, it is a blessing of our computer time and wasn´t possible (at least so easy) 20 years ago… so we are at the beginning of a new revolution in instrument design, cause for the first time the physical sound and the control are decoupled…

    Nice times we are living in :)

    Hope to see many new instrument ideas here soon.

  • Random Chance

    @mat: I think the idea of painting controls instead of just choosing controls from a set of predefined ones and maybe changing some of its attributes, is a good one. I'd really like to see how far one can take it. I could imagine something along the lines of: Paint a shape (like you would in a vector graphics program) and then assign some control functionality to it, so you could draw sliders, dials, buttons etc., but also experiment easily with combinations of those. Could be quite complex to design and even trickier to implement.

  • Peter Kirn

    I'd say the problem with a virtual knob isn't that it's a circle, it's that it's a flattened representation of a cylindrical object – the encoder/potentiometer you grab with your thumb and forefinger, which isn't a gesture you can make on something that's flat.

  • alfonso el sabio

    Thumbjam, NLog Pro, MorphWiz … as a primarily guitar player, these 3 apps were a natural progression for the more structured work that I do and their interfaces lend to that.

    Don't want or need faux knobs, really … too much trouble for the precision I'm looking for when using both iPod and iPad for performing.

  • Aaron

    I think the Knob comments are a little off base. First off, they only really save room. 99% of the knobs on music software out there act as Sliders.. as in you control them with up and down movement and not a circular one. If a VST or tablet/phone app uses a circular motion to control the knob, I consider it a broken application and if I can't override with the host (as is often the case with VST hosts).. then I wave goodbye to that plugin/app forever.

    The main point I'm trying to make is that they look like knobs, but are sliders in actuality and save screen realestate. I like knobs in this sense, and its long been standard programming etiquitte to treat them this way.

    The only time I truly hate a knob is when there is no direct value representation (label or meter).

  • Peter Kirn

    No, I can agree with that: a virtual knob is actually really a circular, compact fader. ;)

  • http://music-interface.com mat

    about virtual rotary encoders:

    They are not so bad! For sure Peter – they are flattend representations of a cylindric object in real world, which is weird. On real knobs both fingers build a counterforce, giving you more precision, assisted by the haptic feedback of your twisting hand. Virtual knobs are controlled with one finger and therefor might miss the (motorical) precision. However, based on the fact that they are round, they offer a rather long way of values compared with a linear control object (fader) in same space on the control panel – better resolution.

    Of course it is necessary to activate "capture" for that virtual knob. Thats a function on the Lemur meaning, once you start the touch movement within the object it is locked to it until you leave your finger. So your rotary movement doesn´t have to fit the knob exactly. (Without that virtual knobs are hell – I thought this is standard also on Ipad, but not sure). btw: I use virtual knobs mostly with my thumb while my palm rests on the metallic border of the Lemur – can draw rotations on endless controllers very fast and quite precise…

  • Peter Kirn

    Absolutely… I think the issues are getting implementation right, and not simply defaulting to virtual knobs because you can. The issue isn't only whether a virtual knob can work, but whether, given the open-ended possibilities of a touchscreen, it's the best solution to a given problem. On hardware, we use potentiometers for specific reasons that aren't necessarily relevant in software. They can be, but it's worth examining each case.

  • http://music-interface.com mat

    uups, the main argument is brought by another comment while writing ;)

  • http://www.gridmode.ca mason bach

    Ohh I wish I knew this earlier!!!! I'm in toronto and have a NXNE pass. But instead of being at the festival I'm home prepping for a gig tonight cause I procrastinated and played with MLRV & Launchpad all week (thanks to you that is!!!)

    If you are heading out, come by Lily Lounge tonight my friend!!!! Be great to meet you :)

    MB

  • JCB

    Yep, the knobs save space, and they serve exactly the same as the faders. But real estate is usually pretty cheap in the virtual world where you can have many windows accessible via tabs on the screen for example. I still believe that the knob representation is not the most intuitive approach.

    @Aaron. I get what you say. If you move up or down the value of the parameter represented by a knob increase or decrease; it works like that in many software, but not all, like you said (and that's part of the problem). It is a design flaw to me, because the movement you have to make isn't directly related to what's happening on screen, and that can be confusing for the user.

  • http://noisepages.com/members/digitalfx/ DiGiTaLFX

    Well I know it's not exactly much use outside of a game, but I do find that Smule's Magic Piano app has a wonderful interface that makes anyone feel like they are a master pianist. There's probably not many interfaces that you can say that about!

  • http://92bpm.com mymanhenri

    i'm upset i didn't see you were coming to Toronto – after going through the whole list and being disappointed w/ most acts. nxne is pretty lame for non indie rock fans, but I would have come out for your talk, hands down. smh. Looking fwd to having you back. Pls keep us posted. 

    How long were you here for?