Part of the beauty of the iPhone from its launch date was the notion of a Mac you could fit in the palm of your hand. That makes it ironic that, for so many developers, mobile platforms in general have turned into a way to fragment software, to make it run fewer places instead of more. There’s something to be said for designing to a specific mobile device, but on the other hand, how many developers would want to restrict where their creations run? And particularly in music, isn’t the appeal of mobile creation the ability to have your tools work in more places? Maybe targeting just one gadget is the right choice for a given tool, but it shouldn’t be the only choice.

On the iPhone, the application Thump has plenty in common with a range of mobile music making tools. It’s simple but fun, a groove box with sequencing, subtractive synths, drums, and a set of basic effects, plus the ability to load your own samples and export songs.

Thump also demonstrates how simplifying sonic capabilities can produce musically-beautiful results, by focusing on the essentials and creating something with personality. Here’s a track by its creator showing off its sounds:
thump soundreel by mazbox

Well worth checking out the app on your platform of choice:
http://www.mrkbrz.com/thump/

What might not be immediately apparent is that under the hood, Thump makes use of the open source environment openFrameworks. As a result, the same code runs on iPhone, Mac, and Windows, as well as Linux. (It’s not distributed for Linux, but it could be. Hint, hint.) Creator Marek Bereza gives back, too – he ported the openFrameworks audio library to the iPhone, where it’s available to anyone.

Updated Marek notes in comments just what this means. The video below is, essentially, the same app. In place of the iPod screen, he has used a massive lattice of physical controls. A separate installation at the same show used a large touchscreen and simplified interface. And this really demonstrates what cross-platform means. Guy Kawasaki in the 80s mocked “ports” as a cheap wine. The idea is not to simply dump your code on a different platform and hope for the best – in fact, in this case, the changes from one platform to another were radical. The key is maximizing what’s essential, what really is not specific to a single device.

Physical Sequencer from Marek Bereza on Vimeo.

Creative Applications has a detailed write-up of the installation with more documentation.

If you’re interested in creating your own projects, oF has an elegant syntax based on Processing, but adapted to C/C++ coding paradigms and libraries.

I would recommend trying desktop development first as you’re learning, but when you do want to go mobile, setup is absurdly easy:

Setup instructions for iPhone

Having this as a foundation means more freedom for users and developers alike, the ability to run where you want, and a shared community of artists helping one another out. It does not mean sacrificing revenue, either – which is important, because revenue is often what enables (and encourages) developers to make new things. Here, we see a commercial mobile tool and distribution via the iTunes store, with free-as-in-beer distribution of desktop apps. But even the Free Software Foundation actually encourages charging for distribution – and to many people, the FSF is the most extreme example.

The bottom line is, you’re free to create what you want, where you want – and it doesn’t even have to have a price of free.

The use of openFrameworks is possible because the iPhone allows the execution of native code. Apple does not allow the Java virtual machine, which locks out many useful platforms and tools — including Processing. Naturally, that should not be the case with forthcoming non-Apple platforms based on Linux; more on that soon.

Thanks to Memo Akten for the tip — and for his exceptional documentation of iPhone openFrameworks development.

motionpage2

GPU-tiful and Cross-Platform

Updated Another framework I left out is the lovely libnui. The tagline says it all: “C++ API, multiplatform (Mac OS X, iPhone, Win32, Win64, Linux), 3D hardware accelerated GUI and more…”

On the iPhone, Intua’s elaborate Beatmaker software and David Wallin’s bleep!box have both been made with libnui. I want to single out bleep!box because of its UI – note how UI elements are simplified and scaled to the size of your fingertip. That’s something that, speaking of multiple platforms, could be equally important with touch hardware coming to Mac, Windows, and Linux platforms (some of it here already). I saw a huge fan of Windows tablets loving Propellerhead Reason on a tablet. He probably would be less enthused running Reason or Record on a touch interface; the UI elements are way too small.

bleep!box is also worth a look for the same reason as Thump: elegant UI, simple functions, and a lot of fun.

Pd, too

Pure Data, the open source patching cousin of Max/MSP, can also run on mobile devices. It’s currently powering the application RjDj, but because Pd is free, that could be just the beginning. In May, I wrote about the work being done by Hans-Christoph Steiner, who has been porting Pd even to ancient PDAs and iPods – including those, notably, without native floating-point capabilities (the kind of number crunching usually associated with sound).

Well, now Hans-Christoph is looking toward other, current-generation devices, too, kicking off with a workshop on making interactive sound on the iPhone using Pure Data. Here’s a taste:

OK, kids, you won’t believe how easy this is. Once you have Pure Data installed on your your iPhone or iPod-Touch, creating and uploading your own sound-processing and sound-generating patches is as trivial as copying a text file to your device! You’ll still need an Apple Developer License, but because Pd patches are treated as media files, they don’t have to go thru Apple’s elaborate code review — so, you can just dive right in, turning your phone into a pocket synth within minutes. In this 3-hour hands-on workshop, interactive sound whiz and Pure Data developer Hans-Christoph Steiner will lead you through all the steps.

Hans-Christoph will be talking Pd on the iPhone at a conference next month at Carnegie Mellon University in Pittsburgh. That will also include Memo Akten and Zach Gage talking about openFrameworks. And for nearly-entirely open source hardware, there’s a tantalizing-looking workshop combining Android with Arduino, too.

http://artandcode.ning.com/

There’s quite a lot more to say about this, but I hope this whets your appetite.

And, of course, riding the train to Pittsburgh or your nearest university or open source workshop, now you’ll have Thump which which to make some thumping tracks on your iPhone.

  • http://www.derrick-s.com Derrick S

    Thx for posting Peter!

    Multiplatform is the way to go on music apps.

    I also would love to see those apps working as a VST plugin, so you could be able to sync it with the iphone

  • The Cody

    I like Thump's interface, it's clear and fairly minimal. It gets you straight to just making music. The sound is great.

    The only real draw back is not being multi-touch. Which means no tweaking synth 1 and synth 2's cutoff at the same time (in the mixer view). No changing patterns at the same time (which wouldn't be too bad if you could set it to queue the pattern changes to the measure's end or the pattern's end).

    Other than that, it's a fun little groove box and worth the price of admission.

  • dovemouse

    im looking forward to flash cs5 with iphone export,

    should open things up for a lot of people who can't get their heads around iphone app coding…like me ;)

    thanks for the post.

  • http://www.3amnoise.net/runagate runagate

    Wow, thanks Peter!

    That should keep me busily reading for a while.

  • http://www.bleepboxapp.com/ David Wallin

    I'd also like to mention that libNUI (www.libnui.net) is another open source framework that compiles under windows, mac, linux and iPhone – one of the biggest iPhone music apps (Beatmaker) is programmed using it, and also my app, bleep!BOX (www.bleepboxapp.com)

  • http://www.luckyframe.co.uk/mujik/ Yann

    Thump looks and sounds great!

    I just wanted to mention that our iPhone app Mujik also used OpenFrameworks…I believe the coder of that app, Jon, is going to be one of the tutors for the OpenFrameworks workshops at Art and Code in Pittsburgh.

  • cobalt

    I'd enjoy hearing your thoughts on Dopplerpad, an iPhone app that I've been interested in since its release, but haven't pulled the trigger on.

  • Ben

    RjDj is interesting. I though Apple wasn't allowing you to port a programming platform to the iphone and that was why flash wasn't availble?

    If it's just running PD patches then isn't that the same thing?

  • http://www.PatternMusic.com RichardL

    "…the notion of a Mac you could fit in the palm of your hand…"

    I disagree. I think that misperceives the platform completely. The iPhone is an entirely different experience from the desktop, and while there may be some synergy between desktop and mobile versions of applications. The similarity ends very quickly. And if you look at the apps that are successful by any terms you will find they are rarely palm-sized versions of Mac apps. In fact it's somewhat inverse.

    The desktop-style music apps on the iPhone (I'll not name names, but there are two $20 music apps on the iPhone that come to mind.) fail quite obviously in their ability transform the process of creating music to appropriate flows for mobile. They tend to rely on File/Open/Save/Import/Export and shrinking down a bunch of fiddly controls to fit on a 3.5 inch screen. Sure you can take your DAW with you in your pocket, but you don't really want to take it out of your pocket. These sorts of apps are better suited to netbooks running a true desktop system in the palms of maybe two hands.

    "That makes it ironic that… mobile platforms in general have turned into a way to fragment software, to make it run fewer places instead of more."

    How is being able to write a single app that runs on 50 million+ devices running in fewer places? Let me repeat that: FIFTY MILLION devices. That's a market size that is bigger than its sister desktop platform.

    So I don't understand the emphasis on the importance of multi-platform from a user's perspective. Since the rise of the GUI multi-platform has been an elusive goal. (I've made my living for 20 years doing compatibility and multi-platform engineering.) But it always means a huge amount of additional engineering or compromised functionality. Anyone who tells you otherwise is selling you something. It may be important for establishing a brand. It may be important for apps with social features. And supporting alternative platforms may be good for competition and innovation in general. But how else is it of importance to an individual end user? I can think of a very small handful of scenarios, but these tend to be edge cases. I understand the desirability of multi-platform from the perspective outside of the huge hegemonic dominant platform. "That game looks fun. If only those kids would come over to my side of the street and bring their ball with them."

  • http://www.createdigitalmusic.com Peter Kirn

    Richard, I'm not saying don't tailor software to mobile, or even specifically target mobile. I'm saying, as a developer, you want to at least have the options of tools that will allow your work to be in both places. The user perspective is dead-simple. The user will say "I want to get my work between my mobile and my computer," or "why doesn't this run on xx," or… I could go on.

    In fact, in all three posts for today, I'm arguing that some of the flaws in desktop apps are flawed *on the desktop*. "Desktop" computing may now increasingly mean something you take onstage with you, something you access at different screen sizes and in different situations, something you use with multi-touch input.

    Multi-platform of course means testing on multiple platforms, designing to multiple platforms, and yes, sometimes making compromises. But why *shouldn't* there be options that use these?

    50 million devices is an artificial number. Mobile platforms are getting much bigger than that. And anyway, developing for myself, I have — maybe half a dozen devices? One's a Mac, one's an (iPhone-based) iPod touch, one's an Android, two are Linux and/or Windows-based PCs. So that's some fragmentation in my own home.

    Anyway, the proof is in the pudding. You're making an abstract argument here. Above, we have a number of devices where these frameworks made the software more flexible, made the developers more productive, made the tools easier to test and to get doing what the developer wanted. As I said at the outset, there's no one solution for every developer or every app. And of course, any idea that there would be some "export to everything" button is extremely misleading; testing and customization is essential. But I still think there's real value in open, cross-platform frameworks and especially open, open-source, cross-platform frameworks.

  • http://www.createdigitalmusic.com Peter Kirn

    Oh, and two other questions/comments:

    1. Surely you'd agree that getting things to run across platforms has improved in 20 years, particularly when set against the massive amounts of additional complexity and functionality we've added in that time.

    2. The iPhone is on the outside, so I'm not sure what you mean by your last comment. It's been Symbian on mobile with dominance, and my money would be on Linux going forward. (On mobile, not on desktop any time soon.) But look at the Linux "platform" – or even Symbian – there's nothing hegemonic about the dominant OS when you get to mobile. Cross-platform development absolutely has its evils, but they're unavoidable. Apple has this lovely little bubble, but it's just that, and even on iPhone you have hardware, API, GPU, audio I/O, connectivity capability differences to consider. Surely that won't involve one resolution forever, either. That's the very nature of testing across devices, and that's on the most closed platform there is.

  • http://www.bleepboxapp.com/ David Wallin

    @RichardL – The advantage of a multiplatform framework is more for developers – for example it enables iPhone development on other platforms than just the mac. It also allows me to port my code more easily – for instance, it shouldn't be too hard to make a desktop/plugin version of bleep!BOX that can read the same preset files and has the same sound engine as the iPhone version.

    I totally agree about the UI and experience needing to jive the platform. I spent a lot of time keeping the UI simple and focused on a single task and the buttons big. I also wanted to focus allowing users to create musical phrases and patterns, rather than trying to recreate the full DAW experience on a tiny screen. I look at mobile devices as a way to generate cool content when away from the computer, that I can bring into my DAW later. Or as a performance device.

    I also agree that the iPhone is a major platform in itself, but there is still a big desire for code to work across all platforms. Developers always want to be able to hit as many targets as they can to maximize their efforts. Having code that works on all of them is a big win, even if the UI is different.

  • http://www.PatternMusic.com RichardL

    1. Yes. We've learned a lot. And the variety and depth of multi-platform alternatives large. But the main tools from Apple and Microsoft are still not multi-platform. That means we either ignore any developments there or we abstract those parts out. And the latter process is still non-trivial and costly.

    I've been developing an iPhone app. The sound engine it uses is multi-plaform. (Alas, no other mobile platforms are supportable yet.) But it's inconceivable to me that I could have built the GUI in a multi-platform way. I certainly was aware when I made that decision and took that path, but there really was and is no alternative given the complexity and constraints. Hopefully that will change.

  • http://www.mrkbrz.com Marek

    I think you said it, Peter, it's making developers more productive, which means they can get on with it and offer more to the user. Rather than being faced with different API's, even languages across platforms, we've now got tools that you only need to learn once to be able to run on (almost) all.

    This app is one of 3 projects (including a large multi-user step sequencer on a multitouch table and a physical real-world sequencer – more details at CAN) to use the same core code, without modification, all on different platforms. I released thump on mac and windows for free so people could try before they buy.

    Obviously writing an app for different contexts or different platforms you need to address GUI differences in code, but more fundamentally it's a usability/design issue which no amount of code is going to compensate for. As you said, apps that cram in too many tiny knobs and sliders just have designers that haven't quite got to grips with this paradigm.

    Note: multitouch coming to thump shortly!

  • http://www.PatternMusic.com RichardL

    In many ways we've rebooted back to 1994 with mobile development. (Reboots are not aterrible thing. Look at Star Trek.) At least that's the dominant impression I've got after working with the nuts and bolts of iPhone. Hopefully we devs all can remember the last 15 years and learn from the experiences of desktop and web development.

  • http://www.createdigitalmusic.com Peter Kirn

    @Marek: Ah, wow, I actually had seen your work there and completely forgot about it; thanks! (In my defense, of course, I'd love to see it here in the US, too!)

    And I think your work perfectly illustrates what *both* Richard and I were saying, even if we were coming at it from different directions. Obviously, to really move your creation from one platform to another means being disciplined both about what you keep (the sequencer guts) and what you leave out (the UI). In fact, I can't think of a better example of what adapting an interface is all about than making something work after you eliminate the screen. :)

    It's great stuff; hope we'll see more.

    @RichardL: Agreed. "You must unlearn what you have learned" definitely applies to mobile development. But you know, I think it's healthy. I mean, on Java ME, basically what you had was a massive pain. But if you look at Android Java development in contrast, I think programmers are having to learn some *good* habits, more flexibility about how they solve problems that allows them to optimize for different hardware. On iPhone and WiMo and Symbian and Android, going back to OpenGL ES 1.x is not a bad way to hit a lowest common denominator of GPUs.

  • http://www.derrick-s.com Derrick S

    @ Marek

    good to hear that multitouch is coming to your app

    have you got any plans for making a vst version that works together with the iphone version of your app?

    i would love to see this finally happens to an iphone app

    For example Beatmaker is really good app, you can export midi files and then import in your sequencer, thats a good thing, but i guess apps could do much more than that

    as a producer for me one big thing stops the i-Phone from really being able to use it in a productive way in the studio and that is you can´t use it as a vst corresponding with your iphone

    there are some really great controller apps like novation automap which allready show how good this can work

  • http://www.jordancolburn.com Jordan Colburn

    If you can get PD on an iphone as a developer, is someone working on getting an app store PD version for regular users(someone who doesn't want to pay apple $99 just to use an iphone like any other mobile device)?

    I've been thinking PD on iphone would be great, the ability to code apps using a language you're already familiar with and tailored to music would bring down the complexity level a lot.

  • Al

    One comment I'd like to make on a lot of these iphone apps like Thump is the panning feature. Apps like Thump and iSequence lack panning. Everthing sounds right in the middle of the stereo spectrum. There are some app like Bleep and Rangrid that incorporate panning and that makes them more functional and sonically pleasing to work with. So come on app developers remember we live in a stereo world; give us panning!!!!