The iPhone 3.0 SDK is a fantastic update, bringing a lot of what was on developer wish lists for the device. But some of the early speculation – that the so-called “library access” would enable music games and DJ apps — may have been premature. Jordan Balagot writes to let us know that, at least in the current SDK, access to media is very limited.

The “library access” in the 3.0 SDK is only a player control API similar to that of the iPod; there is not even read only file access for MP3s nor any way to modify the output from the library. So no iPhone DJing, no BPM detection, no interactive PD or Reaktor patches with your library.

Unfortunately, this seems consistent with Apple’s desire to be the one and only media player on the device. I’m hoping that this is still something Apple plans to add – imagine the ability to add effects or run games based on the library (a la the PC game Audiosurf) or create DJ apps. I know many people who use iPhone or iPod as sample players or backups for live sets; having a custom player app could also be useful.

By comparison, Google’s Android has no such limitations on its MediaPlayer class – the fundamental difference being that you aren’t limited from playing media on your device. Unfortunately, Android has its own limitations: no real audio buffer access, which means it’s not possible to build effects or DJ apps or games on Android, either.

And that’s typical of the sort of situation the newest mobile devices present. We have the iPhone, more sophisticated technically, but limited, apparently, by design in order to protect Apple control over certain functions. Then we have the Android, philosophically unlimited but technically limited by some key missing capabilities.

My question is, which device will evolve first to give us the freedom to make use of its full potential?

No file or output access to iPhone MP3 library – 3.0 SDK still too restrictive

If we’re lucky, perhaps the 3.1 SDK? (Or something we’ll still see in 3.0 that isn’t done yet?)

  • Adrian Anders

    And that’s typical of the sort of situation the newest mobile devices present. We have the iPhone, more sophisticated technically, but limited, apparently, by design in order to protect Apple control over certain functions. Then we have the Android, philosophically unlimited but technically limited by some key missing capabilities.

    Hence why it was foolish for Palm to drop their legacy OS to ape Android/iPhone. Even though the older Palm OSs were somewhat limited by the hardware of the Palm PDAs, at least they gave enough access to the hardware itself in an open platform format on which some really amazing apps (like Bhajis Loops) were developed. Palm had an opportunity to differentiate themselves from Apple and Google by updating their OS while keeping their core philosophy of an open platform for applications and hardware integration intact. Instead they are going to end up as an also-ran in the smartphone game.

    I think Netbooks (especially tablet-enabled ones) are going to pick up much of the market that was dominated by PDAs and later smartphones in the 90s. If an application doesn't need the web-integration that works best on a smartphone (most music applications for example), it will probably end up on smaller form-factor portable computers like netbooks rather than the closed ecosystems found on most modern smartphones.

  • autoy

    So the iPhone has all this great apps like noise.io, iDrum or BeatMaker as opposed to what exactly on Android? Ok, so it's not so "open" but at least it encourages developers to actually build stuff, stand out and make a profit via micropayments. I think it's a reasonable tradeoff. Apple won't let you mangle your iPod songs? Load your own samples in BeatMaker and you're good to go. At least I can DJ with the wonderful TouchOSC hooked up to my Mac.

  • http://www.createdigitalmusic.com Peter Kirn

    Well, right, precisely — the relatively easy access to sound/DSP, the multitouch screen, and the presence of an FPU all enable those apps on iPhone and there's no equivalent on Android. Not yet. On the other hand, the iPhone's first year didn't even *have* an official SDK, so we'll have to see how both platforms mature.

    But in the meantime, yeah, I'm a big fan of what's available now and those apps specifically.

    As to Adrian's netbook comment, that means that for Android to be a compelling netbook OS as Google seems to hope, it is going to need more functionality along the lines of what you get developing for normal Linux distros.

  • richardl

    Besides Apple's megalomaniacal desire for control there may be technical and legal reasons why they don't or can't provide access to the audio playback data on the iPhone/iPod Touch. The main technical issue that comes to mind is that audio decoding and playback is normally done by dedicated hardware. (iPhone 3.0 will include software decoders too) A hardware audio output chain may not provide for access to the data buffers prior to sending the data to the DAC.

    I will leave it to your imagination what sort of legal impediments might get in the way of allowing access to content that is licensed for sale through iTunes for audio playback. Of course, those legal restrictions shouldn't apply to your content, but infectious protectionism has been part of the rules of the game with the iPod since start of the iTunes Music Store.

  • http://www.createdigitalmusic.com Peter Kirn

    @richardl:

    Technical problems – maybe. But you can provide post-decode audio buffer access.

    Legal problems – no way. The data is already in the audio buffer. And Apple isn't providing simple playback controls. There's no legal issue to a non-iTunes app playing a file already on the device.

    Most importantly, on both counts these capabilities are already present in desktop software.

    That said, to me these limitations *aren't* actually deal-killers in the way some of the Android gaps are. There are still some amazing possibilities for iPhone apps that use other capabilities. Actually, even on the Android I could see some creative uses of just doing media playback. So, you know, developers will continue to do what's possible. I just thought the misconception about this capability wasn't necessarily productive.

  • http://jordanbalagot.com Jordan314

    I agree with Peter. There's a lot of speculation out there that 3.0 will allow for serious DJing. For instance:
    http://www.synthtopia.com/content/2009/03/17/5-re
    And it doesn't appear to be the case, at least with the iPod library, anyway. If you include mp3s in your app itself, that's another story, but that's also a hog of an app.

    I suppose one reason they want to limit this is to block potential file sharing apps, i.e. phone-to-phone mp3 transfers, but without the ability to write to the library anyway that seems silly.

    You can control the ipod player or make your own music player, and perhaps you can play them both at the same time and crossfade that way, but you're not going to be able to beat match, pitch shift or time stretch. That's a big disappointment. (You do have access to metadata though so at least there will be some new playlist / organization / recommendation apps).

    Hopefully if the Android or the Pre continue to innovate and compete with their music and file system support, Apple will be forced to open up the phone even more.

  • http://www.intermorphic.com Pete Cole

    Actually, I'm feeling very positive right now about the iPhone platform. We have Mixtikl running (at last!) on real live devices; UI responsiveness is good, sound is fab and the transition from the simulator was painless (estimated time to submit to App store: 2-3 months from now…).

    I agree that it really is a shame we can't do everything we want (my pet grievance: why is it so hard to share data with your desktop Mac or other iPhones!), but today my glass feels half full :) and I'm looking forward to finding interesting ways to work within the boundaries imposed by the platform.

  • Pingback: I-Phone 3.0 #Fail – Dj para I-phone não é possível « www.ilankriger.net

  • autoy

    Tethering apps beign pulled from Android for example. So much for openness http://is.gd/pOD2

  • http://www.createdigitalmusic.com Peter Kirn

    That's a fundamental difference than the app distribution on the iPhone.

    It says Google is dropping the app *as distributor*.

    But you can

    a) download apps from other stores

    b) download (and purchase, if you like) apps directly from the developer

    Both without breaking any kind of legal restrictions on the device. It's simply not so with the iPhone, which requires jailbreaking (a potentially illegal act) to purchase apps through any outlet other than Apple.

    I think Google has done a poor job communicating on this, and I still don't understand why carriers are opposed to tethering — why not just charge extra for it and use it as a valuable revenue source? You could even likely cap bandwidth consumption, as typical tethering users are going to want only light use.

    But as a supposed counterexample of openness on Google Android, sorry, you've got it backwards entirely. It's another illustration of the difference in distribution models. If you don't like it here, you can go around Google, and that to me is the way it should be.

    No, I think the main problem with Android remains a lack of greater developer *functionality* — the question of whether you can develop an app with the broad set of capabilities a Linux app might get, for instance — and the ongoing lack of handsets.

  • http://www.astoundstereo.com Jerry Mahabub

    I am in full agreement with Peter Kim on the legal side of things. The only legal issue is that one can't "permanently" alter the content, however, if an application was to simply process the audio in real-time, much like GenAudio's AstoundStereo Expander software does, and when you turn the processing off it remains unchanged, then legal issues go buh bye! In reality, the way to skin the cat would be to form a licensing relationship with Apple, and this can take some time. My company has been patiently waiting for a kernel extension for iPhone since we became beta developers for version 1.0…Nada, and I do not anticipate that changing anytime soon. Likewise, we are looking to develop an Android app as well, however, the problems that are present for doing this sort of processing on an iPhone are not present for the Google phone, and vice versa! It's like they want to have complete control over certain kinds of applications, and unfortunately, one of them being audio DSP apps.

  • http://www.createdigitalmusic.com Peter Kirn

    Quick note – my name is actually "KIRN" not "KIM" — it's a font thing, almost impossible to tell!

    Actually, I don't believe the issues are legal. There are various DJ apps on the Mac that have iTunes library access, and have been for some time. I can imagine the reason being either control, technical (there's something to that), or some combination.

    As far as Google Android, I think it's best not to see it as a direct competitor — there are lots of differences — but of course we're looking at both. And it does appear that Android will improve over time. I know access to native code is something Google has promised, and already 1.5 SDK is a whole lot better for audio. Once you get native library access, in fact, an app that works on both Android and iPhone – with different code on the top level – should become more feasible. That'll put further pressure on each to open up APIs, because, for instance, you could get features on Android not on iPhone as in this case (or visa versa).

    Both platforms are really young, so I do hope all of this improves down the road.