Samsung Galaxy Nexus remains the phone you want if you're interested in music and audio on Android. Photo (CC-BY-SA) TAKAPPRS (TAKA@P.P.R.S).

Samsung Galaxy Nexus remains the phone you want if you’re interested in music and audio on Android. Photo (CC-BY-SA) TAKAPPRS (TAKA@P.P.R.S).

Saying your device isn’t as responsive to sound as you’d want is a bit like saying you’re feeling sick to your stomach. The symptom is easy to describe, and everyone would agree it’s not a desirable state. But the fix can be rather complex.

And when it comes to engineers who care about music and sound, experiencing latency – or its equally evil mirror cousin, crackles-and-pops – will make you sick to your stomach.

Google I believe is deserving of some criticism over this issue. Years of subsequent updates saw the company largely silent or unresponsive about critical audio issues. It took some time before even basic APIs were reliable and on par with other platforms. At the same time, I don’t believe even all developers – let alone users – appreciate the challenges of making music-quality low latency performance work. There’s no silver bullet: any number of issues with drivers and firmware and battery management can cause things to go wrong, and only a delicate combination of ingredients will make it go right. Indeed, that’s part of why Apple deserves some credit. Being the company making both hardware and software is a big boon, no question, but even that is no guarantee you’ll get results.

One year ago now, we first saw signs that Android would see higher performance audio. This included some general improvements for all newer devices, which my friend Peter Brinkmann quickly baked into the open source libpd library. Low latency audio was promised initially for the Samsung Galaxy Nexus, as I reported, as well, in June.

At this year’s Google I/O, we finally get a presentation from people who really understand the issues involved and explain in atypically-candid detail how you go about solving them.

If you care about this, too, you owe it to yourself to sit through the whole video, as there’s a whole lot of technical detail here. It’s also worth noting that some of the things you think might fix audio actually don’t. Sure, it’s easy for developers to gripe that a platform doesn’t magically give them low latency audio, particularly if they’re familiar with desktop Linux. But on mobile platforms, a number of variables come into play. Some of the obvious fixes can then conflict with battery life, or simply don’t work.

The full video, featuring Glenn Kasten, Ian Ni-Lewis, and Raph Levien:

It’s a pleasure to see engineers who really get this stuff talk about this rather than higher-level managers; this is the sort of conversation I have over beers with people who know what they’re doing far more than I do.

A key slide:

android_todo

The left-hand column, what the Android team needs to do, is summed up in two areas. One is improvements to the platform, and the other has to do with working with hardware partners, whose drivers and hardware and well-meaning battery conversation features and the like are often the source of problems. Most tellingly, while it’s long overdue, it’s good to know that Google is at last adding proper APIs for configuring audio (augh), and being trying more adventurous paths to lower latency as have been found more commonly on desktop systems.

But developers can make headway, too. As evidenced by the availability of software like FL Studio showing up on Android, programming techniques and the use of OpenSL can mean better performance on the platform, as well.

I’ve endorsed it before, but even if you don’t intend to use the free libpd library, you can check out the repository for the latest best-practices code for you to copy and paste in your own Android audio apps:

github.com/libpd

Check out the OpenSL branch.

So, which Android should you get (if any)?

This time last year, we heard about low latency support for specific devices on Android. How are they doing?

Well, frankly, not great. There are three devices that support the low latency profile, up from one device at Google I/O last year:

1. Galaxy Nexus (Samsung)
2. Nexus 4 (LG)
3. Nexus 10 (Samsung)

Certainly, I would only recommend these three Android devices to anyone interested in music or development. Other Android gadgets will perform more poorly with audio latency, including popular devices like the Galaxy S series. You’ll also want a Nexus device in order to have unfettered access to OS updates from Google – useful to developers, but also to anyone wanting to keep pace with improvements.

And of these three, the Galaxy Nexus remains the best choice. Sources tell CDM that the Nexus 4 and Nexus 10 don’t perform as well as the original Galaxy Nexus because of battery conservation concerns.

I still strongly recommend iOS for developers and end users, however. For now, Android can’t match app selection, API quality, availability of MIDI support and wired connections to instruments, or audio performance and flexibility available on iOS.

That said, a Galaxy Nexus could be a great buy for people willing to tinker a bit. Oh, yeah, and there’s the fact that you can get it unlocked for a fraction of the price of a new iPhone.

CDM will have more on these issues over the summer, so stay tuned.

Developers, details from this presentation:
https://code.google.com/p/high-performance-audio/

  • Mark Bryant

    It’s the bit at the end where they say they have to deal with their partners to get them to fix the kernel hardware and driver issues that you know it’s just going to be an ongoing problem. The only way it would work is if they only certified a device to run Android if it met a low latency audio requirement but I’m not sure if they have that control. And the fact that Android has already gained the massive market share with this audio issue makes it less a priority. It’s sad that there is no real competition to iOS at the moment. Windows RT has it’s own low latency audio problems (and no MIDI API). I haven’t seen any information on whether it is any better on Windows phone 8.

    But with the move to the full Windows 8 on tablets and the newer low power CPU’s from Intel this is looking like a good alternative. Just need the DAW’s to implement touch screen support like FL Studio and Sonar.

    • http://pkirn.com/ Peter Kirn

      Actually, Windows is ahead in this regard – there are low-latency prioritization options on RT. Windows Phone, not sure about that, actually.

      To me, though, the price/performance ratio is better on the full Windows tablets and ultrabooks and so on – and we’re seeing lots more alternatives. I guess I’m obligated to take a quick spin through the Computex floor while I’m in Taipei next week, as they’ll all be gathered there. But short story, yeah, if developers can work out how to make use of the touch input, then it’s all interesting.

      FL Studio, SONAR… meh. Their UIs still are designed around a mouse, so the “support” there is overstated. (You get … something. I tried it on a Surface Pro, and it’s really painful.)

      Usine is interesting. Other than that, this depends on more creative development.

    • MatthewWillox

      FLStudio 11 seems to at least be plotting a course into touch land.

  • Peter Brinkmann

    Minor update: I merged the OpenSL branch of libpd into the master branch a little while ago, so the master branch is all you need.

    Also, I’m currently working on a revision of the OpenSL support that’ll use some ideas from the I/O talk. Stay tuned!

    • http://pkirn.com/ Peter Kirn

      Ah, excellent. I will. ;)

    • Peter Brinkmann

      I just created a new experimental branch of Pd for Android called semaphores: https://github.com/libpd/pd-for-android/tree/semaphores

      It’s based on some ideas from the I/O talk, applied to input as well as output. I haven’t yet had a chance to test it much, but it seems to be working nicely, with improvements to both robustness and performance.

  • Ryan Dean

    No wonder this is still an issue when they work on this in their spare time and have to justify the cost of a scope.. Unbelievable!

    • http://pkirn.com/ Peter Kirn

      Um… I think that’s true everywhere to varying extents or on different issues. I wasn’t shocked by that.

    • foljs2

      If you expect Android to catch up audio API wise, you should be.

  • newmiracle

    I haven’t had a chance to put Linux/Ubuntu/GNU options on any of the unlocked cellphones that are compatible. That is also possible on the HDMI “TV Sticks” you see floating around. Dev boards like the Beagleboard, too. My Raspi didn’t handle PureData very well, but I haven’t updated to the newest OS version or tried other USB DACs.
    It really is a shame that there aren’t many easy options that allow for open development, low latency, and portability.

    But that being said, could Linux be the way? Anyone with any success in this area?

    • http://pkirn.com/ Peter Kirn

      Check out extensive discussion on the Pd mailing list of how to make RasPi work well. Maybe that needs a wiki.

      Trying to run Linux on these devices is unlikely to solve your problems, because of the interaction of drivers and firmware.

  • Justin

    Is anyone aware of apps that are utilizing Sonoma Wireworks low latency solution?

    • http://pkirn.com/ Peter Kirn

      Sonoma needs to be adopted by OEMs in order to work; it’s a vendor solution rather than something for developers. I didn’t mention it for that reason – I know of no licensee, so at the moment it isn’t really relevant.

    • Justin

      ah ok. i wonder if OEM’s will pick it up or not… at any rate that was a great video!

  • rockridge98

    Last time I checked, there were no android phones that recorded in stereo. Are there any now. I hate having to carry a flash memory recorder along with my Galaxy 3 when all the iphone owners have lots of choices.

  • Rob Fielding

    If you want to measure the real latency as experienced by a user, run electronic out into L channel of a mixer, and a mic on the glass into the R channel of that mixer. This includes touch latency, app latency, buffer latency as a whole; and it is what the user really experiences.

    • lala

      this

  • lala

    im not in the mood to watch a 40 min video to hear only half the story of androids latency issues

  • Petter Wiik

    Hm, what do you mean by “a fraction of the price of a new iPhone”..? Here in Norway a Nexus and iPhone 4S 16 GB costs about the same, both new and unlocked. (I checked it now: Nexus: NOK 3964.-, iPhone 4S 16 GB: NOK 4090.-). Thats a difference in around €20. A small fraction, yes.

    • http://pkirn.com/ Peter Kirn

      That’s surprising. In the Eurozone, you’re looking at around 300€ for a Galaxy Nexus, or twice or more for a new iPhone. Equivalent pricing in North America.

  • RichardL

    I’ve had two Verizon Galaxy Nexuses (Nexi?) that have had issues with battery life, charging and sleeping. I currently run the Cyanogen Mod RC build on it as my main phone, but I’m having a hell of a time keeping it running for more than about 5 hours. My suspicion is it’s just some wonky floating voltages on the power/USB lines. I don’t know why I’ve had so much trouble with this device because others report good results with this phone.

  • krslick

    Peter, are those 3 Android devices still the only ones that support low latency? How does one determine whether it’s supported on a given device, for future reference?