Yes, I admit it’s getting better. And that could mean more choices for creative music software makers. Photo (CC-BY) Pittaya Sroilong.

Android 2.2 boasts enormous boosts to performance in Java, JavaScript, and the browser, plus nice end-user features like tethering and tons of developer goodies. But developers interested in pushing the multimedia capabilities of the OS have been eager for some specific good news – particularly with Android a candidate for tablets and new embedded platforms (think new, Android-powered DIY music gadgets).

2.2 isn’t likely to satisfy all those concerns, but it is a step forward. There are some subtle but key aspects you might miss, since they aren’t quite headline news for most gadget sites. Apologies for an atypically technical post, but this stuff is important if the platform has a future for readers of this site. And if anyone doubts this is news, let me tell you – talking to music developers from a variety of backgrounds, I hear both immense desire to look at Android, and some significant skepticism about the limitations, many of them specific to performance.

What’s improved:

Audio gets a separate priority. Changes to AudioManager mean that audio can gain focus. Currently, audio processes on Android often get preempted by other processes, so that literally, another service syncing data can screw up your sound. What I can’t tell here is whether the audio focus helps successfully prioritize sound – or if it just helps mix sound with other apps. At the very least, it should avoid other apps cutting into a music performance app without you wanting that. If we’re lucky, Google has also improved audio performance, so that audio apps can set shorter buffer times without adding clicks and pops. (I’ve found disabling data sync stops sound from skipping, but that’s not how it’s supposed to work – not even close.) It’s possible these issues could be positively impacted by improvements to the Java virtual machine (in fact, it’s almost a sure thing).

Android 2.2 blog post, full revisions; NDK changes on the NDK site.

Media recording APIs are finally set up right. As Google puts it, “New APIs in MediaRecorder for specifying audio settings for number of channels, encoding and sampling rates, sampling rate.” Or, as I’d put it, less charitably, “MediaRecorder API no longer involves pain.” This is also a big deal for Android applications that take audio input, including in-progress ports of free synthesis environments Pd and SuperCollider, and Jasuto, whose developer got tripped up on this very problem.

MotionEvent has been improved, for better multi-touch event handling. Developer Luke Hutchison has had to manually write code that works around some unreliable multitouch processing. Google promises better reporting in the new version. Unfortunately, I don’t think we’ll know until this build ships; this can only be tested on devices.

Better storage. Flexibility in storage is a key advantage of Android (cough, Apple) in some areas, but needed work. Now, the ability to (finally) install apps to the SD card means that music-rich apps or interactive music albums become possible. And automated data backup could be a boon to people using Android as a creation device.

A smarter NDK. Debugging. ‘Nuff said. And that’s something native developers (see, again, SuperCollider and Pd) can download and use right now, today.

By the way, motion fans should be happy with doubled camera preview FPS and YUV support in OpenGL, among other tweaks.

Now, the bad news.

What’s Missing

JIT (Just in Time) compilation for Java translates to vastly improved performance, so you can’t wish for more. But there are still things musicians could use.

Everyone’s been writing open letters to Steve Jobs. Let’s make this an open letter to Google.

Why should Google care about musicians? How about because Google threw down the gauntlet today on its comparative “openness,” compared life with Apple devices to 1984, and prides itself on its Linux heritage?

No, actually — let me put this better: don’t you want Android to be a rock star, Google?

  • Native audio access. This NDK added still more support – native access to image buffers, which will help people writing graphics apps. But with that and OpenGL finished, audio should be next on Google’s list. Android devices all appear to use Linux audio standard ALSA. There’s really no reason native DSP code shouldn’t talk directly to the audio output. Google: pay attention. Audio apps have been some of the biggest hits on Apple’s App Store; Smule alone has made ocarinas, AutoTune, and Glee fandom cultural phenomena on the iPhone. Android really does need improved audio performance to compete. Unless a miracle has happened on the Java side, that means providing the NDK as at least an option. And it’s self-selecting: the only programmers who would even try to write their own DSP code and ALSA interfaces would be the ones who already know what they’re doing. You’re not going to get stupid questions about this on IRC like you do with people who haven’t read the UI documentation. Get it?
  • Reach out to better multitouch hardware partners. I’m beginning to think that waiting for OEMs to stop sucking at multitouch on phones and tablets is a losing game. So, Google, you’ve got the biggest tech brand power on the planet now. You’ve got smart people. Find a way to hook up your mobile partners with the people who can make touch hardware and firmware work, and the whole platform wins.
  • Hardware support (question mark). The Droid already supports USB host mode – so why isn’t it a standard? And why shouldn’t Android benefit from the Linux kernel and provide external hardware support in the API? Help the device live up to the “open” hype you keep espousing; it doesn’t make for a flattering comparison if the iPhone OS has more hardware features for developers than Android, especially with tablets on the horizon. So why a question mark? It looks like good stuff is happening here. Tablets show promise, and the announcement of Google’s TV product suggests not only video out but USB, too. So the key is, will developers be able to use those features? It’s not really an “open” platform if the answer is no. It just seems at this point like we’re waiting on standard APIs and documentation. TV video out is a safe bet when Google’s promised TV SDK appears early next year. But by then, Apple may have a similar offering – and it’d be unfortunate if Google didn’t extend capabilities to their whole line, rather than slice up the platform.

Specific as these things are, they could be the detail that makes a (pardon the word) “magical” app for the platform. And hey, that’s also mean some rockstars using Android. That can’t be too bad, can it?

I’m hopeful. And I think the vision of the platform could be extraordinary. Imagine an Android phone that connects to a music rig onstage, an audiovisual app that makes the audio output really shine, or an interactive album you can watch on an Android-powered TV accessory from your couch? On that last question, imagine people listening to albums in their entirety, blissing out to specialized generative visuals? (Return of the psychedelic prog rock album cover!)

Users, take all of this with a grain of salt, because I still want to test 2.2 to evaluate real-world performance. But it’s worth saying, because the fact that we have 2.2 means Google’s talented Android team is already moving on to the next thing.

Updated – More from I/O, and Native Audio Access

Here’s one major difference about developing for Android – developer events aren’t under an NDA, and you can find out about what’s happening to a platform in advance and respond. You can share information, follow open bug reports, and see as they’re changed. This shouldn’t be rocket science; it’s certainly a given in the free software community. But it is a marked difference relative to something else beginning with ‘A.’

Accordingly, while I couldn’t be at I/O, I do get to talk about it. And notes from a session on Advanced Audio Techniques suggests the 2.2 changes to audio are significant – and are accompanied by an upcoming hook for doing native audio.

From the roadmap notes, OpenAL and OpenSL may be the ideal native hook to the audio buffer. OpenAL itself leaves a bit to be desired for musical applications; it’s a fairly primitive API, not really a direct analog to OpenGL. But if well-implemented on Android, particularly in how it talks to the audio output, it could solve many of the problems Android users now face; it’s all in the implementation. What’s coming:

- OpenSL native API to provide audio track stuff in native code
- OpenAL support. Has a shim over OpenSL. should help port games.
- Add autio effects processing
- Expose more low-level APIs.. a Java api that lets you build the player graph in your application. If you have a streaming support that we don’t support, previously there hasn’t been access to codecs.
- WebM support. VP8 + Vorbis
- FLAC decoder
- AAC-LC encoder for device that don’t have hardware support
- AMR-WB encoder
(encoders will be open sourced)

You can read those notes from the session here, thanks to an attendee; I’ll post a link once this goes up on YouTube:
https://wave.google.com/wave/#restored:wave:googlewave.com!w%252B3kgmObZwQ

Via Bug 3434

Also very awesome – hardware-accelerated processing.

NDK FPU and SIMD NEON support means that Android applications can now take advantage of the floating-point and specific hardware acceleration features of the ARM-based hardware architectures underlying these devices. If there was any doubt that Android could become a major solution for embedded musical hardware, this should change that. (The iPhone/iPad will reap some similar benefits, so this is as much about the iron underneath as it is the OS.)

ARM claims 60-150% performance gains as a result. (Thanks to Martin Roth for the tip!)

I expect I’ll be talking to developers more at Droidcamp this coming week in Berlin.

  • electronic_face

    Awesome. I can't wait till Android tablets hit the streets. I'll admit, I've been tempted to get an iPad for its music apps, but then I realized..

    1. I don't agree with Apple's way of doing things (covered by previous posts of yours), and I'd rather not support a company that operates in such a manner, and

    2. I feel like Google is the tortoise in a race against Apple, the hair– slow and steady wins the race, and I would happily support a forward-thinking company like Google.

  • http://naimfalandino.com Naim Falandino

    Peter: As you probably no doubt guessed I jumped up and clicked my heels together when I first learned that the new NDK has better debugging support. :) Incoming pdportable code-a-thon this weekend!

    I was hoping that they'd throw in some support for a native audio API, but no such luck. As we discussed before it seems that Google is asleep at the switch when it comes to this issue, and the longer they put it off the harder it will be correct…

  • http://audiocookbook.org keston

    Thanks for a great article, Peter. You rounded up all the information I'd been seeking about 2.2 and compiled it for us in one place. I'm excited about all the new features I've been reading about. Perhaps our visions of this platform for music apps will be realized sooner than we thought.

  • http://www.numblog.de/ Matthias Gutjahr

    I'm only just beginning to develop for Android, but it's very exciting. Unfortunately, I cannot make it to DroidCamp, although I thought about going there. Looking forward to your reports from Berlin :)

    And I hope HTC is already working hard on an update for my Desire!!

  • http://mediawestpost.com mediawest

    the latency of these devices is still huge, and not really good for midi or vst. but it is a good 2 track recorder, but the internal mics are only fair…… any iphone app i have seen has a latency of over 15ms and some more…..

  • Emile V

    I agree completely with Keston! Thx

  • Jesper

    This is exciting news! Although since I have no experience in programming or developing apps I must admit that I dont fully understand what is happening.. Im thinking of buying a Htc Desire but the musicapps (and poor soundquality) has got me thinking of getting me the new coming iphone instead (although i really dont like apple). My question is: will these new changes in android make it possible for new apps like for example a four track multirecorder for android coming out anytime soon?

  • Jay

    Jesper, I'm in the same boat. I don't really understand the technical side of things but I am excited. I have a 3GS and the music apps are abundant and amazing. However, I am very sick of Apple and their increasingly tyrannical attitude. I want to move to Android but I get so much from the music apps on the iPhone, I'm not ready to leave yet and play the waiting game on Android.

    If a FourTrack and a Thumbjam came to Android right now though, depending on how they sounded, my 3GS would probably be on sale tomorrow and I'd just wait for the Beatmakers, the NLogs, the iRigs, etc.

  • niko

    What you have to also realize is that audio latency is not the only issue here. Android apps are also limited in the amount of memory they are allowed to allocate, depending on the device. Low end devices can only allocate up to 16MB before the OS will shut them down. You can allocate more inside the NDK without that restriction but you still have to be careful. The point is that the iphone does give you more memory to work with.

    With the new 2.2 speedups though of having a JIT, you wont have to write time critical code so much using the NDK, which makes it more complex to debug, etc.

    For example, my ReLoop music sequencer for android sounds great since it's 44Khz 16 bit stereo, but it can only achieve this without bogging down because I'm running NDK code written in C and using all fixed point math. The Drum machine only outputs 22Khz mono since all its sound code is in Java. (And the Electrum Drum also has more limits on the size of samples you can load before you run out of memory because its in java)

  • niko

    I guess the only other thing I have to say as well is while I do support getting more features into the NDK, it also worries me as an independent developer, that if it becomes "too easy" for people to port their code over that I'll get stomped on by the big names in no time flat….. I know, it's just part of the business territory, but I've liked that fact that the little guys as least have a chance yet in the market. When it does happen I will accept it but I dont think its fair IMO if a company can just squash everyone quickly on every platform without having to do hardly any work….they should be required to do a little something…lol.

  • http://www.createdigitalmusic.com Peter Kirn

    @niko: I wouldn't look at it that way. First, if the thing is easier to develop for, it'll be easier for *you* to develop for. Second, it's not quite as simple as just "porting" everything via the NDK. And lastly, if you've got something cool, you know I'll happily, happily stick up for the "little guy." I think Android is going to evolve very differently from iPhone, regardless.

    But I hear you on these other restrictions, for sure!

  • Pingback: Android 2.2: Badly-needed Improvements to Audio, Touch, More; What’s Missing (UPDATED) | VJ Heaven

  • dubremix

    I don't trust Google. Why the hell does everything think they are so pure? They make more money than most countries and they've already been caught spying on people.

    They want to control everyone just as much, just in their own different way. How much data collection are they doing? They will know more about you than your mom.

    Also, the slimy Google CEO sat on Apple's board for several years and then leaves and wow, surprise they copy all Apple's products. First the phone, then the TV, and next the tablet. Talk about two faced. Look if you sit on company's board and the whole time you are there you are gathering intelligence to later launch your own products that compete, then you're a scumbag. I don't trust them. Apple did, look how they stabbed Apple in the back. No wonder Apple is so paranoid.

    Google is going to screw everyone, just wait. This is just a setup.

  • dubremix

    http://www.apple.com/pr/library/2009/08/03bod.htm

    CUPERTINO, California—August 3, 2009—Apple® today announced that Dr. Eric Schmidt, chief executive officer of Google, is resigning from Apple’s Board of Directors, a position he has held since August 2006.

    “Eric has been an excellent Board member for Apple, investing his valuable time, talent, passion and wisdom to help make Apple successful,” said Steve Jobs, Apple’s CEO. “Unfortunately, as Google enters more of Apple’s core businesses, with Android and now Chrome OS, Eric’s effectiveness as an Apple Board member will be significantly diminished, since he will have to recuse himself from even larger portions of our meetings due to potential conflicts of interest. Therefore, we have mutually decided that now is the right time for Eric to resign his position on Apple’s Board.”

  • dubremix

    So in closing to the people bashing Apple and praising Google. Do you think Google's CEO who acts like this can be trusted? He's already shown that he secretly develops competing products while sitting on another company's board. WTF was he just there to take notes and spy on Jobs so he could copy off Jobs' homework?

    You like that sort of behavior? You think that's okay? I don't, because it means he cannot be trusted. Google's is not good. They're getting pretty freaking evil, wake up.

    Are you blind to the Google empire? They are 10X bigger than Apple. They already own most of the web, next they'll own your phone, TV, Tablet, PC and everything else about you.

    About being open? How many severs does Google have? Private. You can't find out what they know about you. They won't reveal what they've cached about you. They are opaque in a number of critical ways.

  • http://thecovertoperators.org Andreas

    Yeah, who can even trust a guy who decides to do a phone? Or a computer? He should be working for a big tech company that doesn't design anything that Apple designs… *groan*

    Honestly, any company is in it to make money, and the Android OS *is* a great thing, and I really do hope they go the extra mile on this in terms of audio capabilities.

    So, what's the current status of PureData on Android? I'd love to be able to design RjDj-type things on a damned phone, that would be killer.

  • http://www.myspace.com/noou (noou)

    I'm looking forward to having a next-gen Android phone, able to run Pure Data and with sound floating point capabilities… Maybe next year will see the coming of such device?

  • HEXnibble

    In case some of you didn't know, you can already install Pure Data on an iPad, iPhone, or iPod-Touch. Creating and uploading your own sound-processing and sound-generating patches is as trivial as copying a text file to your device. And because Pd patches are treated as media files, they don't have to go through Apple's elaborate code review, so you can just dive right in, turning your phone into a pocket synth within minutes.

    Check out pdlib, an open source port of Pure Data to the iPhone:
    http://github.com/bryansum/pdlib

  • http://www.myspace.com/noou (noou)

    @HEXnibble: thanks for the info! I was only aware of RjDj.

    The ipod touch is tempting indeed…

  • http://www.createdigitalmusic.com Peter Kirn

    There's no reason Pd shouldn't also soon run on Android, and both iPhone and Android now support FPU in a way that will work with Pd. I expect we'll take a look at pdlib to compare to what we're working on with the Android port. Part of the hold up is that Pd's code base isn't terribly easy to navigate — but that's why ports like this can be valuable, even if you don't use them directly.

    Unlike iPhone, of course, Android support also means a much wider variety of supported hardware. I don't know that initially we'll get the same *low-latency* audio performance for the reasons above, but that's feedback that will definitely reach Google, as well, and could improve.

  • Evan

    Lookin through the changes it looks like they at least added an interface to notify of sound load completion. This was one major major issue I saw with 'soundpool', which is the low latency audio playback and multi-stream playback section of the sdk. Earlier you would tell it to load a sound to memory and then have no idea when it was complete, basically forcing you to just sleep for a long time and hope it actually finished. Now there is something to go by for loading

  • http://www.document02.com Document 02
  • http://leisuresonic.com/ Christopher Penrose

    The fact that realtime thread support for audio callbacks hasn't been present in Android since beta sdk releases shows that they just don't get it. Audio is often hosed by committee. Again they just don't get it. William Stewart gets it and I am glad that he works at Apple.

  • Pingback: Womp Truck, MIDiPad (not), notes, and optical compression expanded « Denver Ableton User Group

  • hare

    electronic_face: "2. I feel like Google is the tortoise in a race against Apple, the [hair]– slow and steady wins the race, and I would happily support a forward-thinking company like Google."

    hair would be faster if blown by the wind, but might travel a different direction

  • http://www.myspace.com/noou (noou)

    @Peter

    sorry for the naive questions:

    - is there any chance that the work done for PDa http://sourceforge.net/projects/pd-anywhere/ or similar old projects can be re-used for Android-based devices?

    - since rjdj is being ported to Android, there must be a way to get Pd fully working on such devices, right?

    - on the other hand I cannot find any video examples of Pd fully running on a iPod/iPhone… is it true that the gui isn't currently working?

    the bottom line is that i don't know whether buying an iPod touch or an Acer Liquid…