Adding hardware to tablets, as it has with decades of computing technology, can open up new worlds for software and music. It can animate a conventional piano, or provide new physical interfaces for touching music. But let’s not wait for it to happen; let’s get hacking.

Following on today’s line of thinking about hardware-augmented touch, I’d like to look a bit at the recently-transformed landscape on Android. iOS users can connect to external hardware via the Core MIDI protocol or, via official channels, through the Apple Dock Connector. That’s not a perfect situation, however. Hardware developers have to be approved through the Made for iPod program in order to make accessories, there’s no standard class support apart from MIDI, and while Bluetooth can connect you to other iPads, you can’t connect arbitrarily with hardware via Bluetooth. (At least, that’s what I’ve found speaking with iOS developers, though some Bluetooth gamepads appear to work, possibly because they simply act as keyboards). WiFi wireless connection is also a possibility, one I expect will continue to be exploited.

But Android does have some interesting options here – a reminder of why getting familiar with more than one platform can be enlightening. A lot of those possibilities could open up new ideas in music hardware, ideas that could work not only on Android but in some cases (as with standard USB support) with desktop and tablet Mac, Windows, and Linux machines, too.

Case in point: check out a Motorola XOOM controlling a piano above, for a kind of hacker’s Disklavier. (Apologies to Yamaha.) Yes, it’s a great deal messier than existing USB MIDI devices (more on that in a moment), but it’s a visual reminder of why we do this in the first place – and it doesn’t have to be quite this hacky on Android, either.

Simplified, your options are:

Android Open Accessory

Currently the best wired common denominator, the Open Accessory program allows you to use a third-party accessory as the USB host, even on devices that lack USB host support. You just need a device with Android 3.1, 2.3.4, or greater. In a much-ballyhooed feature, there are prototyping possibilities with the open Arduino platform. Google’s own hardware is obscenely pricey, though, at around $400. Instead, DIYers will want to use a standard Arduino. Two early examples:
Android ADK with a standard Arduino Uno and USB Host Shield
Google’s open accessory development kit on standard Arduino hardware

These prototypes could, in turn, be converted into simple kits or even standard off-the-shelf accessories, without asking anyone’s permission.

See a video at top for one of the Arduino Uno examples in action.

Also, for the actual code, look at:
UsbAccessory

More on DIY implementations that don’t require $400 hardware kits (more like … $40, tops):
http://developer.android.com/guide/topics/usb/adk.html

Microchip also promises an open PIC24 library with a driver for ADK

That said, because Google’s implementation is specific to Android, those accessories aren’t useful anywhere else. That’s why it’s worth considering USB host mode.

USB Host Mode

Tablets are the Android devices about which I’m most excited at the moment, because as with the iPad, the tablet devices wind up being more flexible and capable than their phone handset cousins.

One key feature: Honeycomb tablets support actual USB host mode. That means you can connect standard USB HID devices like joysticks. It also means you should be able to make a MIDI interface that isn’t kludged together from a bunch of proto boards and $400 Google Android kits and that works with Android but not your computer, as above.

In fact, the existence of this option made me a bit puzzled when I read Phil Torrone’s* editorial in Make. (See postlog, though, on why listening to Phil is still a good idea.) Phil focuses there on Arduino, and doesn’t mention standard USB host.

Why Google Choosing Arduino Matters and is This the End of “Made for iPod” (TM)?

Now, Phil’s correct that the Arduino is currently the easiest platform for hacking with this stuff. But I actually think even that could change. There are already a number of AVR-based platforms for doing USB host implementations. Like Arduino, you can develop for them with free, open tools and a wide community. Unlike a standard Arduino I/O board, though, it’s easy to create something smaller, cheaper, and more flexible – and to plug your creation into any device that supports standard USB hardware. Now, actually implementing some of these things is nowhere near as simple as Arduino at the moment, but that to me is a testing and documentation problem more than anything else. I’m bullish on the possibilities here; it’s part of the reason we chose a standard AVR platform for our MeeBlip synth and not Arduino. I’ve started messing with its firmware, finally; more on that soon – it’ll also be relevant to playing with DIY hardware for Android.

The Arduino stuff is absolutely cool, don’t get me wrong. I just think you could substitute “open hardware” more generally in that article for “Arduino,” specifically. (The article’s nonetheless well worth a read for some stimulating ideas.)

Executive summary: I betcha we can come up with a really simple DIY MIDI interface, via both wireless and wired connections, for Android that doesn’t touch the Arduino side of things and will work with your computer, too.

See:
android.hardware.usb in the Android documentation

IOIO – a Chat with the Developer

Following CDM’s coverage of the Google announcement of the ADK in the first place, a number of commenters pondered the inexpensive, DIY IOIO board marketed by Sparkfun (pictured).

Developer Ytai Ben-Tsvi shares some thoughts about how IOIO relates to the ADK:

First, you can see some of my (and others’) thoughts on this thread:
https://groups.google.com/d/topic/ioio-users/3JDn6XZJ9aE/discussion

Aside from what’s written there, I have immediate plans for implementing IOIO on top of ADK. This means that when IOIO is connected to an Android device, it will first try to establish and ADK (accessory-mode) connection with it and if that fails, will seamlessly attempt to fallback to ADB. ADB will still be used for firmware upgrades, as I believe this is the most secure way of making sure the user is involved in the decision to update the firmware (via enforcement of a single signed application that IOIO agrees to talk to).

In the (hopefully near) future, a similar behavior will be achieved by connecting a Bluetooth dongle to IOIO instead of an Android device, enabling a remote connection between Android (or even the Android emulator!) and IOIO, on expense of reduced bandwidth and increased latency.

Some points to emphasize:

  • IOIO is targeted at the hobbyist/educational community. I believe ADK is primarily targeted at serious hardware developers. As a result, in terms of support, design considerations, feature prioritization, ease of use etc., IOIO is more likely to stay loyal to the hobbyist community in the future.
  • At the time of writing, IOIO seems to be the cheapest solution for communicating with an Android.
  • IOIO currently provides a high-level Java API which hides underneath it a rich protocol for controlling many of the board’s features from Java. “Mainstream” users will never have to touch embedded programming when working with IOIO. My hope is that this API will eventually become a standard, and that people will implement it on top of other boards as well. If that happens, application developers will be able to easily port their applications to different hardwares.

IOIO is available now for US$49.95. Honestly, if you’ve got a phone that it works with, it’s a fantastic choice.

IOIO @ Sparkfun

Bottom line:

At first it seemed like there is conflict here, but now it appears to me like there really isn’t! IOIO is an end-to-end solution for physical computing (a-la Arduino) from your Android device, ADK is a protocol for enabling you to connect peripherals to an Android device, and this is Google’s motivation in this game. The boards that came with ADK are just reference implementations, and to me they seem in no competition with IOIO at present. So IOIO and ADK will probably be good friend in the future, when ADK provides the underlying connectivity and IOIO provides the high-level platform for the hobbyists.

Updated: There’s already good news.

Since he wrote the comments above, Ytai got everything working with IOIO and ADK:
IOIO over OpenAccessory (ADK) Available [Ytai's "Microcontrollers, Electronics & Robotics" Blog]

He makes a strong argument for why IOIO may be your best solution. In fact, for my money, IOIO plus USB host development is just about perfect.

Bluetooth

Enough about all these wires: you do lose some of the beauty of these mobile devices when you attach a lot of spaghetti to them. Enter Bluetooth. It’s reasonably cheap, light on power consumption and well-suited to embedded hardware (key differences with wifi). It’s also gotten steadily more robust. For wireless MIDI and wireless control, it really can work.

On Android, it’s possible to interface directly with hardware over Bluetooth. That opens up the chance to do, for instance, MIDI over Bluetooth without a wire in sight. For lots of thoughts on this, the best reading is Peter Brinkmann’s blog. (Peter is also the primary author of libpd.)

I hope we’ll get to talk more about this soon. Ahem.

http://nettoyeur.noisepages.com/

I’ll admit: I’m actually kind of a sucker for wires. So I’m glad to have Peter around to encourage me not to create giant, ugly tangles of crap.

Postlog – Phil Torrone and Mobile

I just want to add one background note on Phil Torrone.

Phil deserves loads of credit for seeing this stuff coming long before it was popularized. Here’s what he had to say in 2002:

A simple hiking stick was way too low-tech for Macromedia Flash expert Phillip Torrone.
So Torrone equipped his stick with GPS and Flash applications that let him identify trails and track information on local flora and fauna — and leave messages for other hikers.
He’ll show off the hiking stick and more of his quirky creations, such as a digital belt buckle made from an old Palm V, when he discusses wireless fashion at a Flash conference here Friday.
His inventions are sure to delight his geeky audience of developers and designers at FlashForward. But his audience will start taking notes when he brings out mass-market, Flash-enabled devices such as a Microsoft Pocket PC Phone Edition and an O2 XDA, two Pocket PCs that are also phones.
Flash developers believe mobile devices like these are the wave of the future, Torrone said. “Last year, wireless phones outsold PCs for the first time,” he said.

Lots of Flash, Even More Sizzle [Wired, July 2002]

In music, we have the opportunity to test these same ideas not only for a quick geeky hack, but as a real means of self-expression, in deeply emotional, intimate ways.

And just remember, for anyone caught up on platform debates, at some point even words like “iPad” or “Galaxy Tab” will seem as distant as “Palm” and “Pocket PC.” But the ideas behind actual work for those platforms will remain. (Oh, and… I guess we still have Flash, huh?)

Now … who wants to do some hacking this summer for music? (And science!)

  • Leslie

    What about the infamous Android latency problems..?!

    It hasn't been addressed in the latest Honeycomb 3.1 release and personally I don't think it will be any time soon…

    No wonder Devs do not want to port their Apps from iOS. What a shame… :(

  • Peter Kirn

    @Leslie: This is practically going to have to be a FAQ:

    Not all music apps need extremely low-latency performance.

    Not all music apps need audio I/O.

    Latency on Android, as on many platforms, is partly a function of audio chipset and drivers, not the operating system.

    While some of us have complaints about the way the audio APIs work for all cases, we've found effective workarounds — those workarounds by Peter Brinkmann are shared as open source code in the libpd repository (though usable by other applications).
    http://gitorious.org/pdlib

    Android application development is a robust scene; music developers, though, have limited resources, and a primary obstacle has been marketability of their apps, not just performance.

    It's early days. There are already some amazing tools on Android (Pd, SuperCollider, Nanoloop, Jasuto, MilkyTracker)

    Part of why it's worth considering more than one OS is better understanding and criticizing strengths and weaknesses on each. Android is more open to hardware connectivity than iOS as a rule (thanks to recent advances); iOS has a better audio API and more predictable hardware because it's limited to a few devices from the same vendor.

  • Pat

    In my opinion, we need to move away from the us/them,PC/mac, android/iphone differences when honestly the answer is BOTH. I have recently purchased a motorola xoom and honestly it reminds me of 1994 and linux

  • Leslie

    @Peter

    Your statement in regards to latency issues with Android is incorrect. This is a well known fact not a myth and Google is refusing to address that. This is also the reason that there are no apps like NanoStudio for example on Android OS. This is the statement from rather excellent Caustic app dev:

    "I've learned to live with the latency issues on Android, and most people now are nice enough not to blame developers for it. We can only keep pressing Google to address the issue."

  • Peter Kirn

    I didn't say it wasn't a problem. It's just not a one dimensional problem. And improved performance would require effort from both hardware vendors and Google. As for Honeycomb, again, because hardware comes into play, we just haven't had enough time to test. But the questions above are relevant, too. I'm doing my best to provide technically accurate, complete info on all relevant music platforms.

  • Leslie

    Oops, forgot to provide the link to Caustic app…
    http://causticapp.blogspot.com/

  • Peter Kirn

    @Pat: Read the last paragraph of my first comment above. That's why I try to cover as many of the platforms and details of those platforms as I can.

    @Leslie: The best way to describe latency is as the end result — like the "handling" of a car. You've got lots of factors that contribute to that. That said, I'm interested on Android in potentially doing some sort of standard latency testing app.

    But this wasn't even a story on audio performance – not all music apps even have to make sound, since they might connect to something else that does. I'll talk more about it when there's more to say.

  • Leslie

    This thread is almost 2 years in the making and still no candy..
    http://code.google.com/p/android/issues/detail?id

  • Peter Kirn

    That thread is full of misconceptions and inaccuracy. Read my above comments.

  • http://www.fladd.de fladd

    Well, if it is not the software, but the hardware: Is there any android phone that has a low latency? The Galaxy S2 maybe? Or the HTC Sensation?

  • GD

    This is all so difficult to grasp. I just want to connect a Mototrola Xoom to my main DAW via USB, built a GUI and start controlling synth parameters and stuff like they did on that jazz mutant thing years ago… yet i do not see much happening at that front… or am i wrong… ?

  • http://www.submitresume.in/ Submit Resume

    Quite amazing technology on android piano player. I was looking regarding the american free downloaded songs.

  • http://www.creditcardloginfinder.com/ LOGIN

    Quite interesting app. I need to get it one like these one. I hope it will be available. Looking for more app for Android.