Yesterday, we saw a bit of the idea of making mobile gadgets more sociable with each other. The idea is, through network/wireless and cabled connectivity, you extend possibilities for expression, control, and collaboration with yourself and others. It’s the same thing that makes a recording studio useful: tools work together to let people work together to create music. It’s absolutely not a new idea; the engineering question is just making it work well on new platforms.

On iOS, we’re already seeing some of this: apart from MIDI connectivity, developer like KORG have even tried using wireless features intended largely for multiplayer gaming to connect multiple beatboxes and synths over a network.

Continuing a look at the Android side of things, Andrea Belloni sends an open source project that gets a sequencer working over a network with a computer and Arduino-based hardware project. It’s really a proof of concept – let’s call it a hack of concept – so they say specifically they weren’t concerned with making it pretty. It’s at an earlier stage of the process, but it’s nice to see that. The basic idea is to control the hardware sequencer, built on Arduino, with a Processing sketch running on the phone, all over a WiFi network. Take the same basic idea and streamline the hardware a bit (the Arduino and shield get a bit clunky), and this could actually be practical.

Android Arduino Sequencer [Italian + English]

But I think as an illustration of what I was saying in words yesterday, it’s worth covering this just so I have an excuse to use this picture:

Yep, that’s the idea. Add in some other operating systems to that graph, too, if you like!

Ready to cut the cord and go wireless? With mobile gadgets getting involved in music-making, it seems a logical solution – maybe not reason to throw away your MIDI cabling, but worth at least trying. Bluetooth could be an answer. In fact, it could work even without all those pesky, pricey mobile tablets and phones lying around, just with good, old-fashioned MIDI gear. (‘Bout time.)

Bluetooth and MIDI are a logical match; the big surprise is that these two haven’t paired off (cough) much earlier. We’ve seen the occasional implementation or paper or rant, but not much real-world usage. That could be about to change – that is, provided ample real-world testing.

Bluetooth mobile sync: First off, owners of Korg iOS software just got a Bluetooth-based update that provides sync. (See video below, shot by tipster and reader Danny Fluck.) Entitled “Wireless Sync-Start Technology,” KORG promises their Bluetooth feature now enables two iPads running iElectribe or iMS-20 to sync with one another with zero latency. (The update also includes perhaps more widely-useful features – SoundCloud support and AudioCopy, says Synthtopia.)

The name suggests that what the “sync-start” technique does is actually to match any latency between the two devices so that they start at the same time. That’s how multiplayer gaming typically works, and it applies easily to musical applications: you ping connected devices, then delay the start points of each of them to match the slowest device so that they sound together. (I think I’m saying that correctly and clearly; it’s Friday.)

Danny reports that it works perfectly and “instantaneously.”

Okay, so cool, as long as you only use iPads, and you’re lucky enough to have two iPads handy, and you only use KORG apps, you’ve got a nifty solution – but that’s a little limiting.

Bluetooth MIDI on Android or any hardware device. A bit further-reaching is something Peter Brinkmann, primary author of libpd, has been testing. (See video, top, which is much snappier than my explanation.) Now, Peter is no MIDI fanboy; when several of us talked about adding MIDI to libpd – an embeddable version of the open source patching environment – he described MIDI as a “plague upon humanity.” (I don’t recall seeing that quote anywhere on the MIDI Manufacturing Association website.)

But here’s the thing: MIDI is obsessively compact and simple, and absurdly easy to implement even on the cheapest microcontroller. That makes it ideally suited as a means of making inexpensive hardware inter-operate, and without using up lots of power or bandwidth – just as MIDI was designed to do.

Furthermore, as Apple tablets and phones demonstrated USB MIDI connections, it got Peter (and some of the rest of us) thinking. Yes, it’s too bad that Android devices lack key USB host capabilities that would let them talk to hardware. On the other hand, once you start attaching lots of cables to a device, you might as well use a (more powerful, more flexible) computer in place of the tablet. So low-power, efficient wireless – like Bluetooth – seems the way to go.

Like a good engineer, Peter went and hacked the solution he wanted himself. Partially inspired by Lady Ada’s own, well-documented Bluetooth MIDI experiments, Peter started documenting the process on his blog.

In his first blog post, he describes his hardware setup. (He later updated the rig and moved it off the bread board.

In part 2, he describes latency and jitter. Here’s what I’ve been told by mobile engineers to whom I talked: performance has greatly improved in Bluetooth implementations in recent years. That means that part of the reason Bluetooth MIDI may have been adapted is that, when people first began testing this a few years ago, the implementations weren’t yet good enough – and no one has checked since. (Until now, that is.)

Since then, Peter has released free software for the Android platform, all under a free Apache license, so anyone can try it out. And since it works natively with Pd for Android, this means you can very quickly hook up a Pd patch to Bluetooth support on any Android device. There’s even a sample patch and code to get you started!

As far as I know, Apple doesn’t let you implement a similar solution on iOS, so this would remain Android-only. (Hey, come on – iOS can’t have all the fun.) I’m curious to hear if I have that correct, though, so please do share.

The next step: refining the hardware rig and interface design and most importantly, testing. Correction: I claimed that the MIDI interface was not opto-isolated; it actually is.

Hacklab and testing: With that spirit in mind, on Saturday April 2 from 1p – 6p, we’ll have a free, open hacklab in New York. It’s mainly an informal get-together, but there will be short demos as we go. It’s the first step in broader testing and experimentation with these ideas. If you’re in the NYC area, you can RSVP on Facebook. (just went up, so it may be quiet) The plan is to play a bit with hardware and software and different Android devices.

All are welcome to the hacklab, regardless of experience.

This isn’t just an Android thing, though. Part of the reason to use Bluetooth in place of WiFi is that it’s much simpler and cheaper to implement, and has more modest power requirements. As such, providing Bluetooth MIDI interfaces for other music gear is more manageable than it would be with WiFi.

More resources:

A 2005 IEEE conference presentation describes early work, though this research could use a modern update.

A 2003 open letter encouraging Bluetooth MIDI use.

Meanwhile, in the Land of WiFi

Synthtopia reports on iOS MIDI WiFi, in the case of MoDrum and Bassline, using Apple’s own network MIDI framework. I’m assuming it claims it’s the most extensive such implementation because of the sync functionality.

SoundPrism Pro, in the trailer below, also recently added network (as well as wired) MIDI compatibility.

Incidentally, there are already compatible frameworks for Windows and Linux (in addition to Mac, of course), and no reason Android couldn’t also do WiFi MIDI, too. The advantage of Bluetooth remains doing so with less power consumption, and as direct serial communication over the wireless link.

More resources on the WiFi side…

From January:
New Solutions for Wireless MIDI, MIDI+OSC; Developers Answer Questions

And yes, devices like The Missing Link provide this kind of functionality over WiFi – though the Bluetooth radio could be cheaper and more power efficient. (As for performance, we just have to do more testing.)

Just don’t forget, all of this is a plague upon humanity and an abomination, so try to keep that in mind. Have a good weekend!

StepPolyArp for iPad controlling an Oberheim FVS from Logan Mannstrane on Vimeo.

Logan Mannstrane sends in this lovely video combining an iPad MIDI step sequencer — with an Oberheim FVS. It’s a striking intersection of analog and digital technology. But I wanted to ask Logan to explain why he’d use the iPad in this case instead of other MIDI tools — why crossing this generational gap mattered. He responds:

That fact that I can interface a synth from the 70′s to new device in 2010 is pretty amazing by iteself. I can have the sequencer in my hands and pull a chair up the window while laying back with my feet up it is very inspiring and a comfortable workflow. Also, for people that have multiple analogue synths scattered around, it is very nice to sequence the synth when you are right next to it. In a world full of DJ applications, rompler sequence programs, it sure does feel
good to have something fun and musical to try out. For a version 1 of the software, it is very neat. I heard more Midi apps are coming to the iPad in the future so this is a great beginning to wireless MIDI.

While TouchOSC is great, there are many people that want to start making music without having to spend a week of building an interface to talk with hardware and software alike. The StepPolyArp software was well thought-out and cleanly executed with a elegant interface. With Analogue synths you have instant control and feedback for designing the sound, and now you can step away from the mouse and chair to sequence. It doesn’t get better than that.

If you like the looks of this software and have an iPad, here it is:
StepPolyArp

It uses the free DS MIDI WiFi, a project born – as the name implies – on Nintendo DS. Viva open source.

There’s no question in my mind that MIDI remains lingua franca for interconnecting devices across the previous decades. Of course, that to me also suggests we need to make more progress on standardizing the way network MIDI protocols work.

Through the grapevine, I’m hearing the iOS SDK will incorporate network MIDI capabilities, but I think there are still some challenges there. Apple’s protoco,l while thoroughly standards-based and still essentially MIDI, is nonetheless for now specific to them. I’d also like to see some solid numbers on performance. New gear may want to investigate Bluetooth and not just WiFi, as Bluetooth could work nicely for embedded hardware, DIY synths, and the like. But it’s certainly an interesting time.

As a counterpoint, here’s Logan with a Pro One – no iPad in sight, just physical knobs. I think there’s something to be learned from the interaction design of each, and something unmistakably wonderful about the connection of hardware like this to sound and experience.

The infamous white faced Pro One from Logan Mannstrane on Vimeo.

For more Oberheim goodness, gaze into the glossy ads the company produced in 1981 and 1982 for Keyboard, and dream of the day when we enjoy electronic music ads again: Retro Synth Ads: Oberheim, at one of my new favorite sites.

With the iPad available now and more touch-enabled devices likely coming soon, we finally have hardware that can be both display and controller, visual feedback and input device. While tactile control will maintain its place, these devices can augment performance controls, acting as compositional brains, and can serve as studio creation tools.

Yesterday, we got a glimpse of something called “MB Control,” a custom Ableton Live controller that got a number of people excited for its eminently-practical control layouts. It’s based on the popular, promising, and open source Live Control project, one we’re watching closely, but with its own twists. Today, I have more details on that project from co-creator Bjorn, who has long been a maker of wonderful tutorials and devices for Ableton at The Covert Operators and works here with Uruguay’s International Feel. The bad news is, for now, this isn’t something you’ll be able to get your hands on – it’s not for sale because it’s so particular to the needs of its creators. But the good news is, it demonstrates a number of useful techniques, and at least some devices are on their way. (It’s also further evidence that a runtime for Max for Live could set this community on fire, if Ableton and Cycling ’74 were to decide to go in that direction.)

The project is powered by TouchOSC, the iPad’s apparent killer-app OSC touch controller, and Max for Live.

Bjorn writes:

This project is a customized Ableton Live studio controller made with Max For Live devices for Mark from International Feel Recordings.

http://soundcloud.com/international-feel/

After trying various control solutions ranging from Lemur Clip Launchers to ST8′s LiveControl, there still was a need for a customizable studio controller solution. Most of the solutions currently available are focused on Live Performance. There’s isn’t really anything out there to jam an arrangement with that works in a practical musical sense for studio heads.

In Mark’s case, he was looking for a Clip Launcher that could display 1 Scene of 40 Tracks wide. A change like that isn’t so easily made to a python script.
He also wanted 3 types of sequencers. That would definitely get tricky with python scripts and 3rd party MIDI drivers. So the project ended up being a customized version of ST8′s LiveControl, but entirely built in Max For Live.

MB Control Features:

• A Drum Sequencer that can be dropped onto any MIDI Track in Live. Up to 8 instances are supported which can be selected and controlled from the iPad. It has all the features of a basic X0X sequencer (and more besides). You can change the Rate/Range/Direction/Length/Velocity/Pitch of any sequencer lane individually, you can control all the lanes at once and you can even control all the sequencers at once. There is a randomization feature that allows you to specify the density of a randomized pattern.
• A Melodic Sequencer. Its almost identical to the Drum Sequencer. It also supports up to 8 instances and it has a feature to lock the pitches of all the lanes to the transposition features of the Bouncer Sequencer. It also allows various melodic scales to be selected.
• A Bouncer Sequencer like people know from on the tenori-on. Its supports 2 times 8 notes with 16 steps. There are a few extra buttons to transpose the sequence to a different pitch. The Melodic Sequencers have a Transpose Lock buttons that allows them to follow the transposition of this sequencer.
• 48 Track Clip Launcher of 1 Scene deep. A very useful Clip Launcher if you have lots of tracks and want to control all clips on a scene individually. Has a feature that shows how many clips are in any given track. It also supports group tracks and allows folding/unfolding on the clip slot of the group track. A button underneath each Clip Slot allows control over Arm/Solo/Mute and Stop Clip.
• A Mixer with control over the Volume/Pan/Send A/Send B/Arm/Solo/Mute. The Mixer follows whatever Clip was triggered last. If you trigger a clip on track 9, the Mixer will display track 9 to 18 automatically.
• Device Controller. Its like Automap with 16 Sliders. You can select any device and control all of its parameters. Devices can also be hidden if required.
• 16 Macros Device. Its like Macros in a Rack but there are 16 of them. And they can be assigned to any device regardless of which track they’re in. This device is integrated in the Device Controller page as a simple switch and allows users to personally tailor their multi device control onto one screen, thus keeping it in line with the ‘arrangement jamming’ principle.
• XY Controller. 5 XY Pads to control multiple device parameters.
• Drum Pads. A MIDI Device that can be dropped on any MIDI Track you wish to play on.
• Keys. A Keyboard that can be dropped on any MIDI Track you wish to play on, with velocity, Octave buttons, modwheel and Pitch Bend.
• Arturia Moog Editor. A custom controller layout for the [Arturia Moog emulation] VST. It is a device that converts the incoming OSC to CC data that the VST accepts.
• Most pages have global controls like Overdub/Record/Transport and switching MIDI Quantization between Off and 1/16ths.

This project is finished as far as Mark is concerned. He has the controller he wished for and it works entirely to his specifications. It’s all running very smooth in Max For Live and the response on the iPad is great. TouchOSC keeps improving so that means that eventually we’ll have more than 7 colors to work with and lots of cool little features surely to come. The day this project was finished, TouchOSC 1.6 came out with the ability to change colors and hide objects. Something that’ll be off great use for making customizable interfaces.

MB Control (in it’s current form), is not for sale.

It is far too personalized to be useful to everybody. I will release some of the devices individually soon. As for the core ‘brain’ patch that controls everything, I will keep improving on that until I’ve got a device that supports Clip Launchers of all sizes. Regardless of how many tracks/scenes you want to display. All you should need is ANY OSC controller, configure it to your liking, type in your IP somewhere and never have to set it up again.

Once a M4L runtime is available then we will definitely consider making this (and possibly different versions of it) available commerically.

Thanks to Bjorn for all this great info.

By way of comparison, below is a video of the LiveControl project, which now has a new homepage:
http://livecontrol.q3f.org/livecontrol/

As you can see, there are a variety of approaches to how to make the controller – software setup useful for music.

LiveControl for the iPad/iPod from ST8 on Vimeo.

First up, here’s a demo of the new app FingerPlay MIDI, which turns Android into a simple touch controller. True, we’ve seen similar things on iPhone, and Apple’s platform has precise multitouch which Android lacks, but it’s nice to see the concept extended across platforms. Big thanks to postromantic on Twitter for the tip! (Follow cdmblogs for more.)

FingerPlay MIDI

I can’t tell yet if this will be open source – that would actually be nice, I think, as it’d allow the community of developers to have a shared set of tools. (In fact, it seems a logical model would be to cash in on general-market apps and open source the more music-specific, niche stuff.)

In other Android mobile news, Christopher Souvey continues to work on his Musical application and the Musical Pro desktop app. The desktop client works with MIDI over Wifi, and thanks to the Cupcake OS update, latency is greatly reduced after a complete rewrite and the creation of custom drawing and event handling and controls. Check out the slick new tuner and UI Christopher has been developing, too.

http://www.souvey.com/ [blog with all the latest]
http://www.souvey.com/musical/

This play-along piano is probably not something any of you folks desperately need, but it is a good demonstration of what’s possible.

Another interesting thread to follow will be the growing power of Web apps. On Android, you’re already able to combine a Web app with the Java APIs, and going another level, mobile apps with native ARM code for the processor. Translation: while phones still have a fraction of the power of your computer, it’s getting easier for developers to work across platforms and to take advantage of what power is there. That’s leading to trends that could be of use not only to a single platform (Android, iPhone), but to mobile devices in general.

I love the sound bugs make when you squish them under a solution.

AlexP, whose blog is also a great source for multitouch and the Sony PS3 Eye Camera and Windows drivers we used in the recent hackday, has been diagnosing his MacBook under Windows Vista. Hardware problems are often the source of sound blips on computers. I’ve talked previously about using the DPC Latency Checker to find this issue.

The good news: Alexander has found the problem (the Broadcom Wireless Adapter in some Apple MacBooks) and a solution (switching off Windows’ automatic wireless network search when you don’t need it). I actually wonder if a similar problem was culpable in early problems with network WiFi on Mac OS X Leopard. Whatever is going on, check out the fix here if you’re encountering this problem. And let us know if you’re seeing this on machines other than just the MacBook revision F; I’d imagine any PC with a similar wireless adapter might have the issue:

MacBook Rev. F Audio Skipping in Vista Analysis and Solution – Part 2

And yes, hardware/driver problems may frequently manifest as what Windows terms DPCs – basically, a symptom of hardware usage that can interfere with reliable audio performance. I’m curious whether WiFi connections specifically may be a cause in other cases. The problem is almost certainly not limited to computers from Apple – especially since, in this case, the MacBook is just behaving like any PC laptop with similar specs.

We’ve already seen the Nintendo DS used as a wireless WiFi MIDI controller. But if you’re longing for some good, old-fashioned, hard MIDI connections to your DS, too, you’re now in luck.

Natrium42, the creator of homebrew-launcher classic PassMe, has built a multi-purpose serial device for the DS called DSerial. Via his schematics, you can support both MIDI input and MIDI output via standard 5-pin DIN cords.

That’s cool enough, but of course you need something with which to use all that MIDI goodness. Tobias Weyand has updated his DS MIDI application so it now supports both wifi MIDI connections (to a computer) and hardware, cabled MIDI connections (to MIDI-compatible synths, keyboards, controllers, effects, guitar controllers, etc., etc.) That makes your DS into an all-purpose MIDI controller for everything you own. Since the previous name, DSMIDIWiFi, doesn’t cover the full range, Tobias has redubbed the application DSMI. (Sounds like some evil branch of the US Defense Department creating killer, mutant dolphin cyborgs. Actually stands for DS Music Interface.)

And this should soon get even better: Tobias is working on keyboard support via DSerial and DSMI for his DS tracker, NitroTracker, which would make the DS into an all-in-one mini music studio.

Grab your soldering iron and your DS and read up on the details.

The hardware hack (DSerial + MIDI) for adding input and output jacks:
MIDI In/Out Hardware for DS [Natrium42 blog]
Schematics and instructions for MIDI on the DSerial wiki

The software side, for wifi support and/or communications via DSerial + MIDI:
DSMIDIWiFi gets DSerial support and becomes DSMI [0xtob’s DS coding blog]

With its touchscreen input, compact size, internal Nintendo-style synth, and wireless Internet connection, the Nintendo DS has a lot of appeal for music use. The challenge is how to hook it up to a computer. TheRain has a hardware hack called DSMIDI: with a little soldering and hacking, you can connect the DS to a standard MIDI cable via a cartridge.

DSMIDIWiFi goes much further, though: without picking up a soldering iron or connecting a cable, you can use your DS as both a synth and a controller, all over the DS’ WiFi connection. And now, the good news: the software is finished, and the solution is a free download away:

DSMIDIWiFi — Wireless MIDI for the Nintendo DS [Project site, instructions, and download]

In case you haven’t seen it, here’s the video demo:

Server software on your Mac, Windows, or Linux computer translates the WiFi data into MIDI messages for use with whatever music or visual software you like. (As I observed on Create Digital Motion, this could make the DS a very drool-worthy wireless controller for VJs.) Three applications are included, two for control and one for synthesis:

1. X/Y “Kaoss Pad” controller: The must-have for the DS, you can turn the touchscreen into an X/Y controller. This to me is especially exciting, as it’s a natural for the DS hardware and the units are small enough that they could be propped on a keyboard, etc.
2. Keyboard controller: A MIDI keyboard controller sends notes, pitch bend, and MIDI controller adjustments via an on-screen keyboard. Sure, you might not want to use this to replace a hardware keyboard, but it could be great for programming and for controlling other things (triggering clips, running visuals, etc.).
3. DS Synth: The internal sound generator provides vintage Game Boy-like sounds. Now, I know purists prefer the original 8-bit Game Boy, but I kind of like the idea of bringing chiptune music into the 21st Century … especially if means I can rationalize buying a DS Lite. Erm, I mean … uh … going wireless.

The software is the creation of TheRain and Tobias Weyand, the creator of NitroTracker. The good news for you programmers out there is that they’ve completed open-sourced the code, so you could use this as the basis of your own wireless Nintendo DS creations.

Next up: hacking the Wii. (Any takers?)

Previously:

Wireless Nintendo DS Synth and Controller

Mobile Music: Tracker for Nintendo DS, DS Homebrew Music Roundup

And, since you’ll need a way of running this (just as with any homebrew software for DS):
Homebrew Music on Nintendo DS, Now Easier with DS-X Update; LEDs Dance to Music