Hey, look: a controller, right in your browser, with drag-and-drop editing. No app needed.

Hey, look: a controller, right in your browser, with drag-and-drop editing. No app needed.

Tablet or phone or touch-enabled desktop computer – now it doesn’t matter. A free tool called (for the moment) Nexus lets you make any browser a canvas for music. iOS, Android, Windows, Mac – if the browser is there, your creations become omni-platform.

Shown at the NIME (New Interfaces for Musical Expression) conference in London earlier this month, Nexus is the most complete foundation for this idea seen yet. And since it’s free, it’s open to others to build upon. Right out of the box, it includes basic interface widgets obviously inspired by Lemur (and apps like TouchOSC), so you get faders and knobs and multi-touch arrangements of balls and keyboards and so on. But because it’s all built with Web tech, anyone can create any interface they imagine – with a custom look and feel, and complete with animations. And even in the browser, it uses OSC (Open Sound Control), for flexibility with a range of commercial or custom-built software.

Various demos are featured in a video:

There are a few ways you can work with Nexus, depending on whether you’re an end user or a coder, and which tools you prefer:

1. Drag and drop in a browser. Take any modern browser on desktop or tablet, and you can make your interfaces without coding at all using NxDrop. You can design right on the device, or build on a computer and then send to a mobile gadget.

2. Work with the browser to make sound. You can interface directly with the Web Audio API, meaning you can both create your interface in the browser and use it with sound.

3. Work with libraries like libpd. Another way to integrate interface and sound engine is with tools like libpd, the open source Pure Data engine, which you’ll see in some of the examples.

4. Make interfaces with – and for – Max/MSP. Integration features with Max make it really easy to automatically route messages, and even (in one demo) to use Max to help create the interfaces themselves. That should be welcome for both Max and Max for Live users.

5. Code with JavaScript. It’s clear that Nexus has some potential for non-coders right now, and that more ready-to-use applications will come in time if it attracts interest. In the meanwhile, though, if you do Web technologies as your day job, you’ll be happy as a pig in slop. With just a few lines of JavaScript and HTML5, you can code interfaces directly.

If you do want to give it a go – and you don’t necessarily know about code or Web server operation – there are a couple of useful video tutorials complementing the resources on the official site. These walk you Mac users (works reasonably well on Linux, too, or a Windows *nix setup of some kind) through the Web server bits and gets you up and running:

The advantage of all of this is, controllers can be distributed and shared and operate independent of platform. You could build a synth and let someone run it in a browser or a simple cross-platform app. You could have backups of your live rig run if your iPad breaks or is stolen. You can make mobile music ensembles, or give friends controls over a music creation for jamming. And you know your controllers aren’t tied to a single device or app – they’ll run on whatever you have, making them future-proof.

So far, Nexus is a starting point. It isn’t as polished in look and feel or installation as dedicated apps; you’ll need to do a little DIY (like installing a Web server, though there’s a good tutorial below). It receives data, but doesn’t send it yet, though that’s on the agenda. It seems some graphic designers could give the whole thing a better look, and the drag-and-drop app could be cleaner. And the name needs some work, because it’s too close to Google’s line of mobile devices. (Anyone got any ideas?)

But the technology works beautifully. And on a project-by-project basis, there’s no question it can begin to rival those better-known apps. There’s already a tool called Vox that converts existing TouchOSC layouts for use with Nexus.

There’s quite a lot of work done already – perhaps the most so far along these lines – thanks to the terrific labors of the team Ben Taylor, Jesse Allison, and Yemin Oh (of Louisiana State University).

And there are some interesting projects already. In addition to the video above, some of the LSU folks are working with the idea of laser-cut overlays, adding tactile feedback to tablets. I had a go on their iPads, and the results were impressive – it was still nearly as smooth running my fingers along the iPad, but my fingers could find their way to notes, as you would with frets or keys.

OLYMPUS DIGITAL CAMERA

OLYMPUS DIGITAL CAMERA

OLYMPUS DIGITAL CAMERA

Lindsey Hartman has made a patch remixing of Imogen Heap, showing how Max and Nexus combine (photo via suncanon Instagram):

imogen_nexus

You’ll find all the code in a BSD license, with loads of documentation, on GitHub:

https://github.com/lsu-emdm/nexusUI

No need to dive into the code, though. At NIME, we had people up and running after just a couple of minutes following directions to install a Web server. The main site will take you straight to the getting started tutorials and free downloads:

http://www.nexusosc.com

Academics (or the technically curious), check out the full paper:
Simplified Expressive Mobile Development with NexusUI, NexusUp and NexusDrop

I’m curious to see what you do with this. And I do hope some intrepid Web designers reading this venture in, as well, or let us know if this might interoperate with other browser music projects.

nexusUI

  • BugBareDrums

    Happy days indeed! Big thanks to the team that made this.

    • William Conlin

      You are welcome! Happy nexifying!

  • WetBoy

    This is pretty awesome as there’s a total lack of options on desktops.
    I suppose the lack of comments here reflects the lack of a user base clinging to their iFads hehe

    • http://pkirn.com/ Peter Kirn

      Actually, the first thing people did at NIME was test it on iPad. But I agree … and Windows touch laptops will make good use of this.

      The lack of comments may be that people are just content with the idea. :)

      It’s also not end-user friendly just yet. I think there’s potential for it to get there, though.

    • William Conlin

      Any Ideas are welcome, we are a small team right now. Just 5-6 students at a University. We are hoping to build our user base up more and more

  • nudephotomusic

    To me, the first thing it suggested was using a collection of all the devices I already own. So my old phones, kindle, etc could find a new life as a controller. I love the idea that its not tied to iOS. Also – as Peter mentioned, its exciting for all the new and inexpensive windows 8 machines.

    I’m a developer by day, and this is one of the few projects that’s made me interested in coding / development for musical purposes. :)

  • miguel

    I’m no computer programer but was able to get this up and running without a problem. Wow! Now, can anyone tell how to access this “Web Audio API,” and what the heck it is? I’m reading about it but it has yet to get to the “real world” part of my brain.

    • davydka

      It’s the new HTML5 Javascript methods for generating sound. Basically, you can code sound using javascript in your web browser.

    • William Conlin

      I hope this helps. I the web audio is still new to me as well: http://www.html5rocks.com/en/tutorials/webaudio/intro/

    • Danny Holmes

      http://charlie-roberts.com/gibberish/docs.html

      Check out Charlie Roberts’ Gibberish. It is also open-source, and can be downloaded on github. It’s a higher level web-audio library that does a good bit of the dirty work for you. I know at least one of the apps above uses it…

  • haszari

    This looks superb, have been waiting for someone to do something like this for a while .. and it’s on github. Props.

  • William Conlin

    Peter! this should be corrected: Vox is piece I wrote that used to have a ToucOSC controller, I converted it manually for nexus by coding in HTML. There is no way for converting existing touchOSC to nexus.

  • joshue ott

    this is SUPER exciting- so glad someone finally did this. time to replace mrmr as my method of getting multiDraw going at parties/installations! also, Thicket, etc. etc. etc. etc. very very cool.

  • Luis Melo

    Just to be sure, this can’t be used to turn your devices into midi controllers that adjust parameters on your DAW right?

    • Thomas Wilfred

      It can easily be used, its max, so you can route the osc to control your DAW for example via midi.It can easily be used, its max, so you can route the osc to control your DAW for example via midi.

  • WetBoy

    something similar but not as swish

    http://remict.wikispaces.com/

  • http://bentoncbainbridge.com/ Benton C Bainbridge

    Not working for me – I see the note on nexusosc.com that most of the templates have incorrect routing, but even the templates in the “servers” folder won’t send OSC to VDMX. Testing Chrome/VDMX both over the network and on the same machine.

    • http://bentoncbainbridge.com/ Benton C Bainbridge

      Mea Culpa! Working like a charm on my MBA, sending OSC dial messages from Chrome to VDMX. Guess i gotta run Multibeast to fix those network settings on my tower ;)