Visual programming or “patching”, producing custom software by connecting on-screen objects with patch cords, until recently had only niche appeal. The domain largely of academic computer musicians, patching was scoffed at by computer science departments and unknown to everyone else. Lately, though, something very strange has been happening: this technique, popularized by experimental music synthesists, is being reborn in the Web age.

Patching for the Web

Patching software goes back to the 1980s, best known in its incarnation in Max (later Max/MSP, later Max/MSP/Jitter and Pure Data), software for making music and multimedia. Max is well known in these parts, not so well among the general public. But its basic patching metaphor, itself inspired by early hardware synthesizers like the Moog and Buchla, has filtered into other software.

First, Apple quietly acquired the developer of a little-known live visual/VJ app called Pixelshox, transformed it into a new app called Quartz Composer, made it part of the Mac OS X developer tools, and made it central to their UI efforts. One day, a tiny VJ app with a cult following, the next, central to Cupertino’s OS strategy? Interesting.

Quartz Composer didn’t exactly take the world by storm, but it did update Max’s taking on patching with a pretty, zoomable patching interface. Someone must have noticed, because more recently mighty Yahoo unveiled Yahoo Pipes, an online tool for creating Web remixes, with an almost pixel-perfect, color-exact clone of the QC interface. (Imitation is the sincerest form of Web 2.0, apparently.) Microsoft’s Popfly tool is a bit different, but even it uses an object and patch-cord metaphor.

Lily, JavaScript Patching

Enter Lily: Lily is a JavaScript-powered patching environment. Lily-created software can run standalone, as a Firefox plug-in, or even in a browser. Much of the functionality is Web-focused, as you’d expect: modules for mashing-up data from Amazon, Flickr, Wikipedia, Yahoo, and the like, widget support for popular JavaScript library elements, SQLite database storage and file system access — all good stuff, but primarily Web-based.

Lily

Where things get especially interesting is that Lily has some multimedia support:

• Graphics: SVG, canvas elements.
• Multimedia: Audio and video file support.
• Connectivity: OpenSoundControl support, which in turn could be used to connect to tools like Max/MSP, Flash, Processing, Reaktor, Traktor DJ, and others.
• Custom modules: Code your own modules in JavaScript.

There’s even a demo of Lily being used as a Beatbox.

Ever thought music software would inspire developer tools? Only Apple would try something like this: they’ve copied the interface of their own GarageBand software, almost button for button, in the new Xray developer tool in Xcode 3.0 (part of Mac OS X 10.5). The developer’s work process will be much like someone mixing music in GarageBand.

Xray is designed for visualizing performance and debugging code, which is a linear, time-based process. That means that some kind of timeline interface makes perfect sense. Apple didn’t just stop there, though: the track view, transport controls (including record button), volume, channel controls, ruler, and loop display elements are all there. It’s so close that you wind up with sentences like this:

Add different instruments so you can instantly see the results of code analyzers.

Instruments turn out to be exactly the same word in development; see comments for more details of what this means for real programmers as opposed to weekend coders like me. :)

I’m just waiting for Apple to add an Apple Loop Browser so you can lay down a groovin’ trance or house beat while you figure out why your application is sucking so many CPU cycles.

Mac OS X Leopard Sneak Peak: Xcode 3.0 [Apple.com]

And in a non sequitur at the end, Apple reverts to their usual marketing hyperbole: “Xray. Because it’s 2006.” So we should have a developer tool with the interface from Sonic Foundry’s ACID in 1998? Hey, if it saves developers time and makes the dev tools more intuitive, I’m for it! Developers who want to chime in on this and let us know what you think, please do.

Programming offers incredible possibilities for music creation, and with the free Processing development environment for Mac, Windows, and Linux, even non-programmers can get into the artistic horizons of code. But code doesn’t always think like composers do. That’s why the new sound library jm-Etude looks promising:

New sound library: jm-Etude [Code & form]

jm-Etude for Processing, and the Java library jMusic on which it’s based, allow you to structure your code more musically with notes, phrases, parts, and scores. Combine this with the Sonia synthesis library for Processing (or, for Java development, the corresponding JSyn plug-in), and you have a serious compositional environment. I can also see this as appealing to people coming from the land of Csound and wanting something that lets them code notes and other musically-useful units.

I’d love to see a similar library that helps deal with performance environments, helping structure into scenes and elements — Chris from Pixelsumo was just asking me how you might use Processing in a VJ performance. For live music or visuals, it’d be helpful to have a library that lets you structure what you’re doing over time for performance. Anyone know if there’s something like this already out there (short of coding the thing yourself, which might ultimately be better)?

[Updated:] I missed the major point of this, which is that it lets you use Processing as an interactive MIDI sequencer. (Follow that link for a promising-looking interactive table, built in the “app no one wants to use any more,” Director.)

It’s tough to make money selling music software. The market is small to begin with, made smaller when an unsettling number of users use pirated software, and divided into pieces by a range of different software. Sometimes, dead software winds up disappearing forever (Opcode Studio Vision), but sometimes it winds up free.

CPS is a powerful host for plug-ins. Like Native Instruments’ much-hyped platform KORE, it can be a plug-in in other hosts or host plug-ins itself. CPS is unusually powerful, with some truly unique features (aside from the normal VST/VSTi hosting on Mac and Windows). Director and Web browser supports makes this particularly worth a look if you’re building an interactive kiosk or a project involving live animation: