Visual programming or “patching”, producing custom software by connecting on-screen objects with patch cords, until recently had only niche appeal. The domain largely of academic computer musicians, patching was scoffed at by computer science departments and unknown to everyone else. Lately, though, something very strange has been happening: this technique, popularized by experimental music synthesists, is being reborn in the Web age.

Patching for the Web

Patching software goes back to the 1980s, best known in its incarnation in Max (later Max/MSP, later Max/MSP/Jitter and Pure Data), software for making music and multimedia. Max is well known in these parts, not so well among the general public. But its basic patching metaphor, itself inspired by early hardware synthesizers like the Moog and Buchla, has filtered into other software.

First, Apple quietly acquired the developer of a little-known live visual/VJ app called Pixelshox, transformed it into a new app called Quartz Composer, made it part of the Mac OS X developer tools, and made it central to their UI efforts. One day, a tiny VJ app with a cult following, the next, central to Cupertino’s OS strategy? Interesting.

Quartz Composer didn’t exactly take the world by storm, but it did update Max’s taking on patching with a pretty, zoomable patching interface. Someone must have noticed, because more recently mighty Yahoo unveiled Yahoo Pipes, an online tool for creating Web remixes, with an almost pixel-perfect, color-exact clone of the QC interface. (Imitation is the sincerest form of Web 2.0, apparently.) Microsoft’s Popfly tool is a bit different, but even it uses an object and patch-cord metaphor.

Lily, JavaScript Patching

Enter Lily: Lily is a JavaScript-powered patching environment. Lily-created software can run standalone, as a Firefox plug-in, or even in a browser. Much of the functionality is Web-focused, as you’d expect: modules for mashing-up data from Amazon, Flickr, Wikipedia, Yahoo, and the like, widget support for popular JavaScript library elements, SQLite database storage and file system access — all good stuff, but primarily Web-based.

Lily

Where things get especially interesting is that Lily has some multimedia support:

• Graphics: SVG, canvas elements.
• Multimedia: Audio and video file support.
• Connectivity: OpenSoundControl support, which in turn could be used to connect to tools like Max/MSP, Flash, Processing, Reaktor, Traktor DJ, and others.
• Custom modules: Code your own modules in JavaScript.

There’s even a demo of Lily being used as a Beatbox.

Ever thought music software would inspire developer tools? Only Apple would try something like this: they’ve copied the interface of their own GarageBand software, almost button for button, in the new Xray developer tool in Xcode 3.0 (part of Mac OS X 10.5). The developer’s work process will be much like someone mixing music in GarageBand.

Xray is designed for visualizing performance and debugging code, which is a linear, time-based process. That means that some kind of timeline interface makes perfect sense. Apple didn’t just stop there, though: the track view, transport controls (including record button), volume, channel controls, ruler, and loop display elements are all there. It’s so close that you wind up with sentences like this:

Add different instruments so you can instantly see the results of code analyzers.

Instruments turn out to be exactly the same word in development; see comments for more details of what this means for real programmers as opposed to weekend coders like me. :)

I’m just waiting for Apple to add an Apple Loop Browser so you can lay down a groovin’ trance or house beat while you figure out why your application is sucking so many CPU cycles.

Mac OS X Leopard Sneak Peak: Xcode 3.0 [Apple.com]

And in a non sequitur at the end, Apple reverts to their usual marketing hyperbole: “Xray. Because it’s 2006.” So we should have a developer tool with the interface from Sonic Foundry’s ACID in 1998? Hey, if it saves developers time and makes the dev tools more intuitive, I’m for it! Developers who want to chime in on this and let us know what you think, please do.

Programming offers incredible possibilities for music creation, and with the free Processing development environment for Mac, Windows, and Linux, even non-programmers can get into the artistic horizons of code. But code doesn’t always think like composers do. That’s why the new sound library jm-Etude looks promising:

New sound library: jm-Etude [Code & form]

jm-Etude for Processing, and the Java library jMusic on which it’s based, allow you to structure your code more musically with notes, phrases, parts, and scores. Combine this with the Sonia synthesis library for Processing (or, for Java development, the corresponding JSyn plug-in), and you have a serious compositional environment. I can also see this as appealing to people coming from the land of Csound and wanting something that lets them code notes and other musically-useful units.

I’d love to see a similar library that helps deal with performance environments, helping structure into scenes and elements — Chris from Pixelsumo was just asking me how you might use Processing in a VJ performance. For live music or visuals, it’d be helpful to have a library that lets you structure what you’re doing over time for performance. Anyone know if there’s something like this already out there (short of coding the thing yourself, which might ultimately be better)?

[Updated:] I missed the major point of this, which is that it lets you use Processing as an interactive MIDI sequencer. (Follow that link for a promising-looking interactive table, built in the “app no one wants to use any more,” Director.)

It’s tough to make money selling music software. The market is small to begin with, made smaller when an unsettling number of users use pirated software, and divided into pieces by a range of different software. Sometimes, dead software winds up disappearing forever (Opcode Studio Vision), but sometimes it winds up free.

CPS is a powerful host for plug-ins. Like Native Instruments’ much-hyped platform KORE, it can be a plug-in in other hosts or host plug-ins itself. CPS is unusually powerful, with some truly unique features (aside from the normal VST/VSTi hosting on Mac and Windows). Director and Web browser supports makes this particularly worth a look if you’re building an interactive kiosk or a project involving live animation:

One of the things that makes Max/MSP special is that a wide variety of tools, from simple MIDI utilities to complex interactive art, can be represented by a graphical interface of interconnected visual objects instead of lines of code. But actually building those patchers can involve some repetitive tasks, if you need to connect a large number of objects or want to clean up an on-screen mess of boxes and lines.

Nathanaël Lécaudé has built an elegant solution called the Max Toolbox. A whole series of mouse operations is reduced to single keystrokes: connect and rearrange objects en masse with just one key press. The secret is Max’s JavaScript implementation, which allows objects to be manipulated, created, and destroyed via code. But whether this inspires your own automagical patching JavaScripts for Max or you just take advantage of the keystroke shortcuts Max’s creators forgot, it’s a welcome idea. Still in beta; I’ll be watching this evolve. Via the CDM forums, where you’re welcome to ask or share anything you like related to this sort of thing on our DIY, Physical Computing, and Advanced” boards.

Max/MSP and Xcode, correction: Okay, while it might be handy to have a copy of CodeWarrior around, I’m happy to say I was wrong about developing externals. Despite what the PDF documentation says in the Max/MSP SDK, you can now develop Mac externals using Xcode. Check out the new documentation, posted by David Zicarelli. I’ll be moving happily over to Xcode! Thanks, vallen! Now all we need is someone from Apple marketing to say something snarky about how Xcode is more advanced than anything else on the planet, and anyone who uses anything else is a loser.

It’s the arcane post of the week: Yes, this series will bring all the stories the fewest readers care about, direct to you. But I care about them (because I care about arcane things), and we’ll see if someone else does, too. This week’s entry: it looks like CodeWarrior, the long-beloved Mac development tool, is nearly at its end. Metrowerks is no longer supporting the product (though they’ll still happily sell it to you unsupported — uh, thanks), and “after Q1 2006″ you won’t be able to download at all. (Or thereabouts, since it’s obviously still there.) Why am I bothering to mention a dead development tool in the first place, particularly with Apple’s own Xcode included free with every copy of Mac OS X? Two reasons: first, we’ve already heard noted music developer Propellerhead complaining that their PowerPC code was faster on CodeWarrior than Apple’s Xcode. Now, granted, I’m not sure why this is and I haven’t heard anyone else griping about Xcode performance (though you will hear them griping for other reasons), but I’ll leave that whole issue to others.

More importantly, Max/MSP/Jitter from Cycling ‘74 still requires CodeWarrior for creating externals on Mac, a key means of extending Max’s already-deep capabilities. Yes, I expect Cycling will fix that in a future release, but it makes it sound as though CodeWarrior would be a smart download for backwards compatibility.
Pay US$99 for an unsupported product that’s waiting for the executioner? No way. But if you can convince Freescale’s automated form (hint, hint) that you’re an educator, you can use the learning edition. And I’m guessing if you’re writing Max/MSP externals, you’re either in an academic institution now, or were at some point, or could probably fit in if you happened to visit. Go download and enjoy. Oh, and send us your externals.

Blog haze: it’s the dimmed state of consciousness that comes from reading too many blog entries about such and such a remix creative commons resampled installation art piece about intellectual prop . . . there, see I’ve already fallen asleep. But wait, this is really, truly, incredibly cool. If you don’t believe me, just watch the video.

Okay, Sony PSP owners: get ready to make still more noise. We’ve just seen another major release for the superb href=”http://www.createdigitalmusic.com/index.php?option=com_content&task=view&id=1126&Itemid=44″>PSP Rhythm 3 drum machine. Now, get ready for a PSP sequencer: PSP Seq Ver 0.00 has begun development with basic sequencing functionality all in place. Features are bare-bones — song and sound design functionality is yet to come, among other things — but given the rapid maturation of other PSP creations, this could be good news. We’ll be watching. Thanks to CDM’s redesigner and logo creator Nathanael Jeanneret, who has further thoughts on his blog.

href=”http://dl.qj.net/PSPSeq-v0.00-Gaming-Consoles-PSP-Homebrew-Applications/pg/12/fid/912/catid/140″>PSP
Seq 0.00, as created by Ethan of dsp music

And has been widely reported this week, the href=”http://forums.qj.net/showthread.php?t=35463″>2.60 PSP 2.60 Firmware has now been hacked, so you, too, can run this stuff. A copy of Grand Theft Auto: Liberty City is required, but I assume you bought a PSP for a reason.

Unfortunately, the Nintendo DS doesn’t seem to be getting the attention of homebrewers, but oldskool Game Boy is another matter; I notice even Nanoloop again seems to be available again in both GBA (2.0) and original Game Boy (1.3) versions.

In answer to my earlier question, yes, people are using circuit simulation software to develop music software. Chris Randall of the very cool plug-in development house Audio Damage writes us:

We use SPICE when we’re modeling for Audio Damage products. We recreate the circuitry of whatever unit we’re modelling in order to better understand the signal flow; we also do some analysis using the SPICE model. (By “we” I mean Adam, of course. I sit in my office and play Rise Of Empires until he sends me a build to test.)

To which I say — hey, Rise of Empires sounds great! I’m sure you CDM readers can find something to do (like learning electrical engineering and circuit simulation software); I’m off to teach Chris how powerful my Persian Army is.

In all seriousness, I can’t wait to see Audio Damage’s upcoming recreation of the Mutron Bi-Phase effects pedal. (Warning: Bi-phase goodness, MP3s, and nostalgia after that link.) Any chance you’re busy simulating those circuits right now, Audio Damage?

And while we’re on the subject of geeky software meets cool musical applications, reader m15a spots this fantastic add-on to MatLab for analyzing and visualizing MIDI files. Can you say: slick visuals for your doctoral dissertation? (Oh, yes, I must be getting back to that one of these days. After one more Rise of Empires game, anyway.)

MidiToolbox: MIDI Analysis / Visualization for Matlab

Anyone got some nifty MIDI visualization images? Send `em in!