Visual programming or “patching”, producing custom software by connecting on-screen objects with patch cords, until recently had only niche appeal. The domain largely of academic computer musicians, patching was scoffed at by computer science departments and unknown to everyone else. Lately, though, something very strange has been happening: this technique, popularized by experimental music synthesists, is being reborn in the Web age.

Patching for the Web

Patching software goes back to the 1980s, best known in its incarnation in Max (later Max/MSP, later Max/MSP/Jitter and Pure Data), software for making music and multimedia. Max is well known in these parts, not so well among the general public. But its basic patching metaphor, itself inspired by early hardware synthesizers like the Moog and Buchla, has filtered into other software.

First, Apple quietly acquired the developer of a little-known live visual/VJ app called Pixelshox, transformed it into a new app called Quartz Composer, made it part of the Mac OS X developer tools, and made it central to their UI efforts. One day, a tiny VJ app with a cult following, the next, central to Cupertino’s OS strategy? Interesting.

Yahoo Pipes

Patching cords together mimics the flow of Web data: Yes, the patching interface is intuitive, even for Web development beginners. The source: music software, and before that, vintage hardware synthesizers. Who would have imagined Moogs and Buchlas would some day spawn Web apps? Here: Yahoo Pipes.

Quartz Composer didn’t exactly take the world by storm, but it did update Max’s taking on patching with a pretty, zoomable patching interface. Someone must have noticed, because more recently mighty Yahoo unveiled Yahoo Pipes, an online tool for creating Web remixes, with an almost pixel-perfect, color-exact clone of the QC interface. (Imitation is the sincerest form of Web 2.0, apparently.) Microsoft’s Popfly tool is a bit different, but even it uses an object and patch-cord metaphor.

Lily, JavaScript Patching

Lily JavaScript development environment, Beat box

The Lily interface is unlikely to replace Max/MSP any time soon, but the inspiration is very clear — and the fact that a simple Max clone with Web functionality could be produced with JavaScript says a lot about the future of even Max.

Enter Lily: Lily is a JavaScript-powered patching environment. Lily-created software can run standalone, as a Firefox plug-in, or even in a browser. Much of the functionality is Web-focused, as you’d expect: modules for mashing-up data from Amazon, Flickr, Wikipedia, Yahoo, and the like, widget support for popular JavaScript library elements, SQLite database storage and file system access — all good stuff, but primarily Web-based.

Lily

Where things get especially interesting is that Lily has some multimedia support:

  • Graphics: SVG, canvas elements.
  • Multimedia: Audio and video file support.
  • Connectivity: OpenSoundControl support, which in turn could be used to connect to tools like Max/MSP, Flash, Processing, Reaktor, Traktor DJ, and others.
  • Custom modules: Code your own modules in JavaScript.

There’s even a demo of Lily being used as a Beatbox.

So, will there be a Web-based Max killer? Quite the opposite. As patching interfaces spread, they only serve to validate the idea of patching in the first place. Web developers get introduced to the concept, and meanwhile, musicians schooled in Max have new, more accessible Web tools. Lily could be useful for Web-based, simple patches that complement software like Max and Reaktor: speedy beat calculators, or, via OSC, perhaps even control apps. (Hmm… your Flickr stream, controlling a Max/MSP patch?) And meanwhile, this also validates the Max approach of blending patching with coding in languages like JavaScript and Java, both of which are possible in the current release. We also have a new version of Max to look forward to, as confirmed by Cycling ’74, which should continue to expand on these ideas.

Lily looks really interesting, but I’d like to see more. The feature set in JavaScript is woefully limited compared even to Flash, let alone what’s possible in Java. It’s too bad these other environments haven’t developed toolkits that allow patching for modular combinations, but still let you easily code when you like. Even Max falls short here: much of Max’s capabilities require messy coding in C, so once you get to that point, you probably don’t want to be using Max unless you’ve got a Cycling ’74 paycheck coming. (On the Pure Data side, even, where coding is open source, efforts to integrate Ruby and Python have proven less fruitful for multimedia than simpling coding in something like Java or C#. And all of these tend to separate coding from patching; we haven’t seen a good combination.)

Lily is still in development, with a pre-release beta coming soon, so we have plenty of time to think about what it’s for. But one thing’s for certain: patching is here to stay.

Thanks to Ronnie at the wonderful Rekkerd.org and to Marco Raaphorst, who has a write-up in Dutch.

  • http://www.numblog.de/ Matthias

    This looks awesome, hope I can give it a try sometime. And I can't wait for the first flickr-last.fm mashup. If Lily's learning curve is as flat as Yahoo Pipes' it should be a matter of days or weeks. I really like this stuff, be it only for testing out what's possible.

  • Adrian Anders

    OSC huh? very cool. I imagine you could do some sort of live audience participation sort of thing by setting up a mobile website that users could direct their browser-enabled cells to. By popular demand a show could be controlled by the crowd (or at least by the portion of the crowd with mobile internet on their devices).

    Could also be really cool for rss-fed installations with audio being manipulated by a feed live.

    ATA

  • poopoo

    The patching idea is fantastic( we called visual programming at school). But you've hit the nail on the head with the problem of having to extend it using "Messy C", ruby, python, java etc. The ideal would be using patching to extend the patching. The objects are pretty much all there in pd but the main barrier seems to be the execution efficiency. Perhaps some kind of sub-patch optimizer that works like a de-compilier when you open a sub-patch and a compilier when you exit it.

  • c

    this made the rounds about a year ago and still no download link – feel free to repost again when it doesnt require mailing him asking for access. thanks !

  • http://www.createdigitalmusic.com Peter Kirn

    @c:

    Aw, come on. It's better *imagining* things than actually experiencing them anyway, right? ;)

    In all seriousness, we're in touch with the author. I missed it when it made its "rounds" before — got a string of tips this week — so we'll be sure to have an update when it's actually available. I know it can be frustrating to get vaporware, but the nature of the Web is that word spreads quickly, and sometimes it's helpful to start documenting early.

  • Pingback: sounDesign » Blog Archive » Il web è il futuro della programmazione musicale?

  • http://www.soundclick.com/benhodgson poorsod

    You forgot the level editor for the upcoming Crysis (video game, causing quite a stir for various reasons). All scripted events in a level one is desgining can be controlled by a modular patchy-stylee flowchart, as opposed to lines of computer code.

  • Pingback: Free Encyclopedia » Blog Archive » Lily (software)

  • Pingback: Create Digital Music » Play the NY Times Website Like an Instrument, and Other New Lily Tricks

  • Pingback: Create Digital Motion » Processing, Ported to JavaScript

  • http://xesex.110mb.com xesexcom

    Sex offenders in my area.

    Registered sex offenders in my area…

  • http://unsickness.110mb.com unsickness

    Find the causes of your mysterious symptom or unknown sickness

    through the root cause analysis of a holistic healer for natural, health and wellness

  • http://alphabet.101freehost.com alphabet

    Find Pattern for Alphabet Letters websites, images, videos, news and more

    Pattern for alphabet letters

  • http://h1.ripway.com/peopletracer/ peopletracer

    Search for Free by Name, Business, Address or Phone Number. Accurate!

    Free people tracer…

    Free people tracer

  • http://ipodwork.110mb.com ipodwork

    How does an iPod work? How do I reset an iPod?

    How do I change the language on an iPod? …

    Bookmark How does an iPod work? Latest Blog Posts …

    How does an iPod work

  • http://russiansks.50webs.com russiansks

    ssk fungo bats · worx jawhorse · sks mud flaps · prises on sks gun ·

    russian sks rifles for sale · tapco integrated tool prices · tapco · sks mud …

    Russian sks rifles for sale