Processing Class in New York, Online: Art From Code, For Non-Coders [Create Digital Motion]

It’s on CDMotion rather than CDMusic for a simple reason — Processing is especially well-suited to visuals, 2D and 3D. But there is audio and MIDI support in there, as well, and while it’s not exactly Java’s strong suit, certain applications do benefit from this approach in music. (It works nicely with everything from Monomes to Arduinos, too.)

Actually, on that note, I’d be especially interested to hear if anyone is using Processing for musical applications. Let us know in comments. And there are slots free for the class, so do sign up if interested. (If you’re outside NYC — realizing that’s, um, the HUGE majority of you –  watch for an announcement soon for how we’re sharing some of this information online for the rest of the world.)

Harvestworks also has a terrific-looking Max/MSP/Jitter intensive in March, but I don’t know how many slots are still open, and it will require US$1275 in tuition.

Image by the awesome eskimoblood, a great source of Processing inspiration!

Patching for the Web

Patching software goes back to the 1980s, best known in its incarnation in Max (later Max/MSP, later Max/MSP/Jitter and Pure Data), software for making music and multimedia. Max is well known in these parts, not so well among the general public. But its basic patching metaphor, itself inspired by early hardware synthesizers like the Moog and Buchla, has filtered into other software.

First, Apple quietly acquired the developer of a little-known live visual/VJ app called Pixelshox, transformed it into a new app called Quartz Composer, made it part of the Mac OS X developer tools, and made it central to their UI efforts. One day, a tiny VJ app with a cult following, the next, central to Cupertino’s OS strategy? Interesting.

Quartz Composer didn’t exactly take the world by storm, but it did update Max’s taking on patching with a pretty, zoomable patching interface. Someone must have noticed, because more recently mighty Yahoo unveiled Yahoo Pipes, an online tool for creating Web remixes, with an almost pixel-perfect, color-exact clone of the QC interface. (Imitation is the sincerest form of Web 2.0, apparently.) Microsoft’s Popfly tool is a bit different, but even it uses an object and patch-cord metaphor.

Lily, JavaScript Patching

Enter Lily: Lily is a JavaScript-powered patching environment. Lily-created software can run standalone, as a Firefox plug-in, or even in a browser. Much of the functionality is Web-focused, as you’d expect: modules for mashing-up data from Amazon, Flickr, Wikipedia, Yahoo, and the like, widget support for popular JavaScript library elements, SQLite database storage and file system access — all good stuff, but primarily Web-based.

Lily

Where things get especially interesting is that Lily has some multimedia support:

• Graphics: SVG, canvas elements.
• Multimedia: Audio and video file support.
• Connectivity: OpenSoundControl support, which in turn could be used to connect to tools like Max/MSP, Flash, Processing, Reaktor, Traktor DJ, and others.
• Custom modules: Code your own modules in JavaScript.

There’s even a demo of Lily being used as a Beatbox.

So, will there be a Web-based Max killer? Quite the opposite. As patching interfaces spread, they only serve to validate the idea of patching in the first place. Web developers get introduced to the concept, and meanwhile, musicians schooled in Max have new, more accessible Web tools. Lily could be useful for Web-based, simple patches that complement software like Max and Reaktor: speedy beat calculators, or, via OSC, perhaps even control apps. (Hmm… your Flickr stream, controlling a Max/MSP patch?) And meanwhile, this also validates the Max approach of blending patching with coding in languages like JavaScript and Java, both of which are possible in the current release. We also have a new version of Max to look forward to, as confirmed by Cycling ‘74, which should continue to expand on these ideas.

Lily looks really interesting, but I’d like to see more. The feature set in JavaScript is woefully limited compared even to Flash, let alone what’s possible in Java. It’s too bad these other environments haven’t developed toolkits that allow patching for modular combinations, but still let you easily code when you like. Even Max falls short here: much of Max’s capabilities require messy coding in C, so once you get to that point, you probably don’t want to be using Max unless you’ve got a Cycling ‘74 paycheck coming. (On the Pure Data side, even, where coding is open source, efforts to integrate Ruby and Python have proven less fruitful for multimedia than simpling coding in something like Java or C#. And all of these tend to separate coding from patching; we haven’t seen a good combination.)

Lily is still in development, with a pre-release beta coming soon, so we have plenty of time to think about what it’s for. But one thing’s for certain: patching is here to stay.

Thanks to Ronnie at the wonderful Rekkerd.org and to Marco Raaphorst, who has a write-up in Dutch.

Xray is designed for visualizing performance and debugging code, which is a linear, time-based process. That means that some kind of timeline interface makes perfect sense. Apple didn’t just stop there, though: the track view, transport controls (including record button), volume, channel controls, ruler, and loop display elements are all there. It’s so close that you wind up with sentences like this:

Add different instruments so you can instantly see the results of code analyzers.

Instruments turn out to be exactly the same word in development; see comments for more details of what this means for real programmers as opposed to weekend coders like me. :)

I’m just waiting for Apple to add an Apple Loop Browser so you can lay down a groovin’ trance or house beat while you figure out why your application is sucking so many CPU cycles.

Mac OS X Leopard Sneak Peak: Xcode 3.0 [Apple.com]

And in a non sequitur at the end, Apple reverts to their usual marketing hyperbole: “Xray. Because it’s 2006.” So we should have a developer tool with the interface from Sonic Foundry’s ACID in 1998? Hey, if it saves developers time and makes the dev tools more intuitive, I’m for it! Developers who want to chime in on this and let us know what you think, please do.

Programming offers incredible possibilities for music creation, and with the free Processing development environment for Mac, Windows, and Linux, even non-programmers can get into the artistic horizons of code. But code doesn’t always think like composers do. That’s why the new sound library jm-Etude looks promising:

New sound library: jm-Etude [Code & form]

jm-Etude for Processing, and the Java library jMusic on which it’s based, allow you to structure your code more musically with notes, phrases, parts, and scores. Combine this with the Sonia synthesis library for Processing (or, for Java development, the corresponding JSyn plug-in), and you have a serious compositional environment. I can also see this as appealing to people coming from the land of Csound and wanting something that lets them code notes and other musically-useful units.

I’d love to see a similar library that helps deal with performance environments, helping structure into scenes and elements — Chris from Pixelsumo was just asking me how you might use Processing in a VJ performance. For live music or visuals, it’d be helpful to have a library that lets you structure what you’re doing over time for performance. Anyone know if there’s something like this already out there (short of coding the thing yourself, which might ultimately be better)?

[Updated:] I missed the major point of this, which is that it lets you use Processing as an interactive MIDI sequencer. (Follow that link for a promising-looking interactive table, built in the “app no one wants to use any more,” Director.)

CPS is a powerful host for plug-ins. Like Native Instruments’ much-hyped platform KORE, it can be a plug-in in other hosts or host plug-ins itself. CPS is unusually powerful, with some truly unique features (aside from the normal VST/VSTi hosting on Mac and Windows). Director and Web browser supports makes this particularly worth a look if you’re building an interactive kiosk or a project involving live animation:

1. You can patch stuff: A powerful graphical patching feature set lets you create complex performance scenarios. And if that’s not enough:
2. You can code your own stuff: An SDK supports C++ and Java so you can create your own code


3. You can use it with Adobe (Macromedia) Director: CPS is the only app I’ve ever seen of this type that interfaces with interactive visuals and multimedia in Adobe’s Director, making this perfect if you’re one of the niche of people building museum kiosks or other Director-y applications.


4. It’ll run in your Web browser: Yep, you read that right; thanks to Director support you can publish to a Web-compatible version


5. Csound users will be at home, because it uses the structured audio standard estbalished in MPEG-4


6. It works with sensor input if you’re attaching I/O boards via serial so you can, for instance, make sounds by waving your hands over photocells. (See my ongoing coverage on this topic.)


7. It’s a great learning environment if you want to try out input from sensors, basic programming, patching, MPEG4 opcodes, and more.

CPS Introduction

CPS isn’t going to be entirely without updates, fortunately. It’s not quite as complete on Mac as Windows, but it does run on both, and the creator plans to update it with a new version of Sun’s Java Virtual Machine (so obviously, the whole thing runs on Java). It’s had a cult following in academia, not surprising given its feature set. Now if the creator would just open source the thing (if that’s possible) . . .

Thanks to CDM’s Atomic Afro for this one; Afro’s got his eyes permanently peeled for free music software. More discussion on the CDM forums.

Nathanaël Lécaudé has built an elegant solution called the Max Toolbox. A whole series of mouse operations is reduced to single keystrokes: connect and rearrange objects en masse with just one key press. The secret is Max’s JavaScript implementation, which allows objects to be manipulated, created, and destroyed via code. But whether this inspires your own automagical patching JavaScripts for Max or you just take advantage of the keystroke shortcuts Max’s creators forgot, it’s a welcome idea. Still in beta; I’ll be watching this evolve. Via the CDM forums, where you’re welcome to ask or share anything you like related to this sort of thing on our DIY, Physical Computing, and Advanced” boards.

The bizarrely-capitalized sCrAmBlEd?HaCkZ! takes an audio sample, slices it up into rhythmic slices and categorizes it in a database by spectral content, and then (here’s the good part) retriggers the audio based on an input. In other words, you can beatbox into a microphone and get a totally mashed-up version of MC Hammer in response. Check out the entertaining video for a sense of what’s going on.

The software itself will be unleashed on the masses, open source; it was created with c++, Python and PureData. (Okay, so you have to know not one but three geeky things. So we’ll probably just watch the video and leave it at that.) As near as I can tell from the software page, Pd does the audio/video playback, C++ does the analysis, and Python is the database. There are no effects: this is really just a (very clever) playback mechanism.

For the world beyond cool online videos, I do think vocal input has been wildly underutilized in new music tech. I could see an interface like this completely reconceptualized for a wide variety of material (think melodic or timbral re-slicing) . . . and who’s to say you have to use 80s music videos?

Via Music thing.

Previously:
Recycle TV: Remixed TV Beats

DIYers, take note — this previous TV remixer project also released source code, in Max/MSP (no mucking about with C++ and Python code, if that scares you!)

We use SPICE when we’re modeling for Audio Damage products. We recreate the circuitry of whatever unit we’re modelling in order to better understand the signal flow; we also do some analysis using the SPICE model. (By “we” I mean Adam, of course. I sit in my office and play Rise Of Empires until he sends me a build to test.)

To which I say — hey, Rise of Empires sounds great! I’m sure you CDM readers can find something to do (like learning electrical engineering and circuit simulation software); I’m off to teach Chris how powerful my Persian Army is.

In all seriousness, I can’t wait to see Audio Damage’s upcoming recreation of the Mutron Bi-Phase effects pedal. (Warning: Bi-phase goodness, MP3s, and nostalgia after that link.) Any chance you’re busy simulating those circuits right now, Audio Damage?

And while we’re on the subject of geeky software meets cool musical applications, reader m15a spots this fantastic add-on to MatLab for analyzing and visualizing MIDI files. Can you say: slick visuals for your doctoral dissertation? (Oh, yes, I must be getting back to that one of these days. After one more Rise of Empires game, anyway.)

MidiToolbox: MIDI Analysis / Visualization for Matlab

Anyone got some nifty MIDI visualization images? Send `em in!