Processing is an open-source coding tool, built in Java, designed specifically to be versatile for artists and friendly to non-coders. Code is elegant and simple, but can take advantage of all the potential power and performance (no, really) of Java. Java really can be fast enough to use in live performance situations, though its one Achilles’ heal is that automatic memory management — the very thing that makes coding easier, via something called a garbage collector — can make sound glitchy at lower latencies. (JavaSound seems worst on Mac OS X, as implementation of the sound API by Apple hasn’t kept pace with improvements in Java audio on other platforms. It is possible to build a real-time-ready Java implementation that performs as well as languages like C++ for audio, but right now there’s not yet a mainstream implementation of this type.)

That doesn’t mean Processing isn’t useful for musical applications. With experimentation, sound libraries like Minim can perform quite well, especially if extreme low-latency is unnecessary. (see Processing’s libraries page for more.) And you can always use Processing as a visual front end, while sound comes from elsewhere (Max, Pd, Reaktor, or even Ableton Live or a plug-in.)

There’s plenty of incentive to work with the environment as an artist. People who never coded before are able to build entire projects in Processing, not just uber-programmer-geeks. Even experienced coders can find it a fast way of experimenting with ideas — sometimes better-suited to tasks that are more difficult with patching environments. Despite all the hype around Flash/AIR/Flex and Silverlight, I find Processing easier to develop in, and you have far more robust development options, free and open source tools and libraries, and genuine OpenGL 3D capabilities.

I put out a call for people working with Processing for music, and we’ve already got a handful of interesting examples. Because of the open community around Processing, code is available for a couple of the ideas here, so you can have a peek and learn from fellow Processing coders.

nodeSeq from Jared Arave on Vimeo.

We like to push the outer envelope of music technology geekdom. But what if you’re also an obsessed web geek? Then you start playing the data encoded in a Website design like nytimes.com as a musical instrument.

A new patching environment called Lily, inspired by tools like Max/MSP, works its magic using JavaScript inside a browser. So turning your browser into a music tool becomes more practical. And Lily supports the network-savvy OpenSoundControl (motto: “it’s not MIDI!”), so you can hook up an OSC controller like the Monome and jam with Firefox and the New York Times.

Finally found a use for the NY Times from Bill Orcutt on Vimeo.

How does it work? Get prepared for some Web technospeak, kids!

When the patch starts, the browser enters a DOM inspection mode and mousing over a DOM element highlights the node. Clicking on a node writes the element’s data (its innerHTML value if it’s a text element or the binary data if it’s an image) as a sound file and the file is then loaded in a quicktime player in the patch. The sounds can then be triggered using OSC messages.

Hey, where’d everyone go?

If DOM models don’t exactly get your pulse racing, here’s a strange and elegant physics-based sequencer hooked up to Reaktor. Fans of Processing, that environment is also capable of similar stuff; this is even modeled on the Processing-compatible traer physics library.

SVG Midi Sequencer from Bill Orcutt on Vimeo.

Ready to get going with this yourself? Lily is now in public beta, ready to run for free on Mac, Windows, or Linux. Browser not included.

Lily Public Beta 1

Gobs more examples and documentation on the blog

Previously: Browser Beatboxes and the Rebirth of Max-Like Patching

Looks can be a powerful agent for changing how we think about sound. Pairing liquid, organic speakers with equally fluid and dynamic visualizations, the launch of Muon last month in Italy made this principle readily apparent. I’m all about lo-fi, cheap gear here on CDM, but if you absolutely must launch luxurious aluminum speakers with spectacular animated visuals at a posh party in an Italian salon, I sure won’t complain. Pass the prosecco, please?

Muon Project Page, documentation videos at chrisoshea.org
See coverage at ze | d | esign, toxi’s project blog, MoCo Loco, elsewhere. (Yeah, CDM’s motto is: cover things last. Was a bit busy with Maker Faire!)
Created by Moving Brands

Details on the installation and how it was done:

When those infants graduate from playing with computer music-controlling pacifiers, they can move on to blocks. Our friend Nat points today to a brilliant tangible computing interface that generates sequences of musical events. (Also seen last week on Matrixsynth.) The transmitter (some sort of RF operation) communicates with a receiver connected to the compute, and the computer generates the sounds. Nat’s unsure of the creator, but the YouTube videos were posted by “traer”, which I’m guessing is Jeffrey Traer Bernstein, who has developed a lot of interesting projects and some great stuff for the Processing coding environment we’ve fallen in love with over at Create Digital Motion. (Traer, whoever you are, feel free to say hi!)

Here’s my personal favorite. (YouTube has some other configurations; one even includes beat juggling.)

DIY Interactive Blocks: TileToy

Ever wish you could go beyond oohing and ahing on the Web and experiment with technology like this yourself? You’re in luck: one project has released extensive open source documentation. TileToy is a similar concept in that it uses blocks and radio frequency-to-serial connection to a computer. (It adds a nice extra: interactive LED matrices on top of the blocks let you display different patterns.) The system works via a patch built in Max/MSP. (It could also be easily ported to Pure Data or another free environment if you’re on a budget.) Even if just part of this project appeals to you — like the RF aspect, or the grooving LED lights — it could be well worth checking out both the project and the “do it yourself” section:

TileToy.org

The DIY section includes a parts list, assembly guide, complete guide to controlling the hardware, software for Max/MSP, and software for communicating with the PIC. Might want to cut your teeth on something a little simpler if you’re a beginner, but file this away for when you’re ready to make the jump! (Thanks for the reminder of this goes to Chris at Pixelsumo.)

TileToy is the latest of a series of similar projects. We Make Money Not Art covered Digital Cubes by Simon Schiessel way back in January 2005, and from 2003, the “father” of these kinds of projects is Ryoto Kuwakubo’s Block Jam for Sony (thanks, again, Chris!):

Block Jam [Sony Design]

Other good tangible computing resources out there? Let us know!

Previously:
Babies Making Electronic Music on Video, and More on Traditional Japanese Instruments (Interactive teething ring, also built with Max/MSP)

[tags]DIY, hardware, alternative-interfaces, design, how-to, physical-computing, tangible, electronics[/tags]