Weird Fishes: Arpeggi from flight404 on Vimeo.

Plenty of bands have jumped on the "remix generation" bandwagon, releasing music to be remixed and sampled and visualized as a publicity stunt. But, then, plenty of bands aren’t Radiohead. Readers here may have been disappointed that our favorite superband didn’t embrace Creative Commons sharing when announcing their iTunes-only stems. But a number of the artists we follow came up with some brilliant work.

In Visuals: Robert Hodgin, aka Flight404, has the enviable job of exploring new visual expressions as his day gig of sorts. Working primarily in code developed in the open-source, Java-powered Processing, he develops a technique and then iterates and iterates on it until it goes from computer gimmick to refined artistry. He blogs that process, as well, pushing forward the rest of the Processing community. His video above uses abstract, generative processes to visualize Radiohead’s "Weird Fishes", but is developed enough to become organic. It’s a voyage under the sea. Via our sister site, Create Digital Motion.

In Music: A number of readers tackled the Radiohead remix contest. Here’s my favorite: our friend Alan Molina created a sparse string accompaniment that spotlights Tom Yorke’s vocal part. He explains:

Thanks for listening!  I actually recorded and mixed all the strings.  

They are all a violin (just lots of layers of me).  My profession is an orchestral violinist–this remix was an outlet to do something different!

I used Ableton Live 7 for the effects, and used  the kind of mic that clips on behind the bridge of a violin.  Done on my couch in front of my computer!

Of course, the other direction to go is stretching the tune past the point of recognizability, with strange bizarro-universe remixes pulling the tune to experimental glitch and faux-punk. Here are a few of the more unusual takes on their music:

Everything has an “API” these days, but what that means in practice is often not so exciting. You can make little widgets for Facebook, or post recent Twitter messages, or do other simple developer tricks. Echo Nest’s “Musical Brain” API is more far-reaching: it’s an API for music. All music online. The first of a series of developer tools, “Analyze” is designed to describe music the way you hear it, figuring out tempo, beats, time signature, song sections, timbre, key, and even musical patterns. More developer tools will follow.

Twelve years in development, the Musical Brain is a bit like a digital music blogger. It’s been crawling the Web while you sleep, reading blog posts, listening to music to extract musically-meaningful attributes, and even predicting music trends. It’s like almost like a robotic, algorithmic Pitchfork. (And I’m serious — it may be April Fools’ Day, but this is real. That’s what the “brain” claims to do, backed by research at UC Berkeley, Columbia, and MIT.)

What does all of this mean? The Musical Brain may not be replacing your friendly local music blog any time soon, but what it can do is infuse some musical intelligence into applications like music visualization. The Matmos album at top used the Analyze algorithm to map the timbral profiles of songs on an upcoming album; that graph was then rendered by an artist in watercolor, translating the digital into traditional paint media.

The Analyze API could also enable everything from Web music apps and mash-ups to live audiovisual performance tools, or even smarter music games. That’s the reason co-creators Brian Whitman and Tristan Jehan, both with PhDs from the MIT Media Lab, chose to open up development to a broad audience. I got to speak to Brian a bit about the project.

Processing is an open-source coding tool, built in Java, designed specifically to be versatile for artists and friendly to non-coders. Code is elegant and simple, but can take advantage of all the potential power and performance (no, really) of Java. Java really can be fast enough to use in live performance situations, though its one Achilles’ heal is that automatic memory management — the very thing that makes coding easier, via something called a garbage collector — can make sound glitchy at lower latencies. (JavaSound seems worst on Mac OS X, as implementation of the sound API by Apple hasn’t kept pace with improvements in Java audio on other platforms. It is possible to build a real-time-ready Java implementation that performs as well as languages like C++ for audio, but right now there’s not yet a mainstream implementation of this type.)

That doesn’t mean Processing isn’t useful for musical applications. With experimentation, sound libraries like Minim can perform quite well, especially if extreme low-latency is unnecessary. (see Processing’s libraries page for more.) And you can always use Processing as a visual front end, while sound comes from elsewhere (Max, Pd, Reaktor, or even Ableton Live or a plug-in.)

There’s plenty of incentive to work with the environment as an artist. People who never coded before are able to build entire projects in Processing, not just uber-programmer-geeks. Even experienced coders can find it a fast way of experimenting with ideas — sometimes better-suited to tasks that are more difficult with patching environments. Despite all the hype around Flash/AIR/Flex and Silverlight, I find Processing easier to develop in, and you have far more robust development options, free and open source tools and libraries, and genuine OpenGL 3D capabilities.

I put out a call for people working with Processing for music, and we’ve already got a handful of interesting examples. Because of the open community around Processing, code is available for a couple of the ideas here, so you can have a peek and learn from fellow Processing coders.

nodeSeq from Jared Arave on Vimeo.

Chris O’Shea sends along his latest project, a collaboration with sound designer and composer Owen Lloyd called Drop Spin Fade. Part of the Future of Sound tour, Drop Spin Fade allows users to position, sculpt, and play with sound in a 3D environment using gestural control.

Drop Spin Fade

The music/sound environment: Through a series of iterations, Chris and Owen have started simple and built increasingly-sophisticated sonic control using the setup, manipulating granular samples by spinning and bouncing them around the space. It’s not just positioning at work here: you can actually shape the sounds you’re hearing by interacting with the geometric forms in the environment. Eventually, the designers hope to give users more compositional control, making this into a kind of 3D sequencer.

The guts behind the scenes: The work was built to showcase the Illustrious positional sound system, which can use positioning data to create 3D sound environments. For control, the project uses the Gametrak game controller hardware, which you may have seen used in inexpensive golf and other sport games. It happens to be a very nice gestural controller, as well, with extremely low latency when compared to video camera tracking solutions. Visuals and hardware interface are performed in Adobe Director, routing positional control to Illustrious via MIDI and playing a live sound patch built in Max/MSP via OpenSoundControl data. There will be yet another piece as work proceeds on support for the Nintendo Wii controller.

I’m actually quite surprised that more work hasn’t been done with 3D interfaces — though I suppose I shouldn’t be too surprised, as it’s extremely labor intensive! 3D has focused largely on positioning, but with powerful hardware and software capabilities bring 3D to the masses, 3D interfaces are surely next. Iterations and shared research are vitally important to any medium advancing, so I hope we’ll have more projects in this area. (I happen to be working on something different but related in the 3D space in my own research, which I’ll share when I’m ready.)

Previously from Chris O’Shea land: Muon Speakers, with Processing Visuals

All rational arguments for development on mobile phones aside, some phone owners are driven by the irrational, purely personal need to tap into their phones and do stuff. Our friend Sebastian Tomczak reports back on a first experiment with the mobile version of Processing, the Java-based “coding for artists” tool.

Simple Live Synth for Phone

This is nothing fancy, but that’s the point: simple is the best way to get started. It’s the sound version of “Hello, World.” In programming, I find, even simpler than I think is necessary is often the best place to start. Erm, especially if you’re me and screw things up really quickly.

Mobile Processing is an example of what the future could look like if we could program phones with standard Java libraries. After all, your phone is a computer — often a pretty expensive computer, considering. And then, when you upgrade, it becomes useless without service. Recycling phones into something useful is only natural. And because they’re simple computers, there’s an opportunity to return to the days when programming was actually considered a killer app — for “end users.”

And yes, this does fit in the category of “things you can’t do with Ajax on iPhone and can do with zillions of other, cheaper phones.” I’m glad to be over the “what might actually impact iPhone sales” category and into the “completely useless phone applications that appeal only to insanely geeky musicians” area again. Phew! Much better.

Non-Processing/Java geeks can stop reading here, but I should elaborate a bit.

Looks can be a powerful agent for changing how we think about sound. Pairing liquid, organic speakers with equally fluid and dynamic visualizations, the launch of Muon last month in Italy made this principle readily apparent. I’m all about lo-fi, cheap gear here on CDM, but if you absolutely must launch luxurious aluminum speakers with spectacular animated visuals at a posh party in an Italian salon, I sure won’t complain. Pass the prosecco, please?

Muon Project Page, documentation videos at chrisoshea.org
See coverage at ze | d | esign, toxi’s project blog, MoCo Loco, elsewhere. (Yeah, CDM’s motto is: cover things last. Was a bit busy with Maker Faire!)
Created by Moving Brands

Details on the installation and how it was done:

Hello, coming to you live from San Mateo and the Maker Faire hosted by Make Magazine. If you stopped by the CDM booth today, I hope you stick around a while. This site, for musicians, DJs, composers, and soundmakers, covers the audible side. Create Digital Motion covers VJing and live and interactive visuals (for an audience we like to call visualists). We cover store-bought hardware and software as well as DIY, and love having everyone from beginners to advanced readers around.

Here are some links to more information on some of what I was showing:

Sensors and MIDI: Working with sensors and physical inputs is what some term “physical computing.” We’ve covered lots of physical computing projects here on CDMusic.

I wrote up an introduction to MIDI for Make 07 which would be a good place to start if you’re relatively new to MIDI.

The MIDI sock monkey and infrared and touch sensors are powered by Lady Ada’s MIDIsense, as introduced here:
Sensors-to-MIDI with the MIDIsense Board. See also the Arduino board, which we’ve covered mostly on the motion side.

Barcode scanner to music converter, powered by Processing, a Java-based development language that makes programming friendly for artists. Check out the official Processing site, Processing Blogs aggregator, and Processing coverage on our sister site, Create Digital Motion.

Live visuals: The live video-processing “mirror” via the MacBook was built in Max/MSP/Jitter. Check out our ongoing Jitter coverage on CDMotion. Nope, no “source code” / source patch available yet — it’s a bit too messy, the reason you don’t see more open-sourced Max patches!

And totally off-subject: The keyboard I was using is a ReMOTE 37SL from Novation. Nothing to do with Maker or Maker Faire but very handy to have around … and well worthy of combining with other DIY hardware.

What good is open source hardware? How about hacking in additional functionality by adjusting the code and electronics? The creators of Monome have a new video demonstrating some of the possibilities for user modifications to their hardware project. Included:

1. Optical encoders allow additional multi-touch interactions, for a game of Life (the grid-based life simulation of yore)
2. Two knobs make the world’s most expensive Etch-a-Sketch
3. An accelerometer let you tilt and shake the controller while using it, with full LED feedback (my personal favorite)

Monome documentation of process and prototyping
Via Open-source grid controller - the monome (video)”, Make:blog

The software side is all programmed using Max/MSP and the open source project Processing, which we’ve been following avidly on Create Digital Motion. A reader on Make points out that the device is pricey at $500, but if you have any similar ideas on any scale, you can learn something from the open documentation.

Now just imagine what you could do if Korg posted documentation for hacking their hardware and rewriting their code. A lot of these open source projects have been big business — made bigger by word-of-mouth promotion. Could more music makers profit from letting customers hack their gear? Absolutely.

Previously: Hackable, Playable LED/Pad Music Interface

Programming offers incredible possibilities for music creation, and with the free Processing development environment for Mac, Windows, and Linux, even non-programmers can get into the artistic horizons of code. But code doesn’t always think like composers do. That’s why the new sound library jm-Etude looks promising:

New sound library: jm-Etude [Code & form]

jm-Etude for Processing, and the Java library jMusic on which it’s based, allow you to structure your code more musically with notes, phrases, parts, and scores. Combine this with the Sonia synthesis library for Processing (or, for Java development, the corresponding JSyn plug-in), and you have a serious compositional environment. I can also see this as appealing to people coming from the land of Csound and wanting something that lets them code notes and other musically-useful units.

I’d love to see a similar library that helps deal with performance environments, helping structure into scenes and elements — Chris from Pixelsumo was just asking me how you might use Processing in a VJ performance. For live music or visuals, it’d be helpful to have a library that lets you structure what you’re doing over time for performance. Anyone know if there’s something like this already out there (short of coding the thing yourself, which might ultimately be better)?

[Updated:] I missed the major point of this, which is that it lets you use Processing as an interactive MIDI sequencer. (Follow that link for a promising-looking interactive table, built in the “app no one wants to use any more,” Director.)