Turning music and sound into three-dimensional worlds often yields something that fields like a trip through space. But this feels like a real trip. Through pulsing, glowing starfields, “Versum”‘s audiovisual movements are brain-bendingly transformative. Artist Tarik Barri has created an integrated world of sound and image that makes the interface and the compositional realms seamless. It seems as though this really is a musical universe, through whose harmonies of the spheres you can fly like. Boldly going, indeed.

Ingredients: Max/MSP/Jitter, Processing, Java, SuperCollider, GLSL [the 3D shading language], and … some serious skill and time, I imagine.

The work has been in development for some years (not surprisingly, given the results). But it surfaced again as we brought up the 3Dconnexion SpaceNavigator hardware as a practical controller for 3D. See Create Digital Motion:
Look at Me, I’m Flying: SpaceNavigator Hardware + Blender

Tarik’s work resurfaced after a presentation in the UK. Reader janklug writes:

I’m just back from the M4_u Max/MSP/Jitter conference in Leicester (was great, btw), where Tarik Barri presented his project ‘Versum’, both as an installation and as a performance.
The user (and in case of the performance, Tarik) navigates through this incredible 3D-space-sequencer-universum with the help of a SpaceNavigator; glowing objects floating in this space produce sound, and as you approach them, they even give this nice doppler effect…
It was totally amazing to be able to float between pulsing rhythm-planet-objects and shiny drone-beams; navigation was easy and natural. Tarik uses a combination of Processing and Max/MSP; don’t know which one the SpaceNavigator is connected to.
Having tried this, I immediately ordered one; I think it also could be a great interface for M4L…

More information:
http://tarikbarri.nl/projects/versum
PDF documentation [2009]

Significantly, it’s really the act of flying that controls the music. That remains interactive, but it’s the movement through the three-dimensional space that determines what you hear. As the artist explains:

This virtual world is seen and heard from the viewpoint of a moving virtual camera with virtual microphones attached. This camera, controlled in realtime by means of a joystick (or any other kind of controller) moves through space, similar to how first person shooter games work. Within this space, I place objects that can be both seen and heard, and like in reality, the closer the camera is to them, the louder you hear them. So when the camera moves past several visual objects, you simultaneously hear several sounds fading in and out. Consequently, the way the camera travels past them actually causes melodies and compositional structures to be seen and heard.

The visual position of each object coincides with the panning of its sound: objects to the right of the camera will also be heard on the right, and those behind the camera will be heard from behind in case a surround speaker setup is used. This principle also applies to the Z-axis, meaning that sounds can be heard coming from above and below if the speaker setup supports it.

That’s the essential question, to me, when looking at 3D environments for music. What about the dimensionality will interact with the music? Is it something spatial, or will there be other sorts of interactions? (New Zealander-turned-Berliner Julian Oliver worked extensively with game engines, for instance. One solution for him was modifying the “gun” in those games to be an implement for doing things in the space, turning swords into plowshares after a fact by making the gun produce music rather than kill virtual entities.)

So, now you’ve seen some of the technical demonstration. But Tarik uses his work as an environment in which to make audiovisual performances. Here’s what some actual live playing looks like, in a beautiful, meditative piece called “Eleven”:

In fact, the biggest challenge to me of a piece this awesome is that you want an immersive environment, not just the small, rectangular screens that are often all festivals and venues can afford.

Holodeck, anyone?

More:

Compare the complex model of what a computer can use to control sound and musical pattern in real-time to the visualization. You see knobs, you see faders that resemble mixers, you see grids, you see – bizarrely – representations of old piano rolls. The accumulated ephemera of old hardware, while useful, can be quickly overwhelmed by a complex musical creation, or visually can fail to show the musical ideas that form a larger piece. You can employ notation, derived originally from instructions for plainsong chant and scrawled for individual musicians – and quickly discover how inadequate it is for the language of sound shaping in the computer.

Or, you can enter a wild, three-dimensional world of exploded geometries, navigated with hand gestures.

Welcome to the sci fi-made-real universe of Portland-based Christian Bannister’s subcycle. Combining sophisticated, beautiful visualizations, elegant mode shifts that move from timbre to musical pattern, and two-dimensional and three-dimensional interactions, it’s a complete visualization and interface for live re-composition. A hand gesture can step from one musical section to another, or copy a pattern. Some familiar idioms are here: the grid of notes, a la piano roll, and the light-up array of buttons of the monome. But other ideas are exploded into spatial geometry, so that you can fly through a sound or make a sweeping rectangle or circle represent a filter.

Ingredients, coupling free and open source software with familiar, musician-friendly tools:

• Two projectors
• A monome
• Processing, the elegant and artist-savvy free software for visual code
• Ableton Live and Cycling ’74′s Max for Live, acting as the interactive glue with the sound world
• Drumaxx, Image-Line’s tasty physical-modeled drum synth
• Native Instruments Battery, the sampled drum engine
• Eclipse, the free IDE, for Java coding in this case
• Community Core Vision and reacTIVision (based on our previous info, at least), free and open source community-based projects for making the interfaces you see in movies happen in real life.

Another terrific video, which gets into generating a pattern:

Now, I could say more, but perhaps it’s best to watch the videos. Normally, when you see a demo video with 10 or 11 minutes on the timeline, you might tune out. Here, I predict you’ll be too busy trying to get your jaw off the floor to skip ahead in the timeline.

At the same time, to me this kind of visualization of music opens a very, very wide door to new audiovisual exploration. Christian’s eye-popping work is the result of countless decisions – which visualization to use, which sound to use, which interaction to devise, which combination of interfaces, of instruments – and, most importantly, what kind of music. Any one of those decisions represents a branch that could lead elsewhere. If I’m right – and I dearly hope I am – we’re seeing the first future echoes of a vast, expanding audiovisual universe yet unseen.

Previously:
Subcycle: Multitouch Sound Crunching with Gestures, 3D Waveforms

And lots more info on the blog for the project:
http://www.subcycle.org/

Light-up grids of buttons are nearly commonplace, but the BlipBox is something different: its array of lights is also a sensor, making it both X/Y controller and light-up grid. And it’s designed to be completely open — firmware, hardware, schematics and documentation are all fully GPL-licensed and open source.

For those of us who aren’t ninja coders, it’s also easy to customize, thanks to friendly software (pictured below) .k for making nifty interactive animations on its display and support for the artist-friendly Processing code environment. As the creators describe it, it’s three (three!) pieces of hardware in one:

• a creative tool and musical instrument
• a large, high definition x/y controller with visual feedback
• a uniquely versatile MIDI and OSC controller

Lest you assume such oddities as this come only from non-musician hackers, these are designed by musicians. The project, built right in London, is available in fits and starts and stock becomes available, but a recent run was “Prices are GBP 140 for a complete ‘box with USB and MIDI interface, and 9v external power connection, in a black aluminium case with perspex side panels.” (To the team: apologies for giving you splashy publicity right as you have precisely none in stock. Readers, follow @cdmblogs on Twitter for updates. Or just follow their site:

BlipBox News

Side note: yes, we need to stop putting “blip” in the name of things. Guilty as charged. (I’ll be writing soon about the Blippo Box, which is … completely different.)

If you’ve got a whole bunch of colored lights, it seems only right to do something with them. Cacheflow sends a fun little hack with a Novation Launchpad. Of course, turning a Launchpad into a live EQ display means you can’t simultaneously use its lights to, like, play the Launchpad, but provided you have another controller, this could be a fun way to liven up your stage setup.

We looked at a free e-book on Processing last week; if you’re playing with Processing, you can now use a handy, free library to integrate this simple and elegant coding tool with your Launchpad.

The ingredients:

Music: unreleased Yo Soy Sauce tune.
plasticsoundsupply.com/​release/​yo_soy_sauce_-_juke_box/​

Built with the following components:
processing.org
tree-axis.com/​Ess/​
rngtng.github.com/​launchpad/

By the way, I’ve noticed a lot of great GitHub use for music projects. Music Hack Day Montreal even did their event planning on it. I wonder, is there a way – using GitHub itself or a hack with the GitHub API – for us all to build a little CDM community there? Git gurus, I’d love to hear from you about what you might like.

I love this controller, but I think we should keep it Platonic. Solid.

Sorry, geometry humor. See, the controller in question is constructed as a convex regular polyhedron, such that all its faces are themselves congruent regular polygons meeting at each vertex, and … uh, never mind.

Above, a stunningly gorgeous video from Polish media art group panGenerator, with some lovely chiming music following by the evidently-now-requisite dubstep demo. (Tip all of us could use, guys and gals – makeup. Styling. Now, they just need some post-production so you can’t see the IR sensors or the wires.)

Hedoco, also based in Poland, is the manufacturing and distribution partner, and donated this prototype. I love their stated philsoophy: “a unique brand that connects two trends: open source design and fashion.”

No, seriously. And, seriously, why not?

The controller itself looks actually quite lovely – and it’s open source hardware, too, from top to bottom. MIT license for the source, Creative Commons for everything else. (One trick: by choosing “Non-Commercial,” they don’t technically qualify under the Open Sound Hardware Definition. I’d suggest keeping the ShareAlike, since any commercial user would have to share modifications. I know not all designers are comfortable with that, though.)

Designer Jakub Koźniewski and whole team of panGenerator, seen in these parts building a kinetic sound organ out of cans, is behind this project, too. Ingredients:

• Bluetooth. (The revolution will not have wires.)
• Infrared distance sensors. (Though if you know what those are, you already knew that.)
• Arduino, the open source hardware prototyping platform.
• Processing. (Whatever. No one cool uses that any more.)
• OpenSoundControl.

Source code is available now on GitHub, with hardware schematics coming soon. And that could lead to an all dodecahedronal music festival. You may recall the work of Ted Hayes, whose Neurohedron has the same shape. Ted’s work, by contrast, works with a drum sequencer — meaning these two could even play onstage together. Ted and Jakub each tell us that’s coincidence, and when Jakub did realize the form had been taken, the two connected. Now I say we just need more. Dodecahedronstock. Polyhedrapolaooza. Platonaroo. Euclid Fair. Let’s make it happen.

On Ted’s Neurohedron side, he tells us that his work, featured at a Handmade Music event I produced in New York as well as at NIME, has gotten significant updates, so we look forward to seeing that. Previously:

Pretty, Nodal, Non-Linear Music, on iPad + iPhone and Big Dodecahedrons

Handmade Music NY 8/29: Meet the Musical Inventors, Pong to Dodecahedrons

More:

http://www.pangenerator.com/

http://www.hedoco.com/

Sonaur is a US$1.99 ambient toy for Android mobile devices, with on-screen creatures you can manipulate to generate sound. It’s notable not only for being a fun toy – and on a platform that hasn’t had as many fun toys – but because the tools used to create it are also highly accessible and free.

I’ve taught Processing, a code environment popular among artists and designers to people who never before imagined they could be coders. Pd (Pure Data), here in the form of libpd, is a free graphical patching cousin of Max/MSP. You can check out libpd, which allows Pd to run on Android, at our Pd Everywhere group.

Developer Miles describes the app thusly:

I wanted to create an app that lay somewhere between an ecosystem and a musical instrument. The hope is that sonaur requires less attention both, and still provides a reasonable amount of intrigue.

Sonaur contains three distinct lifeforms. You can interact with them individually, or together; creating new sounds and visual patterns for your enjoyment.

I was also curious if he had advice for people exploring this area.

Miles tells us that he found making both the art and sound generative – rather than pre-drawn and pre-recorded – made a big difference. He also suggests reading Andy Farnell’s book Designing Sound (now on MIT Press) as a way of learning both Pd and sound design, saying it “helped me a lot to create the sound of the flying insects.” Another tip: using vectors and not hard-coded pixel values makes your work adaptable to different devices. And, “Matt Pearson’s book Generative Art talks a lot about this but I’ve found too that randomness is great in controlled amounts.” I couldn’t agree more.

It’s really great work, Miles. And by the way, readers should never be ashamed of plugging their work, individual or group, free or for-sale. We love hearing about it, even if we can’t cover it all.

Find this at:
http://app.net/sonaur

Via discussion on Noisepages.

Also on Creative Applications Network, run by our friend Filip, which covers all sorts of these kinds of creations.

Working in open source code for any platform, Brazilian artist and developer Jeraman has produced a charming project that imagines musical interfaces in dynamic, whimsically-simple sketches. Like doodled knobs, cartoons of guitar pedals, interactive devices on the touchscreen control musical activity.

And because it’s open, cross-platform code, everything from a computer-powered multitouch table to an Android tablet could get in on the fun. Jeraman explains:

It is a digital musical instrument that allows the control of real-time recorded loops through collaborative performances based on relationships between sketches and sounds, intended to be ludic and playful.

Developed in Processing and OpenFrameworks, it mixes multitouch technologies with the interaction metaphor of guitar pedals, by using a FTIR DIY multitouch table – built with PVC tubes, tapes… – with a modified old keyboard as pedal, following a high-end / low-tech approach.

Here’s an example of performance using an early prototype. By the way, sorry for the poor musical quality… I’m not a professional musician!

The project is also open source and the code, that may allow the instrument to be ported to others plataforms like Android, iPad and cardboard boxes (yes! using this http://sethsandler.com/multitouch/mtmini/), will be released soon.

This is possible due to the independence of the gui module (developed in processing) and the looper module (developed in openframeworks).
for the communication between both, it was used OSC protocol.

Some pictures can be found at:
http://www.flickr.com/photos/jeraman/sets/72157626935130739/

For further informations, check this site (sorry, in Portuguese):
http://jeraman.info/illusio/

It’s a fascinating-looking project! Do keep us posted, Jeraman — and readers, if you happen to play with the code, let us know! Oh, and never apologize for Portuguese – it’s a gorgeous language!

See also some nice examples of Jeraman’s previous work:

See, previously:
Robotic Twitter Songwriter Generates Tweet Poetry

illucia is a project at the intersection of lots of forms of goodness and imagination. The physical interface – what you see first in the image and video here – is a DIY modular controller, complete with Buchla-style patch cords and arcade buttons and pretty knobs. But while that might lead you to believe we’re in analog synthesis land, the physical controls are just a tangible cockpit for what the artist calls “code-bending.” Connect the USB controller to software, and it modifies audiovisual games in modular, interactive fashion.

Since you probably don’t have this particular physical controller lying about your home, you can also use these games directly with whatever software or hardware interface you like, thanks to OpenSoundControl (OSC) interfacing.

The result: free, Creative-Commons licensed audiovisual “playgrounds.” The video at top is just a teaser, but there’s more to come, says creator Chris Novello. Chris tells CDM:

In a quick sentence: I make games that can play other games, and I’ve designed a physical console to interact with them.

The vimeo video’s accompanying text offers some explanation, but here is some extra info as well:

“Codebending?”

Codebending is the mutant daughter of live coding, circuit bending, and modular synthesizers. It gives software “patch points,” and allows for connections between otherwise-unrelated computer programs. Anything can control anything, and computer software tropes collapse into strange generative art. In terms of implementation, these patch points are really just OSC addresses, which allow for easy transmission between programs. http://www.paperkettle.com/codebending has more general information on codebending.

“Instrument?”

illucia is a physical instrument for making these connections – it gives the software patch points physical counterparts, and is a real life console for routing information between programs. The act of building and breaking connections (of exploring generative emergences via patching) becomes a form of play. In terms of hardware, I use Atmega mircocontrollers to encode the state of the device (connections between jacks, knob positions, and button presses) into a tiny byte array, which then gets sent to my computer over USB. This state is used to update a software model of illucia on the computer, which determines the routing between several games / programs. I use Max/MSP a lot for rapid development of routing utilities.

All of my games are written in Processing. They utilize classic arcade mechanics, but are more like modules in a modular synthesizer. I also add some twists, like:

“Soviet Life Sequencer,” which is a Tetromino step sequencer that is remixable by Conway’s Game of Life. I remember a CDM post about Conway sequencers, as well as one featuring the game Chime, but I think I’m the first person to combine Cellular Automata, falling Tetrominos, and a Step Sequencer : 0

also, “Pile of Secrets,” which is a codebendable text editor. Office software is a surprisingly fun companion to video game mechanics.

I’m in the process of releasing the first wave of info right now, but I’ve got a ton on the way as well!

For more information:
http://vimeo.com/22732435
paperkettle.com/​
twitter.com/​paperkettle
facebook.com/​paperkettle

Chris also says he’s sending over some more images, so I’ll add those to the story.

This should start the conversation, but if you have questions, fire away and I’m sure we can get Chris to answer them!

More images (all photos courtesy Chris Novello; used by permission):

I’ll be in Stockholm the week of 27 September, speaking at Stockholm’s Android Only conference Wednesday/Thursday 29-30. I’m particularly excited by the lineup, because it emphasizes pushing the envelope of Android and cross-platform development. I’ll talk about how Processing for Android can allow “sketching” audiovisual apps on mobile. Martin Roth of RjDj will talk high-performance audio development. There’s also discussion of advanced uses in embedded hardware, cross-platform PhoneGap development, CouchDB and JavaScript goodness, hardware-accelerated graphics … all good stuff, and all potentially relevant to creative music and visual development not only on Android but future platforms generally.

I need to revisit Android on this site, but in the meantime, if you’re interested in the platform, MusicRadar did an exceptional round-up:
The best Android music making apps in the world today

I’ll have more information for everyone (whether or not you’re in Stockholm) in time for the conference on CDM.

But beyond Android…

Calling All Swedes! CDM is co-hosting a party Wednesday night 29 September at the conference venue in Stockholm, with .SE, Dialect, and most importantly the fantastic Swedish music mag Studio Magazine.

If you’re a CDM reader and music artist in the Stockholm area and have a project/studio to show off, and/or if you’re up for playing (with or without an Android device involved), please let me know right away, send some music/files/info, and we can discuss details. We’ll finalize the lineup over the weekend. I’ll also try to make a visit or two in my free time, limited as it may be.

For all the wonderful tools and toys for sound out there, sometimes you want to find the couple of tools that, like a great kitchen knife, can accomplish the majority of what you actually need. (And as with the kitchen knife, while it may not eliminate your desire for all those other gadgets, it’s worth some sharpening.) So it is with something like Csound, the tested-and-tried, free synthesis tool. Jim Aikin looked at the QuteCsound front end recently, which puts the power of Csound in a more friendly work environment.

Via Synthtopia, there’s also now a screencast series that covers using QuteCsound, starting with digging into presets. (Yes, that’s right – presets. And here you thought you were going to have to do a lot of coding to have any fun.)

I find two YouTube users uploading how-to screencasts:
http://www.youtube.com/user/mantaraya36 (author of the series starting at top)
http://www.youtube.com/user/ketchupok (start with “Where to start?”

Also worth following is Jacob Joaquin’s excellent Csound Blog, hosted on Noisepages:
http://csoundblog.com/
and on Twitter, follow @TheCsoundBlog

It’s very early in development (“alpha”), but Jacob is already doing amazing things integrating Processing, the non-coder-friendly, artist sketchbook-style coding language, with Csound, in a new library called Csoundo. That’s an ideal combination, because you can do logic and visuals quickly in Processing, then turn to Csound for audio. This is where I imagine work in two of Csound’s most popular rivals – the object-oriented, OSC-savvy SuperCollider and visual patching, Max-descendent Pure Data – may lead, as well. Check out Jacob’s roadmap for more.

By the way, I hear some folks are having some trouble building QuteCsound on Ubuntu, so I’ll see what the issue is, and write up some instructions and send them over to Jacob for his blog.

It’s a good time for Csound and free synthesis in general. With this work accelerating, I think doing a series of absolute-beginner tutorials will be very doable soon. And there’s no reason you can’t integrate a tool like this with your favorite host of choice, from Ableton to Cubase.