Turning music and sound into three-dimensional worlds often yields something that fields like a trip through space. But this feels like a real trip. Through pulsing, glowing starfields, “Versum”‘s audiovisual movements are brain-bendingly transformative. Artist Tarik Barri has created an integrated world of sound and image that makes the interface and the compositional realms seamless. It seems as though this really is a musical universe, through whose harmonies of the spheres you can fly like. Boldly going, indeed.

Ingredients: Max/MSP/Jitter, Processing, Java, SuperCollider, GLSL [the 3D shading language], and … some serious skill and time, I imagine.

The work has been in development for some years (not surprisingly, given the results). But it surfaced again as we brought up the 3Dconnexion SpaceNavigator hardware as a practical controller for 3D. See Create Digital Motion:
Look at Me, I’m Flying: SpaceNavigator Hardware + Blender

Tarik’s work resurfaced after a presentation in the UK. Reader janklug writes:

I’m just back from the M4_u Max/MSP/Jitter conference in Leicester (was great, btw), where Tarik Barri presented his project ‘Versum’, both as an installation and as a performance.
The user (and in case of the performance, Tarik) navigates through this incredible 3D-space-sequencer-universum with the help of a SpaceNavigator; glowing objects floating in this space produce sound, and as you approach them, they even give this nice doppler effect…
It was totally amazing to be able to float between pulsing rhythm-planet-objects and shiny drone-beams; navigation was easy and natural. Tarik uses a combination of Processing and Max/MSP; don’t know which one the SpaceNavigator is connected to.
Having tried this, I immediately ordered one; I think it also could be a great interface for M4L…

More information:
http://tarikbarri.nl/projects/versum
PDF documentation [2009]

Significantly, it’s really the act of flying that controls the music. That remains interactive, but it’s the movement through the three-dimensional space that determines what you hear. As the artist explains:

This virtual world is seen and heard from the viewpoint of a moving virtual camera with virtual microphones attached. This camera, controlled in realtime by means of a joystick (or any other kind of controller) moves through space, similar to how first person shooter games work. Within this space, I place objects that can be both seen and heard, and like in reality, the closer the camera is to them, the louder you hear them. So when the camera moves past several visual objects, you simultaneously hear several sounds fading in and out. Consequently, the way the camera travels past them actually causes melodies and compositional structures to be seen and heard.

The visual position of each object coincides with the panning of its sound: objects to the right of the camera will also be heard on the right, and those behind the camera will be heard from behind in case a surround speaker setup is used. This principle also applies to the Z-axis, meaning that sounds can be heard coming from above and below if the speaker setup supports it.

That’s the essential question, to me, when looking at 3D environments for music. What about the dimensionality will interact with the music? Is it something spatial, or will there be other sorts of interactions? (New Zealander-turned-Berliner Julian Oliver worked extensively with game engines, for instance. One solution for him was modifying the “gun” in those games to be an implement for doing things in the space, turning swords into plowshares after a fact by making the gun produce music rather than kill virtual entities.)

So, now you’ve seen some of the technical demonstration. But Tarik uses his work as an environment in which to make audiovisual performances. Here’s what some actual live playing looks like, in a beautiful, meditative piece called “Eleven”:

In fact, the biggest challenge to me of a piece this awesome is that you want an immersive environment, not just the small, rectangular screens that are often all festivals and venues can afford.

Holodeck, anyone?

More:

Texas-born, Detroit-raised, New York-based artist Matthew Dear has a new EP, to be followed by a full-length in 2012. It’s worth mentioning now for two reasons: one, the driving, “chugging” rhythms of the single, “Headcage,” will pop into your head and stay there, led by Dear’s vocal ability to croon and groove simultaneously. Second, the opening of this video may well make your mind go squish. The work of London-based director Morgan Beringer, seen previously milking monochrome textures out of another Matthew Dear collab, the film makes it look like some very colorful part of the Earth’s crust turned a film into magma. It settles down, but the opening frames are to me transcendent, especially when set to a similarly-morphing sonic backdrop.

You can stream and download the single via SoundCloud:

Matthew Dear – Headcage by ghostly

More on the upcoming release from Ghostly:
Matthew Dear: Headcage

The music writing echoes a bit for me Eno and Byrne on My Life in the Bush of Ghosts; perhaps channeling that, the album art by Michael Cina for Dear has washes of indistinct color, like a kaleidoscope set into motion, then blurred. Ghostly reports Dear co-produced the single with Van Rivers and The Subliminal Kid, vets of the acclaimed self-titled Fever Ray. The rest of the album is full of other vocal and producer collaborations. More on this when it arrives.

The possibilities of a microphone and the world are limitless, so as this week we looked at a recording of music made with playgrounds, a mic, and Ableton Live, readers responded in kind with a fantastic spectrum of sampling-inspired, real world-produced musical wonder. From comments, a few examples:

Diego Stocco, a favorite sound designer on this site, ventures in his latest installment into a dry cleaner. Clean, wrinkle-free clothes and great music – see, you don’t actually have to choose. See top.

“Vega” by CDM reader Cordovan Music (Gregory Reeves), is an eerily-lovely ambient score made from LA’s freeways – and perhaps an ominous, if beautiful, portent of a lot of us driving on said freeways for NAMM in January.

Vega by Cordovan Music

From the experimental and minimal ambient release “Photic”, “Vega” is made entirely from field recordings taken on Los Angeles freeways.
About the album:
Gregory Reeves is a Los Angeles-based electronic artist, composer, and musician. His work can be heard on A&E, History Channel, FUEL TV, Universal, EA Games and many more. His sound art and installation work includes pieces for the Gaffa Gallery, Sydney, as well as creating a soundscape for the huge geodesic dome at the Peats Ridge festival in Australia, New Years eve. Electronic releases under various aliases have received winning reviews in XLR8R, URB, JazzTimes, and Rolling Stone. Remixes include Bob Marley and the Wailers (“One Love” from “Roots Rock Remixed” on Tuff Gong/Quango), Sarah Vaughan, and others.
“Photic” is a collection of minimal, generative ambient works. “Vega” is made entirely from manipulated field recordings taken on Los Angeles freeways. “Dark Field” uses convolved urban sounds to create a brooding ambient soundscape (the piece was originally composed for the DUOscope multimedia installation in Sydney). “Anemone” was created using undersea video footage to generate musical events, while “Haiku in C” is based on classical Japanese Gagaku music. For fans of Brian Eno, Murcof, and Loscil

Reader Cillian John, in the more pedestrian-friendly city of Stockholm, looked instead to escalators.

On and on by cillianjohn

He writes:

The urban environment is full of inspiring noises. I particularly enjoy the mechanical rhythms you hear all round.
The rhythmic element of this track is a field recording of escalator I recorded while on a trip to Stockholm.

Audiovisual remix superstar Pogo, aka Perth, Australia-based Nick Bertke, is embarking on a mission to “remix the world” as has inspired the headline of this post. Check out the spectacular results, and perhaps even get involved (thanks, BenAlex):

http://www.pogomix.net/remixing-the-world/
Kickstarter on Tibet

(I’m awaiting further details, but I am personally angered by the apparent mistreatment of Bertke which recently resulted in a 10-year deportation from the United States of America – and even that only after seemingly-extreme prison time and detainment that required the intervention of the consulate of New Zealand; you can follow on his blog. I’ve been deeply frustrated by the apparent targeting of artists, whatever the requirements of the law, in the US and Canada. More on that soon; anyone with expertise or experience in these matters, I’d love to hear from you. I feel it’s worth bringing up, as I’m sure that someone would have done so in comments as this story has spread.)

I’m sure there are many, many other examples, but this seems to me a nice place to begin – and fertile ground for inspiration to make something yourself.

This update I believe is worth a second post, as it makes visible the otherwise-mysterious algorithms producing music in our previous post.

And yes, I believe this is “music,” naysayers aside. Whether it’s good music is in the ears of the listener, but if you can describe this much sound with this little code, imagine what’s really possible in computer music. Whatever it is you want to hear, it’s within the power of your imagination to describe it, on a score or in code, either one.

Thanks to none other than Stephan Schmitt for the tip.

Drasko Vucevic, Toronto- and Santa Monica (California)-based sound designer and artist/composer, is apparently not only interested in playing alone. His upcoming interactive installation at Toronto’s Royal Music Conservatory will have an audience jamming along live via Twitter. And the artistry is crowd-sourced, too – with a range of artists already onboard, Drasko is calling on musical and visual artists (read: you) to be involved with sounds and visuals.

Drasko has sent along extensive notes, so I’m going to let him speak for himself:

I am working through both Drastic Music and Eksperimental (my companies) to create an interactive installation experience for this year’s TEDx Toronto conference taking place at the Royal Music Conservatory.

I am also doing an interactive music performance – an audio visual performance with a complete 3d/visual journey, which allows the audience (physical and digital) to collaborate with me by triggering audio and video elements in real time through their tweets.
So far, I have a few great artists contributing their time to create some beautiful visuals for this first of its kind interactive real-time jam.

Installation [Call for Audio]:
The installation concept is based around redefining collaboration. We are doing this by using real-time data (motion, color, sound, light) and tweets relating to TEDx to trigger, control, manipulate and compose audio elements on our back-end audio library (ableton). This is all happening through Processing, Max 5, Arduino, Ableton.

The massive back-end sound library contains loops, melodies, soundscapes, fx, you name it – produced by some great artists. The beauty of this is also that artists which have never before collaborated, will be remixed and mashed up solely by the audience, in relation to key words, discussions, movement, etc.

So far, the artists contributing audio content are:

Yoko K
Trifonic
Richard Devine
Drumcell
Audioandroid
David Della Santa
Darrin Wiener
Audionerve
Box of Toys
Lodewijk Vos
Matt Davis
Adrian Ellis
Andrew Lauzon
Drasko V

Performance [Visual Call]:

As I mentioned, my performance will be very interactive, musically and visually. Both audio and video elements will be triggered based on tweets in real time. I have some great visual artists contributing their time, such as Murat Pak, Yongsub, Charlie Vicetto, etc, but am looking for more, to create elements for the performance. They would of course get the great exposure of TEDx brand, be mentioned everywhere online, and will be in the final video spread throughout blogs once we launch the digital music version.

So how does the call for works … work?

The TEDxToronto conference is on September 23. Here’s how all the pieces come together for that and how to submit:

Musical system uses a massive library of sound structures – loops, melodies, fx, soundscapes and more – triggered and manipulated solely by tweets (relating to TEDxToronto) and motion, color, sound and light within the RCM venue.

Over 12 compositions will be recorded on the day of the conference. The arrangement, structure and sounds used will depend only on the key words used in tweets, the types of emotional replies, and physical interactivity within the venue.

Beauty of having some great artists be remixed and mashed up by the general public, in a very subliminal way. (again – Through their emotional replies, and physical movement)

This posting is a call to artists who may wish to apply to contribute their audio content and be a part of the soundtrack we will create that day. They should contact drasko (at) drasticmusic (dot) com with a link to their portfolio and we will take it from there.

Our installation progress may be followed on my personal site (drasko-v.com) or via Drastic Music or Eksperimental blogs.

We plan to expand the installation idea and bring it online for an ever-changing musical universe manipulated by truly organic methods (digitally and physically).

Interested to see how this will all come together. We’ll be watching. If you submit, and if you attend, let us know how it goes.

More on Drasko:
http://drasko-v.com/

Sometimes, to quote Depeche Mode, words are very unnecessary. Instead, lose yourself for a few minutes in the vibrating mathemagical lands of numbercult, audiovisual immersions in which sound and geometry fuse in a strange, abstract dance.

Their most recent creation, found via Richard Devine’s prolific Facebook wall and posted earlier this summer, explores an actual audiovisual sequencer. See it at top:

Connected is a graphical/musical sequencer system. a three way flow of information, between graphics, sound and external triggers shape the composition. Recorded in real-time.

But actual functioning interfaces aside, I’ll leave you with some other video clips that traverse similar territory, these syncing up separate visual and audio systems.

These folks make music, too – have a listen to their album, at bottom. And that shifts to body candy, as in, for your butt, with danceable grooves.

All three of the above videos combine vvvv – the Windows-only, graphical patching environment for powerful 3D effects – with Ableton Live for sound.

But lest you think it’s all abstraction, have a listen to their excellent dance release on Bandcamp. Downloading:

http://www.numbercult.com/

By the way, ever wondered what visual software people are using? So did we. Don’t miss this look on our sister site, Create Digital Motion, including where vvvv fits on the spectrum:
What Visual Software Readers Use: Some Clear Favorites, Plenty of Diversity, in Census Results

Presenting artists from around Earth to viewers around Earth, a center in Beijing has found a way to do live performance for a sleepless world without waking the neighbors.

Let me start out by saying this: if you read CDM from China, say hello. We’re in the wrong language, we have no translation, and I seriously doubt our Texas data center is delivering this site with any speed (until we upgrade to an international CDN), but the only reason I still run CDM is in order to reach people, and to hear from a wider world that knows things I don’t, and imagines things I can’t. And if you’re not in China, we still get very nice, high-quality video streaming. Think about that for a second: we’re on a planet that has a circumference between poles of about 24,860 miles (40,000 km), and we can share video and recording as if we’re in the same room. That’s pretty ridiculous; almost more impressive than recording itself. (I had similar thoughts a few years ago, somewhere in the jetlag going from New York to its nearly-furthest point on the globe, Perth, Australia.)

Shan Studios is a platform for artist conversations, residency, audiovisual performance, and learning. If you’re in Beijing, China, this center is forging connections between European audiovisual practice and China — and it’s a place where you can go to learn tools like Ableton Live, SuperCollider, and Max/MSP/Jitter. But if you’re anywhere else in the world, tonight/today you can watch a performance of audiovisuals. (That’s 11:59p Beijing time, 4:59p London time, 11:59a New York time).

The best part of this: by broadcasting to the Web but being silent in person, the performance won’t disturb the neighbors.

Using an array of webcams, DIY synthesizers, medical equipment, projectors, busted radios, and many unconventional instruments, the performers will create a completely immersive audiovisual experience in the Shanstudios sound laboratory. But the actual performance space will be silent – as to not wake the neighbors and simultaneously experiment with the best distortion box ever created (the Internet!) – all sounds will be processed digitally and virtually. The event is entirely exploratory and will hopefully lead to greater investigation of the Internet as a viable medium for other such experimental performances.

Shan Studios is the brainchild of multimedia artist Sheng Jie (gogoj), who returned from studying in France with artists and education to share with young people in China.

That pattern is very familiar. In fact, it’s hard to imagine where we’d be now without international exchange. First, research centers exchanged knowledge and technology – think, for instance, American Miller Puckette visiting Paris’ IRCAM to go on to create what would become Max/MSP, but also investigations spanning Brazil, Japan, Australia, and so on. With more democratized access to technology (read: s*** gets cheaper), that’s gone beyond any centralized locations to knowledge and artistic ideas that cross all six populated continents.

Whereas this was once one-directional – even in the US, aspiring artists often headed to Europe – now I think the compass could spin in all directions.

Anyway, I should be quiet so you can go watch the video:

http://www.livestream.com/gigonline (something interesting happening there already, and I think they’re just warming up)

http://shan-studio.com/?lang=en [English Shan Studio info]

Side note: if anyone is interested in making a Mandarin-native site companion to CDM, do get in touch. We’re not, ahem, sponsored by Intel, but I can see what we can do. Hell, I’d be pleased to have one page, or content in English that does a better job of what’s going on on the other side(s) of the globe.

Music making with your face? It’s just the latest novel way of manipulating your computer with movement, thanks to a revived interest in camera-based interaction spurred by Microsoft’s Kinect and hackers making it work, and other computer vision libraries. One original work: FaceOSC, which uses custom tracking code and a standard computer webcam (no additional hardware required) and free code to send control information for applications like live music performance. Kyle McDonald may have already wowed you with his face-tracking wizardry, but it’s easy to want to know more. Sure, it’s cool, but, um, what is it for? How do you get started? Is the timing quick enough for this to work in music? And what can we expect in the future?

I spoke with Kyle, educator, artist, and coder, about those questions and more. He’s also got some examples of what people are already doing just days after the release of his software – there’s some serious viral quality to open source code.

You’ve done work with both Kinect and now, in FaceOSC, your own camera-tracking software. How is working with Kinect for musical applications, in terms of latency? How have you found latency in your own FaceOSC application?

The only number I’ve heard regarding the Kinect’s latency is from Synthtopia, where they give 100 ms. That seems a little high to me. In my experience, the depth camera seems to have an extra frame of latency compared to the color camera. So I’d put the latency somewhere between 30 ms and 80 ms. In other words: don’t expect it to be a precision tool for live percussion, but for everything else I think there’s just
as much to explore as with any other camera.

FaceOSC feels like adding an extra frame or two of latency on top of what you’re getting from your camera. So you shouldn’t expect to beatbox or do percussion with it, but for controlling parameters in a musical context, you should be set.

How do you imagine this being used? I mean, obviously, in some ways this is (very) experimental, if good, clean fun — is there a
practical application? (EyeWriter, the eye-tracking application that improves accessibility, comes to mind as one possibility.)

I imagine FaceOSC being used to prototype ideas surrounding face-based interaction. I created it because Jason Saragih, the researcher behind FaceTracker, uses an open source non-commercial license for his code. He asks that anyone who wants to use the code email him directly, as a way to keep track of the usage. This is great, but I know that one of the fastest ways to get cool stuff happening is to make new tools and research accessible to a wide audience. So I asked him if it would be ok to make a standalone app for people to prototype their ideas — even if they don’t have access to the code. Everyone already “speaks” OSC so I thought this would be the easiest way to get the technology out there. Eventually, if people need to integrate it into a single application, they can contact Jason directly and use my ofxFaceTracker addon to get started:
https://github.com/kylemcdonald/ofxFaceTracker

And if they need to go the commercial route, there’s FaceAPI
http://www.seeingmachines.com/product/faceapi/

As far as a practical applications, I could imagine it augmenting the way the computer understands us. I’ve been thinking a lot about this
recently. Your computer has a microphone to listen to you, an accelerometer to know when you drop it, a camera to watch you, an
ambient light sensor to know how bright the screen should be. I have to wonder if it makes sense to respond to our pose and facial
expressions.

That said, here are the few experiments I’ve seen so far:

http://jeffwinder.blogspot.com/2011/07/face-gestures-faceosc-and-flash.html

Ingredients above:

FaceOSC
+
RoboFab’s Glyph Math
robofab.org
+
Vanilla
code.typesupply.com
+
Ideal Sans
http://www.typography.com/fonts/font_overview.php?productLineID=100042

Want to talk at all about your approach to developing this — particularly as you’ve been teaching others?

I think everyone learns differently, but for me I learn by playing. So I try to make it easy for other people to play by providing interfaces like FaceOSC (or, with 3d scanning, via my structured light work).

Anything else musicians might want to know about your work?

I haven’t spent enough time recently making music, but I’m always thinking about things in musical terms. My older work has a lot of
musical interfaces and ideas scattered through it, if you dig through http://kylemcdonald.net you might find some inspiration there.

Thanks, Kyle! If you’re in NYC, as Kyle and I are — or, may be, when we’re not traveling to opposite ends of the globe — Kyle has a couple of recommendations. There’s a “no-more-than-monthly” Kinect meetup organized by Sean Kean:
http://www.meetup.com/volumetric/

Also, there’s an amazing “summer school” meetup on July 21. Wish I could be there myself, but I’ll send regards from Berlin. Hope one of the other New Yorkers can report back.
http://eyebeam.org/events/meetup-demo-day

For more on Kinect, we’ve got loads of coverage on Create Digital Motion:
http://createdigitalmotion.com/tag/kinect/

It is, after all, Motion!

Got a creation of your own, or a meetup in your area? Let us know!

For a very different aesthetic take on the United States’ Independence Day celebrations, here’s electronic producer Steve Nalepa joining visualist superstar Johnny de Kam for a collaboration. I find it makes for some nice, chilled-out Monday, July 4 inspiration, wherever you are – no marching-band bombast required. Nalepa has been sharing his Ableton skills with the Dubspot school, online and off, and Johnny de Kam, if you don’t know his work, is one of the leading visualists on the planet, a skilled craftsman of motion and live visual performance, as well as a founder of visual software maker Vidvox.

See, what has America accomplished if not send a man to Jupiter and put human civilians on a permanent space station served by daily commercial Pan Am flights? Wait a second here…

For an alternate take, here’s a second video, also by Johnny for the same track — thanks, ChuckEye!

Now that I’ve opened that can of worms, by posting a 4th of July video — or, erm, box of firecrackers — it’s worth saying that symbolic holidays can be a time for reflection on how we relate to our civil societies.

For all the more troubled parts of America’s history, there are reasons to celebrate, too, as an international community of artists, the sequence of events that would eventually enshrine in US law protections for freedom of expression, religious practice, separation of church and state, and a free press – institutions that around the globe are closely interconnected with our freedom to create digital music and motion. (I can say that, doubly, too, because I’m not actually in America at the moment, and the US is cutting those patriotic fireworks shows to make a gesture toward budget cutting – take that to mean what you like.) But as I talk to artists from Moscow to Sao Paolo, I find common themes in fighting to build free communities of artmakers, that transcend history and borders. These may not involve fighting the British Army, to be sure, but there are legal and civil policy decisions, philosophical ideas, that are part of these historical events, too.

There, in case someone were to decide to go on a flame war about US holidays given the video’s theme, I’ve gotten my word in and can leave you to it. Now, I’m off to celebrate my Fourth of July with the awesome people of the Netherlands. (Neth-er-lands! Neth-er-lands!) And yes, to my home nation, I hope as always for a bright future.

Oh, and if you’re mostly celebrating the Nation of Ableton Live, here’s one of those Nalepa videos I mean… as a tasty, watermelon dessert on this post.

Speaking of sound design, here’s a nice weekend diversion for you: “140 dB” is an experimental short film, combining sonic acrobatics with digitally-synthesized motion spectacle, that meditates on the “interaction between sound and physical bodies. The main idea is to show how sound can change objects form and structure including human mental state.”

Thanks to the flexibility and media-agnosticism of the computer, all the work – art direction, animation, and sound design – come from one person, Tadas Svilainis. It’s a tantalizing teaser; I’d love to see the idea expanded. Tadas, you’re one multi-talented maker.

Of course, as for the content, one man’s torture is apparently another man’s party. I can see a few friends seeing these kinds of sounds as a nice way to unwind. Richard Devine, looking at you.

Found via @TheFilmo and @SoundPlusDesign.

America, enjoy your 4th of July holiday weekend. (I’m in the Netherlands; hopefully I don’t create any fireworks of my own, like accidentally plugging in a non-universal power adapter… again.)