Turning music and sound into three-dimensional worlds often yields something that fields like a trip through space. But this feels like a real trip. Through pulsing, glowing starfields, “Versum”‘s audiovisual movements are brain-bendingly transformative. Artist Tarik Barri has created an integrated world of sound and image that makes the interface and the compositional realms seamless. It seems as though this really is a musical universe, through whose harmonies of the spheres you can fly like. Boldly going, indeed.

Ingredients: Max/MSP/Jitter, Processing, Java, SuperCollider, GLSL [the 3D shading language], and … some serious skill and time, I imagine.

The work has been in development for some years (not surprisingly, given the results). But it surfaced again as we brought up the 3Dconnexion SpaceNavigator hardware as a practical controller for 3D. See Create Digital Motion:
Look at Me, I’m Flying: SpaceNavigator Hardware + Blender

Tarik’s work resurfaced after a presentation in the UK. Reader janklug writes:

I’m just back from the M4_u Max/MSP/Jitter conference in Leicester (was great, btw), where Tarik Barri presented his project ‘Versum’, both as an installation and as a performance.
The user (and in case of the performance, Tarik) navigates through this incredible 3D-space-sequencer-universum with the help of a SpaceNavigator; glowing objects floating in this space produce sound, and as you approach them, they even give this nice doppler effect…
It was totally amazing to be able to float between pulsing rhythm-planet-objects and shiny drone-beams; navigation was easy and natural. Tarik uses a combination of Processing and Max/MSP; don’t know which one the SpaceNavigator is connected to.
Having tried this, I immediately ordered one; I think it also could be a great interface for M4L…

More information:
http://tarikbarri.nl/projects/versum
PDF documentation [2009]

Significantly, it’s really the act of flying that controls the music. That remains interactive, but it’s the movement through the three-dimensional space that determines what you hear. As the artist explains:

This virtual world is seen and heard from the viewpoint of a moving virtual camera with virtual microphones attached. This camera, controlled in realtime by means of a joystick (or any other kind of controller) moves through space, similar to how first person shooter games work. Within this space, I place objects that can be both seen and heard, and like in reality, the closer the camera is to them, the louder you hear them. So when the camera moves past several visual objects, you simultaneously hear several sounds fading in and out. Consequently, the way the camera travels past them actually causes melodies and compositional structures to be seen and heard.

The visual position of each object coincides with the panning of its sound: objects to the right of the camera will also be heard on the right, and those behind the camera will be heard from behind in case a surround speaker setup is used. This principle also applies to the Z-axis, meaning that sounds can be heard coming from above and below if the speaker setup supports it.

That’s the essential question, to me, when looking at 3D environments for music. What about the dimensionality will interact with the music? Is it something spatial, or will there be other sorts of interactions? (New Zealander-turned-Berliner Julian Oliver worked extensively with game engines, for instance. One solution for him was modifying the “gun” in those games to be an implement for doing things in the space, turning swords into plowshares after a fact by making the gun produce music rather than kill virtual entities.)

So, now you’ve seen some of the technical demonstration. But Tarik uses his work as an environment in which to make audiovisual performances. Here’s what some actual live playing looks like, in a beautiful, meditative piece called “Eleven”:

In fact, the biggest challenge to me of a piece this awesome is that you want an immersive environment, not just the small, rectangular screens that are often all festivals and venues can afford.

Holodeck, anyone?

More:

Drasko Vucevic, Toronto- and Santa Monica (California)-based sound designer and artist/composer, is apparently not only interested in playing alone. His upcoming interactive installation at Toronto’s Royal Music Conservatory will have an audience jamming along live via Twitter. And the artistry is crowd-sourced, too – with a range of artists already onboard, Drasko is calling on musical and visual artists (read: you) to be involved with sounds and visuals.

Drasko has sent along extensive notes, so I’m going to let him speak for himself:

I am working through both Drastic Music and Eksperimental (my companies) to create an interactive installation experience for this year’s TEDx Toronto conference taking place at the Royal Music Conservatory.

I am also doing an interactive music performance – an audio visual performance with a complete 3d/visual journey, which allows the audience (physical and digital) to collaborate with me by triggering audio and video elements in real time through their tweets.
So far, I have a few great artists contributing their time to create some beautiful visuals for this first of its kind interactive real-time jam.

Installation [Call for Audio]:
The installation concept is based around redefining collaboration. We are doing this by using real-time data (motion, color, sound, light) and tweets relating to TEDx to trigger, control, manipulate and compose audio elements on our back-end audio library (ableton). This is all happening through Processing, Max 5, Arduino, Ableton.

The massive back-end sound library contains loops, melodies, soundscapes, fx, you name it – produced by some great artists. The beauty of this is also that artists which have never before collaborated, will be remixed and mashed up solely by the audience, in relation to key words, discussions, movement, etc.

So far, the artists contributing audio content are:

Yoko K
Trifonic
Richard Devine
Drumcell
Audioandroid
David Della Santa
Darrin Wiener
Audionerve
Box of Toys
Lodewijk Vos
Matt Davis
Adrian Ellis
Andrew Lauzon
Drasko V

Performance [Visual Call]:

As I mentioned, my performance will be very interactive, musically and visually. Both audio and video elements will be triggered based on tweets in real time. I have some great visual artists contributing their time, such as Murat Pak, Yongsub, Charlie Vicetto, etc, but am looking for more, to create elements for the performance. They would of course get the great exposure of TEDx brand, be mentioned everywhere online, and will be in the final video spread throughout blogs once we launch the digital music version.

So how does the call for works … work?

The TEDxToronto conference is on September 23. Here’s how all the pieces come together for that and how to submit:

Musical system uses a massive library of sound structures – loops, melodies, fx, soundscapes and more – triggered and manipulated solely by tweets (relating to TEDxToronto) and motion, color, sound and light within the RCM venue.

Over 12 compositions will be recorded on the day of the conference. The arrangement, structure and sounds used will depend only on the key words used in tweets, the types of emotional replies, and physical interactivity within the venue.

Beauty of having some great artists be remixed and mashed up by the general public, in a very subliminal way. (again – Through their emotional replies, and physical movement)

This posting is a call to artists who may wish to apply to contribute their audio content and be a part of the soundtrack we will create that day. They should contact drasko (at) drasticmusic (dot) com with a link to their portfolio and we will take it from there.

Our installation progress may be followed on my personal site (drasko-v.com) or via Drastic Music or Eksperimental blogs.

We plan to expand the installation idea and bring it online for an ever-changing musical universe manipulated by truly organic methods (digitally and physically).

Interested to see how this will all come together. We’ll be watching. If you submit, and if you attend, let us know how it goes.

More on Drasko:
http://drasko-v.com/

Sometimes, to quote Depeche Mode, words are very unnecessary. Instead, lose yourself for a few minutes in the vibrating mathemagical lands of numbercult, audiovisual immersions in which sound and geometry fuse in a strange, abstract dance.

Their most recent creation, found via Richard Devine’s prolific Facebook wall and posted earlier this summer, explores an actual audiovisual sequencer. See it at top:

Connected is a graphical/musical sequencer system. a three way flow of information, between graphics, sound and external triggers shape the composition. Recorded in real-time.

But actual functioning interfaces aside, I’ll leave you with some other video clips that traverse similar territory, these syncing up separate visual and audio systems.

These folks make music, too – have a listen to their album, at bottom. And that shifts to body candy, as in, for your butt, with danceable grooves.

All three of the above videos combine vvvv – the Windows-only, graphical patching environment for powerful 3D effects – with Ableton Live for sound.

But lest you think it’s all abstraction, have a listen to their excellent dance release on Bandcamp. Downloading:

http://www.numbercult.com/

By the way, ever wondered what visual software people are using? So did we. Don’t miss this look on our sister site, Create Digital Motion, including where vvvv fits on the spectrum:
What Visual Software Readers Use: Some Clear Favorites, Plenty of Diversity, in Census Results

Presenting artists from around Earth to viewers around Earth, a center in Beijing has found a way to do live performance for a sleepless world without waking the neighbors.

Let me start out by saying this: if you read CDM from China, say hello. We’re in the wrong language, we have no translation, and I seriously doubt our Texas data center is delivering this site with any speed (until we upgrade to an international CDN), but the only reason I still run CDM is in order to reach people, and to hear from a wider world that knows things I don’t, and imagines things I can’t. And if you’re not in China, we still get very nice, high-quality video streaming. Think about that for a second: we’re on a planet that has a circumference between poles of about 24,860 miles (40,000 km), and we can share video and recording as if we’re in the same room. That’s pretty ridiculous; almost more impressive than recording itself. (I had similar thoughts a few years ago, somewhere in the jetlag going from New York to its nearly-furthest point on the globe, Perth, Australia.)

Shan Studios is a platform for artist conversations, residency, audiovisual performance, and learning. If you’re in Beijing, China, this center is forging connections between European audiovisual practice and China — and it’s a place where you can go to learn tools like Ableton Live, SuperCollider, and Max/MSP/Jitter. But if you’re anywhere else in the world, tonight/today you can watch a performance of audiovisuals. (That’s 11:59p Beijing time, 4:59p London time, 11:59a New York time).

The best part of this: by broadcasting to the Web but being silent in person, the performance won’t disturb the neighbors.

Using an array of webcams, DIY synthesizers, medical equipment, projectors, busted radios, and many unconventional instruments, the performers will create a completely immersive audiovisual experience in the Shanstudios sound laboratory. But the actual performance space will be silent – as to not wake the neighbors and simultaneously experiment with the best distortion box ever created (the Internet!) – all sounds will be processed digitally and virtually. The event is entirely exploratory and will hopefully lead to greater investigation of the Internet as a viable medium for other such experimental performances.

Shan Studios is the brainchild of multimedia artist Sheng Jie (gogoj), who returned from studying in France with artists and education to share with young people in China.

That pattern is very familiar. In fact, it’s hard to imagine where we’d be now without international exchange. First, research centers exchanged knowledge and technology – think, for instance, American Miller Puckette visiting Paris’ IRCAM to go on to create what would become Max/MSP, but also investigations spanning Brazil, Japan, Australia, and so on. With more democratized access to technology (read: s*** gets cheaper), that’s gone beyond any centralized locations to knowledge and artistic ideas that cross all six populated continents.

Whereas this was once one-directional – even in the US, aspiring artists often headed to Europe – now I think the compass could spin in all directions.

Anyway, I should be quiet so you can go watch the video:

http://www.livestream.com/gigonline (something interesting happening there already, and I think they’re just warming up)

http://shan-studio.com/?lang=en [English Shan Studio info]

Side note: if anyone is interested in making a Mandarin-native site companion to CDM, do get in touch. We’re not, ahem, sponsored by Intel, but I can see what we can do. Hell, I’d be pleased to have one page, or content in English that does a better job of what’s going on on the other side(s) of the globe.

Matt Davis [namethemachine] is seen here with Microsoft’s Kinect computer vision / 3D camera controller, plus – stealing the show – lasers. The lasers in question are a rig by Henry Strange, which allows computer control of laser direction using the DMX protocol. (DMX is a protocol similar to MIDI – though actually a bit simpler, if you can believe that – generally associated with lighting and show control.)

I could say more, but I’ll let you watch the video and ponder. The ingredients:
OpenNI, the “natural interface” not-for-profit standards body and organization that allows drivers across multiple hardware (Kinect being the best-known)
Ableton Live (sound)
Max/MSP (I believe here just translating OpenNI control to MIDI and perhaps DMX, as well)

The result: audiovisual control, and The Future. (Now, the only problem is, I’m not sure I’d want to watch an entire lineup of people doing these kinds of gestures while performing, but I could certainly see this alongside other alternative control schemes, from breath to good-old-fashioned tangible controllers.)

Thanks, Laura Escude, for the tip. (Laura has her own interface for futuristic electronic performance – she uses a violin!)

Computing technology is an inherently disruptive thing, wonderfully so. It solves problems you didn’t know you had. It creates problems, then creates new problems in even trying to understand those problems. Simply using a computer is a kind of design statement.

You’ve seen questions about what happens with computer performance and audience interaction. But, in AMALGAM, design student Jacob Lysgaard asks those questions, and proposes solutions, in a new way: with a giant talking robot face. (See above.)

Laptop and electronic performance produces a number of symptoms that can be problematic. As the video roboface above puts it, you might find, for instance,

“A lonely man hiding behind a big table onstage.”

Actually, I sometimes do feel lonely and like to hide. Then again, I don’t necessarily have to invite other people for that. So, in that spirit, here’s the latest in a long line of design ideas for re-imagining computer performance. Maybe at this point, this isn’t solving a problem: maybe it’s design, reorganizing the experience of musical activity around a technology that could really be anything.

The solution Lysgaard devises is really rather spectacular, conceptually. Whereas computer performance “solutions” generally involve novel performance interfaces, here, the design delineates the fundamental problem: “real” space (the live performance that’s actually happening) and “virtual” space (the performance that happens only through the machinery of the digital performance, via playback, interactive or otherwise).

In some sense, this is what all responsive visualizations of music do: they create visual evidence of what you’re hearing, producing the artefact of the activity that the virtual sound lacks.

But, then, you’re not always concentrating on what an acoustic musician is doing with their physical instrument, either; you’re often lost in the music. And that is to say, you might just trip out watching all these bobbing cubes and virtual selves. And I think that’s okay.

Here’s just the visualization of how the scheme works, in case you zoned out watching Mr. Roboto in the earlier video:

Various visualizations are presented on the designer’s Behance portfolio. Suffice to say, while the representations here are abstract, other styles are possible – even M.C. Escher variations:
AMALGAM

Read the full explanation of the project, as well as its inspirations, on Lysgaard’s blog:
AMALGAM – my bachelor exam project

The work was a degree project in visual communications for the design department of the Bergen National Academy of the Arts in Norway.

I love the logo for the project:

I’m saving my favorite bit for last: a kind of visualization – or at least visual reduction – of representations of music in Ableton Live.

Terrific work; Jacob. I’ll be interested to see how this evolves in performance.

Artists Leisure-B and Usselino collaborated on this Autechre-inspired audiovisual short, filled with pulsing geometric primitives. The work is a kind of A/V composition, the music arranged with abrupt, video-style edits. For fans of Native Instruments’ thick-sounding Massive synth, you also get some free preset downloads in the deal.

With Native Instruments’ software providing the sound palette, Ableton Live became a context for editing the music as you would video:

Leisure: “I tried to approach the composition as “sound design” as possible. Since the video was animated on the BPM (117) and rhythmical accents of “Vose On”, all I really had to do drum wise was find out which accents Usselino had used for his main video events. After creating the drums on those accents, most of the composition was just tweeking knobs and experimenting with note placement.

Since Ableton Live has excellent video support, I could just run the video in the loop region I was working in, and experiment with Native Instrument’s Massive’s great synthesizing capabilities. I usually start of with an init patch, and work my way towards the sound which I feel is fitting for the events in the video. The only exception to this rule however is the second bassline, which originates from Durk Kooistra’s WODAN bass, available for download in his 10 Free Massive Patches”

Watch the video, then check out the presets and more information:
10 Free Massive Patches
Cinema 4D meets Ableton Live: fret_1

More on the visual side, free Cinema4D download, on Create Digital Motion

What could audiovisual art made for tablets look like? Some dismiss the results as “toys.” The creators of Thicket embrace that very role. Thicket, the product of electronic A/V artists Morgan Packard and Joshue Ott, is intended as a kind of immersive distraction, filling the screen of an Apple handheld or iPad with clouds of abstract color and ambient sound. Free of controls or widgets, playing with Thicket is a bit like sticking your hand into a mist, more gently responsive than overtly interactive.

It’s also lovely, and – for the moment – free. As an experiment, Josh and Morgan are offering up Thicket free of charge on the store, meaning those of you with appropriate devices can try it out as you read.

http://apps.intervalstudios.com/thicket/

A lot of what you read about iOS development and the new “app” markets in general tends to blur into generalities. You’d also be forgiven for assuming the “app” is a gold rush even for avant-garde digital artists. Instead, the picture from Morgan and Josh is subtler. They concede their work may have niche appeal, and view the iPad and iPhone not just as a hot, new market, but as part of a longer narrative of audiovisual expression.

Along the way, they’ve got some tips for learning to develop software, reflections on how interaction can work and why even a baby can play with their creation, and some insight into how their project worked as musical, visual collaboration and not just software creation.

If you had to classify Thicket, how would you describe it?

Morgan: Thicket is a toy. We can get a little more descriptive and call it an audiovisual toy.

Josh: I’d call it portable moving artwork…

What was the original impetus for the idea behind Thicket? What sparked this particular, somewhat abstract notion of how you might interact with sound and image?

Josh: I had been talking about making something for iOS for over a year, ever since Apple released the first SDK, but had been unable to find the time. The impending release of the iPad and Morgan’s interest provided me with the motivation I needed to actually start something. We decided to choose from favorite moments of our past collaborative efforts and find something we could make pretty quickly. Thicket came from one such moment in the “Unsimulatable” DVD/performance that accompanied Morgan’s first album, Airships Fill the Sky. We had referred to this moment (fondly) as the “ball of string”. That particular moment came from multiple experiments and jam sessions with my superDraw program and Morgan’s Ripple program. That was the seed, and once we had that seed, using the unique features of the iOS devices to alter/enhance/affect what was happening seemed fairly natural.

Morgan: My approach to sound for Thicket comes out of a desire to piggyback on the natural ways people want to touch Josh’s graphics. I didn’t want to have any sort of fiddly controls — knobs, sliders, buttons, menus. I simply wanted to sonically respond to the natural, casual interactions which Josh’s visuals encourage. Typically, music apps for mobile devices have a number of controls — buttons, sliders, knobs. And they use simple XY position as a primary source of musical or synthesis parameters. Neither of those input sources — control widgets, or xy position — is compatible with the natural way one wants to interact with Josh’s visuals, so I had to use some different sources of music input — finger speed and multipe touches. I’m really interested in finding other gestural means of input which play nicely with the control of visuals.

While it began on handhelds, Thicket really seems to benefit from being on the tablet form factor on the iPad. What does that device mean to you as a canvas for work?

Josh: It’s a pretty exciting device to work with. As we’ve improved it over the year, we’ve found that we are more and more focused on the experience on the tablet rather than on the phone.

Morgan: I don’t want to discourage anyone from using Thicket on a phone. We’ve made sure the phone experience is solid since the first version. But I’ll admit that Thicket really shines on the iPad. I was a reluctant iPad buyer. I only bought one because Josh forced me to! But now that I have one, I’ve developed quite a passion for it. It’s a seductively humanizing and ergonomic little machine. It’s very book-like, which is a good sign. There’s a reason books are the size and shape they are. Their form has been under development for hundreds of years. The iPad allows us to reach people when they’re in that curl-up-with-a-book mode: relaxed, comfortable, attentive.

Back when multimedia CD-ROMs were what some artists saw as the future (which wound up being both wrong and right, depending on how you look at it), I remember Morton Subotnik talking about his vision for this stuff. He described these devices as a kind of “chamber” experience – the personal quality of them being a strength. What does it mean to the two of you to be able to distribute work to someone on a mobile computer, and not only in a performance or gallery?

Morgan: The mobile art experience has been around as long as radio, so I’m not sure I consider Thicket revolutionary in that sense. I’ve been making music recordings for years which most people experience in solo settings. However, the community aspect of art-making is very important to me. I love performing, and I love what happens when you get a crowd of people together. If we want to test out the “chamber experience” analogy, we can compare the experience of using Thicket to the experience of listening to live (western classical) chamber music. When you listen to the chamber music, you’re out in some sort of venue, you’re setting next to people, probably strangers, you’re in a public place, physically participating in something collective and cultural. Using an app on your own, no matter how artistic that app is, doesn’t have any of that meatspace cultural thing built in to it. The Smule folks have done a great job of allowing you to interact over the Internet with other people using their apps. But I still place a lot of value on getting people physically together in the same space, experiencing the same sounds, the same sights, the same vibe, and actually being interact with one another in the physical world. I’d love to find ways to create software which encourages more connection between people. But on the other hand, sometimes it’s good to recognize what things software is good for, and when it’s time to be a little more old-fashioned.

Josh: I think the most exciting aspect of these devices for me (besides the general human augmentation that all of them increasingly offer us) is their potential to augment the way we communicate in an artistic way. I’ve been exploring this in my performance based work for a while now (performance that involves the audience in new ways, like the multi-user art show earlier this year), but I’m really excited to approach it even more organically, more like little ad-hoc multi-player games that happen to create something aesthetically compelling.

We’ve also seen Thicket adapted to a gallery installation (at TEDxBrooklyn) and theoretically it could be used in live performance. Is this something you’ve been able to perform with, as well? Do you think it’s possible to build a piece that could be both your performance tool and something an end user can pick up?

Josh: I’ve really enjoyed experimenting with how Thicket can be framed (sometimes literally) as a piece of art on a wall. The iPad itself is just big enough to be framed and hung, and people seem to really enjoy interacting with it this way. Another fun use I’ve discovered for it was projecting it on the ceiling for my 4 year old daughter as a moving night light. Thicket was never really designed to be a performance tool, although it’s arguably started to move in that direction. Building a performance tool that is still fun and accessible for everyone, while giving more focused users the flexibility they need, is one of my goals for the next year. I’m not sure whether that will be Thicket or something new, but will definitely be approaching it as something I will use myself for my own performances.

Morgan: I’m wary of allowing Thicket to become too much of a tool. I’d rather make it a better toy. Performers need tools, not toys. For now, I’ll be sticking with Ripple, my performance/composition software for my own music. And trying to make Thicket an easier, richer, and more delightful world to explore.

Game designers have long thought about challenge, reward, and even failure states. And as a result, some of their creations are truly long-form — it’s not uncommon for someone to complain about a 20- or 30-hour game being too short. One of the criticisms of art on devices like the iPhone has been that thing tend to become quickly-digestible toys – perhaps calibrated to the kind of interaction design artists have done traditionally. How do you respond to that criticism? How did you tackle questions of states of interaction in Thicket?

Josh: Being a fairly avid gamer, and having just read a profile in the New Yorker about Shigeru Miyamoto (Nintendo’s Mario creator) I’m really excited about the intersection between art and games. I know this topic is a bit tired, but I’m approaching it from the art side rather than the “gamer” side, and that’s interesting to me. While Thicket isn’t a 20-30 hour experience (though it could be argued that its open ended-ness makes the experience ultimately unquantifiable), making Thicket a deeper experience has been one of our major goals this year. (I think we’re definitely on the right track with our 2.0 update that hit in November.) Games that allow players to be creative in how they play, from open world games like Rockstar’s Grand Theft Auto series, to games where traditional goals can be accomplished in a variety of ways (Bioshock, etc.), to games where creativity is part of the game itself (World of Goo, Noby Noby Boy, Captain Forever), have offered players deeper experiences and replayability. I’m really excited to see these two worlds collide, and possibly try to experiment with that collision in my own work.

Morgan: I am decidedly _not_ an avid gamer, though Josh diligently continues to try and convert me. But I’m intrigued by the idea of guiding a user experience, creating some sort of progression a user (or player) travels through. On the other hand, I’m not sure that’s necessary. There’s room in the world for idle playthings with no specified goal. Take those little magnetic Buckyballs, for example. They’re simply fun and satisfying to play with. They’re wildly popular despite the fact that they’re completely open-ended, with no challenge/reward/failure built in to them. A difference between that toy and Thicket is the range of what you’re able to create with them. We’ve been getting regular suggestions from users asking for the ability to add their own sounds and pictures. I have no idea how we would do this. But it may be that what people are really asking for is greater control, a greater range of end results that they’re able to produce. I’d like to come up with ways to do that without losing the casual, playful spirit that Thicket has now — and without adding any sliders!

Working on Thicket meant collaborating not only aesthetically, but technically. How did the two of you work together, especially as you were in different cities?

Josh: There was (and is) a weekly video chat where we discuss our status, what we’re doing… and we’re of course using an SVN [Subversion version control system] on the technical side to share code. Surprisingly, collaborating technically across a long distance has been quite painless.

Morgan: It’s pretty easy for sound and visuals code not to step on each others’ toes. We have a shared set of data we can examine for things like touch position and speed, and mode changes. Other than that, we’re pretty much in our own sandboxes code-wise. We did do a bit of coordination to syncronize mode changes to tempo. As Josh mentioned, a shared code repository — the Subversion software in our case — is absolutely indispensable.

I know a bit about your coding background – Josh having worked in Processing/Java and Flash, Morgan in SuperCollider. What was the process of learning iOS development like? How was the learning curve? Any lessons learned you can share?
Josh: It definitely has been harder than I thought it would be. Coming from [Flash/Flex] ActionScript and Java, I feel like I had to learn three new languages at once: Objective-C, Apple’s Cocoa Touch framework, and OpenGL. Learning about pointers and handling memory was a big challenge. I would highly recommend the (FREE) Stanford iPhone Application Programming course available on iTunes: It was a huge help for getting started and I still am kind of amazed that you can audit a very well put-together class like this for free.

Morgan: I’ve been comparing the move from higher-level languages to the lower-level iOS environment to stepping out of a go-cart and in to a helicopter. Not only was I learning new languages (C, Objective-C, C++), I was dealing with raw audio data for the first time. Honestly, I’m still in the steep part of the learning curve, though things are beginning to feel a little easier. It’s pretty difficult to simply figure out how to send raw audio data to the speaker, which is necessary if you want to do any dsp. This example project helped me quite a bit with that:
http://sites.google.com/site/iphonecoreaudiodevelopment/remoteio-playback

Morgan, you ultimately decided to code sound from scratch. This was your first project to do that, correct? What led you to that solution, and how did you approach it?

Morgan: Without rolling a significant amount of my own code, I would have been limited to very basic techniques — file playback and looping, pretty much. Most of the audio in Thicket is based just on manipulating loop lengths and playback start points. But even though the techniques I use are quite simple, I wasn’t able to find a higher-level sound library which could do exactly what I needed. Also, I didn’t want to get locked in to something which wouldn’t be flexible enough for me in the future. Unless there’s a really powerful tool available which I’m confident I can grow in to for a very long time, I’m more working at low level and building more things from scratch.

Josh has talked with me a bit about how he feels the iTunes App Store and the Apple ecosystem on iOS, for all the criticism it has garnered, has really made Thicket possible. What was it that drew you to iOS? What is it that Apple is able to provide for you, in terms of hardware, software, development experience, or user experience?

Josh: Apple really does have excellent documentation and support for their SDK. That’s one of a number of reasons why we chose to develop for iOS. Another reason is actually the small number of devices that iOS runs on: from a development perspective, it makes things much easier to test… Then there’s the solidity of the Apple’s hardware and multitouch implementation in general.

Morgan: I like the way the Apple devices and OSs look and feel. It’s a dumb emotional judgement, but sometimes you’ve got to follow that. I also like the fact that people using Apple devices are really into their apps. I wanted to be part of that party.

Do you think there’s potential on new platforms – ranging from Web app stores to Android or Windows and Linux tablets – to do the same thing? Or is there something Apple was able to do for you that was unique? Is there anything those platforms can learn from Apple?

Morgan: We chose Apple in large part simply because that platform seemed to be where the action was. I think we probably could have built Thicket for another tablet platform and the product would have been fine. But I also think fewer people would have downloaded, and we wouldn’t have felt encouraged in the way that we were. We might not have made the version 2 update, which has been much more successful than version 1. At this point, for us to transfer time and energy away from iOS development toward another platform, we’d need to see someone else having significant success selling artsy apps. It’s too big an investment to try another platform simply as an experiment. When we see another app store really taking off, on a device which we can feel some love for, we may reconsider.

How has this business of selling an “app” worked out for you? And why try going free?
Josh: I wish I could say Thicket’s paying the rent, but it’s really not. I think the (emerging) art market on the App store is relatively new, and harder to find (as Apple doesn’t give us a real category in the App Store). The fact that they have a Generative Art & Sound category in their 2010 “best of” list [iTunes link] (which Thicket happens to be in!) suggests this may change in the future. We never really expected it to do well financially and are pretty thrilled when people seem to like it. Making it free is an experiment of sorts. It allows a lot of people who wouldn’t ordinarily try Thicket to do so.

Morgan: I had very low expectations for how many people would be interested in Thicket, and they’ve been clearly exceeded. I’m excited that (a modest number of) people are actually spending money on our art! Sales are much better for Thicket than they’ve ever been for any recorded music I’ve released. Originally we thought Thicket would be a sort of experimental, niche product, and we’d price it a bit high. It seems to have a bit wider appeal than we expected though, and a lower price allows us to reach more people, while still making some money. Going free for the holidays is a way of testing just how wide the appeal is.

Have you seen other interactive creations – on iOS or elsewhere – that you’ve found inspiring?
Josh: I’ve mentioned a couple of games that I play; here’s more stuff.

For PC: Windosill (really everything by Patrick Smith, AKA Vectorpark, is just amazingly beautiful.)

Superbrothers have also inspired me quite a bit…

Machinarium is also really good.

Interactive artwork I’ve been inspired by this year on iOS: Uzu, Sonic Wire Sculptor.

Games (also iOS): The Incident, Acrobots, Enviro-Bear (also on Android)

Morgan: I think BubbleHarp is great. I love the butterfly effect aspect of it — tiny differences in the way you set it up result in huge differences in what you see. Bloom is classy and beautiful, and was a great example of how to create a sequencer without forcing a bunch of fiddly controls on a user. Using Uzu makes me feel like a god.

What’s next for your collaboration (or personal projects)?
Josh: Both of us want to continue to make Thicket better. It’s becoming an outlet for artistic experimentation for ourselves, and we are seriously considering how to involve others as well. Additionally I have lots of ideas and I’m hoping to also release a bunch more software next year: some artistic, some performance based, some more game-like.

Morgan: The next thing we want to try with Thicket is in-app purchase of new modes. It will be really satisfying to have this platform we can be continually adding to. I’m a bit anxious to start devoting a bit more energy toward live performance. I don’t want to get so stuck inside the iPad that I forget that I was once a performing, travelling musician.

Thanks to Morgan and Josh for the insight. See also a superb interview, with code, at the always-excellent Disquiet, which focuses on the sonic and compositional aspects of the app…

BEING DECIMAL: THE ANTICIPATORY PLEASURES OF THE THICKET APP [Disquiet / Marc Weidenbaum]

For more interactive iOS goodness, also free, try Josh’s Snowdrift – not that certain people associated with the East Coast of the United States, or London, among other places, really need an app for that at the moment.
Snowdrift @ iTunes

Commencing in about 12 hours, Visionsonic 2009 showcases Audiovisual artists from around the world. The Thursday show is “for Young Audiences”, but I’ll definitely be watching The Odyssey of Rick the Cube.

L’Odyssée de Rick le cube (extraits du spectacle) from Jesse Lucas on Vimeo.

The festival program has 20 artists performing over Friday and Saturday, which you can view on the Visionsonic TV page, or load the WMV stream in your media player of choice (VLC works beautifully).

There are plenty of reasons to go to live performances, to be in the same physical location as the artists, and to experience them without any technology or production getting in the way. Of course, if the performance is happening on the other side of the planet, then streaming may be the only feasible way to get there. CDM has dabbled in internationally-streamed performance before with NetLag, which attempted to put together a “festival” featuring artists performing from various locations. This was great fun, but beset with various problems, as technical and especially bandwidth limitations are keeping us from the “everything streaming live in HD” world the mainstream media seems to think we inhabit. These limitations are falling away quickly though, so it just remains for us to figure out the most compelling ways to capture live performance for streaming.

For watching the stream? Well I’ve got a big stereo, a projector, a copy of VLC, and a weekend’s entertainment all planned out for me.

Following on from our previous Game On Set. Sam will be kitted out with Serato’s Video-SL (review on CDMo), and I’ll be bringing a brace of live camera feeds with the Vixid VJX16-4 video mixer (minisite | on CDMo).

Last time it went down something like this:

Sampology at Game On – AV Turntablist Set (Part 1) and (Part 2) from Herovision on Vimeo.

Video-SL is fantastic fun, and as a visualist it’s somewhat humbling to discover what a turntable worrier can do when their spinning plastic discs suddenly have power over vision as well as sound. Tune in tomorrow to see.

To sweeten the deal, we’ll be preceeded on stage by Yahtzee (of Zero Punctuation) and Matt and Yug (of Australian Gamer), who will have a screening of their show Game Damage, and then talk about games rather a lot.

Using web production studio Mogulus, the stream will be viewable on the CDMedia channel, and there’s a countdown and embedded player at Herovision.