As an addendum to the last story, Ivica Ico Bukvic sends along an example of the [myu] Max/MSP + Unity game engine combination in action. Here’s the surprise: Unity isn’t generating visuals. Instead, Unity simulates ripples created by movement in the space, and builds physical models that are sonified and spatialized by Max/MSP.

Speaking of work involving art museums and the combination of Max and Unity, VJ Anomolee notes in comments his own work with the pairing. Lightbent Synth is an in-progress piece with alternative controllers and sensors that produces sound with a novel visual representation (sound’s very quiet in this preview — more hopefully once it progresses):

Lightbent Synth from VJ Anomolee on Vimeo.

Ivica explains the top work:

This past fall [myu] had seen its first real-world implementation in an exhibit that was a part of the grand opening of the Taubman Museum of Art in Roanoke, VA (http://www.taubmanmuseum.org/). The exhibit utilized [myu] as part of an interactive aural installation titled "elemental." An online tech
demo video of the installation, including written synopsis is available also via Youtube at http://www.youtube.com/watch?v=PA-9BOgc1gk. Below is a brief synopsis of the installation:

"elemental" interactive communal soundscape premiered in November 2008 as part of the Revo:oveR collection commissioned for the grand opening of the Taubman Museum of Art in Roanoke, VA. The Youtube video focuses primarily on the technical aspects of the installation. Using Max/MSP/Jitter, a homebrew IR webcam with fish eye lens and a LED-based IR spotlights, entire 24×36-foot exhibit space is converted into an aural sandbox giving visitors an opportunity to generate and shape the
ensuing soundscape. Positional data of up to 20 visitors is forwarded to Unity3d using [myu] Max-Unity interoperability toolkit developed at DISIS (http://disis.music.vt.edu). Unity is used for physical simulation of ensuing ripples and the resulting data is sent back to Max for spatialization across a 12-channel (4×3) ceiling-mounted speaker array. Driven by communal interaction, virtual ripples refract from each other spawning an algorithmically generated aural fireworks. The exhibit ran non-stop for approximately 5 months until March 2009.

Bonus video below: an early prototype that did include visuals. After days of looking at emulated knobs and faders, it certainly does speak to some of the possibilities for musical interface and expression.

Take a powerful game engine (for animation, 2D and 3D graphics, physics, and on-screen interaction). Add the flexibility of a visual development environment for programming with virtual patch cords, for rich sonic and musical capabilities plus easy interaction with data and input. That’s the idea of combining something like Unity 3D with Max/MSP. In the example from earlier today, the solution simply routed basic data from a Unity-based game to a responsive music engine in Max.

In the case of [myu] – the Max Unity Interoperability Toolkit – that integration goes further still. Developed at the DISIS (Digital Interactive Sound & Intermedia Studio) at Virginia Tech, [myu] allows bi-directional integration of the Unity engine with Max or Pd. The two tools use netsend/netreceive to send data via TCP and glue the two together.

For visualists using Jitter, you can even exchange texture data, which offers some mind-blowing powers for live visuals.

Download at Virginia Tech — bonus, an extension of the aka.wiiremote object so you can use the lovely Wii Fit controller, among various other projects
Discussion on the Unity Community Forums
Discussion on the Cycling ‘74 forum
Virginia Tech DISIS

As an interactive prototyping tool, this should have a lot of potential for lovers of patch-style programming.

Thanks to Dr. Ivica Ico Bukvic, DISIS Director and researcher, for sending in his project. I’ll be curious to see what other people might do with this.

I’m also connecting with our friends at Boing Boing and Boing Boing Offworld, and we’ll have some joint coverage this week and next. They’re crazy enough to be doing a live video stream, so if it’s during the day West Coast time you can see what they’re up to:

It’s got some impressive capabilities going for it, too:

• Mic input
• 3D audio
• DSP effects
• Compressed samples, MOD, and MIDI

And, in good news for indie studios, it’ll cost just US$500 per title to license.

Of course, you can add this to Pure Data (Pd), which found its way to the mobile platform via the (partially open-source) RjDj project. RjDj is a music platform, not a game platform, but Pd has some powerful audio processing capabilities of its own, and I’d count them both in the category of interactive music. The RjDj gang will be having their next “sprint” – a developer intensive to build interactive scenes for the platform – in Berlin, with New York to follow in January. (I’ll be at the New York event, naturally.)

December is the time for the next RjDj sprint: The Reality Jockeys would like to invite you to the Scene composing session on 12-14 December 2008 in Berlin at this nice location:

Here’s the RjDj event, via the Pd list:

moviepilot

Bluecherstrasse 22

10961 Berlin (Kreuzberg)

The action starts on Friday, 12.12. at 12 am and spans the whole weekend. We’ll post further details/timetable etc. briefly before the actual sprint happens.

What is a RjDj sprint?

RjDj sprints bring together musicians and artists interested in working on reactive, algorithmic music ("Scenes") for mobile devices and do other mad things with sound. Currently supported devices are the iPhone and in an yet unreleased version the iPod Touch.

The goal is to create, test and enjoy many new scenes for the RjDj application. We know that many musisicans don’t have iPhones, so we will provide some devices for development during the sprint. Prior sprints happened in Bizau, Barcelona and Vienna. Check out http://rjdj.me/sprints to see how much fun we generally spread!

Attending the sprint is free, but we kindly ask you to register on our website: http://rjdj.me or via info@rjdj.me

Please forward this invitation to anyone you think might be interested.

And here’s the news on FMOD. So, Firelight, I wonder – will other mobile platforms follow? (And mobile platform makers, this clearly demonstrates the need to have lower-level development capabilities, cough, Android.)

Firelight Technologies today released its popular FMOD Music and Sound Effects System for the iPhone platform. With this release, iPhone developers can now use the same high quality FMOD audio engine that is widely used throughout the games industry.

Brett Paterson, CEO says "We’re proud to bring FMOD to this exciting platform. FMOD is a natural fit for the iPhone with its support for sequenced music formats such as mod and midi, compressed samples and small code size. Users can keep it small, or scale right up to take full advantage of the microphone, 3D audio and suite of DSP effects that FMOD natively supports. Being conscious of the budget nature of iPhone development we are also releasing the API at a special iPhone price of $500 per title. This includes the use of FMOD Designer. By combining the power of the iPhone and FMOD we hope to see some really creative programs and hear some great sounding games."

FMOD Ex download / free evaluation