You’re not, in fact, a lonely bedroom musician with no friends. You’ve got friends. You’ve got collaborators. You’ve got audio and visual artists who want to play with your laptop with sync piped between you, and yet wireless synchronization for laptop performance has often been an elusive ideal. Here, in spectacular fashion, you can see it all come together over WiFi with Ableton Live, a projection-mapped coffeemaker, and everything from percolation sounds to the keypad sound effect from The Hitchhiker’s Guide to the Galaxy.

This just in from Barcelona – a new Max for Live plug-in is making laptops and coffee pots jam in harmony. A proper review is in order from CDM. (Well, once we find a few friends… by astounding coincidence, I’ll be surrounded by Ableton users – visual and audio – next week in Rome. I’ll make it happen.) But that’s no reason to deny you the pleasure of the video now. It’s the best part of waking up.

More: http://www.ooeevv.com/

It starts as just another toy to play around with in a few minutes of distraction in your Web browser – as if the Web were short on distraction. But then, something amazing can happen. Like a musical Turing Test, you start to get a feeling for what’s happening on the other side. Someone’s stream of colored dots starts to jam with your stream of colored dots. You get a little rhythm, a little interplay going. And instead of being a barrier, the fact that you’re looking at simple animations and made-up names and playing a pretty little tune with complete strangers starts to feel oddly special. The absence of normal interpersonal cues makes you focus on communicating with someone, completely anonymously, using music alone.

Dinah Moe’s “Plink” is the latest glimpse of what Web browser music might be, and why it might be different than (and a compliment to) other music creation technology. You can now create private rooms to blow off steam with a faraway friend, or find new players online. It’s all powered with the Web Audio API, the browser-native, JavaScript-based tools championed by Mozilla. That means you’ll need a recent Chrome or Firefox (Chrome only at the moment; this is a Chrome Experiment), and mobile browsers won’t be able to keep up. But still, give it a try – I think you may be pleasantly surprised. (Actually, do it right now, as you’ll probably be doing it with other CDM readers. I expect greater things!)

http://labs.dinahmoe.com/plink/

Thanks to Robin Hunicke, who worked with multiplayer design and play at That Game Company’s Journey on PS3 and now on the browser MMO Glitch. I think her friends were more musical than most, because the place came alive after she linked from Facebook.

The browser is becoming a laboratory, a place to quickly try out ideas for music interaction, and for the code and structure that describe music in a language all their own. As in Plink, it can also benefit from being defined by the network and collaboration.

Dinah Moe’s experiments go in other directions, as well. In Tonecraft, inspired by the 3D construction metaphor of Minecraft, three-dimensional blocks become an alternative sequencer.

http://labs.dinahmoe.com/ToneCraft/

There are many reasons not to use Web tools. The Web Audio API still isn’t universal, and native options (like Google’s Native Client) have their own compatibility issues, stability concerns, and – because of security – they don’t do all the things a desktop application will. Desktop music tools are still more numerous, more powerful, and easier to use, so if you’re a reader out there finishing a thesis project, you might look elsewhere. (Actually, you’re probably in trouble, anyway, by any nation’s academic calendar, given it’s the First of May, but I digress.)

But think instead of this as another canvas, and the essential building blocks of interface design, code, and networking as shared across browsers and desktop apps. Somehow, in the light of the Internet, its new connectedness, and its new, more lightweight, more portable code and design options, software is changing. That transformation could happen everywhere.

If you need something to help you meditate on that and wait for a revelation to occur to you, I highly recommend watching a soothing stream of dots and some pleasing music as you jam with your mouse.

Of course, in the end, like a digital mirror, it might inspire you to go out to the park with a couple of glockenspiels and jam the old-fashioned way. But maybe that’s another reason to make software.

(Here’s a video, in case you’re not near a browser that supports the app!)

More, plus reflections on adaptive music:
http://labs.dinahmoe.com/

Mobile computing has already had an enormous impact on music making. A modern phone or tablet (and yes, most often, these come from Apple) is capable of out-performing a lot of dedicated hardware and easily runs the synths and workstations that required state-of-the-art desktops just a decade or so ago.

But what if this same computing power – low-energy, low-cost chips – could be in other music gear, too? They could offer significant advantages. Bare boards, while on their own not quite road-ready, can wind up in music-friendly housings. (Think stompboxes – without stomping on your phone, or buying a big, silly dock.) You’ll never have to sign a contract with a phone company to get one, or stop your latest song sketch to take a call. And they could be significantly cheaper: the Raspberry Pi isn’t quite ready for mass consumption yet, but it has already begun shipping at US$25, meaning the entire computer costs what a phone car charger might.

In fact, much as the original personal computing revolution took computing to masses of new audiences, this could extend music computational power worldwide. We’re not just talking strange DIY software, either – these boards run Linux, meaning a lot of off-the-shelf music software will “just work,” including even some fine commercial entries.

If you’re ready to stop dreaming and start making music, now’s a great time. CCRMA at Stanford in the United States and STEIM in Amsterdam, NL have each been working on development. STEIM even has a workshop scheduled for June, taught by Edgar Berdahl (CCRMA) and Florian Goltz (DE):
Satellite CCRMA: Interactive design with open embedded computers

The instructors offer some great inspiration about what this is all about in their description:

These small computers combine the connectivity of a laptop with the computational power of a high-end smartphone; however they are less expensive than either and fit inside a cigar box. We will dedicate much of the workshop to prototyping new functional artworks, for example: musical instruments, effects processors, interactive installation works, and anything else you can imagine that requires high computational power in a small, inexpensive footprint.

In the broader sense this workshop deals with interaction design: What happens when human behaviours meet those of machines?

But even if you’re not able to get to California or Holland, you can give the software a try. The BeagleBoard is now supported by a custom distro; the Raspberry Pi seems a logical next frontier once it starts shipping. With Pd (Pure Data) included, you can even copy-and-paste instruments and effects like synthesizers, step sequencers and drum machines, and granulators built by a broad community – even without necessarily being a master patcher yourself. (And then, when you do want to modify the way it functions or sounds or gets controller, you can.)
https://ccrma.stanford.edu/~eberdahl/Satellite/

It’s not all beginner-friendly yet, but these hacklabs seem the perfect way to begin to move in that direction, as more people test the solutions, gather data on how different patches perform, and make tweaks and write documentation.

How do you visualize the invisible? How do expose a process with multiple parameters in a way that’s straightforward and musically intuitive? Can messing about with granular sound feel like touching that sound – something untouchable?

Music’s ephemeral, unseeable quality, and the ways we approach sound in computer music in similarly abstract ways, are part of the pleasure of making noise. But working out how to then design around that can be equally satisfying. That’s why it’s wonderful to see work like the upcoming Borderlands for iPad and desktop. It solves a problem familiar to computer users – designing an interface for a granular playback instrument – but does so in a way that’s uncommonly clear. And with free code and research sharing, it could help inspire other projects, too.

Its creator also reminds, us, though, that the impetus for all of this can be the quest for beautiful sound.

Creator Chris Carlson is publishing source code and a presentation for the NIME [New Interfaces for Musical Expression] conference. But this isn’t just an academic problem or a fun design exercise: he also uses this tool in performance, so the design is informed by those needs. (I’m especially attuned to this particular problem, as I was recently mucking about with a Pd patch of mine that did similar things, working out how to perform with it and what the interface should look like. I know I’m not alone, either.)

The basic function of the app: load up a selection of audio clips, and the software distributes them graphically in the interface. Next:

A “grain cloud” may be added to the screen under the current mouse position with the press of a key. This cloud has an internal timing system that triggers individual grain voices in sequence. The user has control over the number of grain voices in a cloud, the overlap of these grains, the duration, the pitch, the window/envelope, and the extent of random motion in the XY plane. By selecting a cloud and moving it over a rectangle, the sound contained in the rectangle will be sampled at the relative position of each grain voice as it is triggered. By moving the cloud in along the dimension of the rectangle that is orthogonal to the time dimension, the amplitude of the resulting grain bursts changes.

You can see how Chris is imagining this conceptually in a sketch he shares on his site:

An extended demo shows in greater detail how this all works:

Chris is a second-year Master’s student at Stanford University’s Center for Computer Research in Music and Acoustics [CCRMA] in California. The iPad version is coming soon, but you can get started with the Linux and Mac versions right away, and even join a SoundCloud group to share what you’re making. You’ll find all the details, and links to source code, on the CCRMA site. (And if someone feels like building this on Windows, you can save Chris the trouble.)

https://ccrma.stanford.edu/~carlsonc/256a/Borderlands/index.html

I also love this Max Mathews quote Chris shares as inspiration:

Max Mathews, in a lecture delivered at Stanford in the fall of 2010
“Any sound that the human ear can hear can be made by a sequence of digits. And that’s a true theorem. Most of the sounds that you make, shall we say randomly are either uninteresting, or horrible, or downright dangerous to your hearing. There’s an awful lot to be learned on how to make sounds that are beautiful.”

Beyond the technology, beyond this design I admire, anything that sends you on the path to making beautiful sound seems to be a worthy exercise. It’s a challenge you can face every day and never grow tired.

http://modulationindex.com/ [Chris' site, with more information]

Thanks to Ingmar Koch (Dr. Walker) for the tip!

There are a few spots in the world that are active hotspots for music tech geekery, and in San Francisco, it’s definitely all about Robotspeak. CNET’s Donald Bell – known to some of us back in the day as terrific IDM producer Chachi Jones – visits the store that once employed him. (This gives me extra-happy memories, because years ago I was fortunate enough to play an A/V set at Robotspeak alongside Donald and Daedalus.)

Donald explains more of why Robotspeak is cool on a CNET blog post:
Robotspeak: An electronic musician’s toy store

All of this gives me an idea. Austin has Switched On. Berlin has Schneidersladen. LA has Big City and Analog Haven. We need a tourist guide to the planet, don’t we?

So, we’d love to hear from you. What are your favorite music tech stores – places so special, you don’t just buy goods there, but actually would tell tourists to make a special trip in your town to visit them, places you’d hang out, places you know you’ll run into other musicians? Where are the seismic epicenters of music geekdom on Earth, whatever continent you may call home?

Fill out our survey and let us know [direct link]

Or answer below:

Quit your job, leave your loved ones, stock up on food, and get ready to destroy your life with videos from YouTube. Network Awesome, a kind of curatorial “TV” network full of free online videos, has been assembling the best documentaries dealing with electronic music online, with hours and hours and hours of things that could basically keep you from, ironically, reading this site for the rest of the week if you like.

This isn’t just some random assortment, either. There are thoughtful assortments and surprise discoveries, including a celebrity-chosen set of some pretty far-out live shows pulled by Jan St.Werner of Mouse on Mars. (How Jan has time to watch YouTube in the midst of an obsessively-detailed production process and punishing touring schedule, I have no idea.)

As Network Awesome’s Jason Forrest tells CDM:

We have guest curated shows by Soul Clap & Mouse On Mars as well as specials on Aphex Twin, Clara Rockmore, Giorgio Moroder, Daft Punk and pretty much every other interesting electronic musician you can think of.

You can find everything on one massive archive page from last week. But here are a few favorites of mine, if the official Network Awesome page made your eyeballs fall out:

A collection of Aphex Twin videos:

An epic collection of IDM classics (remembering that the IDM dream of the 90s is alive on CDM):

Autechre – Gantz Graf
Plaid – Itsu
Aphex Twin – Donkey Rhubarb
LFO – Freak
Pan Sonic – Telakoe
Cylob – Rewind
Bogdan Raczynski – Ahou Bouken
Squarepusher – Come on My Selector
Seefeel – Fracture
Amon Tobin – Esther’s
µ-Ziq – Zombies
Oval – Ah!
Boards of Canada – Dayvan Cowboy

Channel 4′s 2001 docu “The Shape of Things That Hum” covers a significant lineup of electronic instrumental milestones, including the Minimoog, vocoder, Yamaha DX7, Fairlight CMI sampler, Simmons, Roland TB-303 and TB-808, and Akai sampler.

Live appearances by Jean-Jacques Perrey:

An all-live collection selected by Jan from Mouse on Mars:

1. Whitehouse – Live Action 39 Reseda 6-21-84
one of the coolest live electronic music perfomances ever. if kraftwerk would have performed like that they could have skipped the robot fake part and go straight to pop heaven.

2. Masonna @ Helluva Lounge, Kobe (May 2010)
not really pure electronic, rather an effect distortion cabaret in a beautiful stage design. you don’t know if any of the sounds are live and still it is such an outstanding performance.

3. Yannis Xenakis – Mycenae Alpha
it’s always hard to identify which actions exactly trigger which sounds in an electronic music performance. xenakis’ upic system provides the most obvious and honest translation: you can see exactly what you hear. as live as it gets in granular synthesis.

4. Jacques Tati – PLAYTIME
tati’s playtime is one is of my most favourite movies ever. it’s all choreographed around sound. this is a sequence in a hyper modern office building where the concierge is playing the most fantastic sequencer imaginable. replay, switch off the image and just listen to the sounds.

5. Tatu Tyni’s tap dancing cards
tatu tyni the godfather of skweee in one of his magic dance trigger performances.

6.Radioboy live @ Music Plane
perfect performance with a noble purpose. herbert as radioboy replaces the 808 with flaky cornflakes packages. bonus: introduction by mtv veteran ray cokes.

7. Michel Waisvisz – Crackle Synthesizer
michel waisvisz’ crackle synthesizer, probably the most immediate and beautiful synthesizer ever built. unfortunately it’s not michel who plays it in this video.

8. Michael Waisvisz – the Hands
so here is a clip with michel waisvisz, inventor the crackle synth. michel was also the director of steim, amsterdam’s legendary institute for electronic music research. he is using his other famous electronic instrument “the hands” in this clip.

9. Dick Raaijmakers – Intona (1992)
as there is no category for live microphonic music we lump this into live electronic. dutch composer dick raaijmakers doing all the things you ever dreamed of doing to a microphone.

10. Dj Elephant Power – Scratch Tv- Part 2
to round up this little journey into the world of concrete musical abstraction here is a live scratch anthem by dj elephant power.

Delia Derbyshire, Tom Dissevelt and Luc Ferrari feature in a collection of electronic pioneers working primarily with tape and simple electronics (before modulars and computers transformed the techniques of the medium):

Selected by The Sadnesses, Theremin virtuoso Clara Rockmore is seen in a series of performances and an interview. Whether she’s the only person to “master” the instrument is debatable, perhaps – but she is certainly incomparable.

The complete list:

http://networkawesome.com/special/electronic-music-week/

Designing a piece of gear, an interface, is a musical exercise much as is sketching a written score. It’s particularly apparent in the simple but descriptive “x0x” grid of the step sequencer on the Roland TR-808. Graphic designer Rob Ricketts has made some beautiful, arresting posters that simply provide those patterns. Apologies if you’ve seen these already, but several people sent them into me and I saw them making the rounds, and they’re worth spotting. Now, next: a monome poster? Or what visualization might next be clearest?

Program Your 808 (4 poster series, 2011)

A series of informative posters detailing how some of the most notable drum sequences were programmed using the Roland TR-808 Drum Machine. Each sequence has been analyzed and represented as to allow users to re-programme each sequence, key for key.

http://www.robricketts.co.uk/808.html

(I’ve given up on anything with corners; it may upset readers, but I’m only playing spheres from now on. Practicing “Mary Had a Little Lamb.” But I’m curious what new designs, new patterns, and new visualizations we may see next.)

What you’re watching in the video above doesn’t involve cameras or motion sensors. It’s the kind of brain-to-machine, body-to-interaction interface most of us associate with science fiction. And while the technology has made the occasional appearance in unusual, niche commercial applications, it’s poised now to blow wide open for music – open as in free and open source.

Erasing the boundary between contracting a muscle in the bio-physical realm and producing electronic sound in the virtual realm is what Xth Sense is all about. Capturing biological data is all the rage these days, seen primarily in commercial form in products for fitness, but a growing trend in how we might make our computers accessories for our bodies as well as our minds. (Or is that the other way around?) This goes one step further: the biological becomes the interface.

Artist and teacher Marco Donnarumma took first prize with this project in the prestigious Guthman Musical Instrument Competition at Georgia Tech in the US. Born in Italy and based in Edinburgh, Scotland, Marco explains to us how the project works and why he took it up. It should whet your appetite as we await an open release for other musicians and tinkerers to try next month. (By the way, if you’re in the New York City area, Marco will be traveling to the US – a perfect chance to collaborate, meet, or set up a performance or workshop; shout if you’re interested.)

Hypo Chrysos live at Trendelenburg AV Festival, Gijon, Spain, December 2011.

CDM: Tell us a bit about yourself. You’re working across disciplines, so how do you describe what you do?

Marco: People would call me a media and sound artist. I would say what I love is performing, but at the same time, I’m really curious about things. So, most of the time I end up coding my software, developing devices and now even designing wearable tech. Since some years now I work only with free and open source tools and this is naturally reflected in what I do and how I do it. (Or at least I hope so!)

I just got back from Atlanta, US, where the Xth Sense (XS) was awarded the first prize in the Margaret Guthman New Musical Instrument, as what they named the “world’s most innovative new musical instrument.” [See announcement from Georgia Tech.]

It’s an encouraging achievement and I’m still buzzing, specially because the other 20 finalists all presented great ideas. Overall, it has been an inspiring event, and I warmly recommend musicians and inventors to participate next year. My final performance:

Make sure to use a proper soundsystem [when watching the videos]; most of the sound spectrum lives between 20-60Hz.

Music for Flesh II live at Georgia Tech Center for Music Technology, Atlanta, USA, February 2012. Photo courtesy the artist.

You’re clenching your muscles, and something is happening – can you tell us how this XS system works?

Marco: My definition of it goes like “a biophysical framework for musical performance and responsive milieux.” In other words, it is a technology that extends some intrinsic sonic capabilities of the human body through a computer system that senses the physical energy released by muscle tissues.

I started developing it in September 2011 at the SLE, the Sound Lab at the Edinburgh University, and got it ready to go in March 2011. It has evolved a lot in many ways ever since.

The Xth Sense wearable biosensors by Chris Scott.

The XS is composed of custom biophysical sensors and a custom software.

At the onset of a muscle contraction, energy is released in the form of acoustic sound. This is to say, similarly to the chord of a violin, each muscle tissue vibrates at specific frequencies and produces a sound (called Mechanomyographic signal, or MMG). It is not audible to human ear, but it is indeed a soundwave that resonates from the body.

The MMG data is quite different from locative data you can gather with accelerometers and the like; whereas the latter reports the consequence of a movement, the former directly represents the energy impulse that causes that movement. If you add to this a high sampling rate (up to 192.000Hz if your sound card supports it) and very low latency (measured at 2.3ms) you can see why the responsiveness of the XS can be highly expressive.

The XS sensors capture the low-frequency acoustic vibrations produced by a performer’s body and send them to the computer as an audio input. The XS software analyzes the MMG in order to extract the characteristics of the movements, such as dynamics of a single gesture, maximum amplitude of a series of gestures in time, etc.

These are fed to some algorithms that produce the control data (12 discrete and continuous variables for each sensor) to drive the sound processing of the original MMG.

Eventually, the system plays back both the raw muscle sounds (slightly transposed to become better audible, say about 50/60Hz) and the processed muscle sounds.

I like to term this model of performance biophysical music, in contrast with biomusic, which is based on the electrical impulses of muscles and brainwaves.

By differently contracting muscles (which has a different meaning than simply “moving”) one can create and sculpt musical material in real-time. One can design a specific gesture that produces a specific sonic result, what I call a sound-gesture. These can be composed in a score, or improvised, or also improvised on a more or less fixed score.

The XS software has also a sensing sequencing time-line: with a little machine learning (just implemented few days ago) the system understands when you’re still or moving, when you’re being fast or slow, and can use this data to change global parameters, functions or to play with the timing of events. For example, the computer can track your behaviour in time and wait for you to stop whatever you’re doing before switching to a different set of funcions.

The XS sensors are wearable devices, so the computer can be forgotten in a corner of the stage; the performer has complete freedom on stage, and the audience is not exposed to the technology, but rather to the expressivity of the performance. What I like most about the XS is that is a flexible and multi-modal instrument. One can use it to:

• capture and playback acoustic sounds of the body,
• control audio and video software on the computer, or
• capture body sounds and control them through the computer simultaneously.

This opens up an interesting perspective on the applications of the XS to musical performance, dance, theatre and interaction design. The XS can also be used only as a gestural controller, although I never use it exclusively this way. We have thousands of controllers out there.

Besides, I wanted the XS to be accessible, usable, hackable and redistributable. Unfortunately, the commercialized product dealing with biosignals are mostly not cheap and — most importantly — closed to the community. See the Emotiv products (US$299 Neuro Headset, not for developers), or the BioFlex (US$392.73). One could argue that the technology is complex, and that’s why those devices are expensive and closed. This could make sense, but who says we can’t produce new technologies that openly offer similar or new capabilities at a much lower cost?

The formal recognition of the XS as an innovative musical instrument and the growing effort of the community in producing DIY EEG, ECG and Biohacking devices are a clear statement in this sense. I find this movement encouraging and possibly indispensable nowadays, as the information technology industry is increasingly deploying biometric data for adverts and security systems. For the geeky ones there are some examples in a recent paper of mine for the 2012 CHI workshop on Liveness.

For those reasons, the XS hardware design has been implemented in the simplest form I could think of; the parts needed to build an XS sensor cost about £5 altogether and the schematics looks purposely dumb. The sensors can be worn on any parts of the body. I worked with dancers who wore them on the neck and legs, a colleague stuck one to his throat to capture the resonances of his voice, I use them on the arms or to capture the pumping of the blood flow and the heart rate.

The XS software is free, based in Pd, aka Pure Data, and comes with a proper, user-friendly Graphical User Interface (GUI) and its own library, which includes over one hundred objects with help files. It is developed on Linux, and it’s Mac OS X compatible; I’m not developing for Windows, but some people got it working there too. A big thumb up goes to our wonderful Pd Community; if I had not been reading and learning through the Pd mailing list for the past 5 years I would have never been able to code this stuff.

The Xth Sense software Graphical User Interface. Built in Pd.

The public release of the project will be in April. The source code, schematics, tutorials, will be freely available online, and there will be DIY kits for the lazier ones. I’m already collecting orders for the first batch of DIY kits, so if anybody is interested please, get in touch:
http://marcodonnarumma.com/contact

I do hope to see the system hacked and extended, especially because the sensors were initially built with the support of the folks at the Dorkbot ALBA/Edinburgh Hacklab. I’m also grateful to the community around me, friends, musicians, artists devs and researchers for contributing to the success of the project by giving feedback, inspiring and sharing (you know who you are!).

Thanks, Marco! We’ll be watching!

More on the Work

http://marcodonnarumma.com/works/xth-sense/
http://marcodonnarumma.com/works/music-for-flesh-ii/
http://res.marcodonnarumma.com/blog/

And the Edinburgh hack lab:
http://edinburghhacklab.com/

Biological Interfaces for Music

There isn’t space here to recount the various efforts to do this; Marco’s design to me is notable mainly in its simplicity and – hopefully, as we’ll see next month – accessibility to other users. I’ve seen a number of brain interfaces just in the past year, but perhaps someone with more experience on the topic would like to share; that could be a topic for another post.

Entirely unrelated to music, but here’s the oddest demo video I’ve seen of human-computer interfacing, which I happened to see today. (Well, unrelated to music until you come up with something this crazy. Go! I want to see your band playing with interactive animal ears.)

Scientific American’s blog tackles the question this week (bonus 80s sci-fi movie reference):
Brain-Machine Interfaces in Fact and Fiction

I’ve used up my Lawnmower Man reference quota for the month, so tune in in April.

It started as some compelling demos or proof of concept, but it’s plenty real now: the tools for translating movement, gesture, and dance from the body to interactive music march forward. Empowered by Microsoft’s Kinect and an artist-friendly toolchain, even a single, clever developer can do a lot. Sound designer, music producer, and Max/MSP developer Chris Vik of Melbourne has been one of those busy early pioneers, with an incredible tool called Kinectar.

So, the tech is cool and shiny and impressive: what about the actual music? And, even more importantly, what if all the hand waving and moving about could be meaningful? That’s the next step. For his part, Chris is teaming up with a dancer and choreographer to combine his compositional ideas with someone who knows how to move. The Dubstep-y demos (all below) are impressive, true, but the early tests of the work with the choreographer are simply beautiful, and demonstrate that wobble bass isn’t the limit of what this can do. They also turn the arbitrary arm-waggling into a part of the art.

And as for you: the software’s alpha, but you can fire up your copy of software like Ableton Live and grab this software for Mac or Windows and try it yourself. So if you don’t like the results – be the gesture-controlled basslines too wobbly, be they not wobbly enough – you can put your music, and your movement, where your mouth is.

At top, Chris shows off an early test of the dance collab. (There’s more to come.) Below, a tutorial that shows how this works with Ableton. And read on for more from Chris on what the work with the dancer is about, and what the tool can do.

Chris writes:

Since April 2011 I’ve been working solidly with the Microsoft Kinect, developing my software, Kinectar, to enable its use as a MIDI controller for performing music live. I’ve done a number of performances around Australia since I started the project, however, it’s safe to say that, although I would consider myself an electronic musician, I’m certainly no dancer. Enter, Paul…

Dancer, Paul Walker and I have joined forces to bring the Kinect controlled music concept into the world of contemporary dance. Recently we obtained a residency at PACT theatre (centre for emerging artists), where we spent the week developing different ways of implementing my Kinect music control system in a dance context.

My system is developed in Max and uses OpenNI drivers, OSCeleton and Ableton Live.

via Chris’ blog

CDM will check back in with Chris soon, because:

I’ve got some more videos to release over the coming weeks from a range of my different Kinect music performance applications, including controlling/conducting the Melbourne Town Hall Organ and a 100+ speaker Kinect-controlled diffusion performance. I’ll keep you posted when they’re released!

More on the software:

Kinectar Performance Platform is a toolkit developed by music producer Chris Vik to allow the use of Microsoft’s Kinect motion tracking sensor in computer-based music. The software is designed for electronic musicians to expand the way they control their music in a futuristic and extremely expressive way, using only the waving of hands and a small amount of creativity. It can be used to control the simplest of parameters like a filter or LFO, play notes and chords on a sampler or synthesizer, or be programmed to control an entire live-set through nothing more than gesture.

Key Features:

Movement Tracking UI allows manipulation of the Kinect’s human tracking capabilities, displaying all relevant data extracted from the hands location in 3d-space

Instrument Builder lets the user build virtual ‘instruments’ by outputting MIDI notes in three modes:

• Static – Produces a single note value. Useful for drum triggers, turning on/off effects within a DAW or feed that trigger into Kinectar to switch between presets using your gesture
• Solo – Do sweeping solos by selecting from over 40 musical scale presets or click the notes on the UI to make your own
• Chord – Create a progression of up to 8 chords per preset to play live

Global Flags lets you turn on/off Kinectar’s instruments using a MIDI note sent from your DAW, external MIDI controller or Kinectar itself

MIDI Preset Control lets you switch between Kinectar’s presets and instruments using a single MIDI note

Value Editor enables many more MIDI/OSC outputs, for controlling device values

Visual Metronome popout window sits on top of all programs to make it easy to see if you’re in-time when the music gets messy

It’s labeled “rough alpha,” so don’t expect a finished tool here, but you can go download it and give it a try (or learn more about what’s possible):

http://kinectar.org/download

And now, the obligatory (but quite awesome, Chris) Dubstep demo videos:

FRACT is a curious combination of music studio and puzzle game, merging elements of games like Myst with the sorts of synths and pattern editors you’d expect somewhere like Ableton Live. You have to make sounds and melodies to solve puzzles; by the end of the game, say the creators, you’re even producing original music. The work of a small student team out of Montreal, FRACT looks like it has all the makings of an underground indie hit – at least for music nerds.

As the creators describe it:

FRACT is a first person adventure game for Windows & Mac much in the vein of the Myst titles, but with an electro twist. Gameplay boils down to three core activities: Explore, Rebuild, Create. The player is let loose into an abstract world built on sound and structures inspired by electronic music. It’s left to the player to explore the environment to find clues to resurrect and revive the long-forgotten machinery of this musical world, in order to unlock its inner workings. Drawing inspiration from Myst, Rez and Tron, the game is also influenced by graphic design, data visualization, electronic music and analog culture.

The hub of the game is a virtual studio, collecting patterns and timbres. It’s right now in prototype phase, but it already looks visually stunning, an alien, digital world in which more-conventional step-sequencer views seem to emerge from futuristic landscapes. And you can spot Pd in the background (the free and open source patching tool, Pure Data). Update: the developers confirm that they’re working with the embeddable Pd library, libpd. That enables synths with sounds like phase modulation and classic virtual analog sounds, all modulating and generating sounds in-game.

The developers have also published plenty of sound samples so you can experience the musical side of this. Via SoundCloud:

While never released, one place some similar ideas has shown up is a prototype game inspired by Deadmau5. As in this title, two-dimensional editing screens and synth parameters are mapped to a first-person, three-dimensional environment. However, FRACT appears to take this concept much further, expanding upon the world, building more instruments, and actually turning those interactions into gameplay elements. The video of the Deadmau5 project – apparently done in-house for fun and not endorsed by the mouse-headed artist:

That title was the work of a game house called Floaty Hybrid; music blog Synthtopia got the scoop on this in August:
http://www.floathybrid.com
Mau5Bot Sequencer Lets You Make Music In A 3D World [Synthtopia]

We’ll be watching this one develop, certainly; good luck to the team!
http://fractgame.com/