You might imagine sound in space, or dream up gestures that traverse unexplored sonic territory. But actually building it is another matter. Kinect – following a long line of computer vision applications and spatial sensors – lets movement and gestures produce sound. The challenge of such instruments has long been that learning to play them is tough without tactile feedback. Thereminists learn their instrument through a the extremely-precise sensing of their instrument and sonic feedback.

In AHNE (Audio-Haptic Navigation Environment), sonic feedback is essential, but so, too, is feel. Haptic vibration lets you know as you approach sounds — essential, as they’re invisible. The work of Finland-based DJ/VJ Matti Niinimäki, aka MÅNSTERI (“Mons-te-ri”), the project is part of research undertaken at SOPI Research Group at Media Lab Helsinki. Like some sort of sound sorcerer, the user is entirely dependent on movement, feel, and sound as they move unseen sound sources through space. (More technical details below.)

It’s labeled, as always, “proof of concept.” The creator promises more videos to come; we’ll be watching as this evolves, as it looks terribly promising.

Below, “Tension” is a fair bit simpler, in which users walk through a space and control synth parameters. (“You are the knob,” one might say, though I don’t suggest shouting that at someone you don’t know. They could take it the wrong way.)

More descriptions:

AHNE

This is a demonstration video of AHNE – Audio-Haptic Navigation Environment.

It is an audio-haptic user interface that allows the user to locate and manipulate sound objects in 3d space with the help of audio-haptic feedback.

The user is tracked with a Kinect sensor using the OpenNI framework and OSCeleton (github.com/​Sensebloom/​OSCeleton).

The user wears a glove that is embedded with sensors and a small vibration motor for the haptic feedback.

This is just the first proof-of-concept demo. More videos coming soon.

HEI Project 2011
SOPI Research Group
sopi.media.taik.fi/

Aalto University School of Art and Design

AHNE – Sound and Physical Interaction

Tension

A brief video showing Tension. An interactive spatial sound installation for multiple users.

A person enters the space and a generative sound is assigned to that person. The sound pans around in the 6-channel speaker system following the user in the space.

Up to 5 users can use the installation at the same time. Each person modifies the other sounds based on the distance to the other users. The closer you are to other people the more the tension in the sound increases.

Tension – Sound and Physical Interaction

Side note: watching these two videos makes me want to consult with someone on non-verbal expression, posture, and stage presence. That criticism is mounted at myself – I could use it. Perhaps we need an all-physical, unplugged music event for laptopists, controllerists, and electronic musicians. And I can at least say I’ve had some experience in this, working in the dance program at my undergraduate alma mater, Sarah Lawrence. Anyone game? (Sounds like something we could do while CDM is in Berlin in the fall.)

For their part, the Finnish research facility is working with dancers, along with Nokia Research Center. (Sadly, I can’t find documentation.) But I think interesting things happen when us non-dancers learn movement technique, too.

Elly Jessop and VAMP at the Maker Faire from The Amazing Rolo on Vimeo.

Yann Seznec aka The Amazing Rolo brings CDM his coverage of music tech at the Maker Faire in three episodes today.

Continuing the tradition of computer-augmented vocal performance and interactive gloves, Elena “Elly” Jessop shows off her VAMP system at Maker Faire. Elly is a Masters student at the MIT Media Lab’s Opera of the Future research group, headed by Todd Machover. Interestingly, Elly’s background is in conventional theater, including stage and costume design and choreography.

http://web.media.mit.edu/~ejessop/

VAMP stands for “Vocal Augmentation and Manipulation Prosthesis.” What’s really nice in this demo is that the results sound like more than just effects – they begin to become real augmentation, setting up a complex relationship between the vocalist and the sounds that come out.

It’ll be great to see your work evolve over time, Elly, as you fuse that experience. (And I know what a challenge can be, as I’m still working on fusions of my own, having likewise come from various non-digital backgrounds… heck, I made my way through puppetry class at Sarah Lawrence, even. It’s a lifetime-scale commitment.)

For more on data gloves and such: composer, computer scientist, and futurist Jaron Lanier did lots of seminal thinking about these ideas leading back to the 80s. And you can find some extraordinary work from “augmented vocalists” like Laetitia Sonami and Pamela Z. Here’s a terrific 2006 interview by Sua Constabile for Cycling ’74 with Laetitia:

Friend of the Site Gustavo Bravetti is back, getting the young Brazilian boys and girls on their feet with his virtual reality glove and Wiimotes and gesturally-controlled electronica. Gustavo sends us this video from the 2009 Tribaltech SC Edition in Campinas. Having seen a lot of DJs take the easy way out at festivals in front of throngs of people, it’s great to see someone really play his laptop – and while some of us, ahem, look goofy waving Wiimotes around, Gustavo makes it look good.

Gustavo also gives us the scoop on a new track release, orange. It’s inspired by … wait, Henry Purcell? (Indeed; see also: Wendy Carlos.)

I did produce this track specially for the Tribaltech 2009 SC edition, it was inspired on the classic piece by the baroque composer Henry Purcell (century XVIII), “The Funeral Of Queen Mary”. As usual all synthesizers and fx was made using only Ableton stuff, this time Operator, Analog, and Tension was used to create all synths and effects.

Gustavo also gets a rather eloquent review by our friend David Cross.

The incredibly simple melody of the short ‘Bocuma’ becomes a lump-in-the-throat meditation on man’s place in the universe through subtle pitch shifts and just the right mist of reverb. The slow fade-in on ‘An Eagle in Your Mind’ is the lonesome sound of a gentle wind brushing the surface of Mars moments after the last rocket back to Earth has lifted off.” Why not listen to, Only the Proletariat Floss’s by Screaming at the Mirror. With a truncated syncopation and approach that rivals only Tosh Guarrez pre “FartFlap”, “S.A.T.M” has taken steps to dismantle what was previously only dared mantled by the great Gilda Thrush when she fronted “Cycle Clause”. It’s as if Genghis Kahn got together for breakfast with Oliver Wendell Holmes and Virginia Wolfe and ordered just a bowl of homemade granola and then skipped out on the check. RATING: 11.-111 -David Cross

Previous Gustavo action on CDM:
Live + FM8 = Drum Kit Love: Free FM8 Drum Kit Download
Weekend Inspiration: Ableton Live Follow Actions, Dummy Clips, Making Snares
Gustavo Bravetti Show Us How To Glitch out Ableton Live
Interview: Gustavo Bravetti, Playing Music with Light and Interactive Gloves

Cassette-tape DJ battles are just one of TradeMark G.’s retro, regressive, subversive musical creations. He also likes to put on glasses, a white lab coat, and interactive sewing thimble gloves, in order to produce illegal, copyright-crushing musical performances.

Many of the techno-gimmicks seen here on CDM are one-offs and prototypes. The Evolution Control Committee, by contrast, has been producing “illegal art”, often with the aid of technology, for some 20 years. They’ve been “culture jamming”, dropping Napster bombs (remember Napster?), infamously attracting the ire of CBS, and dressing up as giant pairs of trousers and cans of Parmesan cheese ever since. (I’m especially fond of the giant pants costumes.)

For the last few years, they’ve been perfecting the Thimbletron, a glove with sewing thimbles attached to a hacked M-Audio Oxygen8. (I always knew those Oxygen keyboards would be good for something.) The interface gives them newly-expanded powers of sample triggering. Happily, unlike Wired Magazine, they don’t overuse the term “mash-ups” to describe what they’re doing. Try, instead, “plagiarhythm” or “plunderphonics”: “In the world of The ECC’s music, Public Enemy duke it out with Herb Alpert while TV news anchor Dan Rather is the new frontman for AC/DC.”

Thimbletronic Energy Technology Page (video link at the top)

TradeMark will be performing with the Thimbletron at the Maker Faire, as well as running the cassette tape DJ battle we saw earlier:

Call for Cassette Jockeys @ Maker Faire, Cassette Tech Roundup

CDM (meaning me) will be at Maker Faire all week, sending as much coverage and causing as much havoc as possible. I’m hoping Dan Rather shows up.

More glove music controllers:

Controlling Music with DIY Interactive Gloves

Eric Singer deserves special credit in this category. (See link for projects and videos.) In the early 90s, he hacked the Mattel PowerGlove, a controller for the Nintendo NES, for music. He followed that in 1999 with a Wireless MIDI Glove (right), which sends pressure and bend data for each fingertip.

So, what’s next?

Music thing gets the scoop on the latest iteration of the “gloves for music” trend, with a wireless model developed by 24 year-old Computer-Engineering grad Shaduz of Bologna, Italy:

3DID (3D Interaction Device) Project Page (Basic specs, photo, video)

You can tell the 3DID is a 2005 project: it’s got all the sexy modern specs you could want. Wireless is available via Bluetooth, with wired USB, serial, and Ethernet options. Data is transmitted as “oldskool MIDI,” yes, but also via OpenSoundControl (OSC), and there’s a TCP/IP driver, as well. Unlike Eric’s wireless glove, Shaduz doesn’t just rely on bend data: he adds 3 gyroscopes and 3 accelerometers, so the position of your hand can be expressive, as well. (That’s similar in conception, in fact, to the 3D controller Nintendo is working on for its upcoming Revolution console.) It’s all very cool, but I can’t help but wonder if it’s a little too complicated: sometimes focusing on fewer controls (as with Eric’s wireless glove) can be more expressive. But the really good news here is that all these features are possible with low power consumption, low weight, and EUR150 in parts.

Of course, the next step is taking time to learn the controller as an instrument. Imagine you’ve just invented the piano: you’d better take another few years and actually learn to play it.

If you want a wireless glove without doing all this work (albeit a simpler one), check out the very inexpensive P5 data glove: see CDM’s brief how-to, free Windows software. (The glove itself can be used on the Mac, too, with a little effort.)

Will interactive gloves ever make an impact as an expressive control for computers? The technology is here; my guess is it now depends on whether artful musicians can make that technology expressive, not the other way around.

Flex sensors are fab: these cheap strips send varying voltages when you bend them, seen in use in projects like Eric Singer’s sonic banana (basically, a bendable tube for triggering sounds). The trick is turning that flex data into something useful.

Hypersense Complex is a three-person collaborative working on new musical interfaces, and they’ve been nice enough to post details of the hardware and software they’re using. Hardware — all cheap, off-the-shelf stuff you can play with, too. Software — they’re doing fancy Python script interpretation to turn gestures into music in the free sound app SuperCollider. Check out details, sounds, and gallery. Not much aesthetics to their flex sensor glove — any fashion designers out there? But the exploration of musical gloves continues. Via Turbulence.org’s networked_performance blog.

Friday we looked at a big roundup of game controllers for music, courtesy Chris O'Shea. Ready to look like a cyborg when making music? Want to keep your entire budget under US$20? Here's where to get started (thanks to atariboy for some link pointers here):

1. Get the hardware: Pick up a P5 virtual reality gaming glove. (Check Froogle and the like; they're easy to find. I just picked one up for US$15.)
2. Get something to make music with: Get something to control, like plasq's free/donationware sampler instrument Musolomo.
3. Watch a video demo: Don't believe it will work? Take a pause for inspiration from this geeky video of P5 and Musolomo in action, courtesy OCP)
4. Dig into the research: Check Audiomulch's page for info on P5 research. You've got Melbourne ensemble Simulus to thank.
5. Windows drivers: Windows users will want P5 Glove MIDI.
6. Mac drivers: Mac OS X user will probably need two pieces of software, depending on their setup. P5osc supports new-wave protocol OSC, but not standard MIDI as you'll need for Ableton Live, most virtual instruments, and the like. To convert to MIDI, get P5osc and a copy of DoctorOp's
   Max/MSP-based P5 Glove MIDI Assigner, so you can assign the inputs of
   the P5 to MIDI data. (Click on the software tab to find it. Available
   as standalone in case you don't have Max/MSP.)
7. Start practicing: You're set! Assign the MIDI controllers
   to something interesting, like a sample loop point or filters or a
   synth, and play! Now you just have to figure out a way to make good music.

More music & gaming coverage coming soon; there's simply too much to cover here.

Game controllers have some serious advantages for music:
they're simple, flexible, fun, and because they're shipped in volume,
often cheaper than DIY or even manufactured music products.

Our friend Chris O'Shea has an ongoing roundup of game controllers as instruments:

• Part I

• Part II

He wants more help: if you see some he missed, drop him a line! (between CDM readers and near near future,
we should be set!) Meanwhile, CDM is rounding up some ideas, too; I
just have to get through some paid writing now but expect something
next week.

Good news: White Noise Audio Software's free Memory plug-in
lets you sample loops of audio in real-time, adjusting both loop length
and filter cutoff for some crazy live loop-playing. Did I mention it's
free?

Insanely great news: You can control this mayhem using a $20-30 P5 Data Glove
game controller. I leave it up to your colossal stage presence to make
sure you look cyber-awesome and not just like a dork, but either way
I'll bet you have a great time.

One thing you probably didn't see last weekend at Miami's WMC: this DJ. (via audioserve) Call it world's cutest scratching. Netherlands-based Lejo
specializes in this unique hand + props marionette theater. And the
show tours — if you happen to be passing through, say, Segovia or
Mechelen.

What, you say? You don't think DJs are real musicians? Fine. Check this rockin' accordion duet.