Here’s how interfaces normally break down. You’ve got your conventional, tactile interfaces, like a knob. You’ve got your touch interfaces, which lack tactile feedback (you touch them, but they don’t push back). You’ve got your gestural interfaces, which have you waving your hands in the air without touching anything and without any tactile feedback. (They’re generally the most challenging, because your brain has no feedback for what it’s doing.)

Syntact creates an entirely new category. It’s a gestural interface, of the “waving your hands around in the air” sort. But while your hand is in mid-air and isn’t touching anything, it does provide tactile feedback. It pushes back as you move your hand around, giving you interactive feedback. The way it pulls it off: sound. 121 ultrasonic transducers beam sound at a particular point, so that you feel something as you move.

You can see a bit of what this means in the new video, above. I’m hoping to get a hands-on (erm, hands-off) demo soon from the designer. The basic specs:

• Optical analysis of gestures, using a USB camera built into the interface
• MIDI control, for use with any live performance or music making rig (or other media)
• A control panel for selecting different sonic images and adjusting scaling.
• A built-in music solution visualizes sound and makes it easier to map to your own MIDI files.

More information:
http://www.ultrasonic-audio.com/products/syntact.html

Also well worth checking out the directional speaker tech from these Slovenia-based developers – directional sound is another huge area of innovation.
http://www.ultrasonic-audio.com/products/acouspade.html

If you want to try this in person, it’ll be at the Beam Festival in London in late June.

Side note: Yes, I’m looking into that LEAP thing, for more gestures, albeit without tactile feedback. Stay tuned.

luanna is a beautiful new application out of Tokyo-based visual/sound collective Phontwerp_. Amidst a wave of audiovisual iPad toys, luanna is notable for its elegance, connecting swirling flurries of particles with gestures for manipulation. I imagine I’m not alone when I say I have various sample manipulation patches lying around, many in Pd, lacking visualization, and wonder what I might use in place of a knob or fader to manipulate them. In the case of luanna, these developers find one way of “touching” the sound.

As the developers put it:

luanna is an audio-visual application designed for the iPad
that allows you to create and control music through the manipulation of moving images.

The luanna app has been designed to be visually simple and intuitive, whilst retaining a set of rich and comprehensive functions. Through hand gestures you can touch, tap and manipulate the image, as if you were touching the sound. The image changes dynamically with your hand movements, engaging you with the iPad’s environment.

The interface is multi-modal, with gestures activating different modes. This allows you to select samples, play in reverse, swap different playback options, mute, and add a rhythm track or crashing noises. It’s sort of half-instrument, half-generative.

Phontwerp_ themselves are an interesting shop, descibed as a “unit” that will “create tangible/intangible products all related to sound.” Cleverly naming each as chord symbols, ∆7, -7, add9, and +5 handle sound art, merch, music performance / composition / sound design, and code, respectively. That nexus of four dimensions sounds a familiar one for our age.

Sadly, this particular creation is one of a growing number of applications that skips over the first-generation iPad and its lower-powered processor and less-ample RAM. Given Apple can make some hefty apps run on that hardware, though, I hope that if independent developers find success supporting the later models, they back-port some of their apps.

See the tutorial for more (including a reminder that Apple’s multitasking gestures are a no-no).

US$16.99 on the App Store. (Interested to see the higher price, as price points have been low for this sort of app – but I wonder if going higher will eventually be a trend, given that some of the audiovisual stuff we love has a more limited audience!)

Readers request Audio Copy and sample import right away. I think sample import, at least, could easily justify a higher price, by making this a more flexible tool.

Find it on our own directory, CDM Apps:
http://apps.createdigitalmusic.com/apps/luanna

http://phontwerp.jp/luanna/

Very similar in its approach is the wonderful Thicket, well worth considering:
http://apps.createdigitalmusic.com/apps/thicket

See our recent, extensive profile of that application’s development:
Thicket for iOS Thickens; Artists Describe the Growth of an Audiovisual Playground

See also, in a similar vein, Julien Bayle’s recent release US$4.99 Digital Collisions:

http://julienbayle.net/2012/04/07/digital-collisions-1-1-new-features/

http://apps.createdigitalmusic.com/apps/digital-collisions-hd

When we last caught up with the touch-less, gestural music-making of composer Chris Vik, the Australian musician was sharing his own Kinectar software and playing both dubstep and ambient scores for modern dance. Now, Vik is back playing a very substantial physical instrument: Melbourne’s four story-tall, MIDI-retrofitted Town Hall Organ. Here, the Max-powered software takes on some very big sound from some very big pipes.

He writes:

I’ve created my own software Kinectar, which allows the use of the Kinect to control MIDI devices, ie. playing notes through simple gestures and motion. The Melbourne Town Hall Organ got a referb in the late 90s adding the ability of MIDI messages to active the notes… this happened.

Controlling a 4-story pipe organ with the Kinect

Previously: From Beautiful Ambient Modern Dance to Dubstep, Gestures to Music in Kinect (Download the Tool)

It started as some compelling demos or proof of concept, but it’s plenty real now: the tools for translating movement, gesture, and dance from the body to interactive music march forward. Empowered by Microsoft’s Kinect and an artist-friendly toolchain, even a single, clever developer can do a lot. Sound designer, music producer, and Max/MSP developer Chris Vik of Melbourne has been one of those busy early pioneers, with an incredible tool called Kinectar.

So, the tech is cool and shiny and impressive: what about the actual music? And, even more importantly, what if all the hand waving and moving about could be meaningful? That’s the next step. For his part, Chris is teaming up with a dancer and choreographer to combine his compositional ideas with someone who knows how to move. The Dubstep-y demos (all below) are impressive, true, but the early tests of the work with the choreographer are simply beautiful, and demonstrate that wobble bass isn’t the limit of what this can do. They also turn the arbitrary arm-waggling into a part of the art.

And as for you: the software’s alpha, but you can fire up your copy of software like Ableton Live and grab this software for Mac or Windows and try it yourself. So if you don’t like the results – be the gesture-controlled basslines too wobbly, be they not wobbly enough – you can put your music, and your movement, where your mouth is.

At top, Chris shows off an early test of the dance collab. (There’s more to come.) Below, a tutorial that shows how this works with Ableton. And read on for more from Chris on what the work with the dancer is about, and what the tool can do.

Chris writes:

Since April 2011 I’ve been working solidly with the Microsoft Kinect, developing my software, Kinectar, to enable its use as a MIDI controller for performing music live. I’ve done a number of performances around Australia since I started the project, however, it’s safe to say that, although I would consider myself an electronic musician, I’m certainly no dancer. Enter, Paul…

Dancer, Paul Walker and I have joined forces to bring the Kinect controlled music concept into the world of contemporary dance. Recently we obtained a residency at PACT theatre (centre for emerging artists), where we spent the week developing different ways of implementing my Kinect music control system in a dance context.

My system is developed in Max and uses OpenNI drivers, OSCeleton and Ableton Live.

via Chris’ blog

CDM will check back in with Chris soon, because:

I’ve got some more videos to release over the coming weeks from a range of my different Kinect music performance applications, including controlling/conducting the Melbourne Town Hall Organ and a 100+ speaker Kinect-controlled diffusion performance. I’ll keep you posted when they’re released!

More on the software:

Kinectar Performance Platform is a toolkit developed by music producer Chris Vik to allow the use of Microsoft’s Kinect motion tracking sensor in computer-based music. The software is designed for electronic musicians to expand the way they control their music in a futuristic and extremely expressive way, using only the waving of hands and a small amount of creativity. It can be used to control the simplest of parameters like a filter or LFO, play notes and chords on a sampler or synthesizer, or be programmed to control an entire live-set through nothing more than gesture.

Key Features:

Movement Tracking UI allows manipulation of the Kinect’s human tracking capabilities, displaying all relevant data extracted from the hands location in 3d-space

Instrument Builder lets the user build virtual ‘instruments’ by outputting MIDI notes in three modes:

• Static – Produces a single note value. Useful for drum triggers, turning on/off effects within a DAW or feed that trigger into Kinectar to switch between presets using your gesture
• Solo – Do sweeping solos by selecting from over 40 musical scale presets or click the notes on the UI to make your own
• Chord – Create a progression of up to 8 chords per preset to play live

Global Flags lets you turn on/off Kinectar’s instruments using a MIDI note sent from your DAW, external MIDI controller or Kinectar itself

MIDI Preset Control lets you switch between Kinectar’s presets and instruments using a single MIDI note

Value Editor enables many more MIDI/OSC outputs, for controlling device values

Visual Metronome popout window sits on top of all programs to make it easy to see if you’re in-time when the music gets messy

It’s labeled “rough alpha,” so don’t expect a finished tool here, but you can go download it and give it a try (or learn more about what’s possible):

http://kinectar.org/download

And now, the obligatory (but quite awesome, Chris) Dubstep demo videos:

Building on the original Midi Fighter, a 4×4 array of arcade push-buttons, the Midi Fighter 3D adds interactive, light-up color feedback and gyroscope-powered motion sensing. The work of electronic music site DJ Tech Tools, it’s an impressive-looking piece of work. But if you’re not interested in the “3D” sensing, don’t overlook the clever color feedback and bank shifting, which could prove as much of a draw.

The Midi Fighter 3D, announced today, will ship in April at US$249. There are now orders yet, but there is a preorder list.

DJ Tech Tools is pushing the 3D orientation functionality. In a good way, it mirrors a bit of the branding and design we see from Nintendo (well, at least that “3D” moniker). If you don’t mind moving your controller around as you play, it looks like it can do some impressive things. Dan White of DJTT explains how it works to CDM:

The 3D uses a gyroscope and a compass to track the position of the controller in space. The gyroscope tracks relative position (meaning angling the controller towards any of its sides), and the compass tracks rotation along the same plane that the controller is on (think turning the controller like a steering wheel).

While the sensing may not appeal to everybody, the big advantage here is integrating continuous control of parameters (which buttons obviously lack), in a way that’s integrated into the design and gestural.

A wrist-strap will be available, and designed in such a way that you can access all the controls, including even those on the side.

At $249, though, fans of the original could easily justify the purchase based solely on the new light-up, assignable color indicators on the buttons. Apart from looking cool, they promise to make elaborate control setups possible, with the aid of bank controls and lots of customization in the software. You get four banks of controls via the top, but there are also six nicely-integrated triggers on the side which can be used for whatever you like. That could give you more banks, effect kill switches, or some other function you haven’t thought of yet. The fimware can send up to 68 unique Control Change messages and 70 button messages, so presumably DJTT is betting – as they have with their other product line – on lots of preset ideas for different performance rigs and styles.

All of this communication happens via MIDI, so using it with your favorite software is a cinch.

Specs:

• Included configuration software
• Customizable RGB arcade buttons: 4 x 4 button array, with individually-addressable light-up RGB feedback on each button
• Four banks, six side buttons
• 3D motion tracking of five movements

It’s hard not to notice the cable in the images. DJ Tech Tools tells us that’s their own DJTT USB cable, which will be bundled with the hardware and also available separately. They say it’s a “high-quality” USB cable – I’m guessing the main test is whether it can stand up to moving the hardware around, since it isn’t wireless. Having right-angle USB cables is hugely useful in tight corners, though; Hosa was showing off something like that at NAMM and I’m happy to replace my USB collection with them.

Also worth noting: DJTT says they’re applying for a patent on the five-way motion control tracking method they’ve developed. (I find the patent process to be pricey and arcane, personally, but I’ll be interested to see how it goes for them!)

$249 seems to me a really good deal for this gear, but if you liked the brute-force simplicity of the original controller – and its greater customization options – the Classic remains available, starting at US$119.99.

More details:
Introducing the Midi Fighter 3D [DJ Tech Tools]

Trust your ears.

It seems a simple instruction, in the teaser video for this project by Stockholm-based producer/composer Håkan Libdo. But for those of us used to having vision, focusing in on one sense – even the sense on which we rely for music and sound – can be an extraordinary experience. If digital interface design has done anything, it has forced new ways of looking at design across senses, and not just in a weary repetition of “we always do it this way.”

Playing tennis in InvisiBall becomes a new experience. Sonic cues along direct the ball from side to side. And apart from experiencing the game – blindfolded or even as a non-seeing person – the result can be a performance. Developed with programmers Magnus Frenning and Jonatan Liljedahl, the game relies on three-dimensional sound and senses user play, a new musical game.

A computer, while tracking the players, seamlessly mixes sonic cues with a musical soundtrack, so that the results are melodic and not only sound effects. Played in a darkened room, the rackets emit infrared light. Wiimotes here don’t move, as they do when playing games like Nintendo’s Wii Tennis; instead, they’re just IR sensors.

An audience gets to “see” the position of that invisible ball, so that spectating is a slightly different sensation; a TV displays where the ball would be.

More information on Håkan Libdo’s site; he tells us this could be making appearances at festivals this year. (So… if you’ve got a festival, and you’re reading… uh, send CDM some tix, too, and we’ll visit?)

http://hakanlidbo.com/archives/2075

Alex Nowitz for BodyControlled #2 from CDM on Vimeo.

Electronic media artist Mario de Vega (Mexico City/Berlin) says his work plays with the creation of “unstable systems.” As part of the official Vorspiel, or lead-up, to Berlin’s massive Transmediale festival, here we get to visit two artists working with the materiality of live performance, drawing from the festival theme of “in/compatible.” The sonic environments they create seem poised on the brink of sonic chaos, a dance at the edge of entropy.

CDM will again be editorial co-presenter of BodyControlled; you can see the show for free (donation suggested) in Berlin at LEAP, or tune into the live video stream from anywhere in the world, and we’ll be bringing you details of the artwork. We’re a ticket to Alexanderplatz that’s even cheaper than easyJet, in other words. The performances start at 20h CET Thursday, 26 January. (That’s 2p East Coast time / 11a Pacific, so scare your office mates and turn it up loud.) Full details below.

At top, composer/singer Alex Nowitz demonstrates his gestural performance techniques. I got to see his work for the first time at the Patterns + Pleasure Festival in the fall at Amsterdam’s STEIM research center. While at STEIM, Nowitz built on previous work with the Wii remote, and augmented his gestures with a new instrument, entitled the “Strophonion.” You can see that creation in the video above.

With each contortion of his body, Nowitz rips apart sounds, all while sputtering non-lingual utterances with his gymnastic voice. In the Amsterdam performance, one had the sense of following him into the Schwarzwald (Black Forest), an operatic odyssey echoing with forboding birdsong. But the system can also be dynamic and even, at moments, whimsical.

steim.org/projectblog/?p=3715
nowitz.de/

For his part, Mario de Vega’s “unstable systems” flirt even more with this notion of engineered incompatibility, with sounds that seem like they will explode in an earthquake-like tremor.

Mario de Vega for BodyControlled #2 from CDM on Vimeo.

mariodevega.info/

Films by João Pais, co-curator of the series; edited by CDM.

Also on this program, more works engage the idea of what the curatorial statement terms “hidden acoustics”:

Echo Ho (Canada/Cologne, DE)

Tuned to Site #26012012
This title is from a series of concerts, called “Tuned to Site #…”. As a whole, the series formulates the idea of “musification of urban landscapes”.
In the first performance of this series in 2012 Echo Ho will play a set of instruments: a self-fabricated hybrid semblance of the ancient Qin from China, which combines traditional acoustic and digital interfaces in one unique transparent plexiglas body. Like a sensor box, it will enable Echo Ho to make field recordings of inaudible hidden sounds within
the city environment, such as electro-magnetic fields, variation and wind movements. The performance thus marks the process of generating action by outlining situations in which sounds may occur.

http://www.echoho.net/

Ignaz Schick (DE)
Turntablist, sound artist, performer & composer Schick promises, through motors and objects, genuine accidents:

Site-specific performance with transducers, wireless controllers, feedback systems and back tape
Through accidents and their outcomes, actions, processes and objects that conceptually connect with acoustic information, the work of Mario de Vega researches the value of vulnerability, exploring the causes and effects that determine the construction of realities. In this site-specific performance with transducers, wireless controllers, feedback systems and back tape, de Vega is investigating aesthetic and social realms through a multiplicity of mediums.

http://www.zangimusic.de

Co-curator João Pais tells CDM that this installment, in keeping with Transmediale’s theme, will “give the performers a room where they can show their ways of working with the dissociation of matter (through sound, in this case) and expression.” Pais co-curates the event with Daniel Franke of LEAP.

This episode includes two self-made instruments that expand on existing practice, he says, in the case of Nowitz and Ho, and the hacked and modulated machines of Schick and Vega.

More information; where to see the show

26 January 2012, 20h (free/donation)

Show details

Anywhere in the world – all performances will be available from 20.00 CET via live stream:
http://bit.ly/uXRgyq

Facebook: on.fb.me/AmEtO9

LEAP
Lab for Electronic Arts and Performance
(Berlin Carré, 1. Stock)
Karl-Liebknecht-Str. 13
10178 Berlin

How to find LEAP

Look around the room you’re in. Drum your fingers against some of the objects around you. Now imagine that you could turn those touches into any imaginable sound – and all you’d need to play them is a single contact mic. And we’re not talking just simplistic sounds – think expressive, responsive transformation of the world around you, all with just that one mic, thanks to clever gestural recognition.

Bruno Zamborlin has made that idea a reality, with hold-onto-your-chair results. It’s not available yet for public consumption, but it’s coming.

Bruno explains to CDM:

Mogees is a novel way for transforming any surface into a musical instrument.

By putting a (very cheap) contact microphone over a surface, the software can recognise different types of touch and associate them with different synthesisers.

Users can train the software with their own ‘gestures’, using both bare hands and objects. In the video demo we put the microphone over different surfaces such as kitchen tables and balloons.

The sound synthesis is based on two different techniques:

1 – physical modeling, which consists in generating the sound by simulating physical laws. Different materials can be simulated, such as membranes, strings, tubes and plates

2 – mosaicing, that works as follow: first, users load a sound folder; then, the noise coming from the microphone is analysed and the software continuously finds and plays its closest segment within the sound folder.

Mogees has not been realised yet. It could be published as Max4Live patch in some month.

Yes, we’ll be watching for future versions and publication, with bated breath and eager hands.

http://www.brunozamborlin.com/mogees

Update: Readers point to similar earlier work; obviously, contact mics have long been readily available. I’m not always concerned with whether something is new or not – old and cool can be cool. But what does appear to be new here is the additional gestural analysis to work more accurately with location. That takes an existing technique and refines its musicality. -PK

LA-based bang Fol Chen (Asthmatic Kitty records) wanted to go beyond the computer as the playback and manipulation device for their music. So they worked with collaborators to invent a solution. In a new video, sounds, and an interview, we can share some of how this came into being.

Built with the monome creators (Brian Crabtree and Kelli Cain) and LA research and experimentation center Machine Project, the Tetrafol is a custom, pyramidal sound device. The object warps Fol Chen’s music using gestural manipulation of playback, but can also use your own samples. And with open-source circuit and firmware, the project could be an opportunity to learn or to build your own creation.

Description:

The Tetrafol is a hand-held tangible electronic sound toy. Circuits enclosed by a wooden tetrahedron detect orientation and motion-gestures to modify the playback of a collection of Fol Chen’s micro-compositions, allowing the user to explore sound through physical manipulation.

The battery-powered device has its own internal speaker but can additionally be hooked up to a headphone or amplifier.

The circuit and firmware are based on open-source hardware and is itself published as open-source, allowing anyone interested to learn about its deepest inner-workings.

Here’s a sample of the project, via the Tetrafol-created Fol Chen track “So Good”:
Fol Chen – So Good by WeGetPress

Built by hand in a limited run of 100, the device sells for US$110 direct from Machine Project. We spoke to monome’s Brian Crabtree about the project – and a new, comically-inclined video shows off the project.

Stems from the track “Back on Kent” come preloaded:
Fol Chen, “Back on Kent” by asthmatickitty

CDM: How did this collaboration come about? How did you work together?

brian: kelli and i have a loving and working relationship with machineproject, a phenomenal organization founded by our good friend mark allen. we’re always amazed at the fantastical variety of projects that are born there. a few inspiring works of recent include a cash machine designed for a children’s museum and a workshop on lockpicking. so when mark approached us on behalf of his good friend adam goldman and adam’s band fol chen regarding a possible collaboration we were all ears. the goal was to design and produce some sort of synthesizer-sampler-effect-instrument-toy-object to accompany the release of their new album. that was about a year ago and we’re happy to see it finalized and soon in playful hands.

in the early stages there was much whittling of ideas (too expensive, too complicated, etc). we arrived at some sort of gestural sample player and a demo video was ready to show the proof of concept (we live on opposite coasts so there was much back and forth through internets and mails) the basic build used a waveshield (by adafruit) and an arduino and some very hacky code i modified.

fol chen provided the sound set. kelli and i proposed a series of enclosures– diamonds, stars, ice cream cones, d20. the tetrahedron ended up being the most beautifully minimal, and incidentally the most cost effective. our friend jason voytilla laser cut a prototype from thin birch ply and we sent the “finished” sample to california where it underwent a series of intense focus groups – thanks to the rigorous machineproject laboratories. after more back and forth, and basic design changes here and there we were in agreement. we used our very reliable production chain that we depend on for monome releases… it was nice really helpful to have that all in place and sped up the process considerably.

the tetrafol accompanies the release of some exciting new fol chen tracks, and there will be a release party of sorts in early december at machineproject. should be very interesting, as the current installation is a 30 foot deep window sill of sorts.

What went into the design? The construction of the thing?

the final circuit board is an arduino, [Lady Ada - Limor Fried] waveshield, and accelerometer smashed together and made very small. i really just put existing technologies together– i can’t take a lot of credit here.

the industrial design was more fun. we didn’t want to use plastic so we experimented with felt and wood. coming up with a size, shape, and feel were the main goals- to create something that was pleasant to hold and sturdy enough to be tossed in the air.

What’s the basic notion of the instrument?

it plays sound loops, or “micro-compositions” written by fol chen. when you pick up and tilt the device it modifies playback: in one axis it changes the playback speed, in the other it triggers a variable-speed stutter (playback position jump). sounds are changed by a shaking motion. given the response is immediate, it comes alive very quickly.

i’d also hesitate to endorse it as an Instrument, though it’s very playable. it’s a bit like a responsive buddha box.

Any other documentation?

i posted the firmware/hardware source on github. there is a no “build your own” guide as you’d be much better off just looking at the waveshield documentation (which is very good.)

this was a fun collaborative side project– and it makes me even more curious to see how musicians continue to create tangible objects to accompany their releases.

More information:
folchen.com

Tetrafol @ GitHub (firmware + hardware, under a GPL v3 license)

When we last saw Onyx Ashanti, he was speaking of a grand vision to remake himself into a music-performing Tron. Now, the elements of that vision are coming together, with a crowd-sourced funding campaign that ends today, Friday. Update: Apparently after seeing this story, IndieGogo extended the funding deadline for five days, with the new deadline Thursday, December 1.

I knew Onyx back when he was playing more conventional wind controllers. Now, that fingering arrangement is freed from the virtual wind instrument, handheld and movable through space. Because of the plans to open source everything he’s making, you might yourself pick up that hand controller – or, if you’re like Onyx, go full-tilt with physical training to make your body do new things and a carbon fire, full-body prosthetic transformation.

Onyx has been at auditions for the main TED (the big one, not TEDx), experimenting with a beatbox configuration, and honing alien-like futuristic human reinvention with the help of artist Christopher Logan, aka Loganic. Loganic makes the art, then prosthetic engineer Uli Maier – with doses of carbon fiber – translates those notions into physical form. And the whole thing is mobile; Onyx draws on his busking background to take this thing wherever he goes.

Initially built as an open/proprietary hybrid, the new system is increasingly open source from the ground up, from customized Linux-based software to Pure Data (Pd) patches to open source designs for the molds. The wearable system can be 3D printed. Plans for the system also were featured in Make Magazine.

It’s actually quite a lot to digest, but Onyx has been posting videos, the most recent and illustrative of which I’ve included here. And because there’s a lot to do physically, from personal training to buying clay to engineering the prosthetics, Onyx is relying on crowd-sourced funding. In place of Kickstarter, which has specific requirements for minimum funding and other restrictions and requires US-based banking, he’s opted for IndieGogo.

If you invest just a few dollars, you at least get music; with successively larger donations, Onyx throws in his software, custom artwork and posters, t-shirts, or starting at US$500, the custom hardware itself for your use.

The IndieGogo campaign ends Thursday, December 1:
IndieGogo: Beatjazz System

– but we’ll be in touch with Onyx on an ongoing basis, so let me know if you have questions for him or want to watch this continue to evolve.

Videos showing the making of the elements of the system:

Above: New visualizations in 3D have vastly expanded the now-Pure-Data-based audio system with heads-up displays worthy of the spacesuit. Below: Some of the beautiful concept artwork produced for the project.

IndieGogo Campaign

Previously: Onyx Wants to Make Himself Into Helmeted, Wearable-Music-Tech Tron, With Your Help