Electronic music making has had several major epochs. There was the rise of the hardware synth, first with modular patch cords and later streamlined into encapsulated controls, in the form of knobs and switches. There was the digital synth, in code and graphical patches. And there was the two-dimensional user interface.

We may be on the cusp of a new age: the three-dimensional paradigm for music making.

AudioGL, a spectacularly-ambitious project by Toronto-based engineer and musician Jonathan Heppner, is one step closer to reality. Three years in the making, the tool is already surprisingly mature. And a crowd-sourced funding campaign promises to bring beta releases as soon as this summer. In the demo video above, you can see an overview of some of its broad capabilities:

• Synthesis, via modular connections
• Sample loading
• The ability to zoom into more conventional 2D sequences, piano roll views, and envelopes/automation
• Grouping of related nodes
• Patch sharing
• Graphical feedback for envelopes and automation, tracked across z-axis wireframes, like circuitry

All of this is presented in a mind-boggling visual display, resembling nothing more than constellations of stars.

Is it just me, or does this make anyone else want to somehow combine modular synthesis with a space strategy sim like Galactic Civilizations? Then again, that might cause some sort of nerd singularity that would tear apart the fabric of the space-time continuum – or at least ensure we never have any normal human relationships again.

Anyway, the vitals:

• It runs on a lowly Lenovo tablet right now, with integrated graphics.
• The goal is to make it run on your PC by the end of the year. (Mac users hardly need a better reason to dual boot. Why are you booting into Windows? Because I run a single application that makes it the future.)
• MIDI and ReWire are onboard, with OSC and VST coming.
• With crowd funding, you’ll get a Win32/64 release planned by the end of the year, and betas by summer (Windows) or fall/winter (Mac).

I like this quote:

Some things which have influenced the design of AudioGL:
Catia – Dassault Systèmes
AutoCAD – Autodesk
Cubase – Steinberg
Nord Modular – Clavia
The Demoscene

Indeed. And with computer software now reaching a high degree of maturity, such mash-ups could open new worlds.

Learn about the project, and contribute by the 23rd of March via the (excellent) IndieGogo:

http://audiogl.com

Turning music and sound into three-dimensional worlds often yields something that fields like a trip through space. But this feels like a real trip. Through pulsing, glowing starfields, “Versum”‘s audiovisual movements are brain-bendingly transformative. Artist Tarik Barri has created an integrated world of sound and image that makes the interface and the compositional realms seamless. It seems as though this really is a musical universe, through whose harmonies of the spheres you can fly like. Boldly going, indeed.

Ingredients: Max/MSP/Jitter, Processing, Java, SuperCollider, GLSL [the 3D shading language], and … some serious skill and time, I imagine.

The work has been in development for some years (not surprisingly, given the results). But it surfaced again as we brought up the 3Dconnexion SpaceNavigator hardware as a practical controller for 3D. See Create Digital Motion:
Look at Me, I’m Flying: SpaceNavigator Hardware + Blender

Tarik’s work resurfaced after a presentation in the UK. Reader janklug writes:

I’m just back from the M4_u Max/MSP/Jitter conference in Leicester (was great, btw), where Tarik Barri presented his project ‘Versum’, both as an installation and as a performance.
The user (and in case of the performance, Tarik) navigates through this incredible 3D-space-sequencer-universum with the help of a SpaceNavigator; glowing objects floating in this space produce sound, and as you approach them, they even give this nice doppler effect…
It was totally amazing to be able to float between pulsing rhythm-planet-objects and shiny drone-beams; navigation was easy and natural. Tarik uses a combination of Processing and Max/MSP; don’t know which one the SpaceNavigator is connected to.
Having tried this, I immediately ordered one; I think it also could be a great interface for M4L…

More information:
http://tarikbarri.nl/projects/versum
PDF documentation [2009]

Significantly, it’s really the act of flying that controls the music. That remains interactive, but it’s the movement through the three-dimensional space that determines what you hear. As the artist explains:

This virtual world is seen and heard from the viewpoint of a moving virtual camera with virtual microphones attached. This camera, controlled in realtime by means of a joystick (or any other kind of controller) moves through space, similar to how first person shooter games work. Within this space, I place objects that can be both seen and heard, and like in reality, the closer the camera is to them, the louder you hear them. So when the camera moves past several visual objects, you simultaneously hear several sounds fading in and out. Consequently, the way the camera travels past them actually causes melodies and compositional structures to be seen and heard.

The visual position of each object coincides with the panning of its sound: objects to the right of the camera will also be heard on the right, and those behind the camera will be heard from behind in case a surround speaker setup is used. This principle also applies to the Z-axis, meaning that sounds can be heard coming from above and below if the speaker setup supports it.

That’s the essential question, to me, when looking at 3D environments for music. What about the dimensionality will interact with the music? Is it something spatial, or will there be other sorts of interactions? (New Zealander-turned-Berliner Julian Oliver worked extensively with game engines, for instance. One solution for him was modifying the “gun” in those games to be an implement for doing things in the space, turning swords into plowshares after a fact by making the gun produce music rather than kill virtual entities.)

So, now you’ve seen some of the technical demonstration. But Tarik uses his work as an environment in which to make audiovisual performances. Here’s what some actual live playing looks like, in a beautiful, meditative piece called “Eleven”:

In fact, the biggest challenge to me of a piece this awesome is that you want an immersive environment, not just the small, rectangular screens that are often all festivals and venues can afford.

Holodeck, anyone?

More:

Compare the complex model of what a computer can use to control sound and musical pattern in real-time to the visualization. You see knobs, you see faders that resemble mixers, you see grids, you see – bizarrely – representations of old piano rolls. The accumulated ephemera of old hardware, while useful, can be quickly overwhelmed by a complex musical creation, or visually can fail to show the musical ideas that form a larger piece. You can employ notation, derived originally from instructions for plainsong chant and scrawled for individual musicians – and quickly discover how inadequate it is for the language of sound shaping in the computer.

Or, you can enter a wild, three-dimensional world of exploded geometries, navigated with hand gestures.

Welcome to the sci fi-made-real universe of Portland-based Christian Bannister’s subcycle. Combining sophisticated, beautiful visualizations, elegant mode shifts that move from timbre to musical pattern, and two-dimensional and three-dimensional interactions, it’s a complete visualization and interface for live re-composition. A hand gesture can step from one musical section to another, or copy a pattern. Some familiar idioms are here: the grid of notes, a la piano roll, and the light-up array of buttons of the monome. But other ideas are exploded into spatial geometry, so that you can fly through a sound or make a sweeping rectangle or circle represent a filter.

Ingredients, coupling free and open source software with familiar, musician-friendly tools:

• Two projectors
• A monome
• Processing, the elegant and artist-savvy free software for visual code
• Ableton Live and Cycling ’74′s Max for Live, acting as the interactive glue with the sound world
• Drumaxx, Image-Line’s tasty physical-modeled drum synth
• Native Instruments Battery, the sampled drum engine
• Eclipse, the free IDE, for Java coding in this case
• Community Core Vision and reacTIVision (based on our previous info, at least), free and open source community-based projects for making the interfaces you see in movies happen in real life.

Another terrific video, which gets into generating a pattern:

Now, I could say more, but perhaps it’s best to watch the videos. Normally, when you see a demo video with 10 or 11 minutes on the timeline, you might tune out. Here, I predict you’ll be too busy trying to get your jaw off the floor to skip ahead in the timeline.

At the same time, to me this kind of visualization of music opens a very, very wide door to new audiovisual exploration. Christian’s eye-popping work is the result of countless decisions – which visualization to use, which sound to use, which interaction to devise, which combination of interfaces, of instruments – and, most importantly, what kind of music. Any one of those decisions represents a branch that could lead elsewhere. If I’m right – and I dearly hope I am – we’re seeing the first future echoes of a vast, expanding audiovisual universe yet unseen.

Previously:
Subcycle: Multitouch Sound Crunching with Gestures, 3D Waveforms

And lots more info on the blog for the project:
http://www.subcycle.org/

AudioGL, a project teased in videos first in April and then again last week, is a new concept in designing a user interface for real-time music creation. Visuals and sound alike are generative, with the rotating, 3D-wireframe graphics and symbolic icons representing a kind of score for live synthesized music. The tracks in the video may sound like they’ve been pre-synthesized, polished, and sampled from elsewhere, but according to the creator, they’re all produced in the graphical interface you see – what you see is what you hear.

The newest video, released this week, reveals in detail the project’s notions of how to make a 3D, live music interface work. The UI itself is similar to other graphical patching metaphors, but here, like exploding a circuit diagram in space, routings and parameter envelopes are seen and edited in a freely-rotating view in three dimensions rather than on a flat plane.

There’s a reason interfaces like this have been few. Computer displays and pointing methods tend to be heavily biased to two-dimensional use, modeled as flat planes like pieces of paper. Working in two dimension is simply easier; there’s no reason you can’t take another layer of parameters and represent it on a two-dimensional interface. And rotating around in 3D space can make it difficult to keep your bearings.

Those challenges, though, don’t make this less interesting – they make it juicier and more delicious as design problem and stunning, futuristic musical model. Freed in three dimensions, a complex set of envelopes and parameters has room to spread out visually, making a kind of spatial score. This particular project strikes an interesting balance between traditional, iconic UI – operators are represented with graphic symbols – and more free-flowing geometry representing the sequencing and envelopes. To me, the latter is more compelling, but putting the two together may make the program more flexible and familiar to users of other music software.

What could knock you out of your chair, though, is the sheer depth of the software teased in the video. This is no simple tech demo: it’s an attempt to build an entirely new, live-synthesizing music tool from scratch in 3D. It’s like the International Space Station of music software, assembled in some void. I got a couple of tips on this today, and some are even wondering if it’s real.

It appears to be very real. Whether this particular tool is usable or not to me almost isn’t important: a spectacular failure in this arena would even be useful. Anyone waiting for some sort of “singularity” in music tech, I think it’s coming: it’s just going to be a singularity of human software ingenuity, explosive creativity and invention from independent developers. I can’t wait.

Stay tuned to find out more about this particular project.

See also the earlier video (not able to grab the embed code for some reason).

Thanks, Bodo Peeters, among others, for the tip.

A new tool could be for the expressive, not just the lazy. That’s the read of Auto-Tune for guitar, and it makes me excited to see what people will do with it. It could be the advent of the true digital guitar.

Antares teased their efforts to bring Auto-Tune technology to guitars earlier this month, having gotten as far as working proof-of concept. (See Harmony Central’s exclusive video above, and Axetopia, Synthtopia.) I hadn’t worked out anything intelligent to say about it, perhaps because I was cowering in a corner in fear.

As a technologist, I have great respect for what Antares does, and their portfolio goes far beyond just the flagship vocal pitch correction. But suffice to say, Auto-Tune has been used in recording in some pretty unpleasant ways – the fault of the user, not the software, I’d argue. It’s regularly applied in order to suck the life out of great, perfectly-tuned singers, as well as to cover for people who can’t really sing, to the point that producers seem to not understand what the sound of a human voice is in all its complexity. (Case in point: Glee. The talented cast sounds incredible live and onstage, and like they have android stand-ins when they’re on the show. In fact, if you disagree with those uses, please – go use some of Antares’ terrific software for good, not evil, and I’ll write about it.)

Auto-Tune as a name, then, has come to symbolize a revolution, an extraordinary blockbuster of software – and the butt of a joke. So, it’s hard not to see a product called “Auto-Tune for Guitar” and carry some of that bias. Sometimes, as writers we actually need our readers to add some perspective.

Auto-Tune for Guitars could likewise be misused to smooth out some of the guitar’s natural intonation subtleties, though I think the danger is far less so than it is with the voice. But it’s more than that.

Reader Jesse Engel reflects on what it could mean. He notes that the significant advance is building the intelligence into the guitar, not just the computer, and that applications could be varied:

Don’t know if you saw this, but Antares has taken a fresh swipe at HEX guitar, putting a processor in the guitar and using it to do some more modern (Auto-Tune, emulation, etc.) processing. [Ed.: Hex refers to the practice of adding individual pick-ups to each of six strings. -PK]

The hex has been around for a while, but it’s a big deal to use it in this way for guitarists since you don’t need to try to do any polyphonic pitch recognition. Literally direct note access. Also, signals add nonlinearly, so effecting each string individually has a different sound than doing emulation on the mix.

The tech looks like it will help a lot of people fake being better than they are (especially bending to the right note), at the expense of the beautiful imperfections of great playing, but the potential of using hex pickups in these new ways is fun to think about.

The digital guitar has been a vision for a long time, from working out MIDI output to multichannel output. Gibson has been the name behind many of those efforts. Back in January 2004, Wired ran a glowing portrait of Gibson’s efforts in print:

The 100-Megabit Guitar: Gibson’s maverick CEO wants to shove Ethernet up your ax and rock the music world. [Wired 12.1]

It’s worth reading the whole article; the technical limitations of the Gibson system immediately come to light. Suffice to say, that vision never quite came to fruition; Wired even this year claimed that the project had been killed – at least at Gibson. None other than Adrian Freed, OpenSoundControl and alternative instrument design guru at the University of California Berkeley’s CNMAT research center, led the group – he, his colleagues, and his many students go right on innovating with or without Gibson. Updated: I’m not able to find the reference for that story, which I read in print. See comments for commentary by Adrian Freed, who sees otherwise.

At the time, CEO Henry Juszkiewicz, oddly speaking in the third person, pronounced, “Progress will happen. If Henry Juszkiewicz didn’t build a digital guitar, I can assure you the digital guitar would still happen.” That prediction may prove prescient.

The 2007 video below shows the debut of Gibson’s HD.6x-Pro Digital Les Paul – working with individual strings. I also saw a demo with Gibson, Intel, and Cakewalk that used each string in a surround speaker diffusion. It was a psychedelic effect, if not necessarily the most practical demo, but proof that a technology like this could have many uses.

For their part, here’s how Antares describes their technology. Notice that they aren’t only talking intonation, but other applications, as well.

Incorporating our world-renowned Auto-Tune pitch detection and manipulation along with our proprietary modeling technologies, ATG-6 is an entirely DSP-based suite of functions that offer everything you’ve always wanted from a guitar, along with capabilities you never imagined possible. From flawless intonation to astonishing tonal flexibility to alternate tunings that open up entirely new areas of inspiration and creativity, ATG-6 technology seriously expands the flexibility and range of the electric guitar while letting you continue to play your own way.

… Using our new Solid-Tune™ Intonation system, an ATG-6 equipped guitar constantly monitors the precise pitch of each individual string and makes any corrections necessary to ensure that every note of every chord and riff is always in tune, regardless of variables like finger position or pressure or physical limitations of the instrument. As a result, listening to a guitar with Solid-Tune is a revelation, offering a purity of intonation that has simply never before been possible.

Of course, Solid-Tune is smart enough to know when you want to manipulate pitch, so you can play bends and vibrato exactly as you always do. In fact, Solid-Tune Intonation makes it even easier to bend to the right pitch every time.

Antares ATG-6

Updated: Chris Randall chides me on Twitter (and I agree) for not mentioning Roland, specifically — that’s the reference above in Jesse’s from-the-hips comments to “hex” guitar. Roland has built a whole business around products that track notes played on a guitar, adding polyphonic pitch shifters, open tunings, note-by-note replacement, MIDI output, and even DSP effects processing. The difference in the Roland offering is that Roland has done all this work in a separate processing box you connect to their pick-up; Antares appears to be promising something that’s all-in-one in the guitar. And the analysis Antares is doing may well prove more sophisticated than what we’ve seen in the past in terms of distinguishing, say, a bend from different notes. That could open up additional and radically-new expressive possibilities, even if the underlying fundamental concept is more or less the same.

On the other hand, the other difference with the Roland offering relative to both Gibson’s past attempts and Antares’ upcoming ones: Roland successfully shipped and sold theirs. Until Antares does the same, advantage: Roland. We’ll be watching.

If your music and sound collections seem like outwardly-expanding universes, two new tools promise to bring order by representing media as virtual planets and stars. One works on albums and tracks on the iPad; the other uses computer-aided analysis of loops and samples (not just music) on Windows. One will make your eyeballs pop; one might help you manage gigs of samples for a game design project.

Built in the open-source framework Cinder by an all-star team of media artist-designers (Ben Cerveny, Tom Carden, Jesper Sparre Andersen and Robert Hodgin), Planetary should satisfy space nuts and eye candy lovers. The metaphor is pretty direct: artists are stars, albums are planets around the artists, tracks are moons around the planets, and you can filter “constellations” by letter. That means the actual structure is heavily hierarchical, actually, in the tradition of iTunes (and, before it, its predecessor SoundJam). I’m not sure what happens with, say, compilations. But let’s face it: the real draw is that it’s incredibly beautiful to look at. I’d be just as entertained looking at a visualization of my system folder if it looked this pretty.

For now, Planetary is some fascinating eye candy with at least basic playback capabilities, iPad-only. That brings some good news – Airplay wireless works, and since it makes use of standard media code, even features like Last.fm scrobbles function. It also brings some bad — while Apple added support for libraries to third-party apps, Home Sharing isn’t included, so you’re limited to what’s on your iPad. Playlists aren’t supported, either. But hook this up to a projector or large screen TV with some of your favorite music, and I don’t think you’ll be complaining. And as a free tool, it’s incredible.

Planetary is available now; free for the iPad. As seen on creativeapplications.
http://planetary.bloom.io/
iTunes link

Less pretty, but with greater facilities on the utility side, is the Windows-only Soundtorch. (Thanks to Kristian Gohlke for the tip!) Visually, it offers a similar metaphor: media assets live on a continuous plane. Functionally, though, it’s more algorithmic than hierarchic, using something called the Computer Aided Sound Exploration engine (C.A.S.E.). The set of algorithms, which the creators say were based on evaluation of human listening, performs a sophisticated set of extractions of some 600 features from each sound file.

Rather than limit itself to albums and tracks, C.A.S.E. is tuned for audio files and loops. It’s fast enough that it can plow quickly through gigs of material. So, if you’re on Windows and have amassed an enormous collection of loops, samples, field recordings, sound effects, and the like, Soundtorch will use C.A.S.E. to first map all those relationship, then visualize them. You can use the mouse to produce new collections of assets, map relationships visually, export those relationship to XML, copy sounds to the clipboard, export to WAV, or open them in Windows Explorer. That is, all that eye candy is a genuine interface, not a barrier between you and what you might do (as so often happens with these sorts of experimental interfaces).

In fact, you might argue that, despite outward appearances, Soundtorch is entirely different from Planetary, but they share one common conceptual assumption. Related media “orbit” or attract to common materials. The difference is that Soundtorch is relational. In Soundtorch, if you “magnetize” a file, it – and any similar files – become attracted to attractors called “magnets.”

As is appropriate searching for media, the “torchlight” metaphor shines a light through files. Everything under the light plays back simultaneously, so you don’t have to audition sounds one at a time. (That sounds slightly terrifying to me, but I have to spend more time with it in an actual library.)

The creators describe the magic thusly:

Have you ever listened to a sound and felt that there was a similar one somewhere on your hard disk? And the sound you can’t find would just work so much better right now? Well, Soundtorch also remembers all sounds that you ever listened to. Just select any sound on Soundtorch, and let the system suggest the most similar ones from your whole collection.

In other words, SoundTorch is as much about what you can’t see as what you can – the intelligence to determine similarity behind the scenes. Check out the tech talk in the video above for more information on how “aurally and visually-enhanced audio search” could also apply this technology. More research at:
http://www.accessive-tools.com/

Soundtorch 2.0 entered a free public beta last week. It was developed in Microsoft’s C#-based XNA framework.

Grab the download:
http://soundtorch.com

Finally, if you want to hear the “Optimist” track by Zoe Keating without that voiceover and just enjoy Planetary’s gorgeous visuals, here you go:

From innovation in the visual interface to the intelligence underneath that changes how the computer interprets relationships between files, finally, there’s hope. Music and sound might not forever be trapped in views borrowed from spreadsheets, tables modeled on the needs of accountants 30 years ago.

Nearly as long as we’ve had electronics, musical inventors have tried to imagine new electronic instruments. In the crowded world of new instrument design, the Margaret Guthman Musical Instrument Competition has emerged as a key prize for the best work, with creations battling fiercely for attention.

But in the oddball world of sound and music, how do you judge a winner? As a starting point, organizers this year asked the judges what they personally found important. With an expert panel including synth pioneer Tom Oberheim and reacTable creator Sergi Jorda, those answers are themselves revealing.

As for the competitors themselves, even with eclectic entrants, one theme stands out. Human gesture and performance presence is a strong dimension of the winners. And in perhaps the most promising first-prize winner yet, research begins to crack the code of how to make real gestural analysis work, even allowing everyday objects to become musical instruments.

To help us learn more, Competition founder and Georgia Tech Music Technology director Gil Weinberg grants CDM a window into the philosophy of some of these leading technologists, and introduces us to this year’s winners.

The Winners

First Prize: MO, Interlude Consortium. Everyday objects become novel gestural interfaces.

From the project site:

The MO tangible interfaces are a series modules to capture various gestures, from motion to touch. The central module MO contains motion sensors (3D accelerometers and 3axis gyroscopes) and transmits the data wirelessly. Moreover, two accesorries, i.e. other sensors can be added to both side of MO.

Second Prize: MindBox Media Slot Machine, Humatic Berlin. A vintage slot machine is transformed into a compositional interface.

Personnel:

Christian Graupner , Humatic
….media artist, director, composer

Roberto Zappalà
… performer, choreographer

Norbert Schnell, IRCAM — Centre Pompidou
… interactive music & sound design

Nils Peters, Humatic
…system developer and software artist.

Third Prize: Samchillian Tip Tip Tip Cheeepeeeee, Leon Gruenbaum. It began as an ergonomic computer keyboard, but years of layered work on relative pitch makes it an instrument – a bit like a macro keyboard for composition.

Honorable Mention: Hexenkessel, Jacob Sello. A conventional acoustic timpani is both projection surface and multi-touch input.

From the creator’s description on the video:

The Hexenkessel is a modded 22″ timpani using LLP multitouch technology for control of live-electronics & dmx-light. the realisation of the instrument involves a modified led-projector, webcam and IR-Lasers. the programming is done entirely using max/MSP/Jitter + CCV. The instrument-hack is non-destructive and costs less than 300$.The instrument is intended for the use in multimedial stage performances and innovative concepts of new music.

Pioneering Judges Offer Their Philosophies

A musical instrument design may seem like subjectivity atop more subjectivity, a meeting of the aesthetic of the object with personal musical expression. Judges were asked, therefore, to describe the philosophy they brought to the contest. The reason, explains organizer Weinberg: “To steer it away from general statements – this is the better instrument than this – to make it more personal, about the judge’s opinion and artistic manifesto and instrumental manifesto.”

Tom Oberheim, the man who created the first polyphonic synth product, responded:

The first thing that I look for in a new musical instrument is its musicality. This means where appropriate: does is sound good, is it playable, does it add to the music making language. Then I consider if the device has some sort of universality; in other words, can it be used by a variety of musicians from different backgrounds. Finally, I consider the ease with which the device can be learned.

Sergi Jorda, creator of the reacTable tangible interface:

The ultimate goal for any new instrument could arguably be the potential to create a new kind of music. In that sense, baroque music cannot be imagined without the advances of sixteenth- and seventeenth-century luthiers, rock could not exist without the electric guitar, and jazz or hip-hop, without the redefinitions of the saxophone and the turntable. Yet, this extremely ambitious objective is often beyond the reach of its creator (eighty years separate Adolphe Sax from Coleman Hawkins, and no less than thirty go by between Les Paul and Jimi Hendrix). Being a bit more pragmatic, as a performer, my goal when constructing the instruments I will play is clear. I need instruments that are enjoyable to play and that mutually enhance the experience when playing with other musicians. Thereby allowing me to create or co-create music that will surprise me as much as possible, that will keep revealing little hidden secrets at every new performance. Music not necessarily better, nor worse, than a piece that I could compose in a studio, but music, in essence, that could not have been created in any other possible way. As a ‘professional’ luthier, I need to take some additional considerations into account, but the overall goals do not change: my aim is to create instruments which people can enjoy playing; instruments that will be able to enrich and mature the performers’ experiences in any imaginable way; instruments that allow scope for the performer (particularly in the case of a non-expert user) to be proud of the music created. In order to survive in the extremely demanding instrumental ecosystem, any new instrument should clearly excel in something. It should either be able to do one thing that no other instrument could or, at least it should do it better (whatever this can be and whatever “better” may mean). My last advice would be that when envisaging new instruments one should not only concentrate on the instruments’ sonic capabilities, on their algorithmic power or on the amount of sensors used. One should also be especially careful about the instruments’ conceptual capabilities, and consider how new instruments impose or suggest new ways of thinking to the player, as well as new ways of establishing relationships, new ways of interacting, new ways of organizing time and textures; new ways of playing, in short.

Jason Freeman, a composer, technologist, and Assistant Professor at Georgia Tech:

For me, new musical instruments are significant for their potential to transform our experiences with music. They may enable us to create new acoustic or electronic sounds not previously possible. They may encourage us to think about musical content, structure, and hierarchy in unusual ways. They may suggest new methods of musical collaboration, performance, or education. And they may make musical creativity more accessible to everyone. I am interested in instrument makers who have thought deeply about their work from technical, musical, and design perspectives to create musical instruments that transcend novelty to suggest new paradigms for musical creativity.

A Chat with the Organizer

Now in its third year, the Guthman competition has become a coveted award. As a result, says organizer Weinberg, who is director of the hosting Georgia Tech Center for Music Technology, quality and quantity were up in entrants. And, he says, he feels that entrants have transcended some of the typical designs in the field.

“New interfaces for many [means], let’s think about an object that we didn’t use before, and some kind of gesture, stick on some sensors, make some music … But I think the winners of our competition were outside of this realm, really innovative, completely new approaches for playing music,” he says.

On the prize-winning MO tangible interface:

In a section of the performance, they took a ball – a soccer ball – and did some [musical] gestures with it, threw it … moved it … on the hands, on the floor. Each one of these gestures was recorded with the gesture recognition. And then they actually threw the ball to the audience. The audience members started to throw the ball back and forth. If you threw it in a particular way, it made a particular sound — and everything’s wireless, completely — if you threw it back and forth in a different way, it made a different sound. It was really fun; people threw the ball at each other, threw the ball back at the stage. And all made music that was pretty cool to listen to.

Basically, the instrument becomes an intelligent entity. It can sense similar but different gestures and create something smart and relevant musically.

On the slot machine:

The gesture is mostly visual — the intelligence here is of the human performer. He makes his own gestures, accompanied by sounds. And it allows you to manipulate and change [the sound] — and get some surprises, because it is a slot machine, after all.

You can play, explore it. He was able to very expressively pet and touch and click and manipulate the slot machine to create some very nice — not only musical outcomes, but visual outcomes. In some cases, this guy is lying in the sea and making gestures in the sea. Sometimes he’s hanging stuff on the walls, and making sounds with his mouth. Sometimes it’s basic stuff that you can manipulate in real time, with a pretty unique interface — it’s not a monome, it’s a slot machine. It surprises you.

On the Samchillian:

Some instruments – controllers – have this short or sometimes long learning curve, but once you get to a certain point, you know it, and that’s what it can do. And you cannot get better at it. I think the Samchillian is really an instrument with a learning curve that’s very long, and just like other acoustic instruments, violin, piano, there’s a wide range of [technique]. And this guy was really a virtuoso with this instrument. He was able to play chords, all kinds of arpeggiators.

What I liked about it is it’s an instrument more than a controller. There’s always more to learn about how to become better with it. And I think that’s valuable.

Notably, Weinberg has no illusions about the challenge of making new instruments. It’s no accident that the winners were typically the result of years of development and evolution. “I don’t think any of the great instruments were invented in months,” says Weinberg. “It’s a lot of iteration, a lot of building… only a few are good enough to stick.”

And perhaps the great electronic instrument, while getting nearer, hasn’t yet been created. Weinberg says one example of a new instrument design that doesn’t work particularly well is the legendary Theremin – it’s beautiful in the hands of only a couple of artists, but generally a design that stumps musicians and is hard to play.

Looking at the winners this year, though, there are ideas on which new work can be built, not just impressive one-off instruments but real research into handling pitch and gesture. That, at least, should present a bright future. But with the competition heating up, aspiring engineers may want to get started on those designs now.

Thoughts? Questions about the work? Let us know.

More on the MO tangible interfaces from the IRCAM-based Interlude:
MO Interfaces

That work isn’t yet available for download, but an “augmented score viewer” is.

Onyx Ashanti is insane – in the special, essential way that makes certain brilliant musicians. An experienced busker, having crossed from the US to Berlin, he’s a rare virtuoso of wind instruments and electronic improvisation, the kind of musically-free soul who can just let loose live. But his latest project really crosses into some new territory.

He’s making himself into Tron.

No, really – just having some handheld touch control wasn’t enough, so he’s preparing an open source, wearable rig. He’s hardly the first to attempt this sort of thing, but he has two major advantages: first, he’s already developed the musical idioms and chops he needs, rather than leaving that for some indeterminate time after the thing is built. Second, he has on his team not only himself, but people with experience in prosthetics, plus the co-founder of Ableton. And the work isn’t just a crowdsourced pipe dream: it’s already well on its way.

I could try to describe it all, but this is a project only its mad-scientist, mad-musician creator can really do justice. So I’ll let Onyx take it away.

My Name is Onyx Ashanti. I am a Busker, Author, Beatjazz Artist.

Beatjazz is a term and style of music i came up with back in the late 90s, which described my playing of my wind MIDI controller with beats I had pre-programmed into Fruity Loops [now FL Studio]. That was cool for a while, but as time went on and I got older, It became boring. I wasn’t writing new beats as often as i should have. I had an Ableton phase which gave me a different means of using beats as a sort of hybrid, chopped-up DJ/live set kinda thing, but i got bored of that, as well. It wasn’t until a family tragedy that I realized that it was time to walk that tight-rope, in a sense, to do the weird and the crazy stuff that you convince yourself not to, for the various reasons you give yourself. So beatjazz evolved into a beat-centric form of live music based on live looping, software synth-based sound design, and jazz improvisation. That was three and a half years ago.

It wasn’t easy. It took a year just to get proficient enough to play beats live without a metronome using the wind controller. But after that, Beatjazz evolved in ways i couldn’t have imagined — styles of music that I may have heard once in my life, springing out of one of these extended improvised sessions, which sounds oddly like DJ sets in their continuity.

Over time, I have outstripped the capabilities of my [Yamaha] WX5 wind MIDI controller. It was not designed to do multi-elemental improvisation. There are many things to do and keep track of during a live beatjazz performance, so I have, over the last few years used a wide variety of different secondary controllers, such as the M-Audio Trigger Finger and the Korg microKONTROL, but found them all to be to distracting in performance. A wind midi controller is constructed like a horn. It looks like a clarinet, and as such, it’s hard to play and simultaneously tweak a knob on a secondary controller, because your hands need to be on the horn. And I never liked foot pedals, especially in clubs, because they limit my movements to a very small area — and get drinks spilled on them CONSTANTLY.

Last winter, I started experimenting with using TouchOSC, interfacing with [open source multimedia development environment] Pure Data on my computer, as a gestural controller. In that way, I could simply wave or shake my hand and control many parameters at once, which opened up many new stylistic trajectories that are still very exciting — so much so that I can’t do what I consider to be “my music” without an iPhone running TouchOSC strapped to the back of my hand. This is great and very cool, but isn’t optimal because there are so many gestures I have created and only one accelerometer/GUI, so I set out to design a system that was designed specifically for the presentation of beatjazz. The result is the TRON Beatjazz controller system.

I called it TRON because the system is made up of three main components; a helmet (above) and 2 hand units (top). Each unit is wirelessly connected to the computer. The hand units together are “fingered” the same way one would finger a saxophone or a clarinet, known as the “Boehm” fingering method [see video / below], but the hands do not have to be in a stationary “horn” position the way they are with those instruments. Each hand will have switches for keys, a joystick, an accelerometer, and a color synthesis system based on RGB LEDs to tell the audience what element i am playing (for instance, blue for bass and green for drums,etc). Part of the performance of these units is hand motions very similar to that of a raver using glowsticks (image below), which results in light trails reminiscent of the light effects from the movie TRON.

The purpose of this system is to provide a complete live performance system that incorporates lights, sound, and dance in one cohesive new form. The helmet, which looks like an afro, is made of carbon fiber and will house lip and breath sensors, a wireless microphone system, in-ear monitoring with ambient mics (so I can hear things around me without taking the helmet off), a digital compass for directional processing, an accelerometer, and two very powerful PC fans so I don’t have a heat stroke while wearing it. Why have a carbon fiber helmet? Primarily because a friend who is a professional creator of artificial limbs offered to help me make use of the material, and also because it’s durable and very lightweight. Oh, and I almost forgot — BECAUSE ITS f&%(ING CARBON FIBER! It’s the coolest-looking material on Earth! Ahem.

The brains of the system will reside on the host computer by way of Pure Data [above]. This is for two main reasons. First, as I stated above, I get bored easily. This system will be permanently malleable. Since the controller is just a wireless array of sensors, I can change and adapt them to do many thing I can’t imagine at this time, which leads to the other reason:

This is an open source project. This system would not be possible without open source software and hardware in the form of Pure Data and the many Arduinos that will make up the core components of this system. I am releasing all notes associated with this project once it is completed, as well as detailed notes on the concept and methodology of Beatjazz. By keeping the brains on the computer, by design, the concept should evolve exponentially in multiple directions. People can use these notes and patches as a jump off point for their projects, and it is also a not-so-subtle way of spreading the gospel of open source, of which I am a zealot.

Beatjazz is part of an atomization of sound culture. It is pure, computer-enabled improvisation. I never know what I am going to play onstage, even while I am playing. I combine the vibe in the room with what I am feeling at the moment, limited only by my skills and my sound set, and construct/deconstruct a narrative that provides a singular soundtrack for that moment. It may come out as house or be-bop or latin jazz — that’s the point, I have no idea, and it is very exciting.

I am doing this project now because, for lack of a better way of putting it, it’s just time! I’ve “practiced” for 20 years. This is the stuff i grew up dreaming about. When you reach a point in your life where you have the skills and the determination and the “people resources” to make something actually happen, you have to act! I have direct access to some of the most amazing people with the skills that make this a much more viable project.

Uli Maier is the Prosthetist (video above) that is helping me cast the molds we will use for the helmet and hand units. Chris “Loganic” Logan draws the ideas out in a form that conveys exactly what i see in my head, Tomas Henriques is wind synthesis legend, having already created the Meta-EWI [customization of the Akai wind controller] and the award-winning Double-Slide controller and Native Instruments co-founder Stephan Schmitt is offering to help me design the custom looping system that will form the backbone of the system. (The controller is open source, not the synths and looping system — yet.) I’d be mad not to go for it with all of these stars aligned at one time! These are just a few of the people that make the project real.

This is the most exciting thing I’ve ever been part o, as an artist, but its also, by far, the most expensive, so i am “crowdfunding” it through indiegogo.com, which is another amazing aspect of the modern Internet — the ability to bring people directly into this project that may be really interested in it. I have a created a wide array of “perks”, i.e. cool items in exchange for contributions to this campaign. From an exclusive EP of the twisted beatjazz that I will create with the new system ($5-available in late May), at onyxashanti.bandcamp.com, to a hand-bound copy of my book entitled “The 21st Century Musicians Guide to Busking” ($50, completion by late March). Options go all the way up to various versions of the controller system itself, in wired and wireless varieties, including the $5000 “Ultimate Package” in which you get the same carbon fiber controller i am making for myself (with a different helmet). I will it deliver to you personally, spend two days teaching you how it all works, and then you and I will give a concert for your friends and family! See: www.indiegogo.com/tronbeatjazz.

I have been told that saying that I want to create “the most amazing live music performance system ever” is a bit “bombastic”. But I wouldn’t say it if it weren’t the intended goal and if it didn’t think it were possible.

www.beatjazz.blogspot.com
www.indiegogo.com/tronbeatjazz

Digital models and acoustic instruments have traditionally been studies in contrast. And instrument making has by definition been a craft and an art. But what if making an acoustic instrument was a matter of hitting “print”?

That’s the question asked by MIT Media Lab researcher Amit Zoran. Using the Objet Geometries Connex500 3D printer, one capable of on-the-fly use of multiple materials, he made a flute in 15 hours. The results are surprisingly good for a first attempt. The instrument is playable, but Amit plans additional iteration and improvement. (Be sure to watch through the video for some feedback on the details from flutist Seth Hunter.)

Initially, it’s hard to hear about a 3D printer spitting out a flute from a digital model and not worry a bit about what happens to craft and skill. But that may miss the point. For one, the more access we have to 3D printers, the more we may appreciate the subtleties of human fabrication that these printers can’t reproduce – in case decades of (often inferior) mass production haven’t done that for you already.

Moreover, the 3D printer could represent new potential for instrumental research. Acoustic instrument design hasn’t produced a popular instrument, arguably, in over a century. Part of the problem is that it’s too difficult to prototype ideas. Being able to rapidly prototype a lot of variations inexpensively could mean wild, new instruments (see the fanciful multi-pipes trumpet Amit proposes), new designs that can’t be fabricated by hand, as well as new revelations about historical designs. (Imagine being able to produce a dozen variations of a prehistoric flute, for instance, and be able to try them out with a musician.) Those prototypes might, in turn, ultimately be fabricated by a skilled artisan after perfecting the basics of the design.

I’ve sent some questions to Amit to hear more about his research, but let us know if you have questions for him or thoughts about the project. In the meantime, some great coverage from NPR’s Renee Montagne and Engadget’s Sean Hollister, who each beat us to the story:

3D printed concert flute rapidly prototypes sound (video) [Engadget]

3D Printer Produces Working Flute [NPR Morning Edition / Audio]

You can also visit Amit’s own site, which includes designs for everything from sci-fi motorbikes to re-imagined acoustic guitars:

Amit Zoran homepage at MIT

For a long time, technologists have described a world of in which computing experiences naturally incorporate touch and gesture. The question is, how do we bridge the intuitive desire for those interactions and the actual technologies that get us there?

Few activities test the expressive potential of interaction quite like music. It’s in our cultural DNA; musical activity may even predate written language. So it’s fitting that the story of touch in computing and digital music would be intertwined, as they are with touch pioneer JazzMutant. Years before well-known Apple products, the Lemur, prototyped in 2003 and shown as a musical multi-touch screen, suggested the importance of fusing display and touch, and of tracking more than a finger or two at a time.

The history, and products like Apple’s iPad and iPhone, you may know well, though. The question on everyone’s mind now is, what’s next? (And for some impatient futurists, the question may even be, what’s taking so long?)

To begin to answer that question, I turned to Guillaume Largillier, original co-founder and CEO of JazzMutant, now Stantum Technologies. There aren’t many people on the planet closer to where touch has been and where it might be going. Even as the Lemur gets new features like integration with popular music production and performance tool Ableton Live, Stantum is working to bring the same enabling technologies to other device makers. And even though this is “Create Digital Music,” it’s telling that that technology is showing potential in everything from phones to aviation, not just DJing. Musicians have had a role in technological history before, from Leon Theremin’s work to Max Mathews and computer synthesis. It may be musicians who invent the future, again. This time, the trick is who delivers that future to the hardware makers who can popularize it.

http://jazzmutant.com/
http://jazzmutant.com/behindthelemur.php
http://www.stantum.com/en/
http://www.stantum.com/en/offer/technology-ip

To accompany the story, we also have an exclusive look inside Stantum’s labs, all the way back to the original 2003 prototype of the Lemur.

On Designing for Touch, and the Music Tech Industry

Peter: I remember when I first talked to Darwin Grosse about Lemur, when it was being distributed by Cycling ’74. Darwin just kept saying, “You know, I just think Star Trek: The Next Generation.” (That’s my recollection, Darwin; I hope I’m not misquoting you.) I tended to agree. It’s a cliche, perhaps, but this was clearly hardware that brought into our century part of an imagined vision of a much further-off future (the 24th Century). Was that a conscious influence? In an industry that has sometimes been aggressively traditional, is there a way to channel ideas from something as far out as science fiction?

Guillaume: Before answering your question, allow me to challenge your statement about the computer music industry. I think “ill nostalgic” would describe this industry much better than “aggresively traditional.” Most music software companies have kept being innovative over the last decade, but their creativity has been a slave to this nostalgic obsession. Emulating an analog channel strip, a tube amplifier, or a vintage synth is far from a trivial job. It actually requires as much engineering time and resources as developing a disruptive product such as Ableton Live or Max/MSP/Jitter! On the hardware side, the innovation killer is the price pressure. Despite a common misconception, the computer music industry is not and will never be a mass market. Companies such as M-Audio [Avid], Behringer, or Native Instruments may look like giants compared to JazzMutant, but they are nano-particles compared to large consumer electronic brands such as HP or Nokia. The volume and the gross margins are too small to amortize ambitious research and development plans. When we launched the Lemur in 2005, a lot of people predicted, and somewhat hoped, that Behringer would release a similar device at $200 within the next eighteen months. Five years later, the first serious competitor of the Lemur is about to land – Apple’s iPad – and its entry level price is $500.

Back to the USS Enterprise, whether we want it or not, this parentship is likely to follow the Lemur forever. This is kind of ironic insofar as I’ve never been acquainted with science-fiction culture. I don’t even remember having ever watched a full episode of Star Trek. That being said, I acknowledge that this association has settled spontaneously and durably in people’s mind. Does this association come from the product concept itself? I don’t think so. In my opinion, it comes first and foremost from the fluorescent graphic design of the UI objects, not from the tactile technology.

So, the real question would rather be: “Why did we design the graphic interface this way?” First, we wanted to stand clear of those boring pseudo-vintage brushed-aluminium graphic skins – the cutaneous symptom of the nostalgic flu! Moreover, we anticipated that converting users to virtual controllers would be a difficult task and that trying to mimic the appearance of real-life objects would generate frustration; hence, impeding the adoption of the product.

Having said that, the main purpose of this flashy design was pragmatic and ergonomic. The Lemur is ontologically a live controller, though it might be used in other contexts. This requires that the interface must be visible wherever and whenever a user might be performing, from night clubs to outdoor venues. This is particularly tricky with a touch screen laid horizontally, because the display backlight cannot compete with the specular reflection of sunlight. Human-factor sciences taught us that contrast perception prevails over brightness perception. Hence, highly contrasted graphics- ie, flashy objects on dark background – is the most efficient way to ensure a consistent readability. This is something the aerospace industry has understood for decades. So, if there was one conscious influence behind the Lemur, it would be the Boeing 747 dashboard, not the USS Enterprise.

“If there was one conscious influence behind the Lemur, it would be the Boeing 747 dashboard, not the USS Enterprise.”

I know for me, the appeal of the science fiction aspect was more conceptual than superficial, the idea of the ubiquitous touch interface. But I agree, having experimented with this, that the high contrast, light foreground, dark background formula is really an essential solution. I’m seeing some interfaces on a white background that look aeshtetically lovely, but that I can’t imagine using onstage. I’d at least want a switch for dark environments, when you’re not at your desk.

That reminds me a funny episode of JazzMutant’s story. As early as February 2004, we started showing early prototypes of the Lemur to our friends at [Paris sound research center] IRCAM. At this time, the graphic skin was based on a palette of blue shades, with a few touches of warm yellow for emphasizing elements that needed to stand out, such as text, levels, etc. One day in July of 2004, about one year before the commercial launch of the product, we brought them a new prototype, featuring a brand new touch panel along with the final graphic design. Their only reaction was, “wow, this display is much brighter!” They did not even comment on the tremendeous improvement to the touch panel! That being said, there are other approaches to improve the psychological perception of readibility. I sometimes regret that other developpers are reluctant to dig into them, and mimic the “Lemur style” instead.

Talking about drawing on screen, did you know that Iannis Xenakis’ Upic project has been my main source of inspiration – and also my main motivation to step from music making to technology creation ?

I didn’t know that, but it makes a lot of sense. [UPIC is a tablet-based, visual composition system developed by ground-breaking experimental composer Xenakis. It is now decades old but continues to evolve in new incarnations.]

Below: DJ Mike Relm demonstrates the Lemur for G4 Tech TV. Yes, this is the video to show all your friends who aren’t regular CDM readers and have no idea what the heck this is all about.

Lessons of Lemur

Let’s talk about the place of Lemur’s technology in the current landscape. How does it hold up in 2010? I know a lot of people do get hung up on the price, but can you talk about how it differs from other options out there, or what the source of the cost is?

Once again, the music market being a small niche, it’s hardly possible to be both innovative and affordable at once. In addition, the Lemur is still manufactured in France with components imported from various locations around the globe – not to mention that the US dollar’s agony doesn’t help [when exporting] the manufactured product! Lastly, a large part of the product assembly is still handcrafted. For all the reasons above, the product is far from cost-optimized. I cannot disclose further our plans now, but we are working hard to address most – if not all – of these issues.

Have there been uses of the Lemur in performance and creation that surprised you, or went beyond what you imagined?

Oddely enough, and despite of what I said before, the most surprising uses of the Lemur are sometimes the most conservative ones! As an example, for Björk Volta tour, Damian Taylor and LFO made the most archaic interface layouts one could imagine — a fistful of colored labelled pads and eventually a pair of faders –- nothing more. Their brilliant idea was to create one unique interface for each song. This way, at each moment of the gig, they just had at their disposal the few commands they did actually need. The other big surprise came from video performers. Whereas most musicians are reluctant to use the advanced features of the lemur during their live performances – such as the objects’ physics – video artists do not hesitate to play the Lemur as an instrument, rather than a remote control. For instance, I warmly recommand you to visit Ali Momeni’s website. Of course, it would be unfair to forget all the advanced users who have developed inspiring and unique instruments, but this is less surprising, since the Lemur was designed specifically for that purpose.

OSC is a technology that many of us have advocated, but there’s also, admittedly, a big gap between where we believe it could be and where it is, especially in regards to the lack of mainstream music tech adoption. That said, what would an ideal implementation of OSC look like? What could the protocol do to be better? And what might you imagine could be a tipping point in adoption?

Indeed, it’s fair to say that OSC failed to become the industry standard we hoped it will be! I can see a few reasons for that. First, there is an obvious chicken-and-egg issue, as with any protocol. At JazzMutant, we’ve done our best to evangelize OSC in the industry for about 5 years now, without success. Why should a software company implement OSC if there is no hardware to support it, apart from a $2k product? Why should a hardware manufacturer develop an OSC-compatible controller if there are no mainstream applications to support it? Finally, there are also some intrinsic technical reasons that prevent OSC from becoming a standard anytime soon. In order to overcome them, we started developing a new protocol a few years ago called “Minuit” (“Midnight” in French), as a successor to OSC and MIDI (“Noon” in French). We were discouraged from pursuing this project after assessing the amount of human resources its evangelization would require.

The Big Picture, Stantum, and the Future

We’re looking at an explosion of interest in multi-touch display surfaces in the consumer space. Are any of these, in your view, promising for music? Are there ways in which some of these technologies are deficient for musical performance applications?

The responsiveness of a touch system is the most under-estimated parameter, even though it tremendously influences the perceived usability, transparency and trustworthiness of an input device. This is why a vast majority of Multi-touch systems available fails to meet music makers’ expectations.

Absolutely — you mean responsiveness in terms of latency, accuracy, precision in tracking multiple points, or (I presume) all of the above?

I was pointing out the latency more specifically – even though the perceived responsiveness is a complexe imbrication of all these parameters.

Can you talk about Stantum’s role in the evolution of multi-touch? What can we expect to see in the future?

We envisioned the real potential of the technology we invented long before the iPhone announcement, though we could not imagine that Steve Jobs’ crew would accelerate the market demand [to the extent they did]. We started investigating how we could bring our technology to OEMs in parallel to our computer music activity as early as 2005. We finally made this step in 2007. The role of Stantum in this ecosystem is quite singular. However pretentious it may sound to you, Stantum is still the only company beside Apple to have developped a real multi-touch product, top-down, including all the software and hardware technology bricks. So, despite the small size of our company, we are better placed than any other player in this field to understand the complexe imbrication of software and hardware. You might ask, “Aren’t all these Windows 7 convertible notebooks real multi-touch products?” In my opinion, they are not, insofar as the only multi-touch services these devices offer so far are rotating videos or ten-finger painting. I do not want to offend anyone, but watching videos is much more pleasant fullscreen and if Neanderthal people gave up painting with ten fingers 45,000 years ago, there might be a good reason. At JazzMutant/Stantum, we’ve always considered the multi-touch technology as a milestone, not the final destination. With what we’ve been incubating in our labs for a few months, we expect to reach the next big milestone quite soon.

Do you mean that these PC vendors are missing the actual application of the multi-touch technology in the software they ship with these devices? Certainly, no argument there — the demos, the marketing, the demo apps outside of Apple have just looked horrendous and awful to me. But surely there are developers out there who want to do better? Hasn’t what’s held them back simply the lack of available hardware?

I do agree with you. Unfortunately, that leads to a chicken-and-egg situation; insofar as developing a meaningful, multi-touch-capable application requires a preliminary awareness of the objective capabilities and limitations of a given hardware solution. On the other side, a vast majority of multi-touch panel providers doesn’t look willing to raise the bar until the market identifies a “killer app” requiring full multi-touch capabilities with zero performance tradeoff. Hopefully, the iPad will contribute to reschuffle the cards. Unfortunately, Apple decided to stand clear of handwriting capability – which, I believe, is a huge limitation for creative and productive applications.

“I do not want to offend anyone, but … if Neanderthal people gave up painting with ten fingers 45,000 years ago, there might be a good reason.”

Let’s assume Stantum is successful in popularizing the technology. At some point, will the Lemur be obsolete – and could that perhaps even be a good goal?

The Lemur as it is today is likely to become obsolete at some point – the pet is more than 5 years old in an industry that usually sends hardware products to retirement manu militari at 18 month old! Having said that, there is much more to develop on the hardware side than what we have done in the past. If we succeed in what we are working on today, I believe the Lemur will keep playing in its own category for quite a long while.

Now, that said, how do Stantum’s efforts to engage the larger electronics industry impact these issues of scale and cost?

We understood as early as 2005 that there was only one path to spread this technology – and the underlying vision of how computerized equipments should work – out of the small niche of professional musicians and Max/MSP users. Then we did what we had to do : we licensed the technology to tier-one semi-conductor companies such as ST Microelectronics to embed our multi-touch know-how into dedicated chips. We also teamed up with some of the largest and most trusted touch panel makers to bring our solution onto the consumer market place. The whole supply chain is now in place and you’re likely to see a few Stantum-based multi-touch tablets shipping in the coming months. Will these products match musicians’ expectation ? That’s too early to risk an answer at this stage, since we have no control over what OEMs will make out of our technology. And as you know, a good user experience does not only depend on the quality of the touch system – it’s also a matter of CPU and OS choice, hardware optimization, not to mention the software application running on top of it. That’s why we believe there’s still some room for a dedicated hardware that takes in consideration the very specific needs of electronic musicians and visual artists. In a not-too-far future, we expect the hard work we have done with our partners will have a positive impact on the cost structure of our music products.

Stantum’s Latest Technology, and What it Means

Guillaume is a bit limited in what he can say about his future plans, but that leaves me free to do a bit of (informed) speculation. This is largely my own analysis, so it’s my message, not necessarily Stantum’s.

First, unless it isn’t already clear, JazzMutant is Stantum. Stantum is JazzMutant. Stantum is now the official name of the company, and JazzMutant is just the brand by which their technology caters to musicians. It says something about the company’s lineage – all the founders have a background in electronic music – that they have in the past, continue now, and plan in the future to keep a strong connection to musicians. That’s meant that the rigorous demands of live music have informed their touch technology and made it a better product.

The idea that Apple’s iPad would drive JazzMutant out of business, therefore, is the opposite of correct. JazzMutant is Stantum. Stantum is in the business of licensing its specialization to OEMs. The Lemur shows just how potent that specialization is, in a way that literally gets rooms full of people dancing and gaping at projections. Apple’s technology is available only to Apple. With Microsoft, Google, phone vendors, and PC vendors all getting into the touch business, that means Stantum just became very big news – even if that’s something musicians and VJs figured out years earlier.

Part of the challenge of multi-touch development is that you have to get a lot of pieces working together. You need the physical surface of the controller, the sensors built into that surface, and the firmware that interprets the sensors all to work in tandem. Apple does it, and does the OS and applications, too. 3M is working on a product for OEMs, also working with multiple touch points. But the other big source right now is Stantum.

It’s also significant that Stantum’s technologies are heavily patented (a fact that they advertise on their site). While I’m no big fan of patents, unlike Apple, Stantum is licensing their technology into the marketplace. Given the need to have a patent portfolio just to protect your work, Stantum’s patents give it effectively the right to play ball. By licensing their technology to the manufacturers big enough to make this stuff on a grand scale, Stantum’s OEM program could provide ready access to touch for software developers beyond just the iPad platform. Even if you’re a huge iPad fan, that means greater accessibility in the market, and more than one vendor to provide that access. I’m a great advocate for DIY, but making displays isn’t yet a garage operation. (Yes, I know people building their own multi-touch tables, but they don’t make their own cameras or projectors.)

Stantum’s technology itself is also unique. Their sensing approach supports pen input and even handwriting recognition, features Apple leaves out. For many of the world’s languages, handwriting recognition is a “killer app,” which could further drive touch adoption. For the rest of us, until we evolve smaller fingers, the ability to use a pen can mean amplified accuracy for painting and writing, and yes, even pen-driven music applications. (Somewhere in the great beyond, Xenakis smiles.)

This is not an advertisement for Stantum, either – the list of companies anywhere close to being able to provide this functionality is short enough to count on your (ahem) fingertips.

So, okay, you buy into the concept – when can you get it? (After all, even the Lemur doesn’t quite count. It isn’t set up for pen input, even if its sensing method could work. And the Lemur is a controller, not a computer.)

Right now, Stantum’s technology is available in a series of multi-touch demonstration kits, including one with the guts of a Dell netbook inside:

http://www.stantum.com/en/offer/evaluate

In other words, we’re waiting for someone to ship a product that incorporates their technology. Windows 7 already includes multi-touch APIs out of the box in all but its Starter edition, so the Windows platform is a major candidate. Windows, while proprietary, has none of the developer, language, software, or hardware restrictions that the iPad platform does, so if your application doesn’t fit the iPad model or needs pen input, Windows’ stock just rose. Free software is possible too. Linux already supports the Stantum Slate PC and a number of other digitizers, support that will be baked into the kernels shipping in this year’s major Linux distros. We’re not just talking drivers, either: the whole Linux community is working on everything from libraries for environments like Java to support in the windowing system to touch-centric distros. (More on the Linux situation later this week.) Google’s Android has a multitouch API, too. I’ve used it, and got frustrated quickly not because of the OS, but because the hardware on current phone handsets just doesn’t work well with more than one finger. That could change if Stantum’s tech starts to appear in licensee products; Android as a touch OS could take off.

For specifics on the Windows 7 aspect (old news, from way back in November – but of course, everyone is taking a second look because of the iPad phenomenon):
2009-11-03 Windows 7 Certification

Right now, the one thing Stantum doesn’t have a lot of – aside from OEMs shipping their tech – is competition. Most of the other touch competitors either can’t accurately track fingers in close proximity, or limit tracking to two fingers, or lose tracking fidelity around the edges of the screen, or can’t handle pens, or some combination.

You need musicians, creative artists, and gamers to tell you this, because the mainstream computer market thinks multi-touch has something to do with pinch-zooming their photos. If that were all you could do with multi-touch, this would indeed be an over-hyped technology. But the responsiveness of the Lemur and the demonstration technology from Stantum is something that can be powerful and expressive.

Apple has already brilliantly demonstrated what happens when scale, creativity, and technical competence meet. Now the question is, who else will be able to put this formula together, thus making other options available to developers? Stantum has the competence, and the connection to creative artists and music specifically. If OEMs start to sign on with Stantum’s tech and build useful hardware, we could see both off-the-shelf machines – and cheaper JazzMutant-branded products – for musicians. Indeed, with this larger Stantum perspective, whatever happens with OEMs could in turn be good for JazzMutant-specific, music-specific customers, too. Even with competition from the likes of 3M, the technology is so specific to certain hardware devices, and the emerging markets so large, it’s hard to imagine Stantum not having a big role.

What might surprise people in the larger tech world is how important music has been – and will continue to be – to the big picture.

When it all comes together, the days of computer musicians, DJs, and visualists standing behind screens, able only to stare blankly into them but unable to manipulate what they see directly, could become a relic of the past.