Compare the complex model of what a computer can use to control sound and musical pattern in real-time to the visualization. You see knobs, you see faders that resemble mixers, you see grids, you see – bizarrely – representations of old piano rolls. The accumulated ephemera of old hardware, while useful, can be quickly overwhelmed by a complex musical creation, or visually can fail to show the musical ideas that form a larger piece. You can employ notation, derived originally from instructions for plainsong chant and scrawled for individual musicians – and quickly discover how inadequate it is for the language of sound shaping in the computer.

Or, you can enter a wild, three-dimensional world of exploded geometries, navigated with hand gestures.

Welcome to the sci fi-made-real universe of Portland-based Christian Bannister’s subcycle. Combining sophisticated, beautiful visualizations, elegant mode shifts that move from timbre to musical pattern, and two-dimensional and three-dimensional interactions, it’s a complete visualization and interface for live re-composition. A hand gesture can step from one musical section to another, or copy a pattern. Some familiar idioms are here: the grid of notes, a la piano roll, and the light-up array of buttons of the monome. But other ideas are exploded into spatial geometry, so that you can fly through a sound or make a sweeping rectangle or circle represent a filter.

Ingredients, coupling free and open source software with familiar, musician-friendly tools:

• Two projectors
• A monome
• Processing, the elegant and artist-savvy free software for visual code
• Ableton Live and Cycling ’74′s Max for Live, acting as the interactive glue with the sound world
• Drumaxx, Image-Line’s tasty physical-modeled drum synth
• Native Instruments Battery, the sampled drum engine
• Eclipse, the free IDE, for Java coding in this case
• Community Core Vision and reacTIVision (based on our previous info, at least), free and open source community-based projects for making the interfaces you see in movies happen in real life.

Another terrific video, which gets into generating a pattern:

Now, I could say more, but perhaps it’s best to watch the videos. Normally, when you see a demo video with 10 or 11 minutes on the timeline, you might tune out. Here, I predict you’ll be too busy trying to get your jaw off the floor to skip ahead in the timeline.

At the same time, to me this kind of visualization of music opens a very, very wide door to new audiovisual exploration. Christian’s eye-popping work is the result of countless decisions – which visualization to use, which sound to use, which interaction to devise, which combination of interfaces, of instruments – and, most importantly, what kind of music. Any one of those decisions represents a branch that could lead elsewhere. If I’m right – and I dearly hope I am – we’re seeing the first future echoes of a vast, expanding audiovisual universe yet unseen.

Previously:
Subcycle: Multitouch Sound Crunching with Gestures, 3D Waveforms

And lots more info on the blog for the project:
http://www.subcycle.org/

The history of music and the history of music notation are closely intertwined. Now, digital languages for communicating musical ideas between devices, users, and software, and storing and reproducing those ideas, take on the role notation alone once did. Notation has always been more than just a way of telling musicians what to do. (Any composer will quickly tell you as much.) Notation is a model by which we think about music, one so ingrained that even people who can’t read music are impacted by the way scores shape musical practice.

All of this creates a special challenge. Musical notational systems had traditionally evolved over centuries. Now, we face the daunting question of how to build that language overnight.

This question has been a topic I’ve visited in a couple of talks, first here in New York at in/out fest last December, then most recently for a more general audience at RSVP, a new conversation series in Hamburg, Germany hosted by the multi-disciplinary design studio Precious Forever. (See photo at top, by which we can prove that the event happened. Check out more on the event and how the Precious gang hope this will inspire new interchange of ideas in Hamburg – something perhaps to bring to your town.)

What I’ve learned in talking to people at those events is, music notation matters. It’s more relevant to broad audiences than even those audiences might instinctively think. The most common lingua franca we have for digital music storage, MIDI, is woefully inadequate.

But perhaps most importantly: replacing MIDI’s primitive note message is far from easy. The more you try to “fix” MIDI, the more you appreciate its relative simplicity. And engineering new solutions could take re-examining assumptions Western music notation has made for centuries.

Musical notation and culture

Explaining the importance of notation to expert musicians is easy. But to convey its importance to lay people, you need look no further than the game interface developed by Harmonix for the hit titles Guitar Hero and Rock Band (and in turn descended from a similar interface paradigm used in the Japan-only Konami GuitarFreaks). These games demonstrate that, even among non-musician gamers, certain received wisdoms from Western notation endure. (In fairness, many of the designers of music games have a fair bit of musical experience, but the fact that their work is received by audiences in the way it is nonetheless speaks volumes.)

The Guitar Hero interface actually is a Western musical score, rotated 90 degrees to make it easier to see how the events on-screen are matched to game play input. (For visual effect, the “track” is also rotated away from the screen, so that events further in the future recede into the background – a bit of visual flair that helped differentiate Harmonix from flatter-looking Japanese games.)

Whatever the rotation, the assumptions of the game screen itself are rooted in notation. Pitch is displayed along lines and spaces, just as on a score. Rhythm is displayed along a metrical grid, which reads as a linear track. Not coincidentally, I believe, when Harmonix has deviated from this formula, their titles have tended to be less successful. More sophisticated interactions in titles like Amplitude and Frequency (and the iPod game Phase) were big hits among gamers, but less so among the general public, perhaps in part because they require a more abstract relationship to the music.

Games are just one example, of course. Musical scores reflect basic cultural expectations, and in turn shape the music that people in that culture produce. As with most Western languages, text flows from left to right and top to bottom. Ask people to describe pitch in any culture that uses this notational system, and they’ll use the notions of “up” and “down,” “higher” and “lower” – even though these metaphors are meaningless in terms of sound. (Indonesian culture, for instance, gets it more physically correct, by describing what we call higher pitches as “smaller” and deeper pitches as “larger,” as they are in gongs.) And music in Western cultures are also deeply rooted on a grid, on 4/4 time and equal subdivisions. It wasn’t always so: even in the West, prior to the advent of notation of these meters, metrical structures flowed more freely.

It’s little surprise, then, that some of the biggest successes in electronic musical instruments have adopted the same conventions. From the Moog sequencer to the Page R editor on the Fairlight CMI sampler to the array of buttons on Roland’s grooveboxes, rhythmic sequencers that follow the grids devised in Western music notation are often the most popular. Even if the paradigm of the interface is one degree removed from the notation, the assumptions of how rhythms are divided – and thus the kinds of patterns you produce – remain.

Nowhere is this more true than in MIDI. MIDI is itself a kind of notational system, around which nearly all interfaces in software and hardware have been based over the past two and a half decades since its introduction.

MIDI, keyboards, and piano rolls: An incomplete “standard”

The first thing to understand about MIDI is that it began life as a keyboard technology. A complete history of MIDI should wait for another day, but even as its early history is told by the MIDI Manufacturing Association, it’s a technology for connecting keyboard-based synthesizers, not a solution to the broader question of how to represent music in general.

Many of the tradeoffs in MIDI, though, were made long before the 1980s or the invention of digital technology. When the 19th Century creators of the player piano needed not only standardization but reproduce-ability – before the advent of recording, the power to recreate entire musical performances – they turned to the piano as a way of modeling musical events. Indeed, the first player pianos quite literally reproduced the process of playing a piano, using wooden, mechanical fingers to strike notes on the keys just as a human would, before that mechanism was replaced with the internal players familiar to us today. What these inventors found in the piano was an instrument that, in the name of accessibility, aligned pitch to a simple grid.

The piano is a beautiful instrument, but its great innovation – the grid of its black and white keys – is also its greatest shortcoming. That grid is an imperfect model even of Western musical pitches, let alone other cultural systems. The 12-tone equal-tempered tuning used on modern pianos makes tuning multiple keys easier, but only by way of compromises. Even a modern violinist or singer may differentiate between the inflection of a G flat and an F sharp, based on context, but to the piano, these pitches are the same. And tuning is only the beginning. Piano notes begin with a note being “switched” on and end with it being “switched” off – no bending or other events within that pitch as on most other instruments.

It’s little wonder, given MIDI’s origins as a protocol for communicating amongst keyboards, that the editing view most common in music software is the piano roll, labeled as such. The piano roll is the perfect paradigm for sequencing events played on a keyboard, but that doesn’t mean it’s the best language for describing all music. And the obligation of a digital protocol is actually greater than that of musical notation, because there’s no human being at the other end to fill in missing expression and context.

Consider what’s missing in MIDI:

• Pitch reference: By convention, MIDI note 60 is C4. However, musical practice internationally lacks a consistent standard for what the tuning of C4 is, and any number of variables can interfere, from independent tuning tables to the use of the pitch bend to the activation of an octave transpose key.
• Pitch meaning: MIDI note values use an arbitrary pitch range from 0 to 127, a hypothetical 128-key piano, which itself makes no sense. 4? 8? 15? 16? 23? 42? The numbers themselves don’t mean anything.
• Pitch resolution: Because of the 0-127 resolution constraints, to get notes in between the pitches, you need a series of separate messages like pitch bend, giving you two values with only an incidental relationship to one another. Since pitch range is kept in yet another message, the results are confusing and un-musical, far more complex than they need to be. (Why wouldn’t 60.5 be a half-tone higher than 60?)
• Real expression: Events between note on and note off are represented independently as control change values. But that causes problems, because it means there’s no standard way to represent something as simple as a musical glissando. On a synth, making an expression (like twisting a knob or turning a wheel) separate from a note (pressing a key) makes sense. But that doesn’t make musical sense, and it doesn’t match most non-keyboard instruments. Only aftertouch is currently available, and that again assumes a keyboard and doesn’t expose pitch relationships created by adding the data.
• Musical representations of tuning and mode: The MIDI Tuning extensions require that you dump tuning information in fairly unstructured System Exclusive binary dumps. The standard itself is in some flux, and at best, its reliance on byte messages means that it’s not something a human being can read. And it still must be aligned with 128 otherwise arbitrary values. It’ll work, but it only makes sense on keyboards, and even then, it’s not terribly musical. Looking at number 42 in your sequencer, you’d have no idea of the tuning behind it, or the position in a mode – something any rational musical notational system would make clear.

Ironically, it was this very set of constraints that early innovators on the Buchla and Moog synthesizers hoped to escape. They were fully aware that the very genius of the keyboard was restricting musical invention. Analog control voltage, the basic means of interconnecting equipment prior to digital tech, was more open ended than MIDI, which replaced it. But that’s not to say it was better. Standardization is an aid in communication, as is the ability to describe messages. The question is, how can you do both? How can you be open ended and descriptive at the same time?

How do you build a new system?

Deconstructing is easy; constructing is hard. We certainly have the ability to send more open-ended messages and higher-resolution data; that’s not a problem. (Even by the early 80s when MIDI was introduced, its tiny messages and slow transmission speeds were conservative.) We also have OpenSoundControl (OSC), which has some traction and popularity, including near-viral use on mobile devices and universal support in live visual applications. It’s telling that that protocol is itself not really an independent protocol in the sense that MIDI is, but built on existing standards like TCP/IP and UDP. 2010 is, after all, not 1984.

The hold-up, I think, is simply the lack of a solid proposal for how to handle musical notes. And there are plenty of distractions. It’s tempting to throw out the simplicity of MIDI’s note on and note off schema, but it’s partly necessary: with a live input, you won’t know the duration of a pressed key until that key is released. It’s equally tempting to cling to Western musical pitches, even though those pitches themselves lack solid standardization and don’t encompass musical practices in the rest of the world. (12-tone equal temperament is a recent invention even in the Western world, and one that doesn’t encompass all of our musical practice. World tunings should best be described not by majority, but plurality, anyway – have a look at the current demographics of Planet Earth.)

One solution is simply to express musical events by frequency. That’s not a bad lowest common denominator, or a way to set the frequency of an oscillator. As a musical representation, though, it’s inadequate. It’s simply not how we think musically. The numbers are also unpleasant, because we perceive pitch roughly logarithmically. Pop quiz:

Can you do logarithms in your head? Yes or no?

Can you count?

MIDI gets it half right by using numbers, but then it’s hard to see octave equivalence, another essential concept for perceiving pitch. MIDI note 72 is probably equivalent to MIDI note 60… assuming 12 steps per octave. Or it might not be.

If you need a common denominator that covers a variety of musical traditions, mode (or more loosely, pitch collection) and register aren’t a bad place to start. I don’t think a system needs to be terribly complex. It could simply be more descriptive than MIDI is – while learning from the things MIDI does effectively.

Consider a new kind of musical object, described over any protocol you choose. It would ideally contain:

• Mode/pitch collection: As with MIDI and the MIDI tuning tables, tuning would need to be defined independently, but it can be done in a musical, human-readable way. It then becomes possible even to define modes that have different inflections based on context, as with pitches that are slightly different in ascending and descending gestures (common in many musical systems).
• Relative degree: a notation like “1 1 2 3 5 6″ can work in any musical language. You simply need to know the active mode or pitch collection.
• Register: Instead of conflating register and scale degree, you could simply define an octave register and starting frequency. This retains modal identities and octave equivalence, and makes relative transposition easy to understand. (A “transposition” message could be defined as an actual message, which is more musically meaningful.)
• Standardized inflections, connected to pitch: Pitch bends and glissandi should be relative to a specific note, because notes can have pitches that bend around their relative scale degree. (Think of a singer bending just below a note and into the actual pitch. These aren’t independent events.) A trombonist would never have invented MIDI notes. They would likely have immediately turned to the question of how to universally describe bending between notes.
• Yes, frequency: There will be times when directly referring to frequency makes sense, and that should be possible, as well.
• Relative duration: Musical notation, regardless of musical culture, uses some kind of relative indication of duration. Only machines use raw clock values. The result is that it’s possible to make musically meaningful changes in tempo and have durations respond accordingly. And whereas note on and note on events make sense on input, a musical event would not logically separate these events; there’s some notion of an event with a beginning, middle, and end. If you sing an ‘A,’ that’s one event, with a duration, not an independent beginning of the note and end of the note.

Far from replacing existing standards for music notation, this kind of standard could interchange more gracefully with printed notation. If you import a standard MIDI file into notation software, you get results that are typically full of errors, because the SMF lacks musical information about the events it contains. With more of that information stored, and stored in standard ways, translating to paper would become vastly more effective.

I’m sure attempts to model this in OSC have been attempted before, but it’s worth compiling those ideas and resurrecting the discussion.

What about input?

Ah, you say, but then, let’s go back to the keyboard. None of these events makes sense on a keyboard. You don’t know when a note is pressed how long it’ll last. You don’t know the modal degree of a particular, arbitrarily-played note.

I was stuck on the same problem, until I realized what I had been taking for granted: MIDI conflates two very separate processes. It makes input and output the same. Musical notational systems have never done that. When you look at a score, it’s a set of musical ideas, given meaning and context. If you record a series of events from an input, those events don’t immediately have meaning or context. It’s confusing the mechanical with the musical. It’s the reason MIDI is not just like a player piano – it is a digital player piano.

Separate out the issue of recording mechanical input events, and you can have a system that’s more flexible. That system should fit whatever the input is. An organ, a shakuhachi, a didgeridoo, and an electric guitar aren’t the same thing. Why would they be represented with the same set of input events? That’s pretty daft.

Look at it this way: imagine if instead of being invented by synthesizer people, Aeolian Harp players had invented MIDI. (It’s not so far-fetched: the Aeolian Harp has a millenia-long history and was once quite popular.) An Aeolian Harp sequencer would feature elaborate, high-resolution data recording for wind pressure relative to different strings. It might measure, even, wind direction. In fact, it’d look a lot more like meteorological data than musical data per se. It certainly wouldn’t involve integers from 0 to 127.

This should lead to a simple conclusion with profound consequences:

Physical input and musical output should not be the same thing.

One of the advantages of a protocol like OSC (or any open, networked, self-described protocol) is that it can be open-ended and descriptive, meeting our earlier challenge. For instance, using a hierarchy of meta-data attached to the message, you could describe a set of variables relevant to wind input. If you wanted to transcribe the results in musical terms, you could then use a musical notation, as above – one that used musical identity attached to the resulting frequencies, as in relative modal pitch and rhythmic duration. But the input would be a separate problem. That’s a far piece from MIDI, which is adequate neither as a complete description of the input device, nor of any kind of resulting musical system.

But wait a minute – how is there a standard? How do you standardize something that could include an Aeolian Harp, a vuvuzela, and a bagpipe? Welcome to the problem of music. Music is by its very nature resistant to standardization, because the possibilities of the physical world are so broad. This also suggests how input protocols (and output protocols) can go beyond musically-exclusive data. Again, we can turn back to MIDI as a model. MIDI was intended with specific applications in mind, with messages that referred to MIDI notes and filter cutoff. But that didn’t stop it from being warped to accommodate tasks well outside the standard, ranging from triggering videos to controlling amusement park robotic characters (literally). This suggests to me that what defines a standard protocol of this kind is not what is most strictly standardized, but what is most flexible.

The real challenge with something like OSC, then, is to come up with standardized ways of defining non-standardized events, and using some kind of reflection or remote invocation to allow devices or software that have never communicated before to handle unexpected messages intelligently. At the very least, they should give users clear, understandable options about the data they send and receive. This independent question has been one the OSC community has raised for some time. To me, all that remains is to make some compelling implementations and let the most effective solution evolve and win out. Recent reading on the topic (though this absolutely deserves a separate post, which I’ll get to soon):
Best Practices for Open Sound Control
Minuit : Propositions for a query system over OSC

That’s a separate problem from how to make events musically meaningful. But that to me is the central revelation, and something MIDI completely misses: these are two separate problems, not one problem. Handle input events as input. If it makes sense in a sequencer to record them as musical events (like scale degree pitches), do that. If it makes sense to record them as a series of time-stamped, physical events, do that – but with actual information relative to what was recorded, so that the wind across an Aeolian Harp is recorded in a way that makes sense for that input. And when describing musical events, describe them in musical ways.

This isn’t relevant only to music communities, either: it’s relevant to anyone recording events in time. It’s part of the reason the “sound” needs to be dropped from OSC. MIDI is as specific as it is partly because the specification has messages too small to contain information describing what the events mean. We now have standard network protocols that do that, so they can include information about other kinds of events. There’s no reason someone monitoring water levels in their herb garden and someone recording a sousaphone solo couldn’t use some of the same underlying protocols. There’s also every reason they’d record different kinds of data content.

Promising venues and a call to action

There’s really no need to try to “replace” or “fix” MIDI – if MIDI has endured for a specific application, maybe it actually is well-suited to that application. I think it’s time, instead, to think about how new systems can encompass more musical meaning from our own traditions and traditions around the world, and how we can standardize broad ranges of events instead of trying to fit everything into narrow, rigid boxes.

There is every reason to believe new things can happen now, too. Whereas hardware standardization once was a slow process, requiring the involvement of major manufacturers, we now carry around programmable computers inside our pockets as “phones” and learn to write embedded code in Freshman college classes using $30 Arduino boards. If you want new hardware standards, you can literally make them yourself. We have the ability to share musical notation directly in a Web browser using standard descriptions, as covered here recently. Because browsers in general are demanding newly distributed, networked applications, communicating in standard ways – as Web APIs do naturally – is becoming imperative.

But there’s one thing that makes me especially optimistic: you. Via the Web, we have instant access to your collective knowledge and experience. That means it’s a sure thing that all of us, collectively, knows more about previous research in this area, previous ideas, and what has and hasn’t worked. We also have the opportunity to communicate with each other, to make ideas evolve, at least experimentally. That doesn’t remove the need for eventual standardization, but good standards follow practice, not the other way around – something has to work in one place before it can be a shared standard. We also have mechanisms for self-standardization that didn’t exist before. Spoken languages evolve because people collectively work to share common means of communication. You might argue that this leads to a tower of Babel, but then, I’m writing this in English and you’re reading it in the same language and (hopefully) understanding. The same is true of Mandarin, Portuguese, German, Arabic, Hindi, and so on. It’s also true of volunteer adoption on the Internet of HTML, XML, JSON, and RSS.

Music is not the result of notation or standards. It’s the other way around. Musical practice long predated any attempt to write it down. And mathematics and written language each have abilities to describe music and many other media.

To me, two questions remain:
1. What would an implementation of structured messages for pitch and duration look like, perhaps implemented via OSC? What history has been there in this area, and what do you need?
2. How can smarter implementations of a protocol like OSC allow software and hardware to better handle unfamiliar input – as musicians, as they have done since the dawn of time, invent novel physical interfaces?

I look forward to kicking off this discussion and hearing what you think.

What if you could have any musical technology you wanted – if you had only to imagine something, and it appeared? That was the somewhat insane notion behind the Dreams Competition CDM organized with Rui Penha of Casa Da Musica’s Digitópia research and education program in Porto, Portugal. Earlier this week, Rui and I sat down on the banks of Porto’s famed Douro River with Paulo Maria Rodrigues to pour through stacks of imaginary instruments. Some proposals read like wish lists composed to Santa Claus. Others included exquisite renderings, mock-ups, and even video that made them into serious, near-finished product designs. In the end, we attempted to choose the ideas that seemed the most surprising and original, including a winner that – with some limitation of its scope – would be feasible to actually build.

Far from just being idle fantasy, the winner will be realized by a team of developers as an open-source, free project. And I suspect some of the other entries may yield real tools, too. The line-up offers plenty of indications of what matters to people, and what’s possible. Here are some of our favorite entries out of an impressively high-quality bunch, plus, of course, our winners and the grand-prize selection that will inspire a real project.

Winner: Dream Synthesizer, Andreas Paleologos

The “Dream Synthesizer” lives up to its name: it embodies a laundry list of ideas about how to make a more interesting synth. Building the whole instrument would be wildly impractical for this project. But one central concept caught our eye as both practical and innovative. The idea is, touch interaction with an array of LEDs creates envelopes that provide a single metaphor for all sound design. Those envelopes aren’t just one paradigm among others: every sound parameter is accessed with gestures.

To realize this idea, we’re directing the Digitopia team we’re assembling for this project to focus on the LED array itself, and working with software to produce sounds. The very limitations of the LED display itself have some appeal, even in this age of multi-touch displays. Furthermore, the constructed physical object should produce a reusable part that other people interested in building their own hardware can reuse. Mapping different software synthesis methods, all built in free software, to touch gestures means the sound side should be reusable, too.

Making a project “free and open source” isn’t just about a license; it’s about choosing something that could be a building block for a wide range of ideas, and making that building block as usable and friendly – from engineering to documentation – as possible. Here’s how Andreas describes his concept, the spark that most inspired us:

My Dream Synthesizer has 3 sound modules that together generate the sound.
Digital Sound Module.
Physical Sound Module.
Vocal Sound Module.
They’re high-quality software modules with a lot of identity. The sound is constantly routed through all three
sound modules, whether generating sound or silent.
It has a big Low Resolution LED screen, covered with a transparent high resolution multi touch film for on
screen interaction with support for up to 3 fingers.
Draw the waveform on the screen and get instant control of the sound.
Use one finger to manipulate the Digital Sound Module.
Use two fingers to manipulate the Physical Sound Module.
Use three fingers to manipulate the Vocal Sound Module.
The envelope is basically ADSR, but with looping sustain. Draw your envelope curve.
You can record and automate all waveform and envelope changes making really complex sounding sounds
with just a finger stroke, recording your particular timing.
Select whether all automation should be triggered once or whether it should loop.

There are more details in the proposal we’ll share soon. And since the project is open, it’s one I hope that we’ll share with the CDM community on an ongoing basis, including getting reactions and ideas as the project is implemented.

By the way, check out Swedish-born, Norwegian-resident Andreas’ artist site, Cuckoo — cool stuff:
http://cuckoo.no/

First Runner-Up: Odu, Nicole Weber

Nicole Weber’s (Germany) Odu was probably the most stunning design work in the lot, including a full physical mock-up and UI design concepts. It’s an unusual combination of physical interface and Web-based sonic engine. On the Web side, users find sample content through a browser interface. On the physical side, a handheld interface turns those samples into tangible objects for manipulation and performance.

Nicole describes her “modular” project thusly:

programmer

• With the programmer the user is able to program the controller with samples
• Ships with a sample archive, e.g. created in collaboration with freesound.org or similar
• The user is able to archive and search own samples
• Optional community feature like competitions are provided via the software interface

base

• The base has all the basic features like USB connection, volume and connection ports for effects or other modules
• Integration of sensors in the controller body, e.g. tilt sensor or accelerometer

See her full proposal:
http://topotropic.de/odu

Honorable Mention: Fabric Ghost Controllers, Tycho

Tycho (Germany) sent what was perhaps the most evocative idea. It’s one I hope actually will be realized, but I think one that involves very specific skills. (Our friends the Grant Sisters of felted signal processing have been working on this very problem.) Tycho writes:

I have a dream.
I imagine when playing electronic music live I have sort of a display beside keyboard and computer where seven colored silk scarves are hanging. The cloths are maybe half a meter long, they are square and fixed with their center at a pole or something, their four corners loosely hanging down. It looks remotely like seven ghosts in a row. You get the picture?

These are the scarf ghosts controllers! Blowing against the scarves or touching them of fanning at them, maybe even squeezing them for extreme results shows that all seven of them control parameters of the actual sound(s) playing.

I do sleep concerts with very very low volume electronic music in Berlin, Germany. I believe such a “silk scarf ghost controller” would be perfect to accompany my music. During my nocturnal seven hour concert (plus some wake up music) I refer to (and sort of musically render) the seven chakras––so I would choose the number seven and the six rainbow colors plus white for the “ghost controller”.

But can a cloth be a controller? I learned of a Doepfer device that transforms a signal of 0 to 5 volts into the MIDI range. Unfortunately I have no idea how to make a scarf turn out volts when blown at or touched or squeezed. Could you work in a metal wire or something? Plug in little invisble batteries?

I had the idea just a few days ago when I heard of your competition. Maybe you would like such a beautiful and subtle “ghost controller” as well. I had even the vision of using it as sort of wind chimes: hang it in a breeze with my equipment rigged up in nature and let it produce generative sounds.

Its main purpose is being a live controller though that maybe look like decoration at first but turns out to affect the sounds and music. (As such it’s naturally highly lightweight and portable: Just fold the scarves.) They maybe even interact when one scarf touches another.

And imagine taking the pole into your hand and swinging it gently all seven the scarves flapping in the air! What a finale!

More Terrific Ideas

There are really too many interesting proposals to list here, but just to give you a taste:

The Wheel Array and the Ball Array, Stefan Blixt (Sweden): Blixt proposed a kinetic interface involving physical wheels. The idea is novel and a tangible contrast for the increasingly-minimal digital interfaces in our world.

Vitus, Michael Oneppo (USA): Vitus is a “controller” suite of connected, wireless objects.

each performer has a foot panel that controls expression, recording, looping, and effects for his or her performance. through the panel, the performer can enable or disable effects, modulate parameters of the effects, and mix up to four loopable performance clips on the fly.
a number of different wireless connections are provided that allow any microphone, guitar, keyboard controller, or other instrument to be wirelessly connected to the system. in addition, these interfaces provide a one button control for activating the instrument for recording and looping with the foot panel.
finally, a main controller panel is available for the controllerist/mixer of the group. this unit provides a master view of all performers’ clips and settings, and also allows the performer to manipulate these clips to make unique mixes. the interface is an array of hybrid button knobs, which present the performer with endless possibilities and configurations. the panel instantly slices any clip into eighth notes segments that can be triggered in monome-style phrasing across the button/knob grid.
all devices are truly wireless, eliminating any messy cords and freeing the performers. being battery powered, each unit recharges inductively through a storage case that can plug into any outlet.

Boulier, Yann Girard (France): Boulier has an ingenious take on how to maximize musical functionality in an array of encoders: use color to denote pitch. Someone may have thought of this before, but the execution is lovely.

LUM from Alfredo Duarte on Vimeo.

LUM, Alfredo A. Duarte Jorquera (Chile) is already, in this mock-up demonstration video, a compelling demonstration of the use of the Sony PS3 Move controller for music. Max Mathews, maker of the Radio Baton, would be proud. I hope Alfredo continues with this project, because I think he’s got a clear vision of how it can work. Updated: that’s an actual demo, so the work has already begun! Go forth!

And the rest…

Jacob Adler (USA) easily wins the honorary Luigi Russolo Award for craziest idea. I think you can agree from the first line:

I want to turn a mountain into a giant organ.
In the millennia-old tradition of organ building, I propose to develop a new kind of instrument – one constructed from powerful train horns and heard in an open desert space. The performer of such an instrument will actuate the valves of multiple air compressors remotely through a digital signal processing program and midi keyboard controller (when an E is pressed, the corresponding horn will sound). The harmonic richness of train horns are beautiful and I believe they can be used to create a new kind of music in
vast spaces. The instrument will be mobile and may be installed in any natural, open space for a series of concerts. One ideal location for this instrument is the south face of Papago mountain in Phoenix, AZ – the
mountain itself is visually beautiful and the park is acoustically ideal.
With a rank of 48 horns (corresponding to four musical octaves), free community concerts of new and old music will be performed. Every stage of the project should be documented through digital video, audio, and photos.
Throughout the organ-building process there will be many issues that will need to be resolved such as power/air-pressure requirements, acoustics in relation to topography, tuning, scaling, voicing, shape of the pipe/horn body, aesthetics of the organ case (if any), and design of the overall instrument. I have a background in working with pipe organs (performing and maintaining), and I would love to work in close collaboration with engineers and musicians. Visually, I would like to base the design of the instrument
after a famous antique organ case in Alkmaar (Netherlands) – the Schnitger Organ in St.
Laurenskerk<http://upload.wikimedia.org/wikipedia/commons/6/62/Alkmaar_organ.jpg>.
However, the design and materials used will harmonize organically with the desert landscape.

At the opposite end of the spectrum, Steppophonic Looperformer, Per Boysen (Sweden) is a very practical-looking approach to step sequencing, and is labeled “please steal this!” It’s well within the range of things readers here could accomplish, so perhaps take the creator’s invitation — write back with the results!

Other ideas, like an OSC sequencer or modular musical/MIDI hardware seem like things that are destined to happen, even if this isn’t quite the forum for those problems.

A big thanks to everyone for compelling, creative contributions. If you have any more ideas or questions, if you missed this round and want to continue the conversation, the grand project of dreaming up the future of music tech is an unending one.

Want a flying car? Dream of the flying car. Build the flying car.

A competition I’m hosting with Digitópia, the musical-technological community of Porto, Portugal, extends to readers worldwide a challenge to dream up the digital musical instrument/interface/creation you want. Got something practical you wish could be built? Got something impractical and bizarre? Either way, articulate it in the best way you can — images, words, videos, mock-ups, stop motion animation, beat poetry, whatever you think is best — and send it in. We’ll share the most interesting entries, and pick one that the folks at Digitópia will actually build. (So, if it is unfeasible, we’ll have to find one that at least can be made feasible.)

I hope it’s just the beginning of this kind of big-picture thinking in digital music.

Here’s organizer Rui Penha on the concept behind the call for entries:

Digitópia = Digital Utopia. We strongly believe in the power of communities, of open source endeavors, of sharing and spreading inspiring ideas, either simple or utterly crazy ones. Our goal is to empower the individual with means to achieve a more fulfilling, rewarding and personal musical expression, regardless of his or her experience and motivation. New interfaces and instruments can overcome the steep technique obstacles of some old ones and create new musical languages and thus we want to make them available to everyone. We want to help you build your idea and, together, we’ll share it with the whole world!

We want your ideas, but you have to act fast. The deadline is this Saturday, midnight GMT, April 3.

Submit ideas via email to competitions@digitopia-cdm.net, using whatever medium of illustration you wish. Works will be judged on innovation, originality, feasibility and inclusive potential. If you win, you get your instrument, built for you.

http://digitopia-cdm.net/competitions/

Full rules after the break / bottom of this post.

By the way, if you’re near Porto, Portugal, there’s a Handmade Music event this Saturday 3/27! Go, take videos, photos, enjoy! Details:

In Portugal, Now

Por favor divulgue. Obrigado! / Please spread. Thank you! (english version below)

A quarta edição da Handmade Music Porto terá lugar já no próximo sábado, dia 27 de Março, na Digitópia: uma festa que junta um mostra&conta a uma jam session com instrumentos únicos. De hardware a software feito em casa até circuit bending, kits personalizados ou instrumentos acústicos originais, todos estão convidados a aparecer na Casa da Música pelas 21h30 para montagem de instrumentos. Estarão disponíveis algumas mesas e tomadas, contudo os canais de amplificação serão muito limitados, pelo que será melhor vir prevenido. Pelas 22h abrimos o evento ao público geral – a entrada é livre e recomenda-se -, ocupando a Digitópia e a zona do bar do Foyer Sul. Contamos convosco!

Teremos dois convidados muito especiais: Rolf Gehlhaar e Luís Girão, que trarão alguns dos instrumentos criados para o projecto “instruments 4 everyone”, no âmbito do Festival Ao Alcance de Todos, edições de 2009 e 2010, que agora começa.

Rolf Gehlhaar – http://www.gehlhaar.org/

Luís Girão – http://www.artshare.com.pt/

——

The fourth Handmade Music Porto, a party + show&tell + jam session with unique instruments, will take place at Digitópia next saturday, March 27th. From handmade hardware or software all the way to circuit bending, customized kits or original acoustic instruments, everyone is welcome at Casa da Música around 9:30pm for assembling the instruments. We’ll provide some tables and power sockets, but only a few channels for amplification, so it is advisable not to rely on them. At 10pm we’ll open the doors – admission is free and we’ll have a bar! See you there!

We’ll have two very special guests: Rolf Gehlhaar and Luís Girão, who will bring some of the instruments made for the “instruments 4 everyone” project, part of the Ao Alcance de Todos festival in 2009 and 2010, starting this week.

Rolf Gehlhaar – http://www.gehlhaar.org/

Luís Girão – http://www.artshare.com.pt/

——

You may view the latest post at
http://digitopia-cdm.net/2010/03/handmade-music-digitopia-2703/

Entering the Competition (worldwide)

Rules (PDF download):

RULES · Digitópia Dreams Competition · Digitópia 2010
1 ·
WORKS
1.1 · Entrants shall submit an idea for their dream instrument, interface or software.
1.2 · Only original and yet to be materialized ideas will be admissible.
1.2 · The winning entries shall be developed under a Creative Commons license – http://creativecommons.org/about/licenses .
2 · SUBMISSION
2.1 · Works shall be submitted by email to the address competitions@digitopia-cdm.net , with the contact information of the
applicant – full name, nationality, date of birth, email address – on the email body.
2.2 · Each applicant is free to choose the best way (text, schemes, videos, etc.) to present his or her idea.
2.2 · The closing date for entries is 03/04/2010, at 23:59 GMT.
2.3 · All successful submissions will receive an auto-reply by email.
2.4 · Each applicants may submit up to three ideas.
3 ·
JURY
3.1 · The jury will be comprised of Peter Kirn (president), Paulo Maria Rodrigues and Rui Penha.
3.2 · Judging will be based on each submission’s innovation, originality, feasibility and inclusive potential.
3.3 · The jury will announce its decision on 02/06/2010, through Digitópia’s website – http://digitopia-cdm.net .
3.4 · The jury may decide that none of the works submitted merit selection.
3.5 · The jury’s decision shall be final.
4 ·
PRIZE
4.1 · The winning applicant will be invited to collaborate with Casa da Música and Digitópia’s team on the development of his or her
project.
4.2 · At least two copies of the project will be built, one for the applicant and other for Casa da Música.
4.3 · The complete process will be documented and shared under a Creative Commons license – http://creativecommons.org/
about/licenses attributed to the applicant.

Good luck! I look forward to the results.

When I reviewed JazzMutant’s Lemur at the end of 2005 (printed in the February 2006 Keyboard Magazine), I wondered if what we were really waiting for wasn’t a computer screen. At the time, I wrote:

There’s no question that multi-touch touchscreens represent the future of computer interfaces, and the Lemur is the biggest leap yet toward that science fiction future. For now, the challenge is that the Lemur’s features lie somewhere between a computer display and music controller, without effectively supplanting either one. The Lemur sacrifices the sensitivity and tactile feedback of physical controls in the name of flexibility, but that payoff is limited by the restrictions of its pre-built interface objects and the difficulty of configuring new layouts and assigning them to software controls.

If the Lemur could be truly fused with the computer display, rather than requiring an entirely independent interface, it would become a must-buy.

JazzMutant Lemur Review

At the same time, I marveled at what multi-touch could mean: interfaces that were as flexible as software, powerful live performance capabilities, and the ability to navigate sound spatialization and timbre in new, freer ways. Rather than a solution in search of a problem (as multi-touch image resizing is, arguably), these were tasks that just weren’t possible via any other interface.

The video above, showing multi-touch integrated with the next version of Windows 7 (expected at the end of next year), demonstrates one thing to me: multi-touch is coming, and it’ll be mainstream. And that’s huge for creative performance.

Microsoft demonstrates Multi-touch at D: All Things Digital Conference [Windows Vista Team Blog]

When Touch Makes Sense

Ironically, because Microsoft is the first to show off this technology in something resembling a consumer-ready, standard computer, people are lukewarm. (Do you think the reaction would have been this way if it had been Apple showing the same demo?) Now, I’m all for skepticism. It’s nice to see Lifehacker asking its readers whether touch is really necessary. That was the question I asked in regards to the Lemur, as well: touch isn’t the answer to everything. You lose tactical feedback, and a certain amount of accuracy. On the multi-touch iPhone, this is an especially big deal: I can easily out-type any iPhone user on my Blackberry, and multi-touch doesn’t mean a whole lot on a small form factor that can only comfortably accommodate one or two fingers at a time. Lastly, no technology can change the physical size of your finger relative to, say, a stylus.

But when it comes to music performance, I’m convinced multi-touch can be very powerful. Forget Microsoft’s lame piano demo or obligatory but meaningless photo resizing. Onstage, a multi-touch display is ideal. You can make quick gestures, quickly point at stuff without taking your eyes off the screen, and use large-scale interfaces built for performance. Imagine reaching over to quickly swap instruments, or switch between song sets, or make a rapid gesture to adjust the timbre of a sound, or navigate surround sound spatialization. And imagine that you’ll be able to do this without having to content with another piece of gear, as on the Lemur, but on a mainstream laptop, with any software you like.

Beyond Microsoft

What’s ultimately fantastic about the Microsoft announcement is that it should have implications beyond just Windows. Unlike the proprietary, one-device iPhone, having Windows 7 support multi-touch means lots of hardware should follow, with the economy of scale and access that everyone may benefit. Even Microsoft’s commitment to the relatively niche-oriented tablet PC has driven down digitizer prices (a step, not incidentally, toward this announcement). You can buy an affordable tablet PC right now with Linux installed, if you like. While Microsoft has a leg up in the enabling software for multi-touch, I don’t think it’ll be impractical for other frameworks or open-source frameworks to follow. In fact, the real challenge is to think about interface design in a new way. (In an interview with CDM, the developers of the upcoming Circle soft synth specifically mentioned thinking about making touch work in future as a design goal, and they use the cross-platform JUCE framework.)

And while they didn’t make a specific announcement, I would expect Microsoft to be likewise aggressive about promoting multi-touch capabilities in their own application development frameworks. Ultimately, I believe the most interesting multi-touch interfaces will continue to come from individual developers and researchers, not the likes of Microsoft and Apple. That’s been true already, so imagine what will happen when those folks have cheap hardware ready to go and can focus on design. The OLPC project, of course, promised a multi-touch laptop replacement, as well; that’s basically just a mock-up and I’ll believe it when I see it, but someone is going to deliver a multi-touch machine soon. (It’ll be interesting to see if we hear anything from Apple, as well.)

Yep, I Want It

Don’t get me wrong: tangible, hardware controls aren’t going anywhere. On the contrary, I think the experience of using multi-touch displays, which even with haptics are a long way from giving real tactile feedback, reminds us of the range of ways in which software design and hardware interface can fuse. But by going beyond QWERTY and mouse/trackpad, multi-touch displays could make for an exciting future.

And in answer to Mary Jo Foley’s question, do I want multi-touch in a laptop? Not only do I, but stand next to me or any other digital musician struggling with a tiny trackpad onstage, and you’ll see why.

More multi-touch coverage from CDM

Cybersonica & Encompass Sonic Art Exhibition [YouTube]

Among the delights inside: suspended disco satellites controlled by Korg Kaoss Pads, motion tracking that translates a performer into a shadow puppet monster (complete with roaring sounds), a liquid, fully-3D interface for making music which shall be known at CDM simply as the hotness, a 3D Etch-a-Sketch for sound, an installation with an interface controlled by torn paper, and even a mechanical contraption that samples visitors onto analog tape (it’s not all digital).

Chris is gradually documenting the works on his blog, Pixelsumo. If you’re in London, don’t miss the programs Friday and Saturday, and do file a report so all the rest of us know how it goes!

Jonathan notes some significant bugs in getting the Lemur to work, particularly with control assignments (including problems using multiple arguments with Reaktor). But his conclusion is ultimately positive:

“Using the Lemur as a control surface in live performance feels much more like playing an actual instrument than simply turning knobs or moving faders on a control surface . . . The Lemur is easily my favorite hardware controller, and I hope it becomes popular (especially so that the price will drop).”

My review for Keyboard also found assigning controllers to be unnecessarily difficult. But the primary difference between the two reviews is that I found the Lemur to excel at some tasks, but not at others, and ultimately concluded it was the future of computer interfaces, but not necessarily of musical instruments (at least not on its own):

“The Lemur suggests fantastic possibilities for certain tasks. ItÃÅ’s unparalleled for surround panning and X/Y timbral control, particularly if combined with another hardware controller like a keyboard or ribbon controller.

. . . For now, the challenge is that the LemurÃÅ’s features lie somewhere between a computer display and music controller, without effectively supplanting either one. The Lemur sacrifices the sensitivity and tactile feedback of physical controls in the name of flexibility, but that payoff is limited by the restrictions of its pre-built interface objects and the difficulty of configuring new layouts and assigning them to software controls.”

Do check out the entire reviews, as they’re now both online with full text and images (as there are far too many details of each to summarize here):

JAZZ MUTANT Lemur, Electronic Musician review

Programmable Multi-Touch Control: JazzMutant Lemur, Keyboard review

The hilarious send-up of educational films that was Look Around You: Music was only the beginning. BBC comedy show Look Around You has its own fantastic website filled with still more goodies. And it gives us a much clearer idea of the future of music technology than, say, a teaser from Moog.

Readers have been sending in “Life in the Year 2000″ entries, which include the five-string bass guitar, sex changes using Bach violin concertos, and my personal favorite, Halson Hoek’s invention that improves your keyboard chops by sending electrical shocks through metal gauntlets. At this point, that might be the only thing that can save my piano playing.

Best of all, Look Around You gives us what must be the mascot of Create Digital Music: enigmatic “musechnologist” Synthesizer Patel. He’s shown here with the watery keys of the Liquinth, perhaps inspired by a post here on the water-powered Mocean? There’s more from the new music episode, including a playable Mini-Trelm synth which has sadly been “stolen”. The TV network that gave us the Radiophonic Workshop deeply feels the trade we all ply:

“Synthesizer spends hours at these machines, carefully programming crochets, demi-clefs and arpeggionnes to achieve that special blend of sounds.”