The quickest route to expressing an idea remains the gesture of a hand. That gesture may be crudely interpreted through today’s touch displays, but the immediacy remains. Presumably because of some of the device’s limitations, a lot of the experiments with the iPad have involved controllers that operate independently from sound software, like a remote control. Those interfaces, while useful, largely simulate existing hardware controls in a more flexible form, rather than introduce new ideas. But it seems the long-term potential for touch devices is in designs that unite touch, graphic, and sound in a single piece of software, exploring new paradigms for interaction along the way.

Usine is one of music creation’s most surprising secrets: it’s powerful sound software that incorporates creative touch interfaces as a core design principle. And in the video above, it’s running on a relatively cheap PC two-touch display from Packard Bell. Nay-Seven is one of the founders of the Usine community, all while lecturing internationally, and has been pushing the Usine software to its limits.

Here, he tells us about some of his latest experiments, and the potential they hold.

Always looking for a way to use the computer as a real musical instrument, my latest works try to combine graphics and music using a touchscreen interface. The software Usine from sensomusic gives me the freedom to build my own interfaces. Some examples:

Drawing pitch and pan

Here [at top], the purpose is to draw directly some pitch information on the waveform display of a sample. I’ve also added an LFO [low frequency oscillator for modulation]; this way, the drawing can move slowly according to different speed presets.

[At bottom], I play with pan and volume: the x position of the black ball on the lines gives pan information and y the volume. As I’m on working with a dualtouch screen, I can quickly draw some speed changes. Note that this panel is not only for pan and volume; I can also send this drawing to others parameters like delay and filters, here with the << button.

Geometry …or not

This workspace is also dedicated to drawing. I’ve built four layers, each one with its own color and its own sound. The XY position gives the pitch value of the notes and other parameters, like velocity or pan. The geometry provides sequences; lines give a kind of glissando.

Vertical sequencers and Pads

Using the new Matrix module (thanks to Martin Fleurent), I’ve built this vertical sequencer [seen at top]. I like the idea that notes fly under my hands this way. [At bottom], I‘ve built pads for tablet surfing on the “iPad” mode, adding also a drone option.

Multitouch gestures

On the same idea of movement, here are two screenshots of a video illustrating a new patch made by Olivier Sens (the Usine developer). This patch provides multitouch gesture recognition, opening new doors to ways in which we use our computers and touchscreen. We can easily imagine some new symbols or alphabets, and new forms of interactions in our musical practice. You draw a ‘V,’ you play with volume, you draw a ‘P,’ you play with pitch…

For more on the display, check out the Packard Bell Viseo 200T. It was previewed by Engadget last year and carried a street price – impressively – of only about US$300, all for a 20-inch screen and low latency. I’m gathering either something happened or it was re-branded for distribution outside the UK; anyone with more information, let us know in comments and I’ll update the story.

More on nay-seven’s Flickr:
http://www.flickr.com/photos/usine/

All screen images courtesy nay-seven. Used by permission.

Musicians have long made pictures to represent musical ideas, share those ideas, and allow others to participate. Before computers, we created scores. Now, we can create interfaces, too. Of course, just because you’re using a digital interface doesn’t mean the pencil as prototyping tool has to go anywhere. It’s the quickest way to sketch out an idea. And if your hand is steady, it just might become a lovely, personal interface.

OtoBlock by Tsubasa Naruse is a hand-drawn music sequencer. The basic interface is nothing new, dropping blocks into sequence to make sounds, but the charm is the rough edges on the pencil-made buttons, and the whimsical hand-drawn characters that live on them.

OtoBlock @ iTunes
Tsubasa Naruse website, in Japanese, but don’t miss the other adorable sketches
Via Matrixsynth, by way of Palm Sounds

Here is a 2009 experiment in “sonic interaction” by the same artist, also exploring
some of these ideas. (link) I’m not sure I could even describe it, but the relationship of minimal electronic sounds to handmade animation is utterly irresistible.

So, with mobile music tools like iPhones and portable amps from the likes of Roland and KORG, can you actually go out and make music in a subway? I recall people mocking an old M-Audio ad in which someone was doing laptop production on the subway platform. But when it comes to mobile busking, the same videographer who shot the hands-on video at top also captures an impromptu TB-303 jam in the Tokyo subway. (Apparently, this young woman did not inspire love from the police. Sadly, the app she’s using is entangled in some sort of contract issues.)

More great iOS videos on perfumepod’s channel; it’s a great way to explore different user interfaces:
http://www.youtube.com/user/perfumepod

And yes, I’ve been tipped off to “Tokyo Techno Girl” before; I have to find out more about her.

Imagine drawing an interface on paper, then being able to use it as a musical interface. Or, heck, don’t imagine it – do it. Unfortunately, the kinds of intelligence necessary to make the music video in yesterday’s post just aren’t practical yet. (That is, you could draw a picture of a keyboard, and even use the picture as a music controller, but while you or I could recognize a keyboard from a drum pad and know that line is a fader, a computer would need some sort of advance structure for any recognition to work.) But you can do some really clever things, as folks have shared in comments.

And using some basic paper interfaces, you can make entire instruments for just a few dollars.

Of course, the awesomest way to do anything is with LAZORS. Greg Kellum and Alain Crevoisier presented a paper at last year’s NIME (a conference for new interface designs for music) proposing a system for making any surface a control surface. Like the music video yesterday, you can configure your surface to function however you like – even dividing it up into pads and faders.

By now, you’e likely seen plenty of multi-touch interfaces or means of tracking hands. But, to paraphrase the NIME paper, these either require a special surface (or transparent surface), or they can’t actually detect when you’re touching. You can even use multiple cameras or an IR beam, but there are limitations to accuracy and the size of the usable surface that would result. Kellum and Crevoisier use an infrared camera and two illuminators, each built by pointing a laser at a mirrors.

Yawn, you say, been there, done that, seen Jeff Han’s video… The advantage of this system is that you can use any surface, like your dining room table. And you can configure that surface however you like. There’s even a freely-downloadable Surface Editor you can extend in Java and Processing. The creators claim they can even get input latency down to a reasonable 10 ms using high-speed cameras.

Transforming Ordinary Surfaces into Multi-touch Controllers [PDF paper, NIME 2008]
Future Instruments > Projects
Thanks, Randy Jones!

db3ll has created a keyboard out of paper, and of course it works better than those flimsy rubber “roll-up” pianos you see for sale. “Conductive ink is what I used,” he says, “painted on as traces on the non-printed side of the paper.” That’s the twist – I had assumed you’d use the top of the paper, but the trick is to use the reverse side to provide the “wiring.” He also offers advice for making a fader:

You can make a paper thin fader in much the same way, but it requires a magnet. Cut a slot in a piece of paper, color around the slot with conductive ink (I use the “trace repair” pens sold at electronics supply places… it has a very fine tip), and glue some SVHS tape (resistive side up) under it. Put a thin piece of metal beneath the SHVS tape & use a magnet to conduct between the SVHS tape & the conductive ink. The magnet will stay in position due to the metal (I use package banding) under it, and aside from the magnet, it is roughly the thickness of a couple sheets of paper.

Simon Lacelle is also working on a project I’m eager to see:

In a pad controller I’m making using a HUGE Staples calculator, I’m using strips of aluminium foil separated by a sheet of paper with holes at each button as switches merely a milimeter thick, and these are quite responsive.

A YouTube uploader by the name of DJ Mocap appeared briefly online with a project that seems to show him controlling Traktor with a drawing. There’s a camera and some sort of analog input being fed into a circuit board, but I’m not entirely sure what’s going on – though I can think of a couple of ways to make this work. It stumped DJ Tech Tools’ readers, but I have a feeling it can’t stump CDM readers, so have at it.

UPDATED – FAKE (but possible) Okay, so this turns out to be a Stanton touch controller hiding underneath a piece of paper. Of course, that’s itself not such a terrible idea – by having a drawn overlay, you have visual feedback for specific positions on the controller. But furthermore, while this is fake, the idea remains possible – and more cheaply than buying a piece of Stanton gear to toss under your piece of paper. So I call this “fake but potentially inspiring.”

Thanks to Gizmo from Scratchworx. Now, show Gizmo and Mocap by making a real version of this!

Just to consider moving in the opposite direction, I have to point to Amit Pitaru’s Sonic Wire Sculptor, an interface for drawing virtually and digitally. Because it’s digital, you can draw in 3D, do something you can’t with real-world markers. Here it is in a Tokyo gallery installation version; see more information (or try it yourself online) at Amit’s site.

And back to the realm of the imaginary – could MPCs of the future be made out of cardboard? (Oh, how I love reading YouTube comments. “Doesn’t look too sturdy.” “Why do you have your MPC in a box?” Apparently some people thought this was somehow insulting hip-hop. YouTube comments – pushing the very frontier of stupidity.)

Thanks to dyscode on comments — brilliant.

The cardboard MPC comes from theycontrol.us and our friend Elijah Torn, as seen previously on CDM.

Doing it Yourself

If you’re interested in entering the world of paper, drawing, and controllers, there are two directions I’d suggest.

One way to go is to simply start thinking about drawing as an interface. The creator of Tablet 2 MIDI, a MIDI-graphics tablet interface, suggests that using the pen you can draw any interface you like, then map it to tablet input. That concept could certainly be applied more broadly.

As far as using paper and a conductive pen to doodle your own musical creations, it turns out this is one of the easiest ways to learn about resistance in electronics.

PAiA 2 Transistor “Ribbon” Kit from Create Digital Media on Vimeo.

Learn Musical Electronics, No Soldering: Free PAiA Ribbon Controller Kit for CDM Readers

This project, which we covered at the end of 2007 and featured at our Handmade Music event, is ideal for giving young people (or the solder-phobic) their first step into electronics. The whole kit fits on a business card; you just need speakers to which you can connect.

The Drawdio project uses the same basic circuit and principle, but attaches it to a pen, making the rig a little more portable and allowing other fascinating experiments. It’s also available for purchase.

http://web.media.mit.edu/~silver/drawdio/

You’ll find countless variations of the basic circuit, because it’s so simple, and it’d be a great way to get into the more sophisticated (or at least more complex) ideas here.

Other ideas? Questions? Stuff I’ve left out? Let me know, and I’ll update the story.

Note Pad from Charlie North on Vimeo.

This charming music video from Charlie North imagines creating your own simple music controllers with a piece of paper and a marker. (There’s some similarity to M-Audio pieces there, too.) Of course, that raises another question: could this actually be done?

Computer vision isn’t quite intelligent enough to work out automatically what’s going on here, but it seems to me that you could get a little closer. Another alternative would be using conductive ink or graphite to make the drawing itself a sensor. I’m going to leave you to puzzle out the rest.

It’s technically still a holiday weekend here in the U.S. of A., so I’m going to keep with the whimsical inspiration for the rest of the day.

Miraculously, Nat has escaped alive, and the results are fantastic. Good luck paying a designer to give you gig posters like this. These two warm my heart because of their digital music create-i-ness:

Day 27, Tsuki

28×28 Day 17 – Moulinex + Xinobi

For the complete set, see the lineup on onetonnemusic:

Gig Posters Archives

Have a gig coming up? Need a rocking poster to publicise said gig to the wider community? You should check out Nat’s 28 Posters in 28 Days Poster Challenge! You know you’re going to get a great result, because Nat designed this here website, and CDMo, and the forums. You should get in quick, however, because he doesn’t seem to be starting out in the most positive frame of mind:

They said I couldn’t do it! My girlfriend said I couldn’t do it. I don’t think I can do it… Let me preface this by saying that I have a sneaking suspicion that this isn’t one of my brightest ideas. Good? Clear? Okay.

For the month of February, I am going to attempt to do 1 FREE gig poster per day.

That means I need details for 28 gigs and bands who want posters done. Starting tomorrow, the 1st of February. 

I stepped up first, nominating my collaborators Segue and our show at the Empire, Brisbane on the 23rd of Feb, and I love the results!

I’m sure there are plenty of artists out there with gigs coming up (they don’t even have to be in Feb) who could do with some tasty design to publicise your awesomeness, so go sign up.

Visual Acoustics is an online musical toy built in Flash designed by Alex Lampe (“Ample Interactive”) of the UK. (Via Music Thing.) The motion visuals are beautiful, and the music and interface is very reminiscent of Toshio Iwai’s work (see Nintendo’s ElectroPlankton, for instance). As with Iwai’s designs, just about anything you play will sound good and ambient. Now, there are two schools of thought on that. One suggests that these kind of futuristic interfaces make music accessible to anyone. The other would hold that part of what makes traditional musical instruments lovely is that, while they take a long time to learn, the rewards are much deeper. I’m not sure one is inherently better than the other, but I still wonder if it isn’t possible to build visual interfaces that are harder to master but deeper to play.

If you want some inspiration for moving in either direction, Visual Acoustics certainly shows potential. Now you just need a Wacom tablet-enabled version that, rather than conventional sliders for parameters, adjusts to gesture and pressure.

Windows: Nicholas Fournel writes to tell us he’s just uploaded two new applications for Windows, for free. WMIDI converts tablet input to MIDI, with full support for Z angle and tilt; Theremin takes the next step and turns that into a musical instrument.

Nicholas has a lot of other fun stuff, including apps for painting with sound, granulating, glove using, and touch screen-inating. This is our kind of chap.

Mac: Tablet-to-music apps have a tendency to become vaporware. Not Music Unfolding’s µ midi controller. It appears they’ve updated the interface and features and have even made this a Universal app for Intel Macs. They’ve gone beyond just mapping the tablet input to MIDI: by providing computer keyboard control and lovely visual feedback, they’re really making the tablet into an instrument. Now go use the thing and prove this unusual application could have broader appeal!

The next challenge: make the tablet expressive as an instrument. I was once having lunch with Jon Appleton and talking about his work developing the Synclavier sampler, and he held up a salt shaker and said something to the effect of, as a musician/composer if you spent six months working with this, you would find a way to make it an instrument. Now I just need to re-teach myself to draw.

If you use any of these tools, do let us know how it goes — successes and frustrations alike.

[Updated:]
Loic Kessous stops by in comments to point to previous work with tablets over the past eight years, some of which I’ve mentioned here on CDM before, others I haven’t. Notably, his own site includes audio samples from a vocal synthesizer controlled by graphics tablet, as an example of some of the expressive possibilities of using tablet input.

Matt Wright, as interviewed for Cycling ’74′s site, has done a lot of work with tablets, including mapping tablet input to OSC (see paper with Adrian Freed et al, OSC site. OSC is able to maintain the structure of the data from the tablet at high resolutions, while still supported by apps like Max/MSP, Pd, and Reaktor (among others).

For Max/MSP, see the Max/MSP Wacom object for Mac, or a similar object for Windows.

SuperCollider has built-in Wacom support.

For Pd, see Moonix’s site for Moon Utils, wintablet (Windows-only), or Hans’ HID object for Linux. (There are others, too, I think, so a little Googlin’ / searching of the Pd community sites / experimentation might be in order.)

Have I missed anything?

Interactive music man Amit Pitaru is the creator of the Web sound toy Sonic Wire Sculptor:

Try Sonic Wire Sculptor on your Mac/PC browser

Thanks to Tim, who notes: “I saw it and got to play with it at the ICC gallery in Tokyo last year and on a video projector and touchscreen; it was great fun! The web version is pretty cool, too.”

Sculpting sound in your browser with virtual 3D shapes, complete with optional tablet input? Very cool. But even cooler is the tools behind the magic (mostly developed by Pitaru himself), which also drive a lot of other interactive music tech discussed here on CDM and on sites like Pixelsumo. The system was built in the free development environment Processing, but with some key add-ons:

JSyn, standard soft synth for Java

SONIA, sound library for Processing that using JSyn for lots of cool audio/FFT capabilities inside Processing

Wacom support for Processing

Got all that? Now, it’s way too hot in most of the Northern Hemisphere, so why go out? Sit at home and create new musical interfaces! Let us know how it goes.