Before the iPad, before the iPhone, and indeed before the masses understood touch interfaces would be a big deal, there was the Lemur. Dazzling people with high-contrast, colorful controls, this boutique hardware, priced well over €2000 and running embedded Linux and custom resistive touch technology, brought the future a bit early to a handful of musicians. Star Trek was what you heard most frequently – sweeping your fingers over black glass was nothing if not reminiscent of Geordi LaForge helming the Enterprise. (By the way, talk about prior art: those conceptual designers on The Next Generation, working initially with all-optical effects, were also well ahead of their time.)

Now, at last, Lemur arrives on the iPad, released by a leading iOS developer, Liine. Swept away by Apple’s more-affordable hardware, with the iPad offering a higher-resolution display, slimmer form factor, accurate touch sensing, and wireless capability, the Lemur hardware suddenly looked dated. With iPad software, it’s available to the masses.

http://liine.net/en/

The first question, of course: will anyone care – and will the Lemur software compete, with various other touch alternatives? At US$49.99 / €39.99 / £29.99 34.99, the Lemur app is far cheaper than a Lemur, but spendier than a lot of other touch software. [Ed.: An early press release incorrectly listed the UK pricing as £29.99. It's actually £34.99. Just don't ask us for currency conversions. -PK]

I’ve gotten to see the Lemur in action, and actually was walked through some interactive template ideas. (Unfortunately, I was unable to talk about that, and could only tease what I knew – I got to see more than I could talk about via folks working with Liine and M-nus Records’ stable of artists – Richie Hawtin and Ambivalent, in particular – and was really impressed.)

Just like other apps, the Lemur app will let you control any MIDI or OSC application on your computer from your iPad. But the Lemur brings a few strengths that I think will make it a contender in the iPad age:

Innovative controls: The Lemur’s array of controls is, simply, the largest and most comprehensive anywhere. And for those who want to push beyond just fake faders and knobs, it has an array of more unusual controls, with features like:

Physics: Simulated physics and dynamic movement were, to me, one of those most interesting features of the original Lemur. Whereas I’d almost always choose a physical fader or encoder over a touch equivalent, adding physics to touch allows the controller to play to its strengths.

Scripting: This is a big one. Right now, the only other tool capable of genuinely-dynamic, interactive scripts that modify the behavior of touch is the open source Control by Charlie Roberts. (That, to me, is probably the most compelling alternative, especially as it relies on familiar Web and JavaScript rendering, but it’ll need more input to be fully mature.)

Scripting on Lemur means you get dynamic templates that actually take advantage of the touchscreen. (Think back to Star Trek: mimicking that would require scripts. They use pages and interactive feedback all over the place.)

A mature editor: Now, here, I’m of a mixed mind. I still want a touch app that lets you edit right on the device – guess I’d better go make the one I want. But if you’re going to be editing templates on your Mac or PC, then the Windows/Mac Lemur editor is now tough to beat in sheer power. I was critical of early versions when I first reviewed the Lemur hardware, but it has evolved and matured since.

An installed User Library: This could well be the thing that puts Lemur for iPad over the top – and make no mistake, it’s the biggest obstacle to any newcomer in touch. The Lemur simply has a whole bunch of templates, ready to go, many of them really sophisticated.

The competition: I imagine TouchOSC will continue to dominate the market for touch apps, though interestingly, for many of the same reasons. It has an installed user base and templates, it has a graphical editor that runs on Mac and Windows that people find reasonably easy to use, good documentation and community, and it covers a lot of needs. TouchOSC’s low price also ensures it has nothing to worry about from Lemur, but the Lemur app will appeal to people with more advanced needs, and I think it’ll be a big hit.

Also unique about the iPad: because US$50 is considered “expensive,” it’s really not a zero sum game. You could buy all of the major touch apps for your iPad, assuming you own one, and still be short of the cost of one plastic keyboard.

As for Android? Look, technically, I’m sure you could port Lemur to Android. The fact that they’re not launching with Android support is no surprise – but the problems with Google’s installed base and market and their inability to get OS updates out on devices is a subject for another post. (Preferably one that involves me writing surrounded by candles in a warm salt bath so my blood pressure doesn’t explode.)

Video: How use Lemur + WiFi

Video: How to use OSC and Lemur

Video: How to use Lemur with USB MIDI

Summary:

- Connect the USB Cable to the iConnect MIDI or similar device.
- Open a factory template in the Lemur.
- Open the settings tab and assign the MIDI Ports

http://liine.net/en/products/lemur/

Postlude: What about Existing Lemur Users

Since it’s a matter of some confusion, I asked Liine to clarify their relationship with JazzMutant (now Stantum), the developer of Lemur, and why existing Lemur owners should spend some cash to upgrade. There’s a half-off deal through the beginning of January if you owned the Lemur hardware, but some Lemur owners understandably feel a bit left out, having invested massive amounts of time and money in the now-abandoned hardware platform. On the other hand, even $50 seems to me not unreasonable for updating to the new software, even if a free release for Lemur early adopters may have been nice. I have yet to test it myself, but I imagine I would have no problem recommending the Lemur app to anyone who owns a Lemur and an iPad, certainly if they’ve nailed the software release.

Updated:

CDM: What is the relationship of Liine to JazzMutant/Stantum?

Liine: Members of Liine have a historic relationship with JazzMutant/Stantum. Richie Hawtin and Gareth Williams were very early adopters of the Lemur and have worked closely with them for years. Nick and Gareth also worked alongside Max guru Mathieu Chamagne on the Mu Ableton Live controller for the Lemur. Axel is the former lead developer at JazzMutant who were are very proud to have on board with us for this venture.

In short, Liine is a young independent company, not affiliated with JazzMutant/Stantum, but with a friendship and working relationship going back many years. We are very proud to be contributing to the future of such a revolutionary controller.

CDM: Why not give Lemur for iPad to existing owners for free?

Liine: It costs time, money and resources for Liine to move Lemur to a new platform and relaunch it. In addition, distributing any product always involves costs. Offering a full rebate of the app price is simply non-viable, we would lose money. The initial release of Lemur on iPad is only the first chapter in this second life of the Lemur. Liine is taking JazzMutant’s code and concept into the future, you are going to see a lot of exciting developments (in-app editing, new objects, streamlined workflows…). This will, of course, continue to cost Liine time and money – the small contribution from legacy owners will help ensure the future of their investment in the original machine. Their early support allowed for many updates of the original software. For this, Liine are hugely appreciative as it means that the product we’re able to bring you is the most mature and powerful solution out there. This is why we want to thank those owners by offering them a 50% rebate. We very much appreciate your support.

“Lemur should just run on the iPad.”

“There’s no point to have a Lemur when you can get an iPad for $500.”

“When will the Lemur just run on the iPad?”

Soon, apparently. Sources and an in-person sighting suggest to me you’ll see this in the very near future.

The JazzMutant Lemur, the touch control hardware I reviewed over five years ago, gave musicians the first widely-available, for-sale taste of multi-touch control of music. It established a lot of basic paradigms that would appear on other platforms: high-contrast user interface objects on a black background (so they don’t blind you in a club), widgets that represent familiar elements like knobs and faders, and also some fairly powerful features like unique touch-centric widgets, simulated physics, and scripting. Some of those latter, more advanced features haven’t really been available in other control applications, and Lemur owners have wondered what their long-term solution might be.

So, a funny thing happened to me the other afternoon. I’m looking over the shoulder of M-nus DJ Ambivalent (Kevin McHugh) at Berlin’s Watergate and an afterparty, and I see – no, that’s not TouchOSC. That sure looks like a Lemur step sequencer. And then I might have spotted something similar in the front-of-house at Flughafen Tempelhof’s FLY BERMUDA show, for Richie Hawtin.

It’s possible this was all a dream, of course. So – who believes me?

Interactive touch layouts for sequencers are something of a no-brainer – imagine if an analog pattern machine and the deck of the Starship Enterprise had a love child. But platforms come and go. And just because the iPad is the shiny, new thing – and remains the most affordable solution at the moment – doesn’t mean you shouldn’t learn from ideas beyond just the platform with an Apple logo. Almost a year ago, we saw some compelling sequencer ideas for the Lemur. Sadly, that hardware was discontinued in the fall. But the users keep using it.

Matthias Wille’s Sequencomat has gotten far more powerful since we last looked at it. Far from catching up, indeed, he argues iPad apps are falling further behind – and he makes a good case for that. So hardware and software designers, take note.

It does sync, in both directions. It sends just about everything. It can randomize steps. You need the software on the host computer, not just the controller, but put it all together and there’s some serious power here. Matthias gives us the overview:

• detailed stepvalues for octave, note, velocity, length, CC, delay, steppropability (V2 had only trackvalues for those functions)
• switchable randomfunction on each stepvalue for velocity, length, cc … very nice to variate some aspects of a pattern single and multiple track editing
• 3 clocktype: Master, slave, rewire (and – I still wonder! – my own clock is more stable than most professional DAWs) for sure independent midichannel, timing and tracklength on each track (polyryhthmic patterns!)
• 100 patterns to save and load in realtime

See the excellent overview video at top – or marvel as it works with an analog setup, below.

I asked Matthias to explain more about why he thought this was better than other solutions out there. He took a break from adding new functionality – freely-definable scales, note and octave randomization – to answer at some length.

I could edit this, but I think you’d lose some of the personality of this conversation, so here it unedited.

Lets start with some common differences:
- sequencomat is a plain midisequencer. it can only send midi… while most (all?) Ipad seq have a sample browser. That makes them more “standalone”, and thats the main concept of an app. But to me it makes no sense, cause I have my drumracks inside my DAW (Ableton), so to change the sounds I am triggering I just change the note….
- some Ipad seq also have FX section. again this makes no sense to me, cause I can control that in my DAW by Midicontrolchange (CC) or with another page on my Lemur, which gives me more flexibility.

These both features are more a matter of taste, but it points out the difference of a controller integrated in a bigger system or a standalone you can use everywhere but are restricted if it comes to communication.
Now a list of functions most (all?) Ipad seq miss and even most classical hardware midi stepsequencers have not:

- independent steprange (1-16 steps each track) and timing.
technical this means that each track got its own clock section. Musically it means you can do polyrhythmic patterns…ever overlapping and changing. On typical 4 on the floor music this is meaningless…but if you want to go more experimental…. (Moltons (?) Ipad app (that one that syncs) got something simular, based on quater sections, but only for timing, not for steprange)

- independent midichannel on each track with possibility to change during play and saved within the patterns.
technical it was hard to get rid of the midihung that can appear if you change the channel while a note is played….the “note off” (damn midiprotocol) will be send on the new channel…so I had to cut these notes first. but only these notes, not all on this channel! musically it gives you much freedom, cause while in one pattern track 1 can be an epiano in the next pattern it can be a drumrack. (well, with that freedom some confusion can come in)

- stepvalues for velocity are quite a standard…. but I got also stepvalues for octave, note, length, CC, delay, steppropability.
With “octave” and “note” you can give every step another tone to trigger (most classic hardware seq have that), if you use a well organised drumrack, changing the octave will change the drumsound (different BDs all lay on pitch C) and with changing the note you can change the drumtype (e.g. snare on D).
“Length” is also a stepvalue on some hardware stepsequ, but mostly on a discrete scale (1/4 1/2 1), while I have continious scale. You can set the maximum on the maxpatch for better fine control ranging from 1-16 steps.
You can furthermore control 8 CC-values – each track has one attached, they are boundend in timing and steprange, but not in meaning. You can set the Midichannel and Controllernumber of those independent and – guess what – these are saved within the patterns…so again a lot of freedom in routing.

With “delay” you can delay each step in triggering and therefore create a groove. Swing would be to delay every second step. But you can go much more in detail… cause you can also control the amount of delay for each step. The predefined range is 0 – -50msec, but you can set it to whatever (-2000msec?) on the maxpatch for more experimental settings. The delay of a step is also reflected in the steplight, giving you a visual impression of the groove.
With Steppropability you can set some activated steps to only be triggered in lets say 30% and therefore create some variation of your pattern. Each step independent on each track, all saved within the patterns. The stepvalues reflect more “unlikeliness”, cause the higher they are the more unlikely it is that the tone will be playsed (if set at all in the stepmatrix – sure). The unlikeliness values are compared with a random value that is triggered on 16th, 8th, 4th, 1 bar, 2 bar or 4 bars. Setting this to higher values will cause the same variation several times before changing. To give you visual feedback of the actual propability status (on/off) there are little LEDs on each step: If they are off – no tone.

- stepvalues for velocity, length, CC, delay got a “range” control on the left side. So you can control the range (e.g. 40-66 instead of 0-127) while the relative difference of the stepvalues still work. So you can fade in velocity…. Of course, that range is also saved within the patterns, independent for each track.

- stepvalues for velocity, length and CC got a randomfunction you can switch on for each step independently (!!!!). so if you want the velocity on step5 of track2 to variate, just push the little switch under the stepvalues. Or the length of step9 on track3? Or both? Or all? Every time those marked stepvalues are triggered they generate a new value. But remember – the output will ever stay within the range. (which makes a random much more usefull than plain 0-127) With this function you can surf on the border between total control and random. Thats what I love as an artist….discovering this border of controlled random. And this stepwise random is really a bomb…it makes this static thing “alive”!

- single or multiple track editing. Normally you step through the tracks and choose a function. But what if you want to change the values of more than one track at once? No prob, switch to multiple track editing, choose more than one track (chosen tracks become red) and all values you enter will be routed to all tracks. (Funny, but this concept is not common sense…maybe because with mutliple track editing you can get no more feedback – what should be displayed if the values differ?) So you can change tempo or steprange of different tracks at once (nice breaks). Or fade in the velocity of a couple of tracks with the range object. Or the CCs (!). Or even the propability if you set the random value to “manual”, this will cause fading in the “density” of a pattern.

- all of this saved within 100 patterns handled in realtime. jumps are done immediately giving you a good feel for interacting. But you can also activate a “automatic pattern chain”, like play pattern 2, 3, 4. In random order or reverse? No prob. Jumping on 1/4 bar – 1/2 bar – 1 bar – 2 bar – 4bar….your choice. You can also “exclude” single tracks from pattern jumping if you want.

-step and track mute – independent from patterns for breaks…

-a X/Y pad for controlling a CC on each axis and /or triggering notes (vertical velocity, horizontal speed (syncable!)) all with nice ranges attached to the borders to control the min and max output.

-and finally 3 clock options: master, slave, rewire.
I had rewire only first on V1 but never was satisfied with the results. Especially Abletons Midiclock (using it as master or as rewiremaster) was f***ing bad. As long as you do not reach 50% CPU power it is ok, but after that it turns unstable…sure, these are only Milliseconds…but damn, they call it “Live” !! Some of my users told me, that Cubase is much better. But I decided to build my own clock. I did not rely on max standard clock, I build it from scratch…with very nice results. Now all users confirm, that my clock as master is the most stable one. (I still find it confusing….me building a better clock than Ableton?… the background might be, that ableton drops the clock first before they drop audio, while on my maxpatch the clock has the highest priority)

So – cocky or not – if it comes to plain stepsequencing, SequencomatV3 eats them all

In a future update I will rework the pitch section: Octaves and Notes will be defined by the user. that means scales instead of 12tones each Octave. not only major or minor…nooooo…. free defineable scales – you just enter your keynote and the halftone steps. And for sure – then the random-stepvalue-switches makes also sense and will be there (I cutted it on octave and not only because it sounds so inharmonic on 12 tones)

Okay, that’s all well and good and fantastic – but the Lemur is now discontinued. So I was curious what Matthias’ plans were – would he consider a future beyond the Lemur?

Yes, sure. But not the Ipad.
I thought about going on it…. my core engine is done in max, so why not make a touchOSC surface? Because TouchOSC (as great as it is) is generations behind the Lemur. Not only physics…hey, I do not use physics in my sequencer… but many control objects are missing (range!), leds are not handled in vectors (as far as I get it), there is no light interaction independent from on/off state, no moveable containers (well, I think in the last version they added this, not sure) and so on…. so it will not be simpley changing some paths in the maxpatch – if so I would have been already there, kickin some ass – it will be completely reconstruct everything.
And I do not want to do that if I then have to sell it for 10$. This pricetag of apps makes the Ipad unattrative to me. Not because I am a greedy guy, but because it isn´t worth it. Most users need support for their midisetup. Even this support will be more effortfull than 10$. And furthermore there is still that bidirectional communication issue. The Ipads WiFi can handle over 1500 values each pattern in realtime? Hahahaha…lol, never. It is not made for that.

So instead of competing with all these Apps, I think of giving my Sequencomat a control surface directly in Max and wait for more and more touchscreens coming to couple with any PC or Mac. Just as a 2nd monitor. As my sequencomat never was ment as a standalone, this fits much better. But we will see…this will not happen within the next half year. See, I am so happy that I have my dreamsequencer here… after the next update I will chill and make some music again. Because this is something I missed all the last 1 1/2 years… having time and energy for making music again and not only coding…. (and this is also a reason, why the music in my demovideos is a bit uninspired or boring…)

I’m way over my word quota, so I’m going to leave it at that. But while sometimes I actually prefer a simpler touch device, even I think the guy has some good points here. Keep in mind that we’re talking the combination of the touch layout, the touch hardware, and then the software on the host. The iPad could certainly accomplish a lot of this (though not over an Ethernet cable), and we should assume the iPad is, in the long view, just the leading edge of a large wave of tablets.

So – discuss.

http://www.tonvibration.de/SequencomatV3.html

It’s not so often that I write obituaries for hardware, but this time, it seems appropriate. JazzMutant has announced that its Lemur, the multi-touch hardware controller, is officially at the end of its life.

Their announcement:

Since 2002, JazzMutant has been a acknowledged pioneer in the field of Creative Computing and Multi-touch technology, being the first-ever company to develop and bring to the market a product featuring a multi-touch screen as early as 2005. Since its market launch, the Lemur has been endorsed by a fascinating community of music and video artists. Nine Inch Nails, Richard Devine, Hot Chip, Ritchie Hawtin, Matthew Herbert, M.I.A, Mike Relm, Alva Noto, Ryuchi Sakamoto, Daft Punk, Bjork, … : The list of prestigious and influent artists who have made the Lemur their favorite pet companion on stage would be way too long to be mentioned here. Its visionary concept and groundbreaking technologies allowed the Lemur to win numerous international press awards and was recently elected “Innovation of the decade” by Future Music.

During five years and despite the new fever surrounding touch technologies, the Lemur remained the only Multi-touch device capable to meet the needs of creative people. From now on, this ecosystem is evolving quickly : powerful consumer tablet devices are becoming mainstream, bringing the power of multi-touch to everyone. In the meantime, JazzMutant, renamed Stantum in 2007, has become a technology-centric company and developed partnerships with tier-one industrial partners to speed up this democratization. As a result, the need for a high-end dedicated hardware is doomed to vanish in the near future. This is why Stantum is announcing today that it will close its JazzMutant activity unit and stop selling its legendary Lemur Multi-touch hardware controller at the end of December while the stock lasts! The last batch of Lemurs just came out of the factory. These very last units are now available at a special discounted price from JazzMutant’s webstore and from its authorized distributors and retailers. These very last units are now available with 25% discount! Moreover, the Dexter App and an original Lemur T-shirt will come along for free. Don’t miss this last opportunity and grab the legendary Lemur from authorized retailers while the stock last! The Technical support and after sale service will be handled until December 31, 2011. The jazzmutant website will stay online in order to let the user community access support resources and share their projects.

We would like to thank all the people involved in this fantastic adventure: first of all the user community which exceeded our wildest expectations in creating the most amazing templates; our devoted resellers, who helped us to show the Lemurs all over the world; finallymusic software editors for their support.

http://www.jazzmutant.com/press_release.php

None of this is necessarily news. To me, the surprise is that the transition away from a dedicated multi-touch controller to widespread multi-touch took as long as it did. (Well, actually, it isn’t so much of a surprise in retrospect, but perhaps the Lemur made the coming transformation so vivid that I hadn’t thought through just what that transformation would take to come to pass.)

At the end of 2005, writing a review for Keyboard Magazine of the Lemur, I concluded that powerful as touch was, it should be viewed as a complement, not a replacement, for tactile hardware – a tool ideal for certain tasks:

The Lemur is an unusual piece of hardware. Because it rethinks fundamental questions about what music hardware should be, it raises questions we normally take for granted. But it also suggests that conventional hardware interfaces aren’t as arbitrary as one might think.

If anything, I think at the time I wildly underestimated some of the Lemur’s most important contributions:

• Simple, geometric, high-contrast user interface elements improved legibility and usability.
• It pointed to physics-driven touch interfaces which still haven’t been fully explored – even by the likes of Apple.
• It demonstrated how important multiple touch points could be – not just two, or three, but the same number of touch points for which you have fingers.
• It led adoption of OpenSoundControl (OSC), leading to more intelligently-labeled controls, network-based control schemes (whether Ethernet or WiFi), and higher-resolution data.
• It showed the usefulness of floating point control precision, particularly in the visual space.

But more importantly, in my Keyboard story I said I felt that the Lemur would ultimately be replaced by computers that had touch, rather than dedicated touch controllers:

There’s no question that multi-touch touchscreens represent the future of computer interfaces, and the Lemur is the biggest leap yet toward that science fiction future. For now, the challenge is that the Lemur’s features lie somewhere between a computer display and music controller, without effectively supplanting either one. The Lemur sacrifices the sensitivity and tactile feedback of physical controls in the name of flexibility, but that payoff is limited by the restrictions of its pre-built interface objects and the difficulty of configuring new layouts and assigning them to software controls.

If the Lemur could be truly fused with the computer display, rather than requiring an entirely independent interface, it would become a must-buy. Until that happens, the Lemur could be a worthy acquisition if you need more flexible control of parameters like timbre and surround sound, or want a programmable interface you can touch, and can afford paying a premium for emerging technology. But for most of us, less-expensive and more musical physical hardware will remain the preferred way of interacting with the virtual worlds of computer sound.

In fact, in one odd sense, the Lemur’s separation of control from sound source has hindered the fusion of sound and interface on devices like the iPad; control surfaces remain arguably more popular than dedicated musical instruments and production tools that take advantage of the new touch paradigm.

I imagined that the fusion of touch displays with computers was a couple of years off; it was actually closer to six years. It really took the debut of Apple’s iPad this year – some five years after the Lemur’s introduction – for us to see the computer fused with the touch interface. And I think the year of the tablet is more likely to be 2011, as Windows, Linux (MeeGo and Ubuntu), Android, and presumably Chrome tablets all hit the marketplace. (True, we’ve seen computers with touch, but they’ve all made compromises that prevented them from even matching the Lemur, either in sacrificing performance, adding cost, losing multiple touch points and resolution, or some combination.)

Some of the coming models will seem more like conventional computers with touch displays, others more like tablets in the mold of the iPad.

What remains to be seen is whether we’ll see Stantum’s technology – technology derived from the Lemur – in some of those devices. (See my April interview with JazzMutant’s co-founder.) No new information there. Update – it bears saying, based on what I see in comments: we already know that there are a lot of tablets and touch-equipped laptops and specialized devices coming to market in 2011 and beyond. There are more devices, from what I’ve seen, than there are vendors of touch technologies. That makes this a very desirable marketplace, and could explain why JazzMutant are in no hurry to open source their code. It’s a safe bet that Stantum is still making a play for that enormous market, with everything from general consumer electronics to specialized gear for certain industries in the mix (and not just music). I don’t have any new information, but I expect when they’re able to make something public, you’ll know.

As for the Lemur touchscreen, though, it is now committed to computing history. The present is, primarily, the iPad, and the future, from Apple and others, offers multitouch as inexpensively and seamlessly integrated with computing hardware as the trackpad, keyboard, and mouse have been in the past. The knob, the piano keyboard, strings and frets, and other tactile interfaces live forever, but touchscreens can at least be powerful options for the digital realm between tactile musical control and composition, as a more direct way to reach out and touch interactive interfaces for sound.

For a long time, technologists have described a world of in which computing experiences naturally incorporate touch and gesture. The question is, how do we bridge the intuitive desire for those interactions and the actual technologies that get us there?

Few activities test the expressive potential of interaction quite like music. It’s in our cultural DNA; musical activity may even predate written language. So it’s fitting that the story of touch in computing and digital music would be intertwined, as they are with touch pioneer JazzMutant. Years before well-known Apple products, the Lemur, prototyped in 2003 and shown as a musical multi-touch screen, suggested the importance of fusing display and touch, and of tracking more than a finger or two at a time.

The history, and products like Apple’s iPad and iPhone, you may know well, though. The question on everyone’s mind now is, what’s next? (And for some impatient futurists, the question may even be, what’s taking so long?)

To begin to answer that question, I turned to Guillaume Largillier, original co-founder and CEO of JazzMutant, now Stantum Technologies. There aren’t many people on the planet closer to where touch has been and where it might be going. Even as the Lemur gets new features like integration with popular music production and performance tool Ableton Live, Stantum is working to bring the same enabling technologies to other device makers. And even though this is “Create Digital Music,” it’s telling that that technology is showing potential in everything from phones to aviation, not just DJing. Musicians have had a role in technological history before, from Leon Theremin’s work to Max Mathews and computer synthesis. It may be musicians who invent the future, again. This time, the trick is who delivers that future to the hardware makers who can popularize it.

http://jazzmutant.com/
http://jazzmutant.com/behindthelemur.php
http://www.stantum.com/en/
http://www.stantum.com/en/offer/technology-ip

To accompany the story, we also have an exclusive look inside Stantum’s labs, all the way back to the original 2003 prototype of the Lemur.

On Designing for Touch, and the Music Tech Industry

Peter: I remember when I first talked to Darwin Grosse about Lemur, when it was being distributed by Cycling ’74. Darwin just kept saying, “You know, I just think Star Trek: The Next Generation.” (That’s my recollection, Darwin; I hope I’m not misquoting you.) I tended to agree. It’s a cliche, perhaps, but this was clearly hardware that brought into our century part of an imagined vision of a much further-off future (the 24th Century). Was that a conscious influence? In an industry that has sometimes been aggressively traditional, is there a way to channel ideas from something as far out as science fiction?

Guillaume: Before answering your question, allow me to challenge your statement about the computer music industry. I think “ill nostalgic” would describe this industry much better than “aggresively traditional.” Most music software companies have kept being innovative over the last decade, but their creativity has been a slave to this nostalgic obsession. Emulating an analog channel strip, a tube amplifier, or a vintage synth is far from a trivial job. It actually requires as much engineering time and resources as developing a disruptive product such as Ableton Live or Max/MSP/Jitter! On the hardware side, the innovation killer is the price pressure. Despite a common misconception, the computer music industry is not and will never be a mass market. Companies such as M-Audio [Avid], Behringer, or Native Instruments may look like giants compared to JazzMutant, but they are nano-particles compared to large consumer electronic brands such as HP or Nokia. The volume and the gross margins are too small to amortize ambitious research and development plans. When we launched the Lemur in 2005, a lot of people predicted, and somewhat hoped, that Behringer would release a similar device at $200 within the next eighteen months. Five years later, the first serious competitor of the Lemur is about to land – Apple’s iPad – and its entry level price is $500.

Back to the USS Enterprise, whether we want it or not, this parentship is likely to follow the Lemur forever. This is kind of ironic insofar as I’ve never been acquainted with science-fiction culture. I don’t even remember having ever watched a full episode of Star Trek. That being said, I acknowledge that this association has settled spontaneously and durably in people’s mind. Does this association come from the product concept itself? I don’t think so. In my opinion, it comes first and foremost from the fluorescent graphic design of the UI objects, not from the tactile technology.

So, the real question would rather be: “Why did we design the graphic interface this way?” First, we wanted to stand clear of those boring pseudo-vintage brushed-aluminium graphic skins – the cutaneous symptom of the nostalgic flu! Moreover, we anticipated that converting users to virtual controllers would be a difficult task and that trying to mimic the appearance of real-life objects would generate frustration; hence, impeding the adoption of the product.

Having said that, the main purpose of this flashy design was pragmatic and ergonomic. The Lemur is ontologically a live controller, though it might be used in other contexts. This requires that the interface must be visible wherever and whenever a user might be performing, from night clubs to outdoor venues. This is particularly tricky with a touch screen laid horizontally, because the display backlight cannot compete with the specular reflection of sunlight. Human-factor sciences taught us that contrast perception prevails over brightness perception. Hence, highly contrasted graphics- ie, flashy objects on dark background – is the most efficient way to ensure a consistent readability. This is something the aerospace industry has understood for decades. So, if there was one conscious influence behind the Lemur, it would be the Boeing 747 dashboard, not the USS Enterprise.

“If there was one conscious influence behind the Lemur, it would be the Boeing 747 dashboard, not the USS Enterprise.”

I know for me, the appeal of the science fiction aspect was more conceptual than superficial, the idea of the ubiquitous touch interface. But I agree, having experimented with this, that the high contrast, light foreground, dark background formula is really an essential solution. I’m seeing some interfaces on a white background that look aeshtetically lovely, but that I can’t imagine using onstage. I’d at least want a switch for dark environments, when you’re not at your desk.

That reminds me a funny episode of JazzMutant’s story. As early as February 2004, we started showing early prototypes of the Lemur to our friends at [Paris sound research center] IRCAM. At this time, the graphic skin was based on a palette of blue shades, with a few touches of warm yellow for emphasizing elements that needed to stand out, such as text, levels, etc. One day in July of 2004, about one year before the commercial launch of the product, we brought them a new prototype, featuring a brand new touch panel along with the final graphic design. Their only reaction was, “wow, this display is much brighter!” They did not even comment on the tremendeous improvement to the touch panel! That being said, there are other approaches to improve the psychological perception of readibility. I sometimes regret that other developpers are reluctant to dig into them, and mimic the “Lemur style” instead.

Talking about drawing on screen, did you know that Iannis Xenakis’ Upic project has been my main source of inspiration – and also my main motivation to step from music making to technology creation ?

I didn’t know that, but it makes a lot of sense. [UPIC is a tablet-based, visual composition system developed by ground-breaking experimental composer Xenakis. It is now decades old but continues to evolve in new incarnations.]

Below: DJ Mike Relm demonstrates the Lemur for G4 Tech TV. Yes, this is the video to show all your friends who aren’t regular CDM readers and have no idea what the heck this is all about.

Lessons of Lemur

Let’s talk about the place of Lemur’s technology in the current landscape. How does it hold up in 2010? I know a lot of people do get hung up on the price, but can you talk about how it differs from other options out there, or what the source of the cost is?

Once again, the music market being a small niche, it’s hardly possible to be both innovative and affordable at once. In addition, the Lemur is still manufactured in France with components imported from various locations around the globe – not to mention that the US dollar’s agony doesn’t help [when exporting] the manufactured product! Lastly, a large part of the product assembly is still handcrafted. For all the reasons above, the product is far from cost-optimized. I cannot disclose further our plans now, but we are working hard to address most – if not all – of these issues.

Have there been uses of the Lemur in performance and creation that surprised you, or went beyond what you imagined?

Oddely enough, and despite of what I said before, the most surprising uses of the Lemur are sometimes the most conservative ones! As an example, for Björk Volta tour, Damian Taylor and LFO made the most archaic interface layouts one could imagine — a fistful of colored labelled pads and eventually a pair of faders –- nothing more. Their brilliant idea was to create one unique interface for each song. This way, at each moment of the gig, they just had at their disposal the few commands they did actually need. The other big surprise came from video performers. Whereas most musicians are reluctant to use the advanced features of the lemur during their live performances – such as the objects’ physics – video artists do not hesitate to play the Lemur as an instrument, rather than a remote control. For instance, I warmly recommand you to visit Ali Momeni’s website. Of course, it would be unfair to forget all the advanced users who have developed inspiring and unique instruments, but this is less surprising, since the Lemur was designed specifically for that purpose.

OSC is a technology that many of us have advocated, but there’s also, admittedly, a big gap between where we believe it could be and where it is, especially in regards to the lack of mainstream music tech adoption. That said, what would an ideal implementation of OSC look like? What could the protocol do to be better? And what might you imagine could be a tipping point in adoption?

Indeed, it’s fair to say that OSC failed to become the industry standard we hoped it will be! I can see a few reasons for that. First, there is an obvious chicken-and-egg issue, as with any protocol. At JazzMutant, we’ve done our best to evangelize OSC in the industry for about 5 years now, without success. Why should a software company implement OSC if there is no hardware to support it, apart from a $2k product? Why should a hardware manufacturer develop an OSC-compatible controller if there are no mainstream applications to support it? Finally, there are also some intrinsic technical reasons that prevent OSC from becoming a standard anytime soon. In order to overcome them, we started developing a new protocol a few years ago called “Minuit” (“Midnight” in French), as a successor to OSC and MIDI (“Noon” in French). We were discouraged from pursuing this project after assessing the amount of human resources its evangelization would require.

The Big Picture, Stantum, and the Future

We’re looking at an explosion of interest in multi-touch display surfaces in the consumer space. Are any of these, in your view, promising for music? Are there ways in which some of these technologies are deficient for musical performance applications?

The responsiveness of a touch system is the most under-estimated parameter, even though it tremendously influences the perceived usability, transparency and trustworthiness of an input device. This is why a vast majority of Multi-touch systems available fails to meet music makers’ expectations.

Absolutely — you mean responsiveness in terms of latency, accuracy, precision in tracking multiple points, or (I presume) all of the above?

I was pointing out the latency more specifically – even though the perceived responsiveness is a complexe imbrication of all these parameters.

Can you talk about Stantum’s role in the evolution of multi-touch? What can we expect to see in the future?

We envisioned the real potential of the technology we invented long before the iPhone announcement, though we could not imagine that Steve Jobs’ crew would accelerate the market demand [to the extent they did]. We started investigating how we could bring our technology to OEMs in parallel to our computer music activity as early as 2005. We finally made this step in 2007. The role of Stantum in this ecosystem is quite singular. However pretentious it may sound to you, Stantum is still the only company beside Apple to have developped a real multi-touch product, top-down, including all the software and hardware technology bricks. So, despite the small size of our company, we are better placed than any other player in this field to understand the complexe imbrication of software and hardware. You might ask, “Aren’t all these Windows 7 convertible notebooks real multi-touch products?” In my opinion, they are not, insofar as the only multi-touch services these devices offer so far are rotating videos or ten-finger painting. I do not want to offend anyone, but watching videos is much more pleasant fullscreen and if Neanderthal people gave up painting with ten fingers 45,000 years ago, there might be a good reason. At JazzMutant/Stantum, we’ve always considered the multi-touch technology as a milestone, not the final destination. With what we’ve been incubating in our labs for a few months, we expect to reach the next big milestone quite soon.

Do you mean that these PC vendors are missing the actual application of the multi-touch technology in the software they ship with these devices? Certainly, no argument there — the demos, the marketing, the demo apps outside of Apple have just looked horrendous and awful to me. But surely there are developers out there who want to do better? Hasn’t what’s held them back simply the lack of available hardware?

I do agree with you. Unfortunately, that leads to a chicken-and-egg situation; insofar as developing a meaningful, multi-touch-capable application requires a preliminary awareness of the objective capabilities and limitations of a given hardware solution. On the other side, a vast majority of multi-touch panel providers doesn’t look willing to raise the bar until the market identifies a “killer app” requiring full multi-touch capabilities with zero performance tradeoff. Hopefully, the iPad will contribute to reschuffle the cards. Unfortunately, Apple decided to stand clear of handwriting capability – which, I believe, is a huge limitation for creative and productive applications.

“I do not want to offend anyone, but … if Neanderthal people gave up painting with ten fingers 45,000 years ago, there might be a good reason.”

Let’s assume Stantum is successful in popularizing the technology. At some point, will the Lemur be obsolete – and could that perhaps even be a good goal?

The Lemur as it is today is likely to become obsolete at some point – the pet is more than 5 years old in an industry that usually sends hardware products to retirement manu militari at 18 month old! Having said that, there is much more to develop on the hardware side than what we have done in the past. If we succeed in what we are working on today, I believe the Lemur will keep playing in its own category for quite a long while.

Now, that said, how do Stantum’s efforts to engage the larger electronics industry impact these issues of scale and cost?

We understood as early as 2005 that there was only one path to spread this technology – and the underlying vision of how computerized equipments should work – out of the small niche of professional musicians and Max/MSP users. Then we did what we had to do : we licensed the technology to tier-one semi-conductor companies such as ST Microelectronics to embed our multi-touch know-how into dedicated chips. We also teamed up with some of the largest and most trusted touch panel makers to bring our solution onto the consumer market place. The whole supply chain is now in place and you’re likely to see a few Stantum-based multi-touch tablets shipping in the coming months. Will these products match musicians’ expectation ? That’s too early to risk an answer at this stage, since we have no control over what OEMs will make out of our technology. And as you know, a good user experience does not only depend on the quality of the touch system – it’s also a matter of CPU and OS choice, hardware optimization, not to mention the software application running on top of it. That’s why we believe there’s still some room for a dedicated hardware that takes in consideration the very specific needs of electronic musicians and visual artists. In a not-too-far future, we expect the hard work we have done with our partners will have a positive impact on the cost structure of our music products.

Stantum’s Latest Technology, and What it Means

Guillaume is a bit limited in what he can say about his future plans, but that leaves me free to do a bit of (informed) speculation. This is largely my own analysis, so it’s my message, not necessarily Stantum’s.

First, unless it isn’t already clear, JazzMutant is Stantum. Stantum is JazzMutant. Stantum is now the official name of the company, and JazzMutant is just the brand by which their technology caters to musicians. It says something about the company’s lineage – all the founders have a background in electronic music – that they have in the past, continue now, and plan in the future to keep a strong connection to musicians. That’s meant that the rigorous demands of live music have informed their touch technology and made it a better product.

The idea that Apple’s iPad would drive JazzMutant out of business, therefore, is the opposite of correct. JazzMutant is Stantum. Stantum is in the business of licensing its specialization to OEMs. The Lemur shows just how potent that specialization is, in a way that literally gets rooms full of people dancing and gaping at projections. Apple’s technology is available only to Apple. With Microsoft, Google, phone vendors, and PC vendors all getting into the touch business, that means Stantum just became very big news – even if that’s something musicians and VJs figured out years earlier.

Part of the challenge of multi-touch development is that you have to get a lot of pieces working together. You need the physical surface of the controller, the sensors built into that surface, and the firmware that interprets the sensors all to work in tandem. Apple does it, and does the OS and applications, too. 3M is working on a product for OEMs, also working with multiple touch points. But the other big source right now is Stantum.

It’s also significant that Stantum’s technologies are heavily patented (a fact that they advertise on their site). While I’m no big fan of patents, unlike Apple, Stantum is licensing their technology into the marketplace. Given the need to have a patent portfolio just to protect your work, Stantum’s patents give it effectively the right to play ball. By licensing their technology to the manufacturers big enough to make this stuff on a grand scale, Stantum’s OEM program could provide ready access to touch for software developers beyond just the iPad platform. Even if you’re a huge iPad fan, that means greater accessibility in the market, and more than one vendor to provide that access. I’m a great advocate for DIY, but making displays isn’t yet a garage operation. (Yes, I know people building their own multi-touch tables, but they don’t make their own cameras or projectors.)

Stantum’s technology itself is also unique. Their sensing approach supports pen input and even handwriting recognition, features Apple leaves out. For many of the world’s languages, handwriting recognition is a “killer app,” which could further drive touch adoption. For the rest of us, until we evolve smaller fingers, the ability to use a pen can mean amplified accuracy for painting and writing, and yes, even pen-driven music applications. (Somewhere in the great beyond, Xenakis smiles.)

This is not an advertisement for Stantum, either – the list of companies anywhere close to being able to provide this functionality is short enough to count on your (ahem) fingertips.

So, okay, you buy into the concept – when can you get it? (After all, even the Lemur doesn’t quite count. It isn’t set up for pen input, even if its sensing method could work. And the Lemur is a controller, not a computer.)

Right now, Stantum’s technology is available in a series of multi-touch demonstration kits, including one with the guts of a Dell netbook inside:

http://www.stantum.com/en/offer/evaluate

In other words, we’re waiting for someone to ship a product that incorporates their technology. Windows 7 already includes multi-touch APIs out of the box in all but its Starter edition, so the Windows platform is a major candidate. Windows, while proprietary, has none of the developer, language, software, or hardware restrictions that the iPad platform does, so if your application doesn’t fit the iPad model or needs pen input, Windows’ stock just rose. Free software is possible too. Linux already supports the Stantum Slate PC and a number of other digitizers, support that will be baked into the kernels shipping in this year’s major Linux distros. We’re not just talking drivers, either: the whole Linux community is working on everything from libraries for environments like Java to support in the windowing system to touch-centric distros. (More on the Linux situation later this week.) Google’s Android has a multitouch API, too. I’ve used it, and got frustrated quickly not because of the OS, but because the hardware on current phone handsets just doesn’t work well with more than one finger. That could change if Stantum’s tech starts to appear in licensee products; Android as a touch OS could take off.

For specifics on the Windows 7 aspect (old news, from way back in November – but of course, everyone is taking a second look because of the iPad phenomenon):
2009-11-03 Windows 7 Certification

Right now, the one thing Stantum doesn’t have a lot of – aside from OEMs shipping their tech – is competition. Most of the other touch competitors either can’t accurately track fingers in close proximity, or limit tracking to two fingers, or lose tracking fidelity around the edges of the screen, or can’t handle pens, or some combination.

You need musicians, creative artists, and gamers to tell you this, because the mainstream computer market thinks multi-touch has something to do with pinch-zooming their photos. If that were all you could do with multi-touch, this would indeed be an over-hyped technology. But the responsiveness of the Lemur and the demonstration technology from Stantum is something that can be powerful and expressive.

Apple has already brilliantly demonstrated what happens when scale, creativity, and technical competence meet. Now the question is, who else will be able to put this formula together, thus making other options available to developers? Stantum has the competence, and the connection to creative artists and music specifically. If OEMs start to sign on with Stantum’s tech and build useful hardware, we could see both off-the-shelf machines – and cheaper JazzMutant-branded products – for musicians. Indeed, with this larger Stantum perspective, whatever happens with OEMs could in turn be good for JazzMutant-specific, music-specific customers, too. Even with competition from the likes of 3M, the technology is so specific to certain hardware devices, and the emerging markets so large, it’s hard to imagine Stantum not having a big role.

What might surprise people in the larger tech world is how important music has been – and will continue to be – to the big picture.

When it all comes together, the days of computer musicians, DJs, and visualists standing behind screens, able only to stare blankly into them but unable to manipulate what they see directly, could become a relic of the past.

No doubt about it: touch is coming to more screens near you. But there’s no need to disappoint your current beloved laptop. $200 kits can turn your laptop into a functioning touchscreen.

Now, as I’m working with JazzMutant’s Lemur this week, before you get excited, this is no Lemur – or even anything like your iPhone or iPod touch. Sensitivity and accuracy are workable, but not exceptional, the overlay is pretty simple (as you can see in the video) rather than integrated with the display, and this is single-touch only — not multi-touch. Lastly, on a conventional laptop that isn’t convertible, you may miss the ability to fully extend your laptop perpindicular to your body. (Having the screen be parallel can put your arms in a fatiguing position.)

But that said, there’s a lot of potential once you have the ability to reach over and make quick gestures on a laptop screen that control a set. You might make your own instruments and effects or controller dashboards in a tool like Processing or Reaktor. And at $200, this could be a brilliant way to retrofit a machine and breathe new life into it. There’s support for Mac, Windows, and Linux; you just plug in via USB.

In this case, Sensomusic Usine is perfectly suited to the job, with an interface built just for this purpose. Their Touch Screen Edition earns major kudos for being a full desktop computer music environment built around touch, as covered here previously.

Of course, all of this potential is likely to make you want more sophisticated, multi-touch solutions even more. The outlook is improving. Windows 7 will bring native support for multi-touch gestures – not so much important news in itself so much as a sign that more hardware vendors could add support, ramp up volume and lower prices. HP is already shipping computers (including laptops) with multi-touch.

Also interestingly, the creators of the JazzMutant Lemur multi-touch hardware – specifically designed for music and visuals – have now expanded their mission to targeting general-purpose devices. The new company, Stantum, is showing off fantastic, unique technologies for multi-touch, as seen recently in an Engadget preview. Beautifully designed as the iPhone is, these offer some unique features like intensive accuracy and support for input from objects (like styluses) and not just fingertips. That could mean the Lemur is just the tip of a much bigger iceberg.

Here’s a tantalizing look at the Santum SMK screen working with Max/MSP. Unlike the Lemur, that means direct controlling Max’s widgets, rather than treating the screen like an independent controller.

Of course, that doesn’t mean the simpler, single-touch, cheap solution couldn’t be a great project right now, and a chance to get a leg up on The Future. If anyone tries one, let us know what it’s like.

Even online, words have a tendency to linger long after you write them. And I recognize that the risk here is not only what those words mean to me, but others, too. So I want to revisit a topic today in the interest of moving forward.

I wrote a kneejerk post earlier in the life of CDM about MonoTouchLive, a single-touch interface with UI widgets inspired by JazzMutant’s Lemur. MonoTouchLive was (and is) Windows-only, standalone software for controlling Ableton Live. It’s now free of cost.

I’ll be honest: in my original post, I overreacted, and I didn’t choose my words as carefully as I should have. Some regular readers called me out on it at the time. Since that time, I’ve tried to be more careful. Some comment threads referred back to it, though, and the developer, Pablo Martin, has continued to push his tool and has been outspoken about not liking what I wrote.

It’s now clear to me that I can’t just let this go, as Mr. Martin today has posted a multi-page diatribe focused largely on my short, now nearly three-year-old blog post.

A little of JUSTICE please

To be clear: I got carried away. I have since come to the realization that copying – loosely or closely – isn’t such a bad thing. They’re a learning process, and if something really is original, it does tend to shine through.

Also, to my knowledge, Mr. Martin is correct: the layout on the Lemur was apparently a mock-up to show what the Lemur would look like if configured with the layout of the MonoTouchLive. I incorrectly said that both the widgets and layout had been copied, as I misunderstood the image I saw. Axou created that design as part of a thread with some complaints about the similarity to the Lemur widgets, but I now try to make CDM less like a quick forum post.

I don’t think there’s anything really wrong with MonoTouchLive. It may indeed be useful to some people, and I expect (as I even said in that original, ill-advised post) that the growing availability of affordable touch and multitouch screen hardware will mean many more such creations and ideas. DJTechTools even suggested a US$250 solution involving something like MonoTouchLive and a small, USB-powered external display:

The $250 Lemur

There are some of Mr. Martin’s remaining contentions, however, with which I still take some issue. First, he implies that this was some sort of ongoing thread on CDM. It wasn’t; there were two stories out of several thousand, but as is clear from the URLs he copied, they were posted on the same day, September 11, 2006. I don’t know what was going on with me that day, but I should have thought before I published. I understand his frustration, though – because I didn’t write a follow-up post, those stories continued to come up on Google. But, at least I can say truthfully, I didn’t mean for this to become an ongoing issue.

Second, while it’s true that MonoTouchLive didn’t copy a Lemur layout, it did clearly copy the look of the Lemur interface widgets. My original appeal, if poorly worded, was for creativity and variety, and I stand by that now. Mr. Martin is clearly a developer with some skill, and I’d love to see something that looks different. I stand by that criticism now as before – and because I’d love to see new visual ideas.

It’s worth noting, in fact, that I wasn’t alone – jaded, perhaps, by people copying designs instead of making their own, many in the Ableton forum responded the same way. The lesson here is, if something is a copy, you may want to acknowledge that and explain your intentions. The lesson for the rest of us is, we probably should have considered the author’s intentions before responding.

Lastly, while “JUSTICE” claims Dexter “saw the light” of MonoTouchLive’s fixed layout, I still like the idea of modular, editable layouts over fixed layouts. It’s part of the whole advantage of a touchscreen over hardware. The “JUSTICE” article also implies – intentionally or not – that JazzMutant was following MonoTouchLive’s lead, which I think is unlikely. That said, I’m sure MonoTouchLive could be useful to someone with a fixed layout, and they have their advantages – muscle memory, for one.

What Really Matters to Me: Looking Forward

I won’t say anything more about this matter. I think what JazzMutant did was really important, and I think MonoTouchLive is a worthy idea. But what I’m most interested to cover in the future is work that takes touch in new directions.

For touch and multi-touch to really catch on, I think these interfaces need to be substantially different in function and experience than conventional hardware knobs and faders. I’d aim that criticism at JazzMutant as much as I would at MonoTouchLive. I’m really eager to see more experimentation. I shouldn’t have pushed that desire on Mr. Martin and his work, though, and for that, I’m sorry.

Just complaining in this case was out of line. I believe criticism and frank criticism is part of the function of blogs – not pulling any punches, as the saying goes. But I also believe in being constructive, and I think those comments weren’t.

I expect I’ll continue to make mistakes, and I will be the first to defend the value of writing frankly on the Web – warts and all – and responding. It’s because of a few years of doing that, in fact, that I think I have a different perspective in 2009 than in 2005 and 2006 – because of your feedback. So always keep it coming.

I don’t view my role as a cheerleader, in case that isn’t already clear. But because I believe in the value of criticism, I’ll be working harder than ever to make sure it’s the right kind. Comment threads are comment threads, but I hope we’ll all consider that we’re leaving words for strangers around the world.