With Web data providers offering generous cash prizes and a strong emphasis on harnessing data to transform listening, music consumption took center stage at Music Hack Day’s debut in New York. But it was novel music controllers, the sort that once were commonplace only at academic music conferences, that stole the show. That suggests that whereas building the next MySpace was once the hot music tech, the future might look more like a race to build the next Theremin.

Whatever the cause, the event proved just how productive hotshot DIY coders can be when left to their own devices and given ample sources of electricity and caffeine. The weekend marathon has now been exported to nearly a dozen installments in Europe and the US, though this was its first appearance in the boroughs of New York City. The result: nearly 200 participants, hundreds (yes, hundreds) more on a waiting list, and over 70 projects completed in a weekend. From just Saturday afternoon to Sunday afternoon, programmers working with Web and desktop technologies whip up quick software creations. The emphasis is on “hacking” for a reason: there’s no time to second-guess or obsess over quality, or indeed to waste a moment conceptualizing. This is all about making a working product, trying out an idea in practice, mashing together whatever is most accessible as rapidly as humanly possible. Sure, there aren’t any hard, fast rules against bringing in previously-prepared tools. But make no mistake: very much that was live in a demo Sunday was pure theory just twenty-four short hours earlier.

Coders laid out cushions on the floor and packed toothbrushes. Some were local, but others were still bleary eyed-with jetlag from trips across the Atlantic. Hopped up on coffee and Red Bull (and then beer), they coded projects that often had nothing to do with their employment – even those who came on the dime of some of the Web companies. Nor was there a lot of fishing for venture capital or IPOs. Most gave away code (if they could bear to let anyone else see it) on public code repositories like GitHub, and listening to coders, many even blatantly ignored the promise of cash prizes. It was programming for love.

Here are few of the most promising projects, and a few noticeable trends. If generating automatic playlists or finding music videos that match tastes of friends on Twitter isn’t your cup of tea, don’t despair. We had alternative instruments and music-makers, too – and, take note, they generally took home the cash.

Invisible Instruments, Made with Gestures

Invisible Instruments, the winning hack by Tim Soo, began at the Boston event. I think what made it so compelling – the voting was done by the entire audience, entered via SMS – may have been the recognizable instrumental metaphors. Using Max/MSP and OSCulator, a Wiimote, and iPod touch, the instruments emulate a violin, drum pads, and

Now, none of this is news to regular readers of this site, of course. But that should present another lesson: if you’re doing this kind of cool stuff, you should tell the sorts of people who don’t normally pay attention to such things (even, very often, tech-savvy folks). Music tech involves all sorts of wildly cool things that we’ve inadvertently kept a secret. Let’s change that.

(Or, to put it another way, apparently the whole world isn’t reading this site. If you want to help us with that, let me know.)

Previous videos / project work:
Invisible Instruments

Note that Tim does say, “Scout’s honor,” that he built new invisible instruments just this weekend. And you can grab these and older patches from his site.

Plucking Strings and DJing with Kinect

Just got a Kinect? Want to make it do something? What better than a couple of coder friends to make it happen? The three-person team that worked on Stringer, a musical instrument for plucking strings controlled with Processing, wound up easily paying for their Kinect hardware by pocketing some change in prizes.

Participants Aidan Feldman, Tyler Williams, and Alex Chen contributed. In the process, they found that using a camera to simulate string plucking wasn’t entirely effective; they didn’t have enough tracking intelligence to tell the difference between a pluck and a motion near a string, so wound up going for simpler reactivity. The clever string animation works wonders to make you feel like you’re playing real strings, even with samples, however, and it’s amazing how much they accomplished and learned in a short space of time.

The Processing libraries aren’t quite as complete as some C++-based libraries, but they’re a good place to start. If you’re considering doing something similar, I recommend my friend Dan Shiffman’s posts on his library contributions:
Kinect and Processing
Updated Kinect Library for Processing

And by the way, this work was an extension of the strings featured in Alex’s excellent New York subway sonification, about which we I to interview him:
Music Made with NYC Subway Schedules; HTML5+Flash, Q+A with Artist-Developer

Another Kinect hack: Matt Gattis produced the Bionic DJ project with “Kinect, libfreeconnect, and the OSC MIDI protocol.”

Beat Grids and Sine Waves with ChucK

I unfortunately don’t have good documentation of Jordan Orelli’s project, but he has some fascinating ideas. I laughed and said what he did was build a DIY Tenori-On with a Novation Launchpad and ChucK, but it is actually unique.

The grid of the Launchpad is a pitch sequencer – that we’ve seen many times before, and it’s very useful. But the grid can also become beat-synced modulation, which makes it possible to do some lovely, rhythmic manipulation of sounds.

The top row of the Launchpad is used for selecting instruments. The rightmost column selects “modes” specific to that instrument. The grid controls the current mode. All instruments run concurrently, so you can reasonably have a rack of 7 instruments, with the 8th instrument slot being reserved for the “mixer” instrument, which doesn’t actually mix anything but it lets you change the tempo (generally crashing the application in the process).
Everything is written in ChucK and no samples are used.

http://wiki.musichackday.org/index.php?title=ChucKPad

The SMS DJ

DJs may want to replace the crowd members making requests. Tough – the crowd may just ditch the DJ for a robot.

There were a number of crowd-sourced playlists ideas, including one cleverly named Youzakk and hooked into location check-in service Foursquare.

But djtxt was, amazingly, a whole service built in a weekend, complete with slick user interface. To make it work, it uses a whole lot of services: Twilio for SMS connectivity, Grooveshark for playback, Last.FM and musXmatch for albums and lyrics, and many others. Full details:
http://wiki.musichackday.org/index.php?title=Djtxt

And they did the other thing widely-respected by Web geeks: they deployed to a live site.
http://djtxt.me/

Drum Loops, From Your Browser to SoundCloud, and More HTML5

Two big trends emerged that are relevant to anyone interested in making music in the Web browser – without necessarily giving up your “real” (read: traditional desktop) production tools.

One: HTML5-based Web tech, while not entirely polished yet, is indeed enabling some basic music functionality right in the window of modern browsers.

Two: things like SoundCloud connectivity mean you’ll be able to generate quick ideas and then download samples later. (Ableton Live made a number of cameos in the afternoon demos.)

One great example of that is PatternMusic. It’s a pretty terrific little drum machine. But Ghostly International’s Haig and Miguel, who began the project in last summer’s Visual Music program at Eyebeam (in which I was also a participant), made a big leap forward this weekend: SoundCloud export. In turn, Haig worked out how to make PHP wrappers for SoundCloud much simpler and more effective. That’s a hack I hope we get to share soon.

http://wiki.musichackday.org/index.php?title=PatternSketch

http://patternsketch.com/

Also very cool: battling beats at SoundCloud beat battle. Match your groove-constructing skills against Ghostly’s Miguel or Com Truise. You’re going down, Truise, no matter how cool you are.

http://patternsketch.com/battle/

CDM Coolest Hack: Vib-Ribbon Clone

For the uninitiated, Vib-Ribbon, a Japan-only masterpiece by music game innovator Masaya Matsuura, is one of the high water marks of music games, a trippy walk through cartoon lines animated by sound.

Vib-Ribboff by Robert Böhnke cloned that game entirely in the browser, using SoundCloud music and intelligence engine Echo Nest for analysis, all with JavaScript frameworks Coffeescript and Raphaël.js. It’s a sharp parody of the original, and the sonification works. It’s too bad lawsuits exist, because otherwise it could become the most popular feature of SoundCloud. Can’t someone, like, license this?

CDM Funniest Hack: Faux Geocities Fans

Fans Forever and Ever cracked up the audience with a brilliant, generative version of horrible fan pages. It even fakes the awful GeoCities-era HTML and creepy, stalker-ish poetry (see screenshot). I hope this actually shows up online.

CDM Underdog Bet: Music Notation

Trendspotters no doubt got into the crowd at the hackday. (Famed venture capitalist Fred Wilson was there, for one.)

Trend they almost certainly wouldn’t spot: the likely growth of music notation on the Web and tablets.

Only two hacks really capitalized on this – one a score follower, and the other, pictured here, live annotation. But recall that, alongside the better-publicized MP3, guitar tab was an early hit of music on the Web. (Yes, it made music publishers and copyright holders grown, but that misses the point: huge swaths of the public consume notation.)

The reason is this: even as music education suffers in the US, a mind-boggling number of people play music, and since nothing has really replaced music notation, that means scores still matter.

The ability to mark up a score in a browser and share those markings, live, with anyone with a computer or tablet or other Web-enabled device? Priceless.

http://wiki.musichackday.org/index.php?title=Live_score_annotator

This clever tool will even follow a score in time, coupling algorithmic processing (to hold the right place) with broadcast information (to keep everyone in sync):

Follow the Muse

Drawing Sound with SuperCollider

Drawing Restraints by Mike Clemow was one of a number of pieces that focused on live synthesis and not just clever ways to replace Muzak. I have to give a nod here to Mike, as aside from his own project, he was an anchor of a little corner of the room working on live music apps, a big source of energy and enthusiasm. His work, aside from live performance, also appears in gallery contexts.

Also, bonus points for actually performing in his demo – that takes guts.

Drawing Restraints is a musical work for joystick, pen tablet and digital synthesis software. There are four modes for the instrument, two are buffer-based granulation modes using recordings of meat frying and a group of men talking, respectively. The third is a sine wave granular synthesis mode, and the last uses a saw-tooth wave through a filter bank to generate sound.

The synthesis is done entirely in SuperCollider and the input data is routed through OSCulator in the case of the Wacom tablet and a simple Pure Data patch for the HID based joystick. OSCulator does not have a HID input feature as of this writing. Both send the input data over Open Sound Control to Supercollider. While Supercollider does have a HID interface, I prefer to keep my programming interface unified; I merely have to create OSC responders in Supercollider in order to receive the data.

The different modes have similar parameters, however, each is mapped in a different way to the inputs. The modes can be combined to create complex sound objects that are independent, but their behavior is constrained relative to the state of each of the other modes. Their orchestration is constrained by the mapping scheme.

During Music Hack Day 2011, I came in with the hardware and the idea and brought the instrument to a state of playability. This piece will premiere at Zora Art Space in Brooklyn on Feb 23rd along with two others, “3coil,” a piece for induction coils and laptop, and “Outis,” a piece for video stream, computer vision algorithms, and custom synthesis software.

http://michaelclemow.com (home page has information about upcoming show)

Music Hack Day Page:
http://michaelclemow.com/index.php?/projects/music-hack-day-2011—nyc/

Best Networked/Collaborative Hack

JSONloops, an open-source real-time multi-user audio sequencer for collaboration, was an insanely ambitious project. And it wound up failing, likely for simpler reasons. While a first demo ran into network problems, the second go indeed worked!

The team:

Marak Squires – Created project, invented the JSONloops format, built core sequencing code
Elijah Insua – Writer of C bindings, solver of the hard problems
hij1nx – C Programming, JavaScript, HTML, UX and User Interface Dominator

Oh, and here’s what happened the first time around, according to Marak: “The software was working the whole time, but the machine connected to the projector decided to connect to a different WIFI network and we couldn’t access our local server.”

Yep, been there. But the project looks fantastic and does actually work perfectly well. Networked music-making is a topic for an entirely separate article, so I hope to talk to this crew more.

Updated – Marak lets us know he used the Socket.IO cross-browser sockets library:
http://socket.io/

Seriously cool stuff, as it also supports mobile browsers and older desktop browsers that don’t have direct sockets support.

Three features you wished were in SoundCloud

1. Pulling samples into Ableton Live.
2. Splitting up DJ sets into tracks.
3. Downloading SoundCloud sets as zip files.

Done, done, and done. Hope to see them released.

Fun SoundCloud Tricks

Tweetsonbeats.com turns a Tweet into a synthesized hip-hop memo. You can do it to your own tweets (or perhaps retweet beat poets of our time like Sarah Palin) with hashtag #tweetsonbeats. This is what SoundCloud co-founders do for fun. Really. For instance:

@haynes_dave: It went down but Tweets On Beats did a great demo. Just add this hashtag to a tweet and you’ll get a hip-hop memo by Tweets On Beats

And they composed a theme song for the hackday.

Listen to the NYC #musichackday 2011 theme song. Produced by @ericw, vocals by @lenberg at General Assembly on Sunday afternoon by David Noël

Follow up…

I hope that we see some of the code from this event polished and further developed; if it’s relevant to CDM readers, I’ll absolutely share it. And if you have creation events you’d like to see, let us know.

Liveblog of demos

If you care to read my own notes to myself, I live-blogged the event.

2011 hack list, with some great resources and (for many projects) code

For a long time, technologists have described a world of in which computing experiences naturally incorporate touch and gesture. The question is, how do we bridge the intuitive desire for those interactions and the actual technologies that get us there?

Few activities test the expressive potential of interaction quite like music. It’s in our cultural DNA; musical activity may even predate written language. So it’s fitting that the story of touch in computing and digital music would be intertwined, as they are with touch pioneer JazzMutant. Years before well-known Apple products, the Lemur, prototyped in 2003 and shown as a musical multi-touch screen, suggested the importance of fusing display and touch, and of tracking more than a finger or two at a time.

The history, and products like Apple’s iPad and iPhone, you may know well, though. The question on everyone’s mind now is, what’s next? (And for some impatient futurists, the question may even be, what’s taking so long?)

To begin to answer that question, I turned to Guillaume Largillier, original co-founder and CEO of JazzMutant, now Stantum Technologies. There aren’t many people on the planet closer to where touch has been and where it might be going. Even as the Lemur gets new features like integration with popular music production and performance tool Ableton Live, Stantum is working to bring the same enabling technologies to other device makers. And even though this is “Create Digital Music,” it’s telling that that technology is showing potential in everything from phones to aviation, not just DJing. Musicians have had a role in technological history before, from Leon Theremin’s work to Max Mathews and computer synthesis. It may be musicians who invent the future, again. This time, the trick is who delivers that future to the hardware makers who can popularize it.

http://jazzmutant.com/
http://jazzmutant.com/behindthelemur.php
http://www.stantum.com/en/
http://www.stantum.com/en/offer/technology-ip

To accompany the story, we also have an exclusive look inside Stantum’s labs, all the way back to the original 2003 prototype of the Lemur.

On Designing for Touch, and the Music Tech Industry

Peter: I remember when I first talked to Darwin Grosse about Lemur, when it was being distributed by Cycling ’74. Darwin just kept saying, “You know, I just think Star Trek: The Next Generation.” (That’s my recollection, Darwin; I hope I’m not misquoting you.) I tended to agree. It’s a cliche, perhaps, but this was clearly hardware that brought into our century part of an imagined vision of a much further-off future (the 24th Century). Was that a conscious influence? In an industry that has sometimes been aggressively traditional, is there a way to channel ideas from something as far out as science fiction?

Guillaume: Before answering your question, allow me to challenge your statement about the computer music industry. I think “ill nostalgic” would describe this industry much better than “aggresively traditional.” Most music software companies have kept being innovative over the last decade, but their creativity has been a slave to this nostalgic obsession. Emulating an analog channel strip, a tube amplifier, or a vintage synth is far from a trivial job. It actually requires as much engineering time and resources as developing a disruptive product such as Ableton Live or Max/MSP/Jitter! On the hardware side, the innovation killer is the price pressure. Despite a common misconception, the computer music industry is not and will never be a mass market. Companies such as M-Audio [Avid], Behringer, or Native Instruments may look like giants compared to JazzMutant, but they are nano-particles compared to large consumer electronic brands such as HP or Nokia. The volume and the gross margins are too small to amortize ambitious research and development plans. When we launched the Lemur in 2005, a lot of people predicted, and somewhat hoped, that Behringer would release a similar device at $200 within the next eighteen months. Five years later, the first serious competitor of the Lemur is about to land – Apple’s iPad – and its entry level price is $500.

Back to the USS Enterprise, whether we want it or not, this parentship is likely to follow the Lemur forever. This is kind of ironic insofar as I’ve never been acquainted with science-fiction culture. I don’t even remember having ever watched a full episode of Star Trek. That being said, I acknowledge that this association has settled spontaneously and durably in people’s mind. Does this association come from the product concept itself? I don’t think so. In my opinion, it comes first and foremost from the fluorescent graphic design of the UI objects, not from the tactile technology.

So, the real question would rather be: “Why did we design the graphic interface this way?” First, we wanted to stand clear of those boring pseudo-vintage brushed-aluminium graphic skins – the cutaneous symptom of the nostalgic flu! Moreover, we anticipated that converting users to virtual controllers would be a difficult task and that trying to mimic the appearance of real-life objects would generate frustration; hence, impeding the adoption of the product.

Having said that, the main purpose of this flashy design was pragmatic and ergonomic. The Lemur is ontologically a live controller, though it might be used in other contexts. This requires that the interface must be visible wherever and whenever a user might be performing, from night clubs to outdoor venues. This is particularly tricky with a touch screen laid horizontally, because the display backlight cannot compete with the specular reflection of sunlight. Human-factor sciences taught us that contrast perception prevails over brightness perception. Hence, highly contrasted graphics- ie, flashy objects on dark background – is the most efficient way to ensure a consistent readability. This is something the aerospace industry has understood for decades. So, if there was one conscious influence behind the Lemur, it would be the Boeing 747 dashboard, not the USS Enterprise.

“If there was one conscious influence behind the Lemur, it would be the Boeing 747 dashboard, not the USS Enterprise.”

I know for me, the appeal of the science fiction aspect was more conceptual than superficial, the idea of the ubiquitous touch interface. But I agree, having experimented with this, that the high contrast, light foreground, dark background formula is really an essential solution. I’m seeing some interfaces on a white background that look aeshtetically lovely, but that I can’t imagine using onstage. I’d at least want a switch for dark environments, when you’re not at your desk.

That reminds me a funny episode of JazzMutant’s story. As early as February 2004, we started showing early prototypes of the Lemur to our friends at [Paris sound research center] IRCAM. At this time, the graphic skin was based on a palette of blue shades, with a few touches of warm yellow for emphasizing elements that needed to stand out, such as text, levels, etc. One day in July of 2004, about one year before the commercial launch of the product, we brought them a new prototype, featuring a brand new touch panel along with the final graphic design. Their only reaction was, “wow, this display is much brighter!” They did not even comment on the tremendeous improvement to the touch panel! That being said, there are other approaches to improve the psychological perception of readibility. I sometimes regret that other developpers are reluctant to dig into them, and mimic the “Lemur style” instead.

Talking about drawing on screen, did you know that Iannis Xenakis’ Upic project has been my main source of inspiration – and also my main motivation to step from music making to technology creation ?

I didn’t know that, but it makes a lot of sense. [UPIC is a tablet-based, visual composition system developed by ground-breaking experimental composer Xenakis. It is now decades old but continues to evolve in new incarnations.]

Below: DJ Mike Relm demonstrates the Lemur for G4 Tech TV. Yes, this is the video to show all your friends who aren’t regular CDM readers and have no idea what the heck this is all about.

Lessons of Lemur

Let’s talk about the place of Lemur’s technology in the current landscape. How does it hold up in 2010? I know a lot of people do get hung up on the price, but can you talk about how it differs from other options out there, or what the source of the cost is?

Once again, the music market being a small niche, it’s hardly possible to be both innovative and affordable at once. In addition, the Lemur is still manufactured in France with components imported from various locations around the globe – not to mention that the US dollar’s agony doesn’t help [when exporting] the manufactured product! Lastly, a large part of the product assembly is still handcrafted. For all the reasons above, the product is far from cost-optimized. I cannot disclose further our plans now, but we are working hard to address most – if not all – of these issues.

Have there been uses of the Lemur in performance and creation that surprised you, or went beyond what you imagined?

Oddely enough, and despite of what I said before, the most surprising uses of the Lemur are sometimes the most conservative ones! As an example, for Björk Volta tour, Damian Taylor and LFO made the most archaic interface layouts one could imagine — a fistful of colored labelled pads and eventually a pair of faders –- nothing more. Their brilliant idea was to create one unique interface for each song. This way, at each moment of the gig, they just had at their disposal the few commands they did actually need. The other big surprise came from video performers. Whereas most musicians are reluctant to use the advanced features of the lemur during their live performances – such as the objects’ physics – video artists do not hesitate to play the Lemur as an instrument, rather than a remote control. For instance, I warmly recommand you to visit Ali Momeni’s website. Of course, it would be unfair to forget all the advanced users who have developed inspiring and unique instruments, but this is less surprising, since the Lemur was designed specifically for that purpose.

OSC is a technology that many of us have advocated, but there’s also, admittedly, a big gap between where we believe it could be and where it is, especially in regards to the lack of mainstream music tech adoption. That said, what would an ideal implementation of OSC look like? What could the protocol do to be better? And what might you imagine could be a tipping point in adoption?

Indeed, it’s fair to say that OSC failed to become the industry standard we hoped it will be! I can see a few reasons for that. First, there is an obvious chicken-and-egg issue, as with any protocol. At JazzMutant, we’ve done our best to evangelize OSC in the industry for about 5 years now, without success. Why should a software company implement OSC if there is no hardware to support it, apart from a $2k product? Why should a hardware manufacturer develop an OSC-compatible controller if there are no mainstream applications to support it? Finally, there are also some intrinsic technical reasons that prevent OSC from becoming a standard anytime soon. In order to overcome them, we started developing a new protocol a few years ago called “Minuit” (“Midnight” in French), as a successor to OSC and MIDI (“Noon” in French). We were discouraged from pursuing this project after assessing the amount of human resources its evangelization would require.

The Big Picture, Stantum, and the Future

We’re looking at an explosion of interest in multi-touch display surfaces in the consumer space. Are any of these, in your view, promising for music? Are there ways in which some of these technologies are deficient for musical performance applications?

The responsiveness of a touch system is the most under-estimated parameter, even though it tremendously influences the perceived usability, transparency and trustworthiness of an input device. This is why a vast majority of Multi-touch systems available fails to meet music makers’ expectations.

Absolutely — you mean responsiveness in terms of latency, accuracy, precision in tracking multiple points, or (I presume) all of the above?

I was pointing out the latency more specifically – even though the perceived responsiveness is a complexe imbrication of all these parameters.

Can you talk about Stantum’s role in the evolution of multi-touch? What can we expect to see in the future?

We envisioned the real potential of the technology we invented long before the iPhone announcement, though we could not imagine that Steve Jobs’ crew would accelerate the market demand [to the extent they did]. We started investigating how we could bring our technology to OEMs in parallel to our computer music activity as early as 2005. We finally made this step in 2007. The role of Stantum in this ecosystem is quite singular. However pretentious it may sound to you, Stantum is still the only company beside Apple to have developped a real multi-touch product, top-down, including all the software and hardware technology bricks. So, despite the small size of our company, we are better placed than any other player in this field to understand the complexe imbrication of software and hardware. You might ask, “Aren’t all these Windows 7 convertible notebooks real multi-touch products?” In my opinion, they are not, insofar as the only multi-touch services these devices offer so far are rotating videos or ten-finger painting. I do not want to offend anyone, but watching videos is much more pleasant fullscreen and if Neanderthal people gave up painting with ten fingers 45,000 years ago, there might be a good reason. At JazzMutant/Stantum, we’ve always considered the multi-touch technology as a milestone, not the final destination. With what we’ve been incubating in our labs for a few months, we expect to reach the next big milestone quite soon.

Do you mean that these PC vendors are missing the actual application of the multi-touch technology in the software they ship with these devices? Certainly, no argument there — the demos, the marketing, the demo apps outside of Apple have just looked horrendous and awful to me. But surely there are developers out there who want to do better? Hasn’t what’s held them back simply the lack of available hardware?

I do agree with you. Unfortunately, that leads to a chicken-and-egg situation; insofar as developing a meaningful, multi-touch-capable application requires a preliminary awareness of the objective capabilities and limitations of a given hardware solution. On the other side, a vast majority of multi-touch panel providers doesn’t look willing to raise the bar until the market identifies a “killer app” requiring full multi-touch capabilities with zero performance tradeoff. Hopefully, the iPad will contribute to reschuffle the cards. Unfortunately, Apple decided to stand clear of handwriting capability – which, I believe, is a huge limitation for creative and productive applications.

“I do not want to offend anyone, but … if Neanderthal people gave up painting with ten fingers 45,000 years ago, there might be a good reason.”

Let’s assume Stantum is successful in popularizing the technology. At some point, will the Lemur be obsolete – and could that perhaps even be a good goal?

The Lemur as it is today is likely to become obsolete at some point – the pet is more than 5 years old in an industry that usually sends hardware products to retirement manu militari at 18 month old! Having said that, there is much more to develop on the hardware side than what we have done in the past. If we succeed in what we are working on today, I believe the Lemur will keep playing in its own category for quite a long while.

Now, that said, how do Stantum’s efforts to engage the larger electronics industry impact these issues of scale and cost?

We understood as early as 2005 that there was only one path to spread this technology – and the underlying vision of how computerized equipments should work – out of the small niche of professional musicians and Max/MSP users. Then we did what we had to do : we licensed the technology to tier-one semi-conductor companies such as ST Microelectronics to embed our multi-touch know-how into dedicated chips. We also teamed up with some of the largest and most trusted touch panel makers to bring our solution onto the consumer market place. The whole supply chain is now in place and you’re likely to see a few Stantum-based multi-touch tablets shipping in the coming months. Will these products match musicians’ expectation ? That’s too early to risk an answer at this stage, since we have no control over what OEMs will make out of our technology. And as you know, a good user experience does not only depend on the quality of the touch system – it’s also a matter of CPU and OS choice, hardware optimization, not to mention the software application running on top of it. That’s why we believe there’s still some room for a dedicated hardware that takes in consideration the very specific needs of electronic musicians and visual artists. In a not-too-far future, we expect the hard work we have done with our partners will have a positive impact on the cost structure of our music products.

Stantum’s Latest Technology, and What it Means

Guillaume is a bit limited in what he can say about his future plans, but that leaves me free to do a bit of (informed) speculation. This is largely my own analysis, so it’s my message, not necessarily Stantum’s.

First, unless it isn’t already clear, JazzMutant is Stantum. Stantum is JazzMutant. Stantum is now the official name of the company, and JazzMutant is just the brand by which their technology caters to musicians. It says something about the company’s lineage – all the founders have a background in electronic music – that they have in the past, continue now, and plan in the future to keep a strong connection to musicians. That’s meant that the rigorous demands of live music have informed their touch technology and made it a better product.

The idea that Apple’s iPad would drive JazzMutant out of business, therefore, is the opposite of correct. JazzMutant is Stantum. Stantum is in the business of licensing its specialization to OEMs. The Lemur shows just how potent that specialization is, in a way that literally gets rooms full of people dancing and gaping at projections. Apple’s technology is available only to Apple. With Microsoft, Google, phone vendors, and PC vendors all getting into the touch business, that means Stantum just became very big news – even if that’s something musicians and VJs figured out years earlier.

Part of the challenge of multi-touch development is that you have to get a lot of pieces working together. You need the physical surface of the controller, the sensors built into that surface, and the firmware that interprets the sensors all to work in tandem. Apple does it, and does the OS and applications, too. 3M is working on a product for OEMs, also working with multiple touch points. But the other big source right now is Stantum.

It’s also significant that Stantum’s technologies are heavily patented (a fact that they advertise on their site). While I’m no big fan of patents, unlike Apple, Stantum is licensing their technology into the marketplace. Given the need to have a patent portfolio just to protect your work, Stantum’s patents give it effectively the right to play ball. By licensing their technology to the manufacturers big enough to make this stuff on a grand scale, Stantum’s OEM program could provide ready access to touch for software developers beyond just the iPad platform. Even if you’re a huge iPad fan, that means greater accessibility in the market, and more than one vendor to provide that access. I’m a great advocate for DIY, but making displays isn’t yet a garage operation. (Yes, I know people building their own multi-touch tables, but they don’t make their own cameras or projectors.)

Stantum’s technology itself is also unique. Their sensing approach supports pen input and even handwriting recognition, features Apple leaves out. For many of the world’s languages, handwriting recognition is a “killer app,” which could further drive touch adoption. For the rest of us, until we evolve smaller fingers, the ability to use a pen can mean amplified accuracy for painting and writing, and yes, even pen-driven music applications. (Somewhere in the great beyond, Xenakis smiles.)

This is not an advertisement for Stantum, either – the list of companies anywhere close to being able to provide this functionality is short enough to count on your (ahem) fingertips.

So, okay, you buy into the concept – when can you get it? (After all, even the Lemur doesn’t quite count. It isn’t set up for pen input, even if its sensing method could work. And the Lemur is a controller, not a computer.)

Right now, Stantum’s technology is available in a series of multi-touch demonstration kits, including one with the guts of a Dell netbook inside:

http://www.stantum.com/en/offer/evaluate

In other words, we’re waiting for someone to ship a product that incorporates their technology. Windows 7 already includes multi-touch APIs out of the box in all but its Starter edition, so the Windows platform is a major candidate. Windows, while proprietary, has none of the developer, language, software, or hardware restrictions that the iPad platform does, so if your application doesn’t fit the iPad model or needs pen input, Windows’ stock just rose. Free software is possible too. Linux already supports the Stantum Slate PC and a number of other digitizers, support that will be baked into the kernels shipping in this year’s major Linux distros. We’re not just talking drivers, either: the whole Linux community is working on everything from libraries for environments like Java to support in the windowing system to touch-centric distros. (More on the Linux situation later this week.) Google’s Android has a multitouch API, too. I’ve used it, and got frustrated quickly not because of the OS, but because the hardware on current phone handsets just doesn’t work well with more than one finger. That could change if Stantum’s tech starts to appear in licensee products; Android as a touch OS could take off.

For specifics on the Windows 7 aspect (old news, from way back in November – but of course, everyone is taking a second look because of the iPad phenomenon):
2009-11-03 Windows 7 Certification

Right now, the one thing Stantum doesn’t have a lot of – aside from OEMs shipping their tech – is competition. Most of the other touch competitors either can’t accurately track fingers in close proximity, or limit tracking to two fingers, or lose tracking fidelity around the edges of the screen, or can’t handle pens, or some combination.

You need musicians, creative artists, and gamers to tell you this, because the mainstream computer market thinks multi-touch has something to do with pinch-zooming their photos. If that were all you could do with multi-touch, this would indeed be an over-hyped technology. But the responsiveness of the Lemur and the demonstration technology from Stantum is something that can be powerful and expressive.

Apple has already brilliantly demonstrated what happens when scale, creativity, and technical competence meet. Now the question is, who else will be able to put this formula together, thus making other options available to developers? Stantum has the competence, and the connection to creative artists and music specifically. If OEMs start to sign on with Stantum’s tech and build useful hardware, we could see both off-the-shelf machines – and cheaper JazzMutant-branded products – for musicians. Indeed, with this larger Stantum perspective, whatever happens with OEMs could in turn be good for JazzMutant-specific, music-specific customers, too. Even with competition from the likes of 3M, the technology is so specific to certain hardware devices, and the emerging markets so large, it’s hard to imagine Stantum not having a big role.

What might surprise people in the larger tech world is how important music has been – and will continue to be – to the big picture.

When it all comes together, the days of computer musicians, DJs, and visualists standing behind screens, able only to stare blankly into them but unable to manipulate what they see directly, could become a relic of the past.

Looking at the monome hardware, it could be difficult to understand how a simple array of buttons has become the most important musical design of the decade. It’s been the software that has brought this to life, not least the work of stretta (aka Matthew Davidson).

In the early days of electronic music, the creation of modular systems for synthesizing sound was a major breakthrough. Today, we can produce modular systems for composition, for assembling the music itself. And in a world in which “more” is the key word, many of these systems, by design, do less, focusing on the essential.

stretta reached a major landmark late last week, with the release of the maxforlive monome suite. It’s a set of seven Max for Live devices, with variations, which can be dropped into Ableton Live for use in musical projects. But it’s also more than that – it’s a modular model for how stretta thinks, and each module is designed to be used with the others, all without ever having to take your hands or eyes off the monome controller. Included in the pack:

• obo matrix step sequencer
• pitches for playing notes on the monome
• polygomé 64 for polyphonic, step-sequenced, transposing pitches
• press cafe for repeating patterns of pitches
• spectral display for blinking lights to visualize sound
• step filter step-sequenced filter bank
• automatorgator MIDI- and audio- and OSC- controllable pattern gate

Details and download link (no explicit license coming yet, but Matthew has promised an open license):

maxforlive monome suite released

I got the chance to talk to Matthew about the project, how he created it, how to approach using it, and what it was like working with Max for Live.

All photos by Matthew Davidson; released under a Creative Commons attribution license. Click the images for full-sized versions.

CDM: Can you talk a little bit about what these modules are, and how they fit together, for someone who hasn’t seen them before?

stretta: Obo is like tonematrix as a MIDI plug in. I like analog sequencers because they are tactile. They’re limited, though, so the natural tendency is to add memory to them. I’ve never enjoyed the experience of an analog sequencer with memory. As soon as you add memory, the fun evaporates. You can’t tell where the data matches the knobs, you’re looking at LCD menus… bleah. I’m not saying obo is the answer, as it is a very simple device, but obo (with a monome) does provide a pleasing combination of tactile control, visual feedback and multiple patterns.

Pitches turns the monome surface into a MIDI device. Press a button, get a note. The only issue to resolve is how do you map notes across the grid? One of the first things I did with my monome after I got is was create a pitch grid in columns of fourths and chromatic rows, much like a guitar, but more like a touchstyle instrument like a Warr guitar. I liked the strict grid as it made the geometry of harmony completely portable. The pitches application allows you to customize the interval relationships of the rows and columns and optionally add a modal scale filter on the output so it is pretty much impossible to produce a ‘bad’ note.

Polygomé is one of those things that is most easily explained to people by shoving a monome in their hands. It is kind of like if you took ‘pitches’ and added a step sequencer. You recall the sequence by pressing a button, the sequence is transposed based on which button you start on. Then you can play the sequences polyphonically. Polygomé grew out of this massive, overly complicated project for the 256 I was working on called gomé. The idea was you’d create these geometric patterns then define a ‘path’ or vector across the monome surface that they’d walk across; kind of like how gliders move in the game of life. I was describing this on the monome forums and someone said, "boy, I hope you make this so it works on the 64." I didn’t think there was enough room on the 64 so I re-thought the idea and polygomé was born.

Spectral Display is a non-interactive device that turns the monome into a graphic… well.. spectral display. I developed Spectral Display from vu_spec by [monome creator] Brian Crabtree so this is all his brilliance on display. I merely adapted it to maxforlive.

Press Cafe is another MIDI instrument. The original brilliance of the monome is the fact that the buttons are completely decoupled from the LEDs. Nobody had ever produced a device that did that before. There was always some internally programmed or proscribed functionality that was never entirely suitable for much of anything. So I brainstormed about various ways to leverage this unique ability. Press cafe is a pattern trigger sequencer. It works on any size monome, but if you’re using a 256, you get 16 rhythmic patterns of 16 notes. The rows select which pattern you’re triggering and the columns select which note you’re playing. Naturally, you can use the monome surface itself to edit the patterns.

Step Filter is eight independent sequencers, each triggering a band pass filter. The sequence is edited directly on the monome surface. The real fun begins when you start specifying various loop lengths for each of the filters so they loop independently. You can have one band looping in 7, another lopping in 6, etc…

automatorgator comes in two flavors. One is a MIDI plug in that produces MIDI and OSC automation. The output can be smoothed or stepped. The audio plug-in produces gating effects, or cyclical amplitude modulation.

CDM: What was it like working with Max for Live? How did that impact the way you work?

stretta: I’ve always wanted something like maxforlive. Max lacked a powerful timeline. Most DAWs lacked user-directed internal customization akin to a modular synthesizer. I think this is important for a DAW because it is impossible to be all things to all people. I’d hesitate to say much more because my experience with maxforlive is limited to a couple months thus far and there is so much yet to explore. I would be seriously happy for years on a deserted island with this tool, provided there was some mechanism to share my work with others.

Max 5 brought some concepts like a global transport and the ability to specify time values in a metric-centric way. So, instead of milliseconds and samples, you can think in terms of quarter notes and eight notes. Then maxforlive brought all this inside of a functional DAW which made everything more powerful. It is one thing to be able to manipulate audio like play doh. It is another to be able to do it inside a real production environment, using musically-sensible units.

So, maxforlive is a huge leap forward, but the needs of the monome community were a bit more dire. There are applications for the monome that are functional enough to produce a complete musical statement in real time by itself. A good example of this is mlr. You don’t need anything else. My monome applications are not so clever. I never intended for anyone to sit down and try to express something using polygomé in isolation. I figured it would be one tool you’d use in the context of a greater whole. The problem with this is it relegated polygomé and others to the recording studio.

What the monome needed, was, in my opinion, a meta environment that could host multiple monome applications, ensure they were all running in sync, and switch between them on the fly. It would route audio, host virtual instruments, mix everything and add effects. Then you could save a setup and recall everything by opening a single file. If that could be done, then the monome is transformed from this monolithic standalone device, to a piece of integrated performance hardware.

CDM: Many people are now getting started with Max for Live. Any tips for new users?

stretta: The best tip I can offer so far is adding '---' before any data objects. On run, the --- is turned into a unique identifier so the instance is unique. This allows multiple instances of the same maxforlive device with access to their own data. Of course, you may not want to do this, you may want to retain the ability for multiple maxforlive devices to ‘talk’ to each other and share data which is a cool feature. I’ve uploaded a example patch detailing this at the monome wiki called ‘thisinstance’

http://docs.monome.org/doku.php?id=app:thisinstance

CDM: What if someone is interested in this download, but they don’t own a monome? Is there anything they can do with this pack without the hardware?

stretta: Not much. The only application that operates monomeless is obo. However, there may be enough about obo that is interesting to people to justify the download.

CDM: You should be able to adapt to other controllers, though, with some work – correct?

stretta: Sure, it really is simply a matter of tapping into the matrixctrl object and routing to whatever device. My stuff rarely makes use of monome-specific protocols like ledcol, but I’m doing more of that recently for performance reasons.

CDM: Have you seen other Max or monome work that has inspired or impressed you?

stretta: 7up [SevenUpLive 2.0] is an amazing piece of engineering, although it is mostly Java wrapped inside a layer of maxforlive. Buffer Shuffler was the first example that really blew my mind, and the cool part about this is you can open this stuff up and see how it ticks. When I opened up Buffer Shuffler, I was amazed at how little there really is to it, which says a lot about the level of sophistication of the max objects themselves. I still haven’t explored or opened all the devices that are included with maxforlive. It is a simple matter to lift small sections of max code and repurpose it. It is really early on in the life of maxforlive, so I anticipate the learning process to continue and expect to see more interesting creations as the ideas cross pollinate and everyone starts editing everyone else’s work.

Thanks to Matthew for these thoughtful answers. If you have follow-up questions, definitely let us know. And we’ll be interested to see how you work with these tools, or modify them in your own work.

For the latest, be sure to check out stretta’s blog, The Stretta Procedure:

http://stretta.blogspot.com/

maxforlive: monome integration from stretta on Vimeo.

maxforlive: obo from stretta on Vimeo.

It’s hard out there for a hardware synth. There are all these new-fangled soft synths, capable of producing radical sounds via easy-to-navigate on-screen interfaces. I have a very very short mental list of hardware synths that still matter to me for one reason or another – and the Roland V-Synth GT is one that keeps coming back. I had access to one temporarily for a review. It was like temporarily adopting a puppy. You try not to get too close to the thing, as you know you can’t keep it. The V-Synth is likely out of the budget of a lot of readers of this site, but it’s worth just knowing it’s there, and why it has become so beloved by sound design aficionados.

The V-Synth GT, itself a big upgrade from the original V-Synth, had a major software upgrade this summer that flew under a lot of people’s radar. But now as the days are getting shorter again and people are starting to think sound design, I hope we can give the V-Synth GT some attention as an instrument. It has inspired me even in my software work, just to see the perspective of the engineers at Roland and how the device is programmed.

First, a few notes about what the V-Synth GT is about – something I’m sure you’d like explained, given its US$3000 street price.

The experience of using the V-Synth is really different from a lot of the synths out there. You don’t get this sense of the excess of some of the workstations, the stuff you don’t need. You just get a whole bunch of toys for sound design, which combine in unusual ways that feel really playable but can also be warped to produce far-out results:

• Its AP “Articulative Phrase” synthesis method is really unusual, mimicing the organic qualities of how instruments respond in attack, note transition, and tuning. That’s fascinating enough, but the ability to get at some of these AP principles and create hybrid instruments is what makes the GT worth using.
• The COSM models of favorite vintage Roland gear are decent enough on their own, but the ability to combine them in semi-modular routings helps the GT shine.
• You can manipulate audio on the device in some unique ways, with real-time pitch and tempo stretching of loops and phrases, which can then be resampled. Okay, sure, your copy of Ableton Live can do this, but the experience of doing it on hardware – alongside the other V-Synth synthesis features – is unique.
• It samples external sources, which you can edit on the touchscreen, and routes external audio through onboard effects, including the Vocal Designer vocoder / voice modeler.
• It has lots of control, from the easy-to-navigate color screen to D-Beam controller and the signature X/Y pad.

In other words, it’s really a Roland synth studio in a box. My 2007 review for Keyboard Magazine explains what all of this is like in practice.

From the software perspective, the V-Synth embodies a lot of what I admire about Roland. “Articulative Phrase” synthesis really isn’t a synthesis method in the conventional sense – it’s fair to say it’s a collection of tricks they’ve developed for making their instruments sound good. But coming from the hardware background, working in extreme memory constraints we no longer consider on computers, they’ve had to use tremendous economy with their sound designs. And rather than focusing on a “press a button” approach to sound, they’ve really built responsiveness and change into all of the onboard controllers, something that software sound programmers could, frankly, use.

So, that’s the V-Synth GT as released. But the V-Synth has gotten some significant updates, including 64-bit drivers for Windows and, most importantly, a massive 2.0 OS update.

2.0: More Sounds, More Sampling, More Sound Design

• Import WAV and AIFF directly from USB key, making this more useful as a sample manipulator
• More sounds: Two new sound sets, new patches combining the V-Synth’s various sound shaping abilities, and a third sound set that’s a collection of vintage analog synths from various makers – including the entire Roland back catalog
• More sound design options: new arpeggiation styles, new step modulator templates, and effects ported from the Fantom G

Demo video from Roland, via Synthtopia:

Who’s Using It

I recently spoke to Roland’s Dan Krisher about the V-Synth. His thoughts:

There are many uses for the V-Synth GT, but the V-Synth GT has been exceptionally embraced by the sound design community.  For example, Richard Devine creates sample libraries, from drum loops to intricate soundscapes, and the V-Synth GT is one of his main tools.  The way the menus are built, it is very easy to get started making unique sounds right away.  Players aren’t limited to the sounds inside the V-Synth GT either—any instrument can be plugged directly into the V-Synth GT.  The instrument’s signal can be run through the GT’s synth AND effects engines.

Another popular use for the V-Synth GT is for people who are just looking for a huge-sounding synth to play live.  There are a wealth of real-time controllers such as the D-Beam and front-panel knobs, that allow you to tweak sound in real time.  The Vocal designer allows players to add to a vocal performance with harmonies on a whole different level than was possible before.

Now, of course, this is the usual spiel you get from hardware makers, but this is one of the cases in which my anecdotal experience with users backs up what he’s saying.

Dan also said that the 2.0 update really grew out of user feedback, which makes sense. The V-Synth GT doesn’t get as much attention as a lot of other synths out there – even Roland themselves tend to focus more on their Fantom and JUNO – but the V-Synth’s user base, both users of the original and the GT, is really fiercely loyal.

V-Synth GT Videos

There are some really quite amusing videos of the V-Synth GT out there. Now, don’t necessarily judge the sound of the synth from all of these videos – I was able to push it in ways that didn’t sound like conventional Roland synth sounds, too. (And no, I don’t know why some marketing videos – not the ones I’m listing here, but you know who you are – have to be so cheezy.)

Updated: To really get a sense of the extraordinary sounds that can come out of the V-Synth series, you need to watch the 2003 debut video of the original V-Synth at NAMM. Six years later, nothing in hardware has touched this. And keep in mind, the GT has added a lot of sonic tools that the original didn’t have, so imagine this going even further. (Thanks to James Y for finding this, and Roland, please let’s see more videos like this!)

Is there a word for ear-dropping? (And this is YouTube audio quality…)

Tatsuya Nishiwaki, hilariously, confesses he can’t resist turning the V-Synth into an electric guitar as he does other synths, and then parties like it’s 1989. That name will be familiar to fans of Japanese music – he’s a major keyboard player in Japan, along with working with some big names here Stateside, after starting as a founding member of the band PAZZ. (More videos, direct from Roland.)

The best place to get a sense of the V-Synth GT in action is to watch Jordan Rudess playing it:

Jordan Rudess V-Synth GT Demos, Roland Video Library

Rudess is a tremendously skilled player, but there is a certain conventional sound to Roland keyboards here and, perhaps even more so, in the other demos. You’ll just have to take my word for it that if you abuse some combinations of sounds, you can take the V-Synth in another direction. Here’s what the interface looks like, again via Synthtopia. Now just imagine turning knobs past the places you’re supposed to, and routing things wrong!

And, because we can laugh at what we love, one comic strip, drawn for the awesome blog Wire to the Ear by the site’s creator, Oliver Chesler:

Funny and true – but only if you’re just trying to sound like a Jupiter-8. The nice thing about the V-Synth is that it really can sound unlike anything else, once properly pushed out of its comfortable preset zone to its extremes.

V-Synth GT Users?

One V-Synth user talks about the synth’s ability to produce “wild sounds” and demonstrates some of his own creations on YouTube. He concludes – news to me – that “I do not understan why owners are selling them for $800.”

Umm… Actually, yeah. The V-Synth is really, really awful. You don’t want it. You want to sell it to me for $800, or even less. (Maybe these are users of the original upgrading to the GT, so desperate to get the new model they’re unloading the original dirt cheap?)

I really can imagine the V-Synth GT as a desert island hardware synth. I’m curious to hear from users of either the V-Synth or V-Synth GT. Got sound design techniques you’ve discovered? Raves – or rants – you’d like to pass along to Roland?

Let us know.

And yeah, I know – now that I’ve done this, we need someone to write a love ode to Kyma.

V-Synth GT [warning – Roland annoyingly makes their websites make lots of noise without asking]

2.0 Update Download

That’s the question I ask at the beginning of the Create Digital Music Winter 08 guide. We’ve filled it with good stuff we love, plus good stuff we hear that you love (via our survey of hundreds of readers for the holiday guide). Via Creative Commons-licensed images, you’ve shared your world of music, and so we share the whole guide as fully free work (it’s got a CC Attribution / ShareAlike license).

Here’s some of what’s inside – we wanted stories that you’d want to live with the whole winter season:

• Circuit bending 101 with Michael Una
• Imagining synths: reflections on the design of electronic instruments with Dan McPharlin, creator of wonderful miniature synths handmade from cardboard
• Tutorial on slicing audio to MIDI in Ableton Live 7, with tips from Live guru Francis Preve plus a free accompanying CDM pack designed by Covert Operators at http://covops.org/cdm
• Holiday Guide, with your favorite gear and software of the year, listening and reading suggestions, and ideas on open hardware from monome creator Brian Crabtree
• Creative tips for surviving winter in Berlin, courtesy monolake (Robert Henke)
• Images from the CDM community and beyond

With the help of graphic editor Nathanael Jeanneret, the results are designed to be an object on paper or read on high-resolution displays. The PDF is available free, with an on-demand print version from Lulu available worldwide (US$19.99 before shipping). I just ordered my print copy rush, so I’ll let you know what it looks like as this is the first time we’ve tried this.

Print Edition + Free PDF Download @ Lulu.com

A big thanks to our sponsors for making this possible:

Ableton Live, our premiere sponsor; now with an unlimited 14-day trial

Audiofile Engineering, makers of Wave Editor for Mac

Highly Liquid DIY MIDI electronics maker

Covert Operators, creators of Live Packs and video tutorials for Ableton Live

I’m really eager to hear what you think.

It’s something we take for granted: listen to a track, and it starts at the beginning and goes to the end in a fixed length of time. Wonderful things can be done with music that way, and it’s the traditional model of composition and recording. But the equally old, if not older, tradition of improvisation suggests that music doesn’t always have to be linear. It can be specific to a place, a time, a mood.

Now that the technologies that power music creation can fit on a standard mobile device, listeners could have music that’s as pliable when they listen through headphones as it is in a studio when it’s created. Music could respond to the environment you’re in, and sound different each time you plug in your earbuds. That presents new challenges for the people making the music, but it could be an entirely new medium.

The team behind RjDj, a reactive and interactive music platform for mobile devices, don’t just want to wait around for this to happen. They’ve got it up and running right now, in a just-released application for iPhone. I spoke via Skype to the team in Vienna as a crowd of enthusiastic programmers and volunteers hacked away in a massive patching and music-making fest they call a “sprint.” More sprints are planned around the world, and the entire project is being built with the open-source visual patching environment for multimedia, Pd (Pure Data), cousin to Max/MSP.

If you’re ready to geek out with Pd, in fact, you can have at the patches yourself. But even if you’re just an interested musician, there’s plenty to watch here. It’s about more than just the software (Pd) or device (iPhone) – indeed, this app alone is likely to extend to other devices. What it’s really about is a new approach to how to listen to music, how to develop musical tools, and how communities own and share that work.

And, oh, by the way, team members have been behind everything from the port of Pd to Linux to the launch of Last.fm – the latter sold to CBS as one of the hottest musical properties on the Web, and a personal fave among the CDM team. So don’t doubt for a second that this group can drive some serious change.

If you watch just one video, check out the one above – especially about halfway in, as it starts to get juicy. Even for someone who’s been doing this for a while, watching a tiny device respond to the environment is magical.

Gunter Geiger is a technologist and advocate of free software. He puts his code where his mouth is: he ported the multimedia tools Pd and GEM to Linux a decade ago, helping launch the free community around them. Now he’s harnessing Pd again – but it’s not just about the software, he says.

Gunter: It’s not about if it’s Pd or not. The idea is to be able to create music in a different way. Instead of doing a fixed track, you do something interactive. These kinds of programs have been around for ages, but it really didn’t catch up on the music market.

The important thing is to get momentum behind it — not just one guy doing this thing. [And] it’s not only having people to create things, but [expanding] the audience, which is very small. What we really want to create is some momentum, and a scene. We hope that we get artists who make new [work].

You start to create different forms of music. Some of them are more like classical interactive things. Others are using the sound input a lot. It’s really a very open world, and the good thing about using Pd in there is that basically you can do everything. It’s really so open that we don’t know what’s coming out of it. We’re just trying to improve the things, and all the people working here are constantly changing their scenes and making them better.

I asked specifically about whether they were working to standardize these interactive structures, but Gunter emphasized they’re mainly keeping it open. And that’s important to note here – the actual “scenes” are completely open-ended, limited only by what you can do with the target hardware and the objects in Pd approved for the project.

You have a sort of chicken and egg problem. It’s really hard to make a structure before you know what these things look like.

What he could promise was growth – and on more devices than just the iPhone.

Now it’s the iPhone. In a year, I hope … more. There are sprints happening everywhere.

Michael Breidenbruecker initiated the project, now joined by a team of musical and technological thinkers and coders, with a select group of backers with experience in new Web projects for music. As one of the original co-founders of Last.fm, Michael is familiar with what a platform can do for music listening. He’s committed not only to the free, open source model for the project, but to transforming the way people think about music making – even those who aren’t musicians themselves.

Michael: I think we are all just starting at this, in a way. The scenes that we have right now have a [deep] effect. If you’re producing music, maybe you remember the first time you played with an echo or with a delay. At least for me, I spent ages pushing the button and going "poo, poo." For many people on the street, or what I experienced at Burning Man [with the RjDj], people were really going crazy because it was the first time they had this interactive or reactive experience of music. Music was not just something fixed or something they could consume, but something they could influence.

Ever since Burning Man, I’ve known we have a reason to be on the planet, to do what we’re doing.

When you write about this or talk about this, it’s really hard for people to understand what it is. As soon as you put headphones on them, they actually get it.

Michael says that to make that connection with listeners, they first have to connect with artists – which means their challenge is not only evangelizing interactive and reactive music, but on the tool side, making Pd’s power more accessible.

Michael: The big task now for us … the couple of sprints we’ve had, and the people we have involved already, is just blowing my mind. And that’s something that we really actively want to push. In the next couple of months, we’ll have to do a lot of work on the composing interface. Pd is a bit abstract for people who are used to other production software. So that’s our job in the end.

There are a lot of people standing behind Pd, but in the art scene it’s totally … inadequate. If RjDj can bring the whole idea of Pd and interactive music closer to the market, that would be really great.

We are trying to keep it as free as possible. It makes a lot of sense to use and reuse things. All the stuff that’s done should be provided to the community. We have it all on a public SVN [Subversion, a free, standard server tool for tracking changes to code and collaborating on projects]. All we can say to the artists is, if you don’t want to share it, don’t put it up there now.

Artists selling RjDj scenes could be very possible in the future – and wouldn’t necessarily conflict with providing open-source patches for those savvy enough to run Pd. But so far, Michael says the project is driven by imagining a new shift in music more than a new business model. And, interestingly, the ideas behind RjDj predate the now wildly-successful Last.fm, which was acquired last year by CBS.

I had this idea for a project ages ago. I started to work on this thing in 99. In 2000/2001, I started up Last.fm. When I saw what was happening on the iPhone, I said maybe it’s time to start [this concept] up again. I tried to get a bit of structure, all of our investors.

Michael: To be honest with you, including the investors we haven’t yet said, this is our business model, not at all. We just know we’re working on something new which we think has potential for the future. We’d [be happy to] manage to get the idea of reactive music booted, in two, three, four years even, to see a shift in the music market. So people who are now listening to MP3 songs could also be listening to reactive music, and something that’s customizable, highly dynamic, and personal. We would certainly try to be the driving force in that development, that market. Right now, all we can do is try to make the product as good as possible, that the person from the street would be able to listen to it and enjoy it, and artists would enjoy doing scenes.

I can tell you how the idea was born. It was actually one of these stupid things. In the 90s, people started to wear earplugs to raves because they were so loud. They had to protect their ears. Then I saw people who actually had microphones on their ears, and I thought, wow, that’s crazy. They have a microphone and a headphone, so what they hear is filtered. I found out that’s not what it was; it was a binaural microphone. I thought it was like sound glasses. I thought that was great. Eyeglasses for your ears.

Changing the medium, Michael notes, does transform what music can be – for musicians, as well. They have hooked up RjDj to a P.A. at parties, taking care to avoid feedback since RjDj scenes often make use of the microphone as an input. Even networking is potentially on the table, for collaborative scenes, though no development has taken place yet. (Pd supports networking, so that’s definitely something that could happen, with control data beamed between different devices running RjDj.) In the meantime, RjDj poses problems you might not even have imagined.

There is another interesting topic which we haven’t solved yet. You have the RjDj scene, and your sound experience is in the boundaries of that scene, but what you’re actually hearing is totally individual. That’s something that you can record on the rjdj. What do the artists — if a listener makes a recording of his scene which is very private, it’s his voice, his environment, what about that? Who’s the owner of that?

It’s scalable uniqueness — the RjDj scene, you can copy it a trillion times, it’s still the same, it’s a copy, but the individual experience listening to it. and tha’ts something traditional music is fighting. You have a digital copy of a recorded track. The musical industry wanted that scalable; that’s why they made that digital format, the digital CD. So they had this tremendous scalability, but then they started to realize that the uniqueness [is lost]. That was one reason why we did Last.fm at that time.

[Then] people started to realize they make music with objects. An instrument, it’s an object. But with digital music, music in a way became totally objectless. Look at the iPhone – in the end, it’s so miniaturized. RjDj is really bringing it back to the object. You know how this glass sounds [if you strike it], but with RjDj it sounds different. People begin to experience objects in a different way.

RjDj received its first official release today on the iTunes App Store. Software is available for free, or as an “album” for US$2.99.

Where to go:

RjDj Site / About / Blog

How to Create Scenes (And incidentally, you can work on scenes with a laptop even if you don’t own an iPhone. Testing on the device is, of course, very nice – fellow iPod touch users, I’m working on finding out how that mic solution is coming for us!)

To all of our readers here at Create Digital Music, thank you for 2007. It’s been fantastic to sit at the helm of CDM and get to hear from all of you, from news tips to musical and technological projects, and get to meet you out in the world (at Macworld San Francisco, Maker Faire San Mateo, Handmade Music events here in New York with Etsy and Make, in Chicago at a demo swap, and even in Australia at a coffee shop).

I’m wrapping our own 2007 in review story, but which stories did Webizens choose as the most significant? Here’s 2007 by the numbers, according to our server. First, the most visited stories of the year:

Top Ten Stories By Visit

The top ten start out with Yamaha’s unveiling of the long-awaited TENORI-ON instrument, a tool for mobile recording, a terrific free tool for Windows, and an unusual DJ take on mobile music players. Apple’s Logic Studio manages not to sneak into the top ten, I suspect because it can’t compete with apps that run on two platforms instead of one. But Reason 4 falls just short of matching CDM reader favorite Ableton Live:

#10: Yamaha TENORI-ON Launch: Photos, Videos, Interviews, Demos, Details, and a Music Box

#9: Recording@NAMM: Zoom’s $199 USB H2 Mobile Mic/Recorder, Found in the Wild

#8: Open-Circuit: Powerful Sound-Design Sampler Now Free (Windows)

#7: Pacemaker: 120GB Pocket DJ MP3 Player

#6. Preview: Reason 4 Hits Beta; New Sequencer and Thor Synth; Ships Fall 2007

#5: For CDM readers, the biggest app release of the year was Ableton Live 7 and the new Live Suite, overshadowing even Apple’s now-budget-priced Logic Studio 8. The release wasn’t without controversy — check out that comment thread — but in the end, the core app and features from the small (time signature changes) to the large (Drum Racks) earned some serious respect. The app shipped this month, so I expect actually using Live 7 will be one of the big stories of 2008.

Ableton Live 7, Ableton Live Suite: Quick Look at What’s New

#4: We saw various iPod Touch / iPhone applications for music in 2007 — something I predicted (or at least hoped for) minutes after Steve Jobs’ keynote at the beginning of the year. None was as sophisticated, however, as Alex le Lievre’s Pro Tools controller. It should inspire other applications in future. (For a more app-agnostic solution, see my personal favorite app, i3L.)

Control Pro Tools with an iPhone or iPod Touch

#3: Mike Una made a very usable foot controller out of a QWERTY keyboard without a drop of solder, and Ableton Live users flocked to the step-by-step tutorial so they could do the same:

Get loopy with the DIY $10 Ableton Footcontroller (no soldering required)

#2: The hunger for 8-bit sounds on computers led readers to a free plug-in from one of the legends of 8-bit music. Be forewarned, though: this plug-in has a tendency to misbehave or fail to function at all.

Free YMCK Magical 8bit Plugin Now Universal for Intel Macs

#1: And the #1 story of 2007: Van Halen having a guitar tech disaster onstage. Most amusingly, some CDM readers thought the out-of-tune version was more interesting, but then, we have some major microtonal fans among our readers. See also ensuing controversy about what caused the snafu.

Onstage Tech Disasters: Van Halen Goes Microtonal!

Top Ten Stories By RSS

Feedburner has some stats of its own on readership. Here are the most popular stories for RSS readers. (And that’s a lot of folks — RSS readership peaked near 7000, over three times the readership at the beginning of the year.) Interestingly RSS readers weren’t nearly as impressed by Van Halen guitar catastrophe, being just as concerned whether Leopard would cause technical catastrophes for their own music. Their big story of the year? Ableton. But it’s not all about Live this year: note that the relatively obscure tracker Renoise is moving up the top 10.

And it wasn’t just about news: RSS readers loved practical tutorials, too, including James Grahame on maintaining vintage synths and Liz "Quantazelle" on making good demo discs (shown).

#10: Renoise 1.9 Music App Begins Beta; Why You Shouldn’t Overlook This Tracker

#9: Brian Eno, with Wright on Spore and Generative Systems, Sound, and Paintings

#8: Web 2.0 Sampling: Free Samples at Soundsnap.com, Freesound Project

#7: How To Create a Successful Demo Disc: Tips and Resources, Chicago Event

#6: Leopard Reports: Native Instruments, MOTU, Why Tiger Still Rocks, Java

#5: Onstage Tech Disasters: Van Halen Goes Microtonal!

#4: Yamaha TENORI-ON Launch: Photos, Videos, Interviews, Demos, Details, and a Music Box

#3: Windows Does Jack: Multi-App Audio on Mac, Linux, and Now PC

#2. How Healthy Are Your Vintage Synthesizers?

#1. Ableton Live 7, Ableton Live Suite: Quick Look at What’s New

Top Ten Tags of 2007

Finally, what keywords got readers clicking? Mac beat Windows. Fruity Loops beat Ableton Live. (No, really. And, sorry, Image Line, but no one calls it the boring "FL Studio.") You’d apparently rather make beats or run Pro Tools at some expense than get things for free. And software beat hardware by a ratio of 2:1.

10. Oddities

9. Free

8. Beats

7. Pro Tools

6. Ableton Live

5. Windows

4. Fruity Loops

3. Hardware

2. Mac

1. Software

Number One Top Ten

Beatportal has a list of their own top 10 of 2007, curated by Francis Preve:

2007 Technology Top 10

And we’re number 9 for catering to "exotic tweezer-head minutia," which I rank as my number one phrase of the year.

Server numbers aside, what was your big story of the year? Favorite music of the year? Favorite music making moment? Let us know.

Some readers on this site — wisely, perhaps — cautioned last year against doing a second "Create Digital …" site for interactive visuals, live visuals, and VJing. But to us, the connection between musical performance and technology and visual performance and technology is really essential. I’m pleased to announce that now, following a two-week sojourn on the Australian content, we’re finally kicking createdigitalmotion.com into high gear and lining up what we want the mature site to be. If you haven’t been reading lately, here’s a bit of what you’ve missed:

• Video scratching with Serato: At long last, one of the leaders in digital control vinyl has added video scratching as well as audio via a new plug-in called VIDEO-SL, now in beta; we’re planning our own test but already have one hands-on.
• We’ve got a big list of video and visual goodies we like. Is it a practical holiday shopping guide? Erm … bits of it are. It’s all drool-worthy, at least. And you’ll really want vintage, gigantic planetarium projections. Lay out some cots, and play that four hour ambient electronic set you’ve been working up.
• Want to learn this stuff yourself? vade has some nice online workshops and tutorials for working with visuals in Pure Data (Pd) and Processing. Both also work well for music, so if you want to dabble in custom-programmed audiovisuals (I swear, anyone can do it with some dedication!) this could be a good place to start. We’ve also got tips for inexpensive high-speed photography and not one but two CDMo tutorials on the free Quartz Composer tool in OS X Tiger and Leopard. See Keith’s report on what’s new in the Leopard release of Quartz Composer, plus a beginner-friendly tutorial for driving 3D cubes with audio courtesy VJ Kung Fu’s momo the monster. The latter should be ideal for whipping up some quick sound-driven visuals for your band; you can even host those visuals in the live music host Rax. Incidentally, our CDMo New Years’ Resolution: make more work, post more tutorials.
• Open jamming for visualists: Just as with music, the best way to practice your chops and share your work is to get out of your bedroom/studio and out to an open jam. We’ve got a full report on the Perth, Australia Plug and Play, an ideal example of how such an event could work, as part of their Byte Me! Festival I attended earlier this month. See the video below, and watch for more video soon – Jaymis and I are editing hours of video footage now.

Plug N Play – ByteMe Festival – Perth from Create Digital Media on Vimeo.

If you want to keep up-to-date on Create Digital Motion, you can add the feeds for the site:

RSS Feed (or subscribe via email)

Create Digital Motion Flickr pool

We’re also posting weekend inspiration each week, back next weekend post-holiday break.

From the whole CDM team, very happy holidays. (Yep, New Year’s Eve is among them — we’ll be cracking on 2007-in-review!)

It was 20 years ago today …

It’s easy to take for granted the mature tools available for music creation, and forget their history and the folks who made them real. While today it’s one of the biggest music software developers in the world, Cakewalk‘s first sequencer of the same name started as a college project for a Philosophy major. Cakewalk founder, CEO, and original author of the Cakewalk sequencer Greg Hendershott was that student. For the twentieth anniversary year of the founding of his company (then known as 12 Tone Systems), Greg sat down with me in their Boston headquarters.

This was a personally meaningful meeting for me, as Cakewalk 4.0 for DOS (pictured above) was the first software sequencer I ever used — and remained my favorite some time after going to Windows. In those days, programmer’s names were front-and-center more than they are now, and so Greg’s name popped up every time I sat down to work. Greg also studied with Gary Lee Nelson, who was my first electronic music instructor (albeit for me at a summer camp). Of course, part of the reason it’s meaningful is that I’m far from alone — over 1,000,000 users have used Cakewalk’s software. A look at Cakewalk is also a look at the computer music software industry’s brief but fast-moving development, and the design of the tools that have evolved alongside it.

This winds up being a huge interview — you’ll believe me when I say this is basically a transcript of what Greg said. But it’s also a genuine slice of history, and also a glimpse into what the industry’s next 20 years might be like, so we’ll have at it.

B.C.: Before Cakewalk

It’s your 20th anniversary — that means now’s as good a time as ever to talk about the beginnings of Cakewalk, and where this all came from. Can you take us through some of that history?

So, I was born … no.

I think the natural place to start is that I ended up at Oberlin College. I was technically a philosophy major, but the cool thing about Oberlin is that there’s the college, and then there’s the Conservatory of Music, and they make it pretty easy to cross over. I took a few classes, but mostly electronic music. I was taking several electronic music classes a week, and then trying to put out a philosophy paper.

Who did you work with at Oberlin?

Gary Nelson and Conrad Cummings were the two people. Gary was still doing stuff in APL [programming] language, and he made some extensions that he called MPL. [To program in APL] you have to use this special keyboard; it uses Greek symbols. You can be very succinct and precise when doing vector operations. For music that came in handy, because music was usually an array of notes or pitches or other parameters. But apparently the only people who really use APL are either actuaries, in the insurance industry, and Gary Nelson for music. It was pretty cool, but I think for the average freshman electronic music student, it was a little daunting with the Greek symbols.

What technology were you working with as a student — what were the tools at the time?

What we mostly spent our time on was CP/M [Intel operating system] machines, Osborne computers using Turbo Pascal to do programming. Ed. note: Anders Hejlsberg, the creator of Turbo Pascal, and successor Delphi — in which Fruity Loops was developed — went on to be a chief architect of Microsoft’s C# and .net, so a lot of this stuff is significant down the road.

And the other gear was stuff like, I think when I was there we got the first [Yamaha] DX7 [synthesizer]. That came into the lab, and everyone was really excited — and totally baffled about how to program it, because we were used to the Moog modular kind of synths where you literally had the patch cords around your necks and were making patches by [connecting] cords. So, to figure out FM synthesis was a little tricky. Working with a big modular synth was tactile and interactive. And then you had this small screen.

That’s how I got interested in computers. I was not a math and science guy. In high school, I was much more into English and history. If it wasn’t for music, I doubt I would even have tried to learn how to do software programming. But that made it fun for me. And, of course, over the years I came to be interested in the software engineering part of it for its own sake. But in the beginning, it was the music that made it fun enough to stick with.

So I wrote software. I got out of Oberlin and I procrastinated about getting a real job. I had some temp jobs, and in my spare time I thought I would try to take this software that I had done at Oberlin and really try to start over and write a basic sequencer.

Birth of Cakewalk, and a Five-Person User Base

What was that first experimental sequencer project like, the one you did at Oberlin?

At this point, I don’t even remember. It was kind of maybe crude pieces of what you would need to do a sequencer. It’s kind of fuzzy at this point. I do remember pretty much starting over but using a lot of the concepts. You know, sometimes you write something, and in hindsight, you figure out how to do it much better. But it helps to have done that first one.

So I kind of procrastinated, and thought maybe I could try taking out a small ad and selling a few copies. And I did, and amazingly four or five people saw the ad and called up and ordered. And that was enough to pay for the ad and do another one. It was really honestly a bootstrap. In the beginning, I really kind of backed into it. I had this thing, and it’s fun, and I had enough success to go a little longer, and it took off from there. But in the early days, I’d be writing code and taking tech support calls. I’d be going down to the local UPS place with boxes before they closed at 5:30 each day.

How did you come to name the software Cakewalk?

Right before I placed the ad, literally two days before the deadline, I had picked another name for the program, and I found out it had been used by other software. I think it was something like Opus. So I had this little dictionary of music terms, and I saw Cakewalk. And I thought, wow, that’s nice. It’s a simple English compound word, you know how to spell it, and it has this connotation of ease of use even if you don’t know the musical history.

And what about “12 Tone Systems?” Because who doesn’t like twelve-tone music, I guess, right?

So the name of the company was 12 Tone Systems — because it’s everyone’s favorite, most accessible style of music. [laughs] I thought it was kind of fun. For many years, people would call up and say, “is this Cakewalk?” I decided, instead of correcting people, we’d just change the name of the company to Cakewalk and that’d be simpler.

Early Evolution of Cakewalk, and Long-Lost Competitors

What sorts of things did you have to change for the program to evolve?
I think that the very first versions, behind the scenes was all one track, and all the events were in some big, huge array.

That was the other thing I did when I threw away what I had done in college. I actually did that in C. And remember in those days, too, this was strictly a MIDI sequencer, so it was only transmitting MIDI events to be used with external gear. I think back then I had a Roland MT-32, a Yamaha FP-01, and a Sequential Circuits 6-track. I actually owned a Commodore 64 when I was in college and tried to do some programming with that. I had some weird cartridge from Sequential that you’d plug into your Commodore 64 and it’d talk to your six track.

I think [Cakewalk 4.0] might have been the first version that would work with other MIDI interfaces. In the beginning there was the Roland MPU-401, and it had kind of a smart mode and a dumb mode. Ed.: that’s in fact just what they were called, though not necessarily accurately! The smart mode, you’d give it for example a note and a timestamp and it would transfer that MIDI message at that time. And it had some other features. So in a way it made it easier, but you got locked into certain things, like I think it had 8 tracks I think that it could handle. So Cakewalk for DOS had 256 tracks, and I had to come up with some scheme to take each eight and combine them. And then there started to be other MIDI interfaces that were simply UARTs, simply a serial port without the smart mode chip. And I wanted to try and support those. I think it was version 3 or 4 that I had to go back and rewrite [the software.]

At the time, you were really one of the first major sequencing programs to be available for DOS, correct?
There was a company called Voyetra that had a program called Sequencer Plus. And Roger Powell had a very pattern-based kind of sequencer. Right out of college, there was no way I could pay $500 for Sequencer Plus. So that’s part of why I started to write my own software. I think my motivation back then was to make something that was easy to use, and maybe a little bit more affordable. I was influenced a lot by a company called Borland, and back then they had … Turbo Pascal among other products. They had this whole philosophy, they would make the software affordable, they would not use any copy protection, which was pretty unusual back then, and that kind of basic business model or approach to software really appealed to me, so I borrowed a lot of that in Cakewalk.

Design, Then and Now

I remember having a really strong impression of Cakewalk 4.0 for DOS’ original design, and its simplicity and elegance. Now, obviously, in terms of capabilities and technological evolution, SONAR or anything of this generation is very different from an early DOS program. But I certainly have some of the experience of that original Cakewalk in SONAR. What — if anything — from Cakewalk’s first versions carries into the products today?

My design philosophy for the early Cakewalk software was to make simple things simple and to have more complicated things require a little more effort. So anything that you’d be doing frequently, like selecting tracks or wanting to record on a track, I tried to keep that very up-front in the user interface — a single keystroke — and things that were a little less frequent I’d tried to have buried in a dialog box or away from view. I think that’s one thing that we’ve tried to keep up, although that [gets] more challenging as you have more and more functionality and more features to the software. But I think that’s one principle we try to follow: keep the simple things simple and hide the complexity.

I think today SONAR still does a really good job of that, given the incredible amount of things you can do, especially compared to Cakewalk for DOS. The surface area of the application — plus all of the windows and dialogs — is so much bigger than in the early days. But you do have the choice of how you want to present that surface area to the user. So I think that’s one point where you have some ability to deal with the complexity and decide how to present that to people. The basic approach to the track view to some extent has a lot of the same characteristics of the very early version, but with many more options. And, of course, today we have track folders and all sorts of other options. But the basic keystrokes for moving around the tracks and quickly recording one have pretty much survived from the early days. The basic views, the track view, the piano roll view, obviously continued. Some of them combined, so you don’t have to go to a totally different view to edit controller data or work with notes.

But it really has grown an amazing amount. It was just MIDI, there was no digital audio, there was no graphical music notation — it was a program for DOS, so it was all text, or little blocks for graphical-like things that weren’t really graphics. Obviously that’s been a huge change.

It’s funny to think, too, how these platforms have survived. 20 years ago was before Pro Tools, but if you look at the leading DAWs now, we have Logic that was Notator, Cubase, Performer (now Digital Performer)…
I think it’s easy for me to forget how many programs there were back then. There was the Atari platform and the Amiga platform. And on all four of those platforms [with Mac and PC/DOS] there were three or four pretty good options for people. But it was just too much to sustain over time. So eventually four platforms was just too many, and things started to really boil down to Macintosh and Windows. And on those platforms there are only so many companies — choices at any one time that can capture people’s attention and keep going.

How do you account for Cakewalk’s ability to succeed over that period?
I think we worked hard, but we were also lucky, and we had a lot of great customers who continued to give us feedback. I think in terms of secrets of my success — or our success, probably the single biggest one was we were really trying to listen to customers and take their feedback and their wish list, and really roll things into the next version quickly. Sometimes it’s as simple as listening to your customers, and they tell you what they need and what they want, and you go do it. And a lot of it is really that simple.

But at times, companies for various reasons have difficulty doing that. And it is a little tricky, because sometimes customers tell you they want something, but you shouldn’t take them too literally. So if you take what they’re asking for and do it in a slightly different way, you might be able to satisfy ten different requests that are really variations on the same theme. And sometimes [you'll introduce] things that haven’t even occurred to them yet. There are these discontinuous innovations that you won’t find out from customers. So I think you really need to season that — maybe it’s 80% listening to the customers, and 10% thinking of the things that haven’t even occurred to them, and 10% reinterpreting what they’re asking for. But I really think 80% is listening and doing as much of what people want as possible.

But some observers might disagree with you on that. There’s a lot of talk in design — software design, Web design — about trying to do less, about saying no. Ed.: Since I talked to Greg, I got to ask Ableton co-founder Gerhard Behles about some of these challenges; it’s interesting to hear different takes on this, and we’ll have that soon. -PK
My theory is, a lot of products start off — like Cakewalk for DOS version 1.0 — or maybe Ableton Live version 1.0 — in the beginning, it’s extremely focused and they can only do a few things. But the few things they do, they do very well. And it attracts people who only want to do those things, and for them, it’s perfect — it has everything they need and nothing they don’t need getting in the way.

But over time, as you try to listen to customers and try to accommodate their requests, the product becomes the union and the sum total of all the requests you’ve received over the years. If you could hit the fast forward button, go out five or ten versions, it won’t be that original, really simple — perfect-for-some-people — product that it started off as. I think some companies are able to do a better job of taming that complexity and not letting it get out of control. But it’s pretty hard after 10, 15, 20 years of adding feature requests. That’s a real challenge.

How has perception of Cakewalk evolved, particularly as you’ve expanded the feature set in this way, and had to contend with complexity and change?
I think for Cakewalk there was a period where people knew that we made software that was easy to use, but I don’t think we got credit for being as innovative as we actually were. At times I felt kind of bad for our software engineers (This was the part where I wasn’t writing code any more.) I felt like they’re creating these really amazing things, but we’re not getting credit for it. People might recognize a product like Cubase or Logic as being very innovative, but we had done the same thing two years before. So it seemed like we were maybe a little invisible with the power user community.

That was part of the motivation of doing SONAR. The later era of what was called Cakewalk Pro Audio, 6, 7, 8, 9, I think particularly then we felt that people are recognizing that our product was easy to use, but they were saying, if you’re a real power user, you don’t want to use Cakewalk, you want to use Cubase. After I while that gets to you a little bit. I think that’s changed; [we're] making sure people realize the power user features that we have. Of course, the irony is, as we did that, there’s a group of people looking at Ableton Live saying, wow, this is really cool, it’s really easy to use, it doesn’t have all these features in SONAR or Logic. I think the moral of the story is, there’s not one type of customer, and not one type of product that’s right for everyone. You need to pick your vision, and that may change over time, and may evolve. You’re not going to please everyone all of the time, and that’s fine.

As you say that, it really seems that the market today is more about differentiation, focusing on different approaches instead of just more features — sometimes even intentionally doing less. There was a period, particularly in the 90s, when it seemed to me at least it was more about one-upping features, to the point where programs began to look more like one another; do you have a sense of that, as well?
In particular, my perception was that Logic and Cubase were locked in this feature war, and if one product added a feature then the other one had to. Over time, that’s going to tend to make the products look more like each other, although Logic and Cubase have different feels, but in terms of feature set, they get more and more similar over time. As [we've been discussing], I think some people start off at a certain point in their personal experience using music technology — maybe if you’re just getting started, Cakewalk DOS 1.0 is perfect. But as your needs evolve, the product evolves with you. Other people just getting started, maybe that later version is a little daunting to them.

So I think there’s always a role for new products to come out that are not trying to have every feature the power user might want. Whether that’s Ableton Live or Reason or Project5, there definitely has been a trend in recent years to kind of take out a clean sheet of paper and say, how would we design it today? Which is not to say that Logic and Cubase and SONAR are bad, but they definitely [represent] a certain philosophy of how things should work. For a lot of people that’s perfect, but it’s not the only way. So I think it’s really fun and refreshing to see people take out the clean sheet of paper.

I think as the technology changes, it definitely makes sense, every five years or so to take out that clean sheet of paper. You may discover you’ll still do things pretty much the way things have been done. And sometimes you may realize we could really make things simpler or different.

Were there times when you did a clean sheet, or “reboot”, of Cakewalk’s software? SONAR, I suppose was a kind of reboot, right?
Behind the scenes, pieces of the engine, going from Pro Audio 9 to SONAR 1 there were still huge tracts of code behind the scenes that were basically the same. But we definitely took out a clean sheet of paper with the user interface, and behind the scenes pretty big chunks of the engine and code did change significantly to support other technology. For us, coming from DOS to the first Windows versions was a pretty big change. That was an opportunity to rethink some of the feature set and user interface. And then going from those Windows versions of Cakewalk Pro Audio to SONAR was another opportunity. And then doing Project5 was another opportunity.

Thoughts on Piracy

Cakewalk has managed to grow and remain healthy over the last 20 years. But what sorts of challenges have you — and the industry — had to face?
Piracy continues to be a problem for a lot of software companies. And it’s a problem for Cakewalk, as well. Different companies take different approaches. We have tended not to use any copy protection or copy prevention technology, except maybe the most basic serial number [authentication] — nothing like software activation or dongles. And my thinking behind that is that many of those systems get cracked pretty quickly anyway. And they do cost something to add, and they cost tech support issues, and some inconvenience for customers. I’m always concerned about those systems that won’t really prevent the theft, and all they’ll do is make things a little tougher for the people who are paying for the software.

But that doesn’t mean we like people to copy the software. It really comes down to the culture around intellectual property, and establishing your relationship with customers that is about more than just them buying the product, where they can get good support, there is a good community, a good forum for them to interact with you and other customers. So I think if people are buying into a long-term relationship, that’s part of the key to dealing with that issue. And I notice customers really tend to think that way.

For example, I notice some Cubase customers got upset with Steinberg when they suddenly took out DirectX support in an update. I think many of the customers weren’t even using DirectX; it was the sudden nature of it. And why it bothered them is, as a customer you do have this long-term relationship, you’ve invested a lot in how to learn the software, and you want that company to be around and do good things for you. And I think that’s why sometimes people get personally offended if there’s a change like that, that’s something unexpected or unfair. And not to pick on Steinberg — all of us at some time make a mistake like that or do something that we don’t realize how it’ll be received by people. But I think my point is that it is a long-term relationship, and it’s not just about buying a box that has a disc in it.

Do you have a sense of how many people are pirating your software, what the impact is?
I know people sometimes come up with statistics like that. I’m honestly not sure how; it seems like a tricky thing to measure. If you’re using our software, you should pay for it. But I also know what it’s like to be right out of college and feel like I don’t have a lot of money to buy something. I also remember trading CDs — well, actually cassette tapes — with people in college. A year later or three years later I was going on to buy those albums that initially we had traded. I have a little sympathy or empathy with people who casually try something but then go out to buy it — although we do have an official, fully-featured trial version, so there is that very legal way to do it. But I try not to get personally too bent out shape about someone trying something out for a few days.

Looking Forward: Future Technologies, Growth

We’ve talked mostly about the past, but what about, say, the next 20 years? Do you think there’s more potential for growth in music software and creation?
There’s sort of a hierarchy of interest or commitment to music creation. There are a lot of people who have a very casual interest in playing around with making their own music, or putting beats together, or rearranging loops, and I think that can be potentially a pretty big group of people. But the interest and commitment is probably fairly short-term. And from there you kind of go up to increasing levels of enthusiasm to full-time, professionals or full-time amateurs who do this for many, many years. And so I think realistically the number of really committed people is always going to be a certain percentage of the population. That’s my personal theory — but I think that’s also a function of the particular situation of the country and where it is with its economic development. So I think there are at least 2 billion people where that percentage is pretty low today, but ten, twenty years from now will probably be bigger. So I think, from that point of view, the absolute number of people who will be interested in these more committed levels of music creation will definitely grow. But maybe that’s a little longer term — I don’t think it’s 50 years from now, I think it’s [happening already.]

What about growth around the world — will Cakewalk be a part of that? Will there be more homegrown efforts, more bootstrapped Cakewalks in other parts of the world?
I think there probably will be homegrown efforts — actually, I kind of hope so. Not to harp on the piracy issue again, but to the extent a country has its own software industry, that tends to impact the country in general in terms of piracy. Not to pick on Russia, but maybe today Russia is not the best country [in terms of] software piracy. But they have a growing IT industry or software development industry. To the extent that grows, I think that has a good effect on the culture. So I think having home-grown companies is a good thing. We go to a lot of work to localize our software into different languages. SONAR is available in French, German, Spanish, and Japanese. We haven’t done a Chinese language, but I expect at some point we will do that, also.

Another way to deal with this issue is to realize that probably the predominant way you’re going to sell software is part of a package that includes hardware. So sometimes certain cultures don’t value software except as a freebie add-on for hardware, so you can work with that by creating hardware/software solutions. And that’s where our partnership with Roland can be helpful. It’s not the main reason, but it’s kind of one side of that.

What about the Web; how do you see the Web impacting music software, in terms of actually browser delivery or Internet-rich applications?
It’s a really interesting concept — for example, in theory, and I think in practice in a couple of cases already, you could take a music creation application and deliver it as sort of the Web 2.0 application you use from your browser, much as you can do Google Documents instead of Microsoft Word or Excel. I think that’s really interesting, and maybe in future that will be more practical. I think some of the benefits of doing that with text documents and spreadsheets are not necessarily as relevant for music creation. Collaboration can be a big part of music creation, but it’s not the same factor doing spreadhseets or word documents. I think, too, even though the big thing of software synthesis, software effects, people have a lot of gear — so, okay, you have the software running in the web browser, but what about the rest of your studio connections? That also affects the collaboration and portability a little bit. So I can definitely imagine it all working that way, and maybe someday it will. I think, though, it’s more of the dancing bear stage, where the amazing thing is that the bear dances at all, you don’t care that it dances well. But that will probably change.

Splice is cool, Jump Cut for videos is also cool. We were talking before about increasing levels of interest — maybe for that kind of customer, that’s the best way to reach them, or the way they want to interact with things, through a Web-based application. But I think the other element is that — Google Docs, and so on — there’s also a different business model involved.

At the same time, there’s also this trend with Boot Camp and Parallels where maybe — I don’t have any inside information — but I can also imagine three or four or five years from now, many people will be running both Windows and OS X. How they’ll do it, and with what level of explicit support from Apple or Microsoft, I don’t know how that will turn out. But it seems that that’s something people are already doing. Parallels of course is the coolest version on paper, because you can be running them at the same time. On the other hand, I know people on both Mac and Windows who dedicate a machine to be a DAW, and they don’t want a lot of other things running — and they remind me of gamers. A lot of gamers will reboot into a certain configuration to play a game. So I wonder, is it all that horrible to reboot using Boot Camp to run Windows and run SONAR. So I think at this point, that’s all a little bit new, and it tends to be those who are really technology enthusiasts who are playing that. But maybe a platform won’t be as important five years from now as it is today.

That said, it seems the relationship with Microsoft and Windows has been important for Cakewalk, right?
We’ve focused on the Windows platform ever since Windows 3.1, and DOS before that. We really know it top to bottom. We know how to get things done technically; we have good contacts at Microsoft. It’s something we do really well and we want to keep doing. When it comes to music creation tools like SONAR, I think we’re going to continue to focus on the Windows platform. Realistically, porting SONAR to any other platform would be not a very smart business decision because it would take a long time, and what’s at the other end of that tunnel when you come out a few years later? But, when it comes to instruments, for example, as we expanded into doing instruments, right from the start we’ve made things cross-platform. And we love developing for OS X as well as Windows, and we’re committed to doing that. It’s not kind of a religious issue for us, it’s more the practical reality of when it comes to creation and DAW tools, we can do an amazing job on Windows. We can do things that are really solid, very reliable, high performance, and we’ll keep doing that and when it comes to instruments, we’ll obviously be bi-platform.

Do you ever think about what Cakewalk might look like, 20 years from now? (The fortieth anniversary of Cakewalk!) Or how music technology in general might progress that far out?
More in the kind of speculative, casual interest way. I think in music technology — in most technology — looking even five years out is really a long time. So you do it kind of for fun, or to think broadly about what you should do … that’s good. But to get into any detail that gets pretty sketchy pretty quickly.

I think some of the basic metaphors in applications for music creation will be the same — concepts like tracks for example are pretty fundamental. I think how you interact with some of those may be different. A couple of years ago, I know Apple just “invented” multi-touch technology, but a couple of years ago when I saw the other people who invented that I thought that was a really cool thing, and there are companies already implementing that for music. So to be able to grab objects and manipulate them directly and not have to use a mouse is one thing we’ll see more of in the future.

There’s one holy grail left in music technology, too, which is polyphonic pitch detection. I still have this crazy dream that you should be able to take an audio file or a CD and have the software analyze it and do the amazing thing the human brain does which is know that okay these frequencies go together, these don’t. These are one distinct musical entity, they’re not just harmonics — how we actually do that is pretty incredible, and we do it automatically. If a computer could do that, or do bigger pieces of that, I think it’d be really interesting. That would enable a lot of applications, not just remixing but all sorts of interesting applications. Maybe it’s fundamentally impossible to do a hundred percent, but we’re just so far from a hundred percent — to get even close to that would be remarkable.

Thanks, Greg. And here’s to the next twenty years.

We’ll continue to talk to the people behind the tools we use — the creation of music software hasn’t been nearly as well documented as, say, the creation of vintage hardware. So stay tuned! -PK

Other Faces of Cakewalk

I got a behind-the-scenes tour of Cakewalk HQ when I did this interview in August (hence, SONAR 6, though SONAR 7 product art was being finalized by graphics at the time). Many of the music DSP folks are outside Boston, but there’s a huge support, quality assurance, and engineering effort in Boston — not just the marketing and business stuff. You very rarely get to see the faces behind a lot of your software — something I hope we remedy here on CDM in the future, to try to get the creator-user communication more direct. Here are just a few of those folks I ran into.

Jamie O’Connell, Principal Software Engineer.
Jamie is also the co-creator and primary developer of MIDI-OX and MIDI Yoke, which I’d rank as the most important utilities for Windows music, hands down. You don’t have them and you’re a PC user? Go get them right now.

Scott Stepenuck Sr. SQA Engineer (Software Quality Assurance).
Cakewalk really is a dual-platform house. Most cubicles have side-by-side Mac and Windows machines, and by the time I visited at the beginning of August, all the Macs were running Leopard. They also have big, messy piles of documentation and gear, which is I think the sign of good development and quality assurance. (I’m, erm, using my own desk as part of the measure of that.)

Chad Beckwith, Product Manager Flagship Instruments.
Chad manages all of Cakewalk’s top-of-the-line soft synths. We got a chance to chat about tips and tricks with Rapture — expect more on that soon. He also gets the best toys:

At the 1985 Grammies, Thomas Dolby played alongside Stevie Wonder, Herbie Hancock, and Howard Jones. It was the golden age of synths and keyboard-driven pop. (Yeah, I know, some of us kinda miss those days.) But Thomas Dolby is significant, as well, as one of the pioneers of the computer-driven one-man band. Almost a decade into the age of soft synths, at a time when Logic Pro’s most punishing physical-modeling synths and convolution reverbs run just fine on a $1000 laptop and Ableton Live is becoming commonplace, musicians still struggle with some of the technical details of how to actually make the one-man band work onstage.

Here’s the comforting news: it’s not easy for Thomas Dolby, either. Normally when you write a print interview, invariably there’s a point where you get way off talking about technicalities and they don’t all fit. But because this is online, I’ve decided to reprint most of what Thomas had to say about making the tech work in its original form. These are just the technical details — gear stuff rather than art — but the important thing is that they have to support his performance. Part of why he’s able to bring such great presence to the stage is the gear in back is largely working — and he’s the one in control, rather than backstage techs. Here are all the gritty details:

On Choosing Apple Logic

One technique Dolby uses is to build up loops live in performance, by recording them directly into his sequencer and then triggering them later in the song.

I work in [Apple] Logic, and I actually find I can record layers into a loop and … have a strip that basically aliases them for the appropriate part in the song. If I put it in the main section, the B section … as soon as those bars are filled up, it will alias them everywhere else I need them in the song. So I have the structure pre-determined, but I have a two- or four bar-cycle that I’m feeding stuff into live. It’s in there, it’s in a spawned form in the rest of the song.

I asked, given the emphasis on loop recording and lots of live elements and interactive arrangements, why he chose Logic over a perhaps more obvious choice like Ableton Live.

I probably would have been more successful to have started that in Ableton Live. But it took me actually a couple of months to pick a new sequencer to begin with, because I was a Studio Vision user. [Studio Vision Pro is the audio workstation/sequencer formerly made for the Mac by Opcode, before its demise in the late 1990s. -Ed.] And of course that was no longer an option, and so I looked at Live, and Logic, and various other sequencers when I needed a new axe, and even some relatively unknown ones like Tracktion and so on. I really wanted to like something other than Logic, but it just ended up being the only choice recently.

And I think now that it’s owned by Apple, it’s very hard to bet against the incumbent. You just sort of assume they’ll be a lot closer to the iron as it were, and then they have been working with Apogee on the Ensemble. You know, all audio interfaces are not created equal. There’s quite a lot of difference in what the converters are and how tightly the drivers integrate… So what with all of that, I decided to make Logic my axe, and unlike some other people, I’m quite monogamous with sequencer. I find it hard to switch between Logic and Ableton, although BT is running both onstage and does a lot of his looping and stuff in Ableton. But I mean that sort of loop-based approach — building blocks — never really appealed to me because that’s not how I visualize most of my songs. I see them in a more linear kind of framework, so I can’t really see Live as a recording device. But given that it was built relatively recently from the ground up, it probably would have been better-suited to the real-time stuff that I’m doing.

Live Performance, with Logic and Max/MSP, And Fixing Problems

Logic’s got some things that are very annoying about it, given what I’m doing. One, for example, is that a soft synth in Logic — if it hasn’t been input to for a while — goes to sleep. That means the first time you hit a note, there’s a pause of maybe 20 milliseconds, and then a bit of a glitch and it comes in. And that’s on top of the latency that exists anyway. So it’s very bad for my playing.

I work with a [Cycling '74] Max/MSP programmer named Peter Nyboer. We designed and built a Max app called ZoneOut that basically enables me to map all of my keyboards and input devices to different zones, going into Logic. You could do that in Logic Environment, but there isn’t a graphic interface for just drawing a keyboard and so on. I have a visual representation of all my keyboards and pads [in ZoneOut]. I just draw in the zones that I want and assign the MIDI channel. I can filter velocities; I can turn pitch bend on and off on a per zone basis. And then I save that as a patch, which is opened from the sequence.

Just a few days ago, I got the new version from Pete with a thing called “ping” in it. “Ping” basically sends a false controller message to Logic all the time to keep those soft synths awake. It’s fantastic, actually, it makes a huge amount of difference, but my muscle memory is now set on how to anticipate [the latency], so I still tend to do it. So now I’m coming in early with a lot of the stuff, which is very weird. So I just sort of lunge into it and play a bit early and hope that it’ll go out. I have to re-teach myself to play in time again.

Awaiting an Intel Mac Brain Transplant

And the other thing is the latency itself. It’s just for a lot of my parts it’s no big deal — the string pads or brassy parts, you kind of make a mental adjustment to play them. But on the more funky parts, pianos and things, it’s very distracting. I have a MacBook Pro, which I can run at a buffer rate of 64 instead of 256 which is what I have to run on my G5. But the problem is, not all the stuff I need is available yet on Intel. There are a few plug-ins and things, you know, some little company, two guys in Sweden or something, and they haven’t gotten around to porting it yet. So I’m stuck with a G5.

When I am able to get onto the laptop, it’ll be a big breakthrough. One of the things about this tour is that if I want to change something to night, I’ve got to wait until we set up, we’re sound checking, the bar staff are coming in, the DJ is warming up and everything. And now I’m expected to make edits and hit save, and the next time I open the thing is going to be in performance. Logic does something very annoying, where it borrows memory resources — if you have multiple songs open, and it runs out of memory, it steals something from a song that’s in the background. So if I’m working with one of the songs, and then I save them all, and quit, then I may have actually stolen the drums from the oldest song I had open, and there’s now no drums in there. I’ve saved it and I open it in the middle of a performance and there’s no drums. So this tour has been like a process of two steps forward and two steps back. I’m looking forward to being able to do it on my laptop, when I can just sit on the tour bus or in a hotel room and be working calmly, and when I’m done with it I just take it down and plug it in, and there’s my show.

Working with a Live Visualist

To have a VJ up there of Johnny [DeKam]‘s stature, mixing live feeds with footage of his own, is definitely exciting. Of course, I’ve never seen the full experience myself, you know. [The visuals are projected behind Thomas. -Ed.] But he does a really good job, and he’s very spontaneous. He’s got a lot of different tricks and toys that he tries out on a nightly basis. Other video guys and VJs who’ve come have sort of been in awe of him because he writes his own software that everybody else would like to get there hands on.

More Coverage of Dolby/DeKam Tour Rig

More gory details:
Tech Talk, at Thomas Dolby’s blog (which, thank you very much, is actually written by him, unlike some of the nonsense PR stunts we’ve seen lately!)
Tom Brislin’s Interview/Feature for Keyboard Magazine (interestingly, the setup sounds as though it had been considerably improved by the time I saw it six months later)
Vintage Knob Madness: Thomas Dolby’s Custom-Built MIDI Controller [Create Digital Music]
Thomas Dolby’s Blog, Road Rig, Build Your Rig Cheap

And on the visuals side:
Visualist Johnny DeKam’s Website
Johnny DeKam’s Live Visuals Rig on Thomas Dolby Tour [Create Digital Motion]
Johnny DeKam Profile [M-Audio]

More Dolby

Thomas Dolby, on Music Making Past and Future: The CDM Interview