On tablets, on displays, multi-touch control these days is calibrated largely as a software interface – more Starship Enterprise panel than violin. As such, it works well for production tools and exploring compositional ideas. But it falls far short of being an instrument: even on the much-hyped iPad, touch timing and sensitivity is too imprecise, and the absence of tactile feedback and real, kinetic resistance makes you feel like an operator rather than a musician.

Several projects in experimental instrument research seek to change that. But of all of them, the one that has generated the most enthusiasm is Randy Jones’ Soundplane, co-developed with hardware designer Brian Willoughby. CDM shares a conversation today with Randy about his brilliant Aalto synth, and I’m working on a review soon. But wonderful as Aalto is, many of us are still eager to hear more of the Soundplane controller. I chose to wax poetic and optimistic back in December of 2008:
Intimate Control: Multi-Touch, New Models, and What 2009 is Really About

I shouldn’t have put a year on my predictions, though – good things take time. (If I could clearly recall what happened in 2009, maybe my general prediction was correct. The past tends to blur together for me into a continuum in the manner of the modern technologist, a vague assemblage of stuff that happened in the 60s with things that are actually still in the future.)

The good news: Randy continues working on the Soundplane, and Aalto will help.

Continuing our interview, here are the thoughts most relevant to Soundplane — and a glimpse of what it’s looking like as he works on it in the lab.

First, Randy explains his ideas about running a small business, continuing what he had to say in our Aalto story. The basic idea: Aalto’s software will bootstrap Soundplane’s hardware.

I think the whole idea of venture capital is sort of a poisonous one. It’s a little like bands wanting to get signed right away. The first thing you want to focus on is giving up your autonomy, really?

Instead, why not scrape together whatever you can from friends or family and just make something that you can sell right away, however small. I didn’t have enough saved to finish the Soundplane project so halfway through I switched to putting out Aalto as a plan B for paying the rent. Now it’s out and it’s a product I’m proud of that I think reflects where we’re coming from, and it’s going to fund Soundplane development, and it’s letting tons of people know we exist. Just get a foot in the door, do something useful.

He also shares his feelings about patents:

Some people won’t like to hear this, but I applied for a patent on the sensor used in the Soundplane. I know, the patent system is totally broken, and often, if not usually, used in stupid ways. But if there’s one thing I think it is actually good for, it’s to protect small companies like ours that innovate against a bigger entity simply stealing their R&D. This is why it was designed, right? I don’t know if our patent will save the day if such a thing ever happens, but if it does I’d much rather have one than not. It’s a pain to write one but it’s not impossible, you just need a lot of patience. “Patent it Yourself“, Nolo Press, is a good reference.

The patent question raises some additional questions for me – in fact, I’d love to see open source hardware that’s also backed by patent protection, in the same way that the GPL license is made tenable largely through the existence of traditional copyright laws.

But I do tend to agree that in the case of a truly novel technology, which this is, patent protection may be necessary. The question for projects like this will be whether to operate as a conventional, patent-protected design, or whether some sort of open source model with a patent covenant and a copyleft license like GPL will make sense — both preventing exploitation and allowing free experimentation. If there are any IP lawyers lurking around out there, let us know (I have some contacts, too); and definitely let us know if that’s a conversation you’d like us to continue.

In the meantime, the important thing is that Soundplane lives, and using Aalto could help it come to fruition. We’ll absolutely keep you posted.

As proof, though, more shots from the lab:

Also, must-read article from shortly after Jones’ NIME presentation:
Why Soundplane?

The whole article is worth reading, but Jones argues that not only is it likely many people will try to do tactile multi-touch, but it may be necessary. For those of you not all that good at hardware design, you could be just as essential as well to there being any future for these curiosities. The designers need other designers. The hardware needs software creators – lots of them. The software creators need to try lots of ideas. And everybody needs players, composers … users.

It’s all-too-tempting to sit back on the Web and marvel at what everyone else is doing, to take their genius and novelty as an engraved invitation to give up on your own work. “It’s been done before.” “Someone else is already doing this.” It’s probably a topic for a dedicated article, but it’s simply the wrong reaction. “It’s been done before — maybe it’s worth doing. Or doing again. Or doing better. Or doing over and over again.” “Other people are doing this — that means I have someone else to do it with.”

Historically, revolutions aren’t solo pieces. They’re ensembles.

Updated: speaking of work being ensembles, while Randy’s name is most associated with the Soundplane project, credit is due to hardware designer Brian Willoughby, who did the hardware design for the instrument. As he wrote in comments on CDM in 2010, when we covered Roger Linn’s Linnstrument: “For my part, I’ve been deep into the process of designing the analog circuits, DSP hardware and firmware necessary for the product, so it’s nice to poke my head up for a moment and see interest on this site, as well as to hear about other engineers trying new things and inspiring ideas.”

It’s something we take for granted now, but not so long ago, the only way to scratch and cue records was with analog vinyl. Now, of course, simulating those behaviors using digital records on turntables connected to computers is commonplace. But that hasn’t stopped the question of who owns the technology from spawning legal disputes. Most recently, a suit brought by patent claimants N2IT against M-Audio was dismissed. You can read the history from the time N2IT, a two-person company, launched their first commercial digital DJing (for BeOS, no less) back in the late 90s.

In patents, “first” is everything. And while N2IT had the first commercial product, it seems that broadly speaking the concept of how to make digital DJing work was not exclusively theirs. Chris Bauer writes CDM to share documentation of his own working prototype in 1998, before N2IT shipped their product. Nor is he alone. N2IT hasn’t yet brought suit against digital DJ maker Serato, and Serato’s Steve West publicly demonstrated research at the University of Aukland which leads back to 1996, well ahead of N2IT’s own demos.

It’s well worth reading the whole article for the timeline, but the basic concept is this:

the system created the illusion that the music being heard was actually on the record. and any piece of digitised music could be ‘played’ using this one special record and the spacedeck prototype. the system was fairly crude, but was certainly a working proof-of-concept. you could also perform needle-drops, and very rudimentary scratches. both of these techniques are essential for djing with vinyl, as this is how djs cue and beat-match the records they play.
the main steps of development were as follows:
1. research timecode. it soon became evident that SMPTE timecode would probably be the easiest to work with.
2. burn CDR with SMPTE and write code to ‘listen’ to it and get the speed, direction and position of the code.
3. research and write code to manipulate the speed, direction and position of digital audio files. i used quicktime.
4. write code to playback an audio file according to the incoming timecode data
5. get acetate record (dubplate) with SMPTE timecode on it, test with the system and and fine-tune timecode reading routines
surprisingly, there were no major problems in development. this is probably due to the concept being very simple.

And while N2IT indisputably had the first commercially-available product, this could call their patent claim into question:

the granting of N2ITs patent/s is extremely contentious, as they failed to mention various pieces of ‘prior art’ in their original application, including my project/MA thesis, which they were aware of as early as 2001. patent applicants are obliged to disclose this type of information if they are aware of it.

Chris’ full article:
the spacedeck project 2009

The legal wrangling over patents and who owns digital vinyl technology continues. The latest development: the court has dismissed N2IT’s claim against M-Audio, as covered by djtechtools. Before you strike this as a victory in the M-Audio column, it’s possible the parties settled out of court. Based on my limited legal background, I tend to agree with Ean Golden at djtechtools: this does seem to diminish the likelihood of N2IT successfully pursuing a new case against Serato. (In the Netherlands, it’s not possible to buy Serato, because there is would violate Dutch patent law, in the country in which N2IT is based.)

Previously, background on the story:
NI Ends Legal Dispute Over Traktor Scratch; Digital Vinyl’s Twisty, Turny History

Note that NI did acknowledge N2IT’s claims as valid. That would seem to set some precedent for future legal action by N2IT unless were to M-Audio win a countersuit against N2IT, which apparently has not happened.

It is interesting to hear djtechtools readers defending the N2IT patent. That may well have merit, but the basic technique of using an audio signal for control is something fundamental that well predates any notion of digital vinyl.

Technics 1200 Series: Discontinued (or not)?

Updated to reflect conflicting reports.

As digital vinyl presses on, reports are circulating in press and online communities that the the great emblem of the vinyl era is no more. Parent company Panasonic is reportedly discontinuing all remaining Technics 1200-series turntables (including the 1210).

Technics is dead [inthemix.com.au]

Updated: Sources for statements from Panasonic appear to be unconfirmed and/or conflicting. inthemix is where I had read this story; some CDM readers are describing these as unsubstantiated rumors.

It’s certainly possible that this isn’t the common “wild Internet rumors” phenomenon, but the equally common “large global company representatives aren’t on the same page” phenomenon.

The inthemix story, however, should be regarded as incorrect. As reported in the Australian cnet, the Panasonic Australia source has denied saying quotes attributed to him. There’s perhaps a more interesting (if not at all surprising) story here, which is that analog turntable demand is sagging — but apparently that will not result in the immediate end to 1200 sales.

Analog in ‘decline’ but Technics not dead [cnet AU]

Scratchworx tracks various, conflicting statements from different parts of Panasonic. There hasn’t been a formal press release, but then companies usually send press releases when they’re introducing products, not when they’re discontinuing them, so I wouldn’t read too much into that. One possible explanation could be that circulating rumors about the demise of the 1200 may have increased demand. It isn’t hard to imagine Panasonic responding to their reseller channel if lots of vinyl-heads began demanding turntables.

It’s also possible the rumors were incorrect. It’s easy for such a rumor to start, as vendors often don’t instruct resellers (or sometimes even their own global arms) about the status of products. I’ve often had conversations with press representatives of major music tech makers who weren’t entirely sure of the status of a particular product. And with store inventories low during a credit crunch, it would be easy enough for low stock to be misinterpreted as a discontinued product.

Scratchworx also notes that supposedly officially-discontinued models have remained in the channel.

Bringing some sanity to the Technics rumour [scratchworx]

Let me be absolutely clear: I think that vinyl deserves occasional attention from CDM, but I’m not a vinyl expert, and for that I rely on Scratchworx in particular.

If the Technics were discontinued — or, realistically, hen that day finally comes — what might it mean?

“It’s the end of an era” is the general response of the DJ community, but vinyl isn’t really going anywhere. The 1200 will always be remembered as the iconic scratch turntable, and there’s no question these ultra-reliable devices will continue to flourish, played and repaired by loyal DJs. (In fact, the quality of the gear may be partly to blame, in contrast to the planned obsolescence of a lot of newer equipment.) The 1200′s heydey, meanwhile, is long-since past. I personally think that’s healthy.

The golden age of scratch came about only because artists were irreverent and experimental, misusing and abusing equipment in a way that transformed music. It was not a musical movement born of pure nostalgia, and without a certain experimental drive, we’d be robbed of new experiments in the future. Today, abusing circuits and code have supplanted vinyl, a fitting medium for noisemaking, and one likely to last many more years. The Technics will survive, too. The really sad thing that our gear today is unlikely to last nearly as long as the Technics 1200.

Patenting the use of all episodic media on the Web might sound absurd, but the US Patent and Trademark Office has granted just such a patent, to a company called VoloMedia. It’s a significant issue, one that could threaten the freedom of all media distribution online. Wherever you are in the world, you can help.

Intellectual property law was created in order to protect genuine inventions and innovation from exploitation. But predatory patents, based on bogus claims and attempting to stake out broad rights, threaten to do just the opposite.

Here’s a new idea: fight back.

Lawyers are the heroes this time. The Electronic Frontier Foundation’s patent-busting project aims to take down unfair patents that threaten common-sense uses of technology. A number of these have applied to music and audio. The EFF has already won a big victory against what had been the worst offender – media giant Clear Channel actually successfully patented recording live shows. (No, really — recording a live gig, then burning them on the spot. The EFF was able to bust that patent.) The advocacy group also scored significant victories against patents on sending and receiving online streams and encoding media. (If someone thought they could patent your ears and charge you royalties for hearing, they probably would.)

Lawyers alone haven’t won these battles. The EFF’s clever twist is to crowd-source its case, by getting people like you to help the group document “prior art” – in plain English, to prove that something existed before the patent. (Without basic chronology, I could claim to have discovered electricity.)

In short, you can help save the freedom of online content.

VoloMedia’s Bogus Patent – And Why It’s Dangerous

VoloMedia has been granted a patent for “providing episodic media.” The patent is broad enough to endanger any independent podcast or episodic media producer. Over the summer, Volomedia’s own Murgesh Navar sidestepped concerns about patent abuse to brag on the company blog about just how broad that claim was – that even non-RSS-based episodic media belong to them:

With specific reference to our newly issued 7,568,213 patent, it was filed in November 2003, almost a year before the start of podcasting.  This helps underscore the point, that for nearly six years, VoloMedia has been focused on helping publishers monetize portable media…. and has continued these efforts with the addition of a wide array of smartphone-based applications.  The patent that issued yesterday helps to tie together and reinforce the value of the various technologies and services that VoloMedia has developed to help accomplish this objective.  VoloMedia’s intent is to continue to work collaboratively with key participants in the industry, leveraging its unique range of products to further grow and accelerate the market.  Today, podcasting is 100% RSS-based.  However, the patent is not RSS-dependent.  Rather, it covers all episodic media downloads.  It just so happens that, today, the majority of episodic media downloads are RSS-based podcasts, which is why we titled our announcement the way we did.

Aside from the “before the start of podcasting” lie – and I believe “lie” is the only accurate word – it’s the implied threat that should send a chill down the spine of anyone using the Internet. Make no mistake about it: VoloMedia wants anyone doing podcasting, via any mechanism, to work with them. From that same blog entry:

The impact of a strong growing IP portfolio is such that we would expect new entrants into the podcasting arena to have a collaborative relationship with VoloMedia, just as do many of the current players.

From the patent itself, as approved, the technology VoloMedia claims to own is described as:

A method for providing episodic media, the method comprising: providing a user with access to a channel dedicated to episodic media, wherein the episodic media provided over the channel is pre-defined into one or more episodes by a remote publisher of the episodic media; receiving a subscription request to the channel dedicated to the episodic media from the user; automatically downloading updated episodic media associated with the channel dedicated to the episodic media to a computing device associated with the user in accordance with the subscription request upon availability of the updated episodic media, the automatic download occurring without further user interaction; and providing the user with: an indication of a maximum available channel depth, the channel depth indicating a size of episodic media yet to be downloaded from the channel and size of episodic media already downloaded from the channel, the channel depth being specified in playtime or storage resources, and the ability to modify the channel depth by deleting selected episodic media content, thereby overriding the previously configured channel depth.

Plain English translation: if what you’re doing with media has episodes, you owe VoloMedia.

If this patent were allowed to stand, and if VoloMedia were able to successfully enforce it, it would have a chilling effect on all Internet distribution. Regardless of the likelihood of their legal success, that underlies the fundamental problem with patent law – it has come completely unglued from reality. That alone ought to motivate people to fully document these issues and try to effect change.

Wondering why you haven’t heard of VoloMedia if they supposedly invented all episodic content online? Right now, they advertise “solutions” for advertising and analytics, an iTunes plug-in, and branded mobile apps for platforms like the iPhone. That’s it. RSS and previous formats date back to the 1990s, with the intention of covering episodic media across formats, just as the VoloMedia patent claims. These were published standards years before VoloMedia’s claim. That’s why demonstrating the details of this history become so important: they could strike down VoloMedia’s bogus patent.

Help Write Episodic Content’s History

VoloMedia’s patent twists the law, and common sense. But the same laws also provide for disproving a patent. If you can prove that an invention existed prior to the date for which a patent is claimed, you can undo the damage.

For that reason, the EFF is asking for your help. Knowing the readers of this site, I imagine there are people out there who know those details, or know people who do.

You’re all old enough to remember the Age Before Fall of 2003, right?

Here’s the call to action:

In order to bust this patent, we are looking for additional "prior art" — or evidence that the podcasting methods described in the patent were already in use before November 19, 2003. In particular, we’re looking for written descriptions of methods that allow a user to download pre-programmed episodic media like audio files or video files from a remote publisher, with the download occurring after the user subscribes to the episodes, and with the user continuing to automatically receive new episodes. You can read the entire prior art request here, and if you have something that could help, please send it to podcasting_priorart@eff.org or fill out the form on our Volomedia page.

For more information:

EFF Tackles Bogus Podcasting Patent – And We Need Your Help

Patent Busting Project: VoloMedia

Prior art serves a second purpose. Part of the reason predatory firms can abuse patent law is because technology’s history is so poorly written. I would like to see these kinds of bogus patents struck down, but I’d also like the real history behind today’s technologies to be told. So even beyond this legal battle, I hope that we begin to make the story of technologies like what is now called “podcasting” accurate, complete, and fair. Future generations of technologists will thank us.

Certainly, the VoloMedia patent, if enforced, would do tremendous harm to media today. The entire strength of the Web is that it doesn’t have to have homogenized distribution channels, that anyone can publish without centralized outlets or “collaborative relationships” with any big partner.

If you’ve never cared about intellectual property policy before, this might change your mind. No one should be allowed to un-invent the Internet.

Gibson, the guitar company, has been on an utterly absurd campaign against music games, bringing lawsuits against the developers of both Guitar Hero and Rock Band and even against retailers. In the latest illustration of how screwed up patent law is, and just how over-litigious it has made technology in this country, the patent was based on a Gibson patent for a “System and method for generating and controlling a simulated musical concert experience.” Never mind that Gibson’s patent looks nothing like Guitar Hero, or that if interpreted that loosely, Gibson could theoretically sue any music software maker.

See my previous break-down of the patent and the twisted logic of the case:
Gibson Guitar to Guitar Hero Maker: We Own All Digital Musical Reality

And following development:
Gibson Guitar Loses Mind, Sues Entire Planet

Our friend Nilay Patel gets the scoop at Engadget that Gibson has lost its Guitar Hero case in California US District Court. Engadget also has a PDF of the decision:

Gibson loses Guitar Hero patent lawsuit, gets booed off stage

You can read juicy bits in the final ruling (PDF):

• Gibson’s own counsel withdrew from the case after the guitar maker refused their request for information. That’s right: Gibson wasn’t cooperating with their own lawyers. (Gibson later was represented by different counsel.)
• Gibson’s own corporate general counsel didn’t respond to requests from the court.
• Gibson started trying to force third-party Activision system providers to provide short-notice depositions, much to the dismay of the court and ACtivision, given Gibson’s own lack of cooperation.
• Gibson tried to use a YouTube video of a Guitar Hero hacker on the record, which the court found irrelevant (and, I think, laughable.)
• Gibson variously tried, unsuccessfully, legal gymnastics by which it could redefine musical instruments to enforce its ultimately irrelevant patent.

It’s also fun reading lawyers try to define what a musical instrument is in the context of this case. Ultimately, the determining factor in this case appears to be whether the musical instrument itself produces some kind of audio signal, not control signal. Yep, that’s right: it sounds like Gibson lost out because the Guitar Hero controller was defined as a controller but not an instrument. The court decision, showing unusual technical savvy, notes that the “Musical Instrument Digital Interface” (which they incorrectly call “device interface”) has been used for non-musical purposes, despite its name. In a fit of extreme hubris, Gibson at one point seems to have claimed ownership of MIDI for guitar controllers, despite prior art.

I’m certainly not qualified to interpret the judgment, but we can say this: Gibson lost. And they lost on almost every single point, from apparently abusing the court process to losing just about every detail they tried to prove. The court even says the Gibson arguments “border on the frivolous.”

The extent of their loss says to me the other cases have about a snowball’s chance, which raises the question of what Gibson was trying to accomplish in the first place. You have to wonder if they hoped intimidating legal action could help them win contract terms. But it’s nice to see the law win out — and raises hopes that, in the long run, legal remedies could eventually fix frivolous abuses of the patent system.

Oh, yeah – and we can all be relieved that Gibson neither owns the idea of making things look like guitars, nor musical simulation. Phew.

Apple’s iPhone should be a herald of a new age in interface design. But now, with speculation that Apple and Palm could get into a patent battle, and murky concerns about patents in multi-touch interface design in general, it’s unclear how much intellectual property legal wrangling will have to happen first.

I’m going to resist turning this into a long rant – partly because I think the jury is out on so many issues. It’s never been entirely clear what Apple continues sacred in its intellectual property on the iPhone. It’s even less clear – with similar multi-touch designs spreading back decades and murky law around gestures in general – what their legal standing is. No one knows at this point whether there will actually be a lawsuit between Palm and Apple (or which direction). But one thing I can say with confidence: we need alternatives to Apple. Even if you love your iPhone, I think you’ll agree it’d be tragic if other vendors didn’t push the technology forward. And we need alternatives like Google Android that support real open development, release free and open source code, and provide an option to Apple’s deeply proprietary, restrictive development platform. Innovative music software in particular won’t be able to thrive if alternatives are closed or nonexistent.

Here’s a quick look at where we’ve been, and where things are:

This has been a storm cloud since the beginning. Me, in January 2007, immediately following the keynote: Macworld: Will Apple Keep its iPhone Closed? Multi-Touch Patents? (I wish I had been wrong. No one believed me at the time that these two areas would be big issues.)

The original tech predates the iPhone. Engadget in 2007 on Apple’s multi-touch roots – FingerWorks gave them a patent portfolio and some key technology.

Google may have dropped out of the race. VentureBeat has a source that claims Google voluntarily dropped multi-touch to keep Apple happy. Even if that’s not true, I think potential legal battles with Apple – and the incorrect notion among consumers that this is Apple’s invention – could have a chilling effect. Update: There may indeed be some chilliness in the air, but there’s strong evidence that Google didn’t “cave” to Apple somehow — they just didn’t get around to it. And a multi-touch G1 may not be far off. Just asked the guy who’s already hacked the G1.

Both Apple and Palm are loaded up with patents – and no one knows what will happen. Engadget analyzes the potential for a Palm/Apple legal standoff – but there are two major issues here. One, Palm has a healthy patent portfolio of their own, meaning they could counter-sue. Two, no one knows if anything will come of this – aside from some saber rattling, we’re not even sure there will be a suit.

Just to keep things in perspective, though: I think multi-touch in general is safe. It’s a technology coming to phones, mobile devices, computers, Windows 7, Synaptic trackpads, Linux … the list goes on. To me, the question is whether developers will be free to try ideas without lawyers breathing down their necks, and that’s very much an open question.

And I think the deeper questions about whether open development, as on Android, can be competitive, may prove to be more important in the long run. Apple aside, we need more common-sense, modernized patent law – even if the Android in this case voluntarily dropped a feature, you can see that the issues are linked. And we need to have open development if people are to have freedom to experiment with design. This is about more than Palm and Apple; it’s about how we interact with our tech.

American citizens have turned their eyes to the incoming Obama Administration for all kinds of change. It wouldn’t be overstatement to say that just about every possible hope is being pinned to the new government – practical or not. But there’s good reason to believe some significant changes may be in store for both the areas of arts and technology, in ways that are not only relevant to CDM readers in the US, but could impact the global climate for these areas.

The federal government in the US can’t do everything, particularly when economic pressures are likely to make budgets tight. But they can do something to set the tone. Even more importantly, there should be opportunities for people who want change to become active and vocal, and to learn from each other, wherever we are in the world.

The agenda I think we’ll want as tech-using artists and makers:

• Defend innovation, commercial or common, from patent abuse (see: White House)
• Embrace open source – something that could benefit, again, commercial and community endeavors alike (see: BBC, OSI)
• Make the arts a priority, and one that via technology connects to renewed interest in math and science (see: NYT)

As you can see, regardless of your party affiliations or even country of citizenship, these are things we can work on together. For a start, I’ve already talked about personal changes – not simply governmental or political changes – that can make a difference in our communities:

Your Own Times of Change: Greetings, “Makers of Things”

Here are some additional issues that may well interface with the incoming US government, with impacts on the US and around the world.

Above: Remixing history, through the ears of the UK.
Obama’s Inauguration as Reaktor Mash-Up: Tim Exile

Technology: Patents

You can read the Obama technology agenda on the new White House site (itself a subject of discussion and hopes for new transparency).

http://www.whitehouse.gov/agenda/technology/

A lot here reads like campaign language, so it’s tough to say what the actual policy will be. But this bullet should be especially interesting to digital musicians and visualists:

Reform the Patent System: Ensure that our patent laws protect legitimate rights while not stifling innovation and collaboration. Give the Patent and Trademark Office (PTO) the resources to improve patent quality and open up the patent process to citizen review to help foster an environment that encourages innovation. Reduce uncertainty and wasteful litigation that is currently a significant drag on innovation.

I think flawed patents may be the single biggest to new creative technologies. It impacts both hardware and software, and everyone from DIY makers to useful research in big corporations. (And yes, even big corporations can do research that’s useful to the rest of us. For one thing, even some of that corporate research is open source.)

Patents in the US in particular have been wildly abused. Companies who don’t make anything have effectively “squatted” on ideas that might someday turn into products. Those patents are defined so broadly that by the time a genuine innovator invents something real that works, they often find they’re in “violation” of a nonsense patent. Large businesses, acting defensively, have added to the problem by over-patenting their own research. Clearly, we need some common sense rules so that patents cover people actually making stuff.

There are few political issues more directly relevant to the music and visual technology covered on CDM. I’ve seen patents stifle innovation countless times on this site, and when that hasn’t happened, fear about patents has often been a factor in preventing people from more aggressively pursuing their inventions. It’d be unrealistic to expect the Obama Administration alone to magically solve these problems. But a friendly Administration could invigorate debate, meaning now is the time to get active on this issue. I’m no expert in patent law, but I’ll certainly welcome people who are to become involved.

I’d also like to see the open source community begin to formulate a way of responding to patent issues. Open source has almost exclusively dealt with licenses in copyright terms. Certainly, the community is sensitive to the issue, but just sitting around worrying about patents does nothing: open source inventors need to start formulating a concrete strategy. They’ll need help, not only from the government but experts in the field. But the timing is right.

Whether people want to open-source their inventions or not, I think DIYers and researchers and even businesses who actually create stuff have a common need here. So it will be equally important for that open source community not to just blindly rail against patents, but find policies that work for everyone. “Makers of things,” not just open source advocates, have an opportunity to come together.

Technology: Open Source

The Obamas clearly have the power and popularity to popularize trends and ideas. Sometimes, that borders on the absurd: when it was revealed the Obama children wore J. Crew, the clothing company’s site crashed. It’s little wonder, then, that open source advocates would hope the new Administration would champion their cause. BBC News’ Maggie Shiels has a great story on those possibilities:

Calls for open source government [BBC News, via Slashdot]

One figure behind the rallying cry for open source is Sun co-founder Scott McNealy. That’s interesting, as Sun was actually quite late to the open source party. Sun didn’t open its flagship Java technology until after McNealy’s tenure. The fact that he has been won over I think is telling – McNealy created one of the world’s biggest tech vendors. The rationale for his appeal is simple: open source is cheaper.

I think the case should actually be broader. If the US – and, indeed, the economically-weak planet – want to advocate new growth in education, science, and technological innovation, it’s a no-brainer to have at least some technologies common and shared. That could ultimately lead to benefits for big vendors and individuals and the economically challenged alike.

And if you want to push open technology, artists should be among your first stops. We push the real-time capabilities of computers harder than anyone. For instance, when researchers wanted to demonstrate real-time Java, they chose a Bach performance. Why? Playing Bach turns out to be more timing-critical than one of the other applications – controlling a nuclear submarine. (The Army phrase “Be all you can be” comes to mind.) The drive of self-expression can be a powerful way of to realize technology’s full potential.

Direct quote on that, by the way:

Music synthesis is, in fact, more stringent in its real-time needs than many other hard real-time systems. For instance, avionics typically operate at a period of 20 milliseconds, or about 10 times longer than the synthesizer.

Harmonicon research at IBM

Open source needs music and visuals – and we often need open source. In music and visuals, the lack of interest in basic, open frameworks has often stifled the success and expressivity of the tools we use. I was impressed by the new stuff at this year’s NAMM. But many of the leading technologies – Novation Automap and M-Audio HyperTransport for controllers and Akai’s APC and Native Instruments Maschine among the hardware announcements – were limited by aging standards and proprietary implementations of control. Those same vendors struggle with drivers for proprietary computer operating systems owned and controlled by someone else. The result: music technology is often hard to configure and unreliable, limiting its appeal and reducing the number of customers. The solutions there aren’t all easy, and open source is no panacea, but I don’t think I’m overstating the problem – or the lost potential that could be coming from the open source world.

Of course, the Obama Administration is unlikely to do anything of practical use to artists or musicians when it comes to open source. But it could set a tone – and I’d argue, it already has. The Open Source Initiative’s Michael Tiemann noted just after the election that the Obama campaign had benefited from running open source tools. Whether or not Obama mandates federal offices run OpenOffice or something like that, I’d say the proof of open source’s utility is already out there:

Barack Obama proves the power of Open Source [Open Source Administration blog]

And that should be the main interest of arts technologists and creative tech vendors – politics aside, open source can pay.

Arts

We have mixed blessings in the US. On one hand, government arts funding has often been scant. On the other, we have an artist community that has vigorously defended its own value against the harshest critics, a uniquely-generous private funding climate, and a bootstrap, DIY approach by artists to supporting themselves. Arts advocacy groups are nonetheless eager to use the Obama Administration as an opportunity to get more badly-needed support – and they’re using the economic stimulus as a new angle:

Arts Leaders Urge Role for Culture in Economic Recovery [Robin Pogrebin for The New York Times]

Don’t believe them? Here’s a number for you: US$167 billion. That’s the amount Americans for the Arts says nonprofits contribute to the US economy. (They also employ some 6 million people.) And that’s just nonprofit groups; the impact of the arts and music are of course far bigger than that. As evidenced by this site, that cultural economy is increasingly globalized, meaning the entire business of making things could grow around the planet.

Much of the actual policy here would be more symbolic than practical. The additional US$50 million advocates want for the National Endowment for the Arts would have little meaning to an individual artist, though I’m sure the agency would love to have it. But “reframing” culture as an important part of the business of America is something that’s badly-needed.

Along the same lines, calls for WPA-style support for artists as part of economic recovery:

Will Act for Food [Newsweek]

More practical, I think, is the need for US policy that makes healthcare more affordable and accessible to the self-employed, a significant group of American readers of the site. If individual musicians or visual artists or freelancing coders and visualists and the like didn’t have to worry about spiraling health care costs, they could contribute in other ways a lot more easily.

Globally, we need a climate that’s friendlier to artists in general. The recent struggle of music tech research centers like STEIM in Amsterdam and IRCAM in Paris – places Americans might have assumed would be safe – is solid evidence of that.

Connecting this to the material and business of this site sure isn’t hard. Musicians and visualists increasingly sell to fans and one another, build their own businesses from scratch, innovate technologically, share open source research, teach others, volunteer, and add DIY tech businesses to their portfolio as they make their own hardware and software.

One thing missing from the traditional arts advocacy approach is the ability to use music, movement, and motion to aid in innovating in and teaching math and science. With technology (or even without it), expressive media are a fantastic way of demonstrating math and science concepts and making them creative and personal. I know I would have had a much easier time in school with topics like physics and Calculus if I could have connected them to music and animation, and I don’t think I’m alone.

That’s the philosophical framework, anyway. Given that tone matters for all of these issues, it’ll be interesting to see whom Obama makes NEA chief and what steps that agency and the Obama Administration take in arts policy.

So, thus concludes the post-Inauguration edition of this story. But you can expect to see a lot more on all three of these issues as they directly relate to the subject matter(s) of these sites – and expect more than just the President making some of the headlines.

This November, digital vinyl as we now know it will turn 10 years old. This setup is pretty simple in theory: instead of music, put encoded timecode on a record, then decode that timecode to provide information about where the record is in relationship to the needle. The idea is basic enough that, patent or no patent, it was inevitable that various developers would pursue the technique (and the very difficult work of implementation). Simulate the effect of scratching or needle dropping on a computer, and you’ve got virtual DJing, as found in products from Serato, Stanton, Native Instruments, Ms. Pinky, and others.

And as of Friday, it seems that the ongoing saga of a dispute over digital vinyl, beginning with the 2006 "divorce" of digital DJ titans Stanton Electronics and Native Instruments, may be over. NI released a statement Friday saying they had not only settled a US civil action patent case over their use of digital vinyl in Traktor Scratch, but had agreed to license the technology from N2IT Holdings, the US patent owners for digital DJing.

Apologies for the cat photo cliche, but … this involves patent law. We’d better have something cute and furry around to get through it.

The conclusion — the two have settled, Traktor Scratch is licensed per-use from N2IT, and N2IT’s patents are valid:

Native Instruments acknowledges the validity of patents held by N2IT, and has now fully licensed their usage worldwide for its TRAKTOR SCRATCH digital DJ system and related products.

The patents held by N2IT relate to general principles of digital music playback using time-code records, which are being utilized in TRAKTOR SCRATCH as well as in other manufacturers’ digital DJ systems with time-code control.

Acknowledging the validity of N2IT’s patents is actually pretty sweeping. You can read N2IT’s primary patent on Google Patent Search. The key words here are that N2IT patented the basic idea of using a turntable with encoded timecode on it for DJing. Theoretically, that could open up other digital DJ products to patent liability — keeping in mind that NI is a special case, because it was a development partner on N2IT’s FinalScratch product and was familiar with the technology.

How We Got Here: A FinalScratch History Timeline

I’m neither a patent lawyer nor a historian of digital DJ technology, so I quickly get out of my depth with the twists and turns this plot has taken. But I can offer at least a basic timeline of what’s happened, which puts today’s digital DJing in some context — albeit a somewhat strange context.

It goes something like this:

1. Revolutionizing Digital DJing — on BeOS (Doh!)

November 19, 1998: N2IT announces FinalScratch — for BeOS. [See Harmony Central, 1998] They even show off a working prototype at the COMDEX computer convention in Vegas. The product is developed with input from Richie Hawtin and John Acquaviva, who become its first users. (Incidentally, that’s why NI made a big deal of Hawtin switching to FinalScratch, and why Hawtin claims to have been on the cutting edge of digital DJing. At least in 1998, he certainly was, though you could argue the point now if you like. The choice of Be seems strange, but recall that under two years earlier, an Apple acquisition of Be Inc. had still seemed possible — until Amelio decided to go with Steve Jobs’ NeXT Inc. instead. And BeOS’ unique audio system gives it exceptionally-reliable, low-latency performance that could put today’s Windows and Mac OS X to shame.

Gustavo Lanzas aka Audioelectronic was there in the early days, as he describes on the Ableton forum:

Final Scratch was developed by a small Amsterdam-based 2-person company called N2IT in 1997-1998. The two creators were active in the rave/club scene, and had experience djing.

I was hired in 1998 [think this actually mean 1999 -Ed.] to go to Vegas and demo the very first system seen in the US at a BeOS event at the Treasure Island Ballroom. Here is a link to an archive of the event:

http://www.beatjapan.org/mirror/www.be.com/events/NAB_party.html

The hardware at the time was a hand-soldered prototype in an open aluminum project enclosure. The software it controlled was the BeOS media player. You could open multiple instances, and control whichever one was in front. Very rough, very primitive, but it worked more or less the same as it did now. There were exactly 2 prototypes, and no one had invested 2 million dollars into the company. Smile

Here’s a link to an early press release (I don’t think the product actually existed as such at this date):

http://news.harmony-central.com/Newp/1998/FinalScratch.html

I had worked with Timothy Self, then audio evangelist for BeOS, prior to this at Opcode Systems, makers of Studio Vision & several top-notch midi interfaces.

It looks to me like N2IT actually did show something at the 1998 COMDEX, because the Be folks gave them award and have a photo.

The Version 1.0 release comes out on a modified version of Debian Linux as well as BeOS. (The 1.0 software doesn’t seem to have been released until 2000.) Among early Linux adopters: Josh Wink.

2. N2IT to Stanton

[Date needed]: Stanton Magnetics buys FinalScratch from N2IT. I can’t find a date, which I assume means this wasn’t a publicly-announced acquisition. (And given the fact that the software was primitive and ran on Linux and BeOS, that’s not a huge surprise.) I’m also a bit unclear on the history of N2IT Holdings, though I’m gathering these exist for the purposes of retaining US patent rights.

January 2002: Bearing the Stanton name, the first complete commercial FinalScratch product comes to market at winter NAMM, for BeOS and Linux.

March 2003: Stanton and N2IT release Linux source code to the ScratchAmp driver — but note this only covers the audio interface, which is relatively meaningless; the real value is the timecode on the vinyl.

3. Digital DJing on the Mac, Windows

May 8, 2003: Version 1.1 of FinalScratch began the collaboration between Stanton and N2IT. It’s more like what we now know: Mac compatibility, and (courtesy Native Instruments) more usable DJ software. NI’s Traktor FinalScratch product even included a Linux version, though, for anyone who thinks NI has never done Linux development.

Native Instruments is really the ingredient that takes FinalScratch mainstream. They later port to Windows as well as Mac OS X.

4. Digital DJ Divorce

Late 2006: Native Instruments and Stanton end their relationship. (For some reason, this is listed as late 2005 on Wikipedia, with no reference; the divorce was not formally announced until October 2006, with an effective date of December 31, 2006.)

One stipulation of this separation is that Stanton releases a legal statement requiring NI to remove compatibility with FinalScratch timecode from its Traktor software line.

5. Traktor Scratch, Legal Disputes

Early 2007: The NI / Stanton divorce turns ugly. First, NI releases a competing product, Traktor Scratch. Stanton now lacks the important software capable of doing anything with timecode input, so they can only position the audio hardware — arguably the least important part of the equation — now called Final Scratch Open. Stanton also attacks NI, blaming them for compatibility problems between NI-developed FinalScratch software and Intel Macs. NI fires back that Stanton didn’t give NI enough time to complete updates to the FinalScratch software, while tying NI’s hands on updating Traktor via legal action.

June 16, 2007: N2IT files a claim against NI to try to get Traktor Scratch off the market, claiming NI’s familiarity with the underlying timecode technology.

This also illustrates why N2IT’s patent dispute with NI might not apply to other digital timecode products. From N2IT Holdings’ US legal representative, Bingham McCutchen:

In anticipation of working with Stanton Magnetics and Native Instruments to develop a new version of their product that would operate on Windows, N2IT claims it provided confidential information about its technology, and trained Native Investment employees on how their product worked. Last month, according to Billboard Magazine, Native Instruments released Traktor Scratch, which N2IT claims is similar to Final Scratch and uses its patented technology.

Now: By settling with N2IT and licensing the technology, NI effectively ends the Final Scratch chapter of history — at least to the extent that the FinalScratch / Stanton / Traktor / Traktor Scratch line appears to have passed (legally, now) to Traktor Scratch. Stanton is out of the business, left only with an audio interface. But, of course, with competing vinyl timecode systems, NI’s Traktor Scratch is also no longer the only game in town. And there’s nothing preventing N2IT from pursuing legal action with the competitive products, particularly now that they have legal precedent with the NI settlement. We’ll see.

How RZA "Invented" Digital DJing: The Legendary of Replicator

Of course, then there’s the bizarre footnote to this story — if it’s anything other than a random interview rant. In late 2007, RZA from Wu Tang Clan claimed in an interview with KotoriMag.com claimed that he had invented the digital vinyl technology used in Final Scratch on an Atari the year before N2IT — using technology from NASA, and millions of his own money. (Oddly, he also claims that music tech in general has come from space tech. I always suspected Cubase was powered by Tang… erm, the NASA-tested powdered beverage, I mean. Unless that’s how Wu Tang Clan got its name…)

There’s only one little problem: there’s no actual evidence of any of this. That, and the timeframe RZA describes is concurrent with, not prior to, N2IT’s development. So I have no idea what he’s talking about.

Here’s the take from illdoctrine.com, via a ProHipHop.com post of late last year:

And the complete interview:

So, is NI settling with RZA next?

I doubt it. But a RZA Edition Traktor Scratch — perhaps with a big Space Shuttle emblem — now that’d be cool.

References:

Final Scratch history [Thread on FinalScratch Forum]

Final Scratch @ Wikipedia

Updating the timeline:

If anyone has details to fill in, please let us know in comments.

Are you making music without real acoustic instruments? You know, in, like, virtual reality? Then you may have stepped into a strange, alternate dimension. Let’s call it, for the sake of argument, The Gibson Zone. They control the horizontal. They control the vertical. They invented what you’re doing … right now.

Or, at least, that seems to be the message sent by a recent patent dispute between Gibson Guitar Corporation and Guitar Hero developer Activision. (Harmonix, the original Guitar Hero developer, has moved on to Rock Band.)

I know what you’re thinking: maybe Gibson claims to have invented the guitar, or the Guitar Hero controller looks a little too much like an Epiphone or something. Ah, but that might actually make some sort of logical sense, and this is the topsy-turvy world of intellectual property. In fact, both Harmonix and Activision already have licenses with Gibson for their guitars.

Instead, Gibson is arguing they own the rights to anything that can “simulate participation in a concert,” which they patented in 1999. (Look out, air guitar lovers.) Now, I don’t claim to be an expert in patent law, but being the layperson that I am, I would assume the original Gibson patent would have some passing similarity to Guitar Hero.

System and method for generating and controlling a simulated musical concert experience [Google Patents]

Well, let’s review. The Gibson patent is described as follows:

“A musician can simulate participation in a concert by playing a musical instrument…”

Okay, with you so far.

“…and wearing a head-mounted 3D display that includes stereo speakers.”

Nope. Lost. They do know that Guitar Hero is not available for Virtual Boy, right?

“the musical instrument generating an instrument audio signal at an instrument audio output, the instrument audio signal varying in response to operation of the instrument by the user of the system;”

Okay: musical instrument, check. “Instrument audio signal?” No, not in Guitar Hero.

“a video source providing a source video signal at a source video output, the source video signal representing a video portion of the pre-recorded musical performance

a video display responsive to the source video signal whereby the user can view the video portion of the pre-recorded musical performance on the video display.”

To the extent that Guitar Hero involves a TV, yes. Pre-recorded musical performance? What? I’m lost again. In fact, what exactly is Gibson describing here? Playing your guitar while watching an old Pearl Jam concert DVD? With goggles on your eyes?

“an audio source providing a source audio signal at a source audio output, the source audio signal representing an audio portion of the pre-recorded musical performance, the audio portion including an instrument sound track containing pre-recorded musical sounds that would be generated by the musical instrument in the pre-recorded musical performance”

At this point, I think Gibson is patenting sound itself. Or, at least, karaoke, which as I recall had already been invented in 1999.

a system interface device having a first audio input electrically connected to the instrument audio output, a second audio input electrically connected to the source audio output, and a first interface audio output;

More damning evidence, except for the fact that Guitar Hero doesn’t have an audio interface.

And, still more:

f. the system interface device including a source audio control circuit responsive to the instrument audio signal, whereby a characteristic of the source audio signal is controlled in response to operation of the musical instrument by the user to provide a controlled source audio signal at the first interface audio output; and
g. an audio playback transducer responsive to the controlled source audio signal such that the user can listen to the audio portion of the pre-recorded musical performance on the transducer, in synchronization with the video portion.

Back to my first theory. This, followed by lots of talk about audio signals and audio portions and “video disc machines” still seems to involve you playing your Pearl Jam DVD karaoke-style with a guitar. In other words, both wholly un-patentable, and wholly unrelated to Guitar Hero, an interactive game with computer-generated graphics and a controller with buttons not audio.

It’d be a bit like if the guy who invented the paperclip claimed patent rights for deep sea fishing. But, as I said, this is patent law — so that may be possible.

Now some visual evidence:

Keeping in mind that what you’re looking at involves audio signal, not the control input of a game controller, from that guitar, I’ve provided this marked-up version, removing the stuff not in Guitar Hero:

But Gibson does have a point. To the extent that Guitar Hero involves:

• sound
• image
• people pretending to be musicians
• something shaped like a guitar
• an on/off switch

…Guitar Hero is a dead-ringer, patent-violating copy of what they described in 1999. Then again, so is a group of stoners playing air guitar Pink Floyd to The Wizard of Oz.

‘Guitar Hero’ Subject of Patent Dispute [Associated Press, via Wired.com]

At the very least, it looks like Apple is working on significant new music software features, as you might expect. But depending on how you read their patents, new music hardware — even touchscreen hardware — could be in store.

Apple is a big company with a lot of intellectual property, so teasing out patents can be difficult. But, as others have observed before, looking for Dr. Gerhard Lengeling, the Emagic founder who came to Apple along with his company, reveals interesting results:

20060278058: Frameless musical keyboard
20060272485: Evaluating and correcting rhythm in audio data
20060022956: Touch-sensitive electronic apparatus for media applications, and methods therefor
20050288805: Providing synchronized audio to multiple devices
20050204906: Method and apparatus for simulating a mechanical keyboard action in an electronic keyboard
20050204904: Method and apparatus for evaluating and correcting rhythm in audio data
20050145099: Method and apparatus for enabling advanced manipulation of audio
20040125122: Method of manipulating an audio and/or video signal in a graphical user interface (GUI) and a computer readable medium containing a program code for said manipulation
20040125083: Method of controlling movement of a cursor on a screen and a computer readable medium containing such a method as a program code

Lengeling Query on the US Patent and Trademark Office

Reader Doug Joyce, who sent in the link, observes:

There seems to be ample evidence of a touch screen interface for the next Apple offering. There are a few other interesting tidbits in the patent applications as well if you’re into to reading this stuff.

Some of this is old news: synchronized audio to multiple devices was rolled right into the operating system in the form of network communication and device aggregation features.

But what’s that about a “frameless music keyboard”? Or a “touch-sensitive media device”?

Let’s talk about the touch-sensitive media device first.

“An electronic apparatus, such as an electronic mixing apparatus and an electronic keyboard apparatus, and associated methods are disclosed. The electronic mixing apparatus or the electronic keyboard apparatus is provided on a touch screen that provides user input and display capabilities. In one embodiment, the touch screen is a multipoint touch screen so that multiple user touch inputs can be simultaneously acquired. In another embodiment, surface guides can be provided on the touch screen to assist with user input.”

See Touch-sensitive electronic apparatus for media applications, and methods therefor

Multi-touch Music, a la Lemur? It sounds a lot like the iPhone, but as I noted shortly after Macworld, this particular patent has Dr. Lengeling’s name right alongside Jonathan Ive’s, and describes elaborate audio mixers and music keyboards and sophisticated multi-touch gestures. None of these would be possible on the tiny iPhone screen.

Plug-together keys? Then we see the “frameless keyboard” patent, which is more recent — filed 2005, granted December 14, 2006. Interestingly, this patent makes reference to the fold-up patent design we broke here on CDM, a design which unfortunately, well, folded long before it went into production. This patent is not assigned to Apple; it’s assigned directly to Dr. Lengeling. The idea is to combine keyboards in segments, minus frames, for an arbitrary number of keys. Dr. Lengeling also has a patent for simulating the feel of different mechanical keyboard actions — again, not explictly mentioning Apple. Maybe Dr. Lengeling, an accomplished keyboardist, is working on his dream keyboard. (If you are, do share it with us when it’s done! And maybe we can have a Gerhard vs. Gerhard smackdown with Gerhard Behles, CEO of Ableton.)

Before you get too excited, just because a company is working on an idea and patents it doesn’t mean it’ll get into a shipping product; there are countless intervening steps. If you’ve followed Apple’s design history, you’ll know the company was working on phones and boomboxes in the late 80s that never materialized. On the other hand, Jobs hinted in his iPhone keynote that he’s aware that paperwork can tip your hand. (In the case of iPhone, it was an FCC filing, but Apple product hints have snuck out in patent applications, as well).

A New Logic? Equally interesting is that this stream of patents hints at the plot line of the post from early today: Apple has a next-generation music app, and they’ve been working on it for five years. That’s the rough time period of the various application features here. As I read the patents (always a challenge), it sounds like the “graphical” methods of displaying audio and video signals and what looks like automatic beat detection and tempo correction features are beyond what we’ve seen in existing apps. It’s possible the “graphical” audio display is just the spectrum view in Soundtrack and the tempo correction bits are related to Apple Loops (again, even experimental ideas that may not have gotten into the software), but it seems possible there’s a longer-term project here, as well.

Hmmm … rumors about a sub-notebook, tablet features, multi-touch, the shipment of the iPhone, rumors about touch-sensitive music applications, patents for touch-sensitive music applications, rumors about an app that’s been in development for five years, five years of patents for audio software features …

The honest truth is, it could all be meaningless. In the meantime, feel free to dream. And this being CDM, feel free to continue to create your own wild inventions — they could prove to be cooler, or real-er, anyway.

For more on the “I just read a blog by an Emagic founder and there’s not going to be a Logic 8″ plotline, see:

Rumor Mill: No Logic 8; New Touch-Sensitive “Pro Tools Killer” Instead?