Microsoft Research has done some amazing work; it doesn’t always move me to tears, but there’s some fantastic stuff that deserves real recognition. And MySong is … well, technologically impressive, if musically painful. It’s a sort of collision between AutoTune and Band-in-a-Box: it recognizes a melody as input, then harmonizes that melody.

The vocal input goes well, and illustrates the number of different inputs beyond the mouse you can expect in The Future. Here’s the problem: harmony is extraordinarily difficult to model on a computer because of the number of variables, the amount that’s driven by instinct and art. And let’s be blunt: it doesn’t work right.

In short: if you’re planning to build a Jerome Kern robot, the technology may not be there just yet.

It’s been almost a year since Windows Vista was released to consumers. We know that nearly half of our readers use Windows, so the future of the OS is something we take very seriously — even if many of you, for now, are staying cautious and working (happily, in many cases) on XP. We’ll be examining Vista from various angles over the coming weeks, both measuring the OS and telling you how to make the most of it if for music you are giving it a go.

To start out, we’ve again caught up with Noel Borthwick. Noel CTO of Cakewalk, and one of the most knowledgeable experts on Windows technical details. (He’s also a veteran Linux developer, so his perspective on operating systems goes beyond those from Redmond.)

When we talked to Noel this time last year, a lot of what was new still hadn’t been tested in the real world. Now, Vista has been in the hand of users, and there’s both some good news and bad. A year of Vista has meant a year of improvements, both from Microsoft and third parties. In my own testing, for instance, what began as a disastrous experience running Vista earlier in the year has now become more comparable to XP. (I’m currently on Vista SP1 release candidate on a modest PC desktop.) But there are still areas that could use improvement — and while general Vista improvements were welcome, I think there’s still the real question of whether Vista offers enough that’s unique to compete with its real rival, XP.

We’ll revisit some of those broad issues, but first let’s actually get the technical story, and clear up some misconceptions.

Your ears and mind are incredibly sensitive to tiny details of sound. Result: if your operating system can’t keep up with sound output for any reason, you’ll get a noticeable “glitch” in the sound — and that’s a big deal. Windows Vista promised to be “glitch-free” in development, later reworded to “glitch-resistant” or “glitch-resilient.” Then it shipped, and a lot of us noticed it was, well, just plain glitchy, at least at the beginning of this year when Vista met up with half-finished, buggy drivers.

All operating systems will glitch under certain circumstances, though, and the causes are many. Microsoft has a great post on their Vista Team Blog today from Steve Ball, who seems to be a really sharp guy and has a great handle on how Vista can continue to improve in terms of audio performance.

An Overview of Windows Sound and Music “Glitching” Issues

Well worth reading, whether you’re a Windows user or not. (Linux and Mac can absolutely encounter the same issues, and as you look through the full list of possible causes you’ll see why.) There’s quite a lot missing from this discussion, but the blog promises this is part 1 of 2, and you’ll find some more meat in the discussion in comments.

I did enjoy this description of why we’re so bothered by glitches:

My colleague on the Windows Sound team, Larry Osterman, also pointed out to me recently that humans are actually “hard-wired” to be disturbed by audio glitches. In an exchange about this topic, Larry observed that audio glitches are more obvious than video glitches because the ear’s tuned to notice high frequency transients — his visceral example of this idea is an image of a stick snapping in the woods behind you as an audio event that wakes you up before a bear wanders into your path.

I think I have the same visceral reaction to software bugs. (Help! A bear!)

I’m writing this from Vista right now, and I have to say, I found all kinds of reproducible glitching problems early on. But now, various hotfixes and driver updates better, Vista’s audio performance is running really smoothly for me. I think the major culprit on Vista in the early months of the release was video drivers, an issue which for me, and many others, has finally been fixed.

The writing’s on the wall: DRM for music downloads is deader than the eight track. Okay, actually, that’s not fair: the eight track was relatively good technology.

Just two weeks after Amazon launched their own DRM-free music store, Microsoft is jumping on the bandwagon. DRM won’t be gone completely from the Zune store, but there will be hundreds of thousands of DRM-free tracks going live, apparently in November with the release of a new store and new players. That’s a major departure, given that Microsoft built its Zune and PlaysForSure platforms around DRM, and introduced significant new DRM features in Vista. Normally, I don’t like to quote press releases, but I’ll give my friends at the Digital Freedom Campaign a chance for some deserved gloating:

“The industry standard has shifted in the past six months and the tide has turned in favor of consumers,” Maura Corbett, a spokesperson for the Digital Freedom Campaign said. “The number of digital music retailers offering DRM-free music will soon out number those that do not, and consumers will soon live in a world where they can listen to legally purchased music when, how, and where they want. We congratulate Microsoft for joining the growing number of retailers and labels that have realized the best way to increase the sales of digital music, is to listen to their customers.”

Actually, I’ll argue one point. The number of digital music retailers offering DRM-free music already outnumbers those that don’t.

I’ve found the Windows Vista Team Blog to be largely disappointing in terms of actual OS information, but they’ve got an interesting post this week from Windows Vista audio team Program Manager Richard Fricks:

Using a microphone array to enhance sound capture

Microphone array recording is a technique for improving recording quality by processing signal from multiple microphones. Compare the signal, and you can more easily focus in on the source. The application here is really not music and “pro audio” — in this case, I think they’re targeting consumer-grade mic arrays that would replace, say, the lousy built-in mics in laptops and webcams. It does sound very useful, that said, and I would be interested to know about any “pro” applications that do exist for this sort of thing, beyond making podcasts and Skype chats work better (which is a worthy thing, and something I myself use).

The Bigger OS Picture

But here’s why you should care, even if you don’t really care: in Windows XP, a lot of these sorts of functions were handled by device drivers instead of the OS. That could wreak havoc because of the low-level function required for tasks like echo cancellation. One particular third-party vendor (cough, Logitech, cough!) was shipping XP drivers for its webcams with echo cancellation turned on by default. It caused all sorts of problems, disabling pro audio hardware interfaces without warning and even nasty Blue Screens of Death (at least in my experience). It could entirely disable your music setup until you turned it off. Now, some might say you should have a dedicated machine for audio production, but that ignores the reality that lots of third-party, consumer hardware plays just fine on the same machine. That’s part of what’s great about using computers for music. So, clearly, this unnecessary chaos could be avoided.

Look out, PCs: you’re getting the audio engine from the Xbox 360. That’s the message from Microsoft, which abandoned the old DirectSound APIs in Windows Vista. They’ve got a new audio system called XAudio2 ready and waiting, however, and it looks good — though it also begs the question, why didn’t Microsoft ship it with Windows Vista out of the gate? (Instead, Microsoft actually suggested users turn to the OpenAL open audio architecture, and now appears to be getting XAudio2 ready for Vista SP1.)

Geek alert: the rest of this post may be interesting only to developers…

XAudio2 does look more ambitious than many other audio architectures in that it includes programmable DSP effects baked right in, plus some nice mixing and spatialization features. This stuff is largely aimed at gaming, but it could yield some interesting music applications, as well:

• Multi-channel and surround-sound support with full per-channel volume and mapping control.
• Programmable, cross-platform DSP effects framework.
• Per-voice filtering, arbitrary submixing, and multi-rate processing.
• Multicore optimized, non-blocking API design.
• Pluggable and generalized 3D spatialization support, with a full-featured implementation provided by the independent X3DAudio math library.

“Cross-platform”, though, in Microsoft fashion, should actually mean Xbox 360 + Windows Vista.

For more:

Microsoft Announces DirectX 10.1 Preview, Betas New Audio Tech [ExtremeTech]

Meanwhile, the state of true cross-platform engines is not so fantastic. OpenAL, a multi-platform, open source 3D audio library, comes closest and appears actively updated, though your mileage will vary depending on platform. DSP and many other features have to be provided on your own. Sun, meanwhile, has left the Java platform a mess; the dusty, rusty Java Media Frameworks hasn’t gotten an update since 2003, the open source Java efforts are stumbling on multimedia support because so much of what’s required is proprietary, and no one seems to know what’s happening next.

As noted last night — with some very witty responses from incredulous readers — the record industry is now pushing for DRM on all radio. It’s a bad idea to begin with, and they’re bringing it up in a context in which it doesn’t even belond, negotiations on royalty rates, at a bad time — in the midst of negotiations that have broken down. I’d love to stop covering this issue, but the most recent round is too absurd to pass up. (Feel free to spread the word, since Congress demonstrated that, at least on a basic level, they’re listening to you.)

So, record industry, why is it you would want to push for a broken, proprietary, exorbitantly expensive to a problem that doesn’t exist as part of a discussion to which it’s entirely unrelated? The RIAA’s Senior Vice President of Government Relations (otherwise known as Grand Poo-bah of Politician Lobbying) Mitch Glazier was happy to explain to Technology Daily:

“Why wait until it is a big problem to start addressing it? There are available technologies in the marketplace to address this issue.”

Yes, indeed. Why wait for a problem to actually exist before legally mandating a solution? A technology exists! Therefore, you are obligated to use it — regardless of cost, whether it functions on the devices people use, whether better technologies exist, or whether there was even a problem in the first place. Which would you prefer: a record industry that works to solve today’s real problems, or one that creates massive, new problems to solve the problems they imagine might exist in the future?

Act now, fellow musicians — before Sound Recorder destroys music!

It’s amazing how complete and total crazies can suddenly wind up with the backing of organizations powerful enough to dictate the law. Witness the strange story of the “stream-ripping” scare, and how it somehow led to a push for mandatory, proprietary DRM on all Internet radio.

Gasp as the experience of bringing back Mark Twain’s ghost somehow inspires a company you’ve never heard of to build their own DRM for streams!

Recoil in horror at the evil pirating capabilities of Windows Vista and its Sound Recorder, as Microsoft earns billions — billions! — of dollars by encouraging people to steal music from radio streams!

Sigh with satisfaction at the realization that we can put a stop to these unprotected broadcasts of music forever, saving music itself in the process!

What? None of this sounds familiar? Bizarre, absurd, even illogical and out of touch with any recognizable reality, you say? You’re right, but alas …read on.

(See previous: Internet Radio Wins Temporary Delay, Possible Minimum Rate Break. You knew it wasn’t really going to be that easy, right? Apparently some of you missed my sense of irony. I was on vacation, so I wasn’t trying as hard to make my sarcasm apparent.)

There’s a simple problem: sound is invisible, and sound synthesis concepts don’t have any physical reality. Knobs, faders, patch cords, keyboards, infrared sensors, touchpads, and the like all work quite nicely for synthesizing sounds. But take a closer look at Bjork’s use of the reacTable, an interactive multimedia interface that uses a camera to track the movements of blocks on a surface. They really are using it to make sounds, those sounds really are visualized in a nice new way (watch the waveforms connecting the blocks), and while the result is some swoopy synthy sounds, the interface does make making them a lot of fun.

It helps that Bjork pulls out some of her synthiest, electronicilicious-est tracks, like Pluto:

and Hyperballad:

And, of course, part of what happens is that the computer screen here has become the interface. When it works — when the visuals match the sounds, and suggest some new ways of constructing music — it really does show potential for this kind of instrument. (Even if you don’t buy into the blocks, the way the visualization itself works has a lot of promise.)

That’s the idea behind Microsoft’s Surface, too … but sometimes the gimmick can be a solution in search of a problem. Well, actually, maybe your computer of the future really will be “a big-ass table.” (Thanks, SarcasticGamer.com, for making me laugh so heartily.)