Austrian Composer Peter Ablinger has transformed a child speaking so that it can be played as MIDI events on a mechanically-controlled piano, making the piano a kind of speech speaker. Via Matrixsynth, the readers at Hack a Day get fairly involved with how this may be working.

It seems not quite accurate to describe this as vocoding in the strictest sense, so much as a simple transformation to a (much) lower frequency resolution – that is, the 88 keys of the piano. Ablinger, for his part, describes the events as “pixels.” It’s pretty extraordinary that without a bandpass filter, you get something approximating the noisy sibilance of the speech, but this seems to be the result of having lots of events (that is, lots of resolution in terms of time). Edit: Listening again, the short answer to how you can hear so much of the voice through the piano seems to be, you can’t; the original is almost certainly mixed in. It’s nonetheless an interesting effect, and I’d like to hear the piano on its own. In other words, the basic process is, 1) convert the sound spectrum of the recorded voice to a series of MIDI events, and 2) play back the translated MIDI file. You can see that the MIDI playback is accomplished with Pd (Pure Data) running on a Windows Linux/KDE netbook, though it’s not clear what was used to do the original conversion. (The screen shot with side-by-side audio and MIDI appears as though it may be for demonstration purposes, only.)

Correction: The work is absolutely done in custom software developed by the composer in Pd (Pure Data). It’s an ideal tool for the job, and free and open source. I wouldn’t dare try to replicate the results here, but this is fantastic inspiration for playing with sound in Pd.

One Windows tool that’s capable of the job is TS Audiotomidi, as observed by Hack a Day spacecoyote. Whether or not that’s what’s at work here – and it may well be – that utility is itself interesting. Edit: Yeah, far more likely the whole thing was done in Pd. And Pd should be up to the task.

TS-AudioToMIDI

Of course, this is to say nothing of the lovely work done on the mechanical piano. It’s a beautiful piece. Here’s hoping some government bureaucrats got the message of the declaration. Now, we just need a chorus of something really loud – say a thousand trumpets – shouting out the Universal Declaration of Human Rights.

Sonic scientist Peter Neubäcker of Melodyne has been wowing Internet audiences for some time with the automagical powers of the company’s Direct Note Access (DNA). The vision: manipulate individual pitches as easily as MIDI notes, even in polyphonic passages of a single instrument. At NAMM last month, the company showed the first product, Melodyne editor, due to ship in the spring for US/EUR 349.

There’s just one little catch: a solo developer has beaten them to the punch, at least prior to them shipping their DNA flagship editor tool. And if you want it right now, it’s yours for 25 Euros. (The final version will cost 99 Euros.)

Jonathan Schmid-Burgk, sole developer and a student at Harvard, announces:

The time has come to announce the release of the world’s first published polyphonic tone manipulation software. The dream of musicians to isolate single notes out of chords and so to manipulate most forms of recorded audio has come true on the 20th of January 2009.

Shell out EUR25, and you get a Mac VST plug-in that can manipulate audio easily. With monophonic audio, you can create polyphonic harmonizations. You can isolate and manipulate individual harmonics – meaning not only can you do pitch manipulations, but presumably sound design, as well. You can change individual notes or chords in recorded audio, to fix mistakes or (more interesting) actively recompose audio.

I feel about this the same way I do about Celemony: this gets really interesting when you use it for sound design. For some inspiration, skip this post and head straight for the sound samples on the site:

VisualVox polyphonic 0.9 [improvisator.de]

Via the awesome rekkerd.org

Also check out his Harmony Improvisator which creatively generates harmonies from MIDI input – an interesting thing to mess around with even for those of us who know / have taught (ahem) classical harmonic theory

Now, VisualVox Polyphonic isn’t without some catches, as you’d expect from the solo-student cheap alternative:

Looks can be a powerful agent for changing how we think about sound. Pairing liquid, organic speakers with equally fluid and dynamic visualizations, the launch of Muon last month in Italy made this principle readily apparent. I’m all about lo-fi, cheap gear here on CDM, but if you absolutely must launch luxurious aluminum speakers with spectacular animated visuals at a posh party in an Italian salon, I sure won’t complain. Pass the prosecco, please?

Muon Project Page, documentation videos at chrisoshea.org
See coverage at ze | d | esign, toxi’s project blog, MoCo Loco, elsewhere. (Yeah, CDM’s motto is: cover things last. Was a bit busy with Maker Faire!)
Created by Moving Brands

Details on the installation and how it was done: