Celemony’s Melodyne plug-in could already perform incredible feats of pitch manipulation with audio. But monophonic audio is one thing. Polyphonic audio has long been sound manipulation’s final frontier. With Melodyne 2, it seems Celemony’s audio wizards have finally cracked the problem.

plugin_2_screen

Celemony is showing their new technology at Musikmesse, and they’ve got demos online you can check out:

Direct Note Access

Grab a note inside a chord, and you can manipulate that note directly. Retune it, change timing, adjust formants, change amplitude — timbre, time, and pitch are all accessible. Celemony is largely pushing this as a corrective tool, as that’s an obvious market, but needless to say, creative applications — even creative abuse — become interesting, too.

Melodyne Studio costs US$399 (349 EUR), with various discounts for upgraders, and the technology will be making more limited appearances elsewhere in Celemony’s product line. Now, it is a plug-in — clearly, someday this sort of thing will just be integrated directly in your host of choice, and I’m particularly excited about the day when it becomes a live performance tool. But for now, it could well be worth the cost of ownership.

You’ll have to wait a bit: the new version is scheduled to ship in the fall, though if you buy now, you’ll get the update free. Celemony, I’ll be seeing you at AES, I think.

Compatibility: Mac (Intel/PowerPC), Windows (XP/Vista)

Thanks to everyone who sent this in (Alex, Karsten, Eric, and others)! By popular demand, the demo video SonicState grabbed at Messe, because they’re organized enough to actually be in Frankfurt while I chill out here in NYC:



MESSE08:Melodyne Blows Our Minds

  • Audiomaker

    I hope to see this applied to midi guitar technology as well as sampling / synthesis in general.

    I wonder if / when they will start to license it?

  • http://www.createdigitalmusic.com Peter Kirn

    Well, you actually don't *need* a MIDI guitar with this. You could record a guitar, and then change the recording of what you just played as if it were MIDI, but of course with the full sound content of that recording and all the nuance that contains, which MIDI wouldn't provide.

    License — I hope so. Perhaps as iZotope has done with a lot of their tech and other DAWs, we'll see "powered by Celemony" features in Logic, SONAR, Live, etc.

  • http://www.sighup.ca Steve

    Check out the video Sonicstate posted of their presentation at Messe, it's even more convincing than the stuff on the Celemony site.

  • flip

    I use their current plug-in… Less as a corrective tool, but more for remixing things. Changing a melody to work with a new key or modality is pretty powerful. 2.0 looks even better.

  • http://www.toilville.com peter

    Both evil and awesome, like all things that are truly awesome.

  • http://www.deltasleep.net deltasleep

    "But think about it, this is not meaty."

  • http://www.deltasleep.net deltasleep

    also, on a more valuable tip, i want this! Can't wait to see what I can do with it.

  • nylarch

    I wonder how the artifacts introduce compare to time stretching – i.e. if you stay pretty conservative its barely noticeable, if you crank it it starts to sound bad (which can be good at times of course)…

    Is it comparable in that if I move it a minor third its less artifact-y than if I move it 2 octaves?

  • http://keithhandy.com Keith Handy

    It would be fun to experiment with changing minor chords to major and vice-versa.

    It'll really impress me if it can pick out notes that are only a semitone apart, playing at the same time.

  • http://keithhandy.com Keith Handy

    Okay, I just watched the video, and now I feel like my last comment was redundant. :)

  • Mr. Tunes

    i am skeptical that it will sound this good. but i also don't think they would fudge a demo like that. could they be that amazing at developing software? this is really throwing me for a loop! i'll have to check out the sonicstate video as steve suggests

  • Goobs

    Amazing. His upcoming invention is direct transduction of thought to musical score.

  • http://www.alog.net gola

    absolutely insane this is…. It is a completely new way of working with audio in my opinion.

    Regarding abuse, it will be interesting to see how this will work in sample-based music…. if an hip-hop artist change all the chords, and tweek the melody of some famous old recording and use it as a hook, is it still the same piece? How different has it got to be before they can get away with it?

  • bliss

    OMG!!! It's the ultimate transcription machine! If you can't get a filmscoring gig after buying this software, you should just quit. All the secrets will be revealed. Those mysterious John Williams chord voicings will no longer be a mystery. This new Melodyne will be of enormous value to composers and music students. Yay!!!

  • http://andrew.hicox.com plurgid

    I can't load the video now, perhaps it's slashdotted, so this is an uninformed comment, but … call me skeptical …

    I'd imagine you'd have to feed this a *very* dry signal to get it to work.

    Maybe I'm overly skeptical, but I can't imagine feeding a big complex signal like an orchestra, with all the overtones and reflections and reverbs into a machine and having it spit out the individual notes … and then letting me CHANGE them … AND having the result not sound like crap.

    I CAN imagine feeding an unprocessed guitar into it and getting notes out. But I still can't imagine changing a note and having it not sound like crap … at least not in a complex setting. Maybe if you were real careful on your chord changes and didn't leave any squeaks or ringing strings … maybe.

  • http://www.scs-trc.net/x2008/ c64

    quoted from Jamie K and the paris news group…

    "I can imagine the sessions:

    "Hal, we need you to strum a chord"

    "Just one chord?"

    "Just one chord, Hal, that's right."

    "Uh, what chord?"

    "Doesn't matter."

    "Like this?"

    "Great, thanks."

    "OK, here I go…"

    "No, we're done, Hal. You can leave now."

    "Uh, really? OK, see you next time."

    "We won't be needing you any more Hal, we have the chord to work with."

    "wha…?"

    "Hal, please clear the studio, we have the bass note to record, the horn

    section stab, and then we need the rest of the week to work with your

    chord."

    "But I blocked out four hours…"

    "Sorry Hal, we're only paying you for thirty seconds. But listen, if you

    play that same chord on the piano on your way out we'll double your pay

    and give you a soda. Which is like tripling your pay!"

    "Yo man, you're a real jerk!"

    "Nice, we can use the 'yo.' Thanks Hal."

    "Mother#*&@ ****&#(#^!"

    "Thank you. goodbye Hal."

  • http://www.createdigitalmusic.com Peter Kirn

    For all the skeptics, we actually don't have to speculate much — Celemony already had Melodyne, and I've tried it. It works as advertised. I do expect a cluttered polyphonic recording could cause problems, i.e., you will need a clean recording. But it makes sense that it'd be possible theoretically — ever looked at overlapping overtones of different musical parts? On a spectral view, you can see how they relate to the fundamental in most timbres, if well recorded. So it's a math problem. Not an *easy* one, but not an impossible one — and Celemony's done a great job, it appears, building that into the interface. I'll make sure I'm on the list to test it in the fall.

    @c64: lol.

  • http://blog.ortz.org Ortzinator

    Oh god this makes me moist.

  • Angstrom

    Peter Neubäcker says on the Sonicstate Messe video that large reverbs and dense mixes don't really cause any problems. What does make it go crazy is clusters of swooping or decending notes. It's all discussed at the end of the sonicstate video anyway.

    at 12 minutes in you can see / hear him pull apart a 1950s Chet Baker recording of a whole band.

  • bliss

    What I'd like to know is if after a piece of music processed to reveal its note by note content, does Melodyne 2.0 also provide a look at the material in standard notation? It's not essential since melodic, harmonic, and rhythmic information is already available in Melodyne's "regular" view, but it would be cool to be able to view the information in standard notation.

  • Pingback: Weblog – Future Audio Workshop » Blog Archive » More from MusikMesse 2008

  • http://www.alog.net gola

    SIngers now has to start singing with huge glissandos and chinese opera-style vibrato to protect their work from this monster.

  • http://adaequat.org karl

    well, from a technical point of view:

    - most instruments show a strong base frequency at the basic note, and harmonics with an odd and/or even whole-number multiply factor

    - this means if you do a fine FFT and start to "Look" from the lowest frequency upwards, the first strong discrete freqency should be lowest note in the chord

    - you mulitply by 2/3/4/5… and you find all the harmonics releated to this lowest note

    - you move on from this first tone, and if you hit the next strong frequency which is NOT an odd/even multiply of the first, you have got the next note in a chord …. and so on…

    with this technique, it is plausible that you can separate all the notes from a cluster of 12 half-notes within one octave, played at the same time. if (for the sake of easy explanation) the base note is 100Hz, the corresponding harmonics are at 200, 300, 400, 500Hz etc… the next half-tone above this is 100Hz times 12throot of 2, so it is 105,94Hz, which can be discriminated from 100Hz with a good FFT. so even from this very complex chord, the 12 notes with their harmonics can be clearly identified.

    I think the only problems might be OCTAVES played simultaneousely (as those are precisely double the frequency)and the non-harmonic noises which are important for some instruments.

    fascinating processing anyway.

  • harmonics

    @ karl

    you may wish to do actual experiments to conclude if your claim that harmonics fall at exact integer multiples is true.

    the problem is a bit more tricky.

  • Pingback: Sharing Quality » Celemony Melodyne DNA: Has this man done the impossible?

  • http://www.sonicstate.com Nick@sonic

    Sorry if the video has been a little clunky, we've been rather snowed under this messe. But I was at the demo and it was pretty amazing. I suspect that the audio examples were well chosen, but they were diverse for sure.

    It was kind of like a Steve Jobs keynote speech – cheers and woops etc. I would love to listen in a more sterile environment for sure, but either way it has enormous implications for music production.

    I would think that it will work pretty well, I just cant see Peter Neubaker trying to fool us – he's a pretty straight guy

  • sheabe

    I have never had such a feeling of awe at a piece of software before.

    Hardware yes, but this?

    It's like the Guy has invented Sampling All Over Again just like Sonic Foundry Acid changed looping from something of an art to an auto response.

  • http://www.myspace.com/archiveofeverything Chris Blundell

    all thats left now is turning a full multi-instrumental recording into a series of midi and sound samples and i'd be in heaven. Can't imagining it happening any time soon though. ;)

  • Pingback: Change Audio Notes Like MIDI: New Melodyne 2 Direct Note Access-Download Music Free

  • Pingback: WikiRecording’s Blog » Blog Archive » Celemony Melodyne DNA: Has this man done the impossible?

  • http://droppopandlockit Guitar Player

    I see many comments here about MIDI who cares about MIDI it has always sounded crap anyway do these people not understand what this man has done these are AUDIO files here real sounds data played by real people and they are being used like Midi Files I dont realy think this Achievement has sunk in yet for some people , it has for me being a guitar player ,in brief i think this will go down as one of the greatest achievements in music history ———

  • http://droppopandlockit Guitar Player

    Just as a finale note to Peter Neubacker i will forgive you for putting me out of a good job as one of the best Guitar Players in England if you send me a free copy of DNA lol

    and once again congratulations

  • Javier Ferrer

    wow this program is amazing