The wait is over – and rumors that Melodyne’s bleeding-edge technologies to allow direct access to notes in polyphonic audio had failed to come to fruition turn out to be false. (I was skeptical about those rumors in April.) Melodyne DNA did take longer than expected to ship, but then, that isn’t exactly news in the software business. And now you can try this “note access” concept for yourself and see what you think (well, provided you’re an existing customer). Coupled with time-based manipulation of audio in the form of updated tools in Ableton Live 8, Cakewalk SONAR 8.5, Propellerhead Record, Logic Studio 9, and others, with Melodyne handling the pitch, audio today could be more fluid than ever.

Celemony Melodyne
(Note, since you are a bleeding-edge type — the software is also considered compatible with Snow Leopard, though host-by-host certification is still forthcoming.)

What Melodyne’s editor enables:

  • Harmonies are accessible, note by note.
  • Pitch, position, duration, and loudness and softness can be modified.
  • Formant spectra, vibrato, and pitch drift are accessible.
  • Pitch, amplitude, and formant transitions between notes can be edited.

Now, all of this is in a plug-in, but that plug-in is more capable than previous versions, with better multi-threading, an adjustable window size (sigh of relief), and the ability to audition and scrub as you edit. That’s not quite as good as having this functionality in your host, but it’s more than good enough to make this usable.

I’m especially interested in what unusual sound design possibilities can be harnessed using this technology – abusing it rather than using it as intended.

Here’s how to get it:
Registered Melodyne customers are able to participate in the beta.
Launch is planned for early November at US$349 / EUR349.

I’m testing a copy now, so if you’re not a Melodyne user, I will get to report back.

More videos:

(Part 3 of 3 appears to be temporarily missing; I’ll add it once it is re-posted!)

  • http://UiandUs.com Keith Lang

    This really is phenomenal technology.

  • aaron m

    kind of afraid where this might lead…

  • Adam

    Not vapor-ware after all! I too was skeptical. Now I have a audio-broner.

    And at $349… that's an okay price. Pretty damn cool.

  • poorsod

    my mind is completely blown

  • Jeff

    My mind was already blown when I saw the teaser video like a year ago. But the reason why I'm still exciting now is the very well-thought-out tools to correct imperfections that the cooler heads among us reasonably predicted would still exist.

    The third video (available here) shows how DNA treats overtone misidentification, which is a difficult problem even while listening with human ears. To deal with that, Celemony seems to have added a deep but simple tool to see "under the hood" at how DNA has identified those pitches, and correct them by hand in a very elegant way–separating overtones into new notes, or combining them where they are single notes.

    It's too bad I'm not in on the beta, but I really think it's starting to look like Celemony is delivering on the insane hype with a real, effective tool.

  • http://live.nwrecords.com gbsr

    noway. i honestly thought it was vaporware aswell.

    well, im about as happy as a clam right now then.

    especially if it delivers on nontraditional recordings for experimental use :D

  • http://ardour.org/ Paul Davis

    "a real effective tool".

    tools generally have a purpose. what's the purpose of melodyne w/DNA ?

  • http://ardour.org/ Paul Davis

    another comment. once upon a time (not very long ago), this sort of audio manipulation was the preserve of "academic" musicians using CSound, SoundHack, SuperCollider etc. it was generally dismissed by "mainstream" musicians and audio engineers. how is it that by applying the idea of complete audio plasticity to existing audio recordings, it suddenly becomes so exciting and full of potential? is it just another sign of the complete increasing unwillingness of musicians and even composers to actually do any work, or learn any really hard skills?

  • http://www.createdigitalmusic.com Peter Kirn

    Paul,

    As I understand it, there are now – and as far as I know, have always been – two potential targets:

    1. The recording session at which there's some minute change you want to make.

    2. A creative application of manipulating sound.

    Now, just out of curiosity, what engineer or musician ever thought this was a bad idea? I believe you; I just think it's absurd. The change I've heard from engineers is mostly that they're now believers in digital technology – that they think it can do the job the rest of us always thought it could do. That's not a growing laziness – it's a growing awareness.

    Anyway, the research in doing this still comes from academics. And, incidentally, the academic community has not tended to be terribly forward thinking about releasing their work to the public – they've often been just as eager to find a way of commercializing a lot of the research, particularly recently as other revenue streams have gotten tighter (all around the world). I can't really fault the creator of that idea for finding a commercial utilization for it, to be honest.

    I'm still keen to see open source, published, and non-commercial applications of ideas – and I'm neither interested in or capable of coming up with some super-secret sauce to sell. But I think it's worth considering both the academic and mainstream development of an idea.

    As for whether there's a reason to want to change recorded audio – well, is there a reason to edit, or a reason to record? (I suppose those are reasonable questions to ask, actually.) I'm sure there are completely boring things to do with the technology; I'm sure there are non-boring things, as well.

  • http://keithhandy.com Keith Handy

    Paul: every musical technology that ever comes along is accused of rendering skill and talent obsolete, from player pianos, to mellotrons, to drum machines, to autotune… and I'm just naming the first few things that pop into my head. It never becomes true. There is always a phase when the new toy is used to nauseating overkill, but in the long run it becomes a companion to talent, not a competitor.

    Using Melodyne well will require creativity and a good ear for music. When people trust or expect it to do all the work for them, though, the results will be generic and disposable.

  • Jeff

    To Paul:

    "what’s the purpose of melodyne w/DNA ?"

    What's the point of a reverb effect? A delay effect? Yes, we have uses for them now, but really, who cared when they were first developed? Delay came from someone introducing a tape feedback loop. It was a curiosity and only a tool amongst experimental musicians for a long time. Nowadays, it's a staple effect.

    First, far from a curiosity, DNA has clear and obvious application and usefulness. Granted the software is not perfect, but given what we've already seen: You can edit performances on a per-note level, saving potentially hours of retakes to get that "perfect" take. You can use it for analysis and seamlessly move into editing. You can do macro harmonic editing–i.e., decide after doing hours of painstaking recording to just try to map your instruments onto D# instead of Bb, and have it revise your instruments in seconds (yes, watching the videos *does* yield some interesting information). I find that pretty damn impressive for "old world" applications alone.

    Second, a basic feature that separates Melodyne right now (and DNA in the future) from those "academic" pieces of software, is the ease of use and the fact that the pitch/length modifications actually sound good. Really good. Seamless unless you are going octaves or full measures out of the original recording range. They are appropriate for use in real musical composition and *not* simply a curiosity. Your comparisons to Supercollider (care to spend two weeks learning to program?) and Soundhack (fine if typical, effects plugins) are not just inapt–they're perplexing.

    This doesn't even touch the experimental applications, which I expect to be substantial.

    If you're not seeing practical applications of this technology, and how this differs from "academic software," I don't know what more I can say.

  • http://www.createdigitalmusic.com Peter Kirn

    Well, and I think the idea of simply commercializing a unique technology is best understood as the status quo. The notion of being able to *intentionally* share, in an organized way, as is possible with free and open source software is relatively new, at least in this form. Personally, that still makes me excited, what Paul has done and continues to do. I don't think the two are mutually exclusive, and – without speaking for them -I don't think the folks at Celemony would frown upon continued experimentation.

  • flip

    Wow… Looking forward to trying this out!

    Skeptics: If you used this software, you'd know it wasn't designed for people without talent. The whole user interface revolves around pro knowledge of pitch, time and digital editing.

  • http://bandcamp.corbu.com hexatron

    well… i really wanted to feed it a beatles song in a major key and make it minor, just to test it. it can't handle something like that at all. there were even parts of vocal/guitar that it couldn't really distinguish.

    i'm sure it would be better with something like a singer and an acoustic guitar, like the video example. but i'm pretty disappointed. i know it's in beta, but my results were pretty ugly.

  • http://www.paniq.org Leonard Ritter

    Meh, not open source? fail.

  • http://www.derrick-s.com Derrick S

    I think its an amazing product

    for example

    when you lost your project material for what ever reason you could use your master files to geht the inidvidual parts back

    of course will not be the same, but better than trying to start from 0

    i don´t see this negative at all, when someone wants to steal your ideas or anything he can allready do this, don´t need DNA for this

    i´m very excited to see how people are going to use this in a creative way

  • http://corbucorbu.com hexatron

    derrick: that's what i thought, too, but it doesn't seem NEARLY as precise as that. you might be able to convincingly change some notes that a guitar plays with a single vocal over it, but full band recordings look way out of DNA's league. maybe a few versions down the road.

  • http://www.fernandogros.com fernando

    Spent every moment I could today feeding stuff into it. Did some amazing things changing keys and chord progressions on solo Jazz and Country guitar stuff.

    One thing I'm realising is that my (largely underused) library of loops now just became a lot more valuable. Time to hack into those keyboard loops!

  • http://www.professorwhimsey.com Dr Apostrophe X

    SWEET! I've been waiting for this since August, 2008, when I got Melodyne Plugin (pretty amazing in its own right) and I'm encouraged by the comments here from folks who have tried the beta (I'll download it tonight).

    Hexatron: if you read the blurbage on the webpage (IIRC), it was never intended for multi-timbral sound sources (such as entire tracks with bass, guitar, drums, vocals), but for single multiphonic instruments such as piano, guitar, etc.

    As for me, I never doubted those Celemony guys for an instant :-)

  • Angstrom

    I plan on taking my portable recorder everywhere I go now, and if I see an unattended instrument of excellence (pipe organ, expensive guitar) I'll whap out a small part of one of my tunes. Then I can take that recording home and put it in time/tune with the actual track.

    Wait, is that stealing?

  • http://ardour.org/ Paul Davis

    @jeff: i am perfectly on clear on what DNA can be used for and what it can do. the issue i am raising is one that i've raised many times before. lets look at one sentence you included: You can edit performances on a per-note level, saving potentially hours of retakes to get that “perfect” take. There's nothing inherently wrong with this, and I'm not trying to say that there is. But notice how this (and other related technology) provides both positive and negative reinforcement of certain kinds of aesthetic judgement, rather than leading you to ask questions about the process of judging itself. Why is someone searching for a "perfect" take when none was ever performed? I am not asking this as a leading question – I am asking it as an active question, and one that I think needs answering. It has been around since at least Glenn Gould's time, and possibly before (though presumably, it does not predate the relatively recent invention of recorded music). DNA, in my view, discourages grappling with this question, because it simply provides a new technique for accomplishing this goal, with great ease and enormous fun. Even if the goal of its inventor was more experimental, it is clear that much of its use will be just the example you gave that I cited above.

    I know that the lines between music performance and music recording were more or less erased quite a while back. However, DNA is bringing a notion of music-as-audio-putty into the production of music from a different direction than the kinds of academic sound design/composition environments I mentioned. I'm intrigued by, rather than critical of, the fact that this different direction seems to get all kinds of people excited about music-as-audio-putty who were never remotely interested in the work done with "more experimental" tools, even though the methodological approaches are converging. Is it only because one can be imagined to be about smoothing out a trumpet solo in some smooth jazz and the other is about wierd "modern" music? I don't know.

    I probably didn't make it clear enough that I wasn't comparing/contrasting DNA and its reception with "academic" software. I was really talking about the underlying notions of what recorded sound is, methods of manipulating it and so on and so forth. I am fully aware that the user interface, indeed, even the likely use cases, for DNA look very different than the programs I mentioned.

    Just a note: DNA's pitch/time stretching algorithms are, to my ears, no better than the current crop of good ones from other sources. I suspect you feel the same way, and were referring the quality of it given the context, in which case I agree.

    reverb: this is a means to recreate the acoustic effect of performing in a particular kind of space.

    delay: although it has other purposes, i see delay originating primarily from some affinity with basic music constructs like canons/rounds.

    i don't see either of them as being related to the capabilities of software like DNA, which fundamentally is about manipulating a performance that was recorded using audio sampling.

  • http://live.nwrecords.com gbsr

    stealing, reusing, recycling. all different words with the same result.

    who cares.

    change one note and its a new sample. ;)

  • http://www.createdigitalmusic.com Peter Kirn

    @Paul: Well, that does raise another question — just what issue are people wanting to correct? To me, for this reason, the creative application is far more common than the "fix that take" application. I agree with some of your concerns about fabricating these non-existent perfect takes (and, indeed, some of the Apple demos I heard worked so well that they sounded boring)! But it seems far more common that you'd be tweaking or nudging a rhythmic difference here or there than changing actual notes. I guess it remains to be seen what these applications will be. But yes, for creative sampling – which is what some people are already talking – for abuse, this could be interesting.

  • http://ardour.org/ Paul Davis

    @keith: Using Melodyne well will require creativity and a good ear for music. When people trust or expect it to do all the work for them, though, the results will be generic and disposable.

    And you don't find most of the results of using current music technology (note I said "music", not "audio") to be just that?

    I would dispute your claim about technology not reducing skill. I think it opens up the possibility of aesthetic creation to people who, yes, have less skill than those who use "more difficult" means of production. That doesn't mean that the more skilled, perhaps I should say more virtuosic, creators don't use it, and it doesn't mean that there isn't a level of skill involved in using it well. But there is absolutely no doubt in my mind that music technology has created a set of musicians who are less skilled as virtuosos, less skilled as musicians in terms of their musical intuition and understanding, and less skilled in their ability to interact with others.

  • http://www.createdigitalmusic.com Peter Kirn

    Paul, compared to what? Compared to mastering the bassoon? (Okay, better examples – yes, I expect the concert percussionist is better at reading rhythmic notation than the average home Pro Tools user. And also, very possibly, better than the reed players, but I digress…) It seems like they're different media to me. You could make the same argument that notation, by allowing composers to write music they can't play and give it to someone else who can, allows the composer to get away with less musicianship, and … well, I'd be guilty as charged. But I think you'd also have to agree that music as represented on paper is a different medium than music played by an instrument.

    And all of this is off the original topic. Consider the likelihood of the following two scenarios:

    1. Melodyne DNA causes a massive drop in musicianship, as engineers tell musicians to play as many wrong notes as they possibly can. (Engineer to musician: "We'll fix it in post!")

    2. Melodyne DNA causes a massive headache for engineers, as musicians tell engineers they can play as many wrong notes as they like. (Musician to engineer: "You'll fix it in post!" Engineer to self: [groan].)

    I'm betting on #2, and I think that's quite different from technology somehow enabling a generation of lousy, new musicians.

  • Angstrom

    Paul, your contention seems to be that music technology has reduced the need for music makers to learn traditional musical skills such as 'playing the piano well'.

    Well, this process of musical techno-ification has been underway for at least 10 years now, so I guess the only way of seeing if your theory is correct is a graph of pianists passing grade 8 over the last decade.

    Sadly I can't find one, but my guess is that any such graph would show no decline in standards or numbers of virtuoso players.

    These modern tools are not for, or impacting Grade 8 pianists, these tools are for the people who would have been called "mix engineers" and producers in the 1970's – 80's.

    You may not like the idea that a mixing engineer can now produce what sounds to the casual listener like 'music', but the reality is that is what is happening. Furthermore a reasonable percentage of these people produce music which is enjoyed by a large number of people. You may consider it inferior or detrimental to 'real' music', but it is not. There is more music, that is all. The percentage of generally 'good' to 'bad' is probably about the same as ever.

    Fortunately alongside the tools to help us create music, technology has brought us tools to help us filter music to our tastes.

    So now : If you don't like T-Pain then you do not have to listen to him.

    Whereas in the past there was no escape from the "Bay City Rollers".

  • http://www.createdigitalmusic.com Peter Kirn

    Well, speaking for myself, I'm certainly capable of being a mediocre pianist with or without a DAW. ;)

    I think these are questions worth asking. But in the broad sense, it still seems to me that it's a bit like blaming photography for the downfall of virtuosic painting. And, in fact, you'd be right — the rise of photography resulted in fewer painters, less of an industry around painting, less of a market for painters, and massive changes in how painters saw themselves and the techniques they used. On the other hand, photography is not painting for dummies; despite the dramatic impact it had, it *was* a different medium. And the analog here seems to be as much recording as it is digital recording, let alone specific editing capabilities of digital recording. Nor did photography require that people give up painting; indeed, some of the most radical developments in painting came after photography, not by coincidence.

    Should people be suspicious of technologies that promise to let you rework every pitch and rhythm of a recording? Absolutely, I think so. I'm sure it's very possible to suck the life out of recordings by misapplying those capabilities. But if you look at the fundamental change – the way recordings are becoming malleable – you also have a way to sample yourself and produce new sounds that really change how that recording works as a medium. That can be said of any digital recording technology, I think, not just Melodyne – true of Ardour – but it's a good thing.

  • http://ardour.org/ Paul Davis

    @angstrom: i don't follow your logic. i didn't claim that music technology would lead to a decline in people pursuing "traditional" pathways to performance skills. i was suggesting that it leads to an expansion in the number of performers overall, primarily by allowing those whose performance skills would otherwise be insufficient to nevertheless create, at the very least, quite listenable music. i think this claim is entirely consonant with your observations about "there is more music, that is all". i don't agree with the idea that the percentages haven't changed – i spend a LOT of time on emusic.com, and there is an awful lot of recorded music there which would never, ever have been released in the vinyl-only days. i remember eno commenting on the need for artists to learn to throw most of their work away. in fact, when on emusic, i tend to remember that thought several times a day :)

    btw, there was always an escape from the bay city rollers. when they were #1 on the UK charts, i was listening to tangerine dream and klaus schulze. it might have cost me a girlfriend or two, but in the end it was worth it.

  • http://twitter.com/benzo8 John Sullivan

    @hexatron: The product is not designed to allow you to analyse whole songs in one shot. That's not what is being done in the video either (you'll see it's just the guitar in Melodyne Editor until he says about "using all the controls in your DAW" – the vocals are on another track…)

    If you had the individual tracks of your Beatles' song, you could analyse and rework each seperately in Melodyne Editor but a whole mix is never going to work.

  • http://ardour.org/ Paul Davis

    @peter: is the malleability of a performance recorded via audio sampling a goal that serves the production of art? obviously such questions have no universal answer, but even for any particular individual, i think they should be asking themselves that question. what does someone believe they are accomplishing via the various kinds of manipulations that are performed? why one kind and not another? why perform at all?

    and sure, i'll be the first (or second, i suppose) to say that a DAW, and even before that, the razor blade, are tools that allow one to go in search of "the perfect take" or "the piece i could never perform myself" long before DNA was implemented (i'd say imagined, but my guess is that varese imagined DNA-like capabilities back in the 50's).

    btw, this comment stream made possible by: the length of time it takes to compile ardour on a 1GHz G4 … :)

  • http://www.createdigitalmusic.com Peter Kirn

    Paul, if I understand you correctly, that's assuming that you're just moving notes around in a pristine fashion in a way that could be accomplished by simply *playing different notes*. I'm assuming that, personally, for a creative use to be really satisfying, I'd want to produce something that's impossible for anyone to play, and ideally wouldn't *sound* like something someone had played.

  • http://ardour.org/ Paul Davis

    @peter: not assuming that at all. but there are dozens, hundreds, maybe thousands of ways of getting to what you describe as the end goal that do not involve starting from an audio recording of a performance. this has been done for decades within the electro-acoustic music community. what's the benefit (and cost) of the "new" approach ("perform it, record it, manipulate the recording so that it doesn't sound like a performance")?

  • http://www.createdigitalmusic.com Peter Kirn

    Yeah, but Paul, if this is a synthesis versus sampling argument, I just don't think there's anything to come from that argument. There are zillions of ways of doing things, indeed. You know, to a certain point, technological innovation really is "because it's there." I don't know that you *need* Melodyne DNA, just like you may not need a whole lot of things. So, anyway, let's be patient and see what people do with it.

  • Adam

    I can't wait to use this in conjunction with MS SongSmith…

  • http://compusition.com Adam Murray

    FYI, you can get this for $300 if you buy Melodyne plugin now. From the plugin product page: "You can still buy Melodyne plugin now and upgrade later for free. This will even save you US$/€ 50!"

    Interesting discussion in the comments. I just thought I'd chime in and say I plan on using this tool for 2 purposes. (1) To alter royalty-free loops to turn them into my own creation. (2) To abuse the tool to make experimental sounds, for example by using it on material it can't handle properly or misusing the overtone correction features.

    (1) probably gets into the "you're being lazy and not writing your own music" territory for some people. I used to think that way too and avoid any loops with a passion. But I managed to amass a huge collection of loops over the years, and the fact is, sometimes a sample/loop serves as the inspiration for an entire song. When that inspiration hits, why not run with it and actually make some music? Who cares where the source material comes from, there's still infinite opportunities to be creative and turn any set of digital samples into unique creations of our own, especially with tools like this.

  • griotspeak

    is it wrong that i want this mainly to transcribe stuff (my old songs that i have lost the project for mostly?)

  • Steve Elbows

    Some strands of this conversation remind me of the wider 'long tail' debate about what the ultimate result of blogs and other things would be, what lowering the barrier to entry will actually mean. Personally so far I have loved the idea more than the results. If this stuff ultimately answers the question as to whether there may be millions of talented people out there, who have previously been prevented from doing anything due to high barriers to entry, and the answer turns out to be negative, then its another boost to the age of cynicism, another avenue of hope sealed with a sigh.

    I was a kid who always gave up too easily, desperately wanted to create, but got way too frustrated with failures to draw or play any instrument in a way that showed any sign of ability or talent at all. But I never gave up the dream, and in the last decade or so Ive been getting kit and learning how to use it. But none of it has in any way made up for lack of natural ability or lack of practise. I dont know if things will ever come together, but maybe I will get somewhere in the end, and so I suppose I dont know what the world gains by treating the lowering of barriers with fear or contempt. Granted it is quite easy to see the world as having been dumbed down in many ways, and peoples expectations getting out of whack, hell its even easier for me to think this now as I am getting older and losing touch with the young, but Im more likely to blame television, the prevailing direction of the ideological/political winds in recent decades, and our economic system than the tools.

    I dont care if tools enable people with less ability to create the same old pap that could previously only have been done by greater talents, brilliant performances will always shine through, just as all the fancy graphics in movies dont stop people recognising a very special film when they see one, or completely diminish the chances of one being made. I dont care if the tools enable lazy people to be even lazier, just so long as the potential is there for someone to do something great with them once in a while, whether by design or happy accident.

    I suppose I wear a frown if I think about fake it till you make it, or the quest for perfection leading to sterility, but as already indicated I dont think the tools are the root cause of this, even if they are enablers I dont think they are the true driving force.

    Anyway as I am rambling on I will stop myself now with a quote from Frank Zappa that goes something like 'talking about music is like dancing about architecture'.

  • http://www.myspace.com/jamesyanisko James

    Ok, from a Beta tester here:

    I have tried a couple of small things. Small changes, taking a guitar line where I play single notes, ring them each out while changing the notes and some chords – I am able to change the chord in the editor and it sounds just like I played it.

    It seems to separate the percussive sounds of a note pluck / hammer on differently, and you have to realize that and try and adjust it without getting some strange artifacts.

    I've added extra notes, and changed timings, and it seems to work pretty well on a small scale – trying to fix or change just small things.

    and two reasons I would use it.

    1) just like we use pitch correction now, great performance, slightly off note here or there….fixed…done.

    Maybe a great performance – played the wrong chord…done, fixed.

    2) as a musician who makes music using samples – this pushes the doors wide open on what I can do with a good sample.

  • Scott Meschke

    @paul and peter

    Interesting discussion:

    I'd like to add a few bits.

    Products like this may help people not as talented make good music. Who says this is a bad thing? Don't get me wrong, the Jonas brothers and all the other artists who are autotuned and produced to perfection bug me as well, but I think that's mainly due to a "musician's ego."

    I think music enriches everyone's lives, and is that a bad thing that a tool can help someone make it. If using melodyne DNA on a Chopin piano nocturne helps a student identify whats going on in the music, and learn composition techniques and theory from it?

    I say no. Let the average Joe play with it, learn, experiment, and make music.

    Are writers against thesauruses?

    Are poets against rhyming dictionaries?

    Are most photographers vehemently against photoshop?

    These are just tools, they don't tell you what to do, and don't make the music for you. But, the more accessible things are the better in my opinion. I don't listen to music because ,"they are so talented."

    Occasionally, but more often for music's sake. And if this can help more people REALLY experience music, I'm all for it.

    About the perfect take thing, you aren't taking into account money. Bands/Artists pay a lot for studio time. And if they pay a lengthy piano solo, and the musicality and expression (the things that can't be defined by an algorithm) are there and they miss one note in the middle, they can't use that. And that wasted time adds up in lots of money.

    I'm looking forward to playing with and abusing this thing. Both for creative, correctional, and educational use.

    Tools don't sculpt the sculpture for you. And generally, the easier the tools make it, the better the best sculptures are. These tools allow real musicians to take things to the next level.

  • flip

    @Paul Davis:

    As a professional composer/performer (and melodyne user for 2 years now) I have this to say…

    Nobody is forcing anyone to polish turds with this software. This technology allows you to push the envelope with any sound or instrument. It is just as valuable as a music tool as it is a sound design tool (or the two combined). You can use it to create things which would be impossible to play in reality. That doesn't make me or anyone else any less talented or lazy: It is a skill to use software to it's fullest potential.

  • http://ardour.org/ Paul Davis

    @flip: please don't misunderstand my comments as being about (a) forcing people to use stuff or (b) discrediting the skills of some or even most of those who will use m-DNA.

    When new technology is introduced, there is normally a lot of clammer and noise from people who stand to benefit from it, either financially or through some improvement in some task they perform. We don't tend to hear much from the people who, now or in the future, will be negatively impacted by it. Sometimes it because they haven't even heard of it yet, sometimes its because the negative impact on any one of them is so small as to be apparently negligible, sometimes its because it takes a long time for the results to become clear. Its very clear to me what the capabilities and possibilities for m-DNA are, so I am not really so deeply curious about them. What interests me more area the "long term" results of this kind of technology on aesthetic creation in general. Nobody has to be forced to do anything for a piece of technology to have drastic impacts on the world in which it appears. TV, MP3 are just the two simplest, most obvious examples of this.

  • flip

    @Paul: I don't think I misunderstood you. In your own words…

    "is it just another sign of the complete increasing unwillingness of musicians and even composers to actually do any work, or learn any really hard skills?"

    That implies technological advances in digital editing could be completely transforming musicians or composers into lazy people without skill.

    I never implied that you said anyone was forced to use melodyne. My statement, "Nobody is forcing anyone to polish turds with this software." is pretty simple: Anyone can choose not to use it or even use it for a task they might not have imagined.

    Have you tried melodyne yourself?

    Do you think the previous version has had any "negative impact" on any composer or performer? If so, who?

    Have you seen any change in "aesthetic creation" since the first version came out?

    In my opinion, there is no substitute for talent and creativity. I have never seen any technology that was a substitute for either. You're right that there is a lot of clammer from people who stand to benefit from new technology, but it's usually accompanied by a lot of clammer from those who seem afraid of new technology.

  • http://ardour.org/ Paul Davis

    @flip: i guess i'm going to get have very very careful with my words:

    is it just another sign of the increasing unwillingness of people who currently do not, but are planning to try to perform or compose pieces of music intended for others to listen to to actually do any work, or learn any really hard skills?

    i have tried melodyne. i've even been playing with the free VST that does a moderate approximation of m-DNA for a few months. negative impact? i didn't think that a particular person using it would be negatively impacted. if you live in the USA, you weren't negatively impacted by shopping at a remote mall until most of your neighbours did the same, and your local stores shut down. there can be long term impacts to technology that do not manifest in "the music of X.Y has become worse since they started using it".

    have i seen a change in aesthetic creation? since melodyne? i think thats an artificial break point – the original melodyne have i seen a change since quality timestretching (& pitchshifting) became common in audio software? certainly. has it been for the better? my impression is that its mostly favored a continuing effervesence …

  • http://www.wiretotheear.com/ Oliver Chesler

    Last one to figure out the new genre is a rotten egg.

  • http://live.nwrecords.com gbsr

    @John Sullivan: actually it can analyze whole songs.

    check this thirdparty demo:
    http://www.youtube.com/watch?v=GCiZS_ChCdU

  • http://www.professorwhimsey.com Dr. Apostrophe X

    ZOMG, you guys… this thing is freaking me out. I've been feeding guitar and piano loops into it and it's like you can apply all your favorite Melodyne tweaks to each individual note of the loop… it works as advertised. It really works. I can't believe it!

  • salamanderanagram

    WOW.

    it runs slower than death on my 3 year old computer (takes several MINUTES to load a file and then every edit causes a substantial delay) but other than that, this is totally amazing.

  • http://www.jonathanleonard.com jonathan adams leona

    Hi, awesome new technology that is very exciting and did not exist in academic or commercial form until now.

    I have used Melodyne myself in sessions getting paid by the hour to operate it for clients. Commercial autotune was available but ironically this was smooth jazz and with this culture (jazz) it is not acceptable usually for an engineer to interfere with jazz musicians expression to melodyne is preferred because you can be very specific with what notes are adjusted and by how much. Specifically melodyne excels in the analysis of group recordings or sections and renders them with transparency allowing you to see clusters in the performance and where the outliers are. Very excellent approach and based on the clients direction, we were able to make adjustments to individual notes on stems preserving much of the original expression.

    There is a cultural side to this tool and that is where we get our notions of 'correct', 'fixing', or 'out'. These are not universal principles but apriori baggage inherited from previous cultures and a continuous fight for identity and voice.

    In reaktor some years ago I decided I wanted to make an ottotoon. Not because I cannot sing, or have some 'faschination' with correctness but because I am a wise ass with serious turbulent tendencies and I wanted to learn how to do it on my own and show people that Antares sells some really expensive software. I started with pencil and paper and made some pictures of flow and feedback and came up with ideas like irrational or unintentional pitch and rational ones….how to store pitch tables and lastly how to make a change to audio in realtime. What fun! But it very much made me aware of what these little things mean and people's earnest desire to please. I released the tool for free to the reaktor community and have improved it with fft and vocoding to show that vocal fx are possible and interesting, but I removed any reference in the comments and information to 'pitch correction' to the be replaced with 'pitch quantization' so I could neutralize the cultural contexting. The first version I made offered some scale presets which is some serious cultural bias right there, but later versions had more editing capability but I never went for full on microtonal editing because its free.

    I have used my own pitch quantization tool called ottotoon to create the color and effect of quantization, as well as create nice three part harmonic styles in things like pentatonic harmony which sounds much more interesting than tonic harmony. Anyhow.

    Melodyne is just a great tool and for me even better now with DNA. Besides the uses that others have suggested above, I think it will be useful for analysis of pitch. I guess some group of people will focus on deviations to remove them, but man I think those deviations are down right fascinating, and I think they are signature.

    I know a few things about the human cochlea, but not what they taste like though I suspect slightly crunchie and with a surprise squirt when they pop. What we hear through this interface is not a direct connection, but before processing layers. When I was a student in biology I had access to auditory specialists including those studying the systems in bat. I asked and confirmed that acoustic interference does happen in the nervous system. This means that nervous impulses representing sound are subject to the same physical consequences when sharing a communication channel and might experience additive, subtractive interference.

    The cochlea contains both efferent and afferent nerves. This means that inside that supposed fft detector since the 'mic' is the timpanic membrane, are both things that resonate and create impulses, and cells that pump sound INTO the cochlea. In fact this is where tinnitus comes from. All the afferent hairs are vibrating and won't stop with a feedback into your efferent hairs. Anyhow, this describes some parts of the sensory side, but not the voice side when a person sings and actively controls their voice to produce a certain pitch. The cochlea is organized in its spiral to fit a magic number or natural log. This makes sound wrap around its delicious cinnabon swirls and coincide in a linear fashion across the cochlear axis. Meaning, the standing wave pattern of a sine wave will excite nerves in straight line across the apparatus. Just like god designed it.

    I have found in my own experience being a person with passive perfect pitch that there are many styles of temperament that may only be described as biological in origin. There really is no perfect pitch and even those who claim it to exist know in practice that it must be maintained, is subject to change based on a person' cultural exposure and also a person' age. I have noticed since I am usually more sensitive to pitch, and since Ihave sung in choirs that people have innate rather than simply wrong temperament.

    What I am describing are the deviations from a perfect pitch table corresponding to decimal pitch values. I have observed with my ears that people have a tendency to sharp and flat certain notes consistently. Like an old saxophone that is in tune except for all the F's, people have a tendency to exhibit their own temperament that may reflect their own inherent biology and culture.

    Said another way, people are unique in their pitch. People have fingerprints, and social security numbers. We have DNA of another sort and there is a suggestion that there might be a molecular definition or augmentation of individual identity. I can't wait!

    I have also noticed that people from the same family singing in harmony exhibit a group effect based on their shared inheritance. Aside from obvious cultural influence I very much theorize that a biological influence is involved though I have not produced any evidence yet. To study this one would have to have people sing the tonic under different conditions, collecting those deviations as events to eventually suggest a pitch fingerprint or signature. I suspect that doing this in families and across cultures will produce and reveal a biological force. One must only record people singing, and create a database of their deviations.

    When people sing together they do their best to sing in tune and they listen as well as change their own pitch. You can have 2 musicians play a beat together and get in sync which is a great operative test for AI and maybe modeling 2 particles in a simple universe listening and influencing each other and the shape of space between. Our universe is literally filled with oscillators.

    I very much appreciate the flavor and sound of people and their individuality. When combined in a group things happen that are so completely unpredictable and wonderful. We all have some innate temperament that is like a fingerprint that should be expressed if we have any love for the individual at all.

    I think people of all colors should be celebrated for their differences and the idea there is any pitch more correct than another is rudimentary and is in ignorance negligence and belligerence of the beauty of the pitch inside of all of us.

    Melodyne's use of the term DNA is poetic and useful in many ways. It is a system we can use to analyse and understand our world through pitch, and also perhaps suggesting the parts of a system we cannot escape.

    Thank you Peter, and especially for Ardour and all the hard work, Paul.

    jonathan adams leonard

  • http://www.myspace.com/keatshandwriting Keats'

    @ gbsr

    I don't believe that it is true that if you "change one note … its a new sample. ;) "

    I was writing a paper on this topic recently.

    The 6th circuit recently held that even when a sample wasn't recognizable that it was still a violation of copyright law.

    From Wikipedia:

    Bridgeport Music, Inc. v. Dimension Films, 410 F.3d 792 (6th Cir. 2005), is a court case that has proved important in defining American copyright law for recorded music. The case centered around N.W.A.’s song “100 Miles and Runnin’” and Funkadelic's “Get Off Your Ass and Jam.” Essentially, N.W.A. sampled a two-second guitar chord from Funkadelic's tune, lowered the pitch and looped it five times in their song. This was all done without Funkadelic's permission and with no compensation paid to Bridgeport Music, which owns the rights to Funkadelic's music. In its decision, the court wrote: "Get a license or do not sample. We do not see this as stifling creativity in any significant way." This decision effectively eliminates the de minimis doctrine for recorded music in the Sixth Circuit, and has affected industry practice.

  • flip

    @jonathan adams leonard:

    "There really is no perfect pitch and even those who claim it to exist know in practice that it must be maintained, is subject to change based on a person’ cultural exposure and also a person’ age."

    I'd say perfect pitch is really just a memory skill. I'm not sure how much one's youth effects the development of such memory, but I can speak of it first hand… From an early age I could listen to music and play it back after a first listen. I had teachers who said I had "perfect pitch" because I could identify notes within a melody or chord structure. However, when I started djing and composing music that wasn't in concert pitch, it got a little nutty. If anything, it got way more complex! Either way, it's common knowledge that "concert pitch" has changed quite a bit over the years…so I've always viewed perfect pitch as a parlor trick, if anything.

    p.s.

    Cilia probably doesn't taste good.

  • Pingback: wire to the ear » Blog Archive » Coming soon: A new music genre

  • usedtobe

    oh you guyss :P

    DNA will let us do something we couldn't before. positive!

  • http://curtismacdonald.com/blog Curtis

    im curious to hear how the tool works at different sample rates and bit depths – anyone try testing low or very standard rates over very high ones? bit depth, etc?

    totally gonna buy it once it comes as a full release regardless…

  • http://ww.jhhl.net/iPhone Jhhl

    This is very exciting and takes audio editing a few more steps closer to being as flexible as music event editing.

    One next step would be to take the abstracted notes out and analyze them into synthetic sound sets. (I'm assuming the re-synthesis of Melodyne's audio is not granular sample mashing, but more sophisticated formant based synthesis). Those sound sets could then be played by themselves or swapped and dropped into the mix, completely replacing the instruments or voices playing.

    Another step would be controlled deconvolution of audio as a preparatory step for this software, removing ambient time-based effects and saving the FIR data so it may be manipulated and reapplied after re-synthesis.

  • http://www.personal-computer-music.com f.e

    @paul : i share your feelings. But i would add that high level instrumentists are always bad at creating / composing music.

    #2 : perfect pitch is disgusting. Only perfect feeling / intention / performance is worth.

    #3 : don't use these kind of tools for what it was built for. I would not use it for pitch correction but would rather try to use its defaults to do something else

  • Yeah, yeah

    Keith Handy said:

    Paul: every musical technology that ever comes along is accused of rendering skill and talent obsolete, from player pianos, to mellotrons, to drum machines, to autotune… and I’m just naming the first few things that pop into my head. **It never becomes true.** There is always a phase when the new toy is used to nauseating overkill, but in the long run it becomes a companion to talent, not a competitor.

    It never becomes true to ANY DEGREE? WRONG!!! This is basically, like most of the contra-Paul Davis commentary, dumb naive Whiggishness about the allegedly cost-free nature of our dependence on technology. PD is overstating it to say that this will make musicians lazy, but Jesus, it's either intellectually dishonest or foolish to think that things like this DON'T EVER have negative impacts. Listen to today's popular singers, then go listen to doo-wop from the 50s, with its five-part harmonizing. Our vocalists today often cannot manage to sing in key BY THEMSELVES.

    The naive optimism of technology freaks and scene people is always amazing to me. Oliver Chesler thinks this will create a new musical genre. Well, let's talk first principles and ask what a genre is if we want to do more than hear ourselves talk. As if the multifarious vain, bitchy semantic/typological discussions of electronic music *trends* are serious discussions about actual, autonomous 'genres.'

    Try to get some sense of history, people. It helps one to understand, you know, anything.

  • http://www.lukeharrald.com.au Luke

    Well… having been testing the Beta for the last couple of weeks, I can only say that while DNA may or may not produce new genres of music (in academic circles, Paul Lansky was using this kind of stuff years before autotune), it will certainly make life much more creative for DJs. As a creative tool, it works great for bending loops around how ever you would like them (avoiding any chance of a copyright infringement court case!) It will certainly also revolutionise the world of mashups.

    DNA does do certain things straight out the box, but generally (as has already been suggested) if you don't have some understanding of music & acoustics, and a reasonable set of ears, then you won't get anything that is actually good outside simple pitch correction, which it does do very well straight out of the box as you would expect.

    The arguments about skill / musicianship are irrelevant. The technology is here, and is not going to go away. People will still always go to live shows to see musicians & DJs because it is a totally different experience to listening to music at home or on your mp3 player.

  • Robert McEwen

    Take one previous gangster or blonde bimbo who can speak… mix with some loops of instruments played by people who can actually play….. pitch correct the bad rhythmic blabbing…. pay clear channel to push your "music"….and you have a way to make a quick buck with litle work.

    Oh wait….. that has happened already…. this may just make it even easier…

    God help us…

    QUote:"People will still always go to live shows to see musicians & DJs because it is a totally different experience to listening to music at home or on your mp3 player."

    Live shows by who ? people who need to rely on this stuff can't play live anyhow. Up to now we have people getting caught lip syncing… Who knows what the next tech breakthough will bring… the computers crap out and you discover it's all fake. Soon the mainstream music industry is dominated by "the Monkees 2010 edition"

  • Sean

    Spliting waves by pitch is a handy tool for the audio engineer, and budding musician I would agree.

    But has anyone tried to create a similar expansion of a wave by exploring timbre, as opposed to pitch? I mean if you are recording a guitar it could be kind of cool (useful for analysis) if a wave could be split into the combination of waves that result from different parts of the instrument.

    Picture the melodyne editors expansion of a wave into its harmonic components, now picture the expansion of a wave into its timbral components, based on the acoustic components of an instrument. That really would be interesting, acoustic instruments are notorious for distorting which you shift sample pitch. If for example from a guitar you could focus on the wave which comes from the head, the neck, the strings, and the nut independantly. Isolate them into different waves and explore the sound that way I think that would be a useful development interms of exploring the modern acoustic theory.

  • Sean

    Oops I mean ". . .when you shift sample pitch."

  • Sean

    Maybe little directional mics attatched and positioned at each point of the guitar just to pick up the isolated region could give an idea of what how to isolate sounds in the spectrogram?

    Call it Direct Timbral Access (DTA)

  • Sean