Integrated Melodyne pitch correction in PreSonus’ Studio One is made more interesting by the technology behind it. Celemony this week describes a new technology they call ARA, or “Audio Random Access.” The notion is this: rather than just receiving or generating audio signal, the plug-in gets access to audio data. That means you can actually write a plug-in that rewrites the audio content in a recorded DAW track, as Melodyne does in Studio One.

As developer Celemony describes it, “ARA opens an additional channel of communication through which the DAW and plug-in can exchange information about the audio file, tempo, pitch, rhythm and much more, which allows them to work together considerably more closely.”

It’s the ability to exchange audio data information that seems the most compelling. Previously, audio processing plug-ins simply took buffers of audio signal from the DAW. You could “look ahead” further into that signal by increasing the buffer (and thus latency with it), but generally speaking, you’re doing the processing in something that approximates real-time. ARA in the example of Melodyne gives you access to an entire recorded track without having to transfer the audio file to and from the plug-in.

Celemony says this is “an extension of the existing plug-in interfaces,” not a new plug-in format. (If it were the latter, I’d have to point to this xkcd cartoon.) I’m still obligated to express some skepticism about how widely this will be adopted, or if it can be considered a “standard” extension, though they do promise additional vendors soon. (Implementation would seem to be by necessity on a host by host basis – and then once you have the host, a plug-in creator might add support.) It’s a proprietary technology, but then, so are the plug-in formats currently in wide use (AU controlled by Apple for Mac OS, VST by Steinberg, and RTAS by Digidesign, unless we see more of LV2). For now, though, we’ll have to see if the idea itself can extend what a plug-in can do. Check out the videos for more.

celemony.com/ara (no documentation for developers, but there is an email address to use if you’re interested)

  • http://noisepages.com/members/renzu/ renzu

    That's actually pretty cool that they're tackling this problem. It would be nice to see more plugins that have better integration with DAWs. It's always awkward to deal with plugins that have their own DAW-independent sequencing features, and then face horror of restructuring a track you've made in the DAW, and have to mirror those changes in the plugin's sequencer, etc.

    And of course this opens up the possibility of more analytical plugins that can "look ahead" arbitrarily long distances to make decisions.

  • ABZyrd

    Truely revolutionary! Couple of questions though. Is this tech & seamless integration coming to other DAWs like Ableton & ProTools? Also one feature which would make it even more fluid is if you could change the pitch of the various elements using a midi keyboard instead of a mouse. Also if you could adjust some of the timing or other feature using your midi interface.  

  • sampl

    This 'new' approach is simply offline editing with metadata.

  • Ycros

    Fantastic. Reading the previous article, I was thinking, "we need an API for that!".

    I just hope the licensing on the API is open enough. VST and others' licensing has always annoyed me.

  • Paweł Rychert

    I'm not a technician, but I think it could also be adapted in some very sophisticated dynamics mastering plugins, right? Where the program can see the whole track and automatically adjust parameters in different parts, depending on their harmonic content etc. Another tool in the loudness war, I'm afraid ;) But also great way to make the workflow much faster and nicer, and more human. I'll keep my fingers crossed for ARA

  • http://www.warriorbob.com Warrior Bob

    My first reaction to reading this was "oh look, Celemony has invented Loading a File Into a Computer Program."  But thinking about that for a minute, I don't know of any DAW software that uses its plugins like this, and I suspect the workflow that arises from it will have some neat surprises. It's basically violating one of the assumptions (buffers) that most plugin writers have made about how their design will work, and the general result of that is that someone comes up with something new that they otherwise wouldn't have. So now, instead of skeptical, I'm interested.

  • Dave TD

    To be honest, I'm struggling a little to grasp the complete concept, but it does look very interesting and I'm curious to learn more. the idea of looking ahead would be incredibly useful for dynamic processing, compressors that are truly adaptive or eqs that maintain a constant frequency balance, regardless of the signal would be quite unusual and pretty incredible. For better of for worse though, I always find myself comparing an effect, when possible, to it's real world counterpart. something like reverberation for example acts in exactly the same way as the guys describe conventional plugins, one note at a time. It's an interesting tech for sure, but I'm interested to see which processes it can be successfully applied to.

  • bar|none

    This was a disappointing interview in the sense that the information was presented poorly. Kind of the typical developer trying to explain a technical product and failing. A demo would have been much more enlightening. This sense I get is the new capabilities of the API are designed to solve the melodyne use case. If you are looking ahead at information, then this is stuff that by nature is prerecorded, whereas my interest is more with realtime performance. It's unclear what the advantage is in that case.

  • http://zeroreference.blogspot.com zeroreference

    Question – is this tool accessing the audio file in memory, essentially?

    Is this different (more computational?) than receiving the buffer of an audio file (a stream?).

    I don't know a lot about how audio plugins work. It sounds like this is interacting with the _data_ of an audio file, as opposed to a (buffer/stream?) of that information. Which sounds pretty cool.

  • Greg

    @barnone: See the second video for a detailed ARA showcase. It makes things pretty clear, don't you think? Sure, it's basically about Melodyne use cases, but that's because Melodyne has always been suffering from the limitations of existing plug-in formats. People have blamed Melodyne for its complicated workflow, when it was actually the existing plug-in formats not allowing for more. So, there is nothing wrong with developing a new technology to work around any limitations.

  • http://soundcloud.com/exit_only Nick Inhofe

    The 2nd video does a great job of demonstrating why this is a cool technology. 

    If anyone has worked with melodyne before it's clear why this is a huge leap. While powerful, working with melodyne in Logic can be PITA very quickly. 

    I don't think the ideas behind the technology are groundbreaking by themselves. Anyone who has tried a program like this where there is an analysis stage would have had the same idea in about five minutes. The difference is that they actually did it, and have a working DAW that supports the technology. This obviously makes it more enticing for other DAW developers to support the idea.

    My biggest hope here is that they do the right thing and make this an open format and not attach a bunch of licensing fees on top of it (cough Rewire).  

    This could be incredibly useful for any plugin where there is an advantage of knowing whats going on everywhere in the track. I would love to see a phase vocoder work like this right in Logic. 

  • vaikl

    Ok, the 2nd video shows it clear: ARA is the missing link between the fantastic features of Melodyne and the normal workflow in a DAW, something I was missing in Melodyne/DNA until now and what was promised with their first DNA version. So Celemony has just followed up on this promise now and maybe the other DAWs will follow soon.

    Hope to get SO v2 asap to check the CPU load:))

  • http://ardour.org/ Paul Davis

    another way to think about this sort of thing is that its an attempt to solve a "problem" created by the (probably rational) unwillingness of most audio developers to implement host-level applications and by the lack of source-level integration between hosts and plugins.

    if plugin developers wrote "host-level" applications the issue wouldn't arise in the first place. if plugin developers had full access to the data structures of the host, then the issue wouldn't arise in the first place.

    but plugins and plugin APIs exist to solve another equally real set of issues: writing hosts is hard and complex, especially in comparison to many plugins. and most plugin developers don't want "full access" to the (possibly ever-changing) internal data structures of a series of different hosts.

    so here we see a technological solution to a problem created by a technological solution to a different problem that is also created by the same general technological environment.

    and as peter noted, it seems to be mostly promoted as a set of tools to fix up poor performances or "fix" a guitar line that lacks "freshness". it almost makes me want to weep.

  • Conner

    core audio has something like this, the 'auol' plugin type (offline)

    no one really uses it though

  • http://www.hicox.com plurgid

    Uhm … passing a filehandle to a plugin?

    Revolutionary, or "doing it wrong?"

  • Experimentaldog

    I always want to creatively explore the other side of the technology.  I don't understand why Celemony keeps demonstrating their tools as if they're only built to be a cheap fast way to performance perfection.  Come on now, show us what happens when you throw a stick in the spokes.  Show me the creative potential not the mediocre perfect pop song that anyone can make once they have these plugins in their DAW.  Plus, the self-referetial interviews are a bit much at times.  That being said though, it would be great if there would be an ARA bridge with Max to any DAW.

  • Jason Acuna

    It just looks like Celemony wants their plugin embedded into all major daws..  and hopefully a $$ of the DAW sales perhaps?

  • http://www.avachorda.de Lind Bohm

    Embedded in other DAWS earlist Q2 next year. Something like a blocking period to support Presonus.

  • David

    This type of communication between plugin and DAW is already implemented in Reaper. I used it to communicate with one of my plugin builds last year.

  • BlueSpark

    This could probably be implemented as an LV2 extension.

  • http://ardour.org/ Paul Davis

    @bluespark: will be implemented as an LV2 extension.

  • a1x

    man… carsten scares the hell out of me.

  • http://drobilla.net David Robillard

    Indeed, this would be very simple to implement in LV2.

    The hard part is getting some plugins written that actually have a need for it…