Can editing sounds be as easy as editing pixels in a tool like Photoshop? That’s the question asked yet again by an audio editor, in the announcement of a new tool called Spectral Layers, seen in a new teaser.

Visualizing sound is not a simple problem, but you can do worse than the spectral view. Mapping frequency over time rather than just amplitude, the graphic spectrum illuminates components of a sound as we hear it, showing sonic energy of different frequencies in brightness and color. And audio editors have routinely made use of these views, whether as displays in various audio editors (some editable, some non-editable views), or in graphical tools like the ground-breaking MetaSynth. In fact, even Adobe themselves have weighed in on the “Photoshop for sound” notion with their own Soundbooth app, which, naturally, copies the toolset verbatim from the company’s flagship Photoshop image editor. See also: Photosounder, which perhaps comes closest to this tool, and SPEAR, which is available free on Mac and Windows and has some fascinating resynthesis features. (Spectral sound design probably deserves its own post, later on!)

Spectral Layers nonetheless looks to potentially break new ground by focusing entirely on the idea. Whereas many audio editing tools that use spectral views have had modest editing facilities, here, it’s the entire program — and with some nice twists. On-the-fly selection previewing means that you’re constantly listening to your audio, not just looking at it. Advanced selection brushes make honing in on certain parts of your sound more precise, including by essential harmonic editing tools. (We hear harmonic relationships intuitively, so editing wave spectra at the literal frequency, rather than in the logarithmic proportions with which we hear, doesn’t work nearly as well.)

Spectral Layers also works with visualizing spectra in more compelling ways than just the typical, two-dimensional frequency vs. time view. Three-dimensional visualizations make seeing details in the sound easier.

Then you get into the actual editing. The developers are promising some powerful features, from extraction to independent pitch and time transformations, all moving this well beyond eye candy to the realm of deep sound editing. (The UI shows other features as well.)

There’s a new UI tutorial, but some of the features in brief:

  • Cross-platform Mac and Windows compatibility
  • Non-destructive layers for editing, plus compositing audio either by adding or subtracting a selection from a sound. (The latter sounds fascinating for sound design.)
  • A multi-pane UI, similar to tools from Apple and Adobe and familiar to people with a graphic software background.
  • 32-bit float spectrum.
  • Surround project support.
  • Pattern matching algorithms for still more-sophisticated selection and editing.
  • An “open project format” (presumably something XML-based or the like).
  • SDK for file formats, devices, tools, and filters.

In other words, the whole thing sounds mind-blowing and gives us everything we’d want … on paper. Presently described as “alpha stage 2,” the tool is still in development. But we’ll be watching.

DIVIDE FRAME, the developer, is a Paris-based house led by engineer Robin Lobel. Unrelated to the music side of this site, they also have a GPU-based video decoder, but no trial of the audio software – yet. Stay tuned.

Spectral Layers

Updated: while this is just a teaser, lead developer Robin responds with some more details for CDM:

There are 4 categories of tools: info (to get extensive info on the spectrum), extract (brush, frequency, harmonics, multichannel, noise (wip), time (wip), and others incoming), modify (so far only erase/amplify, but much more coming to transform the sound, like blur and other graphical modifications), draw (any tool to directly draw sound, as frequencies, harmonics, noise, etc).

Available Q4 2011, no price range yet (expect it to be the high, but there will probably be a light, affordable version too)

3D visualisation can display both amplitude or phase velocity using the GPU (OpenGL), it is seamlessly integrated with the 2D view (right clic+drag to make it 3D as you want, double right clic reset to 2D)

I do independent R&D in audio/video for several years now, have worked in some French [post production] companies as R&D developer and [graphics artist], wanted to start my own business (first with GPU Decoder as a small project, then came Spectral Layers). Spectral Layers came from the need to get clean voice tracks when shooting movies (as I do short movies too), then I thought of extending the concept to a general purpose, Photoshop-like tool. iZotope RX and Adobe Audition were not enough for my needs — I found the spectral editing pretty limited — so I decided to do my own.

Thanks, Fahad, for the tip!

  • http://debsinha.com deb

    +1! Can't wait until it is available!

  • http://noisepages.com/members/dmlandrum/ Darren Landrum

    This looks really good, but I'm expecting the final price to be higher than I can justify. Maybe I'll be pleasantly surprised.

  • http://www.ualberta.ca/~mjh7/Hannesson/Welcome.html Mark Hannesson

    Spear is Mac and Windows.

  • df

    wow. this looks pretty great. that multichannel tool clip… very fun tool.

  • Wil Sarmienti

    This reminds me of a framework that several students from Princeton developed a while back.  Check it out &nbsp ;http://taps.cs.princeton.edu/

  • http://www.jhhl.net/iPhone jhhl

    It also reminds me of me (as usual) RGS, Amiga, 1988. http://www.echonyc.com/~jhhl/software.html#RGS , and also Tondo: http://www.jhhl.net/iPhone/Tondo

    The only mystery is why spectral editing technology hasn't been included in every audio editor for the last 20 years.

  • http://www.eardrill.com Chris Muir

    It's also reminiscent of AudioSculpt from IRCAM, and Metasynth.

  • digid

    And not that far from Melodyne.

    Will be interesting to see the price, and to what degree they have managed to get rid of the main artifact of spectral processing: making most processed sounds sound too "sinewavey".

  • http://www.inoutfest.org Flplsx

    This looks ridiculously useful! The possibilities for creative sound design are mind blowing

  • http://www.jonathanleonard.com Jonathan Adams Leona

    Very cool tool here.  I have done spectral editing in the 90's using Spectralab which was very useful at the time.  This looks much more advanced.  Thx!

  • shim

    very cool!

    but…

    what ballpark are we talking about with the price point spec'd at "high"?

    if it's under $200, done. if over that then…well, shame.

    my secret stash of a certain 3 apps working in tandem yields pretty much what is seen here and total cost is under $99. just sayin'.

  • http://musicinablender.com Nick Francis

    Are these types of programs valuable for use in creating sample-based mashups; for instance, making clean accapellas or extracting a particular instrument from a group recording? Yes?

  • poopoo

    This looks absolutely fantastic. I need it now!

  • Martin

    Looks very much like Audiosculpt to me.

    > Nick : I'm very definitely no Audiosculpt expert, but yes, with AS and apparently with this program, you can "isolate" bits of mixed soundfiles and then ( obviously ) mash 'em. Thing is, if you don't mind your "isolated bits" being a bit phasey and weird sounding cos your making lofi bastard pop then its an amazing tool, however if you want anything else, then be prepared to spend a very considerable time fine tuning, and even then … It ain't easy … 

    Really looking forward to trying this, but not necessarily expecting any laws of physics to 

    get broken …

  • http://keithhandy.com Keith Handy

    I like, but I wish they'd demonstrate it with more natural/organic/acoustic sounds, so it would be easier to detect if anything about the resulting sound was glitchy (and all the more impressive if it wasn't).

  • interesting

    could be powerful 

    hopefully out soon

  • http://agargara.iiichan.net Agargara

    The cynical part of me wonders about the extent to which this will be any more revolutionary or useful than the plethora of previous spectral editors (AudioSculpt has already been mentioned, for example) but the PR certainly looks cool.

    The 3D stuff makes me imagine a William Gibson-esque future where sound design involves entering a virtual reality and editing life-size 3D representations of sound.

  • http://keithhandy.com Keith Handy

    Actually, that comment was after only watching the first video… should have watched all four before commenting!

  • Peter Kirn

    Yes, other tools have tried editing graphical views before, as I say — but whether we've had a truly mature tool in that category, one which can replace other audio editors, one that has a complement of editing and selection tools centering on the graphical view, and one that has non-destructive layers I think is an open question. We'll see if this delivers; it's just a teaser.

    But I'm with some of the other comments — why hasn't this become a standard tool, particularly when the amplitude vs. time view is inadequate for many editing tasks? (I guess one answer to that would be, because it makes transients so clear, and because a lot of edits are in time only and not frequency…)

  • http://www.jazarimusic.com Patrick Flanagan

    If Spectral Layers takes the conventional phase vocoder approach and breaks sound down into sinusoids, some processes will sound phasey. Using sines-plus-noise modeling, which has been around since the 90s, would improve improve sound quality significantly and represent a real improvement over existing tools like SPEAR.

  • vaikl

    Shouldn't we *make* some music?

  • http://www.twitter.com/Birds_Use_Stars BirdsUseStars

    You can do a lot of fun stuff with Camel Audios spectral editor (ahem http://youtu.be/qPBKdfZvJCY)

    but a dedicated tool like this is really interesting.

    @vaikl I don't see how an audio editor would stop someone from making music?

  • http://www.lucas-sound.com Oliver Lucas

    iZotope RX advanced, available now.

  • http://dropthedyle.net dropthedyle

    and there is also the Prosoniq Isolate application:
    http://www.prosoniq.com/editing-products/sonicwor

  • http://www.jeanfrancoischarles.com/ Jean-François

    I'm with Olivier here. It will be interesting to see how it compares with iZotope RX Advanced. Hard to be more expensive, but it might be hard to sound better, too.

  • Dubmachine

    Schwa spectro!!!! awesome tool.

    http://www.stillwellaudio.com/?page_id=32

  • http://noisepages.com/members/mfrasconi/ Miguel Frasconi

    +1 for Schwa Spectro! Excellent plugin.

  • Michael

    GPU(CUDA) accelerated spectrum editor:

    http://data.sonicawe.com/sonic_awe-product_summar

  • tahome

    this reminds me of Prosoniq sonicWORX isolate. http://www.sonicworx.com

  • Mirackis

    given the number of tools that can do very close to this, that have been around for a decade (or possibly a bit more), I do not see why this should cost more than $150 for the "enterprise" edition.