photosounder

Sound is a wonderful, if invisible thing. To work with these tiny fluctuations in air pressure that make up what we hear, we always work with some sort of software metaphor. So why not make that metaphor pixels – and why not manipulate the visual element directly?

Translating between sound and image is not a new concept in music software. The deepest tool for these functions is unquestionably the Mac-only classic MetaSynth, which sprang from the imagination of Bryce creator and graphic designer Eric Wenger. To me, one of the most appealing features of MetaSynth has always been its filter tool, the one component that allows you to work directly with sound using imagery and painting tools. The core of the tool, however, turns images into a score for synthesis, which opens up powerful features for microtones and the like but can conversely make simply designing sounds more challenging. (Side note: Leopard users, read this re: MetaSynth.)

Photosounder looks like MetaSynth, but it more directly translates between sound and image. It also has a uniquely straightforward interface for precisely adjusting controls and mappings. Put these together, and you can really use Photosounder as an audio tool. That opens up not only experimental techniques, but even makes conventional tasks more accessible.

Photosounder is also under very active development, with recent additions like a lossless mode for better sound fidelity and loop modes. The result is a really compelling looking tool for audio manipulation.

What can you do with these pixel powers over sound? Users have been experimenting and posting some pretty impressive stuff:

  • Isolating and removing individual instruments – making this an ideal remixing and sampling tool – using Photoshop
  • Making entire tracks from photographs (which, again, was possible with MetaSynth as infamously employed by Aphex Twin, but sounds very different here)
  • Processing using Photoshop filters
  • Making beats by drawing
  • Extreme time processing

Photosounder is currently Windows-only, but Linux and Mac versions are promised. (By the way, I think that’s going to become more commonplace as savvy developers take up cross-platform development tools, toolchains, and frameworks.)

It’s cheap enough to impulse-buy, too, at EUR25 non-commercial or EUR99 commercial.

http://photosounder.com/

Photosounder examples (with video)

I hope to get my hands on Photosounder and show off some features with this soon. Thanks to everyone who sent this in! (And yeah, after four or five people I finally get around to mentioning it!)

The best way to see what’s possible: check out the videos. Here’s a selection of my favorites:

  • thorspo

    Looks like a fancier version of Coagula:
    http://hem.passagen.se/rasmuse/Coagula.htm

  • http://villamil.org Gian Pablo Villamil

    Check out SPEAR too:

    http://www.klingbeil.com/spear/

    It takes a very sophisticated approach to decomposing audio inputs into their component waveforms, which then lets you isolate and distort individual instruments (among other things).

  • http://twitter.com/dmlandrum Darren Landrum

    Geez, Peter, it was something like two months ago when I had pointed this one out to you. :-P

    The core of this application is something called ARSS, which is open-source:

    http://arss.sourceforge.net/

    You'll note the author is the same. I was thinking about trying my hand at making a GUI front-end to ARSS, which is essentially a collection of command-line apps.

  • http://www.combatdave.com combatdave

    Holy crap, this blew my mind. The way the melody was separated from the drums in the first video was amazing, I've never seen/heard that done before.

    Of course, I might just have missed out on a whole bunch of stuff previously… but… is what he did suitable for extracting vocals from songs?

  • http://wowcool.com/engine Marc Arsenault

    And then there is RGB MusicLab – http://www.kenjikojima.com/rgbmusiclab/

    <blockquote cite="http://www.kenjikojima.com/rgbmusiclab/">RGB MusicLab converts RGB (Red, Green and Blue) value of an image to chromatic scale sounds. The program reads RGB value of pixels from the top left to the bottom right of an image. One pixel makes a harmony of three note of RGB value, and the length of note is determined by brightness of the pixel. RGB value 120 or 121 is the center C, and RGB value 122 or 123 is added a half steps of the scale that is C#. Pure black that is R=0, G=0, B=0 is no sounds.

    I had some fun making a song out of a photo of a pizza last year with it. Seems like he's added alot to it since.

  • http://andybest.net Andy Best

    @Darren Landrum it is true that Photosounder is based on ARSS, but the author tells me that the synthesis method used in photosounder is different from the one in ARSS

  • pg13

    im really frustrated with UI for not getting this Leopard update done…I think Ive been waiting over a year. ridiculous. snow leopard will be out before it happens.

  • http://nonplus.us nonplus

    Cameleon 5000 imports bitmaps as wavetables and I'm sure there's others.
    http://www.camelaudio.com/tutorials.php?tID=2

  • http://andybest.net Andy Best

    @combatdave it is useful for that actually- since he added the ability to use the original sound file in the resynthesis, the results can be fairly good.

  • http://photosounder.com Michel Rouzic

    Regarding the relationship to ARSS, as for the analysis Photosounder uses the same algorithm as ARSS 0.2.x, except for the addition of slightly varying pitch resolution (so that it retains the time precision of 24 bpo anywhere while going up to 60 bpo in precision around the 7th octave and up) and calibration (as it was the analysis algorithm would make higher frequency components dimmer and dimmer).

    As for synthesis, noise synthesis in ARSS 0.2.x was awfully slow, so much that it could take 20 minutes to synthesise a regular image. I rewrote the algorithm for Photosounder, and created that new mode that is called lossless in Photosounder that uses the original sound rather than noise.

    By the way I received my Power Mac just yesterday and already got started with learning how to make Cocoa apps, the Mac port is coming!

  • WhiteNoise

    If you're interested in spectral synthesis (and the relationship between images and sounds), there's also my synths, Additive and Doppelmangler:

    http://www.whitenoiseaudio.com/products.htm

  • Eoj

    What a coincidence, literally just been working on a load of special audio stuff today. Frequency is an awesome app, really simple but great… http://www.studiotoolz.net/frequency

  • Eoj

    @Marc Arsenault – that looks wicked, haven't seen that before. Going to have to give that a whirl :)

  • http://antisound.net stk

    Very interesting stuff.

    Can see some exciting potential here for a vst instrument that would use images as 'oscillators' (layers in photoshop parlance) which could be composited and crossfaded together (multiply/overlay/difference, etc) in realtime.

    I guess the only hurdle would be the number crunching required for the realtime fft calculations.

    Additionally, would be fantastic to be able to do this kind of thing using video as a source/destination.

  • http://www.keatshandwriting.com Keats' Handwrit

    Good call nonplus- Camel Alchemy also uses bitmaps
    http://www.camelaudio.com/Alchemy.php
    It's really good, great synth.

  • Dano

    Not even close to Metasynth. Pale imitation I think, and dreadful sound quality. C'mon Eric, get the Leopard thing working!!!!

  • http://photosounder.com Michel Rouzic

    Dano : lol, that sums it up nicely! ;-)

  • http://andrew.hicox.com plurgid

    Holy crap!

    I never knew … I mean, I'd heard of the thing Aphex Twin did, but that you know … sounded pretty much exactly what I thought that shit would sound like.

    First Melodine DNA, now this? … what??

    Now people are editing Dre beats with photoshop???

    I KNEW it … this really IS the future … now I just need my nuclear powered air-car!

    It's 8:05 AM on a friday … I'm considering taking the rest of the day off work because of this.

    Incredible.

  • HomeTown

    I was a big fan of Meta synth. They need to get a Mac version of this soon. I'm very interested.

  • http://myspace.com/fallsastar foosnark

    Not even close to Metasynth. Pale imitation I think, and dreadful sound quality.

    Not in lossless mode. Of course in lossless mode you can't change the frequency/time settings, but you can still paint spectrally and adjust the gamma, and more importantly, import/export between BMP and WAV.

    Even outside of lossless mode, I find this much more interesting than Metasynth. Sometimes blurring, timestretching, and/or rescaling/adjusting the frequency ranges is exactly what you want to do.

    It's a little unstable, and it'd be nice to auto-set the time/frequency knobs to match loaded WAV files… but it's a fun tool for the price. I expect I'll be making some cool IRs with motion blur and so on.

  • http://www.createdigitalmusic.com Peter Kirn

    I love MetaSynth. I think this is entirely different. This is more about focusing on the audio-editing aspect, whereas part of the beauty of MetaSynth is that the pixels become a kind of score. That's not to say that you can't use MetaSynth to synthesize sounds – you can -but the results here in lossless mode do look promising.

    Anyway, I'm happy to have something a little different. I'm finally getting some time to spend with this and will report back!

  • mik

    i just ran it in linux. wine is good.

  • Pingback: Mother and Country» Blog Archive » Turn Sound Into Pictures Into Sound With Photosounder

  • Imani Athena

    IF only Daphne Oram lived to see the day when Oramics would go digital! Actually, to all the other readers thanks for posting similar apps. Downloading now… Thanks Peter!

  • Pingback: Sound to Pixels and Back Again: Isolating Instruments with Photosounder « ????? ?????

  • http://dublab.wordpress.com dlab

    I'm very interested in more conventional applications. Sounds like this would be incredibly useful for cleaning up recordings where sounds bleed in like barking dogs, cars, the back of my chair hitting the desk while I am recording a guitar part…

  • AlexP

    Readers might be interested in this awesome Ircam tool called AudioSculpt too.

    You can find more info here.

    It works really good for the situations dlab posted above: you can remove selectively and visually most noise sources in recordings with it (along a good amount of other features)

  • http://photosounder.com Michel Rouzic

    dlab : that works too, it just would have been less impressive to demo ;-)

  • Pingback: I’m Back

  • Sam

    i use the trial of this software to see what it would really sound like but it distorts my audio everytime. why is that? is there a particular format the audio has to be in for it to sound regular? 

  • Pingback: SonicPhoto Focuses on Transforming Images into Sound, with Precision Harmonies, Stereo [Windows]

  • Pingback: SonicPhoto Focuses on Transforming Images into Sound, with Precision Harmonies, Stereo [Windows] | Uber Patrol - The Definitive Cool Guide