What can a new digital synth be in 2011? How will it work and sound? And given access to so many excellent tools, how can it stand apart? In place of a press release and some marketing-speak, developer Christopher Penrose (Leisuresonic, Cosmovox) sent us an extended essay explaining his thinking behind his just-released SynthTronica synth for the iPad. Aside from getting into the nitty-gritty technical details, it cuts to the crux of the issue: how to make something personal and new that nonetheless can work for other people, and how that idea can be tailored to a tablet.

As the geeks are glued to the latest iPad announcement, let’s look for a moment beyond platforms. Great ideas in synthesis endure long past platforms. The specific medium is wonderful in that it gives designers, engineers, and musicians the opportunity to realize those ideas, while presenting certain conveniences for developer and user alike.

All of this is worth reading in this case as it sounds like Christopher has a synth that isn’t like everything else out there – not at all. With audio files of your choosing transforming the timbres of synthesized sound in a graphical, spectrally-displayed filter, it looks like it could be a brilliant canvas for producing unusual sounds. That might help it find a place wired into your desktop PC or Mac for production.

In fact, it reminds me of the kind of creative synth we’ve seen all too rarely. The design feels heavily reminiscent of the ground-breaking U+I MetaSynth conceived by Eric Wenger (of Bryce fame). I was always disappointed other software didn’t run with some of those ideas; seeing it with some new twists, the take of a different artist, and touch input on the iPad looks terrific.

Christopher’s notes are quite long, but worth including in their entirety, especially knowing we have other developers in the crowd.

And, oh yeah, we could ramble on about this all day, but I think most of you will get the idea from this video below. It sounds great, and since you can input different audio files to get different filter content, you may be able to escape both overt recognizability and the disposability of many mobile and tablet instruments.

Here’s Christopher:

Notes on SynthTronica’s Development

Back in January of last year, I had been following the Apple tablet rumors with great interest. When the iPad was announced, I was surprised by both its name and its operating system. But it took me only a few hours to decide that I was going to design a synth for the new tablet.

Context

I have developed music software, with varying levels of attention, since 1988, and much of that effort has been spent developing idiosyncratic DSP algorithms for sound exploration. In particular, I focussed on spectral techniques for mating sounds — taking the characteristics of two (and sometimes more) sounds to create a new one. These efforts haven’t made it out of the Max/MSP, Pd, and Unix shell software ecosystems largely due to the limitations of audio plug-in hosts. “Side-chain” processing implementations are obscure and clumsy.

I am also a composer, and, until recently, my software was largely designed to aid my personal music-making. I can honestly say without pretense that my music is idiosyncratic; even Illegal Art, a label which has released one of my albums, regularly characterizes my music as being on the “challenging” end of the spectrum of their musical offerings. I think that SynthTronica has been a good project for me. While developing it, I have been challenged to corral and focus my motley DSP technologies into a broadly-accessible musical instrument. I took a music making process that combines synthesis and sampling, which I used often in my music making in the last decade, and put it at the core of a keyboard synthesizer. Hopefully, I have been able to distill an elegant instrument design from my personal composition practices.

Synth Architecture

SynthTronica is a hybrid instrument; its synth engine combines characteristics of virtual analog synthesis and digital sampling. For most “traditional” synthesizers, sound evolution is controlled by parametric filters — combinations of VCFs and LFOs. Instead, SynthTronica uses time-varying filter data to provide spectral evolution; an instance of such data is called a “formant”. Formants can be created in several ways, through the iPad’s microphone, importing audio files, and capturing performances of SynthTronica’s multitouch filter. While formants are currently played in strict loops, the maximum formant duration is fairly large — just over 60 seconds — providing potential for significant, albeit prerecorded, variation. The benefit of formants lies with their generality. A formant can be made from sources as disparate and varied as Nord percussion, cellos, choirs, braying donkeys, or the chorus of Katy Perry’s latest single. The latter example is an interesting consideration: a formant can reflect much of the rhythmic and sometimes vocal characteristics of its source sound, while effectively obliterating its pitch. Pitch is instead provided by SynthTronica’s synthesizer front-end. When readily-discernible formant sources are used, SynthTronica provides a unique musical space that lies between the boundaries of pure sound synthesis and referential sampling. Formant synthesis is not explicit like sampling; you play through the Katy Perry groove as if it was your avatar. With SynthTronica, a performer needs to make pitch choices for any sound to be heard.

Multitouch Filter

The iPad’s large touch surface was a serious attraction for this project. The idea for the Dynamic Multitouch Filter immediately came to mind. SynthTronica’s multitouch filter serves as a live and expressive counterbalance to the static character of formants by providing fluid gesture-controlled filtering of the synthesizer’s output. Given the spectral architecture of the synth engine, adding up to eleven touch-triggered filters (eleven per voice, technically, though they currently are used synchronously) does not overwhelm the processor resources of the iPad.

Hold Mode

Perhaps an ambient musician’s dream, hold mode simply allows notes to sustain by a single touch. They can be released singly by an additional touch, or released en masse via the “all notes off” button. Hold mode is an excellent counterpart for the multitouch filter, as the filter can easily be the focus of both hands when notes are sustained automatically.

Multiple Keyboards and Scale Patterns

While there are several ways keyboard control could be further developed in SynthTronica, I took advantage of several possibilities made available by the tablet design of the iPad. It is clear that a touchscreen does not offer tactile feedback, so I sought to implement interface dynamics that would make up for this lack in several ways. I choose to support multiple keyboard designs providing two piano style formats and a unique grid-based design.

From my iPhone app Cosmovox, I had a large database of musical scales available. I repurposed these in SynthTronica by providing selectable scale mappings for the keyboards. The piano keyboard has a particular design pattern which is accentuated by the contrast of white and black keys. Being a mallet percussionist in a former life, it was clear to me that this color contrast is optional and the key arrangement itself provides enough information to discriminate notes on a keyboard. Thus musical scales can be represented on a keyboard by changing this color contrast pattern. I often desire to escape my habits when creating music, and altering the keyboard scale pattern can be revealing for me. Further, the scale pattern facility allows for the use of a more radical keyboard design: the grid keyboard. The grid keyboard alters note relationships in interesting ways. The keyboard is compact, allowing one hand to access a two-octave range. Large intervals are no longer as physically distant from a given pitch. The keyboard can be bewildering to play if you play by note (which is a positive feature for me particularly as the keyboard is optional), and can reveal fascinating characteristics of scale architecture.

Design

I went with a modernist design aesthetic as I believe that SynthTronica does not have any appropriate analogs in gear. I find that creating interfaces for software that imitate gear introduces problematic usability issues. While I can understand the desire to have every useful performance control accessible on a single screen, there are practical limits to the number of interface elements that can coexist and still remain effective. Virtual knobs use less screen real estate, make a reference to audio gear, but are more difficult to use than sliders. I chose a slider-only interface using color and orientation for contrast. SynthTronica’s multi-screen design may reduce the accessibility of parameters during performance, but I think the architecture of the synth favors pre-performance sound design and emphasizes use of the Dynamic Multitouch Filter for expressive control in live performance.

Christopher also includes some frank thoughts on limitations of the synth for the time being, including some of his concerns about third-party audio interface support generally. This is beyond my area of expertise, so I’ll leave others to reach what conclusions they will – and I suspect we’ll hear some other developer views.

Limitations

While the preset architecture is robust from a database perspective, it can be frustrating for performance in its current state. The design of the reverb processor is one of the culprits. If reverberation time is different between two presets, changing from one to the other while the synth is sounding can produce awkward glitches. It is possible to ignore reverberation settings from presets by adjusting a SynthTronica parameter in the iPad’s settings application. I would like to improve preset change behavior in a future update.

Some goodies that the electronic music literati would desire — MIDI, audio interface support — have yet to be developed. MIDI is actually very high on the list now, as Apple has provided SDK support [Core MIDI] and I have purchased two Akai LPK25s and an Emu XMidi 1×1 for testing. OSC support is minimal at the moment: there are no supported in-bound messages yet, but a few outbound messages are implemented. Full class-compliant audio interface support will not be added until SynthTronica migrates to an iOS 4.x-only architecture, and even then there may be a performance reduction for many interfaces. A rant could be placed here which would be directed at audio interface manufacturers.

What I will say is that SynthTronica is less flexible with respect to audio buffer sizes as it is a spectral synthesizer; it uses power-of-two FFTs. But this is not unheard of in the least for audio processing; there probably isn’t an MP3 player that does not use them. To support audio buffer sizes that are not powers of two would cause a significant reduction in performance for SynthTronica (namely, a 50% reduction in usable polyphony due to CPU spikes). While there may be an audio interface that works out of the box with SynthTronica, I can’t name one at the moment. The class-compliant audio interfaces I have tested refuse to provide a power-of-two buffer when requested. While I am sure their engineers can come up with an excuse, they really should understand that power of two constraints are ubiquitous in computing, particularly for digital audio signal processing; it is bizarre that their hardware forces applications to perform additional buffering to support powers of two.

Anyway, let’s wrap up not with words, but with…

Sound Samples

PlanetMaster by Leisuresonic

HypnoticCaressing by Leisuresonic

PariahEmoting by Leisuresonic

PulseTherapy by Leisuresonic

More

Check out the sites for more. It was a bit unorthodox to include all these thoughts, but I enjoyed reading it and it made me want to spend some time with the synth. Let us know what you think.

http://synthtronica.com/
http://leisuresonic.com/

on the iTunes App Store

  • Christopher Penrose

    SynthTronica is available on the App Store now :)

    I think it is very fair to suggest that Metasynth had some influence on SynthTronica even though I have only used the environment for a few hours. Ironically, there are some that even argue that Metasynth was influenced by my own application I originally created in the summer of 1992, Hyperupic (http://highc.org/history.html).  While my application may have had some direct influence on Metasynth, Hyperupic was blatantly influenced by Iannis Xenakis' UPIC system (http://en.wikipedia.org/wiki/UPIC). Influence can be complicated and bi-directional.

    Thanks Peter!

  • Leslie

    Fantastic – downloading now :)

    BTW; Upcoming CoreMIDI support as well.

  • Christopher Penrose

    But I should make clear, in no way was Hyperupic in the same design league as Metasynth.  Suggesting that SynthTronica may be is a high compliment.  :)

  • http://yasaka.com/ Kenji Yasaka

    Great work, Chris! I'm downloading it now. Also, I'm still using your Hyperupic on my Intel Mac!

  • Blob

    Almost makes me want to buy an iPad just to use it. Will Synthronica ever show up in other platforms?

    Amazing stuff, Christopher. Congratulations!

  • Random Chance

    Sounds roughly like a vocoder combined with a phaser from the demos. But I guess the real "innovation" is in the way it is controlled. Would be interesting to hear from those who bought the application how easy or hard it is to create what they had in mind using the interface offered. One note on the graphical design aspect: It looks like 90s SciFi GUIs and apparently does not blend well with the overall design aesthetics of iOS or other Apple offerings. That's one of the challenges of developing software (or architecture or any other form of art that is also useful, or useful things that than also be art) right there: How to blend into the environment without giving up any elements that might set your product apart, give an identity, something instantly recognizable.

    And having used Apples libraries on the desktop for doing FFT, I wonder what he uses on the iPad.

  • Paul Schwochow

    Downloaded this yesterday, and I'm very happy with it so far. The formant tweaking is great, and the interface is intuitive. There are also some interesting (and humorous) subtle touches, like the random file name generator. My initial impulse was to explore the types of noises the app could make. Here's a recording made with the app about half an hour into playimg with the thing:
    http://bit.ly/g58xTp

    Hats off to you Mr. Penrose for creating this app. 

  • exatari

    Love the multi-touch x/y approach to activating and modifying formants and filters. Chris, have you played with the Yamaha FS1R, a formant synth module that never really caught on because of its insane menu complexity? Also, I would love to see some basic arpeggiator, perhaps controlled by the x/y pad at the top similar to Kaoss pad to loop basic sequences. Girlfriend just got a free ipad from work so I will definitely grab you app soon to give it a test spin.

  • Chad

    Cool!  Bought it.  Cost of a cup of coffee!

    Waiting for that Greg dude to come along and figure out a way to hate this.

  • loopstationzebra

    The graphics are truly awful.

  • Christopher Penrose

    @Paul  glad to hear a sound like that come out of the synth.

    @Blob  SynthTronica would have to become successful enough that it would easily fund a port to another platform.

    @random I use Takuya Ooura's radix-4 DFT algorithm with slight modifications.  I used the Apple veclib FFTs for a long time also under MacOS X, but they weren't available in iOS when I started the project.  They are now available under iOS 4 and are hopefully faster.

    @loop  I have stared at them for so long that I almost agree with you.  But many people who have no interest in flattering me say the opposite.

  • simul

    Peter (and Chris) – I loved reading the detailed notes about Synthtronica's design decisions and intentions.  Would really enjoy seeing similar pieces on CDM in the future.

  • Peter Kirn

    Definitely – I'm always happy to do more of this!

  • matt

    Please forgive my cynicism but I've seen an explosion of super cool controllers in the last few years.

    That's great and all… but I consider myself a musician… I don't hear any new sounds… none.

    In general, I'm a bit disappointed.

  • loopstationzebra

    Christopher, bought the app and have been messing. Very nice job. Would love to see some formant manipulating right in the spectral view – ala' Camel Audio's Alchemy (drawing right on top of the image!). More loop marker points would be great as well.

    I'm also hoping against hope that when MIDI capability arrives, you will do two things: 1. Provide clock sync support. For something like this app, where the tempo is EVERYTHING, it's vital and 2. Support for the CCK/USB connection. :)

  • Jonah

    From the video I really like the way the creative flow appears to work! Added to list of programs to get when I have an Ipad.

  • jhhl

    I too have used Christopher's apps back on the NeXT, and also wrote my own Amiga based response to UPIC called RGS back in the late 80s (get off my lawn…).

    It has contextual help , which you need because of the many tools and sections of the program . I, too, am perplexed with synth interfaces that use pictures of knobs which you cannot turn .

    SynthTronica gets interesting when you start importing sounds to use as your own spectral formant data.  You can use another synth to create the rhythms and format emphasis you want, or just human beatbox it yourself. This is where the real control over the sound is happening.

    But of course, new features pop into mind even though SynthTronica has plenty of features already.

    iPhone audio cut and paste is pretty easy to implement, and helps slosh audio throughout the iPhone audio ecosystem. The synthesizer providing the excitation signal is a pretty simple mix of pulse oscillators, which is needed to provide a broadband timbre, but  augmenting this with a similarly cheap wavetable synth would probably be a good idea. A derivative project piping live audio through the multitouch formant filter would be a good idea too.

  • Peter Kirn

    I'd love to see those other tools, @jhhl – the Amiga "app." (I bet someone said app before two years ago… have to check. Kids today.)

  • http://www.bloomingtonelectronic.com Mark Kunoff

    In the video it states "moderate pitches yield better results", but I must say I'm curious what SynthTronica sounds like in the lower note ranges. I'm always looking for unusual bass sounds. Also a question – Can your finger position be modulated during recording of formants?

  • Burg

    Love it so far! Thanks for all of your hard work?Is it safe to assume that Copy/Paste is on the way? Importing beat/clips for further manipulation and exporting to sequencers is what I'm looking for. Thanks again and hope this becomes a hit!

  • Christopher Penrose

    Copy and paste support ( at least one of the competing APIs, please vote :) ) will come right after iPad 2 Dual Core support (which I can start before the iPad 2 arrives) and right before Core Midi support.

    @Mark if I understand you correctly, the third example in the video linked from this article demonstrates exactly this. The added twist is that the process can be recursive – you can create a formant by use of the multitouch filter, load it into SynthTronica, and repeat the process using the formant you just created, adding as many layers of filtering that you want. The only issue is file management, as you can build up a lot of formants quickly.

  • http://www.bloomingtonelectronic.com Mark Kunoff

    Apologies Christopher and Bravo! – I must have been asleep during that portion of the video! This is absolutely a wonderful synth. I purchased SynthTronica last night and quickly realized this will be one of my new sonic weapons. And yes, my bass needs will be met! "Moderate pitches yield better results" is subjective in my opinion since based on the formant, one can get some very satisfying results in the lower note ranges. I applaud your efforts to create an instrument with a modern design aesthetic and an innovative control scheme! Lastly, will there be plans for a user forum? This is the kind of app which would so benefit from a community of users sharing methods and files.

  • lala

    i love it :)

    was an instant buy

  • lala

    is there a way to loop the playback of the recording? if not this is a feature request