Clean is overrated. If you’re ready for a little digital dirt in your synth life, powered by the open-source Arduino hardware, Marc Nostromo’s Squealer is for you. Built atop the wonderful, Arduino-based Pocket Piano kit by Critter and Guitari, it’s a full-blown, simple, digitally-gritty synthesis engine.

You get a monosynth, some fixed waveforms, a resonant filter, decay, and some aliasing tricks for extra grit. The big news: the Arduino Piano Squealer is now under a GPL license.

Official Arduino Piano Squealer Synth Page has everything you need
Announcement of GPL v3
All at Mustalk@noisepages.com

Here’s what it sounds like:

SAP+BOM+Dodgey Eighties Ringing Reverb:
apbom.mp3

Eery piano:
ap-eery.mp3

SAP+Flanging Mini KP:
ardboy1.mp3

Novation’s Launchpad, its affordable (<$200) "grid" controller, may have a big Ableton logo on it. But underneath, it's just a MIDI controller. Bi-colored LEDs, containing a red and green element for red, green, and amber output (amber = red+green), can be triggered using simple MIDI note and control messages. That means, whether you're looking forward to Max for Live or you're sequencing in a tracker or writing Processing sketches, you can use the Launchpad just like any other MIDI controller.

One of the things I thought was a major demerit for Akai was the fact that they failed to ship a MIDI implementation for the Akai APC40. MIDI implementations are the charts of MIDI messages we've had since the very first MIDI devices came out in the 80s. They're usually printed in the back pages of the manual, and even the cheapest gear has often had one.

Score: Novation 1, Akai 0. Novation has done the MIDI documentation, and then some. Its MIDI “Programmers Reference” is out even before the official Launchpad ship date. And rather than just doing a MIDI chart and assuming people know how to read it, they’ve taken the care to fully explain the way MIDI messages work, how to calculate the right messages, and how to really use this. Experts will have all the information they need, but newcomers will also find they can spend a little time and learn how to do what they want.

Launchpad Support with Downloads (see Programmer’s Reference at the bottom)
Via: Novation released Launchpad Programming Guide, and Protocol [Nezoomie's Zen Wave Blog - great read]

It’s listed as “for Max/MSP programmers,” but anyone using MIDI will want to have a look; that’s obviously relevant to far more than just Max. (In fact, there’s not a single mention of anything specific to Max in the document.)

What might people do with stuff like this? Well, as of just four hours ago, Matt DiFonzo lets us know he’s written a simple monome emulator. It’s even got a clever name:

nonome – monome emulator for Novation Launchpad

There’s some bad news mixed with the good. Even with something as simple as a grid of buttons, MIDI isn’t as friendly as it could be. I still would like to have a MIDI editor for the Launchpad so you can reassign buttons if you like — that’s a feature, incidentally, available on rival Ohm and Block hardware from Livid Instruments. Also, the documentation reveals that Launchpad uses “a low-speed version of USB,” which runs at a maximum of 400 messages per second, thus taking 200 milliseconds to update a Launchpad’s LEDs. (There are some workarounds, but they’re … more work. Clarification: Once you double up messages, though, you can get this to a more acceptable gap, and that’s for updating all the LEDs, not the latency of input messages.)

Also, here’s a hint to Novation: use a Creative Commons license for that document. That way, your users will be free to document even more ingenious solutions and friendly guides. You win, and your users win. For instance, I have the illustration here, which I should be able to do for purposes of reporting on this story. But can I write my own how-to guide using your guide? Why not make it explicit to encourage me to do so? (They list the PDF as “proprietary,” though there’s no explicit license, and I think they just mean “proprietary” as in “what we’ve done on our hardware.”)

That’s a difference between open hardware and closed hardware, but I don’t even want to belabor the point — CC licenses are something a commercial company like Novation could easily use. In fact, if anyone at Novation or Ableton would like to talk to me about why I think it’s a good idea, I’d like to extend an open invitation. I’m no legal expert, but I can explain what it means to me as a user and developer, and connect you with some of the right people at Creative Commons and the CC-using community.

But those gripes aside, kudos to Novation for getting this documentation out here. I think it’s really good news for people experimenting with grid controllers. And we’ll be looking at how all of these tools, hardware and software, fit together, and how open source development can make them more powerful.

Patchers and coders and hackers: if you’re interested in working on interoperability between all this stuff, let us know.

As computer music practice – part composition, part instrumental play – spreads, the idea of software interface as performance tool is becoming second nature. Putting those opposable thumbs and sensitive fingertips to work, multitouch controllers are growing in number, variety, and sophistication. Berlin-based artist Marco Kuhn shows off his beautiful creation, the PlayBox multitouch hardware, and its first app, PlayLive. That first software focuses on Ableton Live performance, but Live could be just the beginning – Marco has worked with Pd in the past and promises other apps to come. He’s interested in selling this device in the future, and he shares with us the tools he used to create this work for those of you doing development along similar lines.

The Project:

“Play Box” is is a User Interface for Natural User Interaction.
? allows multi-touch and object recognition (TUIO marker)
? hovering is also possible
? 22 ” TFT display , 1680 x 1050 pixel
? robust
? plug`n play

“Play Live” is a dedicated multitouch controller surface for Ableton Live.

- GUI Elements support multi-touch interaction
- easy to setup , just load the “Play” Control Surface
- you need no controller assignment
- you can control 32 tracks and 127 scenes, that are 4064 clips
- track controls are mute, solo, record, send1, send2, pan, level
- scene feedback name and state
- clip feedback color, name and state
- Transport control
- 2 Returns
- Master
- support bidirectional communication
- display track level meter, name

The whole app is coded in python.
- use Python Ableton Live API
- and libavg www.libavg.de for the GUI and Trackingengine
- PyPortMidi receive/send Midi Messages

“Play Live” Future plans:
- FX View per Track
- subpage for abstract clip controlling

I tested it with my mac book pro, “Play Live” and “Ableton Live” is running at the same machine. It should work on all platforms (Windows, Linux, Mac OS X). That Project started with my Diploma Thesis in Applied Computer Science 1 year ago.

Everything is coded and built by myself. The GUI – elements has been designed by Gösta Wellmer.

In this work, I created a GUI-Controller Library which allow me to Develop very rapidly Multitouch Interfaces for other Audio Environments.
More Apps coming soon ;-)

I [plan] to sell the” Play Box” and Apps like “Play Live” if anyone is interested.
The price is yet not specified.

That`s not my first multitouch Interface. 2 Years ago I created forfour – http://forfour.hi-pi.de/
- used PD, Processing, Reactivision and OGRE.
But the “Play Box” is another level;-)

For more on the project:
www.hi-pi.de/play

I hope to have video to share soon – and yeah, it’s time to plan another visit to Berlin.

The “space bassoon” Eigenharp seems to have landed from another planet. Today, I’ve got good news: it’s bringing alien gifts with it. By next year, both the software and the high-performance protocol the instrument uses will be open source. Taken together with other advancements in the open source community and with protocols like OSC, that could mean we’re at the vanguard of a golden age for more open, more intelligent, more expressive digital instruments.

Genuinely new music controllers made available commercially don’t come along very often. So this week’s news of a strange but wonderful-looking instrument shaped like a bassoon with customizable key controls turned many heads. With high-resolution, high-frequency data and reliance on the computer for everything from sound generation to mapping the keys to different tunings, the computer connection matters. Eigenharp’s chairman, John Lambert, sets the record straight for CDM on the software, the way it talks to your computer and other gear, and how open the tools and protocol will be.

I’ll be talking more with John next week, but I want to bring you this news now. Part of blogging means that you don’t hold back – you share that first reaction and then learn more. I’m pleased to say I was dead wrong on the Eigenharp. What looked on the spec sheet like MIDI-only communication and proprietary software turns out to be just the opposite. Sometimes, being wrong is great. Here are all the details:

What protocol do the instruments use?

The instruments talk to their base station by a dedicated, high speed, differential, transformer-coupled, error-correcting digital protocol designed (and tested) for use in noisy stage environments by ourselves. The wire used is a 4 core, standard star quad mic cable and can be up to 40M long. The base station then talks to the control software running on the host PC (Mac at the moment) via a dedicated protocol over USB2. In the host system, the native protocol used between agents (our software components) is a network protocol called Eigentalk. This can run over Ethernet wires (with slightly increased latencies due to some smart jitter buffering we do). We will be making this public and freely available (when we open-source our main software) in Febuary next year. There is an open API, to be published at the same time, for our equivalent of AU’s or VST’s, hosted in our software, EigenD. This is rather richer than the AU and VST API’s, for reasons connected with the expressiveness and configure-ability (is that a word?) of our system. A couple of interesting AU vendors are looking at producing native EigenD instruments at the moment, and I think there will be many more in time. We try to avoid MIDI wherever possible as it has so many serious performance limitations.

Had you considered OSC?

We did have a look at OSC but there are a bunch of things that our protocol does that OSC didn’t (not least of which, for example, is clock syncronisation for jitter removal, utterly vital over ethernet for live performance) so we decided to grow our own and release it. Absolutely no reason whatsoever that we can’t interoperate fuly with it though, I suspect that an OSC EigenD Agent will probably come quite soon after the open source release.

Ed.: Actually, with OSC (OpenSoundControl) moving increasingly toward broader input and assimilating other standards, I can easily imagine that a future implementation of OSC could begin to implement these features. More on both protocols in the coming months. -PK

Is it possible to use this as a controller for MIDI hardware? MIDI software?

Yes, EigenD has in and out MIDI gateway programs. The incoming [MIDI] is mainly used for pedalboards at the moment (to control scale, key, tempo, etc.), although there is no reason you couldn’t use a midi keyboard to play notes. We’re using the MIDI out to do a wide variety of things, including playing MIDI sound modules, and controlling video DJ software and lighting systems. There is a huge reduction in effective bandwidth when ‘downsampling’ to MIDI, but for a lot of things it still works very well. And of course, the AU and VST standards (which we fully support) are a kind of MIDI interface.

Is it possible to access its native protocol for custom software / Max-Pd patches and the like?

Well, we’re open sourcing the whole thing early next year, and I think there’ll be a lot of work done to glue all sorts of external stuff in then. Max/MSP has attracted a lot of interest in this area. The whole of EigenD is kind of built for this really. We’re an Open Source company: my main personal desktop is Linux and I have a strong personal commitment to open standards. I offer a number of ‘programming bounties’ for Linux improvements on my personal website at http://johnhenrylambert.com.

Tell us more about open-sourcing the software. What tools did you use to build the software.

It’s written in a mixture of C++ and Python. It’s quite scary code, but very clean and pretty, with little ‘cruft’. I think the open source community are going to have fun with it. Its a big system too, 8 years of R&D in one huge release. I don’t think that there’s been anything quite like this open sourced before in the music world – it is a bit as if Logic or Ableton were GPL’d tomorrow.

Thanks, John! Stay tuned for more details, and let me know if you have specific questions for the creators.

Part of the beauty of the iPhone from its launch date was the notion of a Mac you could fit in the palm of your hand. That makes it ironic that, for so many developers, mobile platforms in general have turned into a way to fragment software, to make it run fewer places instead of more. There’s something to be said for designing to a specific mobile device, but on the other hand, how many developers would want to restrict where their creations run? And particularly in music, isn’t the appeal of mobile creation the ability to have your tools work in more places? Maybe targeting just one gadget is the right choice for a given tool, but it shouldn’t be the only choice.

On the iPhone, the application Thump has plenty in common with a range of mobile music making tools. It’s simple but fun, a groove box with sequencing, subtractive synths, drums, and a set of basic effects, plus the ability to load your own samples and export songs.

Thump also demonstrates how simplifying sonic capabilities can produce musically-beautiful results, by focusing on the essentials and creating something with personality. Here’s a track by its creator showing off its sounds:
thump soundreel by mazbox

Well worth checking out the app on your platform of choice:
http://www.mrkbrz.com/thump/

What might not be immediately apparent is that under the hood, Thump makes use of the open source environment openFrameworks. As a result, the same code runs on iPhone, Mac, and Windows, as well as Linux. (It’s not distributed for Linux, but it could be. Hint, hint.) Creator Marek Bereza gives back, too – he ported the openFrameworks audio library to the iPhone, where it’s available to anyone.

Updated Marek notes in comments just what this means. The video below is, essentially, the same app. In place of the iPod screen, he has used a massive lattice of physical controls. A separate installation at the same show used a large touchscreen and simplified interface. And this really demonstrates what cross-platform means. Guy Kawasaki in the 80s mocked “ports” as a cheap wine. The idea is not to simply dump your code on a different platform and hope for the best – in fact, in this case, the changes from one platform to another were radical. The key is maximizing what’s essential, what really is not specific to a single device.

Physical Sequencer from Marek Bereza on Vimeo.

Creative Applications has a detailed write-up of the installation with more documentation.

If you’re interested in creating your own projects, oF has an elegant syntax based on Processing, but adapted to C/C++ coding paradigms and libraries.

I would recommend trying desktop development first as you’re learning, but when you do want to go mobile, setup is absurdly easy:

Setup instructions for iPhone

Having this as a foundation means more freedom for users and developers alike, the ability to run where you want, and a shared community of artists helping one another out. It does not mean sacrificing revenue, either – which is important, because revenue is often what enables (and encourages) developers to make new things. Here, we see a commercial mobile tool and distribution via the iTunes store, with free-as-in-beer distribution of desktop apps. But even the Free Software Foundation actually encourages charging for distribution – and to many people, the FSF is the most extreme example.

The bottom line is, you’re free to create what you want, where you want – and it doesn’t even have to have a price of free.

The use of openFrameworks is possible because the iPhone allows the execution of native code. Apple does not allow the Java virtual machine, which locks out many useful platforms and tools — including Processing. Naturally, that should not be the case with forthcoming non-Apple platforms based on Linux; more on that soon.

Thanks to Memo Akten for the tip — and for his exceptional documentation of iPhone openFrameworks development.

GPU-tiful and Cross-Platform

Updated Another framework I left out is the lovely libnui. The tagline says it all: “C++ API, multiplatform (Mac OS X, iPhone, Win32, Win64, Linux), 3D hardware accelerated GUI and more…”

On the iPhone, Intua’s elaborate Beatmaker software and David Wallin’s bleep!box have both been made with libnui. I want to single out bleep!box because of its UI – note how UI elements are simplified and scaled to the size of your fingertip. That’s something that, speaking of multiple platforms, could be equally important with touch hardware coming to Mac, Windows, and Linux platforms (some of it here already). I saw a huge fan of Windows tablets loving Propellerhead Reason on a tablet. He probably would be less enthused running Reason or Record on a touch interface; the UI elements are way too small.

bleep!box is also worth a look for the same reason as Thump: elegant UI, simple functions, and a lot of fun.

Pd, too

Pure Data, the open source patching cousin of Max/MSP, can also run on mobile devices. It’s currently powering the application RjDj, but because Pd is free, that could be just the beginning. In May, I wrote about the work being done by Hans-Christoph Steiner, who has been porting Pd even to ancient PDAs and iPods – including those, notably, without native floating-point capabilities (the kind of number crunching usually associated with sound).

Well, now Hans-Christoph is looking toward other, current-generation devices, too, kicking off with a workshop on making interactive sound on the iPhone using Pure Data. Here’s a taste:

OK, kids, you won’t believe how easy this is. Once you have Pure Data installed on your your iPhone or iPod-Touch, creating and uploading your own sound-processing and sound-generating patches is as trivial as copying a text file to your device! You’ll still need an Apple Developer License, but because Pd patches are treated as media files, they don’t have to go thru Apple’s elaborate code review — so, you can just dive right in, turning your phone into a pocket synth within minutes. In this 3-hour hands-on workshop, interactive sound whiz and Pure Data developer Hans-Christoph Steiner will lead you through all the steps.

Hans-Christoph will be talking Pd on the iPhone at a conference next month at Carnegie Mellon University in Pittsburgh. That will also include Memo Akten and Zach Gage talking about openFrameworks. And for nearly-entirely open source hardware, there’s a tantalizing-looking workshop combining Android with Arduino, too.

http://artandcode.ning.com/

There’s quite a lot more to say about this, but I hope this whets your appetite.

And, of course, riding the train to Pittsburgh or your nearest university or open source workshop, now you’ll have Thump which which to make some thumping tracks on your iPhone.

LaDiDa Demo from khush on Vimeo.

There’s no question iPhone/iPod touch development – really, just clever mobile development – has gotten a bit overhyped lately. But that’s all the more reason to do a round-up of genuinely interesting stories, real innovation happening on the platform. So, I’m clearing out my inbox with some of the more creative tools appearing recently on Apple’s mobile gadgets. There’s no better way to kick off today’s festivities than with this unusual “reverse karaoke” creation.

Sure, people may think they’re tone-deaf. But even the layperson has extraordinary powers of musical perception. So how could you train your iPhone to perceive and respond to music? That’s the question asked by LaDiDa for iPhone, the first of a new line of “intelligent” music applications for mobile devices. A “reverse karaoke” tool, the idea is to listen to singing and fake accompaniment, rather than having you sing along to canned backing tracks. Nothing is pre-programmed; everything is generated on the fly on the device.

It’ll even make up a Bollywood accompaniment to your singing:

LaDiDa Bollywood Duet from khush on Vimeo.

Of course, to me, it’s interesting not only what the iPhone is able to musically, but also what these algorithms are unable to make sound musical. Both reveal a whole lot about how we hear and conceptualize music. I think the team deserves real credit for making this fun, though, and on constrained hardware.

The app’s creator Khush follows in the footsteps of Smule in that it takes hard-core academic music research and uses mobile devices as a vessel for getting that tech in the hands (literally) of the general public. (See my interview with Smule founder and ChucK originator Dr. Ge Wang.)

Parag Chordia, developed at professor at the Georgia Institute of Technology and the gentleman you see in the video, spoke to CDM about what’s happening behind the scenes. He tells us about how this application was developed, and how the intelligent algorithms work (or at least try to work, as music analysis and auto-accompaniment remain at early stages).

First, an explanation of the app.

Khush CEO Prerna Gupta explains how it works:

1. You sing into the phone, and LaDiDa will compose music to match.
2. LaDiDa’s patent-pending technology analyzes the pitch and structure of the melody to compose a unique accompaniment for each recording.
3. To be clear, we do not query a database of pre-recorded songs. That is, LaDiDa has been designed to work with any music.
4. After recording your song, you can hear it with different styles. LaDiDa comes with three styles — E Piano Pop, Rhythm Synth Pop and Dub Tone — each of which has been developed using high-quality instrumentation to work specifically with our algorithm.
5. We will be launching new styles every month that will be made available through in-app purchases.
6. LaDiDa also works on rap! This month we’ll be adding three new rap styles.
7. After choosing your style, you can save the song and share it on Facebook, Twitter and email.
8. LaDiDa also has a Discover page, where you can hear songs recorded by other users from all over the world.
9. Khush was founded by music technology enthusiasts from the Georgia Tech Music Intelligence Lab. You can read about us here and also find out more about the research at our lab here.
10. LaDiDa went live in the iTunes store last week and is currently priced at $0.99.

Prena, the woman you see in the video, has some Web experience to boot, too, including founding a popular Indian dating site. Oh, and she’s a better singer than the music researcher, but, hey, that’s why we all went into computer music, right?

In case you’re wondering how you take a research idea and make it run on the iPhone – or how the algorithm works (and might get smarter in the future) – I turned to Parag for those details:

The initial code was developed in my lab in c++. Since the core algorithms are basically mathematical, that portion was relatively easy to port. However, we spent significant time thinking about how to optimize for the iphone and every aspect of the app, from the interface to sound design, has been built with the iphone in mind. For example, there are significant limits on sampler performance — samples have to be short and effects are more or less out — but we thought it was important for our styles to have a rich sound. So we
put great effort into designing light styles that sound realistic.

Another significant challenge was making the analysis robust to external noise; iphone recordings are lo-fi and corrupted with tons of background noises, which makes robust (and again computationally efficient) pitch detection essential.

Our approach to reverse karaoke is somewhat different than what’s been done before. A significant limitation of previous work was a lack of fine-grained key estimation, a problem that we felt was critical to successful vocal accompaniment (most people are not anywhere near a piano or an instrument with fixed tuning when singing into the app).

We also worked on trying to give some larger structure to the
accompaniment, which can often sound locally reasonable but notably lacking in direction. Again, a difficult problem particularly when people are singing snippets. Still it is sometimes possible to detect phrases, and we have tried to incorporate that information as well.

Auto-accompaniment is an endlessly fascinating and deep problem. As we learn more about human perception and cognition of music, as well as improve our tools for machine listening, our systems will become more musical. While we still have a ways to go, we believe that, with LaDiDa, we’ve created a product that is engaging and allows regular people to express themselves creatively.

If all of this talk about musical perception recalls the questions about how culture and background versus neurology can be used to explain music – as seen at the Notes & Neurons conference – that’s no coincidence. Researcher Parag played sarod with a fascinating ensemble at that same conference. Bobby McFerrin sings a really beautiful solo with the ensemble.

In fact, it’s absolutely worth contrasting the elegance and beauty of these all-human musical responses to the somewhat clumsy (sorry, Khush) iPhone responses. That’s not to say the iPhone creation is any less human – it’s a computation model programmed by humans, and is capable of some impressive feats made possible by their musical instincts and training. As such, we really can hear the gap between what advanced musicians can do intuitively and what we can model computationally, atop the restrictions of the device’s ability to sense the world around it.

World Science Festival 2009: Notes & Neurons, Part 5 of 5 from World Science Festival on Vimeo.

Musicians and artists now have the power to fuse visuals, sound, and interaction, to make a spectacle, an album, and a game all at once. But with the blank canvas of three different media before you, what form should that fusion take?

Space shooters with pounding electronic beats behind them have cleared some of the way. Now it’s ambient music’s turn. In the game Osmos, you become a mysterious particle, floating amongst gravity wells in various fields of material. By carefully navigating, applying just the right vector force to move through the shifting landscape, you merge with other particles and escape to safety.

http://www.hemispheregames.com/osmos/

The move from “shoot stuff” to “move” or “eat” seems to be rising in popularity, with games like fl0w and Spore’s initial “cell stage” encouraging nonviolent navigation. To me, there’s something happening to the zeitgeist, perhaps a renewed awareness of cosmic (micro- or macroscopic) being, and of movement that draws on free-floating physics.

Even amongst a wave of games in this mode,when you actually play Osmos, you realize that it is something different and special. The design makes ingenious use of different kinds of movement and pacing through its different modes, at one point calling upon you to hurtle around a black hole, then move at nearly imperceptible speeds through a seemingly impossible-to-traverse petri dish of massive particles. No less than a shooter, it connects to the id and survival instinct. Pac-Man, the most successful arcarde game of all time, and one of the few that sucked in men and women in equal measure, was noted for its emphasis on eating as the mechanic. Consuming stuff appeals to everyone.

Of course, this is on a music site, and with good reason: what makes Osmos work is that Osmos is musical. It’s immediately beautiful and delicate, a perfect aesthetic union of the texture of the music and the on-screen arrangements of particles. More importantly, the music is woven directly into game play, providing subtle cues for dangers, and underscoring the pace of gameplay. You can only solve a level by managing speed and motion, and the music helps provide both the literal indications of speed and help your head get into the right zone to lose yourself in the world. If blips in early arcade games helped create a zone of play trance, now we have spectacular ambient soundtrack of music by Loscil, Gas/High Skies [Microscopics], Julien Neto, and Biosphere.

The music isn’t simply a beautiful soundtrack to the game. The game really feels like an extension of the world of the music. Put it all together, and something magical happens in this $10 game: you hear the music in a new way.

I spoke to the lead designer behind the game, programmer/animator Eddy Boxerman, along with musical-sonic collaborator Mat Jarvis aka Gas aka High Skies.

Peter: I found it amusing that some of the game press have stumbled around looking for a name for a new genre here, the "eat other stuff" category. To me, the basic game controls could be traced back to early titles like Asteroids. What are the games that have inspired you, either specifically or generally?

Eddy: Some people are dubbing the gameplay type as "eat ‘em up", as opposed to the classic "shoot ‘em up". But it’s true, the physics/controls have more in common with classic games such as Asteroids and Gravitar. The only "modern" game that otherwise influenced me was Katamari Damacy. It’s funny: when I sent out the first prototype to friends, I was concerned they would think the game was too similar to Katamari. That’s when I heard about flOw, which I’ve tried to steer stylistically clear of ever since. I guess it’s an archetype though, and the comparisons have been inevitable. As for Spore, Osmos was about one year into development when I saw the first preview footage of its "cell stage". On top of that, I learned that Will Wright had hired Brian Eno to do the music! I definitely had a "why should I bother?" moment at that point. But here we are, a couple years later, and I think we’ve managed to put out a unique game with a great mix of gameplay, sound, and visuals.

Peter: Ha – well, I’m glad you “bothered”! One thing that I think is really unique about Osmos is the gravitational mechanic. How did this come about?

Eddy: I was (and probably still am) a sci-fi kid. And when I read about real phenomena like colliding galaxies, retrograde planets, and the like, my mind goes to stange and wonderful places. That said, the Spacecraft Dynamics course I took during my engineering degree definitely played a large part in putting those mechanics and levels together. Thanks, Professor Misra! :)

Peter: How are the physical mechanics of the game constructed? This is essentially two-dimensional vector math, yes? Were there refinements you needed to make in order to keep performance up, or to keep the levels playable?

Eddy: That’s a big question! But yes, essentially it’s 2D vector math, though we make use of some 3D math (including quaternions) on the particle systems. The majority of the physics is quite simple though — first year bachelor’s stuff: F = ma, conservation of mass and momentum, gravitational laws, basic time integration, etc. We follow the spirit of the laws, but not always the letter. For instance, if you look at the Earth’s solar system in its entirety, you wouldn’t even see most of the planets; that doesn’t make for a very playable game though. As such, we took many liberties when it comes to scale, constants and exponents in the equations. The majority of the work was in the tweaking of those values to make the game /feel /good.

Peter: How did you prototype the game? What was the process of evolution like? Were there any failed attempts along the way?

Eddy: I put together the first prototype starting from a NeHe OpenGL tutorial. I added some basic physics, mouse controls, some highly unoptimised collision detection code, and voila, I had something to play with. After that, I immediately integrated [cross-platform 3D audio API] OpenAL, as the game was just begging for sound and music. I then hunted for some good sounds (discovering freesound.org in the process), put together a simple looping song (my first rough and humbling attempt to create digital music), and sent it out to a few friends. I’ll make that version available for free someday… when I’m feeling brave.

As for failed attempts, there were tons of shelved experiments and features that were cut. One interesting problem was the difficulty curve of the gravitational levels. In fact, the first gravity levels looked a great deal like the "Warped Chaos" levels do today: several free-floating Attractors in a sea of regular motes. I found it incredibly difficult, however, to make a "beginner" version of those levels. If the gravity was too weak, the Attractors had little apparent effect; too strong, and the level became incredibly difficult. I stuggled with this for a long time, until I hit on the idea of putting one, immobile Attractor in the center, with everything else orbiting around it. This allowed for the creation of systems with strong gravity effects, but long-term stability. It seems so obvious now, especially given the example of our solar system, but it wasn’t back then. In the end, these are some of the coolest levels in Osmos — practically its "signature" level type. Moral of the story: necessity really is the mother of invention.

Peter: What is the visual engine like? The sound engine?

Eddy: As I mentioned, I bootstrapped from a NeHe tutorial, but that’s all been replaced at this point. It’s now a homegrown engine using OpenGL. It doesn’t make any use of pixel/vertex shaders, so it runs on a wide variety of hardware. The majority of our texture work is done in Photoshop.

[The sound engine is] a wrapper we’ve built around OpenAL. At a basic level, all we ever do is mix and adjust the gain and pitch of our sounds and music. This allows us to run on generic sound hardware. But we do make extensive use of those "effects", and it’s possible to get a great deal of variety and feedback with them. In the end, it’s all about the quality of our sound sources, and the algorithms that adjust and smooth the gain and pitch of each.

Peter: For pitch shifting, were you able to do that natively in OpenAL?

Eddy: Yes, OpenAL supports pitch shifting at the software level. I ported our sound engine to use DirectSound at one point, and found that it’s pitch shift range is rather limited. We reverted back to OpenAL.

Peter: One thing that really strikes me about the game that is its use of pacing. Even without the obvious technique of allowing the user to control speed, there’s this extraordinary contrast between levels that require you to move quickly, that require you to anticipate the rate of movement along a vector of other objects, or that require you to move almost imperceptibly slowly. Was this an early goal, or something that evolved out of developing the game concepts?

Eddy: It evolved from the game concepts themselves. We tried to fully explore the fundamental controls and game space of Osmos, and converged on a number of interesting regions and structures that were fun to play with/in. Some people actually "complain" that the pacing of the game isn’t consistent across levels — some are fast, some slow — but that wasn’t our goal. In a sense, we’re merely presenting the results of a great deal of research and exploration into the space, with our sole requirement being that the levels are fun and interesting. We wanted it to be a game of concepts and understanding, not speed. The addition of time-warping allowed us to break free from that, expand our scope, and include a much larger variety of levels, as some have both fast and slow moments in them; we wanted the player to be able to control the pace and difficulty to their level of comfort.

Peter: Connecting the action to music, how important is the use of music in conveying time?

Eddy: I think pitch-shifting the music and sound effects helps players track time-warping in an inutitive way. Without these aural cues, the mental connection would be incomplete. That said, I sometimes test the game without sound, and it’s totally playable — just much less immersive.

Peter: At what stage did you involve the musical element? How was that relationship established?

Mat: Eddy first contacted me two or three years ago about using one of my tracks on Osmos (“Gas – Discovery”). He sent me a basic alpha version where the game was literally flat circles on a plain background, but even back then with the simple graphics and no sound it was compelling. I think it was at this early stage that I sent him a copy of another High Skies track, “The Shape of Things to Come,” which he also used a section from. It wasn’t too long later that he sent me a version of Osmos with most of the music that made it to the release version and it all just complimented each other beautifully.

Eddy has done a great job choosing music that compliments the game and graphics, and they also compliment the music. I think he’s done a great job overall as well, keeping his vision consistent for the look and feel of the Osmos world. Even though some of the levels are quite varied; some are quite chilled, others fast paced, nothing seems out of place. He has created a new world that’s both microscopic and macroscopic.

Peter: The musical and sound effect elements really do blend effortlessly. Mat, I know you made substitute sounds – how did this help blend the aesthetic?

Mat: I made the rebound noise when you bounce off the walls/ boundary. The original sound was quite hard and metallic with a lot of reverb, and I suggested a softer echoey sound. There was nothing wrong with the original, but it made the boundary feel quite hard and cold like being in a metal tank. I sent Eddy some analogue samples, using a [vintage Roland keyboard] Jupiter 6 with varying delays/ echoes. Because the tones are so pure and tuned, Eddy pitched them slightly to fit with the various keys of the different music.

Eddy: In general, it was a matter of searching and experimentation for the "right" set of sounds that fit well together — which can be a lengthy but rewarding process. Once integrated, minor gain and pitch adjustments helped fit it all together.

Mat provided the bounce sound, as well as the continuous absorb/being-absorbed sounds. The other sounds were created/shared on freesound by the users Elektrocell, fran_ky and Jovica. (Check out the Osmos credits in the readme.html for details.)

Peter: I recently spoke to Crystal Method, and while they’ve done some notable game scores, they actually said they wouldn’t want their music to be placed in a game context at which the speed changed, that it’d lose something. This seems like the opposite of that. Now, maybe it’s easy to take for granted with ambient music as opposed to something beat-driven like Crystal Method, but do you think there’s something special that allows this music to be adaptive in that way?

Mat: I think that in the right circumstances music with drums could work well and may even highlight the time-shifting more. Of course with the wrong game and wrong music it could sound terrible too.

Eddy: It’s a delicate issue. Out of the context of the game, I think it would seem wrong. There were also some pieces of music I tried which didn’t lend themselves well to this kind of manipulation, and which didn’t get included as a result. But with these songs, and everything working together in-game, I feel the whole experience is stronger as a result. I’m just glad Mat and the other artists agreed — or at least… didn’t object. ;-)

Peter: Hey, I think if you can write music that sounds good played at a different speed, that’s nice.

Osmos seems to me a musical experience, in that the sense of the music is transformed by the visuals and the interaction with them. Do you think it’s possible that audiovisual interfaces could go even further in terms of their impact on the music?

Mat: Yes, it almost seems as though Osmos creates a world for the music to exist.

I’ve always liked music visualisers like the Processing and Cymatics stuff, they’re quite compelling to watch how they react to the music, so it would be interesting to go the other way; by manipulating/ sculpting abstract shapes which then modify or even create sounds and music, especially using the new controllers like the Wii, [Microsoft’s] Project Natal and Sony’s Motion Controller instead of the mouse.

Eddy: I totally agree with Mat. Yes, please! More feedback between music, visuals and interactivity — in all directions. I’d love to experiment more on all of these fronts. In general, I think games such as Audiosurf and Auditorium, while great, have only begun to scratch the surface of what’s possible. Interactive musical "toys" such as [Nintendo/Toshio Iwai’s ]ElectroPlankton or Dub Selector are another great direction; and Rock Band Network is about to expand the market in a huge way. It’s exciting!

Peter: Along those lines, this is obviously a new take on music distribution in a way that goes well beyond what titles like Rock Band have done. Are there other cases in game history that to you have done that? Is there potential in these new outlets, outlets that are more accessible to independent developers (Steam, Direct2Drive, Xbox Live Arcade, iTunes App Store), to really change game development?

Eddy: Wow, that’s another big question. Really, I just felt the music was such an important part of the experience in Osmos that we should show the artist and track names during the game. But now that you mention it, I can’t think of another game that has done this, besides games like Rock Band where the song /is/ the game/level. That said, one way I’d love to push this evolution is to "augment" music videos with interactivity: ie. into music video-games. (The term "game" could be used rather loosely here.) Everyday Shooter does this, though Jon Mak actually created the music /for/ the game in that case — which rocks — but collaborations could lead to some really amazing things in this direction.

Peter: Eddy, Mat, thank you. Whether it’s in the form of a game or an audiovisual performance, we’ll certainly continue to explore these areas. And – hint, hint – Osmos 2, with multiplayer? Just a thought.

Osmos Trailer from hemisphere games on Vimeo.

Take a weekend, and make something: that’s the challenge behind the Music Hack Day, which joins a growing phenomenon of events built around collective creation. (CDM held its own tangible interface hackday online, which I definitely hope to follow up soon!) Initiated by Dave Haynes of music sharing service Soundcloud, the Hack Day has already hit London. Many of the events were Web app-based and focused on consumption rather than creation of music, but we also saw a chordal synth plug-in and beer bottle percussion instrument.

The Berlin Hack Day, which wound up earlier today, offers still more projects focused on the creation side of music hacking. Having Ableton and Native Instruments as sponsors likely helped the mood. And as you’d expect from one of the world capitals of creative hacking, Berliners don’t disappoint.

Among the projects: a beautiful, elegant 3D sequencer, a fun bird-and-sky multitouch soundmaker with multitouch trackpad input, and a robotic xylophone controlled by monome. Someone even worked out a way to turn NI’s Maschine into a rhythm game, complete with Street Fighter sounds.

I’ve got some of my favorite projects here, but see also an eyewitness report (in English and Italian) at Audio News Room:
Just back from Music Hack Day Berlin
… and keep your eye on the wiki:
Berlin Hack Submissions

xylobot run by monome from robb on Vimeo.

Monomist Rob Böhnke and Ramsey Arnaoot created one of my favorite hackday projects so far: a monome-controlled robotic xylophone. The ingredients: one monome grid controller, one Java application for step sequencing to the output, one Arduino open source controller board, and one terrific xylophone “robot” made of an array of servos that strike the bars of the instrument. Oh, and some hot glue and wood, of course.

Project details

Debugging

Audio loop

Of course, what’s especially impressive is nailing this in just a weekend – imagine what they could do with more time and iterations.

Proof that musical visualizers don’t always have to be trippy, futuristic, Tron-like 3D landscapes (and that’s me speaking as a fan of such things), Gernot Poetsch instead chose a whimsical environment with clouds and cartoon birds, inspired by the graphic identity of Twitter. (No actual Twitter is involved, meaning you lose the, ahem, unreliable, buggy, unfiltered chat network but keep the cutesy happy sky! Works for me!)

The visuals are built in Quartz Composer, which via OSC transmits messages to synthesis language ChucK for noisemaking. The surprise is, the multitouch input is not a Lemur or an iPhone – it’s the new MacBook touchpad under Snow Leopard!

iLoveAcid is a beautiful-looking nodal sequencer by Jakob Penca which uses wireframe “tracks” to guide music playback through a sequencer, transmitting events to your soundmaker of choice via MIDI or OSC. By using curved timelines and connections, it’s a veritable model railroad of music, in which formations combine to form more complex structures instead of simply stepping across a grid. Despite appearances, it is so far only two-dimensional – but then, the z displacement could easily be assigned to some form of modulation. I’m really eager to see the video of this.

One ingredient: Processing, which makes it easier to write visual code and to connect to Java libraries.

Project Page

The hardware controller for Native Instruments’ Maschine software drum machine has been adapted to other software, like Ableton Live. But this is surely the first time it’s been made into a game. Using Maschine’s MIDI output as a game control and sounds from Street Fighter, MaschineFighter is a simple, Simon-style rhythm game. Unlike Simon, though, there’s a twist – instead of rote patterns generated in advance, you face off against a friend and try to out-rhythm each other, battle style. I think it’s actually a pretty brilliant idea, and could become a new sensation for us electronic music nerds – not to mention, it’ll finally test our rhythm in a way electronic performance often does not. (Correction: It is Mac-only, making use of the PYMIDI Objective-C based library, which, since everything else that starts with “Py” usually means Python – a la jThings that mean Java – I assumed, incorrectly, was built on Python. But anyway, if you like the idea, carry on! And, actually, having a pure Objective-C CoreMIDI interface is also pretty awesome.)

Hoping for a video of this, too.

If you have a project that didn’t make this list, or if you add documentation after the fact, let us know.

Code, hardware, and software: Hack Days are all about getting actual stuff made. Berliners, the Music Hack Day that took place in London at The Guardian now gets underway in Berlin this weekend. For anyone who thought the first event was overly Web-centric, there are some new players in Berlin. Ableton is a sponsor, and Ableton, Native Instruments, and RjDj are all presenting hack sessions (in addition to the more Web-focused / consumer-focused 7digital, Songkick, Cloudspeakers, Mufin, SoundCloud, and Echonest). The awesome German musician magazine DE:BUG is also in on the action. I also see our friends at Future Audio Workshop (developers of Circle) in the lineup.

Check out the details:
http://berlin.musichackday.org/
Weekend schedule

So, German readers, who’s going? I’d love to have some spies tell us what the discussions are with NI and Ableton.

I’m, as always, interested in how we can get past geography and share work internationally. So if you’re doing a project, be sure to take lots of pictures, screen caps, code pastes, and the like, and we’ll feature your work here on CDM.

Future events are planned for other cities, and I hope CDM will be involved in some of them. Boston will be the first US event, but it’s on a date I can’t make it. Anyone have a space here in New York you’d like to suggest?

Showcasing amazing projects is a good thing. But we know that no creation emerges fully-formed. They start from nothing, with lots of mistakes along the way. You get help and ideas from other people. And you need time.

So this month’s Handmade Music in Brooklyn we’re declaring an Open Lab. Got a kit lying on your shelf, waiting to get made? Got a half-finished project that needs fixing? Just want to hang around some musical and visual DIYers and see what they’re up to? And just need a few hours to make some progress? That’s the idea.

Software projects, hardware projects, gear hacking, circuit bending, coding, patching in Reaktor or Pd or Max – it’s all welcome.

We also have a very special guest this month in the form of Morgan Packard, a talented multi-instrumentalist and computer musician (video at top, with live visuals by superDraw creator Joshue Ott). At 7:30p, Morgan will show off his free Ripple musical environment, built on the powerful open source SuperCollider code-for-sound platform. It’s a great chance to see what SuperCollider can do, how a scratch-built environment can open up musical possibilities, and what you can do with Ripple yourself.

Full details: Handmade Music Brooklyn: Open Lab, Featuring Morgan Packard’s Musical Code [handmademusic@noisepages]

Facebook Event Page

The whole event runs 6:00p-11:00p at 3rd Ward Brooklyn. As always, it’s completely free. Be sure to bring your projects and the tools you’ll need; we can provide power, a PA, space, and other folks to hang out with. And we can help give you an excuse to set aside a few hours of time.

We’ll also be taking notes on how the setup works, as we know this may be something other Handmade Music events want to try in your neck of the woods.