I tend to tune out when it comes to sample libraries, but here’s one that takes the scripting capabilities of Native Instruments’ Kontakt sampler to extremes. The Grid Machine line developed by Lindon Parker (Channel Robot) and distributed by LoopMasters brings to Kontakt the sort of grid-based, sliced-up sample manipulation we’ve seen in the monome community and in custom tools in environments like Ableton Live and Renoise. Using KSP, the scripting environment in Kontakt, these produce entirely-custom instruments that cut, chop, stutter, reverse, mix, trigger, sub-loop, re-trigger, and modulate. You can change speed, mute, skip, reorder, and play patterns, and even mix between loops.

Even before you get to Kontakt’s effects, this kind of work really challenges the notions of what people imagine a “sampler” or “loop library” to be. And that’s been true of the sample sound design community, generally – they can brew things beyond the expected boundaries of a sample. I could even see this becoming a performance instrument.

Now, for those of us not content to use existing loops, I hope we can somehow convince Lindon to explain how he did the KSP scripting work to make it all happen. Drum ‘n Bass and House libraries are £29.95 each.

Loopmasters: Channel Robot

Some House for those of you who weren’t into the DnB version:

In a responsive, real-time sculpture, the simple sonic qualities of a rainstick become electronically enhanced. Rainlith, a “kinetic sound art” work by Rui Gato, makes the rainstick itself robotic, its sounds transformed in space in a way that is itself sculptural. Responding to movement in the space using Microsoft’s Kinect, the apparatus is a geektastic brew of just about every tool you could imagine involved in this sort of construction.

The artist shares full details, reproduced here in both English and Portugese – and Rui, thanks for sending this in:

Rainlith 2 – Kinectic sound art piece.

On Rainlith, the primitive naturally granular sound of a big rainstick gets explored in real-time by cyber-age sound manipulation tools.
It’s an interactive piece in witch the movement of the audience’s body activates an electric motor, making a reflex movement on the structure that embraces the instrument.
The sound of the rainstick is captured and processed in realtime, and sent 24 meters above, filling the empty space of a old industrial cereal container. The reverberated acoustic mix is then received back by the audience in the spot right below the opening of the container.

Na peça Rainlith o som primitivo, naturalmente granular, de um pau de chuva é explorado por ferramentas modernas de sound design, em tempo real.
É uma peça interactiva em que o movimento do corpo do público activa um motor eléctrico, provocando um movimento reflexo na estrutura que sustenta o instrumento.
O som captado em tempo real é processado e enviado 24 metros acima, enchendo o espaço de um silo de cereais industrial. A mistura acústica reverberada é absorvida pelo visitante no local imediatamente abaixo da abertura do silo.

hardware:

microsoft kinect
arduino duemilanove
H-bridge (hand made)
24v 6A DC motor
CPU
zoom H4N
FM emitter / receiver
ion ipa3 portable speaker

software:

MAX/MSP
Max for Live
OSCeleton
OpenNI
Nite
toxiclibs

agradecimentos especiais:

Nicola Henriques
Susana Luiz
Luís Pereira
Paulo Carocinho
André Sier
Daniel Coimbra
Ruben Santos

By the way, if you’re intrigued by the potential of Kinect, be sure to read sister site Create Digital Motion for more. Just today, we have news of a new version of EyesWeb that could be relevant to musical use:
EyesWeb, Powerful Computer Vision Software for Windows, Adds Kinect Support, Fixes, More

With Web data providers offering generous cash prizes and a strong emphasis on harnessing data to transform listening, music consumption took center stage at Music Hack Day’s debut in New York. But it was novel music controllers, the sort that once were commonplace only at academic music conferences, that stole the show. That suggests that whereas building the next MySpace was once the hot music tech, the future might look more like a race to build the next Theremin.

Whatever the cause, the event proved just how productive hotshot DIY coders can be when left to their own devices and given ample sources of electricity and caffeine. The weekend marathon has now been exported to nearly a dozen installments in Europe and the US, though this was its first appearance in the boroughs of New York City. The result: nearly 200 participants, hundreds (yes, hundreds) more on a waiting list, and over 70 projects completed in a weekend. From just Saturday afternoon to Sunday afternoon, programmers working with Web and desktop technologies whip up quick software creations. The emphasis is on “hacking” for a reason: there’s no time to second-guess or obsess over quality, or indeed to waste a moment conceptualizing. This is all about making a working product, trying out an idea in practice, mashing together whatever is most accessible as rapidly as humanly possible. Sure, there aren’t any hard, fast rules against bringing in previously-prepared tools. But make no mistake: very much that was live in a demo Sunday was pure theory just twenty-four short hours earlier.

Coders laid out cushions on the floor and packed toothbrushes. Some were local, but others were still bleary eyed-with jetlag from trips across the Atlantic. Hopped up on coffee and Red Bull (and then beer), they coded projects that often had nothing to do with their employment – even those who came on the dime of some of the Web companies. Nor was there a lot of fishing for venture capital or IPOs. Most gave away code (if they could bear to let anyone else see it) on public code repositories like GitHub, and listening to coders, many even blatantly ignored the promise of cash prizes. It was programming for love.

Here are few of the most promising projects, and a few noticeable trends. If generating automatic playlists or finding music videos that match tastes of friends on Twitter isn’t your cup of tea, don’t despair. We had alternative instruments and music-makers, too – and, take note, they generally took home the cash.

Invisible Instruments, Made with Gestures

Invisible Instruments, the winning hack by Tim Soo, began at the Boston event. I think what made it so compelling – the voting was done by the entire audience, entered via SMS – may have been the recognizable instrumental metaphors. Using Max/MSP and OSCulator, a Wiimote, and iPod touch, the instruments emulate a violin, drum pads, and

Now, none of this is news to regular readers of this site, of course. But that should present another lesson: if you’re doing this kind of cool stuff, you should tell the sorts of people who don’t normally pay attention to such things (even, very often, tech-savvy folks). Music tech involves all sorts of wildly cool things that we’ve inadvertently kept a secret. Let’s change that.

(Or, to put it another way, apparently the whole world isn’t reading this site. If you want to help us with that, let me know.)

Previous videos / project work:
Invisible Instruments

Note that Tim does say, “Scout’s honor,” that he built new invisible instruments just this weekend. And you can grab these and older patches from his site.

Plucking Strings and DJing with Kinect

Just got a Kinect? Want to make it do something? What better than a couple of coder friends to make it happen? The three-person team that worked on Stringer, a musical instrument for plucking strings controlled with Processing, wound up easily paying for their Kinect hardware by pocketing some change in prizes.

Participants Aidan Feldman, Tyler Williams, and Alex Chen contributed. In the process, they found that using a camera to simulate string plucking wasn’t entirely effective; they didn’t have enough tracking intelligence to tell the difference between a pluck and a motion near a string, so wound up going for simpler reactivity. The clever string animation works wonders to make you feel like you’re playing real strings, even with samples, however, and it’s amazing how much they accomplished and learned in a short space of time.

The Processing libraries aren’t quite as complete as some C++-based libraries, but they’re a good place to start. If you’re considering doing something similar, I recommend my friend Dan Shiffman’s posts on his library contributions:
Kinect and Processing
Updated Kinect Library for Processing

And by the way, this work was an extension of the strings featured in Alex’s excellent New York subway sonification, about which we I to interview him:
Music Made with NYC Subway Schedules; HTML5+Flash, Q+A with Artist-Developer

Another Kinect hack: Matt Gattis produced the Bionic DJ project with “Kinect, libfreeconnect, and the OSC MIDI protocol.”

Beat Grids and Sine Waves with ChucK

I unfortunately don’t have good documentation of Jordan Orelli’s project, but he has some fascinating ideas. I laughed and said what he did was build a DIY Tenori-On with a Novation Launchpad and ChucK, but it is actually unique.

The grid of the Launchpad is a pitch sequencer – that we’ve seen many times before, and it’s very useful. But the grid can also become beat-synced modulation, which makes it possible to do some lovely, rhythmic manipulation of sounds.

The top row of the Launchpad is used for selecting instruments. The rightmost column selects “modes” specific to that instrument. The grid controls the current mode. All instruments run concurrently, so you can reasonably have a rack of 7 instruments, with the 8th instrument slot being reserved for the “mixer” instrument, which doesn’t actually mix anything but it lets you change the tempo (generally crashing the application in the process).
Everything is written in ChucK and no samples are used.

http://wiki.musichackday.org/index.php?title=ChucKPad

The SMS DJ

DJs may want to replace the crowd members making requests. Tough – the crowd may just ditch the DJ for a robot.

There were a number of crowd-sourced playlists ideas, including one cleverly named Youzakk and hooked into location check-in service Foursquare.

But djtxt was, amazingly, a whole service built in a weekend, complete with slick user interface. To make it work, it uses a whole lot of services: Twilio for SMS connectivity, Grooveshark for playback, Last.FM and musXmatch for albums and lyrics, and many others. Full details:
http://wiki.musichackday.org/index.php?title=Djtxt

And they did the other thing widely-respected by Web geeks: they deployed to a live site.
http://djtxt.me/

Drum Loops, From Your Browser to SoundCloud, and More HTML5

Two big trends emerged that are relevant to anyone interested in making music in the Web browser – without necessarily giving up your “real” (read: traditional desktop) production tools.

One: HTML5-based Web tech, while not entirely polished yet, is indeed enabling some basic music functionality right in the window of modern browsers.

Two: things like SoundCloud connectivity mean you’ll be able to generate quick ideas and then download samples later. (Ableton Live made a number of cameos in the afternoon demos.)

One great example of that is PatternMusic. It’s a pretty terrific little drum machine. But Ghostly International’s Haig and Miguel, who began the project in last summer’s Visual Music program at Eyebeam (in which I was also a participant), made a big leap forward this weekend: SoundCloud export. In turn, Haig worked out how to make PHP wrappers for SoundCloud much simpler and more effective. That’s a hack I hope we get to share soon.

http://wiki.musichackday.org/index.php?title=PatternSketch

http://patternsketch.com/

Also very cool: battling beats at SoundCloud beat battle. Match your groove-constructing skills against Ghostly’s Miguel or Com Truise. You’re going down, Truise, no matter how cool you are.

http://patternsketch.com/battle/

CDM Coolest Hack: Vib-Ribbon Clone

For the uninitiated, Vib-Ribbon, a Japan-only masterpiece by music game innovator Masaya Matsuura, is one of the high water marks of music games, a trippy walk through cartoon lines animated by sound.

Vib-Ribboff by Robert Böhnke cloned that game entirely in the browser, using SoundCloud music and intelligence engine Echo Nest for analysis, all with JavaScript frameworks Coffeescript and Raphaël.js. It’s a sharp parody of the original, and the sonification works. It’s too bad lawsuits exist, because otherwise it could become the most popular feature of SoundCloud. Can’t someone, like, license this?

CDM Funniest Hack: Faux Geocities Fans

Fans Forever and Ever cracked up the audience with a brilliant, generative version of horrible fan pages. It even fakes the awful GeoCities-era HTML and creepy, stalker-ish poetry (see screenshot). I hope this actually shows up online.

CDM Underdog Bet: Music Notation

Trendspotters no doubt got into the crowd at the hackday. (Famed venture capitalist Fred Wilson was there, for one.)

Trend they almost certainly wouldn’t spot: the likely growth of music notation on the Web and tablets.

Only two hacks really capitalized on this – one a score follower, and the other, pictured here, live annotation. But recall that, alongside the better-publicized MP3, guitar tab was an early hit of music on the Web. (Yes, it made music publishers and copyright holders grown, but that misses the point: huge swaths of the public consume notation.)

The reason is this: even as music education suffers in the US, a mind-boggling number of people play music, and since nothing has really replaced music notation, that means scores still matter.

The ability to mark up a score in a browser and share those markings, live, with anyone with a computer or tablet or other Web-enabled device? Priceless.

http://wiki.musichackday.org/index.php?title=Live_score_annotator

This clever tool will even follow a score in time, coupling algorithmic processing (to hold the right place) with broadcast information (to keep everyone in sync):

Follow the Muse

Drawing Sound with SuperCollider

Drawing Restraints by Mike Clemow was one of a number of pieces that focused on live synthesis and not just clever ways to replace Muzak. I have to give a nod here to Mike, as aside from his own project, he was an anchor of a little corner of the room working on live music apps, a big source of energy and enthusiasm. His work, aside from live performance, also appears in gallery contexts.

Also, bonus points for actually performing in his demo – that takes guts.

Drawing Restraints is a musical work for joystick, pen tablet and digital synthesis software. There are four modes for the instrument, two are buffer-based granulation modes using recordings of meat frying and a group of men talking, respectively. The third is a sine wave granular synthesis mode, and the last uses a saw-tooth wave through a filter bank to generate sound.

The synthesis is done entirely in SuperCollider and the input data is routed through OSCulator in the case of the Wacom tablet and a simple Pure Data patch for the HID based joystick. OSCulator does not have a HID input feature as of this writing. Both send the input data over Open Sound Control to Supercollider. While Supercollider does have a HID interface, I prefer to keep my programming interface unified; I merely have to create OSC responders in Supercollider in order to receive the data.

The different modes have similar parameters, however, each is mapped in a different way to the inputs. The modes can be combined to create complex sound objects that are independent, but their behavior is constrained relative to the state of each of the other modes. Their orchestration is constrained by the mapping scheme.

During Music Hack Day 2011, I came in with the hardware and the idea and brought the instrument to a state of playability. This piece will premiere at Zora Art Space in Brooklyn on Feb 23rd along with two others, “3coil,” a piece for induction coils and laptop, and “Outis,” a piece for video stream, computer vision algorithms, and custom synthesis software.

http://michaelclemow.com (home page has information about upcoming show)

Music Hack Day Page:
http://michaelclemow.com/index.php?/projects/music-hack-day-2011—nyc/

Best Networked/Collaborative Hack

JSONloops, an open-source real-time multi-user audio sequencer for collaboration, was an insanely ambitious project. And it wound up failing, likely for simpler reasons. While a first demo ran into network problems, the second go indeed worked!

The team:

Marak Squires – Created project, invented the JSONloops format, built core sequencing code
Elijah Insua – Writer of C bindings, solver of the hard problems
hij1nx – C Programming, JavaScript, HTML, UX and User Interface Dominator

Oh, and here’s what happened the first time around, according to Marak: “The software was working the whole time, but the machine connected to the projector decided to connect to a different WIFI network and we couldn’t access our local server.”

Yep, been there. But the project looks fantastic and does actually work perfectly well. Networked music-making is a topic for an entirely separate article, so I hope to talk to this crew more.

Updated – Marak lets us know he used the Socket.IO cross-browser sockets library:
http://socket.io/

Seriously cool stuff, as it also supports mobile browsers and older desktop browsers that don’t have direct sockets support.

Three features you wished were in SoundCloud

1. Pulling samples into Ableton Live.
2. Splitting up DJ sets into tracks.
3. Downloading SoundCloud sets as zip files.

Done, done, and done. Hope to see them released.

Fun SoundCloud Tricks

Tweetsonbeats.com turns a Tweet into a synthesized hip-hop memo. You can do it to your own tweets (or perhaps retweet beat poets of our time like Sarah Palin) with hashtag #tweetsonbeats. This is what SoundCloud co-founders do for fun. Really. For instance:

@haynes_dave: It went down but Tweets On Beats did a great demo. Just add this hashtag to a tweet and you’ll get a hip-hop memo by Tweets On Beats

And they composed a theme song for the hackday.

Listen to the NYC #musichackday 2011 theme song. Produced by @ericw, vocals by @lenberg at General Assembly on Sunday afternoon by David Noël

Follow up…

I hope that we see some of the code from this event polished and further developed; if it’s relevant to CDM readers, I’ll absolutely share it. And if you have creation events you’d like to see, let us know.

Liveblog of demos

If you care to read my own notes to myself, I live-blogged the event.

2011 hack list, with some great resources and (for many projects) code

It’s winter in the Northern Hemisphere. For some of us, there’s little need to remind us of snow and ice. But if you fancy adding some frozen sounds to your music, we have both free samples and expert recording tips to help get your cold on.

Frank Bry, a master sound designer, apparently has plenty of access to snow in his home of Idaho, but that hasn’t dampened his enthusiasm for the white, fluffy stuff. He’s devoted an entire library to Ultimate Snow with some 300 locations. You can read an interview with him on the superb field recording site Sonic Terrain:

Recording Snow Sounds: An Exclusive Interview with Frank Bry

Now he’s back at it again, turning his attention to capturing the sound of snow falling from giant fir trees, as they shed the weight of the snowfall. He employs patience, ingenuity, and some serious recording gear. The results from the end of November:
Let it Snow, Let it Snow, Let it Snow (also via Sonic Terrain)

Snow Falling Off Trees 2010 by therecordist

If you prefer to use snow and ice for more directly musical purposes, the good folks at sample house Tonehammer have a massive collection of wintry wonders to give away. The 2009 edition is back as a mass-download (WAV/Kontakt/SFZ), and 2010 brings a new sample each day through December 25, advent calendar-style. It’s like a big box of Turkish Delight.

2009 includes drums made from snow sounds, and percussion produced by throwing stones on ice. There are other, non-precipitation entries, too, including a Gnomish Choir (Helium Choir) and toy glockenspiel.

The Bedroom Producers Blog can get you connected with this and many other freebies:

Gnomehammer Samples Free Until December 25th!

And if all of this has made you feel a chill, warm up with a free Windows VST warmth/saturation plug-in from the same site.

What if you could make any device or any software a re-programmable musical instrument, effect, or soundmaker? Your phone could be a touch-controlled effect, your tablet a sketchpad for interactive drum sequencers. Patches assembled on your desk on a computer could be taken with you in your pocket. And what if you could do all of this for free, using a time-tested environment?

libpd, authored by Peter Brinkmann, takes on that vision. It’s a way of making Pure Data (Pd), the visual development tool for interactive music and media, more accessible across a range of applications and gadgets. It lets you embed Pd pretty much anywhere. It’s not a new version of Pd. Instead, it makes use of the standard, “vanilla” distribution of the free and open source software. What’s different is that it separates the sound processing part of Pd from the part that talks to audio hardware, allowing Pd to run on a greater variety of mobile devices and inside other applications.

libpd:

• turns Pd into an audio synthesis and processing library
• liberates Pd from GUI and drivers
• allows for easy communication between Pd and the code into which it is embedded (so you can send and receive messages with your Pd patch)

Today, a team of developers and testers (including myself) is releasing the first version of libpd. It’s free to use on any device you wish, and free to modify. Because of its licensing, you can even build commercial applications with it. (That is, yes, it’s open source – but yes, it can also be useful if you’re a commercial developer. You don’t have to choose.)

http://gitorious.org/pdlib | community discussion

We’re just pleased to have a tool that makes experimenting with sound and music quicker, easier, and more flexible and compatible. It’s more fun that way.

Supported Platforms, What You Can Make

Right now, today, you can use libpd with:

Android: Thanks to Google’s NDK (Native Development Kit), you can use libpd with any Android device running OS 1.6 or later. Note that devices without the Google Market are often non-standard in other respects, so your mileage may vary, but we’ve found a wide variety of devices work quite well, including the Motorola Droid and Droid X, HTC Legend, and Google NexusOne.

iOS: iPhone and iPad models with the latest, armv7 processors work (3GS, iPad); we’re working to extend compatibility across more devices. Working with Peter Brinkmann, the RjDj development team contributed (and continues to contribute) free code that’s making iOS support compatible and high-performance. But the Objective-C classes mirror the Android and Java classes, meaning the two will stay in sync, and once you’ve learned one, the other will be a piece of cake. (Or coffee. Or cocoa. Or whatever.)

In each case, you just need libpd, Pd for making your patches (graphically), and a copy of the SDK for each mobile platform you want to use.

Additionally, you will soon be able make user interfaces for libpd using cross-platform HTML5, via Chris McCormick’s project WebKitPd. (It’s not quite ready for consumption yet, but will also be free and open source.) Android was the impetus and initial test platform for libpd, so right now it’s the most mature. But we hope to improve iOS compatibility and testing next.

Some Sample Apps to Try for Android

libpd is really aimed at developers who want to embed Pure Data into mobile devices, games, and so on, and soon also people working with Processing, Open Frameworks, and the like.

But if you’re eager to try this out as an end user, there are a number of packages you can try. They don’t show off everything libpd and Pd can do, but they do allow you to load up something on your device and make some noise.

Download the test packages from the libpd site:
http://gitorious.org/pdlib/pages/Packages

It includes a scene player for RjDj (see below).

Among the code included in the repository is one complete app, Peter Brinkmann’s own Circle of Fifths. He tells us:

Circle of Fifths: I wanted a circle of fifths tool for the subway, with exactly this kind of GUI. It also nicely illustrates the newly possible separation of concerns — Pd only does DSP, and an elaborate GUI is built somewhere else. It’s a demo and not optimized for universal consumption. In particular, it’s a bit CPU hungry because it’s actually simulating six Karplus-Strong strings in real time.

Of course, if string simulation is what you want to do, this also illustrates that you can – even on a phone.

Chris McCormick has created two libpd-based apps, one of which I feature in the video above. Can of Beats also makes use of WebKit as its UI rendering engine. Chris describes what he’s made:

Can of Beats: This is a procedural hiphop beat generator. The beats are generated using simple hand-crafted probability weightings for each type of sound at each position in the beat. In the Android app, you can also input simple melodies or basslines to go along with the beats.
http://mccormick.cx/projects/CanOfBeats/

Garage Acid Lab: This is an algorithmic, 303-style acid bassline generator. The app will make you an infinite number of different acid bass lines and garage style beats. You can also have some fun with the cutoff filter and delay unit settings with a kaos-pad style input. I want to work on this app a bit more to provide an ‘advanced’ mode which will let you write custom basslines, beats, and have more control over the effects. http://mccormick.cx/projects/GarageAcidLab/

RjDj, RjDj team. If you don’t know it by now, RjDj is a fantastic application built on Pd that makes interactive musical and sonic experiences deliverable in the same way as a digital album, not only to musicians, but anyone who wants to experience music and sound in new ways. libpd makes use of code contributed by RjDj. Future development on RjDj will use libpd. (More on those libpd-based versions, and the evolution of RjDj and RjDj Voyager, soon.)
http://www.rjdj.me/

Where to Get It, Where to Get Involved

1. Get the library. To get started, download libpd from its Gitorious source repository:
http://gitorious.org/pdlib

You’ll need Pd, too, if you don’t have it; vanilla Pd builds are available from the official Pd download page.

2. Join the community. You can discuss patching for libpd, developing using Pd, and making instruments and effects and other sonic creations for gadgets everywhere on our new community group:
http://noisepages.com/groups/pd-everywhere/

That will be a location specifically dedicated to the unique challenges of working with mobile gadgets; of course, see also the other great community resources for Pd.

Who made this: libpd was conceived by a team including Peter Brinkmann, Hans-Christoph Steiner, and myself, with input from the RjDj team (particulary Martin Roth). It was primarily developed by Peter Brinkmann, who applied his talents and the work he has done in JJACK, a Java API for JACK, with additional contributions and testing by our team and by Chris McCormick. Major thanks to Martin Roth and the folks at RjDj, to Miller Puckette (creator of Pd), and the generous attendees of our first hackday at the NYC Patching Circle, along with others who are testing now.

Tutorial Next Week; Your Feedback Wanted

I’ve been preparing a tutorial for working with libpd Android (initially), to be followed eventually for developing on iOS devices once we have a better handle on making that go smoothly. We’ll have a complete tutorial for you by next week. Processing is then my next priority.

An FAQ will also be available by then. That means, first, ask some questions!

Got specific questions about what this is for? How to get started? What you’d like to see in the tutorial? Ask away.

And please do get the discussion going not only here in comments, but in the Pd Everywhere group. (Noisepages registration is now open; if you have any trouble, let me know and I’ll sort you out.)

Grain Main Frame is a sound sketch, a one-off piece of software that loads audio files and plays them via several inventive, homebrewed sample players. Via granular techniques, methods of slicing sounds into tiny grains and then re-assembling them, a single sound can be stretched, sliced, and retriggered creatively. The software supports drag-and-drop functionality, as well, so you can drop files and go.

It’s a simple app conceptually, but it’s already packed with functionality in this early version. In addition to drag-and-drop file loading and a folder full of homemade samples to play, the software includes:

• Gesture recording of mouse movements, a la KORG’s KAOSS Pad series
• Rate, interval, loop point, randomization, and pan settings for granular playback
• OSC control, so you can manipulate parameters via an iPhone, a Max patch, etc.

The software is built in Processing, the artist-friendly code environment, using the Beads sound library for Java. Because it’s Java-based, it should run on any platform. (I did have an issue with the executable jar on Linux, but I’m working with Jeremy to see if I can fix the problem!)

It’s free-as-in-beer; no license or source is included, but Jeremy tells CDM he does plan to give most of his work away. (If this does develop into a more mature app, you may see a paid iteration some time down the road, but not any time soon, says the author.)

Grain Main Frame is Released [jeremywentworth.com]

Check out Jeremy’s site for some other cool projects in Reaktor, Max, Max for Live, and Processing.

The iPhone and now iPad have spawned a lot of talk about the idea of small, simple apps rather than big, monolithic workstations in software design – musical and otherwise. But projects like this suggest that we could see similar trends in software elsewhere. After all, there’s no reason you couldn’t load up a new netbook or slate with some simple sonic tools, too; no Apple logo is necessarily required. It’ll be interesting to see how that evolves. (And they could be connected, too: to use Java as an example, there is a Java JACK implementation for routing audio to and from other applications.)

Via our friend Richard Devine in comments, here’s another cool app, this one powered by Csound. It’s a separate project, not authored by Jeremy, but Jeremy has done a lovely screencast. (Grain Main Frame could also substitute other back ends for sound processing).

That project is free and open source, working with Python and Csound:

http://code.google.com/p/soundgrain/

Csound downloads

More information on granular options on Jeremy’s blog:
Obsession with Granular Synthesis

Pictured: Looks native, but this app is built with a cross-platform library. And really, for music making – or great, immersive development, in general – does it matter?

The iPad has inflamed plenty of passions online. On this site, I’ve gotten a little flak from iPad lovers and haters alike. It goes something like this: “wait a minute, you’ve got all these criticisms of the iPad’s restrictiveness, but then you’ve got all these amazing music apps.” Or, on the other side: “why do you keep covering all these iPad music apps?”

In a word, yes. They’re not the same issue. I’ve talked to plenty of developers. The business draw on iPad is a big deal for independent, creative developers, so to the extent that Apple strategy makes the store a good place to sell apps, there’s some overlap. But the iPad has also been attracting plenty of music developers because of the quality of the APIs – developers who often aren’t pleased with the restrictions. Does the person who writes the audio drivers and APIs have anything to do with the lawyer who writes the developer agreement? Of course not.

The problem is, just as iPad/iPhone critics sometimes conflate issues in their rush to criticize the platform, some of the defense from the Mac community is getting a bit carried away, too. We’ve seen this with design issues, not just ideological or business issues: you go from “touch can be an expressive way to interact with a computer” to “throw out your QWERTY keyboards! They’re dead!” or “you’ll never read a magazine again” or “multitasking was a terrible idea in the first place.”

In this case, Apple made a fairly specific change to a developer document. That resulted in some criticism that was over the top (namely, people claimed it’d stop specific developer tools before they had verified whether that was actually the case). But it also resulted in some Apple apologism that was downright surreal:

All cross-platform development is bad? Wait – what?

And for that matter, is the mark of great software design now exclusively using Apple’s developer toolkits? Wouldn’t we sort of hope that, beyond those slick Apple UI widgets, someone somewhere might be developing the UI of the future? For that matter, do people not realize that a lot of what makes Apple’s quality exceptional is stuff you can’t see – things like multitouch firmware, high-quality audio drivers, and other fit-and-finish on the plumbing?

So, I invite you, dear reader, travel with me. I think we may actually have something on which iPad critics and fans alike can agree. It’s relevant to music, because music apps (along with games, incidentally) are the ones that are most intimate with this issue. And I suspect a lot of you use cross-platform tools to develop code for your day job.

The Catalyst: Apple’s Legal Change

Apple surprised many in the tech world last week by making a change – mandatory to all developers – that requires that applications for iPhone “must be originally written in Objective-C, C, C++ or JavaScript [running in the WebKit browser engine].” More specifically-worded, “Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited.” Because of the timing, and because of the further clarification, conventional wisdom suggests this is aimed at Adobe CS5′s tool for making native iPhone apps from Flash code. I don’t think it should be any surprise that that would get developers upset, not only those who use Flash, but even some loyal developers who don’t like being told what to do by Apple. (And I know at least some fairly big fans of the iPad weren’t fans of this change.)

Had it not coincided with Adobe working on CS5, I don’t know that this would have been big news; Apple already restricts the languages used to develop on their device. But I think what set people off may have been that very problem: people don’t know what it means, and that (rightfully) makes them nervous. While online debates have devolved into idealogical extremes “All control is good! / Apple just killed Adobe!”, what the press has missed is a sense among developers that they can’t predict or entirely interpret Apple’s developer agreements. I suspect Apple did aim this at Adobe, but that means even non-Flash-using, native-developing software makers now have to face some serious ambiguity in a legal document they have to sign.

That said, if Apple would further clarify the statement, that could be resolved.

I didn’t personally get worked up over this, because it’s consistent with what I and many, many others have been saying about the platform all along. Apple’s control over distribution and desire to control the development process means this is the sort of thing they can do. There are reasons to endure it: they make a really well-engineered platform, and there’s a terrific market and installed base that has a voracious appetite for creative software. There are also clear reasons to look elsewhere if you’re not comfortable with the restrictions. This is the very definition of trade-offs and choices.

I’m going to avoid the debate not because I don’t think it’s important – I think it is – but because I think enough words have been written in service to one side or the other. What seems to be missing, however, is a shared understanding of what cross-platform development actually is.

The Trend: “All Cross-Platform Development is Bad”

Whatever Apple’s thinking, it’s caused some people apologizing for Apple to say really weird things. John Gruber at Daring Fireball, for instance, begins by making an entirely reasonable argument for Apple’s strategy and where they live in the market. I don’t agree with all of it, but it is a well-reasoned, well-argued point. Then, almost as a footnote, John makes this claim:

Cross-platform software toolkits have never — ever — produced top-notch native apps for Apple platforms. Not for the classic Mac OS, not for Mac OS X, and not for iPhone OS. Such apps generally have been downright crummy.

[emphasis mine]

Why Apple Changed Section 3.3.1
[Daring Fireball]

That argument has gotten picked up all over the Web by other Mac fans. And that, to me, is dangerous – because, as worded, this statement appears to be to be entirely indefensible.

John doesn’t say “cross-platform compatibility layers” or “meta-platforms” like Flash and AIR. He says “cross-platform software toolkits,” and I think he means it. (Now, John, if I’m wrong, please correct me – but please stop making statements like this, because “cross-platform” is what many of your readers are coming away with.)

This would likely come as news to those who use music software. Cross-platform software frameworks are at the heart of most of the tools we use. One small but lovely example, specific to the iPhone/iPad and absolutely kosher under Apple’s new developer rules, is LibNUI, a C++ framework for building UIs. (In fact, after playing with this a bit, I may pick it up for a project on a completely different platform.) Popular iPhone apps like bleep!BOX and BeatMaker use it, but it also keeps tools like MOTU’s MachFive plug-in compatible with multiple platforms, without sacrificing native features like drag-and-drop.

If you use Ableton Live, Max/MSP, Cubase, or countless other apps, you’re using software created in cross-platform frameworks – some in-house, but using the same basic technology. Indeed, few of these applications would work the way you expected if they used exclusively “native” features and design patterns, like UI widgets that don’t fit musical applications or don’t work in live music performance.

In fact, John’s statement is so broad and over the top, I think it might even apply to tools like CodeWarrior, the developer tool and, yes, cross-platform framework that was the dominant toolset for developers in the pre-X “Classic” Mac OS era.

This matters to users, too. Sure, you may never write a line of code, but you rely on the community of people who do. Part of what gives you the freedom and flexibility to run great software on a variety of platforms, rather than being locked into just one platform, is the fact that these tools make the differences between those platforms fall into the background. Any developer who thinks this happens automatically without effort or testing is likely to give you a terrible app, but odds are, they’ll give you a terrible app regardless of what tools they’re using.

Develop Once, Run Anywhere?

Macworld editor Jason Snell also picks up the old argument about cross-platform development being inferior. (The title, I think, may be the most insightful part of this piece, but I’m not an Apple employee or investor, so I’ll let them worry about that.)

Apple against the world [Macworld]

…the develop-once-run-anywhere philosophy is something that makes more sense to bean counters and development-environment vendors than it does to platform owners and discriminating users. In the ’90s we were told that Java apps would be the future of software, because you could write them once and deploy them anywhere. As someone who used to use a Java-based Mac app on an almost daily basis, let me tell you: it was a disaster. Java apps didn’t behave like Mac apps. They were ugly and awful and weird, but hey, at least they ran on the Mac.

Ah, yes – this argument again. (It’s one of those things from the 90s that just never gets old, like Ace of Base or plaid t-shirts and pleated khakis.)

Okay, I kid, but Jason – I feel you. Actually, I feel you even as a fan of Java; the language and platform have some real power, but because of some questionable tooling atop them and questionable development practices with them, it produced some really horrible products. Such is development. (Actually, arguably, the folks in the 90s were right – it just turns out to be the browser itself, not Java applets, which have nothing to do with modern Java development anyway.)

I think Jason is mostly hung up on things like UI widgets; he refers specifically to the lack of a menu bar, odd preferences dialogs, and other usability issues in the AIR application TweetDeck. (Part of the reason we don’t nitpick these things in music, of course, is that we’re using extraordinarily complex interfaces for doing other things.)

Jason misses some critical points, however – in this case by omission; he doesn’t make the same, sweeping statement Gruber does. (Jason told me via Twitter that he wasn’t set to write another 2000 words, so Jason, I’ll try to do that for you.)

In regards to Java, the reason Java apps don’t feel like native Mac apps is at least in part because of Apple. It is actually possible to do all the things Jason is describing; Apple themselves touted the feature. You can read the documentation, and the fact that it was deprecated way back in 2005, on Apple’s legacy Mac developer documentation site. I can only speculate about the decision there, but my guess would be that it was practical more than strategic. There’s a new open source project to replace this functionality, Apple themselves recently made interfacing with native code easier for Java developers, and whatever language preferences Apple has on the iPhone, they continue to support projects like Ruby on the desktop Mac.

Generally, I think you’ll see more native feel in apps for Mac, Windows, and Linux from Java, Ruby, Python, and other languages. It’s an area of active development, and it’s improving. It may also benefit from these communities breaking off from big corporate parents, because the developers themselves seem to understand the perspective of the users better than, erm, companies like Sun and Oracle. Bottom line: don’t be surprised if some day soon you again run a Java app (or another language, not necessarily Java) and don’t notice. Those “discriminating users” on the Mac do notice when it’s wrong, and very often want to get it right.

Art, Tools, and Cross-Platform Frameworks That Don’t Suck (Or Break Apple Rules, Maybe)

But it’s not just about the standard Mac widgets. Jason, definitely check out Processing and the fantastic art made with it? It’s Java, though that doesn’t matter and isn’t immediately apparent, which is good.

If you design became only about widgets and preference bars, even nice Mac ones, we’d have a generic, bland, look-alike future for software. I know that escaping bland, cookie-cutter software is what drove a lot of people to the Mac in the first place, so it’s worth reiterating.

Tools like Java aside, though, somehow lost in this debate is the fact that cross-platform development is wildly popular and largely transparent – just in the language C/C++. From games to serious software, a whole lot of software is written in cross-platform C++, with the bulk of the code compiling on different operating systems and even hardware architectures. Developers typically make use of various frameworks to ease this compatibility.

Furthermore, while I still think there are reasons to be wary of Apple’s policies and this decision in particular, it would likely be inaccurate to claim that the recent change blocks these tools. In fact, several specific examples all use native code to link against the official Apple APIs, meaning they should be safe. These applications are exceptions that prove the rule: they’re great cross-platform tools that can produce great apps, they’re allowed on the iPhone/iPad OS as near as I can tell, and in some cases they’ll also be cranking out great apps for non-Apple platforms. Adobe’s big sin may have been allowing development from Windows, meaning you don’t get all those designers buying new MacBooks. Here are some examples of tools likely to be safe:

iPhone Wax uses Lua, but it still uses Xcode templates.

Corona, an awesome development tool for OpenGL-accelerated apps, has a specific response. Oh, and it’s coming to Android, too.

Unity is producing fantastic games and should likewise be safe under the new rules.

OpenFrameworks, a brilliant framework for artists that allows them to produce creative, interactive applications with music, visuals, and media for Windows, Mac, Linux, and platforms like iPhone/iPad is written entirely in C++ and appears to be okay. (Again, you use Xcode and Objective-C to link against official Apple APIs.)

Not incidentally, each of these tools (and LibNUI, above) could make some amazing music apps, some likely developed by readers of this site.

Let’s be clear: critics of Apple’s change likely overestimated how many frameworks would be impacted. That meant people were making an argument that may have been divorced from the facts. That said:

Just because they got the argument wrong doesn’t mean criticism (or defense) isn’t warranted. Apple did make a major change to the developer agreement, and they made it – apparently – as a reactionary response to a particular technology, in a way that could threaten other, unrelated technologies. The debate may have gotten overheated and inaccurate, but it’s understandable that the underlying cause is cause for concern. In fact, I think there’s no reason that Mac-centric media outlets couldn’t point that out. And developers really should consider leaving a platform if they don’t like it. (If it’s Apple’s right to make the rules, it’s certainly likewise the developer’s right to vote with his or her feet.) I think there’s an argument to be made in defense of Apple – I could certainly make that argument if someone dropped me on a debate team and put me on Apple’s side, even if I happen to disagree.

The jury is still out on just what apps are impacted – which should be further cause for concern. In fact, I’m still not entirely sure what the status of the apps above may be. On the blog /dev/why??:

In my opinion this is not purely aimed at Flash, but it is certainly precipitated by Flash CS5. I can’t imagine Apple is happy about environments like MonoTouch, Unity3D, PhoneGap, Appcelerator, or Corona, but I am doubtful they would have changed the license in this way just to stop developers using those environments …

He apparently thinks, however (though even the developers of Corona do not), that these frameworks could become verboten. Furthermore, he notes the case of “interpreted code” and why it’s important (it happens to be useful in music apps, too), though my understanding was that that was already a violation of the agreement. (Perhaps it’s clarified here.)

The more interesting thing from my standpoint is that this makes it a license violation to include a language interpreter inside a game. If you aren’t a game developer you might not be familiar with how large games are structured, but most games consist of a game engine, which is high performance code for doing things like rendering graphics, and an interpreter which runs the game logic (determining how sprites move, determining when to pop up in game text boxes, etc). This is how practically every commercial RPG works, as well as many (most?) other types of games. This affects major app store publishers, like EA, Gameloft, Tapulous, and ngmoco:). Looking at the top ten lists on the app store right now I see several titles that I know have embedded Lua interpreters. In this case I think these apps are genuine collateral damage, though I honestly doubt Apple would attempt to enforce the clause against them. In fact, using an interpreted language for game logic is already technically in violation of section 3.3.2 in the current agreement, though many developers may not realize it because under the original agreement it was okay, and the change that made it verboten was very subtle (changing an “and” to an “or”). I am actually not sure exactly when that changed, and only noticed it myself while I was researching this blog post.

See comments – ultimately, the language question is the big one. It could have a negative impact on developer flexibility, and specifically could impact DSP code. As Richard notes in comments, it’s all a matter of what Apple chooses to enforce. It’s possible that the letter of the law makes all of these things illegal, but in practice, Apple just wants to block Adobe’s tools.

But let’s also be clear:

Major voices in the Mac community are advocating against cross-platform software, even without a complete understanding of what that means. And you can actually defend Apple’s rule change Others (like Jason Snell at Macworld) I think just don’t get the opportunity to be clear. But let’s be clear. Let’s makes sure that idealogical discussions on both sides of this debate don’t obscure the facts.

Digging into Apple’s own, platform-proprietary tools can be a great thing. My friend vade, a sometimes-contributor on Create Digital Motion, has done great work with Quartz Composer, for instance, as an artist, and knows Core Image backwards and forwards because it allows him to express himself.

But that’s just one avenue. I know other developers who have found that working across multiple platforms ultimately makes their software better. Jason Snell unfairly, I think, characterizes this as “lowest-common denominator” development. If that’s what you’re doing, well, yeah, that would kind of suck. But I’d call this “highest-common denominator” development: the more you need to make code work on multiple platforms, the more, very often, you have to optimize all of the platforms, the more you discover opportunities to improve your code and make it a more general solution to a problem.

The truth is, you can use the cross-platform tools above to make fantastic iPhone/iPad apps, apps that feel entirely “native,” but apps that will also – by Jason’s description:

…[create] a world where App X for iPhone and App X for Android are indistinguishable from one another.

When it comes to games, to music apps, to creative applications with alternative interfaces, to immersive applications, to rich media interfaces – developers are creating that world, period. Apple can’t stop developers from doing that. Given that they tout availability of apps for their platform that were built with that model, I’m not even convinced Apple always has a problem with that development model.

The cross-platform world is here already, and it’s growing. And, honestly, I think it’ll be a good thing.

For further reading…

This post is really rhetorical, but it hilariously takes the legal clause to its logical (if not practical or likely) conclusion:
Apple bans modular programming

It’s amusing reading, but as author Jeff Erickson (“Ernie Pan”) responds to comments, the real bottom line comes out: “But it doesn’t matter what Apple means. The license is a legal document; the only thing that matters is what it actually says.” Of course, that leads to still more unpleasant revelations: it doesn’t matter what the document says or Apple means, but what Apple actually does. And Apple can change what it does at any time.

Tao Effect notes, as I do, that cross-platform toolkits can be made to look like native apps, or even that it may not matter what they look like (because as a game, or in my example of a music app, they all look different by necessity). It also responds to what I think we could now call the Jobs Doctrine:

“We’ve been there before, and intermediate layers between the platform and the developer ultimately produces sub-standard apps and hinders the progress of the platform.”

My issue: I”m not sure what Jobs means by the terms “intermediate” or “layers.” In fact, I’m not entirely certain what he means by “sub-standard.”

The blog notes that – while I suggest that maybe some of them are safe – things like MonoTouch, popular apps that feature Lua scripting in their development, and the widely-used Unity 3D game framework may well not be allowed in the store, which could mean more unpredictable rejections.

Ah, to be using a game console, where almost everything is rejected and you only have to worry about the few apps that make the cut…

Whatever the implications for the iPhone platform, though, these stories underly the point I’m really trying to make here – whatever Jobs may seem to be saying or Apple advocates are arguing, the notion that cross-platform development creates bad apps is one that is seriously open to debate.

In the meantime, we have several groups who don’t speak the same language or technical understanding:
1. Apple lawyers.
2. Apple end users / customers / advocates.
3. Developers.

And then we have Steve Jobs making sweeping, provocative generalizations that are themselves enigmatic, because he’s, well … Steve Jobs. (So make that category #4.)

Software doesn’t have to mean virtualizing everything and letting go of physical objects. On the contrary, it can create all sots of imaginative, new ways of mapping musical ideas to the physical world. And that’s how we wind up with a walnut drum sequencer.

There’s something about virtual drum machines and snacks. We’ve seen bubblegum and Skittles, beer bottle caps, soda bottles, and now walnuts. Don’t stop now: someone has to do Cheetos, even if it means dealing with orange stuff all over your fingers.

That said, it’s not walnuts that make d-touch an important project. Built by Enrico Costanza back in 2003, the project is now available for free download as an open source library, a server (in case you don’t want to get into the C++ code but might want to use this in your own projects), a free, usable drum machine, and a set of documentation that can help you make your own stuff easily. Enrico worked on the original reacTable prototype and has done some really important work in this field. Right now, Enrico and co are looking for feedback, but if you’re ready to just be a tester and play with this – and see what you can do musically – now’s your chance.

d-touch also combines high levels of computer readability for accurate tracking with the ability to make your own tags. Instead of using ugly-looking glyphs, you can make patterns that make sense to human beings as well as computers. Oh, yeah – and mobile fans, this runs at a full 14 fps even on S60 phones.

For more, check out the d-touch site:
http://d-touch.org/ [Register first to make the download available]
and follow them on Twitter:
http://twitter.com/audiodtouch

Thanks to Martin (of reacTable, which is moving toward a commercial product) for sending this our way. Thanks, too, to Ben, who’s working on tangible interfaces with special needs students. I really look forward to hearing how that’s going.

The Cupertino-Mushroom Kingdom gap has been closed: you can now mix and match DS and iPhone/iPod touch for wireless control of music and visuals. DSMI, the homebrew library that has enabled wireless and serial MIDI connections from the Nintendo DS, has come to iPod touch and iPhone. That means anyone building instruments and controllers on the iThing can now add wireless MIDI controllers that talk to computers – or other mobile devices, including the DS. It also means that DSMI’s acronym standing for “Nintendo DS Music Interface” has only one word that describes all the things it does.

If you’re a developer, you can grab the open source (LGPL-licensed) code. If you’re a user, apps are already supporting the new wireless features. There’s MIDI Motion Machine, which provides tilt and 16 triggers, and iXY, a 99-cent app for KAOSS Pad-style X/Y touch control. The MIDI Motion Machine author, TheRain, takes an interesting approach: there’s both a free and pay version, and the free version has source code.

Tobias Weyand, DSMI’s original co-creator along with TheRain, writes:

My friend TheRain has ported DSMI to the iPhone! This enables iPhone deveopers to easily integrate wireless MIDI in their applications, making it possible to control any MIDI application on the PC with the iPhone. The Wifi-to-MIDI bridge is the same DSMI server application that is also used for the DS, thus it works with Windows, OSX and Linux.
Also, like on the DS, both OSC and MIDI are supported!

DSMI for iPhone is available from our Google Code site (http://code.google.com/p/dsmi/) together with an open source example application called MIDI Motion Machine that is a tilt-based xy-controller.

The cool thing is that this library takes away all the hassle of communicating MIDI messages to the PC and makes development of MIDI controllers very very simple. So, we hope that people will use the DSMI to create a lot of innovative iPhone MIDI controller apps.

Pretty cool, isn’t it?

By the way, if you’re a loyal Nintendo DS developer and think this whole iNonense thing is useless, the main library for DS also got a lot of improvements, cleaner coding, and collaboration on Google Code.

What about using OpenSoundControl instead of MIDI? On the iPhone/iPod, it’s a non-issue: OSC is a networking protocol, so it’s already wireless-ready. On the DS, DSMI’s source includes an OSC example, and unlike the MIDI in DSMI, you don’t need a piece of software receiving on the computer end.

Now, any suggestions for how to broaden the acronym DSMI so it’s more accurate? Digital Signal Multimodal Interface? Digital Sound and Music Interface? Damned Sweet Machine Instrument?

Or, to go recursive: DSMI Sure Means Ideas.

DSMI Official Site
CM Software Designs (home of iXY, MIDI Motion Machine, more apps and tutorials – must-visit)
DSMI at Google Code

The creator of the wonderful glitchDS, repeaterDS, and cellDS Nintendo homebrew music apps has turned his sights to the free and open coding-for-artists desktop tool Processing. The result: a drum machine that mutates and morphs in wonderful ways via a command-line interface. (I almost put the command line bit in the headline, but while I actually adore command lines, I think the more interesting part of it is the way it mutates its patterns in lovely ways. No boring endless step sequence repeat here.)

The tool is called Quotile, and since it is built in Processing and the code is entirely free, you’re welcome to try it out and change it around if you like! Apparently the Mac camp are having some troubles, but I don’t see any reason it shouldn’t work on Mac; the problem is generally that getting Java MIDI running on Mac has some tricky bits because Apple dropped support for the Java MIDI API, even though it’s a standard part of the Java platform. In this case, I expect it’s the library’s reliance on mmj or people having trouble installing that MIDI subsystem that’s the culprit. Keep the faith: it can work, and I hope we can get a standard, reliable MIDI library soon.

The sound source above: Machinedrum, of course.

I’ll give this a try on Linux later today, on the platform that I think has the best MIDI support, hands-down – yes, even compared to the Mac. (I’ll explain why I think that soon.)

Speaking of MIDI libraries, the Processing library this is based on is a new one called MIDI Bus. It’s very similar to wesen’s rwmidi, which we’ve covered before.

The project:
Quotile – new PC MIDI sequencer written in Processing at glitchDS

The free library for Processing (Mac + Windows + Linux)
Small But Digital – themidibus

Previous musical creations in Processing:
Strange, New Musical Interfaces, Built in Processing
DIY 3D Controller: Inspired by Theremin, Powered by Arduino, Processing
Tiction: Animated, Nodal Generative Music App in Progress, in Processing
Build Your Own Game of Life Sequencer in Processing: Video Featuring rwmidi
Help! I’m Trapped in an Acid-Colored Wash of a Thousand General MIDI Pianos!
Spaces and Roots: Manipulating Sound with Processing + Touch, Tangible Interfaces