NodeBeat is the kind of experimental music application that’s thriving in the age of the multi-touch tablet. Its dynamic interface and sound are built on the foundation of free and open source software tools regularly covered here on CDMusic and Motion. OpenFrameworks, the Processing-like C++ library, handles the UI, as libpd, the embeddable version of graphical media environment Pure Data, manages the sound.

What you get is an open-ended plane on which you can graphically array sequences, far away from the standard grid, for generative and sequenced music. It’s good fun, which made it a hit on iOS. Developer Seth Sandler, working with Justin Windle, did a brilliant job. Then, earlier this month, NodeBeat made the jump to Android, with additional porting work by Laurence Muller. Android has been getting tablets that can hold their own — I’ve enjoyed my Samsung Galaxy Tab 10.1, for instance. But the platform has remained severely starved of applications in contrast to iOS, but at least in place of quantity, there’s some quality: this application being one, tools like Mikrosonic’s RD3 or Reactable orControl or Nanoloop qualifying, too. (I’m not delusional; this does not make it at this point any serious competition for iOS, but it does demonstrate potential for developers. And I’ve already had the chance to use Reactable and Control in live performance, personally.)

That is, NodeBeat was temporarily available on Google’s Android Market. Then, without warning, Google suspended developer Seth Sandler’s seller account. This led to an extended discussion with Seth, other developers, and myself as we watched events unfold, ironically on Google’s own Google+. (Yes, that Google product works, despite what you’ve heard.)

It’s back now, so please, go buy and review it if you get the chance. If you’ve got a compatible Android, you’ve got truly no excuse as it’s a delightful app, and it holds up even in the crowded iOS platform:
NodeBeat @ Android Market
http://nodebeat.com/ (iOS and all versions; there’s even a free, desktop version with source code!)

Okay? Bought it? Good. Now it’s time to talk about how bad this is for a developer.

The account suspension on the Market represents a series of obvious flaws. First, of course, it shouldn’t have happened in the first place – Google support eventually acknowledged the suspension was entirely random, “incorrectly suspended” in the words of support, with no other explanation.

Second, support was largely nonexistent. Days passed during which Seth was left without any information. (Amidst discussions of how “evil” or “open” Google is, I’d sometimes be happy just to see them seem something other than desperately rushed. And that seems to be the primary “Apple-fication” of the market – the company’s rivals now are so rushed to try to compete that they screw things up constantly. “Don’t be crappy.”)

Third, and most bizarre, the application stayed available but payment was impossible, leaving customers confused and unable to buy the app.

Now, horror stories like this weren’t unheard of in the early days of the Apple App Store, and I still hear – with, happily, much less frequency – complaints from developers about Apple’s store and approval process. Apple deserves credit for ironing out those flaws, but from the skeptical perspective of a developer, It’s hard not to draw the conclusion that you may want to consider distributing your software via more than one means. Even as Apple fails to allow that on their mobile devices, that means considering going cross-platform. That’s not a philosophical claim; from the perspective of a developer, you don’t want to be dependent on only one company. Feel free to disagree, but my experience has shown otherwise as I’ve watched developers get burned. (And it’s worth noting that while Google couldn’t sell Seth’s app, Apple could.) Technically, via Android, developers are free on the vast majority of devices to sell direct or sell via alternative stores; unsurprisingly, Seth submitted his app to the competing Amazon App Store and is awaiting approval there.

None of this, of course, excuses Google from a big customer failure on Android Market. And whereas Apple’s earlier hiccups occurred as it was the only game in town, Google is making an uphill battle even worse. With Amazon’s Fire on the horizon, there are two questions to watch: one, can Amazon deliver enough tablets to create the tablet market Android has thus far lacked, and two, will their store deliver a better experience? Meanwhile, Google continues to promise a better Market; it’s all I hear about at developer events, largely because it’s the primary complaint from developers. As tech pundits make largely-unsupported claims like “Android users don’t like to buy software,” as if they’re a bunch of degenerate freeloaders, I’d point to the often-inferior Market and frustrating hardware experiences.

But, let’s put it this way: in addition to the obvious range of iOS choice, yes, there are superb applications beginning to appear on Android. For that, I credit developers like Seth and his collaborators. Even as we push for better audio performance, some of those applications are already running exceptionally well on new tablets and higher-end phones. If you have one of these devices, you can fire these up and enjoy making some sounds. And because you can’t always rely on another vendor to get things right, having cross-platform, free and open source tools behind these applications means developers have the flexibility to adapt to a changing market, and to focus on creative design and not constantly reinventing the wheel.

Here are some notes on NodeBeat’s creation on our forums.

And let us know what you think of NodeBeat, or if you do have an Android device you’re using for music (or a Fire on pre-order, for that matter).

You may have already seen FaceOSC, free software that eases the use of facial tracking from a computer camera for use as a controller, here with music software (top). Synthtopia picked up the story in July, featuring artist and engineer Kyle McDonald. But one FreeKa Tet has done his own implementation (second from top), and while the video is a bit grainy, he sounds wonderfully terrifying, as if his face is trying to slip out of The Matrix.

Sometimes, I’m rendered entirely silent (no, really, it happens), and it’s best to let videos speak for themselves. So here, after the jump, are some whimsical and wild prosthetic sound light-up … hell, I don’t have any idea what’s going on, but I’m enjoying it.

I dare you to start some conversation about musicality. Just don’t be surprised, comment trolls, if you find yourself abducted by a glowing and oddly glitchy-sounding creature with long, monstrous fingers. I’d watch what I say, frankly. Remember the old saying “on the Internet, no one knows if you’re a dog?” I expect that extends to space aliens, too.

Plenty more where those came from.

Oh, look, I could have done my research and seen there’s a bio for Mr. FreeKa Tet, aka Bacon ClapCLAP.
:

Extreme violent bursts, silence, speech, hard rock samples, strange atmospheres, Burgers , American Idol Icon, crackles, retardation, puking static, rocking a gabber party, cutting a worm in half and watching both parts moving, get a watch tatoo on his wrist, confusing videogames with music, drawing little puke characters on friends faces …

Right. Okay. That cleared everything up.

Enjoy your weekend, folks, and stay high and dry, those of you here on the Eastern seaboard of the US.

Music making with your face? It’s just the latest novel way of manipulating your computer with movement, thanks to a revived interest in camera-based interaction spurred by Microsoft’s Kinect and hackers making it work, and other computer vision libraries. One original work: FaceOSC, which uses custom tracking code and a standard computer webcam (no additional hardware required) and free code to send control information for applications like live music performance. Kyle McDonald may have already wowed you with his face-tracking wizardry, but it’s easy to want to know more. Sure, it’s cool, but, um, what is it for? How do you get started? Is the timing quick enough for this to work in music? And what can we expect in the future?

I spoke with Kyle, educator, artist, and coder, about those questions and more. He’s also got some examples of what people are already doing just days after the release of his software – there’s some serious viral quality to open source code.

You’ve done work with both Kinect and now, in FaceOSC, your own camera-tracking software. How is working with Kinect for musical applications, in terms of latency? How have you found latency in your own FaceOSC application?

The only number I’ve heard regarding the Kinect’s latency is from Synthtopia, where they give 100 ms. That seems a little high to me. In my experience, the depth camera seems to have an extra frame of latency compared to the color camera. So I’d put the latency somewhere between 30 ms and 80 ms. In other words: don’t expect it to be a precision tool for live percussion, but for everything else I think there’s just
as much to explore as with any other camera.

FaceOSC feels like adding an extra frame or two of latency on top of what you’re getting from your camera. So you shouldn’t expect to beatbox or do percussion with it, but for controlling parameters in a musical context, you should be set.

How do you imagine this being used? I mean, obviously, in some ways this is (very) experimental, if good, clean fun — is there a
practical application? (EyeWriter, the eye-tracking application that improves accessibility, comes to mind as one possibility.)

I imagine FaceOSC being used to prototype ideas surrounding face-based interaction. I created it because Jason Saragih, the researcher behind FaceTracker, uses an open source non-commercial license for his code. He asks that anyone who wants to use the code email him directly, as a way to keep track of the usage. This is great, but I know that one of the fastest ways to get cool stuff happening is to make new tools and research accessible to a wide audience. So I asked him if it would be ok to make a standalone app for people to prototype their ideas — even if they don’t have access to the code. Everyone already “speaks” OSC so I thought this would be the easiest way to get the technology out there. Eventually, if people need to integrate it into a single application, they can contact Jason directly and use my ofxFaceTracker addon to get started:
https://github.com/kylemcdonald/ofxFaceTracker

And if they need to go the commercial route, there’s FaceAPI
http://www.seeingmachines.com/product/faceapi/

As far as a practical applications, I could imagine it augmenting the way the computer understands us. I’ve been thinking a lot about this
recently. Your computer has a microphone to listen to you, an accelerometer to know when you drop it, a camera to watch you, an
ambient light sensor to know how bright the screen should be. I have to wonder if it makes sense to respond to our pose and facial
expressions.

That said, here are the few experiments I’ve seen so far:

http://jeffwinder.blogspot.com/2011/07/face-gestures-faceosc-and-flash.html

Ingredients above:

FaceOSC
+
RoboFab’s Glyph Math
robofab.org
+
Vanilla
code.typesupply.com
+
Ideal Sans
http://www.typography.com/fonts/font_overview.php?productLineID=100042

Want to talk at all about your approach to developing this — particularly as you’ve been teaching others?

I think everyone learns differently, but for me I learn by playing. So I try to make it easy for other people to play by providing interfaces like FaceOSC (or, with 3d scanning, via my structured light work).

Anything else musicians might want to know about your work?

I haven’t spent enough time recently making music, but I’m always thinking about things in musical terms. My older work has a lot of
musical interfaces and ideas scattered through it, if you dig through http://kylemcdonald.net you might find some inspiration there.

Thanks, Kyle! If you’re in NYC, as Kyle and I are — or, may be, when we’re not traveling to opposite ends of the globe — Kyle has a couple of recommendations. There’s a “no-more-than-monthly” Kinect meetup organized by Sean Kean:
http://www.meetup.com/volumetric/

Also, there’s an amazing “summer school” meetup on July 21. Wish I could be there myself, but I’ll send regards from Berlin. Hope one of the other New Yorkers can report back.
http://eyebeam.org/events/meetup-demo-day

For more on Kinect, we’ve got loads of coverage on Create Digital Motion:
http://createdigitalmotion.com/tag/kinect/

It is, after all, Motion!

Got a creation of your own, or a meetup in your area? Let us know!

Working in open source code for any platform, Brazilian artist and developer Jeraman has produced a charming project that imagines musical interfaces in dynamic, whimsically-simple sketches. Like doodled knobs, cartoons of guitar pedals, interactive devices on the touchscreen control musical activity.

And because it’s open, cross-platform code, everything from a computer-powered multitouch table to an Android tablet could get in on the fun. Jeraman explains:

It is a digital musical instrument that allows the control of real-time recorded loops through collaborative performances based on relationships between sketches and sounds, intended to be ludic and playful.

Developed in Processing and OpenFrameworks, it mixes multitouch technologies with the interaction metaphor of guitar pedals, by using a FTIR DIY multitouch table – built with PVC tubes, tapes… – with a modified old keyboard as pedal, following a high-end / low-tech approach.

Here’s an example of performance using an early prototype. By the way, sorry for the poor musical quality… I’m not a professional musician!

The project is also open source and the code, that may allow the instrument to be ported to others plataforms like Android, iPad and cardboard boxes (yes! using this http://sethsandler.com/multitouch/mtmini/), will be released soon.

This is possible due to the independence of the gui module (developed in processing) and the looper module (developed in openframeworks).
for the communication between both, it was used OSC protocol.

Some pictures can be found at:
http://www.flickr.com/photos/jeraman/sets/72157626935130739/

For further informations, check this site (sorry, in Portuguese):
http://jeraman.info/illusio/

It’s a fascinating-looking project! Do keep us posted, Jeraman — and readers, if you happen to play with the code, let us know! Oh, and never apologize for Portuguese – it’s a gorgeous language!

See also some nice examples of Jeraman’s previous work:

See, previously:
Robotic Twitter Songwriter Generates Tweet Poetry

Digital musician and artist Josh Silverman began the Synplode process with something familiar – a checkerboard. Play a game of checkers on its computer vision-equipped playing field and beats and loops triggered in Ableton Live generated a responsive soundtrack for the game. But as it’s evolved, Synplode has become a general-purpose musical grid. Whether with little robotic insects (the Hexbugs here) or full-sized human persons, the grid can turn any space into a dynamic, interactive dance floor. (I think I may actually prefer those cute little bugs to the people and dancers and whatnot. Robot rave, anyone?)

I prodded Josh to write up more description of what’s going on, so he’s created lots of documentation on the project Website.

The basic interaction:

At the start of the Synplode demo video, it is easy to see that a wave passes over the basic projected grid, flashing one column at a time, each containing 8 trigger regions. When a participant (or microbot) is present on a region, it is activated. When the wave intersects with an activated region, it causes a Synplosion, expressed through a splash of color and a distinctive sound. In the grid, each row represents a distinctive color and pitch or audio sample.

The basic ingredients:
1. Computer vision in OpenFrameworks, the fully open-source, artist-friendly C++ toolkit inspired by Processing.
2. Ableton Live, triggering clips in Set Mode and modulating them with MIDI effects and racks.

For more detail:
How it Works (details, in particular, of what’s happening in Ableton)
Why it Works (some of the thinking behind the interaction)
Synplode Project Page

Josh first demonstrated this system publicly at our Handmade Music series here in New York, and this is just the kind of experimentation and iteration I like to see. Here’s the original, checkerboard version:

Checkerboard Step Sequencer V2 from Josh Silverman on Vimeo.

Shall we play a game?

Have your checkers chops ready, because Josh Silverman’s Checkerboard Step Sequencer, a tangible interface for music, will test both your game mettle and your grooves.

Built with the open source coding tool OpenFrameworks and Ableton Live as sound source, the checkerboard fuses computer vision technology and … well, some beats.

This video should make obvious the relationship between the position of the checkers pieces and the noises they represent and trigger. It’s still a work in progress, but for now I won’t subject you to the cacophony that is the sound of an actual game of checkers.

Aside from the kick drum, which just keeps pace on every beat, all other drum samples are triggered off the board.

In this version, I’ve implemented a Mute Region that surrounds the board. When the application sees activity in the mute region, it disables the updating of the sequencer. This way, my hand doesn’t trigger a cacophony when I move the pieces.

More technical explanation on Josh’s blog:

How it works [prettyextreme]

As it happens, you can meet this project in person if you’re in the NYC area. We’ll be hosting Josh on Sunday night at Handmade Music, at Culturefix’s Lower East Side. 4:30-6p is an open lab, a chance to check out this project and others (including MeeBlip!), followed by cacophonous demos and raucous music starting at 7p.

Detailed Lineup; Handmade Music site
Event on Facebook

Historical precedent: It’s fairly hard to top John Cage and Marcel Duchamp playing chess, with or without sonification, but apparently sonified they were:
John Cage Playing Chess [Uncertain Times]

No information on what the chess game sounded like, however. Backgammon, anyone? Thanks, SkyRon, for the tip! Also, from 1997, a grid game with Toshio Iwai and Ryuichi Sakamoto; thanks, Ctrlsave. (Interesting to reflect on how much easier this is to do in 2010, thanks to more accessible software and greater, cheaper horsepower.)

Pictured: Looks native, but this app is built with a cross-platform library. And really, for music making – or great, immersive development, in general – does it matter?

The iPad has inflamed plenty of passions online. On this site, I’ve gotten a little flak from iPad lovers and haters alike. It goes something like this: “wait a minute, you’ve got all these criticisms of the iPad’s restrictiveness, but then you’ve got all these amazing music apps.” Or, on the other side: “why do you keep covering all these iPad music apps?”

In a word, yes. They’re not the same issue. I’ve talked to plenty of developers. The business draw on iPad is a big deal for independent, creative developers, so to the extent that Apple strategy makes the store a good place to sell apps, there’s some overlap. But the iPad has also been attracting plenty of music developers because of the quality of the APIs – developers who often aren’t pleased with the restrictions. Does the person who writes the audio drivers and APIs have anything to do with the lawyer who writes the developer agreement? Of course not.

The problem is, just as iPad/iPhone critics sometimes conflate issues in their rush to criticize the platform, some of the defense from the Mac community is getting a bit carried away, too. We’ve seen this with design issues, not just ideological or business issues: you go from “touch can be an expressive way to interact with a computer” to “throw out your QWERTY keyboards! They’re dead!” or “you’ll never read a magazine again” or “multitasking was a terrible idea in the first place.”

In this case, Apple made a fairly specific change to a developer document. That resulted in some criticism that was over the top (namely, people claimed it’d stop specific developer tools before they had verified whether that was actually the case). But it also resulted in some Apple apologism that was downright surreal:

All cross-platform development is bad? Wait – what?

And for that matter, is the mark of great software design now exclusively using Apple’s developer toolkits? Wouldn’t we sort of hope that, beyond those slick Apple UI widgets, someone somewhere might be developing the UI of the future? For that matter, do people not realize that a lot of what makes Apple’s quality exceptional is stuff you can’t see – things like multitouch firmware, high-quality audio drivers, and other fit-and-finish on the plumbing?

So, I invite you, dear reader, travel with me. I think we may actually have something on which iPad critics and fans alike can agree. It’s relevant to music, because music apps (along with games, incidentally) are the ones that are most intimate with this issue. And I suspect a lot of you use cross-platform tools to develop code for your day job.

The Catalyst: Apple’s Legal Change

Apple surprised many in the tech world last week by making a change – mandatory to all developers – that requires that applications for iPhone “must be originally written in Objective-C, C, C++ or JavaScript [running in the WebKit browser engine].” More specifically-worded, “Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited.” Because of the timing, and because of the further clarification, conventional wisdom suggests this is aimed at Adobe CS5′s tool for making native iPhone apps from Flash code. I don’t think it should be any surprise that that would get developers upset, not only those who use Flash, but even some loyal developers who don’t like being told what to do by Apple. (And I know at least some fairly big fans of the iPad weren’t fans of this change.)

Had it not coincided with Adobe working on CS5, I don’t know that this would have been big news; Apple already restricts the languages used to develop on their device. But I think what set people off may have been that very problem: people don’t know what it means, and that (rightfully) makes them nervous. While online debates have devolved into idealogical extremes “All control is good! / Apple just killed Adobe!”, what the press has missed is a sense among developers that they can’t predict or entirely interpret Apple’s developer agreements. I suspect Apple did aim this at Adobe, but that means even non-Flash-using, native-developing software makers now have to face some serious ambiguity in a legal document they have to sign.

That said, if Apple would further clarify the statement, that could be resolved.

I didn’t personally get worked up over this, because it’s consistent with what I and many, many others have been saying about the platform all along. Apple’s control over distribution and desire to control the development process means this is the sort of thing they can do. There are reasons to endure it: they make a really well-engineered platform, and there’s a terrific market and installed base that has a voracious appetite for creative software. There are also clear reasons to look elsewhere if you’re not comfortable with the restrictions. This is the very definition of trade-offs and choices.

I’m going to avoid the debate not because I don’t think it’s important – I think it is – but because I think enough words have been written in service to one side or the other. What seems to be missing, however, is a shared understanding of what cross-platform development actually is.

The Trend: “All Cross-Platform Development is Bad”

Whatever Apple’s thinking, it’s caused some people apologizing for Apple to say really weird things. John Gruber at Daring Fireball, for instance, begins by making an entirely reasonable argument for Apple’s strategy and where they live in the market. I don’t agree with all of it, but it is a well-reasoned, well-argued point. Then, almost as a footnote, John makes this claim:

Cross-platform software toolkits have never — ever — produced top-notch native apps for Apple platforms. Not for the classic Mac OS, not for Mac OS X, and not for iPhone OS. Such apps generally have been downright crummy.

[emphasis mine]

Why Apple Changed Section 3.3.1
[Daring Fireball]

That argument has gotten picked up all over the Web by other Mac fans. And that, to me, is dangerous – because, as worded, this statement appears to be to be entirely indefensible.

John doesn’t say “cross-platform compatibility layers” or “meta-platforms” like Flash and AIR. He says “cross-platform software toolkits,” and I think he means it. (Now, John, if I’m wrong, please correct me – but please stop making statements like this, because “cross-platform” is what many of your readers are coming away with.)

This would likely come as news to those who use music software. Cross-platform software frameworks are at the heart of most of the tools we use. One small but lovely example, specific to the iPhone/iPad and absolutely kosher under Apple’s new developer rules, is LibNUI, a C++ framework for building UIs. (In fact, after playing with this a bit, I may pick it up for a project on a completely different platform.) Popular iPhone apps like bleep!BOX and BeatMaker use it, but it also keeps tools like MOTU’s MachFive plug-in compatible with multiple platforms, without sacrificing native features like drag-and-drop.

If you use Ableton Live, Max/MSP, Cubase, or countless other apps, you’re using software created in cross-platform frameworks – some in-house, but using the same basic technology. Indeed, few of these applications would work the way you expected if they used exclusively “native” features and design patterns, like UI widgets that don’t fit musical applications or don’t work in live music performance.

In fact, John’s statement is so broad and over the top, I think it might even apply to tools like CodeWarrior, the developer tool and, yes, cross-platform framework that was the dominant toolset for developers in the pre-X “Classic” Mac OS era.

This matters to users, too. Sure, you may never write a line of code, but you rely on the community of people who do. Part of what gives you the freedom and flexibility to run great software on a variety of platforms, rather than being locked into just one platform, is the fact that these tools make the differences between those platforms fall into the background. Any developer who thinks this happens automatically without effort or testing is likely to give you a terrible app, but odds are, they’ll give you a terrible app regardless of what tools they’re using.

Develop Once, Run Anywhere?

Macworld editor Jason Snell also picks up the old argument about cross-platform development being inferior. (The title, I think, may be the most insightful part of this piece, but I’m not an Apple employee or investor, so I’ll let them worry about that.)

Apple against the world [Macworld]

…the develop-once-run-anywhere philosophy is something that makes more sense to bean counters and development-environment vendors than it does to platform owners and discriminating users. In the ’90s we were told that Java apps would be the future of software, because you could write them once and deploy them anywhere. As someone who used to use a Java-based Mac app on an almost daily basis, let me tell you: it was a disaster. Java apps didn’t behave like Mac apps. They were ugly and awful and weird, but hey, at least they ran on the Mac.

Ah, yes – this argument again. (It’s one of those things from the 90s that just never gets old, like Ace of Base or plaid t-shirts and pleated khakis.)

Okay, I kid, but Jason – I feel you. Actually, I feel you even as a fan of Java; the language and platform have some real power, but because of some questionable tooling atop them and questionable development practices with them, it produced some really horrible products. Such is development. (Actually, arguably, the folks in the 90s were right – it just turns out to be the browser itself, not Java applets, which have nothing to do with modern Java development anyway.)

I think Jason is mostly hung up on things like UI widgets; he refers specifically to the lack of a menu bar, odd preferences dialogs, and other usability issues in the AIR application TweetDeck. (Part of the reason we don’t nitpick these things in music, of course, is that we’re using extraordinarily complex interfaces for doing other things.)

Jason misses some critical points, however – in this case by omission; he doesn’t make the same, sweeping statement Gruber does. (Jason told me via Twitter that he wasn’t set to write another 2000 words, so Jason, I’ll try to do that for you.)

In regards to Java, the reason Java apps don’t feel like native Mac apps is at least in part because of Apple. It is actually possible to do all the things Jason is describing; Apple themselves touted the feature. You can read the documentation, and the fact that it was deprecated way back in 2005, on Apple’s legacy Mac developer documentation site. I can only speculate about the decision there, but my guess would be that it was practical more than strategic. There’s a new open source project to replace this functionality, Apple themselves recently made interfacing with native code easier for Java developers, and whatever language preferences Apple has on the iPhone, they continue to support projects like Ruby on the desktop Mac.

Generally, I think you’ll see more native feel in apps for Mac, Windows, and Linux from Java, Ruby, Python, and other languages. It’s an area of active development, and it’s improving. It may also benefit from these communities breaking off from big corporate parents, because the developers themselves seem to understand the perspective of the users better than, erm, companies like Sun and Oracle. Bottom line: don’t be surprised if some day soon you again run a Java app (or another language, not necessarily Java) and don’t notice. Those “discriminating users” on the Mac do notice when it’s wrong, and very often want to get it right.

Art, Tools, and Cross-Platform Frameworks That Don’t Suck (Or Break Apple Rules, Maybe)

But it’s not just about the standard Mac widgets. Jason, definitely check out Processing and the fantastic art made with it? It’s Java, though that doesn’t matter and isn’t immediately apparent, which is good.

If you design became only about widgets and preference bars, even nice Mac ones, we’d have a generic, bland, look-alike future for software. I know that escaping bland, cookie-cutter software is what drove a lot of people to the Mac in the first place, so it’s worth reiterating.

Tools like Java aside, though, somehow lost in this debate is the fact that cross-platform development is wildly popular and largely transparent – just in the language C/C++. From games to serious software, a whole lot of software is written in cross-platform C++, with the bulk of the code compiling on different operating systems and even hardware architectures. Developers typically make use of various frameworks to ease this compatibility.

Furthermore, while I still think there are reasons to be wary of Apple’s policies and this decision in particular, it would likely be inaccurate to claim that the recent change blocks these tools. In fact, several specific examples all use native code to link against the official Apple APIs, meaning they should be safe. These applications are exceptions that prove the rule: they’re great cross-platform tools that can produce great apps, they’re allowed on the iPhone/iPad OS as near as I can tell, and in some cases they’ll also be cranking out great apps for non-Apple platforms. Adobe’s big sin may have been allowing development from Windows, meaning you don’t get all those designers buying new MacBooks. Here are some examples of tools likely to be safe:

iPhone Wax uses Lua, but it still uses Xcode templates.

Corona, an awesome development tool for OpenGL-accelerated apps, has a specific response. Oh, and it’s coming to Android, too.

Unity is producing fantastic games and should likewise be safe under the new rules.

OpenFrameworks, a brilliant framework for artists that allows them to produce creative, interactive applications with music, visuals, and media for Windows, Mac, Linux, and platforms like iPhone/iPad is written entirely in C++ and appears to be okay. (Again, you use Xcode and Objective-C to link against official Apple APIs.)

Not incidentally, each of these tools (and LibNUI, above) could make some amazing music apps, some likely developed by readers of this site.

Let’s be clear: critics of Apple’s change likely overestimated how many frameworks would be impacted. That meant people were making an argument that may have been divorced from the facts. That said:

Just because they got the argument wrong doesn’t mean criticism (or defense) isn’t warranted. Apple did make a major change to the developer agreement, and they made it – apparently – as a reactionary response to a particular technology, in a way that could threaten other, unrelated technologies. The debate may have gotten overheated and inaccurate, but it’s understandable that the underlying cause is cause for concern. In fact, I think there’s no reason that Mac-centric media outlets couldn’t point that out. And developers really should consider leaving a platform if they don’t like it. (If it’s Apple’s right to make the rules, it’s certainly likewise the developer’s right to vote with his or her feet.) I think there’s an argument to be made in defense of Apple – I could certainly make that argument if someone dropped me on a debate team and put me on Apple’s side, even if I happen to disagree.

The jury is still out on just what apps are impacted – which should be further cause for concern. In fact, I’m still not entirely sure what the status of the apps above may be. On the blog /dev/why??:

In my opinion this is not purely aimed at Flash, but it is certainly precipitated by Flash CS5. I can’t imagine Apple is happy about environments like MonoTouch, Unity3D, PhoneGap, Appcelerator, or Corona, but I am doubtful they would have changed the license in this way just to stop developers using those environments …

He apparently thinks, however (though even the developers of Corona do not), that these frameworks could become verboten. Furthermore, he notes the case of “interpreted code” and why it’s important (it happens to be useful in music apps, too), though my understanding was that that was already a violation of the agreement. (Perhaps it’s clarified here.)

The more interesting thing from my standpoint is that this makes it a license violation to include a language interpreter inside a game. If you aren’t a game developer you might not be familiar with how large games are structured, but most games consist of a game engine, which is high performance code for doing things like rendering graphics, and an interpreter which runs the game logic (determining how sprites move, determining when to pop up in game text boxes, etc). This is how practically every commercial RPG works, as well as many (most?) other types of games. This affects major app store publishers, like EA, Gameloft, Tapulous, and ngmoco:). Looking at the top ten lists on the app store right now I see several titles that I know have embedded Lua interpreters. In this case I think these apps are genuine collateral damage, though I honestly doubt Apple would attempt to enforce the clause against them. In fact, using an interpreted language for game logic is already technically in violation of section 3.3.2 in the current agreement, though many developers may not realize it because under the original agreement it was okay, and the change that made it verboten was very subtle (changing an “and” to an “or”). I am actually not sure exactly when that changed, and only noticed it myself while I was researching this blog post.

See comments – ultimately, the language question is the big one. It could have a negative impact on developer flexibility, and specifically could impact DSP code. As Richard notes in comments, it’s all a matter of what Apple chooses to enforce. It’s possible that the letter of the law makes all of these things illegal, but in practice, Apple just wants to block Adobe’s tools.

But let’s also be clear:

Major voices in the Mac community are advocating against cross-platform software, even without a complete understanding of what that means. And you can actually defend Apple’s rule change Others (like Jason Snell at Macworld) I think just don’t get the opportunity to be clear. But let’s be clear. Let’s makes sure that idealogical discussions on both sides of this debate don’t obscure the facts.

Digging into Apple’s own, platform-proprietary tools can be a great thing. My friend vade, a sometimes-contributor on Create Digital Motion, has done great work with Quartz Composer, for instance, as an artist, and knows Core Image backwards and forwards because it allows him to express himself.

But that’s just one avenue. I know other developers who have found that working across multiple platforms ultimately makes their software better. Jason Snell unfairly, I think, characterizes this as “lowest-common denominator” development. If that’s what you’re doing, well, yeah, that would kind of suck. But I’d call this “highest-common denominator” development: the more you need to make code work on multiple platforms, the more, very often, you have to optimize all of the platforms, the more you discover opportunities to improve your code and make it a more general solution to a problem.

The truth is, you can use the cross-platform tools above to make fantastic iPhone/iPad apps, apps that feel entirely “native,” but apps that will also – by Jason’s description:

…[create] a world where App X for iPhone and App X for Android are indistinguishable from one another.

When it comes to games, to music apps, to creative applications with alternative interfaces, to immersive applications, to rich media interfaces – developers are creating that world, period. Apple can’t stop developers from doing that. Given that they tout availability of apps for their platform that were built with that model, I’m not even convinced Apple always has a problem with that development model.

The cross-platform world is here already, and it’s growing. And, honestly, I think it’ll be a good thing.

For further reading…

This post is really rhetorical, but it hilariously takes the legal clause to its logical (if not practical or likely) conclusion:
Apple bans modular programming

It’s amusing reading, but as author Jeff Erickson (“Ernie Pan”) responds to comments, the real bottom line comes out: “But it doesn’t matter what Apple means. The license is a legal document; the only thing that matters is what it actually says.” Of course, that leads to still more unpleasant revelations: it doesn’t matter what the document says or Apple means, but what Apple actually does. And Apple can change what it does at any time.

Tao Effect notes, as I do, that cross-platform toolkits can be made to look like native apps, or even that it may not matter what they look like (because as a game, or in my example of a music app, they all look different by necessity). It also responds to what I think we could now call the Jobs Doctrine:

“We’ve been there before, and intermediate layers between the platform and the developer ultimately produces sub-standard apps and hinders the progress of the platform.”

My issue: I”m not sure what Jobs means by the terms “intermediate” or “layers.” In fact, I’m not entirely certain what he means by “sub-standard.”

The blog notes that – while I suggest that maybe some of them are safe – things like MonoTouch, popular apps that feature Lua scripting in their development, and the widely-used Unity 3D game framework may well not be allowed in the store, which could mean more unpredictable rejections.

Ah, to be using a game console, where almost everything is rejected and you only have to worry about the few apps that make the cut…

Whatever the implications for the iPhone platform, though, these stories underly the point I’m really trying to make here – whatever Jobs may seem to be saying or Apple advocates are arguing, the notion that cross-platform development creates bad apps is one that is seriously open to debate.

In the meantime, we have several groups who don’t speak the same language or technical understanding:
1. Apple lawyers.
2. Apple end users / customers / advocates.
3. Developers.

And then we have Steve Jobs making sweeping, provocative generalizations that are themselves enigmatic, because he’s, well … Steve Jobs. (So make that category #4.)

Argos Interface Builder, v0.20 from Dimitri Diakopoulos on Vimeo.

You know the game: you decide you want exactly 8 knobs and 10 faders. But your hardware interface has 8 knobs and 8 faders. And then you realize you could use 4 more knobs.

The appeal of touch interfaces is clear: you get controls that grow and change. So now, a generation of mobile apps is working on giving you that flexibility on touch devices. The iPhone is just the start: now the iPad, with greater real estate, will go head to head with 5″, 8″, and laptop-sized screens running Android, Linux, and Windows.

Argos is an early-stages (but usable), free and open-source tool that could help you be ready. Built in openFrameworks, the C++-based cousin to Processing, the app lets you drag in basic widgets like buttons, sliders, toggles, and x-y pads, and assign them to OSC. That opens up control to various music and visual apps. (The OSC assignment tool does bear some similarity to that on the Lemur, though it’s simpler.) The openFrameworks roots should make this easier to port to multiple platforms.

http://argos.dimitridiakopoulos.com/

Developer Dimitri Diakopoulos, a BFA student at CalArts, is looking for developers and actively working on making this work on the iPad and its additional screen real estate – with other platforms possible, too. (If some of the PC “slate”s simply run Windows 7, you might be able to just switch the thing on, no port required — and run the app you’re controlling on the same machine if you so choose. We’ll have to wait to see what ships.) Stay tuned for more news on this, but this is well worth a look now. (Synthtopia was on top of the story earlier today.)

I’m personally interested to see if the protocol established by open iPhone app mrmr could allow over-the-air template sharing, and whether all these apps can interoperate with TUIO, the touch protocol developed for the reacTable. I said it earlier today, but there is some real potential in convergence, so I invite anyone who wishes to join that conversation. The trick is, you want to initially let people do their own thing, but then take all those “my own thing” solutions and put them together into an actual standard. If you try to impose the standard first, it might not actually work in the real world, but if you fail to standardize, you lose the advantage of interoperability. On the other hand, I think this very quandary is best solved by small groups of passionate developers, not overly-formalized process.

Part of the beauty of the iPhone from its launch date was the notion of a Mac you could fit in the palm of your hand. That makes it ironic that, for so many developers, mobile platforms in general have turned into a way to fragment software, to make it run fewer places instead of more. There’s something to be said for designing to a specific mobile device, but on the other hand, how many developers would want to restrict where their creations run? And particularly in music, isn’t the appeal of mobile creation the ability to have your tools work in more places? Maybe targeting just one gadget is the right choice for a given tool, but it shouldn’t be the only choice.

On the iPhone, the application Thump has plenty in common with a range of mobile music making tools. It’s simple but fun, a groove box with sequencing, subtractive synths, drums, and a set of basic effects, plus the ability to load your own samples and export songs.

Thump also demonstrates how simplifying sonic capabilities can produce musically-beautiful results, by focusing on the essentials and creating something with personality. Here’s a track by its creator showing off its sounds:
thump soundreel by mazbox

Well worth checking out the app on your platform of choice:
http://www.mrkbrz.com/thump/

What might not be immediately apparent is that under the hood, Thump makes use of the open source environment openFrameworks. As a result, the same code runs on iPhone, Mac, and Windows, as well as Linux. (It’s not distributed for Linux, but it could be. Hint, hint.) Creator Marek Bereza gives back, too – he ported the openFrameworks audio library to the iPhone, where it’s available to anyone.

Updated Marek notes in comments just what this means. The video below is, essentially, the same app. In place of the iPod screen, he has used a massive lattice of physical controls. A separate installation at the same show used a large touchscreen and simplified interface. And this really demonstrates what cross-platform means. Guy Kawasaki in the 80s mocked “ports” as a cheap wine. The idea is not to simply dump your code on a different platform and hope for the best – in fact, in this case, the changes from one platform to another were radical. The key is maximizing what’s essential, what really is not specific to a single device.

Physical Sequencer from Marek Bereza on Vimeo.

Creative Applications has a detailed write-up of the installation with more documentation.

If you’re interested in creating your own projects, oF has an elegant syntax based on Processing, but adapted to C/C++ coding paradigms and libraries.

I would recommend trying desktop development first as you’re learning, but when you do want to go mobile, setup is absurdly easy:

Setup instructions for iPhone

Having this as a foundation means more freedom for users and developers alike, the ability to run where you want, and a shared community of artists helping one another out. It does not mean sacrificing revenue, either – which is important, because revenue is often what enables (and encourages) developers to make new things. Here, we see a commercial mobile tool and distribution via the iTunes store, with free-as-in-beer distribution of desktop apps. But even the Free Software Foundation actually encourages charging for distribution – and to many people, the FSF is the most extreme example.

The bottom line is, you’re free to create what you want, where you want – and it doesn’t even have to have a price of free.

The use of openFrameworks is possible because the iPhone allows the execution of native code. Apple does not allow the Java virtual machine, which locks out many useful platforms and tools — including Processing. Naturally, that should not be the case with forthcoming non-Apple platforms based on Linux; more on that soon.

Thanks to Memo Akten for the tip — and for his exceptional documentation of iPhone openFrameworks development.

GPU-tiful and Cross-Platform

Updated Another framework I left out is the lovely libnui. The tagline says it all: “C++ API, multiplatform (Mac OS X, iPhone, Win32, Win64, Linux), 3D hardware accelerated GUI and more…”

On the iPhone, Intua’s elaborate Beatmaker software and David Wallin’s bleep!box have both been made with libnui. I want to single out bleep!box because of its UI – note how UI elements are simplified and scaled to the size of your fingertip. That’s something that, speaking of multiple platforms, could be equally important with touch hardware coming to Mac, Windows, and Linux platforms (some of it here already). I saw a huge fan of Windows tablets loving Propellerhead Reason on a tablet. He probably would be less enthused running Reason or Record on a touch interface; the UI elements are way too small.

bleep!box is also worth a look for the same reason as Thump: elegant UI, simple functions, and a lot of fun.

Pd, too

Pure Data, the open source patching cousin of Max/MSP, can also run on mobile devices. It’s currently powering the application RjDj, but because Pd is free, that could be just the beginning. In May, I wrote about the work being done by Hans-Christoph Steiner, who has been porting Pd even to ancient PDAs and iPods – including those, notably, without native floating-point capabilities (the kind of number crunching usually associated with sound).

Well, now Hans-Christoph is looking toward other, current-generation devices, too, kicking off with a workshop on making interactive sound on the iPhone using Pure Data. Here’s a taste:

OK, kids, you won’t believe how easy this is. Once you have Pure Data installed on your your iPhone or iPod-Touch, creating and uploading your own sound-processing and sound-generating patches is as trivial as copying a text file to your device! You’ll still need an Apple Developer License, but because Pd patches are treated as media files, they don’t have to go thru Apple’s elaborate code review — so, you can just dive right in, turning your phone into a pocket synth within minutes. In this 3-hour hands-on workshop, interactive sound whiz and Pure Data developer Hans-Christoph Steiner will lead you through all the steps.

Hans-Christoph will be talking Pd on the iPhone at a conference next month at Carnegie Mellon University in Pittsburgh. That will also include Memo Akten and Zach Gage talking about openFrameworks. And for nearly-entirely open source hardware, there’s a tantalizing-looking workshop combining Android with Arduino, too.

http://artandcode.ning.com/

There’s quite a lot more to say about this, but I hope this whets your appetite.

And, of course, riding the train to Pittsburgh or your nearest university or open source workshop, now you’ll have Thump which which to make some thumping tracks on your iPhone.

Ten minutes. Four or five kids (or adults). Make a song. Go.

That’s the idea behind the Youth Music Box, developed by Silent Studios and Chris O’Shea. (Our friend Chris you may recall from various interactive projects and the blog pixelsumo; he sends this project our way.) The software is build in openFrameworks, the C++-based creative coding environment for artists.

With keys, drums, and yes, even a scratching DJ-style interface, the music box brings together kids for quick music making, inspired by the phenomenon of musical games. The experience is guided by genre, with some effort to make sure whatever they do sounds good, but it’s extraordinary how effective it is at conveying the experience of the successful jam. It’s a bit of a confidence builder, in other words, for a group musical experience, perhaps more so than those ear-splitting, cheap plastic recorder consorts I recall from my youth.

And oh yeah, those kids look super cute once they get rocking out. (See video below.)

Youth Music Box Experience from Silent Studios | Resonate on Vimeo.

All of this raises some fascinating questions, and not always with the answers you might expect. In a normal musical ensemble, you begin sounding like crap, amp up difficulty, and eventually sound something like this – at least as far as coherence goes, assuming you’re not aiming for experimental free jazz. But with the addition of technology, whether musical games or the presets on our favorite synths or the quantization and beat-synced loops of our sequencers, it goes something in reverse. You start out sounding like this, pull apart the mechanisms that make you sound a certain way, and eventually find your way to your own personal approach. (And at some point, you get some of the readers on this site, writing code to produce their own sounds and musical structures line by line.) In fact, one could imagine scaling difficulty of even this particular setup, gradually adding greater musical freedom and taking away the “training wheels” of all the rules-based restrictions that make the results sound a particular way.

Skeptical about the connection of music-based games and actual music making? Think again – even as music education unravels worldwide, games are actually encouraging real music. That revelation was the impetus of the music box project:

Research commissioned by Youth Music found that up to 2.5 million young people in the UK – or 1 million aged between 12 and 18 – have been inspired to progress into ‘real’ music-making because they have played music-based console games.

You got it – they hit those plastic buttons, got inspired, got bored, then decided to go to the real thing. And otherwise, they might have remained passive musical consumers: the game was a gateway drug. Of course, that means that any such interactive experience has to stand up to polished Guitar Hero and Rock Band-style games. But anyone who believes the music games genre has peaked and is on its way out may be dead wrong on many, many levels. On the contrary, this may only be getting started – and the real growth could come in music beyond the realm of games, as people graduate to the unlimited set of possible music experiences.

Chris sends lots more documentation of this project, if you’d like to learn more:

by silent studios and me for uk charity youth music to get kids turned on to music
http://www.vimeo.com/6210259

watch some bbc coverage here
http://news.bbc.co.uk/cbbcnews/hi/newsid_8160000/newsid_8168800/8168881.stm
http://news.bbc.co.uk/1/hi/technology/8154449.stm

Ed.: The video at top doesn’t play outside the UK, because we don’t pay BBC license fees. What, all those Doctor Who videos I bought in the 80s and 90s didn’t make up for it?

here is a press release from roland. the box is ‘powered by roland’
http://www.audioprointernational.com/news/1329/Roland-unveils-Music-Box-for-Youth-Music

some launch pics
http://www.flickr.com/photos/pixelsumo/sets/72157621466657993/

making of pics
http://www.flickr.com/photos/pixelsumo/sets/72157621404410234/

this goes into some of the ideas and details about the musical kit
http://musicispower.youthmusic.org.uk/blog/24/youthmusicboxlaunchesatlondonssouthbankcentre/

on the website there is a very simplified flash version you can try out on a mini timeline, just click play online

its quite funny to read these comments on it
http://www.hypebot.com/hypebot/2009/07/youth-music-box-democratizes-music-creation.html

And yes, you can try this yourself and play online! The official site:

http://musicispower.youthmusic.org.uk/youth_music_box/

The production company:

http://www.silentstudios.co.uk/

And Chris’ own site:

http://www.chrisoshea.org/

Roland is involved, and donated an E-09 Interactive Music Arranger to give kids some toys to explore.

And yes, I did notice a certain kindred spirit in the form of Moldover’s Octamasher. The underlying technology and its results are different, but to me what’s most interesting isn’t the superficial similarity of these projects, but the fact that they array the instruments in a circle. Computer production often simply orients a single person to a screen – not so ideal for collaboration. And even Rock Band and Guitar Hero, like an onstage band, line up artists for a (now nonexitent) audience. Perhaps the circle is about to make a comeback as music restores its social aspect.

Curious to hear other thoughts on these projects as they evolve.