I’m not sure how everyone who owns an iPad uses it for music, but I find myself strangely drawn, more than anything else, to analog step sequencers. With MIDI connections – via a special interface or a standard USB MIDI interface connected via adapter to the tablet – you can even drive hardware. For me, the app of choice has been Little MIDI Machine. Developer Chris Randall has a new application in the analog-style sequencing category, though, called Phaedra.

If you haven’t grabbed it already, you have until the New Year to get it for US$4.99 before the price jumps to ten bucks. And you get an impressive array of features:

• Multiple buses, with 32 steps max each
• Programmable note, velocity, gate time, and two MIDI CC outs for each step
• Send or receive MIDI Clock for sync
• Use MIDI hardware (via Core MIDI), other apps (using “background MIDI” or OMAC), or your computer (networking via a MIDI Network Session

Phaedra for iPad [iTunes Store Link; you'll need iOS 5.0]

I wanted to know more about the creation of Phaedra. Developer and musician Chris Randall, known for his work with boutique plug-in maker Audio Damage, released this under a new moniker, Naughty Panther, which does iOS and MIDI development. Chris has been known to mix old and new, as with his musical use of the Apple II. Here, he gives us some insight into how he went through the design process on this new tool.

CDM: How do you sketch out the UI on a project like this? Paper and pencil?

Chris: Normally I just make notes in a little Field Notes notebook as I think about them — I carry one everywhere — then once I have a clear mental image of what I want to make, I just bust it straight out in Photoshop (or, for skeuomorphic interfaces like this one, a combination of Photoshop for the panel and 3D Studio Max for the knobs and buttons).

What inspiration did you consider as far as hardware?

Adam sent me the page for the Moon Modular 568 “Sequential Trigger Source” and quipped that it would make a fun iPad app, and I kind of ran with it. (Their site is a frame-a-thon, but ’tis here: http://www.lunar-experience.com/home.html) As you can see, the UI for Phaedra closely follows the Moon Modular design. The functionality diverges quite a bit, though. Once I had the initial look, I just started adding features I personally thought would be nice in a step sequencer. Then the beta testers had some more input. And now that it is out, I’m getting a ton of great feature requests from the users. The next update will have some cool tricks in it, provided I can figure out how to code them.

What hardware would you recommend for connecting MIDI?

The best MIDI I/O situation for Phaedra is an Alesis iO Dock. Hands down. I have tested it with a half dozen MIDI interfaces via the [Apple] Camera Connection Kit [for connecting driverless USB MIDI interfaces], and they all work fine except that $5 cheap one that people buy from Amazon, which unsurprisingly, has problems. Phaedra also automatically creates a virtual port and connects to all virtual endpoints running on the iPad, so it can drive NLog Pro, Animoog, whatever. And it also is [Apple MIDI] Network Session aware, so it can just work over wi-fi.

I’m going to add OSC output as an option in the next update, I think. I’m still thinking about how to implement that, but it seems like it would be handy, and then Phaedra’s abilities would be greatly increased.

I’m also adding trigger outputs via the audio ports, so you can use Phaedra to clock an external modular sequencer like Makenoise René, or a pre-MIDI drum machine. I’m giving it 48-ppq clock for Korg drum machines, 24-ppq clock for most every other drum machine, and arbitrary rates from 16-ppq on up to 1-ppq for driving modular sequencers and your Monotribe. This should be handy, and will be sample-accurate in line with the MIDI clock output.

See also Chris’ blog entry on the app at Analog Industries:
Some Cool Shit

Dig into humanity’s past, and alongside the earliest tools, you’ll find some of the earliest instruments. Designing objects for expression seems to be an essential part of civilization.

Martin Kaltenbrunner, a co-designer of the Reactable tangible music interface, is also a professor in Interface Culture at the Linz University of Arts in Austria. There, in the land of Mozart and Haydn, he works with students to explore what interface design is.

So, when I got to spend some time with Martin in New York in September, I was interested in more than just the flashy coolness of the Reactable, the futuristic table-with-blocks interface for music. We got a chance to talk about instrument design generally. The funny thing about the Reactable is that it is closer to the experience of working with a modular synthesizer and oscilloscope than anything else, with the sense of physical connections of sound to object you’d get from classic synths. It is something unique, truly, but that’s its pedigree.

Martin and I got to give a talk together at the Austrian Cultural Forum New York, a terrific hub in which Austrian artists frequently are paired with New York-based folks, all in a lean, tall modern landmark building in Midtown. We also performed together, which for me was a real pleasure; Martin claims not to be a musician as such, but was good fun as an improvisation partner.

The next day, we headed to Manhattan music education center Dubspot, where Martin’s creation quickly attracted crowds of interested students and educators. Dubspot filmed our encounter for the video at top. Amusingly, the prominent synth sounds you hear at the beginning are not the Reactable, but our own MeeBlip open source synth, which I brought along to illustrate conventional tangible instrument design with switches and knobs. (If you’ve been impatiently waiting for news on the MeeBlip, believe me, I’m even more impatient – more announcements on that this week and next, following a production quality issue with a contractor that required us to reboot the run of new instruments.)

Reactable Live @ Dubspot! Interactive Sound Design Workshop Video Recap [Dubspot Blog]

YouTube commenters, that subtle and thoughtful bunch, are complaining that the tangible Reactable will set you back thousands of Euros. But at ACF and Dubspot, I was also equipped with the far more economical and portable alternative: Reactable Mobile runs on both Android and iOS. (I was pleasantly surprised to discover the app runs perfectly on a Galaxy Tab 10.1 from Samsung; I’m still a long, long way from being able to recommend buying an Android tablet, but if you’ve got one, I can certainly recommend this app.) Now, don’t get me wrong: the experience was nowhere near as fun as using the table. On the other hand, you can’t fit the table into a seat-back pocket on easyJet, and the savings in cash is proportional to the sacrifice in experience. What impresses me is that the design of the physical Reactable “flattens” so nicely onto the screen; I think it’s a user experience triumph that you can make that translation. And I was able to load up a few loops of my own music and jam along with the MeeBlip and Martin on the (real) Reactable.

Looking beyond the Reactable, Martin addressed these larger issues of tangible interface design at TEDx Vienna – a fitting locale for talking the history and future of music. His whole presentation, and a sweeping concept map of what he discussed, is available.

Martin Kaltenbrunner: Tangible Music

If you just want to get your Reactable on and can’t afford the table, see Reactable Mobile below. (Seen here on an iPad 2, but I’ve run successfully on the original iPad and the Galaxy Tab – the experience is more or less identical, thanks to portable code.)

And if you can afford the table, you rockstar, uh, can we be your friend?

http://www.reactable.com/products/mobile/
[the awesome table version]
Martin Kaltenbrunner website [with plenty of academic links]

Compare the complex model of what a computer can use to control sound and musical pattern in real-time to the visualization. You see knobs, you see faders that resemble mixers, you see grids, you see – bizarrely – representations of old piano rolls. The accumulated ephemera of old hardware, while useful, can be quickly overwhelmed by a complex musical creation, or visually can fail to show the musical ideas that form a larger piece. You can employ notation, derived originally from instructions for plainsong chant and scrawled for individual musicians – and quickly discover how inadequate it is for the language of sound shaping in the computer.

Or, you can enter a wild, three-dimensional world of exploded geometries, navigated with hand gestures.

Welcome to the sci fi-made-real universe of Portland-based Christian Bannister’s subcycle. Combining sophisticated, beautiful visualizations, elegant mode shifts that move from timbre to musical pattern, and two-dimensional and three-dimensional interactions, it’s a complete visualization and interface for live re-composition. A hand gesture can step from one musical section to another, or copy a pattern. Some familiar idioms are here: the grid of notes, a la piano roll, and the light-up array of buttons of the monome. But other ideas are exploded into spatial geometry, so that you can fly through a sound or make a sweeping rectangle or circle represent a filter.

Ingredients, coupling free and open source software with familiar, musician-friendly tools:

• Two projectors
• A monome
• Processing, the elegant and artist-savvy free software for visual code
• Ableton Live and Cycling ’74′s Max for Live, acting as the interactive glue with the sound world
• Drumaxx, Image-Line’s tasty physical-modeled drum synth
• Native Instruments Battery, the sampled drum engine
• Eclipse, the free IDE, for Java coding in this case
• Community Core Vision and reacTIVision (based on our previous info, at least), free and open source community-based projects for making the interfaces you see in movies happen in real life.

Another terrific video, which gets into generating a pattern:

Now, I could say more, but perhaps it’s best to watch the videos. Normally, when you see a demo video with 10 or 11 minutes on the timeline, you might tune out. Here, I predict you’ll be too busy trying to get your jaw off the floor to skip ahead in the timeline.

At the same time, to me this kind of visualization of music opens a very, very wide door to new audiovisual exploration. Christian’s eye-popping work is the result of countless decisions – which visualization to use, which sound to use, which interaction to devise, which combination of interfaces, of instruments – and, most importantly, what kind of music. Any one of those decisions represents a branch that could lead elsewhere. If I’m right – and I dearly hope I am – we’re seeing the first future echoes of a vast, expanding audiovisual universe yet unseen.

Previously:
Subcycle: Multitouch Sound Crunching with Gestures, 3D Waveforms

And lots more info on the blog for the project:
http://www.subcycle.org/

Think back to playing a simply childhood game like Musical Chairs. The actual gameplay depends only on auditory clues – something you take for granted as a kid, but something apparently lost on game engineers who insist exclusively on advanced 3D rendering engines for visuals. And because you get your body involved, the game becomes dynamic. That musical cue isn’t just off in the background: in the dizzying run around the chairs, the soundtrack can become the singular focus of your brain, an urgent score to the — DIVE, got the chair!

As the scene around game experimentation grows richer, there’s a rekindled interest in how game mechanics can play to different senses. In some cases, it can be a source of whimsy; in others, it’s the only way to design games for people who are absent one of those senses. And an ongoing exploration of music and sound as gameplay mechanic – not just gameplay accompaniment – ought to interest composers and sound designers. When you look at a conventional arcade game, tuning your reflexes to the graphics is key, even if sounds provide reward and ambience. In these games, the sound is where the play is.

Johann Sebastian Joust has a lot in common with Musical Chairs. The game input is the lovely Sony PlayStation Move motion controller, which – yep, you guessed it, is where the jousting comes in. (An earlier version used the Wiimote.) But in place of graphics, listening to the music itself tells you when to act, just as in the childhood game:

When the music plays in slow-motion, the controllers are extremely sensitive to changes in acceleration. When the music speeds up for, this threshold becomes less strict, giving the players a small window to dash at their opponents. If the player’s controller is ever moved beyond the allowable threshold, that player loses.

Little wonder that the game resembles some of those kids’ games: the designers reveal that they got the idea after improvising “folk” games with friends. Now, there is some concession to adding additional feedback – the controllers use the light-up ball on the end and rumble feedback just to make absolutely clear what’s going on; some “sound games” are more pure in their all-sonic interface. But the idea remains the same.

The game is the work of the Copenhagen Game Collective. They describe themselves as “multi-gender, multi-national, non-profit”; I would add to that “blazing hot stuff.” CGC’s games have earned some serious accolades; for one, B.U.T.T.O.N., a group party game, was the runaway hit of the Kokoromi GAMMA party in 2010, and also showed up wowing crowds again at the same Kill Screen / Museum of Modern Art Show at which we saw Pxl Pusher, covered yesterday. (CDM and myself were also involved in that Gamma party, and co-organized a one-button art show at San Francisco’s GAFFTA art space.) But the group has in no small sense put Copenhagen on the map.

The team for this title:
Douglas Wilson: concept, programming, and video
Nils Deneken: graphics and announcer voice
Nicklas “Nifflas” Nygren: music and sound

Composer Nicklas Nygren is a triple threat: game designer, coder, and composer. Check out some of his music on SoundCloud:

Latest tracks by Nifflas

There’s plenty more to say about sound games and music games and interactive music for games. I gave a presentation at the Indie Gaming Showcase in Berlin on the topic at an event hosted by arts network Platoon – see their write-up. I’ll pull those notes together; if you have any nominees of game work you’d like to see covered, let us know in comments.

But for now, I’ll leave you with the image of Johann Sebastian Joust and Musical Chairs. After all, composition and ensemble playing themselves can be seen as games with musical mechanics. They certainly can even have “win” and “fail” mechanics – ask your local orchestra player.

http://www.copenhagengamecollective.org/johann-sebastian-joust/

What happens when you meld the most futuristic Microsoft technology with the most futuristic Apple technology with the most ColecoVision-esque graphics as built in Jitter? Or you create gameplay that couples physical human contortion with the step sequencing rhythms of music? A different take on music games, that’s what.

Developers Matt (“M@tt”) Boch and Ryan Challinor work, in their day jobs, on the music game as most people know it, at Harmonix. Harmonix’s roots remain in the rhythm game, so that music play, even at its most serious, is still about musical timing accuracy. Pxl Pusher is a very different mechanic: imagine a step sequencer grid on an iPad, presenting blocks that, true to the classic game Twister, require another player to balance and stretch their bodies to match.

I caught up with Matt and Ryan over the summer at New York’s Museum of Modern Art. Before throngs of crowds swarmed the game – easily one of the most popular of the night – I managed to get some quick footage of the creators doing last-minute patching and trying out gameplay. (The quiet there is atypical; we got to shoot before the doors opened.) An insane travel schedule kept me from publishing sooner, but here, Matt and Ryan share their process.

Another interesting twist: Max/MSP and Jitter allowed extremely rapid prototyping with Kinect, something of interest to anyone doing this sort of work. (And dig those “3D” images … if they don’t blind you.)

CDM: How did you work together on this project?

Matt: Jamin from Kill Screen had asked me to make a project for the Pop Rally, and I was looking for someone to team up with to make it happen. Around the same time, Ryan was developing Synapse, and I got super excited about that tech. Having worked with him closely on Dance Central 2, I was positive we’d make a killer team, so I proposed we team up and make it happen. He got the ball rolling immediately.

Max/MSP is a pretty ideal environment for the both of us to work in, as it lets us evaluate ideas really quickly, and completely obliterate the lines between disciplines that we’re used to in traditional game development. Each of us designed, coded, sound designed, and made art for the game. It was awesome to be able to share in the creation of all parts of the game.

Ryan: We riffed on ideas with each other until it snowballed into the final product. Max allowed us both to work in the same space, as opposed to the traditional programmer/designer arrangement, where I would work in code and Matt would tweak values that I expose. This allowed us to work much faster and blurred the line between programmer and designer.

Any roadblocks, discoveries along the way?

Ryan: The whole project was driven by discovery: discover a fun interaction, then figure out how to leverage it. There were no real roadblocks, other than Matt being on painkillers for the entirety of the project!

Matt: I fell, knocked out four teeth, and fractured my jaw, so I was on heavy doses of painkillers for much of the development. I’m doing much better now, almost fully recovered.

The greatest discovery for me was more of a rediscovery. I remembered how freeing it is to design a game for a very specific context. Before working at Harmonix, I made arcade games in a contemporary art context. Getting back into that mindset was a blast.

Tell us a bit how you work with Max for rapid prototyping. How did you set it up? Any specific tools you made use of? Any tips, either related to Max or Kinect specifically?

Ryan: No real tips for Max, other than do a lot of commenting when you’re working on a file with someone else. As far as Kinect tips, my main advice would be to explore and discover what fun you can have with he input, other than starting with a specific goal. The only tools we used outside of Max were Synapse for the Kinect data, Adobe Premiere for the background movie, and some web app Matt used to generate some of the sound effects.

Matt: We ended up using Beanstalkapp & Versions for version control, which was huge for us in the late stages of development when we both needed to be working simultaneously. It was great that we structured our patch in such a way that we could easily turn elements of the patch into standalone patches, so simultaneous work could happen. You lose some flexibility, but it’s worth it to be able to work in tandem.

How did you conceive the idea for this game?

Matt: Largely through iteration and idea exchange. I VJ around Boston a fair amount and often used TouchOSC to VJ with my iPhone, so I’m not stuck behind my laptop for hours at a time, so when Ryan brought up the idea of using TouchOSC with MAX and Synapse, I was totally sold. We quickly arrived at the mashup between Twister and Step Sequencer, and every after that was polishing mechanics, arriving at a unified aesthetic, and then finding the best way to ramp difficulty.

Ryan: Matt approached me to work on a game for the PopRally right around when I just finished up Synapse, so I wanted to leverage that in the project. I had also just learned about TouchOSC, so I thought it could be fun to incorporate that. So, the initial concept was born out of experimenting with interactions that combined the Kinect and iPad.

How did things go at MOMA? Any surprises in terms of reception? It was effectively rapid playtesting — anything that you would now incorporate into the game? Or things that worked well, for that matter?

Matt: It was a phenomenal success. I never imagined we’d have a line of people waiting to play the game all night, never mind a line full of people cheering, laughing, and clapping as they watched others play. We’ve talked a bit about ways to modify the existing game to make it a bit deeper and more broadly compatible, but I’m also wary of upsetting the balance we arrived at.

Ryan: I was pleased with the reception at MoMA, it was very satisfying to work on a project and then immediately have people playing it, literally 10 minutes after it was finished (thanks to a last-minute feature request from Matt’s friend Christina)

What does it mean to you to have this game in this landmark art museum?

Ryan: I’m really grateful that we were able to show it off in such an amazing space. It feels a little unreal to me, my brain hasn’t really processed it yet.

Matt: It was hugely validating, especially because my Dad was in town and was able to make it to the event. He’s never seen crowds of people play the games I’ve worked on so to have him there, seeing how much fun everyone was having, was awesome in and of itself. That it was at the MoMA? Still haven’t totally processed it. It was the stuff of dreams when I was getting my art degree; it’s going to take a bit for me to internalize that that dream was realized. Not only was I able to show work at the MoMA, but it was shown in the context of so many great games: Bit.Trip Beat, Limbo, B.U.T.T.O.N., QWOP, the list goes on…

How about the connection to Kill Screen Magazine?

Matt: I think Kill Screen is far and away the best game writing happening right now. Both their daily updates on the web and their themed print issues are staples in my life. To be fair, I might be biased as I’m good friends with [editor-in-chief] Jamin, but nevertheless, I think what they’ve accomplished is undeniably phenomenal. What other publication could convince the MoMA to have a video game night?

Can you tell us a bit about your roles at Harmonix, and how you came to be there?

Ryan: I’m a programmer at Harmonix, I’ve worked on The Beatles: Rock Band and the Dance Central series. I was a CS major at the University of Texas, I got an internship at a game studio in Austin that turned into a job, then a few years later a coworker talked me up to someone he met from Harmonix, and I got recruited to work there.

Matt: I am a Project Lead at Harmonix, though I’ve occupied many roles previously, most recently Senior Designer. I started at Harmonix as a Production Assistant while I was finishing my thesis at Harvard. My friend and VJ partner, Josh Randall, is the Creative Director of Harmonix. We’d been VJing together for a bit while I was in school. He gave me a call one day, suggesting I apply for a Production Assistant role. One of my jobs was to track all the prototype hardware, but I ended up spending most of my time repairing it, as my art practice of building arcades gave me a fair amount of experience about mechanical engineering, electronic engineering, and coding. That landed me a role as hardware designer, and, many Rock Band iterations and trips to China later, I was tasked with investigating tech for a potential dance game. That lead to me heading up the development of a game prototype that would become Dance Central, and I’ve been designing for the franchise ever since. A circuitous path for sure, but I’m really happy with where I ended up, and deeply grateful to Harmonix for allowing me to occupy such varied roles throughout my 4 years tenure.

Synapse is something other folks can use, too, correct? What does it do, and where might they begin?

Ryan: Yep, Synapse is a freely released toolset. The primary focus of the project was for controlling Ableton Live with Kinect, but it can be used with anything that receives OSC input. You can find more information at http://synapsekinect.tumblr.com/

Any future plans for this game, or other Synapse-powered stuff?

Ryan: In the immediate future, we’ll be showing off PXL PUSHR at Indiecade in LA on October 8th. Beyond that, I’m not sure what the future is. At the very least, we’re going to keep the ideas of the gameplay mechanic in mind to be incorporated into future projects. As far as Synapse, I’m currently exploring other ideas so I don’t have any Synapse stuff in the works at the moment, but I know of a few other people working on Synapse-powered projects currently.

Matt: As Ryan stated, we’re showing it at Indiecade, and are hoping we’ll get more opportunities to show the game publicly. I deeply enjoyed working on the game with Ryan and I hope that TeamPXL, the moniker we’ve adopted, will develop additional games with the same working methodology. For now, given that we both just wrapped development on Dance Central 2, we’re taking a much needed break to recharge. I know Ryan’s got some awesome audio manipulation patches in the works, and I’m excited to see what comes of those. In time, I’m sure an awesome idea for a game will cross one of our minds and we’ll have no choice but to dig in and make something new!

http://teampxl.tumblr.com/

Even from the birds-eye view of larger genres, the interrelations and ongoing transformation of music is dynamic, complex, and inter-connected. That’s the view in The Evolution of Western Dance Music, a map of musical styles in five-year chunks across the 19th and 20th Centuries, through Africa, Europe, the Americas, and Asia. The project is the work of London/Seattle/New York Web agency Distilled, pulling genre births from Bass Culture, Last Night A DJ Saved My Life,The All Music Guide to Electronica, and Wikipedia.

Having just edited a book entitled The Evolution of Electronic Dance Music, I find it extremely interesting to watch in this visualization the way in which European synth pop and Jamaican dub can become, at once, vessels for a lot of these other musical idioms, just in terms of their ability to carry musical ideas across geography.

What is peculiar: this is more a selection of a few threads than it is any kind of comprehensive history, and many of those threads in turn trace backwards from a few modern styles more than they do forwards over those 200 years. If you accept that, though, there’s still something interesting to watch. Even hand-picking a few genres shows some fascinating connections.

But before I say any more, I think any methodology here will raise questions, and I’m as interested in reader questions as I am commenting myself. Mark Johnstone of Distilled has offered to answer questions, so from the intricacies of how the data visualization and mapping work to thoughts on how one untangles this musical history, I’d love to start a conversation.

Specifics of the genres aside, I think it’s the geographical connections that are in many ways the most interesting – all the more so as we can inexpensively get on trains and planes, cross increasingly-open borders (with some admitted major caveats), and be somewhere altogether different – or do the same from the comfort of our chair. Appropriately, I now see Thomson are a travel/vacation agency.

Discuss.

How Music Travels – The Evolution of Western Dance Music [Thomson blog]

Interactive Music Map [Thomson]

On the road from futuristic instrumental concept to real-world product, the Yamaha Tenori-On as shipped lacked some of the functionality its creator, gifted media artist Toshio Iwai, originally imagined. Notably, wireless networking, which promised social music-making with other devices, was gone, replaced with a more-limited MIDI connector.

Now, in a surprisingly literal translation from the hardware to iPad, it appears the Tenori-On has added that feature – but lost some of its charm. An iOS developer notes to me that pitches don’t sound when you tap the screen, only when they are played in the sequence. That fundamentally changes the interaction with the sequencer: you can’t hear notes until they’re sequenced, and you would presumably lose the sense of playing an instrument. That report is happily incorrect; both the developer and I were mistaken from our video impressions. That makes this far more useful.

My reaction here should be taken with a grain of salt – this is only a demo video. But in observing what is new (networked features look terrific), it’s likewise worth saying that something is lost when you move to tangible hardware. To me, a lot of the appeal of the Tenori-On was tangible: the machined metal case, with curved edges designed to be comfortable to hold, and the feeling of running your fingers against discrete, round keys on the array of buttons. Those are lost by necessity. Yet, oddly, some of the Tenori-On’s features designed primarily for hardware – the menu system and navigation keys – are reproduced here, features necessary on a hardware design but not a tablet.

Yamaha Japan, apologies for going on a rant on a product I haven’t yet used, but I’m concerned at what seems to be a missed opportunity. And designer Toshio Iwai has already conceived imaginative touch-based interfaces that are designed for a screen, in works before iOS had even been announced, like ElectroPlankton for the Nintendo DS and interactive installation work going back some 15 years or so.

Simply translating hardware designs to a screen is novel, but rarely usable. Just ask Tascam, who were roundly (and rightly) criticized for making a Portastudio app for iPad that required you rewind every single time.

At least the good news is, some of the musical personality of Toshio Iwai’s work remains, and in a form that doesn’t require a costly hardware investment. Updated – also, via readers, there’s evidence of MIDI support.You’ll find other videos on Yamaha’s official Japanese channel.

Just mark my words: the hardware is still cooler, and there’s a lot of potential in hardware and software sequencers alike beyond this yet to be realized, whether by Yamaha or by someone else.

Updated: I want to re-emphasize that there appears to be auditory feedback as you press buttons for sequences, which is great news and vastly improves usability. And while I stand by some of what’s advantageous in hardware, I’m excited to learn that we may get both networked and MIDI functions here, as we’ve seen in apps from makers like KORG.

Reader comments are very positive, so amidst this hopefully constructive criticism, I think it’s encouraging that the software looks promising and people are eager to try it! (And being critical of some features does not mean you can’t eventually like the product – part of why I tend not to shy away from criticism.)

http://www.youtube.com/user/yamahajp

Saturday afternoon in Toronto, I’m giving a talk to the North by Northeast festival on music software and tablets. I’ll explain a bit about what tablets are about, and some of the software that’s out there on the landscape (principally, of course, on the iPad). But I hope to emphasize a deeper issue: how you design software for the tablet, and what’s unique about this convergence of form factor and touch interface. I mean this generically for a reason: on CDM, we covered some of these ideas before even the announcement of the iPhone, and I was an early (and skeptical, I might add) reviewer of the JazzMutant Lemur.

Even looking beyond that, I hope to talk a bit about how representing music graphically has been an essential part of human practice, not only beyond the iPad, but beyond even the current notational system as derived from the Western church. Talk about early tablets: the first known music notation appeared in ancient stone Greek and Byzantine tablets. (On weight and thinness, I don’t think they compete with the iPad.)

That sounds lofty, especially for a potentially-hungover crowd of musicians and designers on a Saturday, so here’s the executive summary: you don’t have to make a bunch of fake knobs.

I’m really mostly curious to start a conversation about design; ideally, I’ll get some designers showing up here in Toronto, but it’s time to make that conversation happen on the Web, too.

With that in mind, I’m curious:

What software designs – iPad or otherwise – have you seen that have most inspired you, in terms of the way the interface was designed?

Fair game: sound toys, music notation (really), art pieces, games, control surfaces … whatever you like.

I’ll post notes from my presentation by early next week, because I’ll probably be assembling it at the last minute it’s already totally done and perfect and rehearsed and I just wouldn’t want to spoil it.

Pictured: Loopseque; previously on CDM

Also, because I’m a huge fanboy of circles in general (as readers of this site know), I love this image and blog post from Loopseque. They didn’t exactly invent the idea of visualizing loops as circles, but let’s join this revolution.
Another step in the evolution of music interface [Loopseque Blog]

Honestly, if tablets are nothing other than an excuse to ask these questions again, all the better – and there’s no reason not to then apply what you’ve learned to computers, embedded hardware, analog hardware, paper notation – anything.

If anyone would like to start a circles versus rectangles fanboy platform war, troll away! I’ll start:

stupd circle &*(&$s you losers got not edges. serious muzos have right angles. go play with your dumba** frisbee shaped toys that dont have even no sides on them and see if you can even figure out PI LOLZ pie like something youd eat its not even a rational number whatevss
real pros use polygons

whaaaa??? ow did someone just hurt on their foursided pointy pointy pointy edge? shoulda used a circle, youd be happier r4d1us 4 l1f3

Seriously, definitely let me know what new interfaces you’ve found inspiring lately, and I’ll be sure to credit you in my talk!

On tablets, on displays, multi-touch control these days is calibrated largely as a software interface – more Starship Enterprise panel than violin. As such, it works well for production tools and exploring compositional ideas. But it falls far short of being an instrument: even on the much-hyped iPad, touch timing and sensitivity is too imprecise, and the absence of tactile feedback and real, kinetic resistance makes you feel like an operator rather than a musician.

Several projects in experimental instrument research seek to change that. But of all of them, the one that has generated the most enthusiasm is Randy Jones’ Soundplane, co-developed with hardware designer Brian Willoughby. CDM shares a conversation today with Randy about his brilliant Aalto synth, and I’m working on a review soon. But wonderful as Aalto is, many of us are still eager to hear more of the Soundplane controller. I chose to wax poetic and optimistic back in December of 2008:
Intimate Control: Multi-Touch, New Models, and What 2009 is Really About

I shouldn’t have put a year on my predictions, though – good things take time. (If I could clearly recall what happened in 2009, maybe my general prediction was correct. The past tends to blur together for me into a continuum in the manner of the modern technologist, a vague assemblage of stuff that happened in the 60s with things that are actually still in the future.)

The good news: Randy continues working on the Soundplane, and Aalto will help.

Continuing our interview, here are the thoughts most relevant to Soundplane — and a glimpse of what it’s looking like as he works on it in the lab.

First, Randy explains his ideas about running a small business, continuing what he had to say in our Aalto story. The basic idea: Aalto’s software will bootstrap Soundplane’s hardware.

I think the whole idea of venture capital is sort of a poisonous one. It’s a little like bands wanting to get signed right away. The first thing you want to focus on is giving up your autonomy, really?

Instead, why not scrape together whatever you can from friends or family and just make something that you can sell right away, however small. I didn’t have enough saved to finish the Soundplane project so halfway through I switched to putting out Aalto as a plan B for paying the rent. Now it’s out and it’s a product I’m proud of that I think reflects where we’re coming from, and it’s going to fund Soundplane development, and it’s letting tons of people know we exist. Just get a foot in the door, do something useful.

He also shares his feelings about patents:

Some people won’t like to hear this, but I applied for a patent on the sensor used in the Soundplane. I know, the patent system is totally broken, and often, if not usually, used in stupid ways. But if there’s one thing I think it is actually good for, it’s to protect small companies like ours that innovate against a bigger entity simply stealing their R&D. This is why it was designed, right? I don’t know if our patent will save the day if such a thing ever happens, but if it does I’d much rather have one than not. It’s a pain to write one but it’s not impossible, you just need a lot of patience. “Patent it Yourself“, Nolo Press, is a good reference.

The patent question raises some additional questions for me – in fact, I’d love to see open source hardware that’s also backed by patent protection, in the same way that the GPL license is made tenable largely through the existence of traditional copyright laws.

But I do tend to agree that in the case of a truly novel technology, which this is, patent protection may be necessary. The question for projects like this will be whether to operate as a conventional, patent-protected design, or whether some sort of open source model with a patent covenant and a copyleft license like GPL will make sense — both preventing exploitation and allowing free experimentation. If there are any IP lawyers lurking around out there, let us know (I have some contacts, too); and definitely let us know if that’s a conversation you’d like us to continue.

In the meantime, the important thing is that Soundplane lives, and using Aalto could help it come to fruition. We’ll absolutely keep you posted.

As proof, though, more shots from the lab:

Also, must-read article from shortly after Jones’ NIME presentation:
Why Soundplane?

The whole article is worth reading, but Jones argues that not only is it likely many people will try to do tactile multi-touch, but it may be necessary. For those of you not all that good at hardware design, you could be just as essential as well to there being any future for these curiosities. The designers need other designers. The hardware needs software creators – lots of them. The software creators need to try lots of ideas. And everybody needs players, composers … users.

It’s all-too-tempting to sit back on the Web and marvel at what everyone else is doing, to take their genius and novelty as an engraved invitation to give up on your own work. “It’s been done before.” “Someone else is already doing this.” It’s probably a topic for a dedicated article, but it’s simply the wrong reaction. “It’s been done before — maybe it’s worth doing. Or doing again. Or doing better. Or doing over and over again.” “Other people are doing this — that means I have someone else to do it with.”

Historically, revolutions aren’t solo pieces. They’re ensembles.

Updated: speaking of work being ensembles, while Randy’s name is most associated with the Soundplane project, credit is due to hardware designer Brian Willoughby, who did the hardware design for the instrument. As he wrote in comments on CDM in 2010, when we covered Roger Linn’s Linnstrument: “For my part, I’ve been deep into the process of designing the analog circuits, DSP hardware and firmware necessary for the product, so it’s nice to poke my head up for a moment and see interest on this site, as well as to hear about other engineers trying new things and inspiring ideas.”

The gaming industry has made their bet, and it’s that touchscreens go better with tactile controls. Might digital musicians reach the same conclusion?

A funny thing has happened on the way to the touch era. The vision of a device like the iPad is minimalist to the extreme: an uninterrupted, impossibly-slim metal slate, as impenetrable as some sort of found alien scifi object. The notion is that by reducing physical controls, the software itself comes to the fore. It’s beautiful conceptually … and then you find yourself tapping and stroking a piece of undifferentiated glass. For navigating interfaces – and even, I’d argue, exploring sound design and composition – it works brilliantly. But for live digital performance (what to game lovers is called “gaming”), for anything that wants tactile feedback, it can be imprecise or unsatisfying, or both.

Watching this shake out as a design problem is fascinating, especially coming from the perspective of music. Digital musicians were exploring alternative interfaces since before it was cool. Given the ability to make any sound we can possibly imagine, the question of how you design an interface around sound is compositional, philosophical, essential.

Whatever winds up working in the marketplace, there are some fascinating ideas for combining touch with tactile. Since both are good at certain tasks, why not do both?

We’ve seen several examples among musicians and researchers exploring how to augment the touchscreen with physical input:

Mike Kneupfel’s research at NYU’s ITP program, in the video at top, investigates adding additional inputs. See: Extending the Touchscreen.

We saw that kind of extensibility in an iPad dock concept by Livid Instruments.

While it lacks additional tangible controls, I/O extensibility is featured in a still-as-yet-unreleased “pro” dock by Alesis, and most recently in a DIY dock by circuit bending pioneer Reed Ghazala.

Now, game vendors are moving in the same direction – even with prototypes that look quite a lot like the research project above. (Sometimes, arriving at the obvious conclusion is necessary for a great design.)

Sony’s PlayStation Vita, successor to the PSP mobile game platform, augments touch input with tactile controls in much the same way as Michael Knuepfel’s work does. Notably, it also proposes how these inputs can coexist in a form factor that’s larger than a phone, but smaller than a tablet – scaled roughly to a comfortable holding distance between your two hands. (Microsoft and Apple each unveiled standard split keyboards on Windows 8 and iOS 5, respectively. The era of thumb ergonomics is now fully underway.)

Nintendo’s Wii U controller combines a lot of sensing capabilities into one device. Like Sony’s effort, the centerpiece is the combination of the interactive touch display with analog controls. But true to its Wii heritage, Nintendo is packing other sensing technology, too. While its evolution has been more piecemeal, the same is true of the Xbox 360 in the Kinect era. The Kinect camera is really a bundle of mic and stereoscopic camera sensing with software intelligence for motion analysis and even speech analysis via a variety of methods. While Kinect is touchless, the conventional gamepad still plays a role.

What’s the relevance of all of this evolution to music? Digital music’s demands parallel gaming, requiring precision, accessibility, scalability from beginners to hardcore experts, and real-time interaction. Also, music research has often been at the forefront of experimentation with a variety of means of translating sensory data to expression. And since musical practice itself is roughly as old in human evolution as language, if not older, it’s a key way of glimpsing how ubiquitous interfaces can become meaningful.

Let me put that another way: the stuff game companies are doing now looks a heck of a lot like what computer musicians have been doing for years.

While much of the acclaim for platforms like the iPad has been for their transparency and unadorned interfaces – and while I believe those are valuable concepts – bundles of capabilities for interacting with the world can become powerful. That means efforts like Apple’s addition of USB MIDI connectivity to the iPad, or Google’s nascent work to standardize USB host mode and open hardware development (based on Arduino), take on new meaning. Add to this additional connectivity via Bluetooth and wifi, and it may be that we only really see what these platforms do when, like the PC, they start geting sociable with a range of other gear.

This could also mean that communities like the music community have a chance to prove that the “post-PC era” is a little different than it’s been described in the mainstream press – and maybe a little less a radical departure. The “post PC era,” we’re told, is less about being a hub for a lot of hardware. But as people look for tactile feedback, some of the coolest applications of these platforms may not be in the mainstream use as “consumption” devices, but at the fringe.

I’ve just come from the launch of the Samsung Galaxy Tab 10.1 in New York. You’re not missing much; there were a handful of people snapping up the tablets. (I think the 10.1, and a few other Honeycomb-based tablets, do have a bright future, though their growth may be a bit slow at first as developers get their hands on them and give people a reason to buy them.)

What was most compelling to people at the launch, though, was a planned appearance by pop star Ne-Yo (at least according to some staffers to whom I spoke).

But the connection was, at best, tenuous. It may be when devices like these tablets are made more viable for musicians onstage that that connection starts to make sense. And that may mean that Apple and Google/Android vendors alike need to start to think more aggressively about the larger ecosystem and hardware applications. Remember all those futuristic promises from Apple about hardware accessories? Right now, the most significant hardware is the Square payment add-on, and it uses a hack to make it work through the audio jack. Both Apple and Google can do more work to open up hardware development.

It’s all well and good for the tablet to be a “post PC” device, to be different from PCs, to be better. But they may simultaneously need some of the openness to other gadgets that made the PC age so revolutionary.