Tenori-On and iPad apps, hardware designs and visual creations: set against the beautifully-generative mind of Japanese/New York artist Yayoi Kusama, the flurries of dots and circles and patterns in musical interfaces take on a richer meaning. This video, from a workshop hosted at the Tate Modern alongside an exhibition of Kusama’s work, needs little introduction. Instead, the dizzying cuts of geometric abstraction, the array of visual ideas for musical interface begin to take on the same personality of her expansive creations. The galaxies produced out of the minds of musicians somehow overlap with this iconic artist. I hadn’t really made the connection before, even as a fan of her work, but with this workshop, the sympathetic vibrations – intentional or not – become clear. Description:

Sonic Kusama:
Workshop exploring connections between the work of Yayoi Kusama and creation and representation of new music & sound art through visual audio interfaces.
Presented by Simon Little and Kelvin Brown with Chase Lane.
Audio track by Capstone Music
Video production by Territory Studio

If you’re in London, Infinite Kusama is on view now at the Tate Modern.

PocoPoco is a delightful, fanciful device that takes music control into the realm of kinetic sculpture. Normally, the relationship of music controller is primarily about the operator making physical actions. With PocoPoco, the hardware itself moves. The essential musical structure is familiar: it’s the grid of light-up buttons, with strong similarity to the ongoing interaction design of Toshio Iwai in the 90s and (Tenori-On) past decade. Even aesthetically, there are similarities – perhaps not coincidentally as this team is also Japan-based.

But adding in the element of solenoid-powered cylinders popping out of the grid adds a major element of surprise. There is also an unmistakable similarity to a certain arcade game, Whac-A-Mole.

Whac-A-Mole might be ideal inspiration. The game itself is based on rhythm and time, and the ability (or inability) of the brain to deal with multiple simultaneous stimuli, much in the same way our brain has to track across lines of counterpoint in music. And Whac-A-Mole’s history might be instructive, too: it’s the creation of Creative Engineering, the pioneering kinetic and animatronic company behind Showbiz Pizza and Chuck E. Cheese. (Achievement unlocked: CDM legitimately references Chuck E. Cheese twice in one week.) Founder and design Aaron Fechter’s animatronic shows might not seem a likely source for futuristic interaction design and music, but with the computer added to the equation, simple mechanical effects take on an entirely new significance.

But back to the PocoPoco. As a musical instrument, I’m dubious. It’s fundamentally another a four-by-four step sequencer, so it’s not as though it actually solves a problem. (Well, if you’ve ever wished your step sequencer were also a game of Whac-a-Mole, it’s the invention you’ve been waiting for.) But even if it’s not actually useful, it’s no less intriguing. It could be seen as a tantalizing reminder that adding motion to interfaces could produce musical devices that double as moving sculptures, and performance tools that move rather than sit around waiting for you.

The timing seems right, too, as touch interfaces like the iPad make physical interaction fairly abstract (running your finger on undifferentiated glass), or gestural interfaces take away any touch at all (Kinect).

There’s a great interview at DJ Tech Tools. That’s fitting, as DJTT has popularized their own MIDI Fighter hardware, which accentuates the tactile feel of playing grids by swapping arcade buttons as the input, and likewise has a strong connection as this does to games and arcades. A must-read:

PocoPoco – The Motorized Controller [Interview, DJ TechTools]

Takaharu Kanai, one of the designers from the IDEAA Lab team at Tokyo Metropolitan University, has some good things to say.

Seen other kinetic hardware, or worked on a design of your own? We’d love to see it.

On the road from futuristic instrumental concept to real-world product, the Yamaha Tenori-On as shipped lacked some of the functionality its creator, gifted media artist Toshio Iwai, originally imagined. Notably, wireless networking, which promised social music-making with other devices, was gone, replaced with a more-limited MIDI connector.

Now, in a surprisingly literal translation from the hardware to iPad, it appears the Tenori-On has added that feature – but lost some of its charm. An iOS developer notes to me that pitches don’t sound when you tap the screen, only when they are played in the sequence. That fundamentally changes the interaction with the sequencer: you can’t hear notes until they’re sequenced, and you would presumably lose the sense of playing an instrument. That report is happily incorrect; both the developer and I were mistaken from our video impressions. That makes this far more useful.

My reaction here should be taken with a grain of salt – this is only a demo video. But in observing what is new (networked features look terrific), it’s likewise worth saying that something is lost when you move to tangible hardware. To me, a lot of the appeal of the Tenori-On was tangible: the machined metal case, with curved edges designed to be comfortable to hold, and the feeling of running your fingers against discrete, round keys on the array of buttons. Those are lost by necessity. Yet, oddly, some of the Tenori-On’s features designed primarily for hardware – the menu system and navigation keys – are reproduced here, features necessary on a hardware design but not a tablet.

Yamaha Japan, apologies for going on a rant on a product I haven’t yet used, but I’m concerned at what seems to be a missed opportunity. And designer Toshio Iwai has already conceived imaginative touch-based interfaces that are designed for a screen, in works before iOS had even been announced, like ElectroPlankton for the Nintendo DS and interactive installation work going back some 15 years or so.

Simply translating hardware designs to a screen is novel, but rarely usable. Just ask Tascam, who were roundly (and rightly) criticized for making a Portastudio app for iPad that required you rewind every single time.

At least the good news is, some of the musical personality of Toshio Iwai’s work remains, and in a form that doesn’t require a costly hardware investment. Updated – also, via readers, there’s evidence of MIDI support.You’ll find other videos on Yamaha’s official Japanese channel.

Just mark my words: the hardware is still cooler, and there’s a lot of potential in hardware and software sequencers alike beyond this yet to be realized, whether by Yamaha or by someone else.

Updated: I want to re-emphasize that there appears to be auditory feedback as you press buttons for sequences, which is great news and vastly improves usability. And while I stand by some of what’s advantageous in hardware, I’m excited to learn that we may get both networked and MIDI functions here, as we’ve seen in apps from makers like KORG.

Reader comments are very positive, so amidst this hopefully constructive criticism, I think it’s encouraging that the software looks promising and people are eager to try it! (And being critical of some features does not mean you can’t eventually like the product – part of why I tend not to shy away from criticism.)

http://www.youtube.com/user/yamahajp

Clavilux 2000 – Interactive instrument for generative music visualization from Jonas Heuer on Vimeo.

Think of playing musical notes for a moment, or close your eyes while fingering a piano keyboard. Odds are, some visual – however abstract – pops into your mind. Visualizing musical notes is second nature in the digital realm, once a note and an image can each be represented with numbers.

Clavilux 2000 by Jonas Friedemann Heuer is one of the latest works to run with the idea. As you play notes, beams of color drift up from the keyboard. In 3D mode, those beams take on a lovely, subtle quality. The model itself isn’t new, owing the notes-as-lines model to player pianos (or even music boxes), and recalling light organs. But there is something intuitive about this model – and I can imagine it being a terrific way to encourage someone to practice. (Well, that or else it could be distracting while practicing!)

Description. Thanks to Yifan Mai for the link; via infosthetics.com, a fantastic resource for exploring ways of visualizing information.

Clavilux 2000 is a music visualization installation that produces generative real-time animations of music. It consists of a computer running vvvv patch hooked up to a MIDI keyboard and projector. Every note played on the keyboard produces a stripe, whose proportions and color correspond to how the note was played. For instance, the color is mapped to the tonality of the note via the circle of fifths, thus visualizing harmonic consonance and dissonance. Besides looking really cool, it also thus creates unique “fingerprints” of each performance.

Clavilux 2000 is extremely close in design to a key 1995 work by media artist Toshio Iwai, known most recently for the Yamaha Tenori-On and Nintendo-published ElectroPlankton DS (each of which uses ideas from the earlier project). Piano–as image media and related works employed both inputs and outputs. (in the installation, visitors could use a trackball to enter note events visually on a screen; in performance with Ryuichi Sakamoto, the work used a piano. In each, events fly off perpendicular to the piano keyboard as beams of light, just as in the work here. That’s not a criticism, incidentally – even without seeing Iwai’s work, it’s a logical solution, because the keyboard organizes notes into an array of thin rectangles (the keys).

Unfortunately, Iwai’s work is not well-documented online; videos of these pieces have been removed. I do have a few resources for you, however. At bottom, there is a video of a 2006 Ars Electronica talk on the visual interface for music. (I have some video of Toshio’s similar thoughts around the launch of the Tenori-On which I should publish.) And for more:

Toshio Iwai talking about the visual-musical interface[artintelligence]

Toshio Iwai keynote at Futuresonic [pixelsumo]

http://www.nydigitalsalon.org/10/artwork.php?artwork=57

Imagine an alternative universe in which simple digital handheld games evolved into sophisticated music tools. Oh, and they also made lots of really purty lights flash. Mmmmm … flashing lights.

Well, that alternative universe seems to be right here. Mike Una gave us a massive dump of unusual new DIY sequencers, crafted from the ground up to rework techno into sonic objects. Some are unquestionably indebted to the analog step sequencer, but others take as much from 80s digital toys.

Working with the Meggy, Jr. DIY handheld game platform – with a stunning 8×8 pixel resolution – Darius Kazemi has begun building a music app. He calls it “MeggySynth,” and says he’s conceptualizing it as much a video performance as it is sonic performance. Let the video get at least part of the way in, as the colors really pick up – full RGB LEDs really are a beautiful thing (and something you don’t get from projects like monome).

Our friend and Handmade Music regular, the talented hacker Collin Cunningham, covers this for MAKE:
MAKE: Blog: MeggySynth makes music

Collin rightfully compares this to Tenori-On. Part of what strikes me about Toshio Iwai’s work – not only Tenori-On for Yamaha and ElektroPlankton for Nintendo, but his installation work stretching back to the 90s – is that it often incorporates game aesthetics. Designs are reduced to their elemental interaction and visual representation, which very often includes low-resolution, pixellated grids. (Photo: Julie Delvaux.)

Now, being the greedy person I am, I really want this style of RGB grid, but with other sound sources. But I think there’s a lot of potential, and just as grids of lights can function on roadsigns, there’s no reason even a small number of pixels can’t be expressive. Just ask your local Tamigotchi.

The best part of simplicity? Darius, designing level editors — in Excel.

Tiny Subversions: My Meggy Level Editor