As an addendum to why trying to make computer models musically creative can be so disastrous, maybe the problem is we fail to understand what creativity is.

Scientists funded by the National Institute on Deafness and Other Communication Disorders (NIDCD) have found that, when jazz musicians are engaged in the highly creative and spontaneous activity known as improvisation, a large region of the brain involved in monitoring one’s performance is shut down, while a small region involved in organizing self-initiated thoughts and behaviors is highly activated.

Study: Prefrontal Cortex In Jazz Musicians Winds Down When Improvising [scientificblogging]

That’s just one study, and I won’t pretend to be an expert in neuroscience. But what the scientists are describing is awfully close to the nuanced way jazz musicians will describe improv. It’s not not thinking. But it’s also not self-monitoring. It’s something else.

In other words, the self-judging prefrontal cortex — the part you can easily model as a set of computer software rules — switches off, but another area of the brain hits overdrive. And “self-initiated” is exactly what’s lacking in computing technology.

But this has another implication, now that so many of us use computers in performance. For one, the lack of initiation from our computer companion means computers may be fundamentally unsatisfying as accompanists or “duets,” no matter how many rules or interactive behaviors we stuff into them. Maybe we don’t have to view them that way — maybe we should think of them as an extension of composition or an instrument. After all, a person with a laptop is usually a solo artist.

But the other likely implication is that, as many readers here have noted, we need to set up computers in ways that allow us to shot down part of the prefrontal cortex when playing. That’s a complex thing: you want your software to help you get into the zone. It doesn’t mean not thinking — quite the opposite. It means taking away distractions, partly feeling good enough about a performance to be able to stop the “self-monitoring” behavior, and partly giving yourself enough to do, musically, that another part of your brain actually has to work harder to proceed. Readers noted earlier this week that music notation can be musically distracting — not surprising, given many musicians make the effort to memorize a piece for exactly this reason.

But in addition to shutting down one section of your head, you want to activate another. That could also mean that tools that automatically limit your playing to specific scales, while they seem to make things easier, prevent your brain from reaching the level of activity when you feel the most inspired — like failing to make an exercise cardiovascular.

Thanks to Richard Lainhart for sending along this article (via the Electronic Music Foundation list).

How do you get into the zone playing live — particularly if you do use a computer?

The vocal input goes well, and illustrates the number of different inputs beyond the mouse you can expect in The Future. Here’s the problem: harmony is extraordinarily difficult to model on a computer because of the number of variables, the amount that’s driven by instinct and art. And let’s be blunt: it doesn’t work right.

In short: if you’re planning to build a Jerome Kern robot, the technology may not be there just yet.

If you’ve got a strong stomach, you can watch the application lay waste to “The Way You Look Tonight.” Speaking of tears: composer Kern actually drove the lyricist, Dorothy Field, to tears with the original. MySong might make you cry … in a different way. It chooses chords that fit a key and fit the melody, but completely unravels when it comes to making chords work horizontally with each other with the melody — which, when you think about it, isn’t all that easy even for experienced musician. The funny thing is, the harmonic structure of the song isn’t that complex (well, until MySong gets cranked to its avant-post-bop setting later in the demo). Harmony is perhaps just harder than the technologists may realize.

The researchers do compare their tool to Band-in-a-Box’s automatic harmony selection module, and this works better than that — but that’s not saying much.

I also have to admit, I’m getting a little fatigued of all these tools that want to dumb down music, as if somehow it’s music’s obligation to be push-button easy. Do we build giant robotic armatures so people can play basketball without practicing? Isn’t it the struggle that makes it fun? The researchers in this point seem to have missed the point: all those hours you spend sitting with an instrument working out chords are perhaps what music is about. There’s not some musical secret the experts are keeping from everyone else. The songwriter with the guitar very likely received very little training. All of that tweaking of melody and harmony is part of the process that eventually yields things like, well, “The Way You Look Tonight.” Jerome Kern and Cole Porter and Richard Rogers did it very quickly; amateurs may do it more slowly. But it may not be possible to reduce to rules in a way that the current generation of computing intelligence can even understand — and even if it does, it may require more than one or two sliders to adjust.

The best part of the video is the editable parameters: sliders for Jazz factor and Happy factor settings. (Theory fans: the approach seems to be for Happy factor to lobotomize to major I/V chords and Jazz factor to eventually turn everything into sus13.) I’d like to suggest a few additional settings for reproducing a broader variety of music:

• Emo angst factor
• Tone deaf factor
• Pretentious techno chords factor
• Stoned factor
• Saccharine-sweet triteness factor
• Community theater audition accompanist factor
• Went to a liberal arts college where everyone on my floor played Ani DiFranco way too much factor

It’s well worth watching the demo. And, of course, this is the reason to tackle artificial intelligence — even if you’re unsuccessful, you’re learning. My guess is, we’ll need genuine AI before we can successfully harmonize melodies.

MySong, from Microsoft Research, makes your singing sound a lot better than it really does  [istartedsomething]

MySong: Automatic Accompaniment for Vocal Melodies [Explanation, Demos, Academic Paper]

The creator writes CDM to tell us more about it:

Hi, My name is Amaury Hazan. I’d like to introduce a software I have developed.

BillaBoop is a real-time audio driven drum controller which allows the user to control up to 3 drum instruments. The user can control any drum synth with the voice (beat box), or any object or musical instrument. Unlike other audio-driven systems wich require a lot of parameter tuning to be able to discriminate the sounds you are playing, BillaBoop incorporates an efficient Machine Learning component which enables the system to learn by demonstration.

Right now, this is just a technology demonstration, but it works on Mac, PC, and Linux, and could eventually be built into all sorts of other tech. It works in real-time, and recognizes up to three distinct kinds of sounds at a time. (Think, bang your friend with the fish, then kick your desk, then knock your head into a wall. You could assign one to a hat, one to a snare — well, you get the idea.) As Amaury suggests, drum controllers are only the beginning; other possibilities include music sequencing, games, interactive installations, sound-sensitive lighting, and so on.

Amaury had some talented folks in his thank you list, including plug-in developer Bram de Jong and music technologist Xavier Serra. He himself is a PhD student in Machine Learning, Music Cognition and Audio Signal Processing at Music Technology Group, a department of Pompeu Fabra University.

BillaBoop [Project Page]

Long-time readers will recognize a resonance between this project and Georgia Tech’s robot drummer Haile we featured some time ago, which analyzed rhythms so it could play “duets” with human musicians:

Robot Drummer Responds to Human Playing; How They Did It

Raymond Scott’s Electronium is one of the great, odd sound inventions of all time. Scott developed the machine as an automatic performance and composing machine, a great, mechanical algorithmic music creation device. For an official source of information, be sure to read up at the Raymond Scott site, which has this fantastic music demo:

Electronium Music Sample

The idea of the machine, with no keyboard and the ability to “automatically” create music, is still a bit radical today. The sonic results are as whimsical and fresh now as then. But it’s the underlying technology that’s impressive: the device “suggests” musical motives, and allows contrapuntal techniques and development of the materials into music. Not bad for the 1950s — and a lot more fun to listen to than a lot of supposedly more-sophisticated computer algorithmic music.

Motown got interested in the results, I think because it was the only hardware at the time to come with a DOOWAH control.

Raymond Scott was also a major inspiration for a young Robert Moog, a relationship described in Moog’s own words on the Raymond Scott website. In fact, had it not been for Scott apprenticing him, it’s possible Bob Moog would have stuck to Theremins and never gotten into the synth business.

The instrument survives, but sadly in non-working order, in the basement of Mark Mothersbaugh’s office. It’s bittersweet looking at the instrument through this video, posted in April, and not hearing it work. But before you despair, Mothersbaugh is promising to fix his Electronium. Let’s hope he does.