Alexander Chen, he of Kinect hacks and subways turned to strings, is back with another string visualization. Built in the browser (an interactive version is available), this work makes a visual accompaniment to Bach’s First Prelude from the Cello Suites. If you read music notation fluently, you may find the score itself suffices, but even so, the math to make this work – and the dance of circles across strings – is compelling. Alex, whose day job is with Google’s Creative Lab, talks to us a bit about the mathematics and process. First, his description:

baroque.me visualizes the first Prelude from Bach’s Cello Suites. Using the math behind string length and pitch, it came from a simple idea: what if all the notes were drawn as strings? Instead of a stream of classical notation on a page, this interactive project highlights the music’s underlying structure and subtle shifts.

Built in: HTML5 Canvas, Javascript, SoundManager
Made while a resident at Eyebeam

CDM: How did you settle on this particular visualization of this famous work? And how did you work out the maths, that is, why this specific number of dots, the distance from the strings, and the length of the strings themselves?

Alex: When I listened to the opening of the Bach, where it repeats the same bar twice, it made me think of a call and response. So I immediately pictured two wheels that echo each other, instead of just one wheel with four dots.

Figuring out the symbolic string lengths in pixels was a fun research project. I wanted explore the simple math behind string length. I learned that you can derive an entire chromatic scale just by using two fractions: 2/3 and 1/2. These correspond to the fifth and octave intervals. It’s called Pythagorean tuning. I stumbled onto this great little worksheet [PDF link] which seems to be intended for students.

Were there other things you tried, any failed experiments?

There were important learnings. It used to begin playing the piece right away. I started the opening tuning animation as an afterthought while I was preloading the strings. But that sequence became really critical.

What’s your sense of the music now having done this? Did it change your hearing of the piece

A lot of music visualization these days is linear, like reading a score. Logic’s editor, or even games like Guitar Hero, all follow that structure. And there’s a reason for that, as it’s convenient, for both computers and humans, since we can read it (and edit it) like a book. But I wanted to try something different. I think some of the magic of watching a performer is seeing such subtle, intricate finger movements produce such moving sounds. When I watch these strings morph, it feels more like the computer is performing, not just checking off notes one by one.

Seeing the Bach Prelude in groups of 8 notes gives me a bigger picture view of the piece. Instead of focusing on the individual notes, you can see each bar as a group. The strings start shifting very subtly, but as the piece builds, the strings seem to be panicking to me, shifting more rapidly. The computer is not expressive. All notes are played at equal volume. But the notes themselves, the data of the song, is inherently expressive.

http://blog.chenalexander.com/
http://twitter.com/alexanderchen
http://www.baroque.me/ [interactive - grab the ... circles ("grab the balls" doesn't sound quite right)]

Oddly enough, I found another – non-digital – visualization of the same work. Photo (CC-BY) Brooklyn-based player and architect George Showman, who explains the process thusly: “Basically it’s strings attached to my wrists, that run around the room to connect to a pen hanging from the ceiling in such a way that the left hand controls up-down, and the right (bow) hand controls left-right. I.e. it turns me into a plotter. Then, when I play cello, the gestures of the playing are transmitted into the line in the drawing.” Compare this to the image above – in particular, two different ways of treating time, each distinct from a conventional score.
  • http://rekkerd.org ronnie

    That is just lovely!

  • Miguel Marcos

    Beautiful!

  • http://brianhouse.net h0use

    I love how elegant this is, it just 'feels' like the piece itself. Chen rocks. one quibble — I dont think Pythagorean tuning was used in Bach's time, and Im guessing that what we're hearing is the standard equal temperament used today. So the strings are not strictly the correct length.

  • Peter Kirn

    @h0use: You're correct, the strings don't represent what yo're hearing. However, they should be a reasonable approximation (especially because we're measuring in pixels, which is pretty coarse).

    As for Bach's tuning, that's a much more complex question; it wasn't Pythagorean, but neither was it modern 12-tone equal temperament.

  • http://i5music.net Jim Aikin

    Bach probably had in mind some sort of mean-tone temperament, but string players (in his day and in ours) unconsciously adjust certain pitches up or down slightly to produce more pleasing sonorities.

    I like this more than I expected to. (I've been playing this piece on cello for many years.) What he distorts are the lengths of the phrases. He's assuming, presumably because it's a limitation of his animation model, that eight-note groups (defined by the bar lines) are the significant units throughout the piece. That's true at the beginning, but obviously not true at several spots in the second half.

    Also, he left out the B in the final chord.

  • http://zeroreference.blogspot.com zeroreference

    awesome! 

  • http://www.jhhl.net/iPhone jhhl

    in re: Bach's tuning: 
    http://www.afmm.org/bachtune.htm
    (In many cases, the tuning answer is Werckmeister III).

    In my experience, music is not linear – it relies on memory and the relation of notes to previous notes and implicit harmonies, more like a lead sheet than a system of staves, which relates notes to some kind of grid work and not to each other.  

  • http://itunes.apple.com/us/app/pairshare/id424429744?mt=8 Brian Tuley

    Great pdf link also.  I'll be reading up on this 1/2 2/3 Pythagorean tuning principle.  Might come in handy for a Max patch experiment with controlling single oscillators, Stockhausen style.    

  • kconnor9000

    Neat.

    Visually the circles are cool, but effectively, they make the tempo non- linear! The errors are small here, but using a point on a circle will make the time-between-notes wobble. If the rotation rate is constant, the projection of that dot is not constant — it's fast in the middle and slow at the edges. y=sin(theta) thing.

    What I get out of this is a transformation of one variable — the pitch — into a graphical representation, over a short time window (1 measure, 8 notes here). So the pitch contour vs. time, which is a very familiar picture for most people, gets mapped onto a single pitch picture per window, a one measure gestalt that looks like an I Ching hexagram. It's interesting, but I'd like to see 32 or 64 of these little pictures in a grid, to watch the larger structure of the piece evolve, see the repetition that way. I'd rather see at one picture, this decomposition of the piece into 8note granules which we see in real-time instead. My pitch-memory is better than my hexagram memory.

    So, beautiful, yes. But not really an effective mapping, in the sense of revealing something hidden.

    Some related beautiful stuff with a traditional pitch vs time contour representation is the Steve Malinowski Music Animation Machine — http://www.youtube.com/watch?v=SQfzNJGnqnw .

    While I'm on a pitch vs. time contour rant, let me ask this. Why, in 2011, can I not take the piano roll display in my DAW and ROTATE it 90 degrees positive, so that I see the notes from left to right, and time runs from top to bottom ?!!! I mean really, it makes a hell of a lot more sense that way to me as a keyboard player. It's not like computers have a hard time with this…

  • Arrgrr

    To think that Back wrote this stuff on an old 16bit Atari computer without even one plugin, it is truly humbling…

  • Rymf

    Got damn I love the future.

    @Arrgrr: That Back was a helluva guy.

  • r

    wow very nice. One thing I find neat is that you can get these minute timing variations by having equidistant lines with different pieces of the arc going between them. I never thought of timing in that way before.

    It kind of reminds me of the Les Paul google doodle, did Alex have any thing to do with that? The tension on the strings looks pretty similar…

  • http://agargara.iiichan.net Agargara

    This is really cool! I love Chen's work.

    My one quibble is that the time between notes feels a bit too weird. Perhaps this is because of the the points traveling on a circle, as kconnor noted.

  • r

    @kc a tracker gives you the vertical transofrmation, but then pitches are written as letters.

    I think the timing imperfections are part of the charm. Because of where the strings are on the circle, there's this breath-like acceleration and deceleration to each phrase.

    The window-size isn't arbitrarily small, it's the size of the phrases in the music, so it lets you see the pitch adjustments from one phrase to the next. The usefulness of course gets kind of subjective and personal though…

  • nickmasiuk

    Not sure how relevant this is, seeing how the piece's inaudible structure isn't nearly as conjectural as the above example. But I hope folks enjoy. 

    http://www.youtube.com/watch?v=xUHQ2ybTejU

  • r

    Hey he is one of the guys les paul google doodle devs:

    http://blog.chenalexander.com/2011/google-guitar-

    Cool!

  • http://www.chenalexander.com Alexander Chen

    Thanks for the comments everyone! Yeah, I had planned to compensate for the circular path timing. But in the end I decided I kind of liked the imperfect, oscillation-like pulsing of the tempo. (I can understand how it sound slightly irritating to musicians with really finely tuned sense of tempo though!) Computers usually play things so perfectly. Even musicians tend to perform this particular piece with very loose, expressive timing. Some of the pieces I'm looking at next (Well-Tempered Clavier and some others) might make more sense more tightly-timed though.