Talk about new ways to discover music on the Web, and invariably you’ll come to Pandora. Using a sophisticated classification of music based on musical attributes or “genomes”, the Pandora player attempts to decode your musical tastes. Based on artists and songs you like, Pandora will build a “radio station” that parades tracks from a variety of artists it deems related, from hits to obscurities, always with the opportunity to skip songs (or bookmark and buy them).

I wanted the chance to pick Pandora founder Tim Westergren’s brain for several reasons. I was especially interested to know how Pandora’s music recommendation engine works: I was skeptical of automatic recommendations initially, as I suppose many music aficionados might be. And I wanted to know where Pandora might go in the future, and how artists could benefit from the service. Could tools like Pandora help raise niche artists out of obscurity?

Little did I know that the week after sitting down with Tim, Pandora and all other internet radio broadcasters would find their future viability threatened by new licensing rules. Tim says Pandora will simply close its doors if the new fees are left to stand. See the follow-up to this interview in a separate story.

Pandora’s founder speaks to users in Charlottesville, Virgina, on a 50+ city US tour. Photo by Steven Toomney [via flickr]

With that in mind, though, I find a lot of what Tim had to say about Pandora all the more valuable. The service offers some unique ways to find music, already has some major advantages for artists, and could have even more possibilities in the future. Tim revealed how the Pandora attributing system works, and how someone named Matt Nathanson was inexplicably the center of the musical universe for a period of time. (Matt Nathanson, proving the Michael Jackson of the dot-com age will be a statistical anomaly?)

Westergren was in New York on a tour that had already taken him to 50 meet-ups in 50 US cities in a year, talking to users. Some users were so passionate that meetings went on for four hours, and Tim promises to keep doing them “until I drop from exhaustion.” We sat down in a Starbucks, of all places, to chat — the pre-programmed, non-interactive Hear Music stream blaring in the background.

Inside Pandora

CDM: You’ve gotten the chance to hear from your users all around the country. Has that impacted Pandora in any way?

TW: A lot of the stuff that we build is dictated from stuff that people are asking for. Recently we did a thing called QuickMix — that’s a feature from listeners. We had “I like this song, but for a different station.” “Don’t play it for a month.” Alphabetize the playlist. The whole backstage, having audience information. Most of the stuff that we do comes from user feedback.

CDM: Talk to us a little bit about your personal background. Before you founded Pandora, you started your career in music making. How did that lead you to create the Musical Genome Project and Pandora?

TW: I’m a musician. I was a jazz piano player when I was little — kind of self-taught, and then studied in college. After college, I played rock bands for about seven or eight years, touring around the US, having the independent musician experience for a while. Then I spent three or four years as a film composer. The job of a film composer is to figure out the taste of a film director. And you do that by a musical conversation — you play songs for each other. And you have to glean from that what they like about music. I did that for a while, and that was kind of the beginning of the idea of attributing, because I was essentially trying to figure out the musical genome for a given film based on the director’s preferences. Between that and my interest in helping independent musicians get heard, which was my own background for many years, the idea sort of came together, propelled by the excitement that was going on in late `99.

CDM: It seems like you were among the first, if not the first, to do this kind of attribution in this way.

TW: I think there’s always been attributing of some time. We were the first ones I think to do it comprehensively. I think most companies have used data mining for it — they draw recommendations or playlists for radios from comparing the other listeners.

CDM: Right, whereas you actually go through with people and evaluate the music itself. And this is done by hand? How does that work?

TW: It’s all manual. So if you’re a musician, you come in the morning and log in and there’d be a menu of songs that need analyzing, and a shelf of CDs. You’d grab one, and launch into an analysis. What you’d see in front of you is page after page of music genome templates of musical attributes. And you go through one at a time and score them, close to 400 musical attributes, and those represent all the details you could imagine of songs. When you’re done, you have this big kind of fingerprint, and you dump it into the database, and you find out what it matches to later.

CDM: How big is that database now?

TW: Over half a million songs.

How Pandora Performs, and How Its Founder Listens

CDM: Are there areas of the algorithm that you feel are particularly weak, or particularly strong?

TW: Where it’s weakest is in lyrics. Where it’s strongest … outside of lyrics, I’d say it’s pretty strong. I think it tends to do better and better the more complex songs get, because there’s more there; there’s more detail to capture. So the more intricate the arrangement, the more intricate the composition, the more it uses the power of these details.

The challenge with an artist is, when you type in an artist to launch a station, you’re giving us really diffuse input. Take Elvis Costello. He’s done a bunch of different things in his musical career. So when you say you like him, we don’t know what you like — what is your sweet spot? And so, we’re going to try stuff out from all over his catalog, stylistically, until you tell us what you like or don’t like. So I think what happens is often times a listener will enter an artist, and they have a very specific idea about what part of that artist’s sound they like.

CDM: Right, and I guess that’s the sort of drill-down that happens as you use the thumbs-up, thumbs-down tools.

TW: It’s kind of like pruning a tree, finding just what part of it you want to listen to.

Our mode of doing this is to find things that are musically similar. There are other reasons that music gets connected, that have to do with socio-cultural things. But that’s not our thing. So we aren’t going to connect two radically different artists musically — even if they’d be on the same bill. We don’t know what their social milieu is.

CDM: So how do you listen to Pandora?

TW: What I do is I tend to listen to a station, and as I like songs I buy them. And when I have a few, I go buy them on iTunes and create my own playlist from them, and then listen to them over and over again until I’m sick of them — and then go back to the radio. I tend to listen intensively with music. I notice it as it’s happening, and it draws me in to do something, and then if I find something I’m very compulsive about buying them and experiencing them over and over, which I can’t with Pandora.

CDM: Do you still listen to traditional radio, or has Pandora completely replaced it for you?

TW: I’m a big NPR junkie.

CDM: The words part.

TW: Yeah. Although they have fabulous, fabulous music. I don’t listen to a whole lot of commercial music. I listen to public radio all the time.

Pandora’s Future

CDM: Now, I’ve noticed there are some areas that are missing, like there’s no classical music. Will any of that change over time?

TW: We’re [working on adding] classical now. The other thing missing is world music … so we have some Latin music, Portuguese, but we’re basically Ameri-centric, English-centric. Eventually we’re going to allow you to search stations for all sorts of things.

CDM: It’s interesting to me that at the same time you have the rise of internet radio stations with human DJs, at one end of the spectrum, like the increasing popularity of stations like KCRW [Los Angeles]. And then, at the other end of the spectrum, you have things like Pandora. Do you suppose both will continue to grow?

TW: I don’t think there’s one right answer. People have different things they like. There are even many right answers for people for how to build an online system that has no DJ at all. I think people look for different things from the radio. Some look for company — they want a personality they can reliably hear and they just like to hear their voice, what they have to say. I think all of the different approaches have their merit. If they have an audience, then that’s enough.

CDM: Well, and I suppose what’s interesting here is that there are human hands involved somewhere in each — both Pandora’s human-ranked attributes system and KCRW’s freeform DJs. So, what’s next for Pandora, as far as what you can talk about? Where do you want to take the technology?

TW: In the long run, Pandora needs to be mobile and ubiquitous. It can’t just be when you’re at your computer. It’s got to connect through stereo systems, when you’re walking, when you’re in a car — just like regular radio is. All those things are fair game for us.

Mobility I would call our #1 priority right now.

CDM: When you go mobile, will that mean adding DRM?

TW: I’m not sure if we’re going to DRM that stuff. It’s going to be a lower bitrate because that’s the only way you can deliver stuff over a cellular network, which may be by itself the answer.

CDM: Beyond mobile, where else do you see Pandora’s future?

TW: I would say our real dream is international, the other big, big bucket. It’s a bit of a mess right now because of licensing, but I’m hopeful that we’ll do that. And I have some really big dreams of what I’d like us to become, not only available legally everywhere but we have music from everywhere in the genome. So you can be here, and have a musical journey that takes you off in all sorts of directions.

What we’re seeing, unfortunately, is a country-by-country thing — it’s a legacy of the old record business, and the way labels are set up — and they have territories, so there’s not one standard like the DMCA in the US. I think over time that’s going to come around because they’re going to see they need to do things to allow these new services to flourish, because that’s where their future is. But I don’t blame them now for being cautious and uncertain about it; they just don’t understand it. They don’t know what it’s going to look like ten years from now. It’s hard to be the one to say, okay, man, let’s pull our finger out of the dyke and see what happens.

Mash-Ups, Users, and Bandwidth Bills

CDM: Pandora has attracted a number of technology mash-ups; the two most popular seem to be one combining it with eMusic for purchasing and another that integrates with Last.fm’s music “scrobbling” and community features. What’s your take on these mash-ups — obviously, you wouldn’t support anything that encourages stealing music, but are some of them positive for Pandora??

TW: It’s the ultimate compliment if someone takes your product and spends the time to make it into something else. We certainly don’t want people who hack it to steal music — that’s not cool. We try to break those things as often as we can; that’s wrong.

CDM: Have you broken them successfully?

TW: Oh, yeah. In the long run, there are a couple of things that we have to be careful of. One is if it allows people to something illegal, of course. The second is if it allows someone to deliver a version of Pandora that we have to support that doesn’t allow us to put advertising on it — so it’s kind of free and no advertising. If someone did that and it got big, then we would have to close it down. It’s expensive to stream those hours.

CDM: How much of your overhead comes from bandwidth?

TW: [Bandwidth cost] is substantial, and it’s only growing because we grow so fast. It’s amazingly linear, but now it’s starting to do this [steepens hands]. As more people hear about it, there are more people to tell people about it. Growth has an inherent exponential curve to it. But it takes a while to hit critical mass.

CDM: What are the demographics of the listeners like?

TW: Demographics of our listenership is literally from people in their late 80s down to people in their early teens. There’s a concentration of people who are in their late 20s and 30s, and they’re a little bit more male than female in that demographic. But I think that’s a reflection of … those are the people who find out about things first, because they’re connected in technology. I can say with total conviction that it doesn’t have a target market. Its target market is anyone who’s been disconnected from music and wants to get back in the game. And that’s everybody, except maybe a 16-year-old who has music coming out of their ears from their friends. Even then.

CDM: Anecdotally, I’ve been very impressed with the variety of people who like this, from people who are very technically savvy to casual computer users. It seems like the major achievement there is the sliding album cover interface. What went into that design?

TW: The challenge when we launched this was how do we create something very simple but that explains how it works, too. It has to convey the idea of a genome — what does it mean when you type in a song, what’s happening? But it can’t do that in a way that requires you to read three paragraphs of text, or six clicks to a glossary or something. That’s why those panes pop up, why we start with just one search box, why there’s no software to download. People have most definitely responded positively to simplicity. Most of what’s built by other Silicon Valley companies is built for themselves — they’re all tech geeks or music geeks, so they build things that they would want. And that tends to be more complicated — that’s the temptation.

CDM: What strikes me about your advertising is that you integrate the music player with ad sites, so you go to a Budweiser site and get a Pandora station from Budweiser. Much to my surprise, a lot of that music was actually quite good. Have these been successful with users?

TW: Oh, yes. The objective that we had was to try and marry [advertising] with the reason people are there in the first place. So, obviously, music-oriented ideas, features, and so on … For a brand, having a musical identity is a huge thing. Music is such a denoter of brand. And so they have pretty strong ideas about really wanting to identify strongly with sounds. And that’s one thing those locations allow.

CDM: How are those playlists built; are they lists of songs?

TW: No, you seed it with the artists and songs, whose sounds — what kind of music — you’d like to see come out of the station. We’re not allowed for legal reasons to program a station.

How Pandora Pays, and How Artists Can Make it Pay

CDM: I did an article two and a half years ago and tried to research licensing fees, via publishing fees from ASCAP, BMI, and so on. It seemed like there was real uncertainty about how this would be licensed, how the model would work. Has that changed now? [Ed. note: This discussion turned out to be eerily prescient as a DMCA court decision threatens the model for Pandora and all internet radio; see my follow-up interview for Tim's thoughts on that.]

TW: Well, two and a half years in this world is an eternity. I would say that it’s gotten much more clear. The business economics around it are a little more transparent now, so you can have a more informed conversation with a record label, like this rate’s not going to work, and here’s why. It’s had a lot of time to breathe. It’s still not fully resolved, there’s still some arbitration involved. It’s a matter of is there a change in the costing of it. But I think the idea of a statutory license is there to stay. That’s going to be the umbrella.

CDM: The fixed license must be good news.

TW: Oh, yeah. Years ago, a lot of online radio services launched and found out a few years later they had a big bill, and they all collapsed [when the DMCA was launched]. They thought they’d be like terrestrial radio, and not have to pay mechanical royalties, just publishing. And it turned out not to be that way.

CDM: The fact that you pay fees to organizations like ASCAP means that artists get some of that income from streaming plays, correct?

TW: [You the artist] give a piece of it to the agency that’s collecting it — like ASCAP or CESAC or SoundExchange, and then you keep the rest depending on whether you have a label.

CDM: So far we’ve talked largely about listeners; what would an artist get out of Pandora — especially as many of us at Create Digital Music, of course, have an interest in that?

TW: I think it benefits artists in lots of ways. First of all, Pandora does not distinguish between well-known and unknown artists at any step of the way. The only way in which we do is, if a song charts we will get it in as soon as we can. So as soon as a song pops into some kind of chart we immediately buy it and get it in right away. And that makes sense because a lot of people are likely to be trying to launch stations from that artist or song. Setting that aside, if you send me a CD or recording in your living room, it has just as much a chance of getting in as the latest release from Universal Music, all based just on musical quality. So if your CD is really great, it’ll go in.

We get tens of thousands of recommendations every month, between listeners, things sent in from artists, listeners, labels, managers, things from our own research … 30, 40,000 a month.

CDM: So is there anything that you can do as an artist, once your album is on Pandora?

TW: You do get played a lot on Pandora, because a lot of people are listening to it — and it’s only just beginning. You can sell stuff of it. And it encourages people just to build fans that come to your shows, that are introduced to you in the context of a well-known artist that they like, that they’re accustomed to. And in the long run, our intention is to make available to bands all sorts of interesting tools so they can take advantage of the data, and use it to drive things like where they tour, or what kind of records they make, or what bands they open for. All the thumb feedback and all the listener information, we can make that available on an anonymous, aggregated basis to an artist. So you could go and say I’m going to tour the southwest, what are the four cities I’m going to hit, and you can go in and look it up and see where the fans are.

I used to do this, I used to be in a band, and I would have killed to be able to drive 15,000 miles less and go to better places because I was able to get my hands on some kind of research like that.

CDM: We’ve seen the player, of course — what would you be able to do from your side, as an artist?

TW: If you’re in the James Jones band, you can send a link to all of your listeners, your email distribution list saying listen to my Pandora radio station you’ve created — you plus your creative brethren, your sound links. And that’s a useful viral tool.

Life, Music, and the Mystery of Matt Nathanson

CDM: You started off as one person trying to figure out how to understand musical taste working with a director. Now, having seen Pandora at work, has your perspective on what relates different music and tastes changed?

TW: I’ve always thought that you couldn’t really totally understand someone’s musical taste. The best that you could do is to get them in a neighborhood where they’re likely to find something they’ll like. I continue to believe that now having seen this — Pandora isn’t 100% accurate by any means. But I also do believe that you can understand someone’s musical tastes in musicological terms. It’s not the only ingredient, but it’s a very substantial and powerful one. And I think it’s describable. I used to do this with friends, I used to pinpoint, this is the part of the song you like — I’ll tell you why you like it. I think that’s a reality. I think it’s proving to have absolute validity. Pandora is really, really good at consistently playing the stuff you like. It’s not perfect, but it’s really good.

CDM: What about when the Pandora algorithm surprises people, though — would that surprise ever be desirable?

TW: I think it depends on how different it is. People get pretty pissed when they hear something they don’t like — it’s, like, what’s going on here? And we do a little bit of our own proactive adventuring for you. We’ll take the song and kind of bend it around and emphasize different aspects through the playlist to make it more interesting.

We wrestle a lot with how repetitive should it be? We could make this completely arepetitive and just never repeat songs. And I’m certainly as a case study, I love repetition if I like something. What’s it like in the first day, in the first week, in the first month, in the first year — how does it evolve to stay fresh over time?

CDM: So which stations have worked best for you personally?

TW: My Ben Folds station — I just can’t miss with that. He’s just so utterly in my sweet spot as a musician that anyone who kind of circles around him — he’s a great pianist, he’s a great melodist, interesting harmonies, interesting arrangements — all the ingredients that I really like. So he works well for me. I have some great bluegrass stuff that I listen to. I got a shared station that I really like that I got from someone the other day.

I don’t usually keep very many stations — I’ll keep six or seven, and then I’ll erase them. But those two have survived a while.

CDM: Are there Pandora artists that are especially popular?

TW: There was an artist — Matt Nathanson — who, for a reason we cannot explain to this day, was far and away the most popular artist on Pandora, played the most often, by a weird order of magnitude. We thought, like, is he the center of the genome? We had no theory about why it happened. It’s diminished over time, but it was a funny sort of strange quirk. I don’t know if we have a clear upstart who’s replaced him, but it became an inside joke that Matt Nathanson was everybody’s favorite artist.

CDM: So, are you making any music now?

TW: I went completely cold turkey for about five years, and I’ve started to get back into it. I’ve dusted off the piano, and I’m back at it. It was a long point in my life. And you know, we had a couple year period where Pandora was really all-consuming. I was so underwater with it … now I definitely feel freer; I have more energy.

Follow-up, in the wake of new royalty rates for online streams: If Streaming Rates Stand, “We’ll Have to Shutter”, Says Pandora Founder

  • Leo

    hi! i live in China and always listen music on Pandora, it's amazing!

  • Pingback: Musicians Online » Pandora’s Founder on Decoding Taste and Promoting Indie Music

  • http://www.andrewswihart.net Andrew Swihart

    Pandora actually gives you new music you aren't likely to have heard, which is pretty closely related to whatever you type in. I've found Audioscrobbler (Last.fm) to be pretty much worthless for this, and Pandora kills it.

    For example, I'm listening to The Beach Boys, some electronic music, The Bee Gees, Al Green, etc. And what does Last.fm spit out: Coldplay? I just wasted all my time for you to recommend Coldplay? Subsequent recommendations have been similarly bad, or predictable, and it rarely gives me any music I haven't already heard a million times.

    So, take home message, I'm impressed with Pandora, much more than Audioscrobbler / Last.fm.

  • http://www.createdigitalmusic.com Peter Kirn

    I've had the same experience, Andrew. That makes sense, because of how differently these recommendation engines are built. Last.fm is guessing based on what other people are listening to; Pandora is actually analyzing the music.

    But it's not a zero-sum game; these tools can compliment one another. You can use Last.fm for concert listings, friends' music, specific genres, etc., and Pandora to find new artists and recommendations. (And I tend to go the route Tim does, and use this to buy albums. Hmmm… buy … albums. Anyone from the RIAA listening to this?)

  • Adrian Anders

    I'm a pretty big fan of Pandora, but it's pretty wonky when it comes to the real underground styles and genres. I've found a quirk where if you play an artist/band with too few connections to other artists/bands in their system, they start to spit out random bands that have NO connection to the sound you're trying to get.

    It took me weeks of conditioning to get my Evil Nine station to spit out a consistant stream of breakbeat tunes minus the club, mainstream, rock, and new age (???) that seemed to creep into the mix. That being said, the lineup while getting better is still pretty limited as their collection isn't nearly as good as what a proper DJ brings into play. It's all good though, as I did discover a few new groups that had very little connection to the scene whose sound fit the vibe very nicely.

    My Aesop Rock (underground hip-hop) station is a whole other story…. Just because I like Ace Rock, doesn't mean I like Bubba Sparxx… Damn you Pandora!!!!! ;-)

    ATA

  • http://www.myspace.com/tricil tricil

    i prefer last.fm but the recommended radio is shite. the neighborhood radio, however is where the real jems are.

  • Pingback: Musiklänkar « false messiah

  • http://lumma.org Carl Lumma

    God, I hate Pandora. Hate the service, hate whoever they sent to Play '06. Their tech isn't even as good as gnoosic, which is free.

    -Carl

  • http://www.createdigitalmusic.com Peter Kirn

    Carl, I'm impressed with gnoosic; hadn't seen that, thanks!

    I think it's all about having multiple tools. I also found the chance to sit down with Tim because he's really evangelized this idea of recommendations, evangelized Internet radio as a discovery and promotion method, and built a business around it. Which tools are best is a whole different discussion, and I have been disappointed with more obscure music — maybe that suggests that music at the fringes doesn't conform as well to the genome. It's an interesting question, and how these tools work for different musical materials could make an entirely different discussion.

    I also think there will always, always be a place for human interaction. Pandora has its own place, but there are other tools (Last.fm) that work better for community building and (stations with traditional DJs) getting personal introductions from a guru. I love great, eclectic, freeform DJs and I hope they never go away.

    There's a lot more growth and development potential here, for sure.

  • http://www.theoreticalplayground.co.uk det

    pandorafm is a funky mashup of pandora and last.fm

  • mat

    I always seem to disagree with Peter so I suppose I shouldn't be surprised to find you rate Pandora above last. I gotta say that last just gets better.Its ability to submit from your ipod and itunes makes it much more effective in connecting you with music that doesn't even exist to the last database. The fact that Pandora is a genome style project speaks loads about why it can't possible keep up with the dynamism of music ecologies…its typological v. topological …. for that reason last needs (your) time to locate your starting position within the ocean of always shifting/flowing musical experience.. As someone else has said Neighbour Radio – which was the old profile radio is the real last….it gives me a million ears and tells me which ones I should filter in/out…..I shut up now…to each his own

  • mat

    Oh..and I should say…thanks for the interview..great. Pandora is one of a few changing distribution in a networked world. Last and Emusic too..Would love to read more interviews along these lines..tah

  • http://www.createdigitalmusic.com Peter Kirn

    Who said I rank Pandora above last.fm? I love last.fm; I've only really started getting into it the last couple of weeks, actually (lots of Web things compete for my time, for reasons you might guess), but for my own listening / tastes, it's the tool I expect I'm likely to use more.

    That said, I think Pandora takes a completely different approach, so it's comparing two different things. The pandorafm mashup illustrates what I was saying, which is that these can be complementary if you like. I'll also say, I've been impressed at how many casual computer users take to Pandora — some of these people I know would find other tools more overwhelming.

    But I do hope this will be the first of a series of interviews. If anyone has any requests for interviews you'd like to see, please send them along!

  • mat

    Sorry if i misunderstood…

    requests for interviews;

    Robert Henke.

    the Emusic people

    the Last.fm people

    to state the most obvious and difficult…

    agree that last suffers from focus and simplicity that pandora doesn't…

  • http://www.musicinterfaces.com/ Colin (Music Interfa

    First up, many thanks for a really interesting interview. I agree that it's a good idea to mix and match – I'm really impressed by Last.fm's Events functionality – I've been looking for something like this for a long time – very nice user-generated listings and listings recommendations. Funny thing is they're not shouting it from the rooftops. I'm in the ongoing process of monitoring and trying out the various music recommendation services as part of my site's remit. My overview page can be found here. Hope it's of interest and thanks once again.

  • Julie

    I just want to say that Matt Nathanson is the everyone's favorite artist – they just don't know it yet!

  • Pingback: Pandora: Benutzer ausserhalb den USA werden ausgesperrt - Digitale Allmend

  • Pingback: Pandora: Benutzer ausserhalb der USA werden ausgesperrt - Digitale Allmend

  • Pingback: Create Digital Music » If Streaming Rates Stand, “We’ll Have to Shutter”, Says Pandora Founder

  • Pingback: Create Digital Music » Musical Brain API: An API for Music on the Web - And it Makes Pretty Pictures

  • Pingback: My favorite case study: the publishing value chain and how to fix it « The Other Reader

  • Pingback: Create Digital Music » Signs of Change, Ingenuity in Music Distribution

  • Pingback: Computer programming and music « LCS Computer Apps 2

  • Techo Nöller

    Tjo, die Sachverhalte können manchmal wirklich trivial sein! Besten Dank für die Erläuterungen :)