Beauty and the Beast: is there any difference between listening to MP3 vs CD quality?

Standard

TL;DR: yes. But come on! There’s a bunch of graphs and some lame jokes if you actually read the post.

Preface

As I sit here at my desk, I am surrounded by audio equipment and CDs. Spotify is open right now (streaming quality set to “Extreme,” thank you very much). My favorite pair of headphones are within arm’s reach. My studio monitors are effortlessly reproducing a lovely Terry Riley piece. Clearly, I am spoiled. But wait, let’s rewind a moment: I’ve got a stack of CD’s next to me, but I’m streaming compressed audio when I could be enjoying clean, uncompressed audio from my CDs? Why would I do that? (I also have a record player and a few choice vinyls, but it’s an obviously inferior format to CD so it’s not part of the comparison.)

I do it because it’s convenient. And there’s a massive amount of diversity on Spotify that simply isn’t legally accessible to me given my grad student budget. And I’m not alone: a whole heck of a lot of people in the US use streaming services. But all of them, save one, stream in what’s called lossy formats. In fact, other than listening to a CD or vinyl, the music you listen to is probably in a lossy format. It means the previously uncompressed and pristine digital audio of a CD is reduced not just in file size, but in information it contains.WAVs, by comparison, are lossless. It’s kind of bonkers to think, but MP3s and other lossy formats throw away a LOT of sound. That’s partially why they’re so small. The goal, of course, is to only throw away things you can’t hear.

It might sound kind of like science fiction (or the fantasy of scared parents of metal fans): unheard sounds in recordings? It’s true, though. In fact, our cognitive systems are really excellent at filtering out unwanted noise. It’s called the cocktail party effect. So why not automate the process and only save the parts that we hear anyway? It might not be that simple. I, along with a classmate and our advisor, decided to test if there was a difference in the subjective enjoyment of music listening between WAVs and MP3s.

The Experiment

We selected eight songs: four recorded before MP3s were even a glimmer in the Fraunhofer Institute’s eye, and four very recent songs. We did this because there’s an idea floating around in audio engineering and audiophile circles that, for example, the Beatles sound better on vinyl than CD because the albums were recorded for the idiosyncrasies of vinyl in mind. The easiest way to control for this was to have two “early” songs and two “recent” songs as MP3 and another set of two and two as WAVs.

The Song List

  • Aretha Franklin – RESPECT
  • Michael Jackson – Thriller *
  • The Eagles – Hotel California
  • The Beatles – Help! *
  • Carly Rae Jepsen – Call Me Maybe
  • Sia – Chandelier *
  • Rihanna – We Found Love
  • Daft Punk – Get Lucky *

* = MP3, 128k, LAME encoder

Note: the oldest available CD mastering was used for the pre-MP3 songs to eliminate / reduce the chance that some modern mastering techniques would be used to make it more MP3 friendly. For example, “Hotel California” was sourced from the original CD release in 1989.

We had people come in, put on headphones we provide them with, and listen to all 8 songs presented to each person in a random order. After each song, they would rate how positive it made them feel, how negative it made them feel, and how much they enjoyed it. The reason we asked positive and negative separately is because we conceptualize those feelings as representing activations of appetitive or aversive systems, respectively. They can activate separately or they can activate together.

Keep in mind, we told the participants nothing about the sound quality, MP3s or WAVs. As far as they knew, they just had to listen to 8 songs and respond to those 3 questions for each.

Results

I instigated this experiment because I didn’t think there would be a difference. We ended up hypothesizing that there would be a difference between the formats, such that people would like WAVs more. But to be honest I was skeptical, even if I had a theory-driven rationalization as to why I thought it would come out this way. (More on that later.) I thought people might even prefer MP3s since our participants are young and have probably been listening to MP3s their whole lives, give or take.

H1 figure.png

F(1, 17) = 2.162, p = 0.16

The graph above shows the mean positivity results by Format. It’s not statistically significant, but it is in the direction we predicted. Admittedly, this one result alone isn’t convincing. But wait — there’s more!

H2 figure.png

F(1, 17) = 5.224, p < 0.05

And this is a prime example of why we split out positivity and negativity into two measurements: the negative scores are significant, and support our hypothesis that people would like MP3s less.

H3 figure.png

F(1, 17) = 1.7, = 0.21

Again, not statistically significant findings here but the data are trending in the direction we predicted.

RQ1 figure.png

F(1,17) = 5.285, p < 0.05

And here’s the kicker: people rated early era songs as MP3s more negatively than anything else. And this finding is statistically significant.

Discussion

So what gives? Well, it could be as simple as our participants just hated “Thriller” and “Help!” as songs. But more than they hated The Eagles‘ “Hotel California?” I sincerely doubt it. But it is possible, I’ll admit that openly.

Here’s what I think went on, though: remember how I said that MP3s strip out a lot of information, most of which you can’t hear anyway? I bet that process is flawed. It clearly works very well, but I bet that it is imperfect and listening to MP3s is actually MORE work for your brain than uncompressed audio (like WAVs). Our minds are very lazy and, under most circumstances, seek the path of least resistance when hit with a task. If MP3s tax the cognitive systems more than WAVs because we need to actively fill in some of the missing gaps or work harder to do our usual filtering, then it seems logical that we would rate the experience more negatively.

Moving Forward

This study isn’t perfect. I would prefer to have run it with a counterbalanced design where some participants heard Song A as MP3 and others heard Song A as a WAV. That would help remove unwanted effects of the song itself. That, and while I have some ideas as to why these results came about, this experiment doesn’t prove or even directly support my ideas. I need more information before I can put that claim forward more strongly.

The good news is that we have a lot more research in the pipeline regarding audio compression and how it impacts the listening experience.

One thought on “Beauty and the Beast: is there any difference between listening to MP3 vs CD quality?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s