31 Jul An analysis of music hits across decades: 1950-2009

Lately, I’ve been wondering how music has changed across decades. Specifically, what are the distinctive features that separate music of one decade from another? To this end, I have created a dataset of the top 100 music hits per decade, spanning six decades between 1950-2009. This dataset represents derived work collected from two sources –tsort.info/music/ and the Echo Nest database. It consists of the Echo Nest’s audio attributes for these 600 songs.  For more information about the dataset and its attributes, how I created it, and how to use it (licensing and citation info), please download the latest release from my github page and refer to the README.md file.

The Echo Nest (in case you haven’t heard of it) is a music intelligence and data platform that was acquired by Spotify in 2014 – an excellent acquisition, in my opinion. From a business strategy perspective, this acquisition positions Spotify very well in the competitive domain of music recommendation/music streaming service providers (e.g., Pandora, Apple Music, Google Play Music). tsort.info, provided by Steve Hawtin et al., is a comprehensive site that uses 130 music charts as its sources for aggregating information about hit songs. More information about the Echo Nest’s song attributes is available here and here.

To replicate the results below in R, download the dataset with the decades data titled “NVDecades.csv”, from the latest release of Music-Decades-Top-100. Then make sure to set your working directory in R to the folder containing this dataset, or mention the full/absolute path of the file in the read.csv function, and follow along with the instructions in the “MusicDecades.pdf” file.

Box plots are a great way to summarize and compare distributions of data from each decade for any attribute. Here are some box plots for each decade.
Box plots of danceability:

The Echo Nest describes danceability as how suitable a track is for dancing. The Echo Nest computes this attribute using a combination of tempo, rhythm stability, beat strength, and regularity. When considering the median danceability for each decade, we notice two obvious jumps in danceability – one for the 1970s and one for the 2000s. The sudden jump in the 1970s clearly represents how the 70s ushered in the disco and disco-funk era – a departure from the rock & roll and R&B hits of the 50s and 60s.

Box plots of energy:

Energy represents how energetic the song is. While danceability is more subjective to the listener, energy is more directly dependent on the audio characteristics of the song. The Echo Nest uses a combination of loudness and segment durations to compute energy. We notice a spike in the energy once in the 60s and again in the 70s. One possible explanation for the increase in energy levels of songs in the 60s could be the popularity of amplified distortion (especially in the latter part of the 60s). As for the 70s, this was when big mixing consoles (still analog, but allowing 32 channels) started gaining popularity in recording studios, representing a big revolution in music recording. More audio content could be accommodated and EQs could be adjusted. Could this perhaps have had something to do with the increase in energy starting with the 70s?

Box plots of tempo:

We don’t notice much difference in median tempi between decades. As one might likely expect, songs in the 50s have the lowest tempi. However, an interesting thing to note is that hit songs from the 50s varied more in tempo than in the other decades, as perceived by the size of the rectangle in the box plot. In other words, the 50s exhibited a wider range of tempi.

Box plots of duration:

We don’t see anything of note here, except that songs in the 50s and 60s were slightly shorter in duration.

Box plots of acousticness:

According to the Echo Nest, acousticness is a measure of the likelihood that a song was created by acoustic means such as voice and acoustic instruments that are not electronically synthesized or amplified. So, the inclusion of electric guitars, distortion, synthesizers, auto-tuned vocals, and drum machines will considerably lower the acousticness of a song. Not surprisingly, we notice a big drop in acousticness from the 50s to the 60s when distorted amplifiers and electric guitars started becoming popular. We also notice another drop, although not as huge, between the 60s and the 70s corresponding to the advances in recording consoles as well as the use of moog synthesizers.

Box plots of valence:

Valence is a subjective measure of pleasantness and is very listener-dependent. It needs to be interpreted with caution. The Echo Nest associates valence with positivity. The slightly lower median valence for the top songs of the 50s and the 90s, when compared with other decades, is hard to explain.

Now that we have a summarized overview of how hit songs are distributed with respect to these attributes across each decade, we can ask some interesting questions:

• The box plots for danceability and energy show a similar pattern. Is a more energetic song generally more danceable or vice versa?
• Do faster songs tend to be more energetic (or more danceable)?
• How do danceability, energy, duration, and tempo affect the perceived pleasantness of a song? In other words, how are these attributes correlated with valence?

A good basic method to help address these questions is to compute cross-correlations (i.e., Pearson’s correlation coefficients).

Perhaps a better way to visualize these correlations would be either as a table of correlations or as a series of ellipses. In the second plot, the closer the ellipse is to a straight line, the more correlated the two features are. This is also indicated by its color. Additionally, the direction of correlation between any two attributes is also indicated.

Coming back to our questions, we notice that energy and danceability are positively correlated, although the correlation is weak. Again there is a weak positive correlation between tempo and energy. This suggests that faster songs and danceable songs might sound more energetic, but not always. The positive correlation between danceability and valence is much clearer, as might be expected. The more danceable a song is, the more positive or pleasant the listening experience. Another clear insight is the strong negative correlation between acousticness and energy. So, the less acoustic or more synthesized (i.e., artificially amplified, compressed etc.) a song is, the more energetic it sounds.

We can also dig deeper and examine each decade. Each decade could be thought of as a genre. This becomes clearer in retrospect. For instance, songs of the 70s when thought of as one unified schema, have a distinct sound, a strong association with disco/disco-funk etc. However, when living in the 70s and listening to music, this might not have seemed obvious, given that the 70s had a variety of music – hard rock, early progressive rock and heavy metal, disco, synth rock, funk and so on. But first, let’s find the top 5 songs with maximum danceability, across all decades.

Three of the top 5 are from the 2000s, two from the 90s, and one from the 80s. Justin Timberlake’s “SexyBack” ranks at #1 (note that this dataset contains songs only up to 2009). What are the top 5 songs with maximum energy, across all decades?

We have a better contribution from the decades here, with at least one song each from the 70s, 80s, 90s, and 2000s. Since we have an officially streamable version available on YouTube, here is Scatman John’s “Scatman“. Gloria Gaynor’s “I Will Survive” is an interesting entry, suggesting that the song remains a mood uplifter perhaps not just because of the lyrics but also because of the acoustic characteristics that make the song more energetic. What are the top 5 most pleasant songs (i.e., songs with maximum valence), across all decades?

Elvis scores high with two entries in this list. What are the top 5 songs with maximum acousticness?

As might be expected, 4 of the songs are from the 50s and the 60s. Just for fun, what are the 5 songs with minimum acousticness?

Let’s recall what lesser acousticness means. Lesser acousticness indicates that a song is more electronically synthesized or amplified. The inclusion of electric guitars, distortion, synthesizers, auto-tuned vocals, and drum machines would have considerably lowered the acousticness of a song. Ricky Martin’s “Livin’ La Vida Loca” requires special mention here. This is a great song, in no small part due to Ricky Martin’s delivery and showmanship, but also because of being a watershed moment in music recording history. This song was noted for its extreme use of dynamic range compression to increase the perceived loudness of the song. More information about the recording process behind La Vida Loca is provided in this article. We can dig deeper and examine the top songs for each decade. What are the 5 most danceable songs from the 50s?

What are the 5 most danceable songs from the 60s?

Elvis again shows up twice, another clear reflection of why he was (and possibly still is?) such a popular artist. What are the 5 most danceable songs from the 70s?

No surprises here with Chic coming on top. The Nile Rodgers/Bernard Edwards combo was legendary in bringing a unique sound that shaped and defined disco funk. What are the 5 most danceable songs from the 80s?

The fact that Michael Jackson shows up on the 80s and 70s lists indicates two things: (a) his exceptional talent as an artist and a performer, and (b) the fantastic production teams he worked with for his “Off the Wall” and “Thriller” albums. What are the 5 most danceable songs from the 90s?

Finally, what are the 5 most danceable songs from the 2000s?

I have just about scratched the surface here and would love to see other analyses with this dataset.

By: Naresh Vempala

Image credit: The Plastic Mancunian