Data Sonification - Hearing Covid Data

Data sonification is the representation of data as sound, and is something I was looking into a while back, in the context of the indices of deprivation. I never really managed anything satisfactory, so parked the idea, and moved on to other things.

As Covid-19 spread around the globe, there were pockets of good things happening in the world of data. One of these was the Scottish Tech Army - a not for profit organisation created by furloughed IT workers to use their skills for good. They used Scottish Government data to create a dashboard of coronavirus-related statistics. One of the features of this dashboard is the ability to listen to the line charts - hearing the change over time. The Guardian covered this 'sonic interface' in an article in September.

I figured it would be good to try and do similar for England - I already had nightly processors set up munging the data at national and local level for my map explorer, so I dusted off my previous scripts to sonify IMD data, and after a fair bit of messing around with various apps, I made this:

I was well-chuffed with the outcome, and it seemed to generate a bit of interest on twitter / reddit / facebook. Most of it was people asking 'when's the drop?', but there were also messages like this:

In light of this, this post sets out the various tools and scripts I used to make this, from start to finish, so that other people can reproduce and build on it.

Firstly - if you don't want the detail, and just want to know the tools used here's a high-level overview:

Alternatively, if you want a bit of a headstart, all the files that I mention in this blog post are in this github repo. Feel free to clone / fork / etc. The sound file and gif are there as well, and the Garage Band project.

And that's it. You could stop now, and go and play around with these tools (or others), and make your own tracks. But if that's still not enough info, then here are the gory details...

Step 1 - Data Processing and making music

I use R to process the data. R was originally a statistical programming language, but it's become much more than that, and is a hugely powerful tool for anyone working with data.

The Covid-19 data is published by the UK government through the Coronavirus Dashboard. As well as the charts and downloads available from the dashboard, there's also an API. For this, though, I'm using R to programmatically get the latest csv file.

The full script for getting the data is below, and *should* work if you paste it into a new Rstudio session (you'll need to install any packages that you don't already have). I've tried to add comments inline, so if you are familiar with R, then hopefully this should be fairly easy to understand.

The basic flow is:

  1. Get the data
  2. Get the data into the right shape
  3. Deal with Buckinghamshire 🙄
  4. Calculate rolling rates
  5. Make the music
  6. Make the animated line chart

The outputs from this script are a wav file:

and the animated line chart:

Animated line chart showing covid case rates over time in England

Step 2 - Adding drums and other effects

Garage Band is an application that comes with Macbooks, iPads and iPhones, and is free. I'd dabbled with it in the past, but never really been able to do anything with it. I figured it would be interesting to see what it could do with my .wav file.

Screenshot of Garage Band app

To do this, I dragged my wav file in as an audio track, and then added a drummer. GarageBand seems to be incredible at putting beats against other tracks that you take in. This isn't the place for a tutorial - if you have a Mac or iPad, just give it a go, and go to YouTube for the basics.

For this track, I chose a Magnus, a Big Room EDM drummer to accompany the track, and set the speed to 240bpm, the same as my Covid track. And for the Covid data, I chose some Surf-style guitars. I've put the .band file on the repo, but not sure whether it contains all media, or just settings.

After messing about for a bit, I exported the song as an mp3 file, and here's the end result:

So that's the track made. My first attempt at putting it out into the wild was just like this, but I realised that it would really benefit from something visual as well, which is where the animated line chart comes in. Putting the two together was a bit of a challenge, but once I'd worked it out, it's fairly straightforward.

Step 3 - Putting the song and animated chart together

The final step, then, is to put the two together. I'm an Adobe Creative Cloud subscriber, so I used Premiere Rush to do this. It's probably overkill, to be honest, but it was easy enough to do.

Screenshot of Adobe Premiere Rush app

To put it all together, I added the mp3 file, and the animated gif, and then made an end credits picture in Powerpoint for when the drums are fading out. I then exported it as an mp4 file saved to disk, and put a copy on YouTube, which you can see here.

Next Steps

There's a few places I'd like to go with this, I think.

If you manage to use any of the information or resources in this post - I would really love to hear about it. Best place is twitter (@northernjamie), but comments below, and email is good as well. If there are any errors here, then please please let me know, and I'll correct them.

Thanks for reading..!