Create your own ‘Reading Stats’ page for medium.com using Python
Part 2: Presenting the data!
This article is part 2 of ‘Create your own ‘Reading Stats’ page for medium.com using Python’. Part 1 makes the argument that the ‘stats’ page in your medium account is all about the writers, and offers almost no useful stats for the compulsive readers (or just those who live a quantified life!), so I came up with a system to capture and present my own statistics. Part 1 is focused on the python script that extracts the information about each story read and saves it to a database. This part will look at the html used to present the statics (albiet via a python script). Read part 1 here:
Structuring the Data
Following on from the script detailed in part 1, we move on to a function that takes in the results of a SQL query and structures this data in a form that can easily be converted into graphical information later in the script. When referencing each result, the array positions correspond to the columns in our database.
The Counter() function is the perfect tool if you want to produce graphs. It takes in a list and counts up the occurrences of each unique item. So in our case, we are giving it a list of tags that may look something like this: ['Programming', 'Software Development', 'Programming', Software Development', 'Software Development', 'Technology']. And it then returns a dictionary that looks like this: {'Software Development:3, 'Programming':2, 'Technology':1}.
We call the getChartInfo() function here (the date variable was defined in part 1), so we now have the chart data for the partial current month, the last month and the last 6 full months:
cursor.execute('SELECT * FROM readArticles WHERE month = {} AND year = {}'.format(date.month, date.year))
currentMonth = getChartInfo(cursor.fetchall())
cursor.execute('SELECT * FROM readArticles WHERE month = {} AND year = {}'.format(date.month-1, date.year))
lastMonth = getChartInfo(cursor.fetchall())
cursor.execute('SELECT * FROM readArticles WHERE (month BETWEEN {} AND {}) AND year = {}'.format(date.month-6, date.month, date.year))
lastSixMonths = getChartInfo(cursor.fetchall())The final bit of code before we begin to write the html document is just to work out some stats for the entire length of the medium.com membership. We query the database for the number of rows in total, calculate how many months we’ve held our membership for and work out the individual article cost. (A lot of the data we will be presenting won’t make sense since I have only been a member for a month. So, all time stats are the same as ‘current month’ stats, and also the same as the ‘last 6 months’ stats, but this will change over time!)
Presenting the Data
We move on to writing the html file, this will contain all the inline CSS, JavaScript and HTML tags as one big string which we keep appending to. We call this variable html. At the end of the script, we will dump this string to a file and directly upload it to our web host. The first thing to notice is the <script> tag inside the <head> where we declare some external JavaScript that we will be using. The www.gstatic.com/charts/loader.js points to google’s visualization chart library.
When I was looking into displaying graphs for my stats page I discovered there were lots of 3rd party libraries available, but they all appeared to be either expensive or complicated. Googles Visualization is free, easy to implement and has great documentation. Here’s the tutorial page I followed while coding up the pie charts.
We next define a JavaScript function to handle the clicks from some buttons. It finds the div that the button is nested in, it then finds the following div in the list and defines it as the moreinfo div, this is set to be hidden initially (or style.display = “none”) and the button changes the display style to inline. Every time the button is pressed after the first press, the display value of the div and the text on the button is toggled. This hidden div will contain a list of the articles read with a link to the story on medium.com.
You will see <div id="daysChart"> which is an empty div but this will be filled with a google chart later on in the script. After inserting the overall stats, we set up a table with some headers, the rest of the table we will populate in the next function.
We pass in some of the calculated statistics from the getChartInfo() function we looked at earlier and put them into the table rows. The last column item is a button.
Here is how the table looks on the site and then with the expanded view.
Drawing the Graphs
OK next is the fun bit! We set up a few more empty div containers for our graphs, then we define a function which draws the charts. It takes in our Counter() dictionary and a string of the chart name. We use this string to locate the right div as well as using it in the labels for each chart.
As more and more articles were added to my reading database, the pie charts were becoming less useful, especially the ‘tags’ chart, since each article can have up to 5 tags, the chart was split into hundreds of segments which wasn’t very visually pleasing. I added the sliceVisibilityThreshold variable to the options for each pie chart which clumps all the slices that are below a certain threshold to an ‘others’ segment, this really cleaned up the data. We also add the pieHole variable to the options dictionary, which turns the Pie Chart into a polo mint, which (at least to my eyes) seemed to make the data easier to read.
This is how the graphs look on the stats page
The very last thing the script does is write the html string to file and uploads it to our web host. This project took a couple of days to write, but uploading the file to the server via the SFTP protocol took a disproportionate amount of this time. Now I have it figured, I wrote a step by step guide to help you get there quickly.
There are a few improvements I would like to make:
- The ‘number of articles read each day this month’ graph will look strange at the beginning of the month as there is only a bar for each day the script was run. So I plan on generating a bar graph that has a bar for each day of the month with a default value of zero, which will get overridden as the data becomes available.
- Currently the script is run manually when I remember. I plan to set some kind of task scheduler to automatically run the script each morning.
- The ‘authors’ chart is useless currently as I realise I don’t tend to read several articles from the same author. This will probably be deleted in favour of something more useful.
- Once I’ve completed numbers 1 & 2, I could add some stats related to days of the week. I could even plot a graph for ‘articles by day of the week’, and look at what date of each month was my highest frequency of reading.
- Look at some of the free data dashboard online tools.
Thank you so much for reading this article and I hope you’ve found it interesting. If you want to see my stats page in real time here it is!
