DS - Data visualization 101

The idea behind this rule is that depending on the people we’re targeting we should add more or less information to our charts. It has to do with the context the audience has before seeing the data visualization.

Ok, imagine your boss entering the room with the chart on the left and yelling Have you seen these numbers ? I’m really worried. You may think the man didn’t get his pills on time this morning right ? What are those numbers, what do they mean ? Your boss is showing something to an audience that doesn’t know the context, he didn’t realize that the audience he’s talking to needs more information, more context. If he only had added more information to the chart, maybe you could have had a chance. What about the chart on the right, with more information ?

Ok now I can understand his concern, I know he lives in PUENTE DE VALLECAS, and he must be worried due to the number of POSSESSION OF PROHIBITED WEAPONS cases in his district. It’s clear that if my boss were a police officer and I was working in the weapons related department I better know what the first chart was about!

The figure must match the idea of the message. So only when the the message you try to express is clearly identified you can think of the best figure expressing that message.

There’re examples about how to represent really complex things in a visualization, but in the end is just a matter of using the chart type that best represents the idea you wanted to communicate. For example, Imagine I’d like to represent the increment of the felonies in Madrid city during the last six months. I wouln’t be doing any favor to the message using for example a pie chart. On the other hand using a line chart could help seeing if the numbers are rising or falling. In the chart on the right I’m using a barchart with a tendency line.

There is a great article by Jami Oetting that verses on how to choose the right chart depdending on the data you want to show.

In this case the medium is a blog post that is going to be read by someone in a laptop or a home computer, but tomorrow it could be a big screen in a conference room in front dozens of people. Depending on the medium the amount of information in my figures should be different. If someone is reading a blog post, I can add more details because the reader can stop and review the details at his/her pace whereas in a presentation the slides are going to be shown in a limited period of time. It’s not only about the amount of information, it also about to prioritize which information is most valuable depending on the medium and the time.

Imagine we were in a room showing the evolution of numbers of drug posession in Madrid from March to August in 2020, using a slide like the one on the left, even people in the first rows could have a hard time figuring out which line is the one we’re talking about. Instead the chart on the right side highlights just the part I want the people to focus reducing the noise.

The caption helps to understand the figure. Imagine a simple pie chart with three areas but with no percentages, no titles telling what the areas represent, that is important information missing right ?

Another example, the chart on the left has little information, it can’t help the reader to understand what the figure is about, or what the percentages mean. However the chart on the right has the minimum information to express what the chart is about, and what the different areas represent.

By default most visualization libraries normally use a subset that can be used in a wider range of cases. Because of that, the default can lack many of important information we’ve been talking about earlier. If that happens we need to learn how to customize the library in order to add whatever additions/changes necessary to create the right message. That could include, adding captions, changing colors, completing axis information, legends…​ etc.

The following example shows how the default horizontal bar plot (left) looks like. It’s not so bad, it has the x-axis but everytime you want to know how many premises a given type has, you have to guess it, and even for the top category the x-axis is a little bit to far. Now on the right, is the same data visualized using a customized horizontal bar plot. Numbers and labels for every type makes a better use of the space. Also colors have been changed to maximize the contrast and make it easier to read. A little bit more about the color in the next section.

Using colors wisely could help to enhance your charts, and improve the understanding of the information shown. But it also could lead to a complete disaster misleading your audience from the real message if used wrong. A wrong use of the color could also become a phisical barrier, e.g. if you chose a colormap with low contrast, people with color blindness won’t be able to see the difference between colors.

Following the previous chart, I’m adding colors to it with no special reason, maybe I though it could be more appealing. Look at the chart on the left. Yeah it may look nicer, but the colors don’t add any meaningful value, and a reader could do a wrong interpretation based of the colors: maybe red means danger, green means safer…​etc. If the colors don’t mean anything, it’s better to keep it simple like the chart on the right and avoid misinterpretations.

Another practical use of colors in bar charts is to associate categories (could be companies, political parties…​) to colors, or setting threshold limits. Here you have two examples of these ideas.

Alberto Cairo in his work Graphic Lies, misleading visuals writes about what are the principal techniques used to mislead the reader, or in his words how Becoming a good liar. In his paper Cairo points out three important ideas used to mislead the reader:

Chartjunk most of the time refers to everything we could add to chart without adding anything useful or meaningful to it. We’ve already seen a good example of this:

So yes, in general chartjunk should be avoided, but sometimes, chartjunk in the form of infographics has been proved to be useful to memorize the message and the chart itself in the long term as Bateman, Mandryk, Gutwin stated in their paper Useful Junk? The Effects of Visual Embellishment on Comprehension and Memorability of Charts. Judge by yourself, which of the following charts will you remember tomorrow ?

Sometimes it’s difficult to figure out which chart will fit best our message, and is in these situations when we are tempted to improvise and create new types of visualizations. We can even create a beautiful masterpiece, but before trying to be the next Michael Angelo it’s always better to look whether somebody has already dealt with this problem before, or if maybe something that’s already there could help us with the message we’re trying to send.

The main reason is that there’re tons of scientific documents written by really smart people out there, showing hundreds of ways of visualizing data and it’s very likely that some of these works will fit somehow in our problem if not perfectly. Avoid reinventing the wheel, and remember that people is more likely to have some exposure to pre-existent scientific visualizations helping them to avoid the effort of understanding a new way of visualizing data plus the message behind the chart.

Depending on your area of expertise, the medium you’re going to use and the technology you are confortable with, there’re plenty of tools and libraries out there to visualize the data in the way that best expresses the message you’re trying to communicate.

However I would like to differenciate between the concept of libraries and tools. In the present article I’m using Matplotlib which is a Python library for data visualization but it’s been used inside a Jupyter notebook. Jupyter is a tool, but in the sense of a full featured environment where I can integrate the data I’m using with some analitic tools with some visualization libraries to show the result of my findings. Another example of such type of tool could be Weka which is another tool for loading, analyzing and visualizing data, or BeakerX which is the Java version of Jupyter. Of course this section will be incomplete without mentioning the R project, which could be considered as well a full featured data-oriented environment.

Here you can find a tar file with all the sources of the examples I did for this article. You should find both the Jupyter notebooks and the data.

If you’d like to have like a cheatsheet to quickly know which chart fits best the data you’re trying to communicate, please don’t forget to take a look to these:

Some papers and posts worth reading:

If you’d like to have a laugh watching horrendous or suspicious charts, take a look at these:

Data I’ve used to create the charts in this article:

Libraries you can use with your favorite programming language.

A library alone can’t solve all the problems trying to get some meaningful insights from data, and integrated environment normally does.