Principles of Data Visualization

Why is data visualization important?

Anscombe’s quartet highlights the importance of data visualization and warns against relying solely on summary statistics for data analysis.

  1. In a Python notebook (or not), read the data in.
  2. What’s the mean and sd for x and y for each group?
  3. Plot a scatter plot of x and y for each group

Anscombe’s quartet

import pandas as pd
import seaborn as sns
anscombe = pd.read_csv("data/anscombe.csv")
anscombe
   experiment  group   x      y
0           A      1  10   8.04
1           A      2  10   9.14
2           A      3  10   7.46
3           A      4   8   6.58
4           B      1   8   6.95
5           B      2   8   8.14
6           B      3   8   6.77
7           B      4   8   5.76
8           C      1  13   7.58
9           C      2  13   8.74
10          C      3  13  12.74
11          C      4   8   7.71
12          D      1   9   8.81
13          D      2   9   8.77
14          D      3   9   7.11
15          D      4   8   8.84
16          E      1  11   8.33
17          E      2  11   9.26
18          E      3  11   7.81
19          E      4   8   8.47
20          F      1  14   9.96
21          F      2  14   8.10
22          F      3  14   8.84
23          F      4   8   7.04
24          G      1   6   7.24
25          G      2   6   6.13
26          G      3   6   6.08
27          G      4   8   5.25
28          H      1   4   4.26
29          H      2   4   3.10
30          H      3   4   5.39
31          H      4  19  12.50
32          I      1  12  10.84
33          I      2  12   9.13
34          I      3  12   8.15
35          I      4   8   5.56
36          J      1   7   4.82
37          J      2   7   7.26
38          J      3   7   6.42
39          J      4   8   7.91
40          K      1   5   5.68
41          K      2   5   4.74
42          K      3   5   5.73
43          K      4   8   6.89

Mean and sd per group

anscombe.groupby('group').agg(
    mean_x=('x', 'mean'),
    sd_x=('x', 'std'),
    mean_y=('y', 'mean'),
    sd_y=('y', 'std')
).reset_index()
   group  mean_x      sd_x    mean_y      sd_y
0      1     9.0  3.316625  7.500909  2.031568
1      2     9.0  3.316625  7.500909  2.031657
2      3     9.0  3.316625  7.500000  2.030424
3      4     9.0  3.316625  7.500909  2.030579

Plots per group

sns.FacetGrid(
    anscombe, col="group", col_wrap=2, height=4
    ).map(sns.scatterplot, "x", "y", s=100)

Visual Perception

Edges, contrasts, and colors

  • Our visual system (i.e., all the visual processing models from our eyes to our brain) re-constructs what we are looking at
  • Our visual system uses relative differences instead of absolute values for brightness and colors
  • This relativization of brightness is demonstrated in the image in the next slide, in which we perceive darker blobs where grid lines meet at the edge of our visual field

Herman grid effect

Herman grid effect

  • At the very center of the visual field, we see the lines as they are (just white).
  • The center is where our vision is really good
  • Our ability to distinguish details decreases at the edges of our visual field (Ware, 2008)
  • Visual illusions (like the image in the previous slide) are side effects of our visual system processing.

Contrast

  • Another side effect of our visual system processing is perceiving the same shade as lighter or darker depending on the background behind shade
  • In the image on the next slide, the smaller squares are all the exact shade of gray
  • We perceive the gray to be lighter in darker backgrounds and darker in lighter backgrounds

Contrast

Edges, colors, and contrasts

  • Our visual system is not a physical light meter, what we perceive is different from actual values for colors and brightness
  • Humans are better at seeing edge contrasts for monochrome images
  • We are also better at distinguishing dark shades (compared to light shades)

Color

Let’s start with some definitions regarding color. “Color” refers to three components:

  • Hue refers to the dominant wavelength of the color (e.g., red, blue, purple)
  • Chroma refers to how intense of vivid the color is
  • Luminance refers how bright a color is

HSL

Color

  • Color schemes need to represent accurately differences in our data.
  • For example, changes from one level to next need to be perceive as having the same magnitude
  • We should be careful when picking colors for our variables in our data visualizations

Order these colors

Order these colors

Color

  • Gradients should be used as sequential scales from low to high to represent numeric continuous/ordered variables (e.g., population size). Light colors should represent low values, and dark colors high values.

Color

  • Diverging scales should be used where there is a neutral midpoint and then there is variance in ether direction from that neutral point (e.g., temperatures can be positive and negative).

Color

  • Qualitative palettes where each color has the same valence (i.e., no color dominates the other) should be used for unordered categorical variables (e.g., political parties, countries). Different colors in the palette should not imply differences in magnitude.

Color

  • We also need to take into account people who are color-blind, and how they perceive colors

Shape

  • We can also use shape as a channel to encode information
  • The color channel, however, is more salient, and shape can be used as a secondary channel.

Visual Search and Pop-out Effect

Try to find the type B data point in the three plots in the next slides

Visual Search and Pop-out Effect

Try to find the type B

Visual Search and Pop-out Effect

Try to find the type B

Visual Search and Pop-out Effect

Try to find the type B

Color and shape

  • The type B data point should have been easier to find in the first and last plots, and harder in the middle plot.
  • Color is a channel that is easier to process than shape
  • Combining both shape and color is at times a good strategy.

Color and shape

  • Encoding multiple channels in the same visualization can overtax the viewer when the multiple channels are not highlighting some structure in the data

see the two plots on the next slides

Color and shape – overtaxing

Color and shape – overtaxing

Encoding channels

Other encoding channels that are even less easily processed than shape are:

  • size
  • elongation
  • angle
  • movement

Gestalt Principles

  • humans are extremely good at finding patterns, even when no pattern exists
  • the ability to perceive different shapes and lines as a unified image rather than just a bunch of elements that are not related to each other follow what we call Gestalt principles (Healy, 2018).

Gestalt Principles

Visual elements are perceived as being related to each other when these elements are close to each other – i.e., Proximity principle:

Gestalt Principles

Visual elements are perceived as being related to each other when these elements are similar in shape, color and size – i.e., Similarity principle:

Gestalt Principles

Visual elements are perceived as being related to each other when these elements are similar in shape, color and size – i.e., Similarity principle:

Gestalt Principles

Visual elements are perceived as being related to each other when these elements are visually tied – i.e., Connection principle:

Gestalt Principles

Visual elements are perceived as being related to each other when these elements are perceived as being part paths, lines, curves, even when some elements are “hidden” – i.e., Continuity principle:

Gestalt Principles

  1. not complete; we construct the incomplete form into familiar shapes – i.e., Closure principle