data-visualization

# Data Visualization

Data visualization is the translation of data into visual form for the purpose of communication, exploration, or analysis. A good visualization reveals what the data has to say; a bad one obscures or distorts it. This skill covers the principles, techniques, and design decisions involved in creating honest, effective data displays, drawing primarily on Edward Tufte's principles, Leland Wilkinson's Grammar of Graphics, and Jacques Bertin's visual semiology.

**Agent affinity:** tufte (design principles, critique), nightingale (routing visualization tasks), cairo (pedagogy of visual literacy)

**Concept IDs:** data-chart-types, data-visual-design, data-misleading-graphs, data-data-storytelling

## Core Principles

### Tufte's Principles

Edward Tufte's work established the foundational principles of honest data display:

1. **Data-ink ratio.** Maximize the proportion of ink devoted to data. Every element that does not convey data is potential chartjunk -- decorative gridlines, 3D effects, gradient fills, clip art.

2. **Chartjunk.** Visual elements that do not represent data: moiré patterns, heavy gridlines, unnecessary ornamentation, 3D perspective on 2D data. Remove it.

3. **Lie factor.** The ratio of the visual effect to the numerical effect. A bar chart where a 2x increase in data is represented by a 4x increase in area has a lie factor of 2. Lie factor should be between 0.95 and 1.05.

4. **Small multiples.** Repeating the same chart structure across panels to show variation across a categorical variable. More effective than animation or overlapping colors for showing change.

5. **Data density.** The amount of data per unit area of the graphic. High data density is efficient -- sparklines, for example, embed a time series in the space of a word.

6. **Micro/macro readings.** A good visualization works at both levels: the overall pattern is visible at a glance (macro), and individual data points can be read precisely (micro).

### Bertin's Visual Variables

Jacques Bertin (1967) identified the fundamental visual channels available for encoding data:

| Variable | Data type | Effectiveness | Notes |
|---|---|---|---|
| **Position** | Quantitative | Highest | Most accurately perceived; always the first choice for the most important variable |
| **Length** | Quantitative | Very high | Bar charts; humans compare lengths well |
| **Angle/Slope** | Quantitative | Moderate | Scatterplot slopes; pie chart angles are poorly perceived |
| **Area** | Quantitative | Low-moderate | Bubble charts; humans underestimate area differences |
| **Color saturation** | Quantitative (ordered) | Low-moderate | Sequential palettes; limited dynamic range |
| **Color hue** | Categorical | High for categories | Qualitative palettes; maximum ~8 distinguishable hues |
| **Shape** | Categorical | Moderate | Point markers; maximum ~6 distinguishable shapes |
| **Texture** | Categorical | Low | Rarely used in modern visualization |

**Design implication:** Map the most important variable to position, the second most important to length or color, and use less effective channels for less important variables. Never use area or angle for the primary quantitative comparison.

### Perceptual Principles

- **Weber's law:** We perceive relative differences, not absolute ones. A 10% change is equally noticeable whether the baseline is 100 or 1000.
- **Pre-attentive processing:** Color, orientation, size, and motion are detected before conscious attention. Use these for the feature you want the viewer to notice immediately.
- **Gestalt principles:** Proximity, similarity, enclosure, connection, and continuity govern how we group visual elements. Leverage these for layout.
- **Change blindness:** Small changes in a complex display go unnoticed. If comparison is the goal, place the things being compared adjacent or overlapping.

## Chart Selection Guide

### Choosing by Data Relationship

| Relationship | Chart type | Example |
|---|---|---|
| **Distribution (1 variable)** | Histogram, density plot, box plot, violin plot | Distribution of household incomes |
| **Comparison (categories)** | Bar chart (horizontal for many categories), dot plot | Revenue by department |
| **Trend (time series)** | Line chart | Monthly temperature over 10 years |
| **Correlation (2 quantitative)** | Scatter plot | Height vs. weight |
| **Part-to-whole** | Stacked bar, treemap, waffle chart | Market share by company |
| **Ranking** | Sorted bar chart, bump chart | Top 20 countries by GDP |
| **Geospatial** | Choropleth, dot map, cartogram | COVID cases by county |
| **Flow/Network** | Sankey, alluvial, node-link diagram | Customer journey through website |
| **Composition over time** | Stacked area, streamgraph | Energy sources as share of total over decades |

### Charts to Avoid

| Chart | Problem | Better alternative |
|---|---|---|
| **Pie chart** | Angle perception is poor; cannot compare non-adjacent slices | Bar chart or dot plot |
| **3D bar chart** | Perspective distorts lengths; occlusion hides data | 2D bar chart |
| **Dual-axis chart** | Arbitrary scaling creates false correlations | Two separate panels with aligned x-axes |
| **Radar/spider chart** | Area is meaningless; axis order affects shape | Parallel coordinates or small multiples of bar charts |
| **Stacked 100% area (many categories)** | Inner series are impossible to read | Small multiples of line charts |

Pie charts are acceptable only when: (a) showing parts of a whole, (b) with 2-3 slices, (c) where the takeaway is "roughly half" or "roughly a quarter." For anything more precise, use a bar chart.

## Color

### Color Palette Types

| Type | Use | Example |
|---|---|---|
| **Sequential** | Ordered data (low to high) | Light yellow to dark blue for temperature |
| **Diverging** | Data with a meaningful midpoint | Red-white-blue for above/below average |
| **Qualitative** | Categorical data (no order) | Distinct hues for product lines |

### Color Rules