Module Overview
...
Week 1: Data Literacy
Students learn the basics of data acquisition, formatting, and data conversions. They get to know a wide variety of data sources.
Please bring your own laptop to the course!
Week 2: Reading Week
Students conduct personal research, read articles, watch videos, and listen to podcasts relevant to the topic of data visualization.
Week 3 – 5: Data Visualization
Students learn and apply the basics of data visualization within a group assignment to build their own interactive visualization.
Expectations and Grading
Grades will be based on group presentations, class participation, home assignments, documentation (journal) and final work. Contributing to constructive group feedback is an essential aspect of class participation. Regular attendance is required. Two or more unexcused absences will affect the final grade. Arriving late on more than one occasion will also affect the grade.
- 10% Participation (Data Literacy)
- 60% Final work (Data Visualization)
- 30% Documentation (Data Visualization)
Any assignment that remains unfulfilled receives a failing grade.
Deliverables
- Graphical Poster (Representation) & Interactive Protoype (Experience)
- Extensive Documentation as PDF (Process)
- Assets for IAD Documentation (Images & Videos, Text)
Graphical Poster
We expect all groups to deliver and present at least a static data visualization of an appropriately sized data set which is relevant to the overarching topic of instability. The visualization should make the data set accessible to people not familiar with the subject matter and highlight important aspects like trends, patterns, or outliers. The poster should include, but is not limited to, the following content: Title, subtitle, introduction, visualization with scales, legends, annotations, methodology, sources, credits.
Interactive Prototype
More advanced groups are asked to deliver a prototype of an interactive version of the visualization represented on the poster. The prototype should illustrate the intended functionality in terms of interaction and animation.The prototype can be built using any of, but not limited to, the following tools and technologies: Invision, Principle, Adobe After Effects, Adobe Animate, HTML, CSS, JS, etc.
Course Materials
Essentials
Add a short list of specific articles, chapters, videos, podcasts, etc. here.
Books
...
Articles
...
Websites
Podcasts
- Data Stories by Moritz Stefaner and Enrico Bertini
Tools
A variety of useful toolkits have been designed to help support information visualization applications. Some include support for the full visualization pipeline from data to interactive graphics, while others focus only on a subset, typically graphics and interaction.
Visualization Cheatsheets
Visualization Toolkits
...
Visualization Tools
...
Visualization Programming Environments
- Bl.ocks: Code examples, hosting, and reviews
- Bl.ocks Explorer: Explore and discover bl.ocks based on D3 API calls
- Block Builder: Environment to create, edit, and fork D3 examples
Network Analysis Tools
...
Color Tools
Data Literacy Resources
General / good reads
Data quality
- The Quartz Guide To Bad Data
- Tidy Data – although the principle of tidy data stems from an R developer and the examples in this document are made in R, "tidy data" is a very valuable standard that you should achieve when working with data. Once your data is "tidy", visualization in R (or in any other language / framework) becomes easier. You can also look at the more formal and less R-heavy scientific paper.
Data formats / conversions
- http://www.convertcsv.com for converting data from a myriad of formats into others, e.g. from CSV to JSON.
- http://www.reformattext.com for formatting text files and extracting information out of them.
- https://jsonformatter.curiousconcept.com/ for validating and formatting JSON data.
- An introduction to APIs
- How to use APIs with Python – The normal way of working with APIs is via a programming language. Python lends itself to communicating with many APIs, so once you know a bit of Python, this interactive course might be good for you.
Regex
- https://regexr.com/: Interactively test and learn Regex
- This tutorial doesn't look too bad: Learn Regex the Easy Way
Geospatial data
- An overview of common spatial data formats
- QGIS is an open source geographical information system and a good option for working with geodata on a non-regular basis (e.g. to look at some data or to compute areas of some regions). This tutorial collection will introduce you to it.
- https://tools.retorte.ch/map/: Convert coordinates into/from the Swiss projection system.
- Municipal boundaries and districts of Swiss cities by the BFS. Jeremy Stucki (former IAD student) has processed these and other data into TopoJSON (easier to use for web applications).
Working with the CLI
The command line interface (CLI, often referred to as "shell" or "terminal") is a very powerful tool, and each operating system has one. Working with the CLI is easiest on Linux and Mac, and they are both similar since they are both based on Unix. I really recommend getting to know the 101 of working with the terminal, e.g. through this tutorial. Here are some more tips for working with data on the CLI. This Twitter account gives useful and sometimes funny tips on how to make the most of the terminal.
Data processing in R
- Data Analysis with R: Recent and good introduction, no prior knowledge needed
- rddj.info: Resources for doing data journalism with R
Calendar
...
Week 1
...
Tuesday 31.10
...
Wednesday 1.11
...
Thursday 2.11
...
Friday 3.11
...
Data Literacy
09.30-17.00
4.D12, TG
Introduction
From Data to Knowledge
Data Sources
Please bring your own laptop to the course!
Data Literacy
09.30-17.00
4.D12, TG
Data Sources & Quality
Data Types / Formats
Data Tools / Working with Data
...
Week 2
...
Tuesday 7.11
...
Wednesday 8.11
...
Thursday 9.11
...
Friday 10.11
...
Data Literacy
09.30-17.00
4.D12, TG
Data Quality
More Data Tools
Geospatial Data
tbd.
...
Data Visualization
Introduction & Brief
09.00-12.0
4.D12, BW, JG
Research
...
Design Input 1
Basic Techniques
09.00-12.00
4.D12, BW
Research
...
Tech Input I
09.00-12.00
4.D12, JG
Mentoring
13.00-15.00
Atelier, BW, JG
Analysis
...
Week 3
...
Tuesday 14.11
...
Wednesday 15.11
...
Thursday 16.11
...
Friday 17.11
...
Analysis
...
Analysis
...
Concept
...
Concept
...
Week 4
...
Tuesday 21.11
...
Wednesday 22.11
...
Thursday 23.11
...
Friday 24.11
...
B&A
...
Design Input 2
Intermediary Techniques
09.00-12.00
4.D12, BW
Mentoring
13.00-15.00
Atelier, BW, JG
Concept
...
Tech Input 2
09.00-12.00
4.D12, JG
Concept
...
Aesthetics of Interaction
09.00 - 12.00
Concept
...
Mentoring
09.00-12.00
Atelier, BW, JG
Production
...
Week 5
...
Tuesday 28.11
...
Wednesday 29.11
...
Thursday 30.11
...
Friday 1.12
...
B&A
...
Production
...
Aesthetics of Interaction
09.00 - 12.00
Mentoring
13.00-17.00
Atelier, BW, JG
Production
...
Production
Final Presentation
09.00-12.00
4.D12, BW, JG
Documentation
TG: Timo Grossenbacher, BW: Benjamin Wiederkehr, JG: Joël Gähwiler
Inputs
...
Data Visualization Foundation
- Purpose
- History
- State of the Art
- Future Frontiers
...
Basic Techniques
- Perception
- Cognition
- Encoding
- Visualization Types
...
Intermediary Techniques
- Color
- Interaction
- Animation
- Exploration
- Explanation
...
- Online Tools (Plotly)
- Data In & Out
...
- Programming
- Export => Illustrator
...
Assignment
The students collaborate in groups of three people. We recommend to form cross-functional groups where technically-oriented people team up with design-oriented people. We propose the following process to achieve the expected results in time.
- In the first phase, the discovery phase, the students explore indicators of instability in the world around us. Such indicators can be found in economics, politics, or finances, but also in social diversity or climate phenomena. The students will research data sets which hold the potential to describe or even explain this instability.
- In the second phase, the definition phase, the students will conduct a visual analysis of the selected data sets to find their key insight. This can include visualizing different facets of a data set, visualizing a data set with different graphical methods, or putting different data sets in relation to each other. The students will define a question which they can answer using the right data set and the right visualization technique.
- In the third phase, the development phase, the students design and develop their visualization into a final visual artifact. This is either a graphical poster(minimum requirement) or an interactive prototype(advanced requirement). Depending on the student composition and the intended reader / user experience, the priority between these two artifacts can be defined for each group individually.
- In the fourth phase, the delivery phase, the students will exhibit and present their work to the rest of the class. Besides the final result, we’re interested in the overall development process from interest to question to answer. These stages should be documented as a visual journal including sketches, mockups, and prototypes.
Deliverables
Graphical Poster and Interactive Prototype
Documentation as PDF
Documentation for the web(text, images, videos)
Graphical Poster
We expect all groups to deliver and present at least a static data visualization of an appropriately sized data set which is relevant to the overarching topic of instability. The visualization should make the data set accessible to people not familiar with the subject matter and highlight important aspects like trends, patterns, or outliers. The poster should include, but is not limited to, the following content: Title, subtitle, introduction, visualization with scales, legends, and annotations, as well as methodology, sources, credits. Format: A3 Portrait
Interactive Prototype
More advanced groups are asked to deliver a prototype of an interactive version of the visualization represented on the poster. The prototype should illustrate the intended functionality in terms of interaction and animation. The prototype can be built using any of, but not limited to, the following tools and technologies: InVision, Principle, Adobe After Effects, Adobe Animate, HTML, CSS, JS, etc. Format: Clickable Prototype
Documentation
The documentation should include:
- Abstract
- Introduction into the topic and research question
- Data analysis with sources and methodology
- Design and development process with stages and variations
- Result with explanation and evaluation
- Conclusion and outlook
Grading
Grades will be based on group presentations, class participation, home assignments, documentation (journal) and final work. Contributing to constructive group feedback is an essential aspect of class participation. Regular attendance is required. Two or more unexcused absences will affect the final grade. Arriving late on more than one occasion will also affect the grade.
- 10% Participation (Data Literacy)
- 60% Final work (Data Visualization)
- 30% Documentation (Data Visualization)
Any assignment that remains unfulfilled receives a failing grade.
Calendar
Week 1 | Tuesday 31.10 | Wednesday 1.11 | Thursday 2.11 | Friday 3.11 |
|---|
|
|
| Data Literacy
09.30-17.00
4.D12, TG Introduction From Data to Knowledge Data Sources Please bring your own laptop to the course! | Data Literacy
09.30-17.00
4.D12, TG Data Sources & Quality Data Types / Formats
Data Tools / Working with Data
|
Week 2 | Tuesday 7.11 | Wednesday 8.11 | Thursday 9.11 | Friday 10.11 |
|---|
| Data Literacy
09.30-17.00
4.D12, TG Data Quality More Data Tools
Geospatial Data
tbd. | Data Visualization
Introduction & Brief 09.00-12.0
4.D12, BW, JG Research | Design Input 1
Basic Techniques 09.00-12.00
4.D12, BW Research | Tech Input I 09.00-12.00
4.D12, JG Mentoring
13.00-15.00
Atelier, BW, JG Analysis |
Week 3 | Tuesday 14.11 | Wednesday 15.11 | Thursday 16.11 | Friday 17.11 |
|---|
| Analysis | Analysis | Concept | Concept |
Week 4 | Tuesday 21.11 | Wednesday 22.11 | Thursday 23.11 | Friday 24.11 |
|---|
B&A | Design Input 2
Intermediary Techniques 09.00-12.00
4.D12, BW Mentoring
13.00-15.00
Atelier, BW, JG Concept | Tech Input 2 09.00-12.00
4.D12, JG Concept | Aesthetics of Interaction
09.00 - 12.00 Concept | Mentoring
09.00-12.00
Atelier, BW, JG Production |
Week 5 | Tuesday 28.11 | Wednesday 29.11 | Thursday 30.11 | Friday 1.12 |
|---|
B&A | Production | Aesthetics of Interaction
09.00 - 12.00 Mentoring
13.00-17.00
Atelier, BW, JG Production | Production | Final Presentation
09.00-12.00
4.D12, BW, JG Documentation
|
TG: Timo Grossenbacher, BW: Benjamin Wiederkehr, JG: Joël Gähwiler
Inputs
| Input | Date | Content | Instructor |
|---|
| Introduction | 8.11 | Data Visualization Foundation - Purpose
- History
- State of the Art
- Future Frontiers
| BW |
| Design Input 1 | 9.11 | Basic Techniques - Data Properties
- Graphic Properties
- Human Properties
- Graphical Encoding
- Graphical Methods
- Graphical Excellence
| BW |
| Design Input 2 | 21.11 | Intermediary Techniques - Color
- Interaction
- Animation
- Exploration
- Explanation
| BW |
| Technology Input 1 | 10.11 | - Online Tools (Plotly)
- Data In & Out
| JG |
| Technology Input 2 | 22.11 | - Programming
- Export => Illustrator
| JG |
TG: Timo Grossenbacher, BW: Benjamin Wiederkehr, JG: Joël Gähwiler
Module Materials
Books
- The Visual Display of Quantitative Information(2nd Edition), E. Tufte. Graphics Press(2001)
- Envisioning Information, E. Tufte(2005)
- Visual Thinking for Design, Colin Ware, Morgan Kaufman(2008)
- Interactive Data Visualization for the Web(2nd Edition), Scott Murray, O’Reilly(2017)
- Visualization Analysis and Design, Tamara Munzner, CRC Press(2014)
- The Functional Art, Alberto Cairo, New Riders(2012)
- Design for Information, Isabel Meirelles, Rockport(2013)
Websites
Podcasts
- Data Stories by Moritz Stefaner and Enrico Bertini
- Policy Viz by Jon Schwabish
- Especially Big Data by Fathom
- Data Skeptic by Kyle Polich
Videos
Articles
On ethics:
On color:
On interactivity:
Tools
A variety of useful toolkits have been designed to help support information visualization applications. Some include support for the full visualization pipeline from data to interactive graphics, while others focus only on a subset, typically graphics and interaction.
Visualization Cheatsheets
- Data Viz Project by Ferdio
- Visual Vocabulary by Financial Times
- Chart Choser by Juice Analytics
- Ggplot2 Cheatsheet– Cheatsheet for ggplot2(see below)
Visualization Toolkits
- D3– JavaScript library for data-driven DOM manipulation, interaction and animation. Includes utilities for visualization techniques and SVG generation.
- Vega– Declarative language for representing visualizations. Vega will parse a visualization specification to produce a JavaScript-based visualization, using either HTML Canvas or SVG rendering. Vega is particularly useful for creating programs that produce visualizations as output.
- Vega-Lite– High-level visualization grammar that compiles concise specifications to full Vega specifications.
- Processing or p5.js– Java-like graphics and interaction language and IDE. Processing has a strong user community with many examples. p5.js is a sister project for JavaScript.
- Leaflet– Open-Source mapping library
Visualization Tools
- Tableau for Students – Free version of Tableau for students
- Tableau Public– Free version of Tableau for publishing on the web
- Voyager and Polestar– Web-based data exploration tools from UW's Interactive Data Lab
- Lyra– Interactive visualization design environment
- GGplot2– Graphics language for R
- GGobi– Classic system for visualizations of multivariate data
Visualization Programming Environments
- Bl.ocks: Code examples, hosting, and reviews
- Bl.ocks Explorer: Explore and discover bl.ocks based on D3 API calls
- Block Builder: Environment to create, edit, and fork D3 examples
Network Analysis Tools
- Gephi– Graph analysis application for Windows, Mac, and Linux
- SNAP– Graph analysis library for C++ and Python
Color Tools
- Chroma.js– Javascript library for dealing with colors
- D3.js– Javascript library with modules for dealing with colors
- HCL Wizard– Tool for viewing, manipulating, and choosing HCL color palettes
- I Want Hue– Tool for generating and refining palettes of optimally distinct colors
- Colorbrewer– Tool for finding sequential, diverging, and qualitative color palettes
- Color Picker for Data– Tool for picking color palettes
- Accessible Color Matrix– Tool for building accessible color palettes
- Contrast Finder– Tool for finding good contrasts between two colors
- Chromaticity– Guidance for accessible visualization color design
- Color Oracle– Free color blindness simulator for Window, Mac and Linux
Data Literacy Resources
General / good reads
Data quality
- The Quartz Guide To Bad Data
- Tidy Data – although the principle of tidy data stems from an R developer and the examples in this document are made in R, "tidy data" is a very valuable standard that you should achieve when working with data. Once your data is "tidy", visualization in R (or in any other language / framework) becomes easier. You can also look at the more formal and less R-heavy scientific paper.
Data formats / conversions
- http://www.convertcsv.com for converting data from a myriad of formats into others, e.g. from CSV to JSON.
- http://www.reformattext.com for formatting text files and extracting information out of them.
- https://jsonformatter.curiousconcept.com/ for validating and formatting JSON data.
- An introduction to APIs
- How to use APIs with Python – The normal way of working with APIs is via a programming language. Python lends itself to communicating with many APIs, so once you know a bit of Python, this interactive course might be good for you.
Regex
- https://regexr.com/: Interactively test and learn Regex
- This tutorial doesn't look too bad: Learn Regex the Easy Way
Geospatial data
- An overview of common spatial data formats
- QGIS is an open source geographical information system and a good option for working with geodata on a non-regular basis (e.g. to look at some data or to compute areas of some regions). This tutorial collection will introduce you to it.
- https://tools.retorte.ch/map/: Convert coordinates into/from the Swiss projection system.
- Municipal boundaries and districts of Swiss cities by the BFS. Jeremy Stucki (former IAD student) has processed these and other data into TopoJSON (easier to use for web applications).
Working with the CLI
The command line interface (CLI, often referred to as "shell" or "terminal") is a very powerful tool, and each operating system has one. Working with the CLI is easiest on Linux and Mac, and they are both similar since they are both based on Unix. I really recommend getting to know the 101 of working with the terminal, e.g. through this tutorial. Here are some more tips for working with data on the CLI. This Twitter account gives useful and sometimes funny tips on how to make the most of the terminal.
Data processing in R
- Data Analysis with R: Recent and good introduction, no prior knowledge needed
- rddj.info: Resources for doing data journalism with R