Data

• Portal für Schweizer Open Government Data
• Stadt Zürich Open Data Katalog
• Open-Data-Plattform öV Schweiz
• Bundesamt für Statistik

Books

• The Visual Display of Quantitative Information, Edward Tufte (2001)
• Semiology of Graphics, Jacques Bertin (1983)
• Visualization Analysis and Design, Tamara Munzner (2014)
• Design for Information, Isabel Meirelles, Rockport (2013)
• The Functional Art, Alberto Cairo, New Riders (2012)
• Interactive Data Visualization for the Web, Scott Murray (2017)

Websites

• Data to Viz
• Flowing Data by Nathan Yau
• Information Aesthetics by Andrew Vande Moere
• Information is Beautiful by David McCandless
• Visual Complexity by Manuel Lima
• Visualising Data by Andy Kirk
• Edward Tufte: Ask E.T. Forum by Edward Tufte
• Visual Business Intelligence by Stephen Few
• Tutorials & Resources by Alberto Cairo
• Points of Interest by Mapbox
• Design × Data by IDEO
• Visualizing, the Field by Various authors
• Datazar Blog
• Dataveyes Blog
• Accurat Blog
• Vizzuality Blog

Podcasts

• Data Stories by Moritz Stefaner and Enrico Bertini
• Policy Viz by Jon Schwabish
• Especially Big Data by Fathom
• Data Skeptic by Kyle Polich

Videos

• Eyeo Festival
• Visualized Conference
• Information+ Conference
• OpenVis Conference
• Tapestry Conference

Articles

On ethics

• Data Humanism by Giorgia Lupi
• Turning Data Around by Jer Thorp
• You Say Data, I Say System by Jer Thorp
• Data (v.) by Jer Thorp
• Data Will Help Us by Jonathan Harris

On color

• Subtleties of Color by Robert Simmon
• Finding the Right Color Palettes for Data Visualizations by Samantha Zhang
• How To Avoid Equidistant HSV Colors by Gregor Aisch
• Mastering Multi-hued Color Scales by Gregor Aisch

On interactivity

• Why We Do Fewer Interactives by Archie Tse
• Data Visualization and the News by Gregor Aisch (Video)
• The Problem With Interactive Graphics by Meg Miller
• The Death of Interactive Infographics? by Domenikus Baur
• In Defense of Interactive Graphics by Gregor Aisch
• Static Visualizations Do Not Exist by Domenikus Baur
• The Superpower of Interactive Datavis? A Micro-Macro View! by Domenikus Baur

Tools

Visualization Cheatsheets

• Data Viz Project by Ferdio
• Visual Vocabulary by Financial Times
• Chart Choser by Juice Analytics
• Ggplot2 Cheatsheet – Cheatsheet for ggplot2 (see below)

Visualization Toolkits

• D3 – JavaScript library for data-driven DOM manipulation, interaction and animation. Includes utilities for visualization techniques and SVG generation.
• Vega – Declarative language for representing visualizations. Vega will parse a visualization specification to produce a JavaScript-based visualization, using either HTML Canvas or SVG rendering. Vega is particularly useful for creating programs that produce visualizations as output.
• Vega-Lite – High-level visualization grammar that compiles concise specifications to full Vega specifications.
• Processing or p5.js – Java-like graphics and interaction language and IDE. Processing has a strong user community with many examples. p5.js is a sister project for JavaScript.
• Leaflet – Open-Source mapping library

Visualization Tools

• Tableau for Students – Free version of Tableau for students
• Tableau Public – Free version of Tableau for publishing on the web
• Voyager and Polestar – Web-based data exploration tools from UW's Interactive Data Lab
• Lyra – Interactive visualization design environment
• GGplot2 – Graphics language for R
• GGobi – Classic system for visualizations of multivariate data

Visualization Programming Environments

• Bl.ocks: Code examples, hosting, and reviews
• Bl.ocks Explorer: Explore and discover bl.ocks based on D3 API calls
• Block Builder: Environment to create, edit, and fork D3 examples

Network Analysis Tools

• Gephi – Graph analysis application for Windows, Mac, and Linux
• SNAP – Graph analysis library for C++ and Python

Color Tools

• Chroma.js – Javascript library for dealing with colors
• D3.js – Javascript library with modules for dealing with colors
• HCL Wizard – Tool for viewing, manipulating, and choosing HCL color palettes
• I Want Hue – Tool for generating and refining palettes of optimally distinct colors
• Colorbrewer – Tool for finding sequential, diverging, and qualitative color palettes
• Color Picker for Data – Tool for picking color palettes
• Accessible Color Matrix – Tool for building accessible color palettes
• Contrast Finder – Tool for finding good contrasts between two colors
• Chromaticity – Guidance for accessible visualization color design
• Color Oracle – Free color blindness simulator for Window, Mac and Linux

Data Literacy Resources

Slides

Day 1: https://docs.google.com/presentation/d/e/2PACX-1vSq7ytkZLPS-Nbs5JYbeKG8EuYotCxRgUewmbgWWWCqGIV-KUX4AXCa-_5bbdYNilRd46n6p3F9IbmT/pub?start=false&loop=false&delayms=3000

General / good reads

• «The world’s most valuable resource is no longer oil, but data» vs. «Data is not the new oil»

• Data Scientist: The Sexiest Job of the 21st Century

Data quality

• The Quartz Guide To Bad Data
• Tidy Data – although the principle of tidy data stems from an R developer and the examples in this document are made in R, "tidy data" is a very valuable standard that you should achieve when working with data. Once your data is "tidy", visualization in R (or in any other language / framework) becomes easier. You can also look at the more formal and less R-heavy scientific paper.
  

Data formats / conversions

• http://www.convertcsv.com for converting data from a myriad of formats into others, e.g. from CSV to JSON.
  
• https://json-csv.com/ Another pretty good JSON-to-CSV-converter.
• http://www.reformattext.com for formatting text files and extracting information out of them.
  
• https://jsonformatter.curiousconcept.com/ for validating and formatting JSON data.
• An introduction to APIs
• How to use APIs with Python – The normal way of working with APIs is via a programming language. Python lends itself to communicating with many APIs, so once you know a bit of Python, this interactive course might be good for you.

Regex

• https://regexr.com/: Interactively test and learn Regex
• This tutorial doesn't look too bad: Learn Regex the Easy Way

Geospatial data

• An overview of common spatial data formats
• QGIS is an open source geographical information system and a good option for working with geodata on a non-regular basis (e.g. to look at some data or to compute areas of some regions). This tutorial collection will introduce you to it.
• geojson.io can be used to quickly visualize GeoJSON. Mapshaper can be used to generalize geometries and reduce file size, and it can convert between different file formats.
• ogr2ogr (and ogrinfo) are two very helpful CLI tools – just google for some tutorials!
• https://tools.retorte.ch/map/: Convert coordinates into/from the Swiss projection system.
• Municipal boundaries and districts of Swiss cities by the BFS. Jeremy Stucki (former IAD student) has processed these and other data into TopoJSON (easier to use for web applications).

Working with the CLI

The command line interface (CLI, often referred to as "shell" or "terminal") is a very powerful tool, and each operating system has one. Working with the CLI is easiest on Linux and Mac, and they are both similar since they are both based on Unix. I really recommend getting to know the 101 of working with the terminal, e.g. through this tutorial. Here are some more tips for working with data on the CLI. This Twitter account gives useful and sometimes funny tips on how to make the most of the terminal.

Data processing in R

• Data Analysis with R: Recent and good introduction, no prior knowledge needed
• rddj.info: Resources for doing data journalism with R
• Datacamp.com: Good, beginner-friendly and interactive courses on R (and Python). Very recommended - also check out my own course :-)

Exercises

KEV

Kontext:

https://www.srf.ch/news/schweiz/abstimmungen/abstimmungen/energiestrategie/die-profiteure-der-energiewende

Datensatz (Ausschnitt):

https://docs.google.com/spreadsheets/d/1c6A356Eg9zDX3dkYGGvDJuzfpmSgHnp-Zf7ucSodYRA/edit?usp=sharing

Sind folgende Auswertungen / Applikationen mit dem Datensatz möglich?

• Falls ja, wie würdet Ihr vorgehen?
• Falls nein, wieso nicht?

1. Durchschnittliche Vertragslänge (Inbetriebnahme bis Vertragsende)
2. Gesamtstromproduktion nach Kanton
3. Gesamtsumme der Geldbezüge für einerseits Herren, andererseits Frauen
4. Durchschnittliche Länge der Strassennamen nach Kanton
5. Karte mit Standorten der KEV-Bezüger (= ein Punkt-Symbol für jeden genauen Standort, ähnlich wie in diesem Artikel)

Exercise solutions day 1

1. Manillio:
1. Browse https://developer.spotify.com/console/get-search-item/ to get his ID: 7uxtLjuqkJ3cnjQQuW6Cul
2. Browse https://developer.spotify.com/console/get-artist-top-tracks/, fill in values and get JSON data
3. Copy and paste into https://json-csv.com/
4. Download as Excel - do the math (36.60705 min)
5. Take-Aways:
1. Sometimes, data is accessible via an API
2. The preferred data format of APIs is JSON
3. JSON can be converted into CSV
4. The preferred way of talking to an API is with code
2. Wasserstation Tiefenbrunnen
1. First approach: Scraping data
1. Browse https://www.tecson-data.ch/zurich/tiefenbrunnen/index.php (as probably shown on Google)
2. Select “windchill”, 2.11.2018/7.11.2018 and “all values” at the very bottom
3. Copy stuff into Excel by hand and calculate median
2. Second approach: Open Data Zürich / API:
1. Browse https://tecdottir.herokuapp.com/docs/#/measurements
2. Enter parameters
3. Copy curl string and pipe into a file
4. Upload JSON and paste into  https://json-csv.com/ (bonus: use matrix style)
5. Download CSV, open in Excel and calculate median (don’t forget to filter unneeded dates)
3. Take-Aways:
1. Copying and pasting stuff from HTML tables should be avoided
2. Always look out for an API
3. Try out different settings of your tools - they might bring you better results (“matrix style”)
4. Get to know the terminal
5. Excel / LibreOffice / OpenOffice have some good filters: get to know how to use them
6. If you run out of queries, delete cookies
3. Schlichtungsverfahren
1. Google it and go to https://www.bwo.admin.ch/bwo/de/home/mietrecht/schlichtungsbehoerden/statistik-der-schlichtungsverfahren.html
2. Download first PDF
3. Download Tabula and launch, upload PDF (or use Adobe Reader DC)
4. Select last table, lattice extraction format
5. Download as CSV
6. Open in LibreOffice and make chart
7. Take-Aways:
1. Many interesting data are buried in PDFs
2. Use proprietary software or Tabula to extract the data