Bits & Atoms IV 2021

This seminar series covers spatial perception of machines. We focus on the role of computer vision and its relevance for design in contemporary contexts and in future scenarios.

The course is closely tied to Spatial Interaction.

Schedule

Inputs

Watch the presentation and write down questions and thoughts that come to your mind.

Password: iad2021

📼 IAD-CV-Introduction Video (24.03.2021)

• facebookresearch/detectron2

• ClearView AI

  • CNN Interview

  • admin.ch

• Philipp Schmitt: Camera Restricta

• hownormalami.eu

📼 IAD-CV-Processing & Media Video (31.03.2021)

• Video Library 2.0

• PImage (Processing)

• Filter (Processing)

• Computer Vision Wiki IAD

• How do common virtual reality tracking systems work?

📼IAD-CV-Processing & OpenCV Video (07.04.2021)

• OpenCV

• OpenCV Processing

• Greg Borenstein

• Make Things See (Book)

• Introduction to Computer Vision (Daniel Shiffman)

• IR Tracking (Tactile Table)

• Spatial Interaction Examples

📼 IAD-CV-MachineLearning Video (14.04.2021)

• Backpropagation

• Deep Vision Processing Library

• Machine Learning for everyone

• Lena Test Image

• Algorithms

  • YOLOv3 / YOLOv4

  • Ultra-Light-Fast-Generic-Face-Detector-1MB

  • Lightweight openPose

  • MNIST CNN Classifier

  • Age / Gender Classifier

Tasks

Task 0 - Introduction

• Try out the experience of hownormalami.eu and document your feelings about it.

• Think about other hidden computer vision applications in your daily life and describe them.

Task 1 - Find the Moon

Create a new processing sketch which loads this image and displays it. How can we find the position of the moon?

Task 2 - Webcam

Install the Video Library 2.0 and display the webcam image!

Task 3 Interactive Paint

a)

Combine brightest point tracking with the webcam capture to track the brightest point in your webcam video.

b)

Create an own drawing software that is controlled by the light of your smartphone flashlight.

• Think about additional features you could implement!

Task 4 Brightest Spot Tracking OpenCV

Use OpenCV to track the brightest spot in the image (interactive paint with OpenCV).

• opencv.blur(10);

• PVector location = opencv.max();

Task 5 Detecting Apples

Use the provided image and mark the green apples in the image. Check out the example code to extract hue values.

Additional: Implement a color picker that directly adapts to the sensed range.

Task 6 Locate Apples

Locate the position of the extracted apples with the contour detection. Try to indicate the position with a circle or directly draw the area.

Additional: Use the apples as input device for the interactive drawing or another application (a game?)

Task 7 Deep Vision

Download and install the Deep Vision library. Try out the examples and get an overview how the framework works by reading the readme.

Task 8 Project

Think about a simple project you could implement with the tools you already know (and Deep Vision).

• Document in a few words / sketches

• Upload it till the end of today to the filer

Deliverables

Documentation of your work in the provided template together with your code. This should be placed on the server in smb://fileredu.ad.zhdk.ch/DDE/BDE_VIAD/01_ABGABEN/21_FS/Sem4_BitsAtomsIV/

80% Attendance during lessons

Links

• Dokumentation Template

• Vimeo Repeat and Speed Browser Extension