Computer Vision Input

Here you will find all the necessary resources for the computer vision input.

Timeline

Introduction - What is Computer Vision? (9:00 - 10:00)

This is a general presentation about computer vision. The main questions that the video should answer are, what computer vision, where it is used and how it is related to interaction design in general?

Watch the presentation and write down questions and thoughts that come to your mind! We can discuss them later in the Zoom conference.

📼IAD-CV-Introduction Video

Zoom Conference (10:00)

Let’s talk together about computer vision in general and discuss questions and annotations.

https://zhdk.zoom.us/j/4710337215

Topcis

What do you expect of today?
How was it to watch the presentation?
Specific ideas for your group project?
How is the input structured?
Use Slack / Zoom for help

Processing & Media (10:30 - 12:00)

Do your own first steps with processing and computer vision by creating a paint tool with your own interaction method.

📼IAD-CV-ProcessingMedia Video

Task 1

Create a new processing sketch which loads this image and displays it. How can we find the position of the moon?

int w = img.width;
int h = img.height;

for(int y = 0; y < h; y++) {
  for(int x = 0; x < w; x++) {
    
    // do something
    img.set(x, y, color(0)):
  }
}

Task 2

Install the Video Library 2.0 and display the webcam image!

Task 3 a)

Combine brightest point tracking with the webcam capture to track the brightest point in your webcam video.

Task 3 b)

Create an own drawing software that is controlled by the light of your smartphone flashlight.

Think about additional features you could implement!

Zoom Conference (13:00)

Discuss the morning & share what everyone has created.

https://zhdk.zoom.us/j/4710337215

Technical Input
Was it possible to follow?

OpenCV (15:00 - 16:30)

Getting deeper into computer vision by using OpenCV, a framework to analyse images with traditional computer vision algorithms.

📼IAD-CV-ProcessingOpenCV Video

Task 4

Use opencv to track the brightest spot.

opencv.blur(10);
PVector location = opencv.max();

opencv.loadImage(cam);

// process
opencv.blur(10);
PVector location = opencv.max();

image(opencv.getSnapshot(), 0, 0);

noFill();
stroke(100, 255, 80);
circle(location.x, location.y, 10);

import processing.video.*;
import gab.opencv.*;
import org.opencv.core.*;

Capture cam;
OpenCV opencv;

void setup() {
  size(500, 500, FX2D);

  String[] inputDevices = Capture.list();
  cam = new Capture(this, 640, 480, inputDevices[0]);
  cam.start();

  opencv = new OpenCV(this, 640, 480);
}

void draw() {
  background(0);

  if (cam.available())
    cam.read();

  opencv.loadImage(cam);

  // do something

  image(opencv.getSnapshot(), 0, 0);
}

Task 5

Write your own sketch that is able to detect motion! Use either background subtraction, image difference or your own algorithm.

How will you notify yourself?

// import the following at the top of your sketch:
import org.opencv.core.*;

// run this after you have loaded the image to opencv
int count = Core.countNonZero(opencv.getGray());

import processing.video.*;
import gab.opencv.*;
import org.opencv.core.*;

Capture cam;
OpenCV opencv;

PImage last = null;

void setup() {
  size(640, 480);

  String[] inputDevices = Capture.list();
  cam = new Capture(this, 640, 480, inputDevices[0]);
  opencv = new OpenCV(this, 640, 480);
  
  cam.start();
}

void draw() {
  background(0);
  
  if(cam.available())
    cam.read();
  
  opencv.loadImage(cam);
  
  if(last != null) {
    opencv.diff(last);
    opencv.threshold(40);
  }
  
  last = cam.copy();
  
  float count = Core.countNonZero(opencv.getGray());
  float difference = count / (cam.width * cam.height) * 100;
  
  if(difference > 10.0) {
    println(millis() + "WARNING!");
  }

  image(opencv.getSnapshot(), 0, 0);
  
  fill(0, 255, 255);
  text("Difference: " + difference, 20, 20);
}

Task 6

Use Contour Detection to implement a multi-user drawing system.

import processing.video.*;
import gab.opencv.*;

Capture cam;
OpenCV opencv;

PImage last = null;

void setup() {
  size(640, 480);

  String[] inputDevices = Capture.list();
  cam = new Capture(this, 640, 480, inputDevices[0]);
  cam.start();

  opencv = new OpenCV(this, 640, 480);
}

void draw() {
  background(0);

  if (cam.available())
    cam.read();

  opencv.loadImage(cam);
  opencv.threshold(240);

  blendMode(BLEND);
  image(cam, 0, 0);

  noFill();
  stroke(255, 0, 0);
  strokeWeight(3);
  for (Contour contour : opencv.findContours()) {
    contour.draw();
  }
}

Zoom Conference (15:30)

Discuss the current state and mood.

https://zhdk.zoom.us/j/4710337215

Tuesday 30.03.2020

Depth Sensing (Tuesday 9:00 - 10:00)

Watch the video about depth sensing and on how machines are able to perceive our world.

📼IAD-CV-DepthSensing Video

DenseDepth - High Quality Monocular Depth Estimation via Transfer Learning [paper][code]

Zoom Conference (Tuesday 10:00 - 10:30)

Questions, notes and comments.

Deep Vision Input (Tuesday 10:30 - 11:45)

This input is about computer vision combined with modern machine learning techniques. It is also an introduction about the topic of machine learning in general. How does a machine learn? How does it see?

📼IAD-CV-MachineLearning Video

Task 7

Download the Deep Vision Library (direct download link) for Processing and install it into your library folder. Try out the examples and see how the library works.

Task 8

Use the FaceDetector Webcam Example and try to combine it with the emotion classifier. To create an emotion detector, use the following code sample:

FERPlusEmotionNetwork emotionNetwork = vision.createFERPlusEmotionClassifier();
emotionNetwork.setup();

// run on a single face
emotionNetwork.run(face)

// run on multiple detected images
emotions = emotionNetwork.runByDetections(testImage, detections);

A Java example on how to combine these two methods you can find here, it is Processing as well, but you may have to adapt it slightly.

Furthermore: Have a look at the Gender & Age prediction example and try to add it to the sketch as well.

Data Analysis and Connection (Friday 13:00-17:00)

Task 9

Try to analyze your own set of images and think about what you could use this information for. Use the example provided here.

Task 10

Try out the mouse example to train your own classifier.

What else could you classify?
Where do you get the data from?
How can you include this into your main project?

Content

Please do not share the slides and the content from this page. The images and videos in the presentations are sourced but not licensed.

Computer Vision Input

Timeline

Introduction - What is Computer Vision? (9:00 - 10:00)

Zoom Conference (10:00)

Topcis

Processing & Media (10:30 - 12:00)

📼IAD-CV-ProcessingMedia Video.css-vhh4ux{display:inline;outline:none;background-color:transparent;border:none;color:var(--ds-icon, #44546F);cursor:pointer;right:0;}

Task 1

Task 2

Task 3 a)

Task 3 b)

Zoom Conference (13:00)

OpenCV (15:00 - 16:30)

📼IAD-CV-ProcessingOpenCV Video

Task 4

Task 5

Task 6

Zoom Conference (15:30)

Tuesday 30.03.2020

Depth Sensing (Tuesday 9:00 - 10:00)

📼IAD-CV-DepthSensing Video

Zoom Conference (Tuesday 10:00 - 10:30)

Deep Vision Input (Tuesday 10:30 - 11:45)

📼IAD-CV-MachineLearning Video

Task 7

Task 8

Data Analysis and Connection (Friday 13:00-17:00)

Task 9

Task 10

Content

Related content

Computer Vision Input

Timeline

Introduction - What is Computer Vision? (9:00 - 10:00)

Zoom Conference (10:00)

Topcis

Processing & Media (10:30 - 12:00)

📼IAD-CV-ProcessingMedia Video.css-vhh4ux{display:inline;outline:none;background-color:transparent;border:none;color:var(--ds-icon, #44546F);cursor:pointer;right:0;}

Task 1

Task 2

Task 3 a)

Task 3 b)

Zoom Conference (13:00)

OpenCV (15:00 - 16:30)

📼IAD-CV-ProcessingOpenCV Video

Task 4

Task 5

Task 6

Zoom Conference (15:30)

Tuesday 30.03.2020

Depth Sensing (Tuesday 9:00 - 10:00)

📼IAD-CV-DepthSensing Video

Zoom Conference (Tuesday 10:00 - 10:30)

Deep Vision Input (Tuesday 10:30 - 11:45)

📼IAD-CV-MachineLearning Video

Task 7

Task 8

Data Analysis and Connection (Friday 13:00-17:00)

Task 9

Task 10

Content

Related content

📼IAD-CV-ProcessingMedia Video

📼IAD-CV-ProcessingMedia Video