Digital Image Processing - Multimedia Computing and Computer ...

Digital Image Processing 

SS 2008 

Prof. Dr. Rainer Lienhart 

www.multimedia-computing.{de,org}

Prelude 

• Language 

– Lecture: in English, but questions can be phrased in 

English or German 

– Exam: Questions are in English, but answers can be 

given in English or German 

– Homework: Questions are in English, but answer can 

be given in English or German 

– Practice lecture (Übung): in German 

© 2004-2008, Prof. Dr. R. Lienhart, Head of Multimedia Computing, Institut für Informatik, Universität Augsburg 

Eichleitnerstr. 30, D-86135 Augsburg, Germany; email: Rainer.Lienhart@informatik.uni-augsburg.de

Introduction (1) 

• Who am I? 

• What are the research topics? 

– Automatic content analysis of images, videos, and audio 

(Multimedia Content Analysis) 

– Automatic content analysis of multimodal and multi-lingual documents 

– Data Mining on large scale distributed multimedia databases 

(Multimedia Google) 

– Aware rooms (with AV sensor and actuators) 

– Autonomous car robots (cognitive robotics) 

– Automatic learning to identify specific people in audio and video. 

– WYSIWYH – What You See Is What You Hear 

– Machine learning / statistical computing 

– Computer vision 

– Image processing 

Automatic ≡ Learning Independently 



Who are you? 

Please introduce yourself one-byone

Illusions 

Five deadly illusions of students at the University of Augsburg 

• Scores / marks do not matter (especially in Vordiplom or Bachelor) 

• One week of learning is enough 

• Only the final academic degree matters, not the time needed to get it 

• You only have to be slightly better than the mass 

• I understand everything, but stupidly memorizing and writing exams 

are boring 

If you think that any of these statements are true, you will have a 

hard time now and later. 

A university is a place for people who have a burning desire to learn, 

acquire knowledge, and apply it in novel ways! 



Passion Matters ! 



What is your passion? 

Why are you 

passionate about 

Uni Augsburg? 

What is your 

passion? 



Courses 

• SS 08 

– Lecture: Digital Image Processing (2+2) 

– Lecture: Bayesian Networks (2+2) 

– Lecture: Probabilistic Robotics (2+2) 

– Praktikum: Audio Signal Processing (2+4) 

– Praktikum: Video Signal Processing (2+4) 

– Oberseminar: Selected Aspects of Media Mining (2) 

• WS 07/08 

– Lecture: Media Mining I (4+2) 

– Praktikum: Audio Signal Processing (2+4) 

– Praktikum: Video Signal Processing (2+4) 

– Seminar: Object Recognition (2) 

– Oberseminar: Selected Aspects of Media Mining (2) 



Schedule 

• SS08: 14 Apr 2008 – 20 Jul 2008 

• Lecture (Room 207, Eichl. Str. [alte Uni]) 

– Tue: 08:15-09:45 

– First lecture: Apr. 15 th , 2008 

• Exercise (Room 202, Eichl. Str. [alte Uni]) 

– Wed : 10:00-11:30 

– First exercise: April 30 th , 2008 

• Important (National holidays): 

– No lecture on Tuesday May, 13 th 

• Examine (Room 207, Eichl. Str. [alte Uni]) 

– Tue: 08:15-09:45 

– Jul. 15 th , 2008 



Outline 

1. Introduction 

2. Imaging Sensors 

3. Cameras & Camera 

Calibration 

4. Image Representation 

5. Color Spaces 

6. Image Statistics & Image 

Models 

7. Pixel Operations 

8. Neighborhood 

Operations 

9. Filtering: Averages & 

Edges 

10.Salient Points 

11.Salient Point Descriptors 

12.Segmentation 



Book Recommendations 

• Bernd Jähne. Digital Image Processing. Springer Verlag. highly 

recommended 

Relevant chapters 1-5, 8, 10 (4 th edition) 

• David A. Forsyth and Jean Ponce. Computer Vision: A Modern 

Approach. Prentice Hall, Upper Saddle River, New Jersey 07458. 

(http://www.cs.berkeley.edu/~daf/book.html) 

• Martin Schader and Stefan Kuhlins. Programmieren in C++. Springer- 

Verlag. ISBN: 3540637761 

This is a perfect resource for all your questions relating C/C++; 

recommended if you are not skilled in C/C++ 



Software Tools 

Every student must request access to MSDNAA-Software-Center 

Read instructions at http://www.informatik.uni-augsburg.de/service/msdn/ 

Get and install Microsoft VisualStudio .NET 2005 or 2008 

OpenCV (Open Source Computer Vision Library) 

Universal toolbox for research and development in the field of Computer Vision (win & linux) 

Get and install OpenCV from http://sourceforge.net/projects/opencvlibrary 

Required include paths (must be set in your DevEnv): 

– C:\Programme\OpenCV\cv\include 

– C:\Programme\OpenCV\cxcore\include 

– C:\Programme\OpenCV\otherlibs\highgui 

Required library paths: C:\Programme\OpenCV\lib 

Required libraries: 

cxcore.lib, cv.lib, highgui.lib 

YOUR 

ACTION REQUIRED 

(AR) 

!!! 

Problems/deficits with/in C/C++: 

Martin Schader and Stefan Kuhlins. Programmieren in C++. Springer-Verlag. ISBN: 3540637761 

Required for Exercise 



Rules 

Exam: 

• Written examine: July 15th from 8:15-9:45am. 

• Lecture, exercise, and all specified reading notes are relevant for the exams 

independent of how thoroughly the reading notes have been discussed during the 

lecture. 

Exercise: 

• The homework (“Übungsblätter”) will be given out during the Tuesday lecture and 

must be returned by the following Monday 10am sharply (no exceptions) by (a) email 

to Eva Hörster or (b) by delivering your solution into our mailbox at Eichleitnerstr. 30 

in the entrance hall. During the exercise lecture on Wednesday the students will have 

the chance to present their answers. 

• First homework will be given out on April, 22 nd 2008. 

• Exercise will start on Wednesday, April. 30 th 2008. 

• IMPORTANT: In order to be admitted to the final exam, students are required: 

– to score at least 50% of the points archievable on the weekly assignments 

– to attend the weekly exercise sessions (Wednesday, 10:00-11:30pm). Students are allowed 

to miss the exercise session at most three times. No exceptions 

Unterschleif 



Demos 

Media Content Analysis

Demo – Face Recognition 



Demo – Mouth Detection 



Demo – Bowl Detection 



Mining for Commercials 







Demos 

Effortless Sensor and Actuator 

Infrastructure

Business Scenario - 

Meeting room 

2 

PC server 

1 

Digital Home Scenario – 

Connected devices 

5 

1 

PDA’s or 

phones 

mobile PC 

6 

sensors 

3 

4 

wireless networks 

Requirements to make vision happen: 

• Sync between wireless I/O and wireless 

computing devices 

• 3D position and 3D orientation of 

sensor/actuators 

Enable multi-microphone conferencing using 

available computing devices in room (laptop, 

PDA, cell phones, other sensors) 

Enable true 3D audio in the home without 

tedious calibration 



Goal 

Given 

– A location/place 

– N c cameras, 

– N m microphones, 

– N d displays, 

– N l loudspeakers, 

– Enough computational 

resources 

– Enough power outlets at the 

right place 

– Wired/Wireless connectivity 

get a smart room running within 

an hour (55 minutes for 

connecting cables, 5 minutes 

for calibration). 

What’s needed: 

– Simple, scalable, selforganizing, 

self-healing, autocalibrating 

system setup 

Believe: 

– This would be a perfect 

system for us researchers 

– Progress and creativity is 

hindered by current inflexible 

and non-scalable systems 

(e.g., Aware Home Research 

Initiative (AHRI)) 



Position Calibration of Audio Sensors/Actuators 

Z 

Y 


Eichleitnerstr. 30, D-86135 Augsburg, Germany; email: Rainer.Lienhart@informatik.uni-augsburg.de 

X

Speaker 

2 

Speaker 

4 

Room Length = 4.22 m 

Synchronized setup 

bias 0.08 cm sigma 3.8 cm 

Room Height = 2.03 m 

Speaker 

3 

1 2 

Mic 

3 

Mic 

4 

Mic 

2 

Mic 

1 

Speaker 

1 

Z 

Room Width = 2.55 m 





Providing a Common Space for Multiple 

Cameras and Flat-Panel Displays 



Intrinsic Calibration 

• Control point extraction: 

– Pattern is displayed on a laptop screen 

– Images from different orientations are 

captured by waving the screen 

– Projected pattern points are determined by 

SIFT-feature matching 



Extrinsic Calibration of Multiple Cameras 

Bundle adjustment is superior method 

min 

ˆ j ˆ 

 

– Minimization of the reprojection error : P , Xi 

ij 

d( ˆ 

j 

P Xˆ 

i 

, x 

j 

i 

) 

2 

Initialization of P j and X i : 

– Cameras are partitioned into 

manageable subgroups that 

share a common view 

(triplets) 

– Hierarchical registration of 

the different coordinate 

systems based on common 

cameras/points 



Optimal Sensor Placement 



Wie gewinnt man ein 

Rennen mit 

computergesteuerten 

Autos?

Darpa Grand Challenge 

• First held in 2004, 2nd held on Oct, 8 th 2005 

• Designed to accelerate research and development in 

autonomous ground vehicle technology that will help 

save lives on the battlefield 

• Team whose fully autonomous ground vehicle finishes a 

course, from Los Angeles to Las Vegas, in the fastest 

time and under 10 hours, wins $1M (2004) / $2M (2005). 

• Course: 

– defined by ~ 1000 waypoints, given 2 hrs before race. 

– on-road, off-road, trails, water, obstacles 

– 2005: 131.6 miles in Mohave Desert; ~ 25 mph average speed 

• Only publicly available signals (e.g., GPS) may be used. 



Terrain (1) 

Image taken from 

http://www.darpa.mil/grandchallenge04/media_images.htm 



Image taken from 

http://www.darpa.mil/grandchallenge04/media_images.htm 



Computer Vision Problem 

- Lasers have limited range 

- Car is bouncing 

- Maximum speed is 25 mph but we want to go 35 mph 

- 25/35 mph decision is made with computer vision 



Define „Street“ 

- Laser maps 3D terrain ahead of the car 

- Finds path and flat road surface 

- Projects 3D data into 2D video image 

- Learn the looks of the street using machine learning 



Extend “Street” 

- What we know for sure: laser brick is road 

- Use color texture under laser brick in order to extrapolate 

road to the horizon 

- Improve raw results by appropriate heuristics 

Unfiltered Result 

Filtered Result 



Gewinner 2005 

http://www.stanfordracing.org/ 

6:53:58 19.1mph 

Start demo

Digital Image Processing - Multimedia Computing and Computer ...

Create successful ePaper yourself

Delete template?

Save as template?