Lecture 1 - Stanford Vision Lab; Prof. Fei-Fei Li

Fei-Fei Li! 

Lecture 1: 

Introduc.on to “Computer Vision” 

Professor Fei-‐Fei Li 

Stanford Vision Lab 

Lecture 1 - ! 1 

24-‐Sep-‐12

Welcome to CS231a: Computer Vision 



24-‐Sep-‐12 

Slide adapted from Svetlana Lazebnik


Today’s agenda 

• Introduc.on to computer vision 

• Course overview 


24-‐Sep-‐12


Quiz? 


24-‐Sep-‐12


What about this? 


24-‐Sep-‐12

What is (computer) vision? 

Image (or video) Sensing device Interpreting device Interpretations 



garden, spring, 

bridge, water, 

trees, flower, 

green, etc. 

24-‐Sep-‐12

Engineering 


What is it related to? 

Speech 

Robo.cs 

Image 

processing 

Physics 

Biology 

Neuroscience 

Computer Vision 

Cogni.ve 

sciences 

Machine learning 

Psychology 

Informa.on retrieval 

Maths 


graphics,algorithms, 

system,theory,… 

Computer 

Science 

24-‐Sep-‐12


The goal of computer vision 

• To bridge the gap between pixels and “meaning” 

What we see What a computer sees Source: S. Narasimhan 


24-‐Sep-‐12








green, etc. 

24-‐Sep-‐12


1981: Nobel Prize in medicine 

Hubel & Wiesel 


24-‐Sep-‐12

Human vision is superbly efficient 


Potter, Biederman, etc. 1970s 


24-‐Sep-‐12



Thorpe, et al. Nature, 1996 

24-‐Sep-‐12


150 ms !! 


Thorpe, et al. Nature, 1996 

24-‐Sep-‐12


Change blindess 


Rensink, O’regan, Simon, etc. 

24-‐Sep-‐12


Change blindess 


Rensink, O’regan, Simon, etc. 

24-‐Sep-‐12


segmenta.on 


24-‐Sep-‐12


Percep.on 


24-‐Sep-‐12

Fei-Fei Li! Lecture 1 - ! 18 

24-‐Sep-‐12

Fei-Fei Li! Lecture 1 - ! 19 

24-‐Sep-‐12








green, etc. 

24-‐Sep-‐12


The goal of computer vision 

• To bridge the gap between pixels and “meaning” 

What we see What a computer sees Source: S. Narasimhan 


24-‐Sep-‐12


Origins of computer vision: 

an MIT undergraduate summer project 

L. G. Roberts, Machine Percep,on of 

Three Dimensional Solids, Ph.D. 

thesis, MIT Department of Electrical 

Engineering, 1963. 


24-‐Sep-‐12


What kind of informa.on can 

we extract from an image? 

• Metric 3D informa.on 

• Seman.c informa.on 


24-‐Sep-‐12


Vision as measurement device 

Real-time stereo Structure from motion 

NASA Mars Rover 

Pollefeys et al. 

Reconstruction from 

Internet photo collections 


Goesele et al. 

24-‐Sep-‐12

Vision as a source of semantic information 

Slide credit: Kristen Grauman 

The Wicked 

Twister 

deck 


ride 

Lake Erie 

sky 

tree 

tree 

bench 

water 

Ferris 

wheel 

amusement park 

tree 

Cedar Point 

tree 

ride 

carousel 

12 E 

pedestrians 


ride 

people waiting in line 

people sitting on ride 

umbrellas 

Objects 

Activities 

Scenes 

Locations 

Text / writing 

Faces 

Gestures 

Motions 

Emotions… 

maxair 

24-‐Sep-‐12


Why study computer vision? 

• Vision is useful: Images and video are everywhere! 

Personal photo albums 

Surveillance and security 

Movies, news, sports 

Medical and scientific images 


24-‐Sep-‐12


Why study computer vision? 

• Vision is useful 

• Vision is interes.ng 

• Vision is difficult 

– Half of primate cerebral cortex is devoted to visual 

processing 

– Achieving human-‐level visual percep.on is probably 

“AI-‐complete” 


24-‐Sep-‐12

Why is computer vision difficult? 



24-‐Sep-‐12

Challenges: viewpoint variation 

Michelangelo 1475-1564 



slide credit: Fei-Fei, Fergus & Torralba 

24-‐Sep-‐12

Challenges: illumination 



image credit: J. Koenderink 

24-‐Sep-‐12

Challenges: scale 



24-‐Sep-‐12 

slide credit: Fei-Fei, Fergus & Torralba

Challenges: deformation 

Xu, Beihong 1943 



slide credit: Fei-Fei, Fergus & Torralba 

24-‐Sep-‐12

Challenges: occlusion 


Magritte, 1957 


24-‐Sep-‐12 


Challenges: background clutter 



24-‐Sep-‐12 

slide credit: Svetlana Lazebnik

Challenges: Motion 



24-‐Sep-‐12 


Challenges: object intra-‐class varia.on 



24-‐Sep-‐12 



Challenges: local ambiguity 


24-‐Sep-‐12 


Challenges or opportuni.es? 

• Images are confusing, but they also reveal the 

structure of the world through numerous cues 

• Our job is to interpret the cues! 



24-‐Sep-‐12 

Image source: J. Koenderink


Depth cues: Linear perspec.ve 


24-‐Sep-‐12 



Depth cues: Aerial perspec.ve 


24-‐Sep-‐12 



Depth ordering cues: Occlusion 


24-‐Sep-‐12 

Source: J. Koenderink


Shape cues: Texture gradient 


24-‐Sep-‐12 



Shape and ligh.ng cues: Shading 


24-‐Sep-‐12 



Posi.on and ligh.ng cues: Cast shadows 


24-‐Sep-‐12 


Grouping cues: Similarity (color, texture, proximity) 



24-‐Sep-‐12 



Grouping cues: “Common fate” 


24-‐Sep-‐12 

Image credit: Arthus-Bertrand (via F. Durand)


Bogom line 

• Percep.on is an inherently ambiguous problem 

– Many different 3D scenes could have given rise to a par.cular 2D picture 


24-‐Sep-‐12

Bogom line 

• Percep.on is an inherently ambiguous problem 

– Many different 3D scenes could have given rise to a par.cular 2D 

picture 

• Possible solu.ons 

– Bring in more constraints (more images) 

– Use prior knowledge about the structure of the world 

• Need a combina.on of different methods 



24-‐Sep-‐12

Computer Vision in the Real World 



24-‐Sep-‐12

Special effects: shape and mo.on capture 



24-‐Sep-‐12 

Source: S. Seitz


3D urban modeling 

Bing maps, Google Streetview 


Source: S. Seitz 

24-‐Sep-‐12

3D urban modeling: Microsoj Photosynth 


hgp://labs.live.com/photosynth/ Source: S. Seitz 


24-‐Sep-‐12


Face detec.on 

• Many new digital cameras now detect faces 

– Canon, Sony, Fuji, … 



24-‐Sep-‐12


Smile detec.on 

Sony Cyber-shot® T70 Digital Still Camera Source: S. Seitz 


24-‐Sep-‐12


Face recogni.on: Apple iPhoto sojware 

hgp://www.apple.com/ilife/iphoto/ 


24-‐Sep-‐12


Biometrics 

How the Afghan Girl was Iden.fied by Her Iris Pagerns 



24-‐Sep-‐12


Fingerprint scanners on 

many new laptops, 

other devices 

Biometrics 

Face recogni.on systems now beginning 

to appear more widely 

hgp://www.sensiblevision.com/ 



24-‐Sep-‐12

Op.cal character recogni.on (OCR) 

Technology to convert scanned docs to text 

• If you have a scanner, it probably came with OCR sojware 


Digit recognition, AT&T labs 

License plate readers 

http://en.wikipedia.org/wiki/Automatic_number_plate_recognition 



24-‐Sep-‐12


Toys and Robots 

Lecture 1 - !

Mobile visual search: iPhone Apps 



24-‐Sep-‐12


Automo.ve safety 

• Mobileye: Vision systems in high-‐end BMW, GM, Volvo models 

– “In mid 2010 Mobileye will launch a world's first applica.on of full 

emergency braking for collision mi.ga.on for pedestrians where 

vision is the key technology for detec.ng pedestrians.” 


Source: A. Shashua, S. Seitz 

24-‐Sep-‐12


Vision in supermarkets 

LaneHawk by EvolutionRobotics 

“A smart camera is flush-mounted in the checkout lane, continuously watching for items. 

When an item is detected and recognized, the cashier verifies the quantity of items that 

were found under the basket, and continues to close the transaction. The item can remain 

under the basket, and with LaneHawk, you are assured to get paid for it… “ Source: S. Seitz 


24-‐Sep-‐12

Vision-‐based interac.on (and games) 

Microsoft’s Kinect 


Assistive technologies 


Sony EyeToy 


24-‐Sep-‐12

Vision for robo.cs, space explora.on 

NASA'S Mars Explora.on Rover Spirit captured this westward view from atop 

a low plateau where Spirit spent the closing months of 2007. 

Vision systems (JPL) used for several tasks 

• Panorama s.tching 


• 3D terrain modeling 

• Obstacle detec.on, posi.on tracking 

• For more, read “Computer Vision on Mars” by Maghies et al. 


24-‐Sep-‐12 

Source: S. Seitz


The computer vision industry 

• A list of companies here: 

hgp://www.cs.ubc.ca/spider/lowe/vision.html 


24-‐Sep-‐12


Today’s agenda 

• Introduc.on to computer vision 

• Course overview 


24-‐Sep-‐12


Overall philosophy 

• Breadth 

– Computer vision is a huge field 

– It can impact every aspect of life and society 

– It will drive the next informa.on and AI revolu.on 

– Pixels are everywhere in our lives and cyber space 

– Lectures are high-‐level, meant to be informa.ve, and covers many topics 

– Lots of links to references. Know where to look for references 

– Speak our “language” 

• Depth 

– Computer vision is a highly technical field, i.e. know your math! 

– Homework meant to be challenging, both theore.cal ques.ons and 

programming exercises 

– Master bread-‐and-‐buger techniques: face recogni.on, corners, lines, features, 

op.cal flows, clustering and segmenta.on, basic object recogni.on 

techniques 

– Course projects are your hands-‐on experience in computer vision systems and 

research 


24-‐Sep-‐12


Contac.ng instructor and TAs 

• ALL EMAIL CORRESPONDENCES TO ANYONE OF US: 

– cs231a-‐aut1213-‐staff@lists.stanford.edu 

• Professor: Fei-‐Fei Li 

– Office hour: Tues 3:30-‐4:30pm 

• Jon Krause, Ph.D, CS 

– Office hour: Mon 4:30-‐5:30pm 

• Vignesh Ramanathan, Ph.D, EE 

– Office hour: Wed 3:00-‐4:00pm 

• Jinchao Ye, master, CS 

– Office hour: TBD 

• Zixuan Wang, master, CS 

– Office hour: Fri 3:00-‐4:00pm 


24-‐Sep-‐12

• Go to website… 


Syllabus 


24-‐Sep-‐12


Course Project: overview 

• 40% of your grade 

• Form your team: 

– either 2 people or 1 person 

– but the quality is judged regardless of the number of 

people on the team 

– be nice to your partner: do you plan to drop the 

course? 

• No late days 

• Mandatory agendance on Dec 6 for all non-‐SCPD 

students 


24-‐Sep-‐12


Course Project: overview (con.nued) 

• Start immediately 

• Some important dates: 

– Oct 16 

• Finalize team 

• Project proposal due for “open project” teams 

– Nov 6 

• Milestone due (2-‐3 pages) 

– Dec 3 

• Final codes due 

– Dec 4 

• Final writeup due 

– Dec 6 

• Presenta.on 


24-‐Sep-‐12


Course Project Op.on #1: 

the Finding Mii Challenge 


24-‐Sep-‐12

• Original research ideas encouraged 

• Useful datasets: 

– ImageNet (www.image-‐net.org) 

– PASCAL 

• Need Fei-‐Fei’s approval 

– Email is the best way 

– Do it BEFORE Oct 16 (proposal submission 

deadline) 


Course Project Op.on #2: 

Open Project 


24-‐Sep-‐12


Grading policy 

• Problem Sets: 40% 

– We have 5 problem sets 

– Homework 0: very important! (more details…) 

– Late policy 

• 5 free late days – use them in your ways 

• Ajerwards, 25% off per day late 

• Not accepted ajer 3 late days per PS 

– Collabora.on policy 

• Read the student code book, understand what is ‘collabora.on’ 

and what is ‘academic infrac.on’ 

• Midterm Exam: 20% 

– In class: Tues, Oct 30 


24-‐Sep-‐12


Grading policy 

• Course project: 40% 

– presenta.on: 5% 

– write-‐up: 10% 

• clarity, structure, language, references: 3% 

• background literature survey, good understanding of the problem: 

3% 

• good insights and discussions of methodology, analysis, results, 

etc.: 4% 

– technical: 15% 

• correctness: 5% 

• depth: 5% 

• innova.on: 5% 

– evalua.on and results: 10% 

• sound evalua.on metric: 3% 

• thoroughness in analysis and experimenta.on: 3% 

• A word about ‘the curve’ 


24-‐Sep-‐12

Lecture 1 - Stanford Vision Lab; Prof. Fei-Fei Li

Create successful ePaper yourself

Delete template?

Save as template?