Slides (pdf, 1.5MB) - Robotics Summer Schools at ETHZ

UAVs Visual Servoing 

AIRobots: Summer school 2012 – Aerial Service Robotics 

Vincenzo Lippiello 

Università di Napoli Federico II 

vincenzo lippiello – UAVs Visual Servoing 

vincenzo.lippiello@unina.it

Vision sensors 

Camera model 

Image point differential kinematics 

CAMERA MODEL 

vincenzo lippiello – UAVs Visual Servoing 2


• The task of a camera as a vision sensor is to 

measure the intensity of the light reflected by 

an object 

• A photosensitive element (pixel) is employed 

– CCD (Charge Coupled Device) 

– CMOS (Complementary Metal Oxide 

Semiconductor) 



• Camera is a complex system comprising several 

devices: 

– Photosensitive sensor 

– Lens (responsible for focusing the light onto the sensor 

plane, i.e. image plane) 

– Analog/Digital preprocessing electronics 


Camera model 

• Perspective transformation 

f is the focal length 

SENSOR PLANE 

LENSE CENTER 

The image of an object 

appears upside down on 

the image plane 


Camera model 

• Frontal perspective transformation 

LENSE CENTER 

IMAGE PLANE 

Simplified model: lens and mounting 

imperfections cause image distortions 

(e.g. aberrations and geometric distortion) 


Camera model 

• Digitalization of the measurement 

– Spatial sampling (CCD/CMOS sensor): pixels 

– Temporal sampling (shutter) 

• Pixel coordinates of a point: 

λ > 0 is a scale factor (optical ray) 


Camera model 

• Normalized image coordinates (f = 1m) 

• Feature vector of an image point 



• Coordinates of a 3D point fixed in the world 

frame w.r.t. the camera reference frame 

• Feature vector of the point 

• Time derivative of s 



• By observing that 

• The differential kinematics of an image point 

c 

s = Ls s, z c v c 

v c = [o c ! c ] T 



• Interaction matrix of an image feature point 


Rotary wings vehicles 

Quadrotor actuation system 

Quadrotor dynamic model 

QUADROTOR DYNAMIC MODEL 


Rotary wing vehicles 

• Main features 

– High maneuverability 

– Vertical take off and landing (VTOL) 

– Able to perform stationary/slow flight 

– Can fly in small and cluttered environment 



• Four motor/propeller systems 

arranged in a cross-like shape 

• Each motor spins at a proper 

angular speed (control input) 

• Each propeller produces a 

force that is proportional to 

the square of angular speed 

f i = α! i 

2 

• Two motors rotate C.W. and 

two A.C.W. 



COLLECTIVE SPEED 

RELATIVE SPEED IN THE GROUP 

RELATIVE SPEED BETWEEN GROUPS 

TRUST FORCE 

TORQUE AROUND x AND y 

TORQUE AROUND z 



• Motion is given by 

– Mechanical components (gravity, inertia, spinning propellers, …) 

– Aerodynamic effects (blade thrust/drag, blade flapping, …) 

• The complete model is rather complex 

– Some components can be estimated by identifying 

inertial/aerodynamic coefficients 

– It is common to have low level control loops to stabilize propeller 

speed, altitude, attitude 

– A simplified model can be used to design control for high level tasks 

• We consider the following simplifying assumptions: 

– No wind 

– UAV is symmetrical (diagonal inertia matrices) 

– Secondary inertial/aerodynamic effects are neglected 

– Self-inducted aerodynamic disturbances are not modeled 



• Simplified dynamic model 

Trust T = α ! 

2 

1 + !2 

2 + !3 

2 + !4 

2 

Roll torque μx = α ! 

2 

4 − !2 

2 

Pitch torque μy = α ! 

2 

3 − !1 

2 

Yaw torque μ z = α ! 

2 

1 − !2 

2 − !3 

2 + !4 

2 

mx = 

cψs ϑ c ϕ + sψs ϕ T 

mp = −TR b z −mgz 

J ! = μ 

my = sψs ϑ c ϕ − cψs ϕ T 

mz = c ϑ c ϕ T − mg 

(ψ ϑ ϕ) : yaw(Z)–pitch(Y)–roll(X) angles 

g = 9.81 


Image-based visual servoing (IBVS) 

Case 1: hovering control with one image feature point 

Case 2: hovering and landing control with four image feature points 

VISUAL SERVOING 


Visual servoing 

• Vision allows a robotic system to obtain 

geometrical and quantitative information on 

the surrounding environment 

• Control based on feedback of visual 

measurements is termed visual servoing 

• Main approaches to visual servoing 

– Position-based visual servoing (PBVS) 

– Image-based visual servoing (IBVS) 


Image-based visual servoing (IBVS) 

• Control the UAV to ensure the convergence of the 

image features error in the image plane 

– Desired reference assigned in the image plane! 

– Image features differential kinematics has to be considered 

– For UAVs the image features resulting motion depends on 

the vehicle motion w.r.t. the fixed environment 

UAV 


Case 1 : hovering control with one image 

feature point 

• Hovering task: regulate the 

position (x, y) of the UAV, while 

keeping the target in the center of 

image plane 

– Using a down-looking camera, fixed 

and centered in the UAV CoG (body 

and camera reference frame 

coincident) 

– Using a single point feature 

• It is not sufficient to control all the 

degrees of freedom 

• Motion along and around the image 

feature optical ray is unobservable 

Y 

s d 

IMAGE PLANE 

s 

X 




• Additional assumptions: 

– Attitude is stabilized by a high frequency low level 

controller (commonly available on most systems) 

– Altitude is separately controlled (the control input 

T becomes an exogenous signal, i.e. the measured 

trust) 

– Yaw angle is known and separately controlled 




• Image feature differential kinematics 

s = Ls s, z c v c c = Lv s, z c 

p c c + L! s ! c 

c 

• Error in the image plane (e.g. s d = 0) 

es = s d − s 

• By setting a desired velocity as 

c 

p c,d = Lv # s, z c Kses − L! s ! 

c c 

a decupled exponential convergence achieved 

es = −Lv s, z c 

pc 

c,d 

− L! s ! c c = −Kses 




• Dynamic model of the UAV w.r.t. a frame 

rotated around the z-axis of the yaw angle ψ 

mx = cψs ϑ c ϕ + sψs ϕ T 

my = sψs ϑ c ϕ − cψs ϕ T 

mz = c ϑ c ϕ T − mg 

xψ 

y ψ 

zψ 

= 

cψ sψ 0 

−sψ cψ 0 

0 0 1 

x 

y 

z 

• Roll (ϕ) and pitch (ϑ) angles as control input 

xψ = T m s ϑ c ϕ 

yψ = − T m s ϕ 




• Let consider a inner velocity control loop 

xψ = Kv x c,d − x c 

y ψ = Kv y 

c,d − y c 

• By inverting the model 

ϑ d = asin 

m 

T K v x c,d − x c 

c ϕ 

ϕ d = −asin 

m T K v y 

c,d − y c 




• Velocity estimation by inverting the image feature 

point differential kinematics 

p c c = Lv # s, z c s − L! s ! c 

c 

• Advantages 

– only feature coordinates and the attitude required 

– easy implementation/low computational cost 

• Disadvantages 

– not so accurate (numerical derivation of image feature 

coordinates s is required) 




• Control scheme 


Case 2: hovering and landing control with 

four image feature points 

• Hovering and landing task: regulate 

the position (x, y, z) and the 

orientation (yaw angle) of the UAV 

w.r.t. an observed object 

– Using a down-looking camera, fixed 

and centered in the UAV CoG 

– Using four point features (three is the 

minimum needed number) 

– Attitude is stabilized by an high 

frequency low level controller 

s d 

X 

IMAGE PLANE 

s 

Y 




• Interaction matrix of a set of image points 

n = 4 

• Error in the image plane 

es = s d − s 

– s d constant (or slowly varying) 




• Control problem 

mp = −u − mgz 

• Control input 

u = TRcz 

• Control input with a gravity compensation 

u′ = u + mgz 

• Inner velocity control loop 

c c 

u′ = KvRc p c,d − p c 




• Desired linear velocity evaluated as 

c 

p 

c,d 

= Lv # s, z c Kses − L! s ! c 

c 

• Desired yaw velocity evaluated as 

ψ d = z T R c L! # s Kses − Lv s, z c p c 

c 

– Linear velocity p c c 

chosen equal to p c c 

c 

or p 

c,d 

– Yaw angle control loop: PI controller 




• Control scheme 


REFERENCES: 

o B. Siciliano, L. Sciavicco, L. Villani, G. Oriolo, “Robotics: Modelling, 

Planning and Control”, 2nd Ed., 2009 

o L. Rosa, “Autonomous and Mobile Robotics: Visual Servoing for 

Unmanned Aerial Vehicles” 

THANK YOU FOR YOUR ATTENTION

Slides (pdf, 1.5MB) - Robotics Summer Schools at ETHZ

Create successful ePaper yourself

Delete template?

Save as template?