# Lecture 5 - Image processing 1 Lecture 5 - Image processing 1

Overview Block 2: Image processing Two main topics Registra7on Lecture 1:Lecture 2: Basics of parametric image registra9on Mul9-­‐modality registra9on using Mutual Informa9on, Outlook towards deformable registra9on Segmenta7on Lecture 3 and 4

Motivation 1. Mul7ple imaging modali7es different images (CT, MR, PET, …) show different features CT: bone, soT 9ssue, air absorp9on coefficients for radiotherapy dose calcula9on T1-­‐MRI: tumor, ventricles, white maVer

1. Mul7ple imaging modali7es Motivation different images (CT, MR, PET, …) show different features 2. Images from different 7me points examples: planning CT + cone beam CT in treatment posi9on diagnos9c image + follow-­‐up images

1. Mul7ple imaging modali7es Motivation different images (CT, MR, PET, …) show different features 2. Images from different 7me points examples: planning CT + cone beam CT in treatment posi9on diagnos9c image + follow-­‐up images 3. Atlas registra7on for atlas based segmenta9on

Overview Today: Intra-­‐modality registra7on e.g. CT to CT (Hounsfield number is quan9ta9ve) Formulate as op7miza7on problem Problem formula9on Image transforma9on Gradient calcula9on Next lecture: Mul7-­‐modality registra7on Mutual informa9on

Problem formulation Goal: find a transforma9on that maps every voxel in one image to the corresponding voxel in the reference image Image Reference Image

Problem formulation Image I i, j( ) Reference Image I Ri, j( )jjii

Problem formulation Image Reference Image jjii

Problem formulation Consider the image as a func9on in 3D (here 2D) I x, y ( )(scalar, real valued) We have a discrete representa9on of the func9on I i, j ( )(evaluated at voxel (i,j)) Image transforma9on means • coordinate transforma9on I R( i, j) ~ I ( x = f x(i, j), y = f y(i, j) )• not changing the “image func9on” itself

Image transformation !#"xy\$&%= f i, j;' ( ) =!##"f x(i, j;')f y(i, j;')\$&&%Point in transformed image Image transforma9on func9on evaluated at (i, j)

Image transformation !#"xy\$&%= f i, j;' ( ) =!#"x(i, j;')y(i, j;')\$&%Point in the image Image transforma9on func9on evaluated at (i, j)

Transformation function Specifica7on of the transforma7on func7on fRigid / Affine registra9on • few parameters • global representa9on of the transforma9on Deformable registra9on • many parameters • local representa9on of the transforma9on

Rigid registration Angles and length of line segments are preserved • transla9on • rota9on In 2D: • 2 parameters for transla9on • 1 angle for rota9on !#"xy\$ !& = cos! 'sin!#% " sin! cos!\$ !&#%"ij\$ !& + #%#"t xt y\$&&%In 3D: • 3 parameters for transla9on • 3 angles for rota9on

Rigid registration Angles and length of line segments are preserved • transla9on • rota9on In 2D: • 2 parameters for transla9on • 1 angle for rota9on Homogeneous coordinates: !###"xyw\$ !& #& = #&%#"(single matrix mul9plica9on) cos! 'sin! t xsin! cos! t y0 0 1\$ !&#&#&#%"ij1\$&&&%

Affine registration Most general transforma9on that preserves parallelism between lines • transla9on 2 • rota9on 1 • scaling 2 • shearing 1 2D: 6 parameters 3D: 12 parameters !###"xyw\$&& =&%!###"a b t xc d t y0 0 1\$ !&#&#&#%"ij1\$&&&%

κvector v and κ v ( R∈ ) represent the same point, an important advantage of usinghomogeneous coordinates is that original andBasictransformed transformationspositions, as well as compositionof transformations, are related by matrix multiplications [Hartley and Zisserman, 2000].Figure 3, illustrates the Euclidian, Similarity, Affine and Projective transformation.transla9on rota9on similarity affine projec9ve Figure 3. Linear Transformations in Homogeneous Coordinates: (a) Original, (b) Euclidean,(c) Similarity, (d) Affine and (e) Projective!##xy5.1 Euclidean Transformations\$ !& #& = #a b t xc d t yIn the case of Euclidian Transformation, angles and length of line segments are preserved,and only translations and rotations are allowed (see Figure 3(b)). This transformation have#three parameters, Θ = {φ, t i w&" , t j },%where #e φ is the f rotation 1 &#angle, 1&and t i , t j are translation in"directions i and j respectively. Using HC, x(Θ,i,j)=x h /w h%"%and y(Θ,i,j)=y h /w h , can be representedby:\$ !&#&#ij\$&&

Optimization problem Determining the transforma7on parameters … “make transformed image similar to the reference image” Formulate as op7miza7on problem minimize!'i, j#\$ I x(i, j;!), y(i, j;! % & 2( ) " I ( Ri, j)C =! g( e ) ijg e iji, j( ) = e ij2

Most generic algorithm: gradient descent ! t+1 " ! t #! \$ !COptimization algorithm \$&&! "C =&&&&%#C#" 1!#C#" n'))))))(Note: calcula9ng the gradient is also the basis for more advanced algorithms e.g. Quasi-­‐Newton methods

Calculating derivatives Calcula9ng deriva9ves (chain rule) !C!" k=!C# !e ij!e ij!" ki, jinfluence func9on “how much does the error in voxel (i,j) change if the transforma9on parameters change” “how much does the objec9ve func9on change if the error in voxel (i,j) changes” ( ) " I R ( i, j)( )= 2e ij= 2 I x(i, j;!), y(i, j;!

Calcula9ng deriva9ves !e ij= ! I x(i, j;"), y(i, j;"!" k!" kCalculating derivatives ( ( ) # I ( Ri, j))independent of Θ #%\$!x(i, j)!" k!y(i, j)!" k#%&%(%'%\$(chain rule in higher dimensions) !I ( x(i, j;"), y(i, j;") )!x!I ( x(i, j;"), y(i, j;") )!y&(((('

Calculating derivatives !e ij!" k=#%\$!x(i, j)!" k!y(i, j)!" k#%&%(%'%\$!I ( x(i, j;"), y(i, j;") )!x!I ( x(i, j;"), y(i, j;") )!y&(((('Coordinate matrix model “how much does the point (x,y) change when changing the transforma9on parameters” Image gradient “how much does the image change at point (x,y)”

Calculating derivatives ! "C =i, j#C#e ij\$&&! "e ij=&&&&%#x(i, j) #y(i, j)#" 1#" 1! !#x(i, j) #y(i, j)#" K#" K') \$)&)&)&)&)%(\$ ! " e ijMatrix (K=#parameters) ( )#I x(i, j;"), y(i, j;")#x#I x(i, j;"), y(i, j;")( )#y'))))(K ! 2

Example: affine transforma9on !###"#%%%%%%\$xyw\$ !& #& = #&%#"' 1' 2' 3' 4' 5' 60 0 1!x(i, j) !y(i, j)!" 1!" 1! !!x(i, j) !y(i, j)!" K!" K& #( %( %( %(= %( %%(' %\$Coordinate matrix model \$ !&#&#&#%"ij1i 0j 01 00 i0 j0 1\$&&&%&((((((('

Calculating image gradients Image is given at discrete posi7ons (i,j) But, registra9on requires evalua9on at (x,y) I x(i, j;!), y(i, j;!( )Gradient at (x,y) #%%%%\$!I ( x(i, j;"), y(i, j;") )!x!I ( x(i, j;"), y(i, j;") )!y&(((('

Bilinear interpolation (i, j)I ( x, y) = I ( i +! x! , j +! y!)(x, y)= ! xI ( i +1, j +! y!) + ( 1"! x )I ( i, j +! y!)! yI ( i +1, j +1) + ( 1!! y )I ( i +1, j)! x! (1!! x)"(i +1, j +1)!

Calculating image derivatives Simplest method: finite differences !I ( i, j)!i( ) # I ( i #1, j)" I i +1, j2Main problem: • Sensi9ve to noise poten9ally beVer: Deriva9ves of Gaussian filters

Consider smoothed image: I SCalculating image derivatives ( x, y) = I(x', y')g(x ! x')g(y ! y')dx'dy'Deriva9ve: !I S ( x, y)!x!I ( i, j)!i##"= !!x##\$\$#I(x', y')g(x " x')g(y " y')dx'dy'= I(x', y')g'(x ! x')g(y ! y')dx'dy'"( )Deriva9ve of the Gaussian " # I i', j' g'(i \$ i')g( j \$ j') (discrete version) i'#j'Gaussian

Approxima9ng deriva9ves at (x,y) Calculating image derivatives Method 1: calculate deriva9ves of input image use bilinear interpola9on to get !I ( i, j)!i!I x(i, j;"), y(i, j;")( )!xMethod 2: calculate transformed image through bilinear interpola9on approximate deriva9ves of the transformed image