Lecture 5 - Image processing 1

gray.mgh.harvard.edu

Lecture 5 - Image processing 1

Overview Block 2: Image processing Two main topics Registra7on Lecture 1:Lecture 2: Basics of parametric image registra9on Mul9-­‐modality registra9on using Mutual Informa9on, Outlook towards deformable registra9on Segmenta7on Lecture 3 and 4


Motivation 1. Mul7ple imaging modali7es different images (CT, MR, PET, …) show different features CT: bone, soT 9ssue, air absorp9on coefficients for radiotherapy dose calcula9on T1-­‐MRI: tumor, ventricles, white maVer


1. Mul7ple imaging modali7es Motivation different images (CT, MR, PET, …) show different features 2. Images from different 7me points examples: planning CT + cone beam CT in treatment posi9on diagnos9c image + follow-­‐up images


1. Mul7ple imaging modali7es Motivation different images (CT, MR, PET, …) show different features 2. Images from different 7me points examples: planning CT + cone beam CT in treatment posi9on diagnos9c image + follow-­‐up images 3. Atlas registra7on for atlas based segmenta9on


Overview Today: Intra-­‐modality registra7on e.g. CT to CT (Hounsfield number is quan9ta9ve) Formulate as op7miza7on problem Problem formula9on Image transforma9on Gradient calcula9on Next lecture: Mul7-­‐modality registra7on Mutual informa9on


Problem formulation Goal: find a transforma9on that maps every voxel in one image to the corresponding voxel in the reference image Image Reference Image


Problem formulation Image I i, j( ) Reference Image I Ri, j( )jjii


Problem formulation Image Reference Image jjii


Problem formulation Consider the image as a func9on in 3D (here 2D) I x, y ( )(scalar, real valued) We have a discrete representa9on of the func9on I i, j ( )(evaluated at voxel (i,j)) Image transforma9on means • coordinate transforma9on I R( i, j) ~ I ( x = f x(i, j), y = f y(i, j) )• not changing the “image func9on” itself


Image transformation !#"xy$&%= f i, j;' ( ) =!##"f x(i, j;')f y(i, j;')$&&%Point in transformed image Image transforma9on func9on evaluated at (i, j)


Image transformation !#"xy$&%= f i, j;' ( ) =!#"x(i, j;')y(i, j;')$&%Point in the image Image transforma9on func9on evaluated at (i, j)


Transformation function Specifica7on of the transforma7on func7on fRigid / Affine registra9on • few parameters • global representa9on of the transforma9on Deformable registra9on • many parameters • local representa9on of the transforma9on


Rigid registration Angles and length of line segments are preserved • transla9on • rota9on In 2D: • 2 parameters for transla9on • 1 angle for rota9on !#"xy$ !& = cos! 'sin!#% " sin! cos!$ !&#%"ij$ !& + #%#"t xt y$&&%In 3D: • 3 parameters for transla9on • 3 angles for rota9on


Rigid registration Angles and length of line segments are preserved • transla9on • rota9on In 2D: • 2 parameters for transla9on • 1 angle for rota9on Homogeneous coordinates: !###"xyw$ !& #& = #&%#"(single matrix mul9plica9on) cos! 'sin! t xsin! cos! t y0 0 1$ !&#&#&#%"ij1$&&&%


Affine registration Most general transforma9on that preserves parallelism between lines • transla9on 2 • rota9on 1 • scaling 2 • shearing 1 2D: 6 parameters 3D: 12 parameters !###"xyw$&& =&%!###"a b t xc d t y0 0 1$ !&#&#&#%"ij1$&&&%


κvector v and κ v ( R∈ ) represent the same point, an important advantage of usinghomogeneous coordinates is that original andBasictransformed transformationspositions, as well as compositionof transformations, are related by matrix multiplications [Hartley and Zisserman, 2000].Figure 3, illustrates the Euclidian, Similarity, Affine and Projective transformation.transla9on rota9on similarity affine projec9ve Figure 3. Linear Transformations in Homogeneous Coordinates: (a) Original, (b) Euclidean,(c) Similarity, (d) Affine and (e) Projective!##xy5.1 Euclidean Transformations$ !& #& = #a b t xc d t yIn the case of Euclidian Transformation, angles and length of line segments are preserved,and only translations and rotations are allowed (see Figure 3(b)). This transformation have#three parameters, Θ = {φ, t i w&" , t j },%where #e φ is the f rotation 1 &#angle, 1&and t i , t j are translation in"directions i and j respectively. Using HC, x(Θ,i,j)=x h /w h%"%and y(Θ,i,j)=y h /w h , can be representedby:$ !&#&#ij$&&


Optimization problem Determining the transforma7on parameters … “make transformed image similar to the reference image” Formulate as op7miza7on problem minimize!'i, j#$ I x(i, j;!), y(i, j;! % & 2( ) " I ( Ri, j)C =! g( e ) ijg e iji, j( ) = e ij2


Most generic algorithm: gradient descent ! t+1 " ! t #! $ !COptimization algorithm $&&! "C =&&&&%#C#" 1!#C#" n'))))))(Note: calcula9ng the gradient is also the basis for more advanced algorithms e.g. Quasi-­‐Newton methods


Calculating derivatives Calcula9ng deriva9ves (chain rule) !C!" k=!C# !e ij!e ij!" ki, jinfluence func9on “how much does the error in voxel (i,j) change if the transforma9on parameters change” “how much does the objec9ve func9on change if the error in voxel (i,j) changes” ( ) " I R ( i, j)( )= 2e ij= 2 I x(i, j;!), y(i, j;!


Calcula9ng deriva9ves !e ij= ! I x(i, j;"), y(i, j;"!" k!" kCalculating derivatives ( ( ) # I ( Ri, j))independent of Θ #%$!x(i, j)!" k!y(i, j)!" k#%&%(%'%$(chain rule in higher dimensions) !I ( x(i, j;"), y(i, j;") )!x!I ( x(i, j;"), y(i, j;") )!y&(((('


Calculating derivatives !e ij!" k=#%$!x(i, j)!" k!y(i, j)!" k#%&%(%'%$!I ( x(i, j;"), y(i, j;") )!x!I ( x(i, j;"), y(i, j;") )!y&(((('Coordinate matrix model “how much does the point (x,y) change when changing the transforma9on parameters” Image gradient “how much does the image change at point (x,y)”


Calculating derivatives ! "C =i, j#C#e ij$&&! "e ij=&&&&%#x(i, j) #y(i, j)#" 1#" 1! !#x(i, j) #y(i, j)#" K#" K') $)&)&)&)&)%($ ! " e ijMatrix (K=#parameters) ( )#I x(i, j;"), y(i, j;")#x#I x(i, j;"), y(i, j;")( )#y'))))(K ! 2


Example: affine transforma9on !###"#%%%%%%$xyw$ !& #& = #&%#"' 1' 2' 3' 4' 5' 60 0 1!x(i, j) !y(i, j)!" 1!" 1! !!x(i, j) !y(i, j)!" K!" K& #( %( %( %(= %( %%(' %$Coordinate matrix model $ !&#&#&#%"ij1i 0j 01 00 i0 j0 1$&&&%&((((((('


Calculating image gradients Image is given at discrete posi7ons (i,j) But, registra9on requires evalua9on at (x,y) I x(i, j;!), y(i, j;!( )Gradient at (x,y) #%%%%$!I ( x(i, j;"), y(i, j;") )!x!I ( x(i, j;"), y(i, j;") )!y&(((('


Bilinear interpolation (i, j)I ( x, y) = I ( i +! x! , j +! y!)(x, y)= ! xI ( i +1, j +! y!) + ( 1"! x )I ( i, j +! y!)! yI ( i +1, j +1) + ( 1!! y )I ( i +1, j)! x! (1!! x)"(i +1, j +1)!


Calculating image derivatives Simplest method: finite differences !I ( i, j)!i( ) # I ( i #1, j)" I i +1, j2Main problem: • Sensi9ve to noise poten9ally beVer: Deriva9ves of Gaussian filters


Consider smoothed image: I SCalculating image derivatives ( x, y) = I(x', y')g(x ! x')g(y ! y')dx'dy'Deriva9ve: !I S ( x, y)!x!I ( i, j)!i##"= !!x##$$#I(x', y')g(x " x')g(y " y')dx'dy'= I(x', y')g'(x ! x')g(y ! y')dx'dy'"( )Deriva9ve of the Gaussian " # I i', j' g'(i $ i')g( j $ j') (discrete version) i'#j'Gaussian


Approxima9ng deriva9ves at (x,y) Calculating image derivatives Method 1: calculate deriva9ves of input image use bilinear interpola9on to get !I ( i, j)!i!I x(i, j;"), y(i, j;")( )!xMethod 2: calculate transformed image through bilinear interpola9on approximate deriva9ves of the transformed image


Image gradient Image y-­‐component of gradient


Local minima Mean Squared Difference for shiUed image mean squared error shiT in x [# pixels] è Local minima occur


Local minima Mean Squared Difference for shiUed image gray value posi9on x The image is a non-­‐convex func9on


Summary Topic: Parametric image registra7on explicit representa9on of coordinate transforma9on (rigid, affine) Today: Intra-­‐modality registra7on Minimize difference of reference and transformed image Use chain rule to calculate gradients • influence func9on • image gradient • coordinate matrix model

More magazines by this user
Similar magazines