PHD Thesis - Institute for Computer Graphics and Vision - Graz ...

6.1. Wide-baseline region matching 92
Property 1 is achieved by a 2-step approach. In a first step, tentative correspondences are
identified by nearest neighbor matching in feature space. The tentative matches however still
contain a lot of outliers. In a second step the tentative matches are verified by area based
matching, calculating the correlation over the whole interest region. This step ensures with
maximal certainty the correctness of the match.
To achieve property 2 matching patches get exactly registered onto each other, by an iterative
registration procedure. Registration is performed with sub-pixel accuracy which results in highly
accurate point correspondences.
Unlike other approaches this algorithm does not simple use the center point of a region
match as final correspondence. Instead, within the matched and registered image regions, new
point correspondences are detected. Each matched image region yields about 20-50 new point
correspondences (property 3). The registration is done by computing the inter-image homography
for each region which maps one region exactly onto the other. Therefore the method is
restricted to planar interest regions only. In fact, non-planar matches will be rejected by this
method.
6.1.1 Matching and registration
Let us now have a close look at the details of the method. It is a 2-step approach consisting of
generating tentative matches and verification (see Algorithm 2 for a compact description). First
we will describe the generation of the tentative matches. Input is a wide-baseline image pair I
and I ′ . In each of the images interest regions are detected. We denote the set of interest regions
in I with L and in I ′ with L ′ . The method is not restricted to one special detector, every affine
interest region detector (see [76] for examples) is possible. After detection a local affine frame
(LAF) is computed for every region in L and L ′ . Next the interest regions are normalized using
the LAF. Normalization tries to remove the perspective distortion of a viewpoint change and two
corresponding regions will appear almost identical. Some normalization methods create multiple
normalized images for a single interest region. The multiple appearances are simply added to
the region set. For the set of normalized regions L and L ′ SIFT descriptors are extracted and
stored in D and D ′ . Each entry in D and D ′ is a vector of length 128 describing the appearance
of a normalized patch using orientation histograms. Corresponding interest regions can now be
found by nearest neighbor search in this 128-dimensional feature space. For efficient matching a
KD-tree K is built with the feature vectors in D ′ . Corresponding interest regions for the entries
in D are now found by querying the KD-tree. The corresponding region for D i is the closest
feature in D ′ return by the KD-tree query. As distance metric the Euclidean distance is used.
To avoid random matches a measure based on the ratio of the nearest to the second closest
feature vector is used. A match is accepted if
d 0
d 1
< d th , (6.1)
where d 0 is the Euclidean distance between the query feature and the nearest neighbor. d 1 is the
distance from the query feature to the second closest feature vector. d th is a user set threshold.
According to [67] an appropriate threshold is 0.8. We denote correspondences detected in this
way as tentative matches. T is the set of tentative matches with T i = (L i , L ′ j ) and is the
prerequisite for the verification step. The tentative matches T are now verified by area based
matching. Correspondence is checked by normalized cross-correlation. This procedure is quite
slow, but it is applied to the set of tentative matches only, which is significantly smaller than
the initial set of detected regions. The cross-correlation is calculated on a registered pair of