Fast Robust Large-scale Mapping from Video and Internet Photo ...

tablish the spatial relationship between the images. In principle we can use
feature based techniques similar to the method of Philbin et al. and Zheng
et al. [16, 17] to detect related frames within the photo collection. These
methods use computationally very intensive indexing based on local image
features (keypoints) followed by loose spatial verification. Their spatial constraints
are given by a 2D affine transformation or filters based on proximity
of local features. Our method effectively enforces 3D structure from motion
constraints (SfM constraints) for the dataset. Similarly methods like [51, 52]
enforce 3D SfM constraints for the full set of registered frames. Their methods
first exhaustively evaluate all possible pairs for a valid epipolar geometry
and then enforce the stronger multi-view geometry constraints. Our method
avoids the prohibitively expensive exhaustive pairwise matching using an initial
stage in which images are grouped using global image features prior to
indexing based on local features. This gives us a bigger gain in efficiency
and an improved ability to group similar compositions mainly corresponding
to similar viewpoints. We then enforce the SfM constraints on these groups
which reduces the complexity of the computation by orders of magnitude.
Snavely et al. [54] also reduced the complexity of the SfM constraints by
minimizing the number of image pairs for which it is computed to a minimal
set expected to obtain the same overall reconstruction.
4.4.1. Efficiently Finding Corresponding Images in Photo Collections
To efficiently identify related images our system uses the gist feature [78],
which encodes the spatial layout of the image and perceptual properties of
the image. The gist feature was found to be effective for grouping images by
perceptual similarity and retrieving structurally similar scenes [79, 80]. To
22