Semi-automatic generation of full CityGML models from images

Semi-automatic generation of full CityGML models from images
Kerstin Falkowski, Jürgen Ebert
Work Group Software Technology
University Koblenz-Landau
Universitätstr. 1, 56070 Koblenz, Germany
{falkowski|ebert}@uni-koblenz.de
Abstract
This paper reports on an experiment to generate 3d models
from a pair of images captured by a single calibrated
color camera and to augment and transform them semiautomatically
into an annotated internal graph model. After
some exemplary algorithmic manipulations on the graph
the result is exported using the CityGML format. This case
study shows that in principle full CityGML models might be
generated automatically with all their facets.
1. Introduction
CityGML [1] is a common information model for the
representation of 3d urban objects and an official standard
of the Open Geospatial Consortium (OGC) since August
2008 [3]. Besides representing geometry only, CityGML
can also be used to model topological and semantical properties
of 3d city models and to attach appearance information
like textures. We call these models full CityGML models.
The additional information of full CityGML models allows
for powerful queries in the context of augmented reality
and pose tracking applications. Thus, CityGML is also
suited to function as an interchange format in semanticsbased
processing of 3d objects in computer vision applications.
In this paper we report on an experiment to generate 3d
models from stereo images and to augment and transform
them semi-automatically into an annotated internal graph
model. After some exemplary algorithmic manipulations on
the graph the result is exported using the CityGML format.
This case study shows that in principle full CityGML models
might be generated automatically with all their facets.
Generating a model from stereo images instead of modeling
it from scratch by some graphical modeling tool has
plenty of advantages. Taking photos is much easier than
reconstructing a model from accurate blueprints, and often
photos are even the only source for modeling. Furthermore,
the textures come along for free.
Peter Decker, Stefan Wirtz, Dietrich Paulus
Work Group Active Vision
University Koblenz-Landau
Universitätsstr. 1, 56070 Koblenz, Germany
{decker|wirtz|paulus}@uni-koblenz.de
2. CityGML model generation
In the following the generation process is exemplified
in the context of building recognition in GIS applications.
Several pictures of a building are taken and the task is to
derive a full CityGML model thereof.
The generation process described here consists of four
different phases. At first, a 3d model is extracted from
stereo images, then an internal integrated graph model is
derived from the input. This model is manipulated and refined
to generate a full model. Finally, the resulting model
is exported as a CityGML model and rendered by some appropriate
viewer.
2.1. Extracting geometry from images
To be able to reconstruct a 3d geometry from two images,
one has to find corresponding features in each of them.
Since it is our goal to reconstruct a building, those features
should be relevant to the geometry of that object. If the correct
features are chosen, very few will suffice to reconstruct
a large part of the geometry. Since automatic detection of
suitable features like corners at the top and bottom of buildings
or at corners of windows is rather difficult, we chose
to do it semi-automatically in this case study. Hugin [4] 1
turned out to be an excellent tool for marking corresponding
point features in a pair of images with automatically
improved subpixel accuracy (figure 1).
The resulting hugin files are parsed and the pixel coordinates
of corresponding points are used to estimate a fundamental
matrix. Radial distortion is corrected using previously
acquired calibration data. Since the features were
selected manually, we do not need to worry about outliers.
Using the calibration matrix from our camera, we can extract
the essential matrix which is then decomposed into a
rotation and a translation up to a scalar factor. Having obtained
both projection matrices of the cameras this way, we
1 Hugin is a photo stitcher invented to connect single photos into one
large panorama.

can reconstruct the 3d coordinates of all points by doing a
triangulation with minimal squared back projection error.
This process results in 3d point sets where geometric
primitives like linear rings are assembled and annotated
manually with rough semantical descriptions like ”wall”,
”roof”, ”ground”, ”window”, . . . Each of the rings describes
a part of the building’s geometry and a has list of 2d positions,
representing the building’s wall appearance in terms
of texture coordinates concerning one of the input images.
2.2. Building an integrated model
After extraction, all information is composed to a integrated
model which is kept as a TGraph data structure.
This model contains geometric, topological, semantical and
appearence information. TGraphs [2] are directed graphs
whose vertices and edges are typed, ordered and attributed.
Thus, they are a versatile representation for semantically
annotated, attributed geometric models. The combination
of a graph API (JGraLab) with a graph query language
(GReQL) and a modeling approach for graphs and graph
schemas (grUML) provides a base for definition, creation
and processing of models. Built on this, we derived a
TGraph schema for CityGML and implemented an importer
and an exporter for GML/CityGML models in JGralab.
The TGraph CityGML schema extends the tree-like
XML schema for CityGML because its modeling power is
much higher. This explicit representation of geometry and
boundary representation allows for an automatic model refinement
(section 2.3).
After composition, our examplary integrated model is a
city model containing one building. This building has an
explicit level of detail 1 (lod 1) geometry consisting of a set
of polygons with the given linear rings as exteriors, representing
the building’s parts. It has also got an explicit appearance
in terms of a surface data object relating one of
the input images to texture coordinates for every building
part. Furthermore, it is annotated by an implicit semantics
in terms of the given semantic key words.
2.3. Refining the integrated model
After its generation the TGraph model can be refined automatically.
There are several schema dependent refinement
activities concerning all kinds of model elements. Based
on the (TGraph) CityGML schema integration of semantics
and geometry, there are also special activities accentuating
semantical and/or geometric aspects. The model can be manipulated
using the following refinement techniques.
Basic refinement activities are verifying the existence of
mandatory attributes and element parts and their automatic
addition as well as checking existent attribute values and
element parts and their automatic correction.
Semantical refinement activities are inference and supplementation
of further semantical elements (e.g. building
walls) or further attributes of existent semantical elements.
Therefore general and/or application dependent semantical
background information (e.g. the above mentioned simple
semantical keywords) are required.
Geometric refinement activities are computation and
supplement of further geometric elements (e.g. bounding
boxes) or further attributes of existent geometric elements
(e.g. the area of a surface). Therefore general geometric
background information can be supplied. Another refinement
activity for geometric elements is the analysis of the
element’s position in world coordinates and the automatic
correction of the position or the supplement of position dependent
attributes (e.g. attribute value ”isHorizontal” for a
surface).
2.4. Exporting the CityGML model
After model refinement the integrated model contains
one building and a bounding box for the whole model. The
building has a lod 1 geometry and an appearance (section
2.2). Furthermore the building contains an explicit semantics
in terms of lod 2-4 geometry elements. The lod 2
geometry consists of building walls and a building roof,
represented by those polygons of the lod 1 geometry that
were annotated with the simple semantic keywords ”wall”
or ”roof”. The lod 3-4 geometries consist of windows, represented
by those polygons of the lod 1 geometry that were
annotated with the semantic keyword ”window”. Moreover
it has new inferred building walls with a lod 1-4 geometry
and an appearance. At last, there are a lot of explicit building
attributes, like height and width of a building wall, the
surface area of a building wall, and so forth.
As a proof of concept the semi-automatically generated
TGraph CityGML feature model of a building from two images
was exported to CityGML and visualized using the
IFC-Explorer [5] (figure 2 and 3).
3. Conclusion and future work
We sketched a procedure which leads from two stereo
images via a 3d model and its annotation to an integrated
model of an urban object. This model contains geometric,
topological, semantical and appearance information, which
can be further analyzed, refined and exported via CityGML
to other tools.
Further work will concentrate on a fully automatic
derivation of the 3d model, a more comfortable annotation
of semantics and more services supplied for the integrated
model.

Figure 1. Computer-assisted correspondence
generation using hugin
Figure 2. Semi-automatic generated CityGML
model
Figure 3. Semi-automatic generated CityGML
model (textured)
References
[1] CityGML. Offical web site: http://www.citygml.org, 12 2008.
[2] J. Ebert, V. Riediger, and A. Winter. Graph Technology in
Reverse Engineering, The TGraph Approach. In R. Gimnich,
U. Kaiser, J. Quante, and A. Winter, editors, 10th Workshop
Software Reengineering (WSR 2008), volume 126 of GI Lecture
Notes in Informatics, pages 67–81, Bonn, 2008. GI.
[3] Forschungszentrum Karlsruhe. Official web site of
IFCExplorer for CityGML: http://www.iai.fzk.de/wwwextern/index.php?id=1570&L=0,
12 2008.
[4] G. Groeger, T. H. Kolbe, A. Czerwinski, and C. Nagel.
OpenGIS City Geography Markup Language (CityGML) Encoding
Standard. Technical Report 1.0.0, Open Geospatial
Consortium Inc., 08 2008.
[5] Hugin. Official website: http://hugin.sourceforge.net, 12
2008.