research activities in 2007 - CSEM

Generic Framework for Feature Extraction in Vision
T. Zamofing, P. Seitz
A vision system that classifies objects in complex, natural scenes has been realized. This software project tries to map structures of the cerebral
cortex into a hierarchical binary matched filter [1] implemented on a computer.
One of the most ambitious goals in digital image processing is
the development of universal classification algorithms,
capable of “understanding” natural scenes with robustness
and reliability similar to those demonstrated by natural vision
systems, especially by the human visual system. In particular,
the functionality of such natural vision systems under very
adverse conditions is a highly desirable property for practical
applications in machine vision. This robustness can
encompass translation invariance, rotation invariance, as well
as a high tolerance to distortion (e.g. perspective), to partial
occlusion, to reflections and shadows, to unsharp images
(focus, movement). It should moreover be independent of
local contrast and illumination variations, independent of the
background and independent of object texture (surface
texture, dirt, etc.). All classification algorithms are faced, from
the outset, with the problem that the given continuous-tone
images contain a vast amount of information that must be
substantially reduced in order to label the different image
areas (“the objects”) correctly, according to the class to which
they belong.
This feature-matching process is realized as a binary matched
filter following three assumptions (see Figure 1). (1) Local
orientation is a central source of relevant image information.
This assertion is corroborated by neurobiologists’ findings on
how natural vision systems work, using directionally selective
filter banks. The process employs local orientations as the
fundamental picture primitives, rather than the more usual
edge locations. (2) The procedures are based on retaining
and exploiting the local arrangement of features of different
complexity in an image. The technique is based on the
accumulation of evidence in binary channels, followed by a
weighted, non-linear sum of the evidence accumulators. (3)
The algorithm proceeds in a hierarchical fashion, starting at
low feature complexity, and raising the level of abstraction at
each successive processing step.
Figure 1: The feature matching process is based on a successive
hierarchical approach
This algorithm can be implemented very easily in a computer
program. The essence of the algorithm (i.e. without graphics
38
and file handling) can be written using 60–70 lines of a highlevel
language (e.g. Pascal, Fortran or C). Because of the
homogeneity and the simple, reusable characteristics of the
feature-matcher, it should be possible, with reasonable effort,
to develop a hardware implementation running at low power in
real time.
The current implementation uses a black and white firewire
camera with a resolution of 640x480 pixels and is
implemented on a PC using SSE (single instruction multiple
data) for speedup. With this setup a frame rate of 5 to 25
frames per second (depending on the complexity of the
templates) could be achieved. The sample below shows an
example (Figure 2) with its templates to detect traffic signs
that runs at about 20 fps.
Figure 2: Application example: Identification of traffic signs
Future work could use the ViSe (www.csem-devise.com)
sensor. The advantage of the ViSe sensor is that it directly
delivers orientation images with a high dynamic range.
Furthermore a FPGA implementation will lead to a smart, low
power and portable vision system.
[1] G. Lang, P. Seitz, „Robust classification of arbitrary object
classes based on hierarchical spatial feature-matching“, MVA,
1997