Fusion of Visual and Thermal Face Recognition Techniques: A ...

false recognition and rejection rates. If the face detection is confident enough in visual
images and eyeglasses are detected and replaced in thermal images, we can fuse visual
and thermal images for a better recognition system. In other words, this system can be
operating with either visual, thermal, or data fusion images depending on different
situations. On the other hand, decision fusion systems rely on the output of both visual
and thermal recognition results and do not need any sensor fusion.
The remainder of this paper documents the details of our algorithms developed to
fuse visual and thermal images or recognition results properly. Chapter 2 deals with
image acquisition from public database, our own image acquisition device, and data
fusion. In Chapter 3, automatic eyeglasses detection algorithms are proposed with ellipse
fitting methods and statistical thermal facial variations. After detecting eyeglasses,
eyeglass removal and replacement are also discussed. Chapter 4 contains a comparison of
visual, thermal, and fused face recognition, and claims fusion of data or fusion of
decisions can increase the overall performance of face recognition systems which may be
an important step toward more reliable systems. Finally we conclude in Chapter 5.
2. Data Acquisition
In order to evaluate face recognition algorithms, one should use standard databases
so that the evaluation can be reasonable. The FERET database for the visual images is
used in comparison with holistic and feature based recognition algorithms in section 4.1
and for the robustness of FaceIt® which we mainly discuss in section 4.2. Regarding
visual and thermal images, the Equinox Corporation collects an extensive database of
face imagery using co-registered broadband-visible/LWIR, MWIR, and SWIR camera
15