Fusion of Visual and Thermal Face Recognition Techniques: A ...

2.2 Fusion of Visual and Thermal Images
With use of the database as shown in Table 1, we fused visual and thermal images.
Ideally, the fusion of different sensors can be done by pixel-wise weighted summation of
visual and thermal images.
F(x, y) = a(x, y)V(x, y) + b(x, y)T(x, y)
(1)
where F(x, y) is a fused output of a visual image, V(x, y) , and a thermal image, T(x, y) ,
while a(x, y) and b(x, y) represent the weighting factor of each pixel. A fundamental
problem is: which one has more weight at the pixel level. This can be answered if we
know the illumination direction which affects the face in the visual images and other
variations which affect the thermal images. Illumination changes in the visual images and
facial variations after exercise in the thermal images are also one of challenging problems
in face recognition technology. Instead of finding each weight, we make use of the
average of both modalities constraining both weighting factors a(x, y) , b(x, y) as 1.0. The
average of visual and thermal images can compensate variations in each other, although
this is not a perfect way to achieve data fusion. Figure 6(c) shows a fused image based on
average intensity using (a) visual and (b) thermal images. We have produced the same
number of fused images using datasets as described in Table.1. The fused output contains
visual and thermal characteristics at the same time, thus, can be regarded to be more
informative and synthetic than the original inputs.
21