Target Detection and Verification via Airborne ... - IEEE Xplore

IEEE SENSORS JOURNAL, VOL. 10, NO. 3, MARCH 2010 707
Target Detection and Verification via Airborne
Hyperspectral and High-Resolution Imagery
Processing and Fusion
Doron E. Bar, Karni Wolowelsky, Yoram Swirski, Zvi Figov, Ariel Michaeli, Yana Vaynzof, Yoram Abramovitz,
Amnon Ben-Dov, Ofer Yaron, Lior Weizman, and Renen Adar
Abstract—Remote sensing is often used for detection of predefined
targets, such as vehicles, man-made objects, or other specified
objects. We describe a new technique that combines both spectral
and spatial analysis for detection and classification of such targets.
Fusion of data from two sources, a hyperspectral cube and a
high-resolution image, is used as the basis of this technique.
Hyperspectral imagers supply information about the physical
properties of an object while suffering from low spatial resolution.
The use of high-resolution imagers enables high-fidelity spatial
analysis in addition to the spectral analysis. This paper presents
a detection technique accomplished in two steps: anomaly detection
based on the spectral data and the classification phase, which
relies on spatial analysis. At the classification step, the detection
points are projected on the high-resolution images via registration
algorithms. Then each detected point is classified using linear discrimination
functions and decision surfaces on spatial features.
The two detection steps possess orthogonal information: spectral
and spatial. At the spectral detection step, we want very high probability
of detection, while at the spatial step, we reduce the number
of false alarms. Thus, we obtain a lower false alarm rate for a given
probability of detection, in comparison to detection via one of the
steps only. We checked the method over a few tens of square kilometers,
and here we present the system and field test results.
Index Terms—Anomaly suspect, high-resolution chip, probability
of detection–false alarm rate (PD–FAR) curve, spatial
algorithm.
I. INTRODUCTION
WE describe a new technique that combines both spectral
and spatial analysis for detection and classification of
predefined targets, such as vehicles, man-made objects, or other
specified objects. Fusion of data from two sources, a hyperspectral
cube and a high-resolution image, is used as the basis of this
technique.
Manuscript received August 31, 2008; revised January 08, 2009; accepted
January 08, 2009. Current version published February 24, 2010. The associate
editor coordinating the review of this paper and approving it for publication was
Dr. Neelam Gupta.
D. E. Bar, K. Wolowelsky, Y. Swirski, A. Michaeli, Y. Abramovitz,
A. Ben-Dov, O. Yaron, and R. Adar are with Rafael Advanced Defense
Systems Ltd., Haifa 31021, Israel (e-mail: bardor@rafael.co.il).
Z. Figov was with Rafael Advanced Defense Systems Ltd., Haifa 31021,
Israel. He is now with MATE Intelligent Video, Jerusalem 91450, Israel.
Y. Vaynzof was with Rafael Advanced Defense Systems Ltd., Haifa 31021,
Israel. She is now with the Optoelectronics Group, Cavendish Laboratory, University
of Cambridge, Cambridge CB2 1TN, U.K.
L. Weizman was with Rafael Advanced Defense Systems Ltd., Haifa 31021,
Israel. He is now with the School of Computer Science and Engineering, The
Hebrew University of Jerusalem, Jerusalem 91904, Israel.
Digital Object Identifier 10.1109/JSEN.2009.2038664
The Compact Army Spectral Sensor (COMPASS) is a hyperspectral
sensor. In addition, it includes a high-resolution
panchromatic imager. Using COMPASS, Simi et al. [1] describe
the following technique: hyperspectral anomalies were
extracted, and a subregion from the high-resolution image (in
the following text, we refer to this as a “chip”) was matched to
each anomaly. This chip is displayed for the operator.
We take this technique one step further and add automatic
spatial algorithms on the chips at the classification phase. The
technique is described in the next section. Data and results are
described in Sections III and IV. A summary concludes this
paper.
II. TECHNIQUE
We mounted a hyperspectral imager and a high-resolution imager
on an airborne platform. The bore-sighting of the two cameras
was verified. We collected data over different areas, landscapes,
and seasons. The data were transferred to the algorithm
block, whose main steps are as follows.
1) Extract hyperspectral anomaly suspects.
2) Each suspect is matched to a high-resolution chip.
3) Apply spatial algorithms to each chip in order to incriminate
or exonerate the suspects.
4) Pass incriminations on for further investigation.
The first step, extracting hyperspectral anomaly suspects, was
done using unsupervised detection algorithms on the hyperspectral
data. Two detection algorithms were used: local [2] and
global [3]. After applying the algorithms, we obtained the algorithms
results in a fuzzy map of scores, represented by nonnegative
numbers. We used a four-connected neighborhood criterion
in order to group pixels with score above a given threshold score
into segments. The centers of mass of these segments were used
as a list of suspect points.
The second step, matching high-resolution chips to the suspect
points, was done in three substeps: approximate translation
based on global position system (GPS) time tags, improved
translation based on global image matching algorithms (such as
feature based or region based), and final translation based on
local algorithms. At the end of this process, each suspect point
in the hyperspectral image is matched to a point in the high-resolution
image. This point is defined as the center of the chip for
further analysis.
Using linear discrimination functions and decision surfaces
on spatial features, each detected point is classified in the third
step as incriminated or exonerated. The spatial features are built
1530-437X/$26.00 © 2010 IEEE

708 IEEE SENSORS JOURNAL, VOL. 10, NO. 3, MARCH 2010
Fig. 1. Hyperspectral image with anomaly suspects and the matched chips.
Fig. 2. High-resolution image with chips that were incriminated (bold square)
and were exonerated (square) are plotted. For chips A–D, see detailed explanation
in the text.
in three substeps: extract line segments and shadow segments,
build vehicle hypotheses from lines, then to each hypothesis
match a shadow segment [4]. Each hypothesis is assigned a nonnegative
score. The shadow segments are used as support for the
vehicle hypotheses. Thus, when partial shadow is presented the
third substep is omitted.
We used MATLAB and ENVI/IDL software to implement the
algorithms.
To demonstrate the technique, we plot an example in Figs. 1
and 2. At the center of Fig. 1, the band with 700-nm wavelength
from the hyperspectral cube is plotted. White patches represent
pixels that got a score higher than a given threshold. To each
segment of such pixels, a high-resolution chip is matched. Those
chips are seen around the spectral image.
In Fig. 2, a high-resolution image with chips that were incriminated
by the spatial analysis (bold square) and were exonerated
(thin square) are plotted. For three chips, vehicle hypotheses
and shadow segments are plotted. Since no shadow segment was
matched to the vehicle hypothesis in chip A, this chip was exonerated.
In chip D, no vehicle was found, though there was a
car in the hyperspectral image. This is due to the fact that the
car was on the move and there is slight time difference between
capturing a scene by the two imagers.
III. DATA
Two cameras were used to demonstrate this technique.
• A hyperspectral ASIA camera of SPECIM company. The
camera operates at the visual near-IR range with dispersive
system based on a prism-grating-prism component.
The data are collected via the pushbroom technique. Instantaneous
field of view (IFOV) 1 mrad.
• A high-resolution camera with 11 mega pixels, red-greenblue
(RGB) Redlake camera. IFOV 0.1 mrad.
The optics were chosen so that the RGB resolution would be
ten times higher than the resolution of the hyperspectral image.
The image capture rate was chosen, depending on flight speed,
to ensure a slight overlap between images.
We mounted the imagers on a light aircraft. Inertial navigation
system and GPS systems were mounted on the platform and
recorded for the duration of the flight.
Preprocessing of the data was performed for each algorithm
block.
• For the hyperspectral stage: calibration of hyperspectral
raw data, and coregistration of hyperspectral bands [5].
• For the matching stage: Georectification of the hyperspectral
data and RGB image enhancement and cropping.
• For the spatial stage: conversion of RGB chips to gray-level
chips.
We collected data at three different times of year: summer,
spring, and winter. The cloud cover was 0/8, 4/8, and 7/8, respectively,
with some rain during the winter data collection.
The landscape included open fields, forests, and various types
of roads and buildings.
IV. RESULTS
A. Comparing the Parts to the Whole
Operations research was done at each algorithm step by
marking the targets on the images—hyperspectral and high
resolution—and calculating the probability of detection (PD)
and false alarm rate (FAR).
Calculations of PD–FAR graphs are different for each algorithm
block.
At the hyperspectral algorithm step, one obtains a score map.
This map is cut at a threshold to give a list of pixels above the
threshold. We used a connection criterion in order to generate
segments for those pixels. For each segment, its center of mass
is considered as the location of the suspect. Thus, we count suspects
that fall inside a marked target as hits and the others
as false . We repeat the process for different thresholds
to obtain a PD–FAR graph for this algorithm step. A PD–FAR
curve, is calculated from these data based on the total number
of targets in the area considered and its area
At the spatial algorithm step, we looked at the hypotheses’
scores. Thus, each chip gets the highest score of the hypotheses
that appear inside it, or zero score if there is no hypothesis. A
chip is considered as a hit if its score is above the threshold and
there are marked targets inside it.
To check a spatial-only algorithm, we divided the high-resolution
images to chips and performed the aforementioned check
(1)

BAR et al.: TARGET DETECTION AND VERIFICATION VIA AIRBORNE HYPERSPECTRAL AND HIGH-RESOLUTION IMAGERY PROCESSING AND FUSION 709
Fig. 3. Results—comparing the parts to the whole. PD–FAR curves are plotted
for the hyperspectral-only algorithm step (line-dot), for the spatial-only algorithm
step (lined) and for the combined step system (bold line). This analysis
was preformed over 2.94 km with 11 marked targets.
Fig. 4. Comparison of two flight altitudes.
for all chips. For the combined algorithm, we checked the chips
that had been extracted by the matching step.
Each target, in the area checked, is assigned the score of the
chip, which it is in, or zero if it is not included in any chip. If a
target is inside multiple chips, it is assigned the highest score of
those chips. All chips with no targets in them are considered as
false alarms. Thus, for a given score , we count the number of
targets detected and the number of false alarms .
Define and as the number of targets hit and the
number of false alarms, respectively, with a score higher than
the given score
(2)
A PD–FAR curve is calculated, see (1).
We present a comparison between the algorithms on an area
of 2.94 km with 11 marked targets. In Fig. 3, the results for
the hyperspectral-only algorithm step, for the spatial-only algorithm
step, and for the combined steps are presented. Inspection
of the results shows that the number of false alarms is reduced
by an order of magnitude as a result of combining the two algorithms.
At 80% PD, the false alarm incidences are 14, 13, and
1.1 for the three algorithms, respectively. At 90% PD, the false
alarm incidences are 28, 24, and 2.5, respectively.
B. Resolution Compression
To check the dependency of the algorithms on ground resolution,
we compare two flight lines at two different altitudes:
4000 and 6000 ft above ground level. These altitudes produce
ground resolutions of the hyperspectral image of 1.0 1.0 and
1.5 1.5 m to pixel, respectively. A comparison of the results
at these two altitudes is seen in Fig. 4.
There is a reduction in the PD for a given FAR at the higher
altitude flight line. This is due to the loss of spatial information
in the high-resolution images.
(3)
Fig. 5. Combined results over several databases.
C. Combined Results Over Several Databases
We checked the whole algorithm on the combination of three
databases totaling 55 km , 270 targets, different cloud conditions/weather
and different landscapes. The results overall are
shown in Fig. 5. The ground resolutions were in the range of
1.0 1.0 to 1.5 1.5 m /pixel. A global threshold was chosen
for the hyperspectral algorithm step.
Both winter and summer experiments showed similar results
of 3 false alarms per square kilometer with 80% detection. In
the spring experiment, we saw a reduction in the detection, 62%
detection for 3 false alarms per square kilometer. This may be
due to the scattered cloud cover during this experiment.
Although that the three databases are very different from each
other. The individual PD–FAR curves,
(where
is the set of databases), are similar for the different databases.
Therefore, it was justified to combine those curves, to get a
global PD–FAR curve
(4)

710 IEEE SENSORS JOURNAL, VOL. 10, NO. 3, MARCH 2010
For the databases we checked, one may expect 2.5 false
alarms per square kilometer with 70% detection or 4 false
alarms per square kilometer with 80% detection.
V. SUMMARY
A system—composed of airborne sensors and various automatic
algorithms—was presented. We present results that compared
the system to its parts. The results show a reduction by
an order of magnitude of the number of false alarms for a given
PD. Thus, the fusion of spectral and spatial algorithms is better
than the sum of the parts.
A reduction in the PD was observed at higher altitude. This
is due to the loss of spatial information in the high-resolution
images.
The results of the combined algorithms are similar when the
cloud cover is 0/8 or 8/8. However, when one has scattered cloud
cover—such as 4/8—a reduction in the PD is observed and further
investigations should be made.
Reduction of false alarm rate needs further investigation. Due
to the algorithmic process, the false alarms detected were rectangular.
Thus, improving spatial algorithms might not improve
results significantly. However, improving unsupervised algorithms
on the hyperspectral data might reduce false alarms due
to better understanding of the background model [6].
We checked the system over different landscapes—open
fields, forests, roads, buildings—and in different illumination
and weather conditions—seasons, sun angles, and cloud coverage.
The results showed that the algorithms are robust.
ACKNOWLEDGMENT
The authors would like to thank the members of the Image
Processing Group at Rafael Advanced Defense Systems, Ltd.,
for various algorithms used in this research, and all the people
who helped in preparing and carrying out the experiments and
data collection. They would also like to thank A. Kershenbaum
for his helpful comments.
REFERENCES
[1] C. G. Simi, E. M. Winter, M. J. Schlangen, and A. B. Hill, S. S. Shen
and M. R. Descour, Eds., “On-board processing for the COMPASS,
algorithms for multispectral, hyperspectral, and ultraspectral imagery
VII,” Proc. SPIE, vol. 4381, pp. 137–142, 2001.
[2] I. R. Reed and X. Yu, “Adaptive multiple-band CFAR detection of
an optical pattern with unknown spectral distribution,” IEEE Trans.
Acoust., Speech Signal Process., vol. 38, no. 10, pp. 1760–1770, Oct.
1990.
[3] O. Kuybeda, D. Malah, and M. Barzohar, “Rank estimation and redundancy
reduction of high-dimensional noisy signals with preservation
of rate vectors,” IEEE Trans. Signal Process., vol. 55, no. 12, pp.
5579–5592, Dec. 2007.
[4] Z. W. Kim and R. Nevatia, “Uncertain reasoning and learning for
feature grouping,” Comput. Vis. Image Understanding, vol. 76, pp.
278–288, 1999.
[5] Z. Figov, K. Wolowelsky, and N. Goldberg, L. Bruzzone, Ed., “Co-registration
of hyperspectral bands,” Image Signal Process. Remote Sens.
XIII. Proc. SPIE, vol. 6748, pp. 67480s-1–67480s-12, 2007.
[6] L. Boker, S. R. Rotman, and D. G. Blumberg, “Coping with mixtures
of backgrounds in a sliding window anomaly detection algorithm,” in
Proc. SPIE, Electro-Opt. Infrared Syst.: Technol. Appl. V, 2008, vol.
7113, pp. 711315-1–711315-12.
Doron E. Bar was born in 1962. He received the
Ph.D. degree in applied mathematics from the Technion,
Haifa, Israel, in 1996.
Since 1999, he has been with Rafael Advanced
Defense Systems Ltd., Haifa, Israel, as an Image
Processing Engineer. His current research interests
include computer visions, image processing, and
remote sensing tasks.
Karni Wolowelsky, photograph and biography not available at the time of
publication.
Yoram Swirski was born in Jerusalem, Israel,
in 1955. He received the B.Sc. degree in physics
from Tel-Aviv University, Tel-Aviv, Israel, in 1976,
the M.Sc. degree (cum laude) in applied physics
and electrooptics from The Hebrew University of
Jerusalem, Israel, in 1978, and the Ph.D. degree in
physics from the Technion, Haifa, Israel, 1992.
Since 1979, he has been with Rafael Advanced Defense
Systems Ltd., Haifa, where he is currently engaged
in research on IR radiometry, image generation
and simulation, and computer vision.
Zvi Figov studied computer science and mathematics
at Bar-Ilan University, Ramat-Gan, Israel. He
received the B.Sc. degree, in 1999, and the M.Sc.
degree with specialization in neuroscience, in 2002,
from Bar-Ilan University.
From 2001 to 2008, he was with Rafael Advanced
Defense Systems Ltd. (formerly Rafael Armament
Development Authority), Israel, where he was
engaged in research on image processing and remote
sensing. He is currently with the MATE Intelligent
Video, Jerusalem, Israel, where he is engaged in
research on developing video analytics, computer vision, real-time analytics,
and remote sensing.
Ariel Michaeli, photograph and biography not available at the time of
publication.
Yana Vaynzof was born in Tashkent, Uzbekistan, on
December 2 1981 and immigrated to Israel in 1991.
She received the B.Sc degree (summa cum laude) in
electrical engineering from the Technion-Israel Institute
of Technology, Haifa, Israel, in 2006, and the
M.Sc. degree in electrical engineering from Princeton
University, Princeton, NJ, in 2008. She is currently
working towards the Ph.D. degree in physics at the
Optoelectronics Group, Cavendish Laboratory, University
of Cambridge, Cambridge, U.K.
During her undergraduate studies, she worked
part-time as a Student Engineer in Rafael Advanced Defense Systems Ltd.,
Haifa, in the Image-Processing Group of the Missile Division. During
2000–2002, she was with the Israeli Defense Forces as an Instructor in the
Flight Academy. Her current research interests include development of hybrid
polymer solar cells and the improvement of their efficiency and stability.
Miss Vaynzof was the recipient of a number of fellowships and awards, including
the Pinzi Award for Academic Excellence (2004), Knesset (Israeli Parliament)
Award for contribution to the Israeli Society (2005), Gordon Y. Wu
Fellowship (2006–2008), and the Cavendish Laboratories Award (2008).

BAR et al.: TARGET DETECTION AND VERIFICATION VIA AIRBORNE HYPERSPECTRAL AND HIGH-RESOLUTION IMAGERY PROCESSING AND FUSION 711
Yoram Abramovitz was born in Affula, Israel, in
1962. He received the B.Sc. and M.Sc. degrees in
physics from the Technion, Haifa, Israel, in 1994.
Since 2000, he has been with Rafael Advanced Defense
systems Ltd., Haifa, where he is currently engaged
in research on remote sensing R&D of electrooptical
systems and radiometric measurements.
Lior Weizman received the B.Sc. (with distinction)
and M.Sc. degrees in electrical engineering from
Ben-Gurion University of the Negev, Beer-Sheva,
Israel, in 2002 and 2004, respectively. He is currently
working towards the Ph.D. degree at the School of
Computer Science and Engineering, The Hebrew
University of Jerusalem, Israel.
From 2005 to 2008, he was with Rafael Advanced
Defense Systems Ltd., Haifa. His current research interests
include image processing, pattern recognition,
and statistical signal processing.
Amnon Ben-Dov was born in 1955. Since 1981, he has been an Electronics
Practical Engineer at the Physics Development Laboratories, Rafael Advanced
Defense Systems Ltd., Haifa, Israel.
Ofer Yaron was born in 1965. He received the B.Sc. degree in physics from the
Technion, Haifa, Israel, in 1992, and the M.Sc. degree in physics from Tel-Aviv
University, Tel-Aviv, Israel, in 1998.
Since 1992, he has been with Rafael Advanced Defense Systems Ltd., Haifa,
where he is currently engaged in research on remote sensing, image generation,
and simulation.
Renen Adar was born in Afula, Israel, in 1955. He
received the B.Sc. and M.Sc. (cum laude) degrees
in physics and mathematics from The Hebrew
University of Jerusalem, Israel, and the D.Sc. degree
in microelectronics from the Department of
Electrical Engineering, Technion-Israel Institute of
Technology, Haifa, Israel.
From 1989 to 1993, he was a Member of Technical
Staff with the Passive Optical Component Research,
AT&T Bell Laboratories, Murray Hill, NJ. Since
1994, he has been with Rafael Advanced Defense
Systems Ltd., Haifa, where he is currently engaged in research on algorithm
development activities related to machine vision and image recognition tasks.