U. Glaeser

must be found via a finite set of observations. In this method, this problem is solved under the assumption
of a random texture input (the plane in the frequency domain consists of a single constant value).
Optical Flow via the 3-D Gabor Transform
One shortcoming of a filterbank approach (if the filters are not orthogonal or do not provide a complete
basis) is the possibility of loss. Using the 3-D Gabor functions as the basis of a transform resolves this
problem. A sequence of dimension N × M × P can then be expressed at each discrete point (x m, y n, t p) as
© 2002 by CRC Press LLC
J−1 K−1 L−1 Q−1 R−1 S−1
( ) =
cxq ,yr ,ts ,ξxj ,ξyk ,ξtl f x m, y n, t p
∑∑∑
∑∑∑
j=0 k=0 l=0 q=0 r=0 s=0
(28.55)
where J⋅K⋅L⋅Q⋅R⋅S = N⋅M⋅P for completeness, the functions gxq ,y denote the
r ,ts ,ξxj ,ξyk ,ξ (x
tl m, yn, tp) Gabor basis functions with spatiotemporal and spatiotemporal-frequency centers of (xq, yr, ts) and
(ξxj , ξyk , ξtl ) respectively, and cxq ,y are the associated coefficients. Note that these coefficients
r ,ts ,ξxj ,ξyk ,ξtl are not found by convolving with the Gabor functions, since the functions are not orthogonal. See [30]
for a survey and comparison of methods for computing this transform.
In the case of uniform translational motion, the slope of the planar spectrum is sought, yielding the
optical flow vector r.
A straightforward approach to estimating the slope of the local spectra [31,32] is
to form vectors of the ξx, ξy, and ξt coordinates of the basis functions that have significant energy for
each point in the sequence at which basis functions are centered. From equation 20, the optical flow vector
and the coordinate vectorsξ x, ξ y, and ξ t at each point are related as
ξ t = – ( rx ξ x + ry ξ y ) = – Sr
(28.56)
where S = (ξx |ξy). An LMS estimate of the optical flow vector at a given point can then be found using
the pseudo inverse of S:
(28.57)
In addition to providing a means for motion estimation, this approach has also proven useful in
predicting the apparent motion reversal associated with temporal aliasing [33].
Wavelet-Based Methods
A number of wavelet-based approaches to this problem have also been proposed. In [34–37], 2-D wavelet
decompositions are applied frame-by-frame to produce multi-scale feature images. This view of motion
analysis exploits the multiscale properties of wavelets, but does not seek to exploit the frequency domain
properties of motion. In [38], a spatiotemporal (3-D) wavelet decomposition is employed, so that some
of these frequency domain aspects can be utilized. Leduc et al. explore the estimation of translational,
accelerated, and rotational motion via spatiotemporal wavelets in [39–44]. Decompositions designed and
parameterized specifically for the motion of interest (e.g., rotational motion) are tuned to the motion
to be estimated.
28.6 Image Sequence Compression
rest
− S T ( S)
1 –
=
g xq ,y r ,t s ,ξ xj ,ξ yk ,ξ tl
( xm, yn, tp) Image sequences represent an enormous amount of data (e.g., a 2-hour movie at the US HDTV resolution
of 1280 × 720 pixels, 60 frames/second progressive, with 24 bits/pixel results in 1194 Gbytes of data).
This data is highly redundant, and much of it has minimal perceptual relevance. One approach to reducing
this volume of data is to apply still image compression to each frame in the sequence (generally referred
to as intraframe coding). For example, the JPEG still image compression algorithm can be applied frame
by frame (sometimes referred to as Motion-JPEG or M-JPEG). This method, however, does not take
advantage of the substantial correlation, which typically exists between frames in a sequence. Compression
techniques which seek to exploit this temporal redundancy are referred to as interframe coding methods.
S T
ξ t
⋅