U. Glaeser

where the superscripts n and n + 1 indicate the iteration number, λ is a parameter allowing a tradeoff
between smoothness and errors in the flow constraint equation, and rx( k, l)
and ry( k, l)
are local
averages of rx and ry. The updated estimates are thus the average of the surrounding values, minus an
adjustment (which in velocity space is in the direction of the intensity gradient).
The previous discussion relied heavily on smoothness of the flow field. However, there are places in
image sequences where discontinuities should occur. In particular, the boundaries of moving objects
should exhibit discontinuities in optical flow. One approach taking advantage of smoothness but allowing
discontinuities is to apply segmentation to the flow field. In this way, the boundaries between regions
with smooth optical flow can be found, and the algorithm can be prevented from smoothing over these
boundaries. Because of the “chicken-and-egg” nature of this method (a good segmentation depends on
a good optical flow estimate, which depends on a good segmentation …), it is best applied iteratively.
Spatiotemporal-Frequency-Based Methods
It was shown in section 28.2 that motion can be considered in the frequency domain, as well as in the
spatial domain. A number of motion estimation methods have been developed with this in mind. If the
sequence to be analyzed is very simple (has only a single motion component, for example) or if motion
detection alone is required, the Fourier transform can be used as the basis for motion analysis, as examined
in [23–25]; however, due to the global nature of the Fourier transform, it cannot be used to determine
the location of the object in motion. It is also poorly suited for cases in which multiple motions exist
(i.e., when the scene of interest consists of more than one object moving independently), since the
signatures of the different motions are difficult (impossible, in general) to separate in the Fourier domain.
As a result, although Fourier analysis can be used to illustrate some interesting phenomena, it cannot be
used as the basis of motion analysis methods for the majority of sequences of practical interest.
To identify the locations and motions of objects, frequency analysis localized to the neighborhoods of
the objects is required. Windowed Fourier analysis has been proposed for such cases [26], but the accuracy
of a motion analysis method of this type is highly dependent on the resolution of the underlying
transform, in both the spatiotemporal and spatiotemporal-frequency domains. It is known that the
windowed Fourier transform does not perform particularly well in this regard. Filterbank-based
approaches to this problem have also been proposed, as in [27]. The methods examined below each
exploit the frequency domain characteristics of motion, and provide spatiotemporally localized motion
estimates.
Optical Flow via the 3-D Wigner Distribution
Jacobson and Wechsler [28] proposed an approach to spatiotemporal-frequency, based derivation of
optical flow using the 3-D Wigner distribution (WD). Extending the 2-D definition given earlier, the 3-D
WD can be written as
Wf ( x, y, t, ξx, ξy, ξt) f x α
-- , y
2
β
⎛ τ
+ + --, t + --⎞
f
⎝ 2 2⎠
∗ x α
– -- , y
2
β
∞ ∞ ∞
τ
=
⋅ ⎛ – -- , t – --⎞
∫ ∫ ∫
⎝ 2 2⎠
It can be shown that the WD of a linearly translating image with velocity r = (rx, ry) is
© 2002 by CRC Press LLC
– ∞ – ∞ – ∞
× e −j2π(αξx βξ + y + τξt )
dα dβ dτ
Wf ( x, y, t, ξx, ξy, ξt) = δ( rxξ x + ryξ y + ξt) ⋅ Wf ( x– rxt, y – ryt, ξx,ξ y)
(28.51)
(28.52)
which is nonzero only when rxξx + ryξy + ξt = 0.
For a linearly translating image, then, the local spectra Wfx,y,t ( ξx,ξ y,ξ t)
contain energy only in a plane
(as in the Fourier case) the slope of which is determined by the velocity. Jacobson and Wechsler proposed
to find this plane by integrating over the possible planar regions in these local spectra (via a so-called
“velocity polling function”), using the plane of maximum energy to determine the velocity.