Approximation of Hessian Matrix for Second-order SPSA Algorithm ...

1.8 VERSIONS OF SPSA ALGORITHM
the Hessian matrix of L ˆ θ ). Hence, equation (1.10) is a stochastic analogue of the
( k
well-known Newton-Raphson algorithm of deterministic optimization. Since gˆ
( ˆ θ ) has a
known form, the parallel recursions in equations (1.9) and (1.10) can be implemented once that
k
k
Ĝk
is specified. The SP gradient approximation requires two measurements of
L( ⋅):
y
( + )
k
and
y . These represent measurements at design levels θˆ k
+ ck∆k
and θˆ
k
− ck∆k
respectively, where
(−)
k
c
k
is a positive scalar and
∆
k
represents a user-generated random vector satisfying certain
regularity conditions, e.g
∆
k
being a vector of independent Bernoulli ± 1 random variables
satisfies these conditions but a vector of uniformly distributed random variables does not. The
SP comes from the fact that all elements of
θˆ k
are perturbed simultaneously in forming
gˆ
( ˆ θ ) , as opposed to the finite difference form, where they are perturbed one at time. To
k
k
perform one iteration of (1.9) and (1.10), one additional measurement, say
(0)
y
k
is required; this
measurement represents an observation of L (⋅)
at the nominal design level θˆ k .
Main Advantage:
- 1st-SPSA gives region(s) where the function value is low, and this allows to conjecture in
which region(s) is the global solution.
- 2nd-SPSA is based on a highly efficient approximation of the gradient based on loss function
measurements. In particular, on each iteration the SPSA only needs three loss measurements to
estimate the gradient, regardless of the dimensionality of the problem. Moreover, the 2nd-SPSA
is grounded on a solid mathematical framework that permits to assess its stochastic properties
also for optimization problems affected by noise or uncertainties. Due to these striking
advantages, 2nd-SPSA is recently used as optimization engine for adaptive control problems.
17