Approximation of Hessian Matrix for Second-order SPSA Algorithm ...

2.2 THE SPSA ALGORITHM RECURSIONS
1st-SPSA [17]:
θ ˆ = θˆ
− a gˆ
( θˆ
), 0,1,2,...
(2.1)
k + 1 k k k k
k =
2nd-SPSA [18]:
ˆ ˆ
−1
θ = θ − a H gˆ
( ˆ ), H = f ( H )
(2.2 a)
k + 1 k k k k
θ
k k k k
= k
1
H H
ˆ
1
+ H , = 0,1,2,...
k + 1
− k + 1
k
(2.2 b)
k k
k
where
a
k and a
k are the scalar gain series that satisfy certain SA conditions [18], ĝ
k
is
the SP estimate of the loss function gradient that depends on the gain sequence
c
k
(representing a difference interval of the perturbations),
Hˆ
k
is the SP estimate of the Hessian
matrix, and
f
k maps the usual non-positive-definite H
k
to a positive-definite pxp matrix.
The two recursions are showed in Fig. 2.1. Let
∆
k be a user-generated mean zero random
vector of dimension p with its components being independent random variables.
Fig. 2.1. The two-recursions in 2nd-SPSA algorithm
(solid-line eq. 2.2 a, dashed-line eq. 2.2 b).
The i-th element of the loss function gradient is given by [18].
( gˆ
) = (2c
∆
k
i
k
ki
−1
) [ y( ˆ θ + c ∆ ) − y( ˆ θ −c
∆ )], i=1, 2, … , p (2.3)
k
k
k
k
k
k
21