Approximation of Hessian Matrix for Second-order SPSA Algorithm ...
Approximation of Hessian Matrix for Second-order SPSA Algorithm ...
Approximation of Hessian Matrix for Second-order SPSA Algorithm ...
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
ABSTRACT<br />
gives algorithms that can be efficiently implemented. Application to the Steiglitz-McBride (SM)<br />
and Simple Hyperstable Adaptive Recursive Filter (SHARF) algorithms is presented. Also, our<br />
proposed M2-<strong>SPSA</strong> algorithm is presented in <strong>order</strong> to get the coefficients in a lattice <strong>for</strong>m more<br />
efficiently and with a lower computational cost and complexity. The results are compared with<br />
previous lattice versions <strong>of</strong> these algorithms. These previous lattice versions may fail to<br />
preserve the stability <strong>of</strong> stationary points.<br />
Finally, the M2-<strong>SPSA</strong> algorithm is addressed to the problem <strong>of</strong> estimation <strong>of</strong> unknown static<br />
parameters in non-linear state-space models. The M2-<strong>SPSA</strong> algorithm can generate maximum<br />
likelihood estimates efficiently. The per<strong>for</strong>mance <strong>of</strong> the proposed algorithm is assessed through<br />
simulation. Here, the M2-<strong>SPSA</strong> algorithm is compared with the finite difference stochastic<br />
approximation (FDSA) in <strong>order</strong> to show its efficiency.<br />
There<strong>for</strong>e, in this dissertation, we have proposed a modification to <strong>SPSA</strong> algorithm where the<br />
main objective is to estimate the parameters in complex systems, improve the convergence and<br />
reduce the computational cost. Then, this modification to the simultaneous perturbation seems<br />
particularly useful when there are number <strong>of</strong> parameters to be identified is very large or when<br />
the observed values <strong>for</strong> what is to be identified can only be obtained via an unknown<br />
observation system.<br />
Finally, this dissertation is organized as follows. In Chapter 1, we describe an introduction to<br />
<strong>SPSA</strong>, so that we explain the mean concepts, advantages, disadvantages, recursions,<br />
<strong>for</strong>mulation and implementation <strong>of</strong> <strong>SPSA</strong>. Our proposed <strong>SPSA</strong> algorithm is analyzed in detail<br />
in Chap. 2. The asymptotic normality, the <strong>Hessian</strong> estimation and the efficiency between<br />
M2-<strong>SPSA</strong> and the previous versions <strong>of</strong> <strong>SPSA</strong> are shown. In addition, we show how the<br />
M2-<strong>SPSA</strong> algorithm is applied to parameter estimation, and prove its efficiency in several<br />
simple numerical simulations. The first important application <strong>of</strong> M2-<strong>SPSA</strong> algorithm is<br />
described in Chap. 3, in this case, in the control area; M2-<strong>SPSA</strong> is applied to parameter<br />
estimation <strong>of</strong> some methods <strong>for</strong> controlling the vibration in the proposed system. Other<br />
application <strong>for</strong> M2-<strong>SPSA</strong> algorithm is described in Chap. 4. In this application, our proposed<br />
algorithm is applied to signal processing, here M2-<strong>SPSA</strong> calculates the coefficients in some<br />
adaptive algorithms. In the final application, M2-<strong>SPSA</strong> algorithm is addressed to the problem <strong>of</strong><br />
estimation <strong>of</strong> unknown static parameters in non-linear state-space models, this is described in<br />
Chap. 5. Finally, the conclusions and future work are given in Chap. 6.<br />
v