Diffusion Processes with Hidden States from ... - FU Berlin, FB MI

3.13 Global Optimization MethodsEven though genetic algorithms seem to include a high power in the determinationof global optima, they suffer from certain facts. First, no guarantee exists to reallydetermine the global optimum. Absolute guarantee of a global minimum exists only,if the whole parameter space has been sampled. But even though genetic algorithmsare just heuristics, they provide a powerful way to explore the parameter space andrestrict to essential regions in that parameter space.Especially when applied to HMM-VAR additional problems have to be taken intoaccount:• The parameter space of HMM-VAR parameters is large,• numerical instabilities and problems due to improper parameters can occur duringHMM-VAR calculations,• mutation of HMM-VAR parameters is difficult,• the likelihood landscape is far from being ’nice’."3.13.2 Simulated Annealing (SA)Simulated Annealing (SA) is a global optimization algorithm, originally developed by Kirkpatricket al. [52] and has a physical background which is laying in the area of material sciences. Kirkpatrickwas inspired by Metropolis et al. [51] who have developed a Monte Carlo method for“calculating the properties of any substance which may be considered as composed of interactingindividual molecules”.When heating a metal and let it cool down again, the overall atomic configuration of the metaltend to a ground state, meaning that with decreasing temperature the inner energy of the systemdecreases, until it has reached an energy minimum (see Figure 3.17). This so called annealingprocess has to be slow enough, since when it is too fast, the material particles will be in an irregularconfiguration afterwards, and due to this the material possesses bad energy balance.Physically the probability distribution of a configuration, characterized by a set of particle positions{q k }, is given by the Boltzmann distribution(P({q k }) = exp − E({q )k}),k B Twhile E({q k }) denotes the energy of the associated configuration, T the temperature of the systemand k B the Boltzmann’s constant (k B = 1.380650524 · 10 −23 J/K).While Metropolis et al. provided an exact copy of this physical process [46], Kirkpatrick et al.advanced this treatment to a global optimization method for general optimization problems, wherethe energy function can be replaced by an arbitrary objective function f : X ↦→ R, with X denotingthe problem space.The SA algorithm starts with an initial individual p i ∈ X and one iteration randomly mutates thisindividual to a new one p i+1 , calculates the energy difference of the two individuals∆E = f (p i+1 ) − f (p i )59