Diffusion Processes with Hidden States from ... - FU Berlin, FB MI

3 TheoryRelation 3.1 (Parameter Estimate).λ ∗ = argmaxL (λ|O). (3.27)λOften one has to maximize the log-likelihood ln(L (λ|O)) instead of L (λ|O) for the sake of analyticaland computational reasons. Analytical would mean, that we would ask for the derivatives ofthe likelihood function with respect to the single parameters λ and setting these derivatives equalto zero, in order to get analytical expressions for the maximum likelihood estimators. While forthe problems in consideration the probability distributions of interest are complicated functionsof exponential type, it is anything but easy to accomplish this analytical procedure. Numericalproblems are arising here supplementary, as the likelihood function is a product of probabilitydensities, and since these probability densities can and in fact will be very small-valued, the productfunction will tend to zero very fast. Building the logarithm of L (λ|O) is a solution to bothproblems. In the analytical case we get rid of the exponentials, and in the numerical case theproblem of small numbers is vanishing, as by building the logarithm of the likelihood function thejoint probability product transforms to a sum of probabilities.The heaviness of the problem is depending on the form of P(O|λ). In the case of Gaussian distributionswhere it is λ = (µ,σ 2 ), the problem can be solved by setting the derivative of ln(L (λ|O))with respect to λ equal to zero, and solving directly for µ and σ 2 . This will be accomplished laterin the text (see section 3.10). Else wise when the problem is not so easy to solve, one has to watchfor effective computational techniques to contend with the problem.When we speak about computational methods with regard to a complex modeling problem, wehave to introduce the notion of optimization. Optimization of an arbitrary problem with regardto the belonging parameter space is an important case in computational natural sciences and ingeneral.By this reason a short introduction to this purpose will be delivered in the next section, before wecome to those special optimization algorithms that are needed for the purpose of this thesis.22