Omar Khalil - Queen's University

More documents

Recommendations

Info

Contents 1 Introduction 3 2 Background 3 3 Summary of Results 6 4 Critique and Concluding remarks 8 5 References 11 2
1 Introduction The problem of establishing recurrence and ergodicity of Markov chains is a fundamental issue in probability theory as well as in the numerous techniques where stochastic stability of Markov chains plays an important role such as dynamic programming and Markov Chain Monte Carlo. One of the main techniques of determining whether a Markov chain is stable or not is the standard Foster-Lyapunov approach which requires that the one-step mean drift of the chain be negative outside of some appropriately finite set. Although this technique has proved powerful in analysing many models, as the models get more and more complex analy- sis of the one-step drift often proves somewhat difficult. Malyˇshev and Men’ˇsikov developed a refinement of this approach for countable state space chains in [5] allowing the drift to be negative after a number of steps depending on the initial state. The goal of this project is to summarize and discuss an important paper [1] by Sean P. Meyne and R. L. Tweedie which shows that these countable state space results are special cases of a wider class of chains. The paper also develops a random step approach giving similar conclusions as well as state dependent drift conditions sufficient to establish geometric ergodicity of the chain; i.e, the n-step transition probabilities converge to their limits suffi- ciently geometrically fast. Although we will attempt to introduce the necessary mathematical concepts as we go by, a general background in probability theory and the theory of Markov chains is assumed and one may consult a text such as [9] which covers all the necessary material. 2 Background Throughout this project we will be working with an underlying measure space (X, F), and a time homogeneous Markov chain Φ = {Φn, n ∈ Z+}. The transition probabilities of the chain 3
Page 1: State-Dependent Criteria for Conver
Page 5 and 6: for all x ∈ X and n ∈ Z+. We me
Page 7 and 8: We will also define for non-negativ
Page 9 and 10: not necessarily geometric in nature
Page 11: 5 References [1] Meyn, S.P. and Twe

Omar Khalil - Queen's University

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?