Linear Algebra

Topic: Markov Chains 2770 − 01 − 00 − 12 − 01 − 10 − 23 − 02 − 11 − 20 − 34 − 03 − 12 − 21 − 30 − 44 − 13 − 22 − 31 − 44 − 23 − 32 − 44 − 33 − 4n = 0 n = 1 n = 2 n = 3 n = 4 n = 5 n = 6 n = 71 0 0 0 0 0 0 00 0.5 0 0 0 0 0 00 0.5 0 0 0 0 0 00 0 0.25 0 0 0 0 00 0 0.5 0 0 0 0 00 0 0.25 0 0 0 0 00 0 0 0.125 0 0 0 00 0 0 0.375 0 0 0 00 0 0 0.375 0 0 0 00 0 0 0.125 0 0 0 00 0 0 0 0.0625 0.0625 0.0625 0.06250 0 0 0 0.25 0 0 00 0 0 0 0.375 0 0 00 0 0 0 0.25 0 0 00 0 0 0 0.0625 0.0625 0.0625 0.06250 0 0 0 0 0.125 0.125 0.1250 0 0 0 0 0.3125 0 00 0 0 0 0 0.3125 0 00 0 0 0 0 0.125 0.125 0.1250 0 0 0 0 0 0.15625 0.156250 0 0 0 0 0 0.3125 00 0 0 0 0 0 0.15625 0.156250 0 0 0 0 0 0 0.156250 0 0 0 0 0 0 0.15625Note that evenly-matched teams are likely to have a long series — there is aprobability of 0.625 that the series goes at least six games.One reason for the inclusion of this Topic is that Markov chains are oneof the most widely-used applications of matrix operations. Another reason isthat it provides an example of the use of matrices where we do not considerthe significance of the maps represented by the matrices. For more on Markovchains, there are many sources such as [Kemeny & Snell] and [Iosifescu].ExercisesUse a computer for these problems. You can, for instance, adapt the Octave scriptgiven below.1 These questions refer to the coin-flipping game.(a) Check the computations in the table at the end of the first paragraph.(b) Consider the second row of the vector table. Note that this row has alternating0’s. Must p 1 (j) be 0 when j is odd? Prove that it must be, or produce acounterexample.(c) Perform a computational experiment to estimate the chance that the playerends at five dollars, starting with one dollar, two dollars, and four dollars.2 We consider throws of a die, and say the system is in state s i if the largest numberyet appearing on the die was i.(a) Give the transition matrix.(b) Start the system in state s 1, and run it for five throws. What is the vectorat the end?