Chapter 10. Markov chains.
                                  Manual for SOA Exam MLC.
                                          Chapter 10. Markov chains.
                                         Section 10.2. Markov chains.
                           c
                          
2008. Miguel A. Arcones. All rights reserved.
                                                     Extract from:
                 ”Arcones’ Manual for SOA Exam MLC. Fall 2009 Edition”,
                            available at http://www.actexmadriver.com/
                                                                                                                          1/110
               c
               
2008. Miguel A. Arcones. All rights reserved.      Manual for SOA Exam MLC.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     Since X takes values in the countable set E, X has a discrete
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 n                                                              n
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     distribution.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     The set E in the previous definition is called the state space.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     Usually, E = {0,1,2,...} or E = {1,2,...,m}. We will assume
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     that E = {0,1,2,...}. Each element of E is called a state. If
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     X =k, where k ∈ E, we say that the Markov chain {X }∞                                             is at
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        n                                                                                    n n=0
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     state k at stage n.
                                 Chapter 10. Markov chains.      Section 10.2. Markov chains.
   Markov chains
          Definition 1
          Adiscrete time Markov chain {X : n = 0,1,2,...} is a
                                                                  n
          stochastic process with values in the countable space E such that
          for each i ,i ,...,i ,j ∈ E,
                         0 1            n
                    P{X           =j|X =i ,X =i ,...,X                            =i        , X = i } (1)
                           n+1            0       0     1      1           n−1         n−1      n       n
              =                                P{X          =j|X =i }.
                                                      n+1            n      n
                                                                                                                       2/110
               c
              
2008. Miguel A. Arcones. All rights reserved.     Manual for SOA Exam MLC.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     The set E in the previous definition is called the state space.
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     Usually, E = {0,1,2,...} or E = {1,2,...,m}. We will assume
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     that E = {0,1,2,...}. Each element of E is called a state. If
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     X =k, where k ∈ E, we say that the Markov chain {X }∞                                             is at
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        n                                                                                    n n=0
                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     state k at stage n.
                                 Chapter 10. Markov chains.      Section 10.2. Markov chains.
   Markov chains
          Definition 1
          Adiscrete time Markov chain {X : n = 0,1,2,...} is a
                                                                  n
          stochastic process with values in the countable space E such that
          for each i ,i ,...,i ,j ∈ E,
                         0 1            n
                    P{X           =j|X =i ,X =i ,...,X                            =i        , X = i } (1)
                           n+1            0       0     1      1           n−1         n−1      n       n
              =                                P{X          =j|X =i }.
                                                      n+1            n      n
          Since X takes values in the countable set E, X has a discrete
                      n                                                             n
          distribution.
                                                                                                                       3/110
               c
              
2008. Miguel A. Arcones. All rights reserved.     Manual for SOA Exam MLC.
                                 Chapter 10. Markov chains.      Section 10.2. Markov chains.
   Markov chains
          Definition 1
          Adiscrete time Markov chain {X : n = 0,1,2,...} is a
                                                                  n
          stochastic process with values in the countable space E such that
          for each i ,i ,...,i ,j ∈ E,
                         0 1            n
                    P{X           =j|X =i ,X =i ,...,X                            =i        , X = i } (1)
                           n+1            0       0     1      1           n−1         n−1      n       n
              =                                P{X          =j|X =i }.
                                                      n+1            n      n
          Since X takes values in the countable set E, X has a discrete
                      n                                                             n
          distribution.
          The set E in the previous definition is called the state space.
          Usually, E = {0,1,2,...} or E = {1,2,...,m}. We will assume
          that E = {0,1,2,...}. Each element of E is called a state. If
                                                                                                     ∞
          X =k, where k ∈ E, we say that the Markov chain {X }                                              is at
             n                                                                                   n n=0
          state k at stage n.
                                                                                                                       4/110
               c
              
2008. Miguel A. Arcones. All rights reserved.     Manual for SOA Exam MLC.