DiscreteMarkovChains

DiscreteMarkovChains is a package that supports various functions relating to discrete Markov chains. In particular, it deals with discrete-time discrete-space time-homogenous finite Markov chains.

This library also deals with continuous Markov chains. Any function in the documentation that takes "some kind of Markov chain" as an argument can be a DiscreteMarkovChain or a ContinuousMarkovChain. Sadly there are very few examples for continuous Markov chains but they operate in the same way as discrete Markov chains.

Installation

DiscreteMarkovChains should be up on Julia's package registry.

Simply type ] add DiscreteMarkovChains into the Julia REPL.

Documentation

See the documentation hosted on GitHub Pages.

Usage

Discrete Time

We want to find out if this chain is an absorbing chain.

using DiscreteMarkovChains

transition_matrix = [
    0.0 1.0 0.0;
    0.5 0.0 0.5;
    0.0 1.0 0.0;
]
chain = DiscreteMarkovChain(transition_matrix)
is_absorbing(chain)

# output

false

Let's try find the communication classes, see if they are recurrent and what their periodicity is.

periodicities(chain)

# output

([[1, 2, 3]], Any[true], Any[2])

This means that we have one communication class with 3 recurrent states. Their periodicity is 2.

Since we have a single communication class, we can calculate the mean recurrence times.

mean_recurrence_time(chain)

# output

3-element Array{Float64,1}:
 4.0
 2.0
 4.0

So the first and third states take an average of 4 time steps to return to itself. The second state takes an average of 2 steps to return to itself.

Continuous Time

There is support for continuous Markov chains as well.

generator = [
    -3 1 2;
    0 -1 1;
    1 1 -2;
]
chain = ContinuousMarkovChain(generator)

communication_classes(chain)

# output

([[1, 2, 3]], Any[true])

So we have one communication class that is recurrent.

Calculate the stationary distribution of the chain.

stationary_distribution(chain)

# output

3-element Array{Float64,1}:
 0.125
 0.5
 0.375

Calculate the mean first passage time of the chain.

round.(mean_first_passage_time(chain), digits=2)

# output

3×3 Array{Float64,2}:
 0.0  1.0  0.67
 3.0  0.0  1.0
 2.0  1.0  0.0

Authors

  • Chris du Plessis

License

MIT