Once we formulate the markov model correctly, we can obtain the generating function for each entry in the matrix, where there's a possibility of getting the explicit formula. Let's take a look at one such problem:

A six faced unbiased die is rolled :math:`n` times. What is the probability that we get to see all the six numbers in the sequence?

Setting up a markov chain is easy:

\begin{equation*} \displaystyle A= \begin{pmatrix} 0 & 1 & 0 & 0 & 0 & 0 & 0\\ 0 & \frac{1}{6} & \frac{5}{6} & 0 & 0 & 0 & 0\\ 0 & 0 & \frac{1}{3} & \frac{2}{3} & 0 & 0 & 0\\ 0 & 0 & 0 & \frac{1}{2} & \frac{1}{2} & 0 & 0\\ 0 & 0 & 0 & 0 & \frac{2}{3} & \frac{1}{3} & 0\\ 0 & 0 & 0 & 0 & 0 & \frac{5}{6} & \frac{1}{6}\\ 0 & 0 & 0 & 0 & 0 & 0 & 1 \end{pmatrix} \end{equation*}

The states indicate the number of faces shown up. E.g. the row above the last row indicates that when 5 faces are seen, there's a probability of \(5/6\) remaining in the same state and \(1/6\) moving to the final state.

So, \(A^n[0,6]\), gives the required answer. But we can also find the generating function for that entry by computing \((I-x\, A)^{-1}\), which is

\begin{equation*} \displaystyle G(x) = \frac{10 \, x^{6}}{{\left(5 \, x - 6\right)} {\left(2 \, x - 3\right)} {\left(x - 1\right)} {\left(x - 2\right)} {\left(x - 3\right)} {\left(x - 6\right)}} \end{equation*}

and on partial fractions it's

\begin{equation*} \displaystyle G(x) = \frac{36}{5 \, x - 6} - \frac{45}{2 \, x - 3} - \frac{1}{x - 1} + \frac{40}{x - 2} - \frac{45}{x - 3} + \frac{36}{x - 6} + 1 \end{equation*}

and the probability can be written by extracting \([x^n]G(x)\) as

\begin{equation*} \displaystyle \mathbb{P}(n) = 1-\frac{6}{6^n}+\frac{15}{3^n}-\frac{20}{2^n}+15\left(\frac{2}{3}\right)^n-6\left(\frac{5}{6}\right)^n \end{equation*}

which can be verified by a simulation in J:

1
2
3

n=:10
sim=: 3 : '6=+/~:?n#6'
(+/%#)(sim"0)1e5#0 NB. about 0.27

and

\begin{equation*} \mathbb{P}(10) = \dfrac{38045}{139968} \approx 0.271812128486511 \end{equation*}