Sample Solution

MMT-008 Solved Assignment 2024 Sample Solution

mmt-008-solved-assignment-2024-ss-8e24e610-06c9-4b43-84f6-a5bf6ef5ab5c

MMT-008 Solved Assignment 2024

a) Consider the Markov chain having the following transition probability matrix.

p = [\begin{array}{cccccc} 1 & 2 & 3 & 4 & 5 & 6 \\ \frac{1}{3} & \frac{2}{3} & 0 & 0 & 0 & 0 \\ \frac{2}{3} & \frac{1}{3} & 0 & 0 & 0 & 0 \\ \frac{1}{4} & 0 & \frac{1}{4} & 0 & \frac{1}{4} & \frac{1}{4} \\ \frac{1}{6} & \frac{1}{6} & \frac{1}{6} & \frac{1}{6} & \frac{1}{6} & \frac{1}{6} \\ 0 & 0 & \frac{1}{4} & \frac{3}{4} & 0 & 0 \\ 0 & 0 & \frac{1}{5} & \frac{4}{5} & 0 & 0 \end{array}]

i) Draw the diagram of a Markov chain.
ii) Classify the states of a Markov chain, i.e., persistent, transient, non-null and a periodic state. Also check the irreducibility of Markov chain.
iii) Find the closed sets.
iv) Find the probability of absorption to the closed classes. Also find the mean time up to absorption from transient state 3 to 4 .

Answer:

i) Diagram of the Markov Chain:

To draw the diagram of the Markov chain, we represent each state as a node and draw directed edges between nodes with the corresponding transition probabilities as labels. Here’s the diagram:

Here’s the diagram of the Markov chain with states labeled as 1 to 6:

        +--------+     +--------+     +--------+     +--------+     +--------+     +--------+
        |  1     | ----> |  2     | ----> |  3     | ----> |  4     | ----> |  5     | ----> |  6     |
        +--+-----+     +--+-----+     +--+-----+     +--+-----+     +--+-----+     +--+-----+
           |       \     |       \     |       \     |       \     |       \     |       \
           | 1/3     | ----> | 2/3     | ----> (0)     | ----> (0)     | ----> (0)     | ----> (0)
           |       /     |       /     |       /     |       /     |       /     |       /
        +--------+     +--+-----+     +--+-----+     +--+-----+     +--+-----+     +--+-----+
           |               |       \     |       \     |       \     |       \     |       \
           |       \        | 1/4     | ----> | 1/4     | ----> | 1/4     | ----> | 1/4     |
           |        -----> |       /     |       /     |       /     |       /     |       /
        +--------+     +--+-----+     +--+-----+     +--+-----+     +--+-----+     +--+-----+
                                                |                |               |
                 (1/6)                          | (1/6)          | (1/6)          | (1/6)

           +--------+     +--------+
           |  6     | ---- |  4     |
           +--+-----+     +--+-----+
              |               |
              | (3/4)         |
              |               |
           +--------+     +--------+

Explanation:

Each circle represents a state (1 to 6).
Arrows show the possible transitions between states.
The number on each arrow represents the probability of that transition.
States 3, 5, and 6 are absorbing states as they have no outgoing arrows.
States 1, 2, and 4 are transient states as they can leave and enter again.

ii) Classification of the States of the Markov Chain:

Persistent States: These are states that, once entered, the process has a nonzero probability of staying in forever. In this chain, there are no persistent states.
Transient States: These are states that, once left, the process has a zero probability of returning to. In this chain, states 1, 2, 3, and 4 are transient states.
Null States: These are states that do not lead to any other state. In this chain, there are no null states.
Periodic States: A state is periodic if the process can return to it only at multiples of some integer greater than 1. In this chain, there are no periodic states, as all states can potentially be returned to at any time.
Irreducibility: A Markov chain is irreducible if it is possible to get from any state to any other state. This chain is not irreducible because there are states that cannot be reached from other states (e.g., there is no path from state 1 to state 6).

iii) Closed Sets:

A closed set in a Markov chain is a set of states such that once the process enters any state in the set, it cannot leave the set. In this chain, the closed sets are:

{5, 6}

iv) Probability of Absorption to the Closed Classes:

The probability of absorption to a closed class is the probability that the process will eventually end up in that class starting from a transient state. To find these probabilities, we usually set up and solve a system of linear equations based on the transition probabilities. However, in this case, it’s straightforward to see:

From state 3, the probability of absorption to the closed class {5, 6} is 1, as there is a direct transition to state 5 with probability 1/6.
From state 4, the probability of absorption to the closed class {5, 6} is also 1, as there is a direct transition to state 5 with probability 1/4.

Mean Time Up to Absorption from Transient State 3 to 4:

The mean time up to absorption is the expected number of steps it takes to reach an absorbing state from a given transient state. To find this, we can use the fundamental matrix method, which involves inverting a matrix derived from the transition probabilities. However, in this simple case, we can observe:

From state 3, the process moves to state 5 with probability 1/6 in one step, so the mean time to absorption is 1.
From state 4, the process moves to state 5 with probability 1/4 in one step, so the mean time to absorption is also 1.

In conclusion, the Markov chain has transient states 1, 2, 3, and 4, with closed set {5, 6}. The probability of absorption to the closed class {5, 6} is 1 for states 3 and 4, and the mean time to absorption from states 3 and 4 is 1 step.

b) Determine the parameters of the bivariate normal distribution:

f (x, y) = k \exp [- \frac{8}{27} {(x - 7)^{2} - 2 (x - 7) (y + 5) + 4 (y + 5)^{2}}]

Also find the value of

k

Answer:

The given bivariate normal distribution function is:

f (x, y) = k \exp [- \frac{8}{27} {(x - 7)^{2} - 2 (x - 7) (y + 5) + 4 (y + 5)^{2}}]

This can be written in the standard form of a bivariate normal distribution as:

f (x, y) = \frac{1}{2 π σ_{x} σ_{y} \sqrt{1 - ρ^{2}}} \exp [- \frac{1}{2 (1 - ρ^{2})} (\frac{(x - μ_{x})^{2}}{σ_{x}^{2}} - 2 ρ \frac{(x - μ_{x}) (y - μ_{y})}{σ_{x} σ_{y}} + \frac{(y - μ_{y})^{2}}{σ_{y}^{2}})]

By comparing the given function with the standard form, we can identify the parameters of the bivariate normal distribution:

Mean of $x$ ( $μ_{x}$ ): 7
Mean of $y$ ( $μ_{y}$ ): -5
Variance of $x$ ( $σ_{x}^{2}$ ): The coefficient of $(x - 7)^{2}$ is $\frac{8}{27}$ , so $σ_{x}^{2} = \frac{27}{2 \times \frac{8}{27}} = \frac{27}{16}$ , and $σ_{x} = \sqrt{\frac{27}{16}} = \frac{3 \sqrt{3}}{4}$ .
Variance of $y$ ( $σ_{y}^{2}$ ): The coefficient of $(y + 5)^{2}$ is $\frac{32}{27}$ , so $σ_{y}^{2} = \frac{27}{2 \times \frac{32}{27}} = \frac{27}{64}$ , and $σ_{y} = \sqrt{\frac{27}{64}} = \frac{3 \sqrt{3}}{8}$ .
Correlation coefficient ( $ρ$ ): The coefficient of $(x - 7) (y + 5)$ is $- \frac{16}{27}$ , so $ρ = \frac{- \frac{16}{27}}{2 \times \frac{3 \sqrt{3}}{4} \times \frac{3 \sqrt{3}}{8}} = - \frac{1}{2}$ .

To find the value of

k

, we use the fact that the integral of the bivariate normal distribution function over the entire plane is 1:

\iint_{- \infty}^{\infty} f (x, y) d x d y = 1

k \iint_{- \infty}^{\infty} \exp [- \frac{8}{27} {(x - 7)^{2} - 2 (x - 7) (y + 5) + 4 (y + 5)^{2}}] d x d y = 1

Since the integral of the exponential part is the same as the integral of the standard bivariate normal distribution, which is

2 π σ_{x} σ_{y} \sqrt{1 - ρ^{2}}

, we have:

k \times 2 π \times \frac{3 \sqrt{3}}{4} \times \frac{3 \sqrt{3}}{8} \times \sqrt{1 - {(- \frac{1}{2})}^{2}} = 1

k \times 2 π \times \frac{27}{32} \times \sqrt{\frac{3}{4}} = 1

k = \frac{32}{27 π \sqrt{3}}

So, the parameters of the bivariate normal distribution are:

$μ_{x} = 7$
$μ_{y} = - 5$
$σ_{x} = \frac{3 \sqrt{3}}{4}$
$σ_{y} = \frac{3 \sqrt{3}}{8}$
$ρ = - \frac{1}{2}$
$k = \frac{32}{27 π \sqrt{3}}$

Verified Answer

 5/5

Back to Course

Next Lesson