MST-018 Solved Assignment 2024

Next Lesson

Sample Solution

MST-018 Solved Assignment 2024 Sample Solution
Expert Answer
  1. State whether the following statements are true or false and also give the reason in support of your answer:
(a) The covariance matrix of random vectors X X XXX and Y Y YYY is symmetric.
Answer:
The statement is true. Let’s provide a justification for this:

Covariance Matrix Definition

The covariance matrix of two random vectors X X XXX and Y Y YYY is defined as:
Σ = [ Cov ( X , X ) Cov ( X , Y ) Cov ( Y , X ) Cov ( Y , Y ) ] Σ = Cov ( X , X ) Cov ( X , Y ) Cov ( Y , X ) Cov ( Y , Y ) Sigma=[[“Cov”(X”,”X),”Cov”(X”,”Y)],[“Cov”(Y”,”X),”Cov”(Y”,”Y)]]\Sigma = \begin{bmatrix} \text{Cov}(X, X) & \text{Cov}(X, Y) \\ \text{Cov}(Y, X) & \text{Cov}(Y, Y) \end{bmatrix}Σ=[Cov(X,X)Cov(X,Y)Cov(Y,X)Cov(Y,Y)]
Here, Cov ( X , X ) Cov ( X , X ) “Cov”(X,X)\text{Cov}(X, X)Cov(X,X) is the variance of X X XXX, Cov ( Y , Y ) Cov ( Y , Y ) “Cov”(Y,Y)\text{Cov}(Y, Y)Cov(Y,Y) is the variance of Y Y YYY, Cov ( X , Y ) Cov ( X , Y ) “Cov”(X,Y)\text{Cov}(X, Y)Cov(X,Y) is the covariance between X X XXX and Y Y YYY, and Cov ( Y , X ) Cov ( Y , X ) “Cov”(Y,X)\text{Cov}(Y, X)Cov(Y,X) is the covariance between Y Y YYY and X X XXX.

Symmetry of Covariance

To show that the covariance matrix is symmetric, we need to prove that Cov ( X , Y ) = Cov ( Y , X ) Cov ( X , Y ) = Cov ( Y , X ) “Cov”(X,Y)=”Cov”(Y,X)\text{Cov}(X, Y) = \text{Cov}(Y, X)Cov(X,Y)=Cov(Y,X).

Definition of Covariance

The covariance between two random variables X X XXX and Y Y YYY is defined as:
Cov ( X , Y ) = E [ ( X − E [ X ] ) ( Y − E [ Y ] ) ] Cov ( X , Y ) = E [ ( X − E [ X ] ) ( Y − E [ Y ] ) ] “Cov”(X,Y)=E[(X-E[X])(Y-E[Y])]\text{Cov}(X, Y) = \mathbb{E}[(X – \mathbb{E}[X])(Y – \mathbb{E}[Y])]Cov(X,Y)=E[(X−E[X])(Y−E[Y])]
Similarly, the covariance between Y Y YYY and X X XXX is:
Cov ( Y , X ) = E [ ( Y − E [ Y ] ) ( X − E [ X ] ) ] Cov ( Y , X ) = E [ ( Y − E [ Y ] ) ( X − E [ X ] ) ] “Cov”(Y,X)=E[(Y-E[Y])(X-E[X])]\text{Cov}(Y, X) = \mathbb{E}[(Y – \mathbb{E}[Y])(X – \mathbb{E}[X])]Cov(Y,X)=E[(Y−E[Y])(X−E[X])]

Proof of Symmetry

We can see that:
Cov ( X , Y ) = E [ ( X − E [ X ] ) ( Y − E [ Y ] ) ] Cov ( X , Y ) = E [ ( X − E [ X ] ) ( Y − E [ Y ] ) ] “Cov”(X,Y)=E[(X-E[X])(Y-E[Y])]\text{Cov}(X, Y) = \mathbb{E}[(X – \mathbb{E}[X])(Y – \mathbb{E}[Y])]Cov(X,Y)=E[(X−E[X])(Y−E[Y])]
Since multiplication is commutative, we have:
E [ ( X − E [ X ] ) ( Y − E [ Y ] ) ] = E [ ( Y − E [ Y ] ) ( X − E [ X ] ) ] E [ ( X − E [ X ] ) ( Y − E [ Y ] ) ] = E [ ( Y − E [ Y ] ) ( X − E [ X ] ) ] E[(X-E[X])(Y-E[Y])]=E[(Y-E[Y])(X-E[X])]\mathbb{E}[(X – \mathbb{E}[X])(Y – \mathbb{E}[Y])] = \mathbb{E}[(Y – \mathbb{E}[Y])(X – \mathbb{E}[X])]E[(X−E[X])(Y−E[Y])]=E[(Y−E[Y])(X−E[X])]
Thus:
Cov ( X , Y ) = Cov ( Y , X ) Cov ( X , Y ) = Cov ( Y , X ) “Cov”(X,Y)=”Cov”(Y,X)\text{Cov}(X, Y) = \text{Cov}(Y, X)Cov(X,Y)=Cov(Y,X)

Conclusion

Since Cov ( X , Y ) = Cov ( Y , X ) Cov ( X , Y ) = Cov ( Y , X ) “Cov”(X,Y)=”Cov”(Y,X)\text{Cov}(X, Y) = \text{Cov}(Y, X)Cov(X,Y)=Cov(Y,X), the off-diagonal elements of the covariance matrix are equal, making the covariance matrix symmetric.

Symmetric Covariance Matrix

Therefore, the covariance matrix Σ Σ Sigma\SigmaΣ is symmetric:
Σ = [ Cov ( X , X ) Cov ( X , Y ) Cov ( Y , X ) Cov ( Y , Y ) ] = [ Var ( X ) Cov ( X , Y ) Cov ( X , Y ) Var ( Y ) ] Σ = Cov ( X , X ) Cov ( X , Y ) Cov ( Y , X ) Cov ( Y , Y ) = Var ( X ) Cov ( X , Y ) Cov ( X , Y ) Var ( Y ) Sigma=[[“Cov”(X”,”X),”Cov”(X”,”Y)],[“Cov”(Y”,”X),”Cov”(Y”,”Y)]]=[[“Var”(X),”Cov”(X”,”Y)],[“Cov”(X”,”Y),”Var”(Y)]]\Sigma = \begin{bmatrix} \text{Cov}(X, X) & \text{Cov}(X, Y) \\ \text{Cov}(Y, X) & \text{Cov}(Y, Y) \end{bmatrix} = \begin{bmatrix} \text{Var}(X) & \text{Cov}(X, Y) \\ \text{Cov}(X, Y) & \text{Var}(Y) \end{bmatrix}Σ=[Cov(X,X)Cov(X,Y)Cov(Y,X)Cov(Y,Y)]=[Var(X)Cov(X,Y)Cov(X,Y)Var(Y)]

Conclusion

The statement "The covariance matrix of random vectors X X XXX and Y Y YYY is symmetric" is true, as demonstrated by the equality Cov ( X , Y ) = Cov ( Y , X ) Cov ( X , Y ) = Cov ( Y , X ) “Cov”(X,Y)=”Cov”(Y,X)\text{Cov}(X, Y) = \text{Cov}(Y, X)Cov(X,Y)=Cov(Y,X).
(b) If X X XXX is a p p ppp-variate normal random vector, then every linear combination c ′ X c ′ X c^(‘)Xc^{\prime} Xc′X, where c p × 1 c p × 1 c_(pxx1)\mathrm{c}_{\mathrm{p} \times 1}cp×1 is a scalar vector, is also p p p\mathrm{p}p-variate normal vector.
Answer:
The statement is false. Let’s analyze and justify this.

Understanding the Statement

The statement says:
"If X X XXX is a p p ppp-variate normal random vector, then every linear combination c ′ X c ′ X c^(‘)Xc^{\prime} Xc′X, where c c ccc is a p × 1 p × 1 p xx1p \times 1p×1 scalar vector, is also a p p ppp-variate normal vector."

Breaking Down the Components

  1. p p ppp-variate Normal Random Vector: X X XXX is a random vector with a multivariate normal distribution in p p ppp-dimensional space.
  2. Linear Combination c ′ X c ′ X c^(‘)Xc^{\prime}Xc′X: c ′ X c ′ X c^(‘)Xc^{\prime}Xc′X represents a linear combination of the elements of X X XXX, where c c ccc is a vector of coefficients.

Correct Interpretation

Let’s rephrase the correct version of the statement for clarity:
  • If X X XXX is a p p ppp-variate normal random vector, then every linear combination c ′ X c ′ X c^(‘)Xc^{\prime} Xc′X, where c c ccc is a p × 1 p × 1 p xx1p \times 1p×1 vector, is also normally distributed (not necessarily a p p ppp-variate normal vector, but a univariate normal distribution).

Proof of Correct Interpretation

If X X XXX is a p p ppp-variate normal vector, then:
X ∼ N p ( μ , Σ ) X ∼ N p ( μ , Σ ) X∼N_(p)(mu,Sigma)X \sim N_p(\mu, \Sigma)X∼Np(μ,Σ)
where μ μ mu\muμ is the mean vector and Σ Σ Sigma\SigmaΣ is the covariance matrix.
Now consider a linear combination c ′ X c ′ X c^(‘)Xc^{\prime}Xc′X:
Y = c ′ X Y = c ′ X Y=c^(‘)XY = c^{\prime}XY=c′X
We need to show that Y Y YYY is normally distributed.

Mean of Y Y YYY

The mean of Y Y YYY is given by:
E [ Y ] = E [ c ′ X ] = c ′ E [ X ] = c ′ μ E [ Y ] = E [ c ′ X ] = c ′ E [ X ] = c ′ μ E[Y]=E[c^(‘)X]=c^(‘)E[X]=c^(‘)mu\mathbb{E}[Y] = \mathbb{E}[c^{\prime}X] = c^{\prime}\mathbb{E}[X] = c^{\prime}\muE[Y]=E[c′X]=c′E[X]=c′μ

Variance of Y Y YYY

The variance of Y Y YYY is given by:
Var ( Y ) = Var ( c ′ X ) = c ′ Σ c Var ( Y ) = Var ( c ′ X ) = c ′ Σ c “Var”(Y)=”Var”(c^(‘)X)=c^(‘)Sigma c\text{Var}(Y) = \text{Var}(c^{\prime}X) = c^{\prime}\Sigma cVar(Y)=Var(c′X)=c′Σc

Distribution of Y Y YYY

Since X X XXX is multivariate normal, any linear combination of its components is also normally distributed. Therefore, Y = c ′ X Y = c ′ X Y=c^(‘)XY = c^{\prime}XY=c′X is normally distributed with mean c ′ μ c ′ μ c^(‘)muc^{\prime}\muc′μ and variance c ′ Σ c c ′ Σ c c^(‘)Sigma cc^{\prime}\Sigma cc′Σc:
Y ∼ N ( c ′ μ , c ′ Σ c ) Y ∼ N ( c ′ μ , c ′ Σ c ) Y∼N(c^(‘)mu,c^(‘)Sigma c)Y \sim N(c^{\prime}\mu, c^{\prime}\Sigma c)Y∼N(c′μ,c′Σc)

Conclusion

While Y = c ′ X Y = c ′ X Y=c^(‘)XY = c^{\prime}XY=c′X is normally distributed, it is not a p p ppp-variate normal vector. It is a univariate normal random variable. Hence, the original statement is false.

Correct Statement

The correct statement should be:
  • "If X X XXX is a p p ppp-variate normal random vector, then every linear combination c ′ X c ′ X c^(‘)Xc^{\prime} Xc′X, where c c ccc is a p × 1 p × 1 p xx1p \times 1p×1 vector, is also normally distributed (a univariate normal random variable)."
This highlights the distinction between being normally distributed in a univariate sense and being a p p ppp-variate normal vector.
(c) The trace of matrix ( 3 − 2 − 2 6 ) 3 − 2 − 2 6 ([3,-2],[-2,6])\left(\begin{array}{cc}3 & -2 \\ -2 & 6\end{array}\right)(3−2−26) is 9 .
Answer:
The trace of a matrix is defined as the sum of the elements on the main diagonal (the diagonal that runs from the top left to the bottom right) of the matrix.
Given the matrix:
A = ( 3 − 2 − 2 6 ) A = 3 − 2 − 2 6 A=([3,-2],[-2,6])A = \begin{pmatrix} 3 & -2 \\ -2 & 6 \end{pmatrix}A=(3−2−26)
We need to find the trace of this matrix.

Definition of Trace

The trace of a 2 × 2 2 × 2 2xx22 \times 22×2 matrix A = ( a b c d ) A = a b c d A=([a,b],[c,d])A = \begin{pmatrix} a & b \\ c & d \end{pmatrix}A=(abcd) is given by:
tr ( A ) = a + d tr ( A ) = a + d “tr”(A)=a+d\text{tr}(A) = a + dtr(A)=a+d

Applying the Definition

For the given matrix A A AAA:
A = ( 3 − 2 − 2 6 ) A = 3 − 2 − 2 6 A=([3,-2],[-2,6])A = \begin{pmatrix} 3 & -2 \\ -2 & 6 \end{pmatrix}A=(3−2−26)
The elements on the main diagonal are 3 and 6. Therefore, the trace of A A AAA is:
tr ( A ) = 3 + 6 = 9 tr ( A ) = 3 + 6 = 9 “tr”(A)=3+6=9\text{tr}(A) = 3 + 6 = 9tr(A)=3+6=9

Conclusion

The statement "The trace of the matrix ( 3 − 2 − 2 6 ) 3 − 2 − 2 6 ([3,-2],[-2,6])\begin{pmatrix} 3 & -2 \\ -2 & 6 \end{pmatrix}(3−2−26) is 9" is true.

Justification

The trace of the matrix was calculated correctly by summing the diagonal elements:
3 + 6 = 9 3 + 6 = 9 3+6=93 + 6 = 93+6=9
Therefore, the statement is justified as true.
(d) If a matrix is positive definite then its inverse is also positive definite.
Answer:
The statement "If a matrix is positive definite then its inverse is also positive definite" is true. Here’s a detailed justification:

Positive Definite Matrix

A matrix A A AAA is positive definite if for any non-zero vector x x xxx,
x T A x > 0. x T A x > 0. x^(T)Ax > 0.x^T A x > 0.xTAx>0.

Inverse of a Positive Definite Matrix

To show that the inverse A − 1 A − 1 A^(-1)A^{-1}A−1 of a positive definite matrix A A AAA is also positive definite, we need to show that for any non-zero vector y y yyy,
y T A − 1 y > 0. y T A − 1 y > 0. y^(T)A^(-1)y > 0.y^T A^{-1} y > 0.yTA−1y>0.

Proof

Let A A AAA be a positive definite matrix, and let x = A − 1 y x = A − 1 y x=A^(-1)yx = A^{-1} yx=A−1y. Since A A AAA is positive definite, we have:
x T A x > 0. x T A x > 0. x^(T)Ax > 0.x^T A x > 0.xTAx>0.
Using the substitution x = A − 1 y x = A − 1 y x=A^(-1)yx = A^{-1} yx=A−1y, we get:
( A − 1 y ) T A ( A − 1 y ) > 0. ( A − 1 y ) T A ( A − 1 y ) > 0. (A^(-1)y)^(T)A(A^(-1)y) > 0.(A^{-1} y)^T A (A^{-1} y) > 0.(A−1y)TA(A−1y)>0.
Simplifying the left-hand side:
y T ( A − 1 ) T A A − 1 y . y T ( A − 1 ) T A A − 1 y . y^(T)(A^(-1))^(T)AA^(-1)y.y^T (A^{-1})^T A A^{-1} y.yT(A−1)TAA−1y.
Since A A AAA is symmetric (as all positive definite matrices are), ( A − 1 ) T = A − 1 ( A − 1 ) T = A − 1 (A^(-1))^(T)=A^(-1)(A^{-1})^T = A^{-1}(A−1)T=A−1. So the expression becomes:
y T A − 1 y > 0. y T A − 1 y > 0. y^(T)A^(-1)y > 0.y^T A^{-1} y > 0.yTA−1y>0.
This shows that for any non-zero vector y y yyy, y T A − 1 y > 0 y T A − 1 y > 0 y^(T)A^(-1)y > 0y^T A^{-1} y > 0yTA−1y>0, implying that A − 1 A − 1 A^(-1)A^{-1}A−1 is positive definite.

Conclusion

The statement "If a matrix is positive definite then its inverse is also positive definite" is true. The proof is based on the property that if A A AAA is positive definite, then x T A x > 0 x T A x > 0 x^(T)Ax > 0x^T A x > 0xTAx>0 for any non-zero vector x x xxx, and by substituting x = A − 1 y x = A − 1 y x=A^(-1)yx = A^{-1} yx=A−1y, we showed that y T A − 1 y > 0 y T A − 1 y > 0 y^(T)A^(-1)y > 0y^T A^{-1} y > 0yTA−1y>0 for any non-zero vector y y yyy, thereby proving that A − 1 A − 1 A^(-1)A^{-1}A−1 is also positive definite.
(e) If X ∼ N 2 ( ( 2 1 ) , ( 1 0 0 1 ) ) X ∼ N 2 2 1 , 1      0 0      1 X∼N_(2)(([2],[1]),([1,0],[0,1]))X \sim N_2\left(\left(\begin{array}{l}2 \\ 1\end{array}\right),\left(\begin{array}{ll}1 & 0 \\ 0 & 1\end{array}\right)\right)X∼N2((21),(1001)) and Y ∼ N 2 ( ( − 1 3 ) , ( 1 0 0 1 ) ) Y ∼ N 2 − 1 3 , 1      0 0      1 Y∼N_(2)(([-1],[3]),([1,0],[0,1]))Y \sim N_2\left(\left(\begin{array}{c}-1 \\ 3\end{array}\right),\left(\begin{array}{ll}1 & 0 \\ 0 & 1\end{array}\right)\right)Y∼N2((−13),(1001)), then
X _ + Y _ ∼ N 2 ( ( 1 1 ) , ( 1 0 0 1 ) ) . X _ + Y _ ∼ N 2 1 1 , 1      0 0      1 . X_+Y_∼N_(2)(([1],[1]),([1,0],[0,1])).\underline{X}+\underline{Y} \sim \mathrm{N}_2\left(\left(\begin{array}{l} 1 \\ 1 \end{array}\right),\left(\begin{array}{ll} 1 & 0 \\ 0 & 1 \end{array}\right)\right) .X_+Y_∼N2((11),(1001)).
Answer:
The statement is false. Let’s justify this by carefully analyzing the properties of the sum of two independent multivariate normal random vectors.

Definitions and Properties

Let X ∼ N 2 ( ( 2 1 ) , ( 1 0 0 1 ) ) X ∼ N 2 2 1 , 1      0 0      1 X∼N_(2)(([2],[1]),([1,0],[0,1]))X \sim N_2\left(\left(\begin{array}{l}2 \\ 1\end{array}\right),\left(\begin{array}{ll}1 & 0 \\ 0 & 1\end{array}\right)\right)X∼N2((21),(1001)) and Y ∼ N 2 ( ( − 1 3 ) , ( 1 0 0 1 ) ) Y ∼ N 2 − 1 3 , 1      0 0      1 Y∼N_(2)(([-1],[3]),([1,0],[0,1]))Y \sim N_2\left(\left(\begin{array}{c}-1 \\ 3\end{array}\right),\left(\begin{array}{ll}1 & 0 \\ 0 & 1\end{array}\right)\right)Y∼N2((−13),(1001)).
  • X X XXX is a bivariate normal random vector with mean vector μ X = ( 2 1 ) μ X = 2 1 mu _(X)=([2],[1])\mu_X = \left(\begin{array}{c}2 \\ 1\end{array}\right)μX=(21) and covariance matrix Σ X = ( 1 0 0 1 ) Σ X = 1 0 0 1 Sigma _(X)=([1,0],[0,1])\Sigma_X = \left(\begin{array}{cc}1 & 0 \\ 0 & 1\end{array}\right)ΣX=(1001).
  • Y Y YYY is a bivariate normal random vector with mean vector μ Y = ( − 1 3 ) μ Y = − 1 3 mu _(Y)=([-1],[3])\mu_Y = \left(\begin{array}{c}-1 \\ 3\end{array}\right)μY=(−13) and covariance matrix Σ Y = ( 1 0 0 1 ) Σ Y = 1 0 0 1 Sigma _(Y)=([1,0],[0,1])\Sigma_Y = \left(\begin{array}{cc}1 & 0 \\ 0 & 1\end{array}\right)ΣY=(1001).

Sum of Two Independent Multivariate Normals

If X X XXX and Y Y YYY are independent, then the sum Z = X + Y Z = X + Y Z=X+YZ = X + YZ=X+Y is also a bivariate normal random vector. The mean vector and covariance matrix of Z Z ZZZ can be determined as follows:

Mean Vector of Z Z ZZZ

The mean vector of Z = X + Y Z = X + Y Z=X+YZ = X + YZ=X+Y is the sum of the mean vectors of X X XXX and Y Y YYY:
μ Z = μ X + μ Y = ( 2 1 ) + ( − 1 3 ) = ( 2 + ( − 1 ) 1 + 3 ) = ( 1 4 ) μ Z = μ X + μ Y = 2 1 + − 1 3 = 2 + ( − 1 ) 1 + 3 = 1 4 mu _(Z)=mu _(X)+mu _(Y)=([2],[1])+([-1],[3])=([2+(-1)],[1+3])=([1],[4])\mu_Z = \mu_X + \mu_Y = \left(\begin{array}{c}2 \\ 1\end{array}\right) + \left(\begin{array}{c}-1 \\ 3\end{array}\right) = \left(\begin{array}{c}2 + (-1) \\ 1 + 3\end{array}\right) = \left(\begin{array}{c}1 \\ 4\end{array}\right)μZ=μX+μY=(21)+(−13)=(2+(−1)1+3)=(14)

Covariance Matrix of Z Z ZZZ

The covariance matrix of Z = X + Y Z = X + Y Z=X+YZ = X + YZ=X+Y is the sum of the covariance matrices of X X XXX and Y Y YYY:
Σ Z = Σ X + Σ Y = ( 1 0 0 1 ) + ( 1 0 0 1 ) = ( 1 + 1 0 + 0 0 + 0 1 + 1 ) = ( 2 0 0 2 ) Σ Z = Σ X + Σ Y = 1 0 0 1 + 1 0 0 1 = 1 + 1 0 + 0 0 + 0 1 + 1 = 2 0 0 2 Sigma _(Z)=Sigma _(X)+Sigma _(Y)=([1,0],[0,1])+([1,0],[0,1])=([1+1,0+0],[0+0,1+1])=([2,0],[0,2])\Sigma_Z = \Sigma_X + \Sigma_Y = \left(\begin{array}{cc}1 & 0 \\ 0 & 1\end{array}\right) + \left(\begin{array}{cc}1 & 0 \\ 0 & 1\end{array}\right) = \left(\begin{array}{cc}1 + 1 & 0 + 0 \\ 0 + 0 & 1 + 1\end{array}\right) = \left(\begin{array}{cc}2 & 0 \\ 0 & 2\end{array}\right)ΣZ=ΣX+ΣY=(1001)+(1001)=(1+10+00+01+1)=(2002)

Conclusion

The correct distribution for X + Y X + Y X+YX + YX+Y is:
X + Y ∼ N 2 ( ( 1 4 ) , ( 2 0 0 2 ) ) X + Y ∼ N 2 1 4 , 2 0 0 2 X+Y∼N_(2)(([1],[4]),([2,0],[0,2]))X + Y \sim N_2\left(\left(\begin{array}{c}1 \\ 4\end{array}\right), \left(\begin{array}{cc}2 & 0 \\ 0 & 2\end{array}\right)\right)X+Y∼N2((14),(2002))
Therefore, the statement:
X _ + Y _ ∼ N 2 ( ( 1 1 ) , ( 1 0 0 1 ) ) X _ + Y _ ∼ N 2 1 1 , 1      0 0      1 X_+Y_∼N_(2)(([1],[1]),([1,0],[0,1]))\underline{X} + \underline{Y} \sim N_2\left(\left(\begin{array}{l} 1 \\ 1 \end{array}\right),\left(\begin{array}{ll} 1 & 0 \\ 0 & 1 \end{array}\right)\right)X_+Y_∼N2((11),(1001))
is false because the correct mean vector is ( 1 4 ) 1 4 ([1],[4])\left(\begin{array}{c}1 \\ 4\end{array}\right)(14) and the correct covariance matrix is ( 2 0 0 2 ) 2 0 0 2 ([2,0],[0,2])\left(\begin{array}{cc}2 & 0 \\ 0 & 2\end{array}\right)(2002).
Verified Answer
5/5
Back to Course
Next Lesson
Scroll to Top
Scroll to Top