Variance and Covariance
Variance means spread of a distribution
Let X be a random variable with mean μ. The variance of X – denoted by σ2 or σ2X or V(X) or VX is defined by
assuming this expectation exists. The standard deviation is sd(X) = √V(X) and is also denoted by σ and σX.
Variance has the following properties.
1. V(X) = E(X2) – μ2
2. If a and b are constants then V(aX+b) = a2V(X)
3. If X1, . . . , Xn are independent and a1, . . . , an are constants, then
Example: Let X ~ Binomial(n, p). We write X = ∑iXi where Xi = 1 if toss i is head and Xi = 0 otherwise. Here, P(Xi = 1) = p and P(Xi = 0) = 1 – p.
Using the formula of expectation,
E(Xi) = [p x 1] + [(1 – p) x 0] = p
E(Xi2) = [p x 12] + [(1 – p) x 02] = p
Now, V(Xi) = E(Xi2) – p2 = p – p2 = p(1 – p).
Finally, V(X) = V(∑iXi) = ∑iV(Xi) = ∑ip(1 – p) = np(1 – p).
If X1, . . . , Xn are random variables then we define the sample mean to be
and the sample variance to be
Let X and Y be random variables with mean μX and μY and standard deviation σX and σY. Define the covariance between X and Y by
Cov(X, Y) = E[(X – μX) (Y – μY)]
and the correlation by
Covariance satisfies the following property
Cov(X, Y) = E(XY) – E(X)E(Y)
The conditional expectation of X given Y = y is
If r(x, y) is a function of x and y then
Conditional expectation corresponds to the mean X among those times in which Y = y.