Variance and Covariance
Variance means spread of a distribution
Let X be a random variable with mean μ. The variance of X – denoted by σ2 or σ2X or V(X) or VX is defined by
assuming this expectation exists. The standard deviation is sd(X) = √V(X) and is also denoted by σ and σX.
Variance has the following properties.
1. V(X) = E(X2) – μ2
2. If a and b are constants then V(aX+b) = a2V(X)
3. If X1, . . . , Xn are independent and a1, . . . , an are constants, then
Example: Let X ~ Binomial(n, p). We write X = ∑iXi where Xi = 1 if toss i is head and Xi = 0 otherwise. Here, P(Xi = 1) = p and P(Xi = 0) = 1 – p.
Using the formula of expectation,
E(Xi) = [p x 1] + [(1 – p) x 0] = p
E(Xi2) = [p x 12] + [(1 – p) x 02] = p
Now, V(Xi) = E(Xi2) – p2 = p – p2 = p(1 – p).
Finally, V(X) = V(∑iXi) = ∑iV(Xi) = ∑ip(1 – p) = np(1 – p).
Sample mean & Sample variance
If X1, . . . , Xn are random variables then we define the sample mean to be
and the sample variance to be
Note the denominator in sample variance as n-1 instead of n. This is called Bessel’s correction. The mean of the sample is closer to the sample data points than the actual mean of the population. Thus the numerator term in sample variance is smaller than expected. To balance this, the denominator is reduced by 1, known as Bessel’s correction.
Sample mean = (X1 + X2 + . . . + Xn) / n
Given a sample of size n, consider n independent random variables X1, X2, …, Xn, each corresponding to one randomly selected observation. Each of these variables has the distribution of the population, with mean µ and standard deviation σ.
Numpy Variance: 909.9216 Calculated variance 1: 909.9216 Calculated variance 2: 909.9216
Let X and Y be random variables with mean μX and μY and standard deviation σX and σY. Define the covariance between X and Y by
Cov(X, Y) = E[(X – μX) (Y – μY)]
and the correlation by
Covariance satisfies the following property
Cov(X, Y) = E(XY) – E(X)E(Y)
Let X and Y have joint probability mass function as follows:
Lets calculate µX and µY.
µX = ∑xfX(x) = (1 x 1/2) + (2 x 1/2) = 3/2
µY = ∑yfY(y) = (1 x 1/4) + (2 x 1/2) + (3 x 1/4) = 2
Lets calculate σX and σY.
σX = √∑(x – µX)2fX(x) = √( (1 – 3/2)21/2 + (2 – 3/2)2 1/2) = 1/2
similarly σY = √(2/3)
Now cov(X, Y) is
and correlation coefficient,
ρX, Y = cov(X, Y)/ (σXσY) = (1/4) / (1/2 x √(3/2)) = 0.61
Covariance matrix : [[ 985.42897959 229.23591837] [ 229.23591837 95362.44244898]] Covariance: 229.23591836734713 Pearson correlation: 0.023647286832189817 Calculated Pearson coeff: 0.024129884522642694
The conditional expectation of X given Y = y is
If r(x, y) is a function of x and y then
Conditional expectation corresponds to the mean X among those times in which Y = y.