Variance and Covariance

Variance

Variance means spread of a distribution

Let X be a random variable with mean μ. The variance of X – denoted by σ2 or σ2X or V(X) or VX is defined by
variance

assuming this expectation exists. The standard deviation is sd(X) = √V(X) and is also denoted by σ and σX.

Variance has the following properties.
1. V(X) = E(X2) – μ2
2. If a and b are constants then V(aX+b) = a2V(X)
3. If X1, . . . , Xn are independent and a1, . . . , an are constants, then
variance properties

Example: Let X ~ Binomial(n, p). We write X = ∑iXi where Xi = 1 if toss i is head and Xi = 0 otherwise. Here, P(Xi = 1) = p and P(Xi = 0) = 1 – p.
Using the formula of expectation,
E(Xi) = [p x 1] + [(1 – p) x 0] = p
E(Xi2) = [p x 12] + [(1 – p) x 02] = p
Now, V(Xi) = E(Xi2) – p2 = p – p2 = p(1 – p).
Finally, V(X) = V(∑iXi) = ∑iV(Xi) = ∑ip(1 – p) = np(1 – p).


Sample mean & Sample variance

If X1, . . . , Xn are random variables then we define the sample mean to be
sample mean
and the sample variance to be
sample variance
Note the denominator in sample variance as n-1 instead of n. This is called Bessel’s correction. The mean of the sample is closer to the sample data points than the actual mean of the population. Thus the numerator term in sample variance is smaller than expected. To balance this, the denominator is reduced by 1, known as Bessel’s correction.

When writing
Sample mean = (X1 + X2 + . . . + Xn) / n
Given a sample of size n, consider n independent random variables X1X2, …, Xn, each corresponding to one randomly selected observation. Each of these variables has the distribution of the population, with mean µ and standard deviation σ.

Numpy Variance: 909.9216
Calculated variance 1: 909.9216
Calculated variance 2: 909.9216

Covariance

Let X and Y be random variables with mean μX and μY and standard deviation σX and σY. Define the covariance between X and Y by

Cov(X, Y) = E[(X – μX) (Y – μY)]
and the correlation by

co-variance

Covariance satisfies the following property
Cov(X, Y) = E(XY) – E(X)E(Y)

Example:
Let X and Y have joint probability mass function as follows:

y
f(x, y)123fX(x)
11/41/401/2
x201/41/41/2
fY(y)1/41/21/41

Lets calculate µX and µY.
µX = ∑xfX(x) = (1 x 1/2) + (2 x 1/2) = 3/2
µY = ∑yfY(y) = (1 x 1/4) + (2 x 1/2) + (3 x 1/4) = 2

Lets calculate σX and σY.
σX = √∑(x – µX)2fX(x) = √( (1 – 3/2)21/2 + (2 – 3/2)2 1/2) = 1/2
similarly σY = √(2/3)

Now cov(X, Y) is



and correlation coefficient,
ρX, Y = cov(X, Y)/ (σXσY) = (1/4) / (1/2 x √(3/2)) = 0.61

Covariance matrix : 
[[  985.42897959   229.23591837]
[  229.23591837 95362.44244898]] 
Covariance: 229.23591836734713 

Pearson correlation: 0.023647286832189817 
Calculated Pearson coeff: 0.024129884522642694

Conditional Expectation

The conditional expectation of X given Y = y is
conditional expectation

If r(x, y) is a function of x and y then

conditional exp 2

Conditional expectation corresponds to the mean X among those times in which Y = y.

Leave A Comment