Law of Large Number and CLT

The Weak Law of Large Numbers (WLLN)

If X1, X2, . . . , Xn are IID, then

This theorem says that the distribution of  X̄ becomes concentrated around µ as n gets large. For example, the proportion of heads in a large number of tosses is expected to be close to 1/2. This, however, doesn’t mean that X̄ will be numerically equal to 1/2. It means that, when n is large, the distribution of X̄ is tightly concentrated around 1/2.

PROOF:
Using Chebyshev’s inequality,

which tends to 0 as n → ∞. Note that the Variance of the Sample mean is σ2/n. Find the proof here.

Example:
In tossing a coin, let p=0.5. How large should be n so that P(0.4 ≤ X̄ ≤ 0.6) ≥ 0.7?
Solution: E(X̄n) = p = 1/2 and V(X̄n) = σ2/n = p(1 – p)/n = 1/(4n)
P(0.4 ≤ X̄n ≤ 0.6) = P(|X̄n – µ| ≤ 0.1)
= 1 – P(|X̄n – µ| ≥ 0.1)
≥ 1 – (1 / 4n(0.1)2) = 1 – 25/n
which will be larger than 0.7 if n=84.

Central Limit Theorem

The central limit theorem says that the sample mean X̄n is approximately Normal with mean µ and variance σ2/n.

If X1, X2, . . . , Xn are IID with mean µ and variance σ2. Let X̄n be the sample mean then

where Z ~ N(0, 1).

1 Comment

    Leave A Comment