Law of Large Number and CLT
The Weak Law of Large Numbers (WLLN)
If X1, X2, . . . , Xn are IID, then
This theorem says that the distribution of X̄ becomes concentrated around µ as n gets large. For example, the proportion of heads in a large number of tosses is expected to be close to 1/2. This, however, doesn’t mean that X̄ will be numerically equal to 1/2. It means that, when n is large, the distribution of X̄ is tightly concentrated around 1/2.
PROOF:
Using Chebyshev’s inequality,
which tends to 0 as n → ∞. Note that the Variance of the Sample mean is σ2/n. Find the proof here.
Example:
In tossing a coin, let p=0.5. How large should be n so that P(0.4 ≤ X̄ ≤ 0.6) ≥ 0.7?
Solution: E(X̄n) = p = 1/2 and V(X̄n) = σ2/n = p(1 – p)/n = 1/(4n)
P(0.4 ≤ X̄n ≤ 0.6) = P(|X̄n – µ| ≤ 0.1)
= 1 – P(|X̄n – µ| ≥ 0.1)
≥ 1 – (1 / 4n(0.1)2) = 1 – 25/n
which will be larger than 0.7 if n=84.
Central Limit Theorem
The central limit theorem says that the sample mean X̄n is approximately Normal with mean µ and variance σ2/n.
If X1, X2, . . . , Xn are IID with mean µ and variance σ2. Let X̄n be the sample mean then
where Z ~ N(0, 1).
