Chi-square Distribution – Advanced

We now give some additional technical details about the chi-square distribution and provide proofs for some of the key propositions. Except for the proof of Corollary 2 knowledge of calculus is required.

Observation: For any positive real number k, per Definition 1, the chi-square distribution with k degrees of freedom, abbreviated χ2(k), has probability density function

Using the notation of Gamma Function Advanced, the cumulative distribution function for ≥ 0 is
Property A: The moment generating function for a random variable with χ2(k) distribution is

Proof:

Set y = x/2 (2θ–1). Then x = 2y/(1–2θ) and dx=2 dy/(1–2θ). Thus

The result follows from the fact that by Definition 1 of Gamma Function

Property 1: The χ2(k) distribution has mean k and variance 2k

Proof: By Property A, the moment generating function of χ2(k) is

And so

Since

it follows that

Theorem 1: Suppose x has standard normal distribution N(0,1) and let x1,…, xk be k independent sample values of x, then the random variable

has a chi-square distribution χ2(k).

Proof: Since the xi are independent, so are the $x_i^2$. From Property 4 of General Properties of Distributions, it follows that

But since all the xi are samples from the same population, it follows that

Since x is a standard normal random variable, we have

Set  y = $\sqrt{1-2\theta}$. Then x = y/$\!\sqrt{1-2\theta}$ and dx = dy/$\!\sqrt{1-2\theta}$, and so

But since

is the pdf for N(0, 1), it follows that

Thus,

Combining the pieces, we have

But by Property A this is the same moment generating function as for χ2(k). Thus, by Theorem 1 of General Properties of Distributions, it follows that w has distribution χ2(k).

Property 2: If x and y are independent and x has distribution χ2(m) and y has distribution χ2(n), then x + y has distribution χ2(m + n)

Proof: Since x and y are independent, by Property A and Property 4 of General Properties of Distributions

But this is the moment generating function for χ2(m + n), and so the result follows from the fact that a distribution is completely determined by its moment generating function (Theorem 1 of General Properties of Distributions).

Theorem 2If x is drawn from a normally distributed population N(μ, σ) then for samples of size n the sample variance s2 has distribution

Proof: Since (µ)2 is a constant and

we have

Dividing both sides by σ2 and rearranging terms we get

Now we use the fact that x ~ N(μ, σ) with each xi independently sampled from this distribution. Thus the left side of the equation is the sum of n variables, each the square of a z-score, and so by Theorem 1, the left side of the equation has chi-square distribution with n degrees of freedom:

Next consider the last term of the equation above. Since the sampling distribution of the mean for data sampled from a normal distribution N(μ, σ) is normal with mean μ and standard deviation σ/$\! \sqrt{n}$, it follows that the last term is the square of a z-score, and so by Theorem 1, it has chi-square distribution with 1 degree of freedom

By Property 2, it follows that the remaining term in the equation is also chi-square with n – 1 degrees of freedom.

Dividing by n – 1, it follows that

But since

It follows that

which completes the proof.

Corollary 2s2 is an unbiased, consistent estimator of the population variance

Proof (unbiased): From the theorem and Property 1 it follows that:

Proof (consistent): From Property 1 and Property 3b of Expectation:

And so,

5 Responses to Chi-square Distribution – Advanced

1. Min says:

Excellent, helps me explain a lot of things!

2. mzalendo says:

Thanks for your analysis I take it for my assignment

3. Carla says:

I loved your explanation, I’ve been looking in many books for this.
Thank you for this easy to follow proofs.

4. Sydeny Bhebe says:

This is refreshing and easy to follow for begginers. All too often many qualified/advanced staticians skip basic explanations and say ” …it is trivial to prove that… ” ; yet for the begginner, she/he would be expecting those very basics in order to come to a level where they will see the triviality! Thanks a lot and I will keep coming back for more as I am just starting statistics especially the mathematical statistics.

• Charles says:

Sydeny,
Thanks for your comment. I try hard to give some of the theory and basic explanations, without getting too theoretical, for just the reasons you stated.
Charles