Chi-square Distribution – Advanced

We now give some additional technical details about the chi-square distribution and provide proofs for some of the key propositions. Except for the proof of Corollary 2 knowledge of calculus is required.

Observation: For any positive real number k, per Definition 1, the chi-square distribution with k degrees of freedom, abbreviated χ2(k), has probability density function

Chi square pdf

Using the notation of Gamma Function Advanced, the cumulative distribution function for ≥ 0 is
Chi-square distribution functionProperty A: The moment generating function for a random variable with χ2(k) distribution is

image3315

Proof:

image3316

Set y = x/2 (2θ–1). Then x = 2y/(1–2θ) and dx=2 dy/(1–2θ). Thus

image3320 image3321

The result follows from the fact that by Definition 1 of Gamma Function

image3322

Property 1: The χ2(k) distribution has mean k and variance 2k

Proof: By Property A, the moment generating function of χ2(k) is

image3315

And so
image3323

Since
image3324

it follows that

image3325 image3326

Theorem 1: Suppose x has standard normal distribution N(0, 1) and let x1, …, xk be k independent sample values of x, then the random variable

image774

has a chi-square distribution χ2(k).

Proof: Since the xi are independent, so are the x_i^2. From Property 4 of General Properties of Distributions, it follows that

image3330

But since all the xi are samples from the same population, it follows that

image3331

Since x is a standard normal random variable, we have

image3332

Set  y = \sqrt{1-2\theta}. Then x = y/\!\sqrt{1-2\theta} and dx = dy/\!\sqrt{1-2\theta}, and so

image3336

But since
image3337

is the pdf for N(0, 1), it follows that

image3339

Thus,
image3340

Combining the pieces, we have

image3341

But by Property A this is the same moment generating function as for χ2(k). Thus, by Theorem 1 of General Properties of Distributions, it follows that w has distribution χ2(k).

Property 2: If x and y are independent and x has distribution χ2(m) and y has distribution χ2(n), then x + y has distribution χ2(m + n)

Proof: Since x and y are independent, by Property A and Property 4 of General Properties of Distributions

image3343

But this is the moment generating function for χ2(m + n), and so the result follows from the fact that a distribution is completely determined by its moment generating function (Theorem 1 of General Properties of Distributions).

Theorem 2If x is drawn from a normally distributed population N(μ, σ) then for samples of size n the sample variance s2 has distribution

image786

Proof: Since (µ)2 is a constant and

image3345 we have

image3346

image3347

Dividing both sides by σ2 and rearranging terms we get

image3348

Now we use the fact that x ~ N(μ, σ) with each xi independently sampled from this distribution. Thus the left side of the equation is the sum of n variables, each the square of a z-score, and so by Theorem 1, the left side of the equation has chi-square distribution with n degrees of freedom:

image3350

Next consider the last term of the equation above. Since the sampling distribution of the mean for data sampled from a normal distribution N(μ, σ) is normal with mean μ and standard deviation σ/\! \sqrt{n}, it follows that the last term is the square of a z-score, and so by Theorem 1, it has chi-square distribution with 1 degree of freedom

image3351

By Property 2, it follows that the remaining term in the equation is also chi-square with n – 1 degrees of freedom.

image3352

Dividing by n – 1, it follows that

image3353

But since
image320

It follows that

image3354

which completes the proof.

Corollary 2s2 is an unbiased, consistent estimator of the population variance

Proof (unbiased): From the theorem and Property 1 it follows that:

image787

Proof (consistent): From Property 1 and Property 3b of Expectation:

image788

And so,
image789

5 Responses to Chi-square Distribution – Advanced

  1. Min says:

    Excellent, helps me explain a lot of things!

  2. mzalendo says:

    Thanks for your analysis I take it for my assignment

  3. Carla says:

    I loved your explanation, I’ve been looking in many books for this.
    Thank you for this easy to follow proofs.

  4. Sydeny Bhebe says:

    This is refreshing and easy to follow for begginers. All too often many qualified/advanced staticians skip basic explanations and say ” …it is trivial to prove that… ” ; yet for the begginner, she/he would be expecting those very basics in order to come to a level where they will see the triviality! Thanks a lot and I will keep coming back for more as I am just starting statistics especially the mathematical statistics.

    • Charles says:

      Sydeny,
      Thanks for your comment. I try hard to give some of the theory and basic explanations, without getting too theoretical, for just the reasons you stated.
      Charles

Leave a Reply

Your email address will not be published. Required fields are marked *