Estimators

The following are desirable properties for statistics that estimate population parameters:

  • Unbiased: on average the estimate should be equal to the population parameter, i.e. t is an unbiased estimator of the population parameter τ provided E[t] = τ.
  • Consistent: the accuracy of the estimate should increase as the sample size increases
  • Efficient: all things being equal we prefer an estimator with a smaller variance

Property 1: The sample mean is an unbiased estimator of the population mean

Proof: If we repeatedly take a sample {x1, x2, …, xn}  of size n from a population with mean µ, then the sample mean can be considered to be a random variable defined by

image309

Since each of the xi are random variables from the same population, E[xi] = μ for each i, from which it follows by Property 1 of Expectation that:

image313

Property 2: If we repeatedly take a sample {x1, x2, …, xn}  of size n from a population with variance σ2, then

image315

Proof: Since the xi are independent with variance σ2, by Property 6.3b and 6.4b of Expectation

image316

Observation: The proof of Property 2 depends on the sample members being distributed independently of each other. This may not be the case where the sample is not a small fraction of the population. In this case it can be shown that

image317

where N = the size of the population.

Observation: If we repeatedly take a sample  {x1, x2, …, xn}  of size n from a population, then the variance s2 of the sample is a random variable defined by

image320

Property 3: The sample variance is an unbiased estimator of the population variance.

Click here for a proof of Property 3.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>