# Estimators

The following are desirable properties for statistics that estimate population parameters:

• Unbiased: on average the estimate should be equal to the population parameter, i.e. t is an unbiased estimator of the population parameter τ provided E[t] = τ.
• Consistent: the accuracy of the estimate should increase as the sample size increases
• Efficient: all things being equal we prefer an estimator with a smaller variance

Property 1: The sample mean is an unbiased estimator of the population mean

Proof: If we repeatedly take a sample {x1, x2, …, xn}  of size n from a population with mean µ, then the sample mean can be considered to be a random variable defined by

Since each of the xi are random variables from the same population, E[xi] = μ for each i, from which it follows by Property 1 of Expectation that:

Property 2: If we repeatedly take a sample {x1, x2, …, xn}  of size n from a population with variance σ2, then

Proof: Since the xi are independent with variance σ2, by Property 6.3b and 6.4b of Expectation

Observation: The proof of Property 2 depends on the sample members being distributed independently of each other. This may not be the case where the sample is not a small fraction of the population. In this case it can be shown that

where N = the size of the population.

Observation: If we repeatedly take a sample  {x1, x2, …, xn}  of size n from a population, then the variance s2 of the sample is a random variable defined by

Property 3: The sample variance is an unbiased estimator of the population variance.