We now consider an experimental design where we want to determine whether there is a difference between two groups within the population. For example, let’s suppose we want to test whether there is any difference between the effectiveness of a new drug for treating cancer. One approach is to create a random sample of 40 people, half of whom take the drug and half take a placebo. For this approach to give valid results it is important that people be assigned to each group at random. Such samples are independent.

When the population variances are known, hypothesis testing can be done using a normal distribution, as described in Comparing Two Means when Variances are Known. But population variances are not usually known. The approach we use instead is to pool sample variances and use the* t* distribution.

We consider three cases where the *t* distribution is used:

- Equal variances
- Unequal variances
- Paired samples

We deal with the first of these cases in this section.

**Theorem 1**: Let *x̄* and ȳ be the sample means of two sets of data of size *n _{x}* and

*n*

_{y}respectively. If

*x*and y are normal, or

*n*and

_{x}*n*

_{y}are sufficiently large for the Central Limit Theorem to hold, and

*x*and y have the same variance, then the random variable

has distribution *T*(*n _{x} + n*

_{y}– 2) where

**Observation**: *s*, as defined above, can be viewed as a way to pool *s _{x}* and

*s*

_{y}, and so

*s*

^{2}

*is referred to as the*

**pooled variance**. Also note that the degrees of freedom of

*t*is the value of the denominator of

*s*

^{2}in the formula given in Theorem 1.

**Click here** for a proof of Theorem 1.

**Real Statistics Excel Functions**: The following supplemental functions are provided in the Real Statistics Resource Pack.

**VAR_POOLED**(R1, R2) = pooled variance of the samples defined by ranges R1 and R2, i.e. *s*^{2} of Theorem 1

**STDEV_POOLED**(R1, R2) = pooled standard deviation of the samples defined by ranges R1 and R2, i.e. *s*^{ }of Theorem 1

**STDERR_POOLED**(R1, R2, *b*) = pooled standard error of the samples defined by ranges R1 and R2. This is equal to the denominator of *t* in Theorem 1 if b = TRUE (default) and equal to the denominator of *t* in Theorem 1 of Two Sample t Test with Unequal Variances if *b* = FALSE. When the sample sizes are equal, *b* = TRUE or *b* = FALSE yields the same result.

**Observation**: Each of these functions ignores all empty and non-numeric cells.

**Example 1**: A marketing research firm tests the effectiveness of a new flavoring for a leading beverage using a sample of 20 people, half of whom taste the beverage with the old flavoring and the other half who taste the beverage with the new favoring. The people in the study are then given a questionnaire which evaluates how enjoyable the beverage was. The scores are as in Figure 1. Determine whether there is a significant difference between the perception of the two flavorings.

As we can see from the box plot in Figure 1 the data in each sample is reasonably symmetric and so we use the *t* test with the following null hypothesis:

H_{0}: *μ*_{1} – *μ*_{2} = 0; i.e. there is no difference between the two flavorings

Since the sample variances are similar we decide that the population variances are also likely to be similar and so apply Theorem 1.

And so *s* = = 4.01. Now,

Since p-value = TDIST(*t, df*) = TDIST(2.18, 18) = .043 < .05 = *α,* we reject the null hypothesis, concluding that there is a significant difference between the two flavorings. In fact, the new flavoring is significantly more enjoyable.

The same result can be obtained by use of Excel’s **Two-Sample Assuming Equal Variances** data analysis tool, the results of which are as follows.

**Observation**: The Real Statistics Resource Pack also provides a data analysis tool which supports the two independent sample t test, but provides additional information not found in the standard Excel data analysis tool. Example 3 in Two Sample t Test: Unequal Variances gives an example of how to use this data analysis tool.

**Example 2**: To investigate the effect of a new hay fever drug on driving skills, a researcher studies 24 individuals with hay fever: 12 who have been taking the drug and 12 who have not. All participants then entered a simulator and were given a driving test which assigned a score to each driver as summarized in Figure 3.

As in the previous example, we plan to use the t-test, but with a sample this small we first need to check to see that the data is normally distributed (or at least symmetric). This can be seen from the histograms. Also the variances are relatively similar (15.18 and 17.88) and so we can again use the t-Test: Two-Sample Assuming Equal Variances data analysis tool to test the following null hypothesis:

H_{0}: *μ _{control} = μ_{drug}*

Since *t _{obs}* = .10 < 2.07 =

*t*(or p-value = .921 > .05 =

_{crit}*α*) we retain the null hypothesis; i.e. we are 95% confident that any difference between the two groups is due to chance.

**Observation**: The t-test is quite robust even when the underlying distributions are not normal provided the sample size is sufficiently large (usually over 25 or 30). The t-test can be valid even with smaller sample sizes, provided the samples have similar shape and are not too skewed.

### Effect size

The Cohen effect size *d* can be calculated as in One Sample t Test, namely:

**Example 3:** Find the effect size for study in Example 2.

This means that the control group has a driving score 4.1% of a standard deviation more than the group that is taking the hay fever medication.

I am using a t test to compare before and after weights using a diet. n=78. In excel, I am not sure whether to use 1, 2, or 3, under type in the formula box. Can you explain clearly the differences? Thanks.

Jo,

You use type = 1 when the two samples are not independent. E.g. (1) when the first sample contains men and the second contains their wives or, as in your case (2) the first sample contains each person’s weight before dieting while the second sample contains their weight after dieting. In example (2) the same person is being sampled and so the samples can’t be independent.

Type 2 and 3 are used when the two samples are independent. E.g. 20 people are selected at random and half are randomly put in group 1 and half are randomly put in group 2. The difference between type 2 and type 3 relates to the variances of the populations from which the samples are drawn. If the variances are equal then use type = 2, while if the variances are unequal use type = 3. In reality the variance don’t have to be identical to use type = 2. Even if they are close you will usually get good results You usually judge two populations to have equal variances if the two samples have variance that are not too different; in fact even if one sample has a variance which is 4 times the other, the results will be pretty good even if you use type = 2.

For your situation, it looks like you want to use type = 1.

Charles

Dear Charles

First , many thanks for sharing such valuable knowledge . I have a question regarding this page . In example 1 , while I understand that the null hypotheis is rejected because of P value of tow-tail is lower than 0.05, I can not understand this sentence that you have added at the end of example :”In fact, the new flavoring is significantly more enjoyable.” I do not know how should I recognize whether the dependent variable in sample one is increased or decreased significantly compared to the sample 2. All my appreciation for any advice as I am stucked in it

Dear Marzieh,

Before you collect data and conduct the test you don’t know whether the mean of the sample will be in the right tail, left tail or neither. After the test you have evidence as to whether or not the sample mean is in one of the tails and if so which tail. It is on this basis that I drew a conclusion. Of course this conclusion won’t always be correct, but the evidence points in the direction indicated.

Charles

Assuming I have 2 treatments and three trials per treatment. How should I solve the statistical problem of this?

Noemi,

Are you saying that you have two treatments and a sample of 3 for each treatment? You aren’t going to do much with such a small sample, but it is probably best to use the Mann-Whitney test instead of a t-test in this case.

Charles

a number of samples 170, in 2012 mean 12366 std dev 3891 and in year 2011 mean 12549 std dev 4232 . corelation-coeff .92879. Can it be concluded that the value of stock holding decreased? how to solve this problem using excel? i mean i am facing trouble to determine std err

You are referencing the webpae regarding two sample t-test with independent samples, but the correlation coeffcient of .92879 shows that these samples are not likely to be independent. Perhaps you are trying to run a two paired samples test — e.g. where the stock holdings are in say 170 different factories, comparing 2011 with 2012. In this case you might be able to use the fact that var(x-y) = var(x) + var(y) – cov(x, y) and cov(x,y) = corr(x,y) * sqrt(var(x))*sqrt(var(y)). This is the best I can do with the information you have provided. I hope that it is helpful.

Charles

Hi Charles,

In the first example, can I know the meanings of data entered in Old and New columns?

For example, does the greater the number explain the greater the enjoyment?

What scale could be used?

Clarification could be helpful.

Thank you in advance for the post and the answer as well.

Presumably the higher the score the more enjoyable the beverage (although for the t test it really doesn’t matter whether a higher score represents more or less enjoyment). Scale also really doesn’t matter, but let’s assume that each point represents the more enjoyment option in a series of True/False questions.

Charles

What do you do if the number of cases are not equal? How would you calculate the df then?

The formulas and functions on the referenced webpage don’t require that the number of cases be the same. Just use the formulas and functions shown on the webpage.

Charles

I typed a comment last evening asking for help trying to decide how to handle a statistics problem. I need to form a hypothesis where two groups of students at a Private Catholic School one group who lives in North Bethesda (120 observations) are said to receive higher grades on the SATs. Those students living in South Bethesda (132 observations) received lower test scores on the SAT. Sample mean is 86 and 87 respectively with 8.1 and 7.3 populations variances and an .01 level of significance. Is there evidence that students living in South Bethesda may be getting lower grades on the SATs?

Kennedy,

I sent a response yesterday. I am resending it below:

Since you have the population variances you can use a two sample test using the normal distribution, as described in Theorem 1 of Comparing Two Means.

The null hypothesis is mean1 >= mean2 (these are population means). The test statistic is z = (m1-m2)/stdev, where m1-m2 = 88-87 = 1 (sample means) and stdev = sqrt(var) where var = v1^2/n1 + v2^2/n2 = 6.2^2/132 + 7.0^2/125. If NORMSDIST(z) > .99 then you reject the hypothesis that the the workers receive the same pay. This is a one tailed test. If you want a two-tailed test you need to replace .99 by .995.

If instead of the population variance you had the sample variances you would use Theorem 1 of Two Sample t Test instead.

Charles

Pingback: About Significant Test | Lin Chen's Blog