Two sample t

Introduction

Suppose we have two independent random Normal samples of sizes \(n\) and \(m\), namely \[X_1,\ldots,X_n\sim\text{independent }N(\mu_1,\sigma_1^2),\] \[Y_1,\ldots,Y_m\sim\text{independent }N(\mu_2,\sigma_2^2).\] Typically our aim will be to estimate or test \(\mu_1-\mu_2.\)

As in the one sample case, \(\bar X \sim N(\mu_1^2,\frac{\sigma_1^2}{n})\) while \(\bar Y \sim N(\mu_2^2,\frac{\sigma_2^2}{m}).\) Moreover, these two will be independent of each other.

Equal variances?

We can calculate \(S_1^2\) and \(S_2^2\) to estimate the two samples’ variances. Under our usual assumptions of independence and Normally distributed samples, then the ratio of sample variances \(F=S_1^2/S_2^2\sim F_{n-1,m-1}\), where \(F_{n-1,m-1}\) denotes the F distribution on \(n-1\) and \(m-1\) degrees of freedom.

This allows us to conduct an \(F\)-test to determine whether an assumption of equal variances for the two samples is appropriate. Do note, however, that the \(F\)-test is very sensitive to non-Normality, so for practical purposes with data you’re not certain is Normal, alternatives like Levene’s test or Bartlett’s test may be more appropriate.

Unequal variances

As we’ve seen in the lectures, \[ESE(\bar{X}-\bar{Y})=\sqrt{\frac{S_1^2}{n}+\frac{S_2^2}{m}}.\] The \(t\)-statistic for unequal variances is \[T_A=\frac{(\bar{X}-\bar{Y})-(\mu_1-\mu_2)}{ESE(\bar{X}-\bar{Y})}.\] To a very good approximation, \(T_A\) is distributed as \(t_{[\nu]}\) where \[\nu=\frac{(f_1+f_2)^2}{\frac{1}{n-1}f_1^2+\frac{1}{m-1}f_2^2},\] where \(f_1=S_1^2/n\) and \(f_2=S_2^2/m\). If the result isn’t an integer, round down.

Equal variances

The pooled sample variance is \[\begin{aligned} S^2&=\frac{1}{n+m-2}\left\{\sum(X_i-\bar{X})^2+\sum(Y_i-\bar{Y})^2\right\} \\ &= \frac{(n-1)S_1^2+(m-1)S_2^2}{(n-1)+(m-1)}.\end{aligned}\] Then \[ESE(\bar{X}-\bar{Y})=\sqrt{S^2\left(\frac{1}{n}+\frac{1}{m}\right).}\] The \(t\)-statistic is then distibuted as \(t_{[n+m-2]}\).

Note

Click here for the slides we used in the lecture to formulate the example problem we used for the two-sample t test.

Interpreting p-values

The following gives a useful guide to translating \(p_0\) into a verbal conclusion. Note that if \(p_0 \leq \alpha\) then we can say that \(H_0\) is rejected at the \(100\alpha\%\) level.

  • \(p_0\leq0.05\): significant at the 5% level, moderate evidence against \(H_0\), in favour of \(H_1\).

  • \(p_0\leq0.01\): significant at the 1% level, strong evidence against \(H_0\), in favour of \(H_1\).

  • \(p_0\leq0.001\): significant at the 0.1% level, very strong evidence against \(H_0\), in favour of \(H_1\).

  • \(p_0\leq0.0001\): significant at the 0.01% level, extremely strong evidence against \(H_0\), in favour of \(H_1\).

  • \(p_0>0.1\): not significant at the 10% level, little evidence against \(H_0\), not enough to reject.

  • \(0.05 \leq p_0 \leq 0.1\) : not significant at the 5% level, but significant at 10%. A borderline case: sometimes more data is needed before a firm conclusion can be reached.