# 1. The T-Test Formula A t-test compares the means of two groups by standardizing their difference against the uncertainty in that difference: $ T = \frac{\bar{X_1}-\bar{X_2}}{SE} $ How the standard error ($SE$) is calculated depends on whether the two groups are assumed to have equal variance. **When variances are assumed equal** (the classic Student's t-test), the standard error pools the two groups' variances together: $ SE=\sqrt{S^2_p\left(\frac{1}{n_1}+\frac{1}{n_2}\right)} $ with degrees of freedom $df = n_1 + n_2 - 2$. In Python, this is run with `ttest_ind(equal_var=True)`, and the equal-variance assumption itself can be checked with `scipy.stats.levene`. **When variances are not assumed equal** (Welch's t-test), each group's variance is kept separate: $ SE = \sqrt{\frac{S^2_1}{n_1}+\frac{S^2_2}{n_2}} $ run in Python with `ttest_ind(equal_var=False)`. Welch's version is generally the safer default when there's no strong reason to believe the two groups share the same variance. The deeper reason a *t*-distribution is used at all, rather than a normal (Z) distribution, comes down to what's known versus estimated: the standard Z-score formula $Z = \frac{\bar{X}-\mu}{\sigma/\sqrt{n}}$ requires knowing the true population standard deviation $\sigma$. When that's unknown — which is almost always the case in practice — and has to be estimated from the sample instead, that extra layer of uncertainty is exactly what the t-distribution accounts for, producing a distribution that's slightly wider than the normal distribution (see the Statistical Knowledge note for more on the t-distribution's shape). # 2. Assumptions A t-test relies on two core assumptions: the observations must be **independent** of one another (not drawn from a network, friend group, or otherwise clustered structure that would violate independence), and the **residuals should be approximately normal**. In practice, this normality assumption becomes less important as sample size grows: once $n>30$, the Central Limit Theorem usually makes the t-test fairly robust to non-normality, and once $n>1000$, normality violations rarely matter at all. # 3. Types of T-Tests **One-sample t-test.** Compares a single sample's mean against a known or hypothesized value, using `scipy.stats.ttest_1samp`. **Independent two-sample t-test.** Compares the means of two independent groups, using `ttest_ind()`. **Paired t-test.** Compares two measurements taken on the *same* subjects, typically before and after some intervention, using `ttest_rel()`. # 4. When Not to Use a T-Test A t-test is the wrong tool in several common situations, each of which calls for a different test instead: - **More than two groups:** use ANOVA or Welch's ANOVA instead of running multiple pairwise t-tests. - **Repeated measurements across more than two occasions:** use Repeated Measures ANOVA or a mixed-effects model. - **Strongly non-normal data with a small sample:** use the Mann-Whitney U test (for independent samples) or the Wilcoxon Signed-Rank test (for paired samples). - **Binary outcomes** (e.g. click / no click): use a two-proportion z-test, a chi-square test, or logistic regression. - **Count outcomes** (e.g. number of orders or messages): use Poisson or Negative Binomial regression.