Note on Notation

Once again, we will use different notations for the various things $F$ can denote:

• The test and distribution’s name: $F$

• The distribution: $\mathcal{F}$

• The test statistic: $\phi$

• The pdf: $f_F$

• The cdf: $F_F$

  Cheat sheet

• Description: Tests if the variance of two normally-distributed samples, $\vec{x}_1$ and $\vec{x}_2$, are equal,

• Statistic: $\sigma_1^2/\sigma_2^2$ (ratio of variances)

• Distribution: $\mathcal{F}(n_1-1,n_2-1)$ ($F$)

• Sidedness: Two-sided

• Null Hypothesis: $H_0: \sigma_1^2 = \sigma_2^2$

• Alternative Hypothesis: $H_a:\sigma_1^2 \neq \sigma_2^2$

• Test Statistic: $$\phi=\frac{s_1^2}{s_2^2}$$

  Description

  In the last section, we went over how the $\chi^2$ distribution is derived from summing standard normal distribution. Well, the $F$ distribution is derived from dividing two $\chi^2$ distributions. Specifically if $S_1\sim\mathcal{K}(k_1)$ and $S_2\sim\mathcal{K}(k_2)$ then $$\frac{S_1/k_1}{S_2/k_2}\sim\mathcal{F}(k_1,k_2)$$ This $\chi^2$ distribution might seem like they come from nowhere in this test, but recall the definition of $s^2$, the sample variance.

$$s^2 =\frac{1}{n-1}\sum_{i=1}^n(x_i - \hat{\mu})^2$$ Since $x_i\sim\mathcal{N}(\mu,\sigma^2)$ under the assumptions of the $F$ test, $$\hat{\mu}\sim\mathcal{N}(\mu,\sigma^2/n)\Rightarrow (x_i-\hat{\mu})\sim\mathcal{N}(0,1)\Rightarrow (x_i-\hat{\mu})^2\sim \mathcal{K}(1)$$ A final thing to note is that, unlike other tests that assume that samples are normally distributed, the $F$ test is extremely sensitive to violations of non-normality. Thus, it would take an ever larger sample size than for other tests for a $F$ test to stay valid for non-normal samples.

Another thing to note is since $F$ tests are only two-sided, we can’t determine the “direction” of the test. In this test, for example, the result doesn’t tell you if $\sigma_1^2$ or $\sigma_2^2$ is larger than the other.

$F$ Test for Comparison of Multiple Means (Omnibus Test of Means)

Cheat Sheet

• Description: Tests if the means of $k$ normally-distributed samples, $\vec{x}_1,\vec{x}_2,\dots,\vec{x}_k$, with $n_1,n_2,\dots,n_k$ observations each ($N$ total)differ in at least one pair-wise comparison
  

• Statistic: $\mu_i-\mu_j$ (difference of means between any two groups)

• Distribution: $\mathcal{F}(k-1,N-k)$ ($F$)

• Sidedness: Two-sided

• Null Hypothesis: $H_0: \mu_1=\mu_2=\cdots=\mu_k$

• Alternative Hypothesis: $H_a:\mu_i\neq\mu_j$ for at least one pair $i,j\leq k$ where $i\neq j$

• Test Statistic: $$\phi=\frac{V_e}{V_{\bar{e}}}$$

  Description

  The $V_e$ and $V_{\bar{e}}$ in $\phi$ denote the “explained variance” and “unexplained variance” respectively. They are also called be the “between-group variability” and “within-group variability”. The idea is that we can take the ratio of the sum of the variances of each group and the variance of the samples taken as a whole as a proxy to determine if there is a difference of means between groups. This is because we would expect these two “variances” to not differ if all groups share the same mean.

As for definitions, $V_e$ is defined as:

$$V_e=\sum_{i=1}^k \frac{n_i(\hat{\mu}_i-\hat{\mu}_A)}{k-1}$$ where $\hat{\mu}_A$ is the mean of all samples when combined.$V_{\bar{e}}$ is defined as:

$$V_{\bar{e}}=\sum_{i=1}^k\sum_{j=1}^{n_i}\frac{(x_{i,j}-\hat{\mu}_i)}{N-k}$$ Where $x_{i,j}$ is the $j$th observation in the $i$th sample.

This test is useful because using multiple $t$ tests can greatly increase the chance of making a type I error. This is because running multiple $t$-tests exponentially increases the probability of getting false positives (also called “type I errors”). “Exponentially” here is not a placeholder for “a lot.” If each test has false-positive probability $\psi$, the probability of never getting a false positive in $n$ many tests is $\left(1-\psi\right)^n$, which clearly tends to zero as $n\rightarrow\infty$. The $F$-test doesn’t have this issue since it’s an “omnibus test”, meaning it tests all of these hypotheses “all at once.”