Fisher's F-test

A F-test is a parametric statistic based on F-distribution. The test assumes that the underlying distribution of observations is a Fisher’s F distribution under the null hypothesis. Exact F-tests are specifically applied to least-sqaures estimations.

The test was developped by the American mathematical statistician George Waddel Snedecor and named after the English statistician and biologist Sir Ronald Aylmer Fisher, who earlier, had developed the statistic as the variance ratio.

Assumptions

F-test assumes an underlying F-distribution and is applied in severaral cases to test variability of one or multiple samples, related or unrelated.

§ F follows a F-distribution under the null hypothesis

§ Under the null hypothesis, the sums of squares should be statistically independent, and each should follow a scaled chi-squared distribution.

§ Hence, the bservations must be independent and normally distributed with a common variance.

Cases of F-Test

In parametric cases, F-test may be used to assess the following hypothesis:

Equality of means of several normally distributed populations, all having the same standard deviation. This most common application of F-test plays an important role in the analysis of variance (ANOVA).
Lack-of-Fit test: assess whether a proposed regression model fits the data well.
Adjusting significance levels in a linear regression analysis to account for multiple comparisons, also know as Scheffé's method for multiple comparisons adjustment in linear models. Scheffe’s method was named after the American Statistician Henry Scheffe.
Data set in a regression analysis follows the simpler of two proposed linear models that are nested within each other.

General Cases of F-test and Calculations

Explicit expressions that can be used to carry out various F-tests are given in the AroniSmartLytics section explaining the choice of a parametric test. The description of F-distribution is given in the AroniStat 1.0.1 and the probability distribution module of AroniSmartLytics. The formula used to estimate the F statistic under the null hypothesis is given in each case.

To test the significance of each test, one tailed, left or right, or two tailed test may be conducted.

Once an F value is determined, the p-value can be computed either from the AroniStat 1.0.1 and AroniSmartLytics distribution modules or the data analysis module of AroniSmartLytics. If the calculated p-value is below the statistical significance level chosen by the researcher of practictioner, then the null hypothesis is rejected in favor of the alternative hypothesis.

AroniSmartLytics covers multiple situations where F-test is appropriate, both in parametrics and non-parametric statistics settings.

In general, F-tests are derived from the decomposition of the variability in the samples of observations in terms of sums of squares. The test statistic in an F-test is the ratio of two scaled sums of squares. The sums of squares describe the sources of variability. Under the null hypothesis this ratio of the sums of squares tends to zero, whereas the statistic tends to be larger when the null hypothesis is not true. In order for the statistic to follow the F-distribution under the null hypothesis, the sums of squares should be statistically independent, and each should follow a scaled chi-squared distribution. A corolary to the chi-squared assumption is that the observations are independent and normally distributed with a common variance.

F-test and Analysis of Variance (ANOVA)

F-test is extensively used in assessing the variability among multiple groups. In this case, it is an extension of chi-square test of variability. In multiple-comparison settings, the purpose is to assess whethere the expected values of quantitative observation variables differ across the groups of interest. The problem is known as Analysis of Variance or ANOVA.

While t-test is useful in comparing two groups, in several instances researchers and practitioner need to assess the causes of variations from several potential sources. In this case, Analysis of Variance, usually designated by its acronym ANOVA, could be the suitable statistical technique. ANOVA, instead of being one formula, is a collection of statistical models and procedures. A variable is observed and then the researcher uses ANOVA techniques to identify and explain the different sources of variation. In general, multiple dependent variables may be observed and analyzes at the same time. In that case, ANOVA becomes MANOVA, or Multiple Analysis of Variance. In case where the independent or predictor variables are a mix of categorical (discrete ) and continuous variables, with at least one categorical and one continuous, and the independent or outcome variable is continuous, the ANOVA version of Analysis of Covariance (ANCOVA) is used.

The ANOVA F-test can be used to assess whether the mean of the observations for a group, sample or treatment is superior or inferior to the means of the others, with the null hypothesis that all groups, samples or treatments have the same mean response. In some cases, the researecher may be interested in paiwise comparisons, among all the groups, samples or treatments, hence requiring at M(M − 1)/2 tests for M groups.

The ANOVA F-test is known as an omnibus test, because one single test is used to detect several possible differences. In the case of multiple comparisons, the significance level must be adjusted to account for increased error and pairwise comparisons need to be specified a priori. The omnibus factor of the F-Test constitutes a disadvantage, as without multiple comparisons, it may not be clear which of the groups, samples, or tretaments has the greatest or smallest mean. Several methods exist to hadle multiple comparisons. These methods may be classified into two groups:

Single-step procedures

§ Tukey-Kramer method, also knownw as Tukey Range test, Tukey Method, Tukey’honest significance test, or Tukey’s Honestly Significant Difference (HSD) test, named after the American statisticians John Wilder Tukey and Clyde Kramer.

§ Scheffe Method, developped by Henry Scheffe

Multi-step procedures based on t- statistic (Studentized range).

§ Duncan’s new multiple range test (MRT), developped by the American Statistician David B. Duncan

§ Nemenyi test, developped by the American-Hungarian statistician and Civil Rights activist Peter Nemenyi.

§ Bonferrnoni-Dunn test, developped by the italian mathematician Carlo Emilio Bonferroni and

§ Student Newman-Keuls test also know as Student Neuman-Keuls Post-Hoc Analysis Test, developed by D. Newman and M. Keuls .

· ANOVA problems

The formula for the one-way ANOVA F-test statistic is based on the decomposition of the varaiance. The variance may be decomposed as follows:

Hence

The "explained variance", or "between-group variability" is:

where denotes the sample mean in the i^th group, n_i is the number of observations in the i^th group, and denotes the overall mean of the data.

The "unexplained variance", or "within-group variability" is

where Y_ij is the j^th observation in the i^th out of m groups and n is the overall sample size. Under the numm hypothesis, the F-statistic follows the F-distribution with m-1 and n-m degrees of freedom under the null hypothesis. The statistic will be large if the between-group or explained variability is large relative to the within-group or unxplained variability. When the populations groups means are equal, the statistic will be very small.

As a special case, when there are only two groups for the one-way ANOVA F-test, F = t² where t is the Student’s t statistic.

· Regression problems

For a regression in which Model i is nested within Model j. Model i is the Restricted model, and Model j is the Unrestricted one. That is, Model i has fewer paramerers, pi than Model j, pj . pj > pi,

In the regression case, under the null hypothesis of no differences among the two models, Model i and Model j, in getting a better fit, i.e. Model j does not offer a significant improvement in the fit than Model i.

Under the null hypothesis that model 2 does not provide a significantly better fit than model 1, F statistic will have an F-distribution, with (p_j – p_i, n – p_j) degrees of freedom.