The ANOVA F-test is used to test for difference in means between groups, and requires the conditions of normality (or large sample size), independence, and constant variance. A common rule of thumb for the constant variance condition is that the ratio of largest to smallest standard deviation is less than or equal to two. This application implements a user-guided simulation study to assess the consequences of non-constant variance on the Type I error rate of the ANOVA F-test. The application enables the user to visualize data with different standard deviations, reinforces the concepts of sampling distribution, null distribution, and Type I error, and allows the user to uncover a rule of thumb for the constant variance condition.
At left, the user specifies standard deviations for three hypothetical populations and sample sizes to be drawn from each of the populations. When the user presses the “Draw samples” button, data will be simulated from normal distributions with mean zero and the specified standard deviations and sample sizes and displayed in dot plots in the left graph. The ANOVA F-statistic for the simulated data is plotted in the graph at right, and the critical value for a 0.05-level hypothesis test is shown in red. As more samples are drawn (with the option to draw up to 1,000 samples at a time), more F-statistics are plotted in the sampling distribution on the right. The Type I error rate is estimated as the proportion of samples for which the null hypothesis was rejected, and is displayed below the graphs. Below the graphs (not included in the picture above) is guidance for a suggested series of simulation studies allowing the user to compare different specifications systematically and uncover the rule of thumb for the constant variance condition.