Post Hoc Analyses of ANOVA Interaction Effects

Lawrence Myers and his co-authors, Glenn Gamst and A. J. Guarino (2006), have made a valuable contribution to the field of applied statistics by clarifying an issue that has not been adequately addressed by others. I refer to the treatment of interaction effects in factorial ANOVAs involving two or more factors. Myers et al. make two important points. First, they state that when an interaction effect is found to be significant, understanding the nature of the interaction should become the dominant focus of the analysis. The importance of the main effects of the factors involved in the interaction becomes very much subordinated. This is because, in the presence of a significant interaction, any effort to interpret the main effects of the factors involved will be based on the false premise that differences on one factor exist across all levels of the other factor(s). Myers et al. state it this way:

A second important point that Myers et al. make is that post hoc analyses of the “simple effects” encompassed by an interaction should proceed by pairwise comparison of the levels of each factor within the levels of the other factor(s) in the interaction. For example, if A is a 2-level factor and B is 3-level factor, pairwise comparisons should be made between the 3 pairs of B levels within each of the two A levels. The Type I error levels of these comparisons should be corrected for family-wise error (e.g., using Bonferroni, LSD, or other procedure). Myers et al. also recommend that the comparisons be done both ways (i.e., between B levels within each A level, and between A levels within each B level), although they note that others (e.g., Keppel, 1991) suggest that only one of these be chosen on a priori conceptual grounds. In either case, this is quite a different approach than other widely read authors recommend. For example, Howell (2009, pp. 424-426) recommends analyzing whether the overall differences are significant between each level of a factor within each level of the other factor(s) involved in an interaction. This is adequate when there are only two levels of the factor being compared within each of the other factor’s levels. However, when there are 3 or more levels being compared (e.g., 3 levels of B within each level of A), Howell’s overall difference approach does not tell us which specific pairs of factors differ significantly within each level of the other factor. In order to fully understand the nature of the interaction, we must use the pairwise comparison approach.

This leads to a final point that this message should cover: How does one obtain pairwise comparisons of interaction category means? I will address this question in relation to the use of SPSS. The menu options in SPSS do not allow for post hoc analyses of the pairwise combinations of interacting factors in factorial ANOVA analyses conducted using the GLM methods. In order to obtain the desired output we need to add a statement to the syntax of the GLM univariate ANOVA command. If we paste the syntax from a simple 2-way between-subjects ANOVA with post hoc tests specified (which the SPSS menu system limits to only the main effects) and the minimum of other options selected, we get the following (assume A = 2 levels, B = 3 levels):

UNIANOVA C BY A B /METHOD=SSTYPE(3) /INTERCEPT=INCLUDE /POSTHOC=B(TUKEY) /CRITERIA=ALPHA(.05) /DESIGN=A B A*B. In order to obtain the results of the post-hoc pairwise comparisons of the interaction categories (i.e., 3 comparisons within each of the 2 levels of A), we need to add the following line to the above syntax: /EMMEANS=TABLES(A*B) compare (B) adj (BONFERRONI) This would result in the following syntax statement: UNIANOVA C BY A B /METHOD=SSTYPE(3) /INTERCEPT=INCLUDE /POSTHOC=B(TUKEY) /EMMEANS=TABLES(A*B) compare (B) adj (BONFERRONI) /CRITERIA=ALPHA(.05) /DESIGN=A B A*B. The resulting output for the post hoc analyses of the interaction categories would look like this (with simulated data):

In the above example of output, I have modified the standard SPSS output to eliminate redundant categories. I have not explored the command syntax requirements necessary to get comparable output from SAS, STATA, or Minitab, but I strongly suspect each has provisions for producing these comparisons. References Howell, D. C. (2009). Statistical methods for psychology, seventh edition. Belmont, CA: Cengage Wadsworth. Meyers, L.S., Gamst, G., & Guarino, A. (2006). Applied multivariate research: Design and interpretation. Thousand Oaks, CA: Sage Publishers.