Statistical conclusion validity and type IV errors in rehabilitation research

The incidence of type IV errors was examined in 71 rehabilitation research studies, including a two-way analysis of variance with a statistically significant interaction. The interpretation of the significant interaction was examined to determine whether it qualified as a type IV error. A type IV error was defined as the incorrect interpretation of a correctly rejected null hypothesis. Statistically significant interactions were classified in one of the following categories: (1) correct interpretation, (2) cell mean interpretation, (3) main effect interpretation, or (4) no interpretation. Interpretations classified as cell means or main effects were considered type IV errors. The analysis revealed that 21% of the interactions were correctly interpreted and 63% of the interactions were classified as type IV errors. The remaining 16% of the studies showed a statistically significant interaction but no interpretation. The implications of statistical conclusion validity and type IV errors are discussed, and methods to eliminate or reduce the incidence of type IV errors in rehabilitation research are pressented.