Tip of the Week: What ANOM Does that ANOVA Cannot

In the Tip of the Week for October 27, 2016, we listed a number of things that ANOVA can and can't do. One of these was that ANOVA can tell us whether or not there is a Statistically Significant difference among several Means, but it cannot tell us which ones are different from the others to a Statistically Significant amount.    
​
Let's say we're comparing 3 Groups (Populations or Processes) from which we've taken Samples of data.
 
ANOM calculates the Overall Mean of all the data from all Samples, and then it measures the variation of each Group Mean from that. In the conceptual diagram below, each Sample is depicted by a Normal curve. The distance between each Sample Mean and the Overall Mean is identified as a "variation". 

ANOM retains the identity of the source of each of these variations (#1, #2, and #3), and it displays this graphically in an ANOM chart like the one below. In this ANOM chart, we are comparing the defect rates in a Process at  7 manufacturing plants. So, there are 7 variations that are being compared.

The dotted horizontal lines, the Upper Decision Line, UDL and Lower Decision Line, LDL, define a Confidence Interval, in this case, for α = 0.05. Our conclusion is that   Eastpointe (on the low side) and Saginaw (on the high side) exhibit a Statistically Significant difference in their Mean defect rates. So ANOM tells us not only whether any plants are Significantly different, but also which ones are.
 
In ANOVA, however, the individual identities of the Groups are lost during the calculations. 

The 3 individual variations Between the individual Means and the Overall Mean are summarized into one Statistic, MSB, the Mean Sum of Squares Between. And the 3 variations Within each Group are summarized into another Statistic, MSW, the Mean Sum of Squares Within. So, there is no way to identify individual variations, we just have Means of the variations from the individual groups. 

Here's a summary of the calculations in ANOVA:

• The formulas for MSB and MSW are specific implementations of the generic formula for Variance.
• So, MSB divided by MSW is the ratio of two Variances.
• The Test Statistic F is the ratio of two Variances.
• ANOVA uses an F-Test (F = MSB/MSW) to come to a conclusion.
• If F ≥ F-Critical, then we conclude that the Mean(s) of one or more Groups have a Statistically Significant difference from the others.

This makes sense, because If F ≥ F-Critical, then F must be relatively large. So, the variation of the Groups from the Overall Mean (the SSBs) must be substantially larger than the variation within each group (the SSWs). This would happen if there were a Statistically Significant difference between the individual Means.