Modeling both mean and standard deviation to achieve on target results with minimal variation

My colleague Richard Williams just completed a very thorough three-part series of blogs detailing experiment designs aimed at building robustness against external noise factors, internal process variation, and combinations of both. In this follow up, I present another, more simplistic, approach to achieve on target results with minimal variation: Model not only the mean outcome but also the standard deviation. Experimenters making multiple measurements for every run in their design often overlook this opportunity.

For example, consider the paper helicopter experiment done by students of my annual DOE class at South Dakota Mines. The performance of these flying machines depends on paper weight, wing and body dimensions and other easily controlled factors such as putting on a paper clip to stabilize rotation. To dampen down variability in launching and air currents, students are strongly encouraged to drop each of their ‘copters three times and model the means of the flight time and distance from target. I also urge them to analyze the standard deviations of these two measures. Those who do discover that ‘copters without paper clips exhibit significantly higher variability in on-target landings. This can be seen in the interaction plot pictured, which came from a split plot factorial on paper helicopters done by me and colleagues at Stat-Ease (see this detailed here).

Putting on a paper clip dramatically decreased the standard deviation of distance from target for wide bodied ‘copters, but not for narrow bodied ones. Good to know!

When optimizing manufacturing processes via response surface methods, measuring variability as well as the mean response can provide valuable insights. For example, see this paper by me and Pat Whitcomb on Response Surface Methods (RSM) for Peak Process Performance at the Most Robust Operating Conditions for more details. The variability within the sample collection should represent the long-term variability of the process. As few as three per experimental run may be needed with the proper spacing.

By simply capturing the standard deviation, experimenters become enabled to deal with unknown external sources of variation. If the design is an RSM, this does not preempt them from also applying propagation of error (POE) to minimize internal variation transmitted to responses from poorly controlled process factors. However, to provide the greatest assurance for a robust operating system, take one of the more proactive approaches suggested by Richard.