Stat-Teaser September 2014

---

Guest author Sebastian Hoffmeister, STATCON (Germany), with cupcakes at the 5th European DOE User Meeting

Split-plot designs are probably the most important new feature in Design-Expert^® software, version 9 (DX9). In this article we will be exploring the art of baking cupcakes as one example of the numerous use cases for this specific type of design. The first section will describe typical applications and the basic terminology. This is followed by a section illustrating the need for split-plot designs from a statistical point of view. The last section will discuss the process of setting up and analyzing a full-factorial split-plot design.

My experience as a trainer in the field of DOE confirms Daniel Cuthbert’s sentence: “Every industrial experiment is a split-plot experiment.”  There is nearly no basic-level DOE training that does not end in a discussion about the independence of observations and “hard-to-change-factors”.  Since we have version 9 of Design-Expert software available, I am able to quickly end any discussion with a nice example illustrating the problem that split-plot designs solve and how this is done in Design-Expert!

The Cupcake Experiment

Let’s imagine we are working for a cupcake company. Part of the job is to optimize the production process. Our primary goal is to maximize the volume of the cupcakes produced with a given amount of dough. A restriction is that we are not able to control the basic formulation of the dough (no mixtures here!). We’ll have to solve the problem only by changing the baking process—its temperature and time. Additionally, we can choose between adding chocolate chips to the cupcakes or not.

Let us have a quick look at those factors and what I call the “data collection protocol”. The first step when doing one experiment here is of course to take the dough and put it into a baking pan. Potentially chocolate chips are added to the dough before it is put into the pre-heated oven where the cupcakes will be baked at a specified temperature for a specified time.

Now here is the important part of the experiment: the baking pan. Typically baking pans for cupcakes have six cups and allow us to produce six cupcakes at a time. The problem here, of course, is that those six cupcakes have to be baked at the same temperature and time settings. Now this is where split-plot designs become beneficial.  While classic DOEs use complete randomization (i.e. a complete reset of all factors between each run), split-plot designs allow you to use non-independent experiments.

In this context we will call the temperature and the baking time hard-to-change factors, as we cannot randomize the whole design over these two factors. Contrary to this, chocolate chips are an easy-to-change factor, as there is no restriction on randomization concerning this third factor. We are able to freely decide for each cup in the baking pan, if we want to add chocolate chips or not.

Why Independence is Relevant

The previously described scenario leaves us with groups of experiments. We will be baking groups of six cupcakes at a time. Each of the groups will be baked at one temperature and one baking time. But an individual cupcake inside of a group might have chocolate chips or it might not. In statistics we call these groups whole plots and each single cupcake a subplot.

Now why is it important that there are these groups of experiments? It is important, because most statistical methods (including least squares regression and ANOVA) assume that your observations are independent of each other. Be aware that the problem in this case is not that we did a couple of cupcakes in a row with same temperature and time. The problem is that we made them all together!

If you do a couple of experiments with the same factor settings there is no problem with that, as long as you perform those experiments independently from each other. For us this would involve cooling down the oven to room temperature after each individual cupcake was made (a full reset of process parameters)! 

For economic reasons, this is clearly not the way to go. So the alternative is to tell Design-Expert that these groups of runs are not made independently. Then Design-Expert is able to choose an adequate statistical method for the analysis (this happens to be mixed models). If you don’t do this you will underestimate some kind of variation, as DX9 will treat the cupcakes inside of each group as being independent. But as they are made all together they will be more similar than we could expect from really independent cupcakes! This will lead to an increased Type 1 error rate for the effects of temperature and time.

How to Do it Right

In the screening section of DX9 you will find the option to set up Split-Plot Regular Two-Level Factorial Designs (see Figure 1). This is what we need to do here. We are setting up a full-factorial design for the three factors—two of them being hard to change. As we are baking six cupcakes for each combination of time and temperature, we will use 6 replicates. This automatically creates replicates for the chocolate chip factor, too. We will have to remove those replicates manually at the end.

Figure 1

 Entering the factors is easy to do (see Figure 2).

Figure 2

The resulting plan is shown in Figure 3. We then used DX9 design editing tools to modify the layout into groups of six cupcakes.

Figure 3

After gathering the data you will see that the analysis is very similar to what you are used to. The only change is that there are two tabs with half-normal graphs that will help you to graphically analyze the effect of whole-plot effects (temperature and time) and subplot effects (chocolate chips). (See Figures 4a & 4b.)

Left: Figure 4a–whole plot. Right: Figure 4b–subplot

This is needed because we are using different grades of information when estimating whole-plot and subplot effects. As we did not completely randomize over the whole-plot factors we have less information about those factors compared to the information about the subplot factors. You will see that when analyzing the standard errors in the ANOVA tab, too.

Figure 5

The standard error of the whole-plot effects is much larger (0.27) than the standard deviation of the subplot effect (0.075). (See Figure 5 above.)

All in all the reports and graphs for analyzing split-plot designs are very similar to those of classical regression models (see Figure 6).

Figure 6 (Note the low p-value for the effect of chocolate chips!)

It is now easy to see that chocolate chips are the crucial factor for maximizing the volume of the cupcakes. Sadly for those with a sweet tooth, the experiment favors elimination of the chocolate chips.  Luckily this whole scenario is imaginary, though, thus hope remains for you chocolate lovers to discover differently.

Happy baking!

—Sebastian Hoffmeister

↑Go back to the top

---

Not listening in on what’s being said behind your back—DOE fine-tunes hearing aids

Tom Burns, Starkey Labs, Inc.

Have you ever happened to listen in on someone saying stuff about you that you’d really rather not have heard? Awkward! Until the development of directional microphone arrays in the 1970’s, those equipped with assistive listening devices suffered this undesirable side effect. Hearing-aid manufacturers continue to fine-tune control of sound on a directional basis to improve speech intelligibility, that is, the signal-to-noise ratio (SNR).

Now the power of design of experiments (DOE) in the hands of expert engineers paves the way to ever better and more robust setups for medical devices benefitting the hearing impaired. At our 5th European DOE User Meeting this past summer in Cambridge, England I learned all about a successful application of DOE (and got a great education on hearing loss to boot!) by Design-Expert® software (DX) user, Thomas Burns, PhD, an Engineering Principal with Starkey Hearing Technologies, Inc.

Tom’s story started with a major categorical design choice between these omnidirectional sound-detection arrays:

T1. Two microphones in a first-order endfire configuration (most common)
T2. Three microphones in a second-order endfire (less common)
T3. A dual dipole omni (DDO) with promise for improved SNR together with greater robustness

A Google search on technical terms such as “second-order” brought up a page from Chapter 6 on “High Directionality Microphones” by John Eargle in his 2004 Microphone Book that left me only slightly more knowledgeable than before; thus I will refer to these three hearing-aid configurations simply as T1, T2 and T3.  What really caught my attention was Tom’s deployment of a measurement manikin that rotated around in a field of sound to assess in situ the sensitivities of the various microphones operating over a range of levels, at differing positional placements on the head in ten degree resolution.  Using response surface methods (RSM) aided by DX, Tom determined how each of a number of configurations responded to the various environmental factors; thereby predicting which microphone array provided the most robust directionality.  As you can see in his talk,* he illustrated his results using polar patterns in 2D and colored 3D that make it easy to see the results.**

After an impressive amount of work collecting data from optimal experiment designs on each configuration and putting most of the statistical analysis, modeling and multiple response optimization tools for DX to good use, Tom confirmed T3 as the best and most robust of the bunch and he came up with the optimal directional axis angle for in situ performance.  That sounded very good to me—a message received loud and clear: For best results equip your top engineers and scientists with the knowledge and tools of DOE.

—Mark J. Anderson

*See Tom’s detailing of “robust design of directional microphone arrays for hearing aids” via this link.

**According to the “Polar Plot” primer posted at www.audiologyonline.com/ask-the-experts/polar-plots-588, “if a hearing aid wearer is listening to speech from the front and the noise is directly behind them, then the cardioid pattern [one of four pictured] would provide the best audibility because the most attenuation occurs directly behind the listener (180 degrees).”

↑Go back to the top

---

The 5th European DOE User Meeting in Cambridge, UK—a recap

Downing College, Cambridge, UK

In early July Stat-Ease, Inc., together with their partner in the UK, PRISMTC, held the 5th European DOE User Meeting at Downing College in Cambridge, UK.  Attendees came from around Europe, as well as from North and South America and China, to increase their knowledge of DOE, share their experiences, network, and enjoy the peaceful and beautiful surroundings in this historic city. 

On the first day of the three-day conference two optional workshop tracks were offered including: Real-Life Experiment Design Made Easy, and Robust Design and Tolerance Analysis to Improve Decisions and Mitigate Risk.  During the next two days, speakers from a variety of industries presented case studies on their own applications of DOE.  There were also DOE talks on split plots, mixture design, DOE in the context of measurement systems qualification (MSA), and more.  Dr. Paul Nelson and Andrew Macpherson's presentation on Acquiring a Taste for Mixtures was a fun interactive event which involved beer tasting!

Left: Pre-meeting workshop. Right: Pat Whitcomb speaking on split-plot designs

A highlight of the conference was a scientific walking tour of Cambridge followed by a candlelit dinner in the 16th century hall of historic Magdalene College.


Left: Tour guide showing where Isaac Newton used to work, with an offshoot from his famous apple tree. Right: Magdelene College

The 5th European DOE User Meeting was interesting, informative, and enjoyed by all.  The next user meeting will be held in 2016, location to be announced.  Plan now to attend!

↑Go back to the top

Workshop schedule

Experiment Design Made Easy (EDME)
Dec 9-10: Minneapolis, MN
Mar 24-25: Minneapolis, MN
May 13-14: San Diego, CA
$1295 ($1095 each, 2 or more)

Factorial Split-Plot Designs for Hard-to-Change Factors (FSPD—Half Day)
May 15: San Diego, CA
$395 ($295 each, 2 or more)

Response Surface Methods for Process Optimization (RSM)
Dec 11-12: Minneapolis, MN
Mar 26-27: Minneapolis, MN
$1295 ($1095 each, 2 or more)

Mixture Design for Optimal Formulations (MIX)
Nov 4-5: Minneapolis, MN
Mar 9-10: Edison, NJ
May 5-6: Minneapolis, MN
$1295 ($1095 each, 2 or more)

Advanced Formulations: Combining Mixture & Process Variables (MIX2)
Nov 6-7: Minneapolis, MN
Mar 11-12: Edison, NJ
$1495 ($1195 each, 2 or more)

PreDOE: Basic Statistics for Experimenters Online Course
Free (a $95 value). Learn more here.

Workshops are limited to 16. Receive a $200 discount per class when you enroll 2 or more students or enroll in consecutive 2-day workshops. Receive a $100 discount for enrolling in the FSPD workshop along with another class.  

For more information, contact Rachel at 612.746.2038 or via e-mail.

↑Go back to the top