Split-Plot RSM

Introduction

This tutorial illustrates Stat-Ease^® software tools for applying split-plot design to response surface method (RSM) experiments for process optimization.

In many experiments, the levels of one or more factors are hard to change. For example, heating an industrial oven from 300°F to 400°F may take hours. In this case, it will be far easier to group experimental runs by temperature rather than waiting for the oven to heat up and cool down all of the time. A design called a “split-plot” does just this. To get the proper p-values from a split-plot, specialized statistical tools such as restricted maximum likelihood (REML) must be performed. (More on REML later.)

Before getting started, note that throughout this tutorial we provide boxed-up details that are good to know but not essential for working through the program. Bypass these and press ahead if you are in a great hurry.

To illustrate how Stat-Ease designs and analyzes a split plot for RSM, let’s follow a wind tunnel experiment on an aircraft wing—specifically the flap (Kowalski, Parker, and Vining, “Tutorial: Industrial Split-plot Experiments” example from the Quality Engineering, V1, #19, 2007, originally a Box-Behnken design).

If you’ve ever sat in the window seat of an airplane near the wing, the schematic below will look familiar to you.

The two parameters shown, gap and deflection angle, are hard-to-change (HTC) factors. Changing either variable requires opening up the environmentally controlled test chamber. This requires the process to be shut down, which is very costly both in terms of time and expense. A split-plot design minimizes the number of times these HTC factors must be changed, i.e., it restricts their randomization. Two other parameters to be tested are the angle-of-attack and the Reynold’s number, both of which can be changed remotely from outside the chamber, making them easy-to-change (ETC) factors.

Design the Experiment

Click the blank-sheet icon on your toolbar or click New Design. On the left, under Standard Designs, click on Response Surface to expand its tree, then look under the Split-Plot section and select Central Composite (CCD). Enter 4 as the number of factors.

Enter Gap as the name for Factor a. Notice that this factor is already labeled as HTC since a split-plot design was chosen. Click twice on Factor B and enter Deflection as the second factor. Tab to the Change column and type h to change this factor to Hard. You could also click on the droplist and change it, but typing is a nice keyboard shortcut for those who like them.

There is no need to enter the low and high levels of the factors because the authors of this example provided only the coded (-1 to +1) levels—not the actuals, perhaps because these needed to remain proprietary. Press ahead to Factor C and enter Angle as the name. Then, enter Reynolds as the Factor D name. Since these are ETC factors, leave the change column at the default of Easy.

Click on the Options button near the bottom and middle of the screen and change the Alpha value to Practical.

This value determines what levels the axial (star) points in the CCD will be run at. Alpha is the distance (in coded units) from the center of the design.

Note

The makeup of a split-plot CCD: The three-factor layout for this CCD is pictured below. It is composed of a core factorial that forms a cube with sides that are two coded units in length (from -1 to +1 as noted in the low and high levels above). The stars represent axial points. How far out from the center these points should go is a matter for much discussion between statisticians. They designate this distance “alpha” – measured in terms of coded factor levels. As you see, Stat-Ease offers a variety of options for alpha. The default “rotatable” alpha gives the best distribution of the standard error (uncertainty) in your predictions, so it is desirable, though not completely necessary. The rotatable alpha requires the star points to be farther from the center (2.0 in coded units in this case), so it is not always feasible. If the levels for the star points are too far away from the center, the experiment may not produce measurable results. For this reason, the practical alpha can be used, which pulls the star points in a little bit to make sure the experiments are able to be run and produce meaningful results. Practical Alpha is recommended for more than 5 factors (k > 5), since the rotatable alpha becomes unwieldy when you get that many factors.

In addition to the alpha levels, in a split-plot Central Composite Design, there is also the complication of groups. At the bottom of the page, near the options button, you will see how this design was broken up into groups.

For each type of point, it shows the number of groups and the number of runs per group. These can also be adjusted in the Options dialogue box, but we’ll keep them at the defaults for this example.

Press Next. For the R1 Name, enter Lift as the response.

Press Finish to complete the design-building wizard. You will get a warning to reset the factor levels between groups. Just click OK to bypass this warning, for now. The program now displays, in random run order grouped by Gap (Factor a) and Deflection (Factor b), the runs needed to complete the experiment. Your runs will most likely be different due to the randomizing (where not restricted) of the design. Also, note that the convention that has been adopted is that HTC factors will be denoted with lower case letters, as seen in the column header. This sets them apart from the ETC factors, shown with upper case letters.

The Group column has been added to keep track of HTC factor combinations, which form the “whole plots”. Note that the groups are separated by a horizontal black line. All of the runs in any given group maintain a constant level of the HTC factors. For example, in group 1, Gap is set to 0 and Deflection is set to 1.41421 for both runs. For all four of the runs in Group 2, the gap is set to the mid-level (0) and factor B is also at the mid-level (0).

To reduce the number of significant digits, right-click on the column header for Deflection and choose Edit Info. Click on the Format droplist and choose 0.00 and you will be left with 2 decimal points. Do this for all four of the factors to make the design cleaner to look at and run.

The grouping of HTC factors makes the experiment much easier to run. As noted in the Introduction, the sealed experimental chamber would have to be opened every run to change the Gap and Deflection angle. With the split-plot design, this disruption occurs only between the groups. Along the way, the aeronautic engineers sometimes go as many as four runs without having to take the time-consuming step of opening, resealing and, finally, re-pressurizing the chamber. One thing to remember, however, is that all the factors should be reset between groups, even if the HTC factor levels don’t change. You may recall the warning that we clicked OK on earlier, alluding to this. For instance, in groups 2 and 3 in this design, factor “a” and “b” stay constant at 0. To capture the variation that really occurs between groups, the experimenter should open the chamber and adjust these factors to some nominal value and back to 0 before running the third group. That way, the noise caused by resetting these levels is properly accounted for.

Analyze the Results

To avoid potential data-entry errors and ensure you get the same layout shown, read in the data by going to Help, Tutorial Data and selecting Airfoil.

Click on the Design node on the left. You should now see the data for the Lift Response as depicted in the screenshot. Only the first 22 of the 37 runs are shown, to save space.

The response has been multiplied by a factor of 10,000 to convert the lift coefficient to integers for the most part. Positive values indicate an increase in lift relative to average and negative values indicate a decrease.

To get started with the analysis, click the node labeled R1: Lift under the Analysis branch and press the Start Analysis button. As with a normal RSM analysis, a new set of tabs appears at the top of your screen, arranged from left to right in the order needed to complete the analysis.

There are a variety of Transforms that can be applied on this page. Not knowing if they will help at this point, click ahead to the Model tab. There are diagnostics that are checked later that can determine if a transform will help.

On the model tab, the quadratic model is presented for consideration (denoted by the green “” by each term from the linear terms to the squared terms). Clicking ahead to the ANOVA (REML) screen at this point will evaluate the full quadratic model. However, it’s best to do some analysis to select the best model from among the quadratic terms. To allow the computer to do this automatically, click on the Auto Select… button. For Criterion select AICc and for Selection choose Forward (if not already selected) and click the Start button to run the analysis. The program will go through the terms in the quadratic model and select which ones improve the AICc criterion the most and add them to the model one at a time until adding terms will no longer improve the criterion.

The program shows you the terms added in selecting the model, showing AICc criterion for each step. Click the Help button for more details on algorithmic model selection and the criterion used. Otherwise, click Accept to continue and evaluate the resulting model.

The model statistics are shown by clicking on the ANOVA (REML) tab. You will get a warning that the model you have selected is not hierarchical. Be sure to click Yes to correct for hierarchy. This will give you a more statistically sound model, ensuring lower order terms (in this case b) are present to support higher order terms (like ab), even if they are insignificant. This is good statistical practice. Click on the help button in the warning box for more information.

This is not your standard ANOVA analysis, which relies on randomization for validity. The analysis done for split-plot designs is a form of maximum likelihood estimation, more specifically, restricted maximum likelihood (REML), as noted at the top of the results table.

The big difference between the statistics on this table and a normal ANOVA is the grouping of variance terms into a Whole-plot section for the HTC factors and a subplot section for ETC factors. Look at the p-value (Prob>F) for the whole-plot. It is highly significant with a p-value much less than the generally acceptable 0.05 alpha level. That is for all the terms making up the whole-plot (HTC) portion of the model. You can also look at the p-values for the individual whole-plot terms, which look fine except for the b-Deflection factor, which has a p-value of 0.28. Remember, though, that this term was included for hierarchy to support the significant ab interaction.

The subplot terms as a whole are significant as are all the individual subplot terms. Next, click the Variance Components tab (location will vary based on your pane layout) to see various statistics to augment the REML analysis.

Here, you will see more details on the variance components and likelihood ratios for the selected model, including the information criteria (AIC, BIC, and AICc) that score how good the model is. Just remember that when comparing models with these criteria, lower numbers are better, and you should only compare models from within one design that are nested to one another. For example, the model A+B can be compared to the model A+B+AB as all the terms in the larger model are also in the smaller model.

Another good statistic to study is the Adj. (adjusted) R-squared. This number goes from 0 to 1, with 1 being the best. In this case, the 0.99 Adj. R-Squared shows that we’ve captured most of the variation in the data (~99%) with the selected model.

That’s enough on the model statistics. It seems we have come up with quite a strong model. The next tab over is the “Diagnostics” tab, but those have been covered extensively in other tutorials. For instance, see the Response Surface tutorial. The diagnostics used in a standard RSM design are calculated and can be applied here. They all pass with flying colors in this, case so we’ll move on. The focus of this tutorial is on the differences caused by this being a split-plot and the experimental results achieved with less effort.

Jump ahead to the Model Graphs to explore graphical presentations of the predictions stemming from the selected model.

The first contour plot leaves a lot to be desired. The color scales being used are not highlighting anything, with the whole plot being green. Normally going “green” is a good thing, but some color variation will make this plot far more useful. Do this via a right-click on the color scale in the legend to pull up the Edit Gradient Range dialogue box. Change the low value to -400, which is one of the lowest contour values on the graph, and the high value to 200, the highest contour on the graph.

This gives a much more colorful (and thus informative) plot, which highlights the higher values of Lift achieved in the upper left corner at low values of Gap and high values of Deflection.

To see if these trends hold elsewhere and how Factor C (Angle) and Factor D (Reynolds #) affect the results, use numerical optimization. Click on the Numerical node under the Optimization branch. Click on Lift and set a goal to Maximize with an Upper limit of 3000.

When working with quadratic models, it’s good to put in a little stretch goal like this that exceeds the values achieved in the experiment. This ensures achieving the highest possible value.

To see the optimal solutions, click on the Solutions tab.

The best result occurs at a low level of Gap (a) with high Deflection (b), high Angle (C), and high Reynolds (D). This agrees with what we saw on the contour plot between Gap (a) and Deflection (b). With these settings, a Lift of 2667.9 is achieved. All of the other solutions, which can be explored by clicking on the other numbers across the top, give less Lift.

To see what the contour plot looks like at this recommended number 1 solution, click on the Graphs tab. Note that solution 1 is automatically selected in the solutions drop-down menu on the right. By default, All Responses is selected on the Response droplist. This shows the Desirability plot, which is used to find the optimum, alongside the Lift plot. To focus on the Lift plot, click on the Response droplist and select Lift. Now, there is again little stratification of colors on the plot, so right-click on the color scale and change the low to 1500 and high to 2500. This results in a better plot, highlighting the region where the high lift optimum is achieved in red. The program also plants a flag at the optimal solution (the upper left corner) found by the numerical optimization.

For a better view of the response surface, click on the 3D Surface plot from the Graphs toolbar and then click and drag the graph to a new orientation with the optimum solution in the back to better highlight the slope of the surface.

Summary and Conclusion

In this experiment, there were HTC factors (gap and deflection) that required the costly step of opening the environmental chamber. By using a split-plot design, the number of times the chamber has to be opened is reduced by sorting the design into whole-plot groups. Since randomization was restricted by sorting, a REML analysis was then applied to get the proper p-values for the model. Numerical optimization then displayed the conditions that will optimize the Lift of an airplane wing. Using a split-plot design allowed the experimenters to save time and money and run a design which might well have been infeasible if fully randomized. If you ever find yourself tempted to sort your randomized design to make it easier to run, try a split-plot design instead. This will give you the proper p-values and do the sorting for you.

Split-plot designs are the epitome of efficiency when it comes to the experimental effort needed. Just be aware that the easier experimenting comes at a cost: less precision and power in determining the HTC factor’s effects. Be sure to use power or fraction of design space (FDS) sizing tools to make sure you have enough experimental runs to overcome the loss in power and precision that come with restrictions in randomization. These tools are discussed in other tutorials and in webinars you can find here: https://statease.com/training/webinar.html. Then, just follow the steps outlined above to design and analyze your experiment.