Stat-Ease Blog

So, I’ve gotten thru some of the basics (Greg's DOE Adventure: Important Statistical Concepts behind DOE and Greg’s DOE Adventure - Simple Comparisons). These are the ‘building blocks’ of design of experiments (DOE). However, I haven’t explored actual DOE. I start today with factorial design.

In this case, the horizontal line (x-axis) is time and vertical line (y-axis) is temperature. The area in the box formed is called the Experimental Space. Each corner of this box is labeled as follows:

1 – low time, low temperature (resulting in crunchy, matchstick-like pasta), which can be coded minus-minus (-,-)

2 – high time, low temperature (+,-)

3 – low time, high temperature (-,+)

4 – high time, high temperature (making a mushy mass of nasty) (+,+)

One takeaway at this point is that when a test is run at each point above, we have 2 results for each level of each factor (i.e. 2 tests at low time, 2 tests at high time). In factorial design, the estimates of the effects (that the factors have on the results) is based on the average of these two points; increasing the statistical power of the experiment.

If we look at the same experiment from the perspective of altering just one factor at a time (OFAT), things change a bit. In OFAT, we start at point #1 (low time, low temp) just like in the Factorial model we just outlined (illustrated below).

Here, we go from point #1 to #2 by lengthening the time in the water. Then we would go from #1 to #3 by changing the temperature. See how this limits the number of data points we have? To get the same power as the Factorial design, the experimenter will have to make 6 different tests (2 runs at each point) in order to get the same power in the experiment.

After seeing these results of Factorial Design vs OFAT, you may be wondering why OFAT is still used. First of all, OFAT is what we are taught from a young age in most science classes. It’s easy for us, as humans, to comprehend. When multiple factors are changed at the same time, we don’t process that information too well. The advantage these days is that we live in a computerized world. A computer running software like Design-Expert®, can break it all down by doing the math for us and helping us visualize the results.

Additionally, with the factorial design, because we have results from all 4 corners of the design space, we have a good idea what is happening in the upper right-hand area of the map. This allows us to look for interactions between factors.

That is my introduction to Factorial Design. I will be looking at more of the statistical end of this method in the next post or two. I’ll try to dive in a little deeper to get a better understanding of the method.

Expand your design of experiments (DOE) expertise at the 2019 Fall Technical Conference.

Martin Bezener will be teaching a pre-conference short course on Practical DOE and giving a session talk on binary data (abstracts are below). Shari Kraber will be hosting our exhibit booth and can discuss your DOE needs.

Short Course Abstract (Wed, Sep 25): Practical DOE: ‘Tricks of the Trade’

In this dynamic short-course, Stat-Ease consultant Martin Bezener reveals DOE tricks of the trade that make the most from statistical design and analysis of experiments. Come and learn many secrets for design of experiment (DOE) success, such as:

• How to build irregularly-shaped DOE spaces that cover your region of interest
• Using logistic regression to get the most from binomial data such as a pass/fail response
• Clever tweaks to numerical optimization
• Cool tools for augmenting a completed experiment to capture a peak beyond its reach
• Other valuable tips and tricks as time allows and interest expressed

Session 6A Abstract (Fri, Sep 27, 1:30pm): Practical Considerations in the Design of Experiments for Binary Data

Binary data is very common in experimental work. In some situations, a continuous response is not possible to measure. While the analysis of binary data is a well-developed field with an abundance of tools, design of experiments (DOE) for binary data has received little attention, especially in practical aspects that are most useful to experimenters. Most of the work in our experience has been too theoretical to put into practice. Many of the well-established designs that assume a continuous response don’t work well for binary data yet are often used for teaching and consulting purposes. In this talk, I will briefly motivate the problem with some real-life examples we’ve seen in our consulting work. I will then provide a review of the work that has been done up to this point. Then I will explain some outstanding open problems and propose some solutions. Some simulation results and a case study will conclude the talk.

Be sure to mark these as must attend events on your FTC schedule!

Hurry – Early Bird Registration ends Aug 25. Hotel block ends Aug 27. Final Registration ends September 13.

Links for more info:

www.falltechnicalconference.org

We recently published the July-August edition of The DOE FAQ Alert. One of the items in that publication was the question below, and it's too interesting not to share here as well.

Original question from a Research Scientist:

"Empowered by the Stat-Ease class on mixture DOE and the use of Design-Expert, I have put these tools to good use for the past couple of years. However, I am having to more and more defend why a mixture design is more appropriate than factorials or response surface methods when experimenting on formulations. Do you have any resources, blogs posts, or real-world data that would better articulate why trying to use a full factorial or central composite design on mixture components is not the most effective option?"

Answer from Stat-Ease Consultant Martin Bezener:

“First, I assume you are talking about factorials or response surface method (RSM) designs involving the proportions of the components. It makes no sense to use a factorial or RSM if you are dealing with amounts, since doubling the amount of everything should not affect the response, but it will in a factorial or response-surface model.

"There are some major issues with factorial designs. For one thing, the upper bounds of all the components need to sum to less than 1. For example, let’s say you experimented on three components with the following ranges:

A. X1: 10 - 20%
B. X2: 5 - 6%
C. X3: 10 - 90%

then the full-factorial design would lay out a run at all-maximum levels, which makes no sense as that gives a total of 116% (20+6+90). Oftentimes people get away with this because there is a filler component (like water) that takes the formulation to a fixed total of 100%, but this doesn't always happen.

"Also, a factorial design will only consider the extreme combinations (lows/highs) of the mixture. So, you'll get tons of vertices but no points in the interior of the space. This is a waste of resources, since a factorial design doesn't allow fitting anything beyond an interaction model.

"An RSM design can be ‘crammed’ into mixture space to allow curvature fits, but this is generally a very poor design choice. Using ratios of components provides a work-around, but that has its own problems.

"Whenever you try to make the problem fit the design (rather than the other way around), you lose valuable information. A very nice illustration of this was provided in the by Mark Anderson in his article on the “Peril of Parts & the Failure of Fillers as Excuses to Dodge Mixture Design” in the May 2013 Stat-Teaser.”

An addendum from Mark Anderson, Principal of Stat-Ease and author of The DOE FAQ Alert:

"The 'problems' Martin refers to for using ratios (tedious math!) are detailed in RSM Simplified Chapter 11: 'Applying RSM to Mixtures'. You can learn more about this book and the others in the Simplified series ('DOE' and 'Formulation') on our website."

Links for additional information:

• Back issues of The DOE FAQ Alert
• Signup to receive the bimonthly DOE FAQ Alert via email
• Learn more about mixture design by attending our computer-intensive workshop Mixture Design for Optimal Formulations

As I learn about design of experiments, it’s natural to start with simple concepts; such as an experiment where one of the inputs is changed to see if the output changes. That seems simple enough.

For example, let’s say you know from historical data that if 100 children brush their teeth with a certain toothpaste for six months, 10 will have cavities. What happens when you change the toothpaste? Does the number with cavities go up, down, or stay the same? That is a simple comparative experiment.

“Well then,” I say, “if you change the toothpaste and 6 months later 9 children have cavities, then that’s an improvement.”

Not so fast, I’m told. I’ve already forgotten about that thing called variability that I defined in my last post. Great.

In that first example, where 10 kids got cavities. That result comes from that particular sample of 100 kids. A different sample of 100 kids may produce an outcome of 9, other times it’s 11. There is some variability in there. It’s not 10 every time.

[Note: You can and should remove as much variability as you can. Make sure the children brush their teeth twice a day. Make sure it’s for exactly 2 minutes each time. But there is still going to be some variation in the experiment. Some kids are just more prone to cavities than others.]

How do you know when your observed differences are due to the changes to the inputs, and not from the variation?

It’s called the F-Test.

I’ve seen it written as:

Where:

s = standard deviation

s2 = variance

n = sample size

y = response

ӯ (“y bar”) = average response

In essence, this is the amount of variance for individual observations in the new experiment (multiplied by the number of observations) divided by the total variation in the experiment.

Now that, by itself, does not mean much to me (see disclaimer above!). But I was told to think of it as the ratio of signal to noise. The top part of that equation is the amount of signal you are getting from the new condition; it’s the amount of change you are seeing from the established mean with the change you made (new toothpaste). The bottom part is the total variation you see in all your data. So, the way I’m interpreting this F-Test is (again, see disclaimer above): measuring the amount of change you see versus the amount of change that is naturally there.

If that ratio is equal to 1, more than likely there is no difference between the two. In our example, changing the toothpaste probably makes no difference in the number of cavities.

As the F-value goes up, then we start to see differences that can likely be credited to the new toothpaste. The higher the value of the F-test, the less likely it is that we are seeing that difference by chance and the more likely it is due to the change in the input (toothpaste).

Trivia

Question: Why is this thing called an F-Test?

Answer: It is named after Sir Ronald Fisher. He was a geneticist who developed this test while working on some agricultural experiments. “Gee Greg, what kind of experiments?”. He was looking at how different kinds of manure effected the growth of potatoes. Yup. How “Peculiar Poop Promotes Potato Plants”. At least that would have been my title for the research.

We recently wrapped up the 2019 Analytics Solutions Conference in Minneapolis, MN. This was the first time we organized a US conference on analytics with our Norwegian partners, Camo Analytics. It could not have gone better! A BIG congratulations on a job well done to Shari Kraber who spearheaded the organization of this conference.

The goal of the conference was to help attendees transform their business, from R&D to production, using data-driven tools. If I may toot our own horn here, we did that. There was a lot of information to digest at this meeting, but the camaraderie of attendees made it easy. We overheard many conversations in the halls between attendees about what they just learned or discussing potential solutions to business problems they were facing.

The conference started with a short course day. Two all day seminars were conducted in parallel, one titled “Practical DOE: ‘Tricks of the Trade’” and the other “Realizing Industry 4.0 through Industrial Analytics”. We saw a lot of people experienced with design of experiments (DOE) go to the Industry 4.0 course and a lot of the analytics pros headed to learn about DOE. What a great way to start!

Day one wrapped up with an opening reception for all attendees and speakers, fostering networking and building relationships.

The conference days were filled with session talks that wrapped around keynote presentations (a fantastic trio of speakers, find out more at the link below). The result was a great mix of academic-style overviews with down-to-earth, real-world case studies. It struck a perfect balance.

If you couldn’t make it to the conference, details on each presentation (including pdf’s of the presentations), are posted on the ASC speakers page.

To add some fun to the mix, the evening’s social event was a dinner cruise on the Mississippi River for all attendees. It was quite the treat, especially to those of us who live here! We rarely take the opportunity to enjoy activities like this in our own backyard. We traveled thru a lock, spotted a bald eagle in the trees, and enjoyed the casual atmosphere.

The next Stat-Ease hosted conference will be our 8th European DOE Users Conference. This year it will be held in Groningen, Netherlands from June 17-19. We hope to see you there!

If you would like to be kept in the loop about the conference, sign up for our mailing list. We will be sending out information regarding a call for speakers and registration later in the fall. Sign up now, before you forget!

See you in Groningen!