Issue: Volume 4, Number 7
Date: July 2004
From: Mark J. Anderson, Stat-Ease, Inc. (www.statease.com)

Dear Experimenter,

Here's another set of frequently asked questions (FAQs) about doing design of experiments (DOE), plus alerts to timely information and free software updates. If you missed previous DOE FAQ Alerts, please click on the links at the bottom of this page. If you have a question that needs answering, click the Search tab and enter the key words. This finds not only answers from previous Alerts, but also other documents posted to the Stat-Ease web site.

Feel free to forward this newsletter to your colleagues. They can subscribe by going to http://www.statease.com/doealertreg.html. If this newsletter prompts you ask to your own questions about DOE, please address them to stathelp@statease.com.

1. FAQ: The use of factorial design for market research

Question
From: Ohio

"We own a copy of Design-Expert® software, which we want to use for market research. Our goal is to evaluate customer needs for a new product. A classical tool for this purpose is "conjoint" or "trade-off" analysis. I'd appreciate your advice on how to perform such analysis with your software."

Answer:

This can be done via Design-Expert, but you won't see it labeled as "conjoint analysis." It's been a while since I've heard this term, but I recall studying this type of analysis for my MBA (University of Minnesota, 1980). I found the example below via Internet.* It can be easily set up in Design-Expert (or Design-Ease(R) software) via the two-level factorial design feature.

"Conjoint analysis presents choice alternatives between products/services defined by sets of attributes. This is illustrated by the following choice: Would you prefer a flight with regular seats, that costs $400 and takes 5 hours, or a flight which costs $700, has extra-wide seats and takes 3 hours? Extending this, we see that if seat comfort, price and duration are the only relevant attributes, there are potentially eight flight choices:

Choice: Seat-Comfort, Price, Duration
1 extra-wide, $700, 5 hours
2 extra-wide, $700, 3 hours
3 extra-wide, $400, 5 hours
4 extra-wide, $400, 3 hours
5 regular, $700, 5 hours
6 regular, $700, 3 hours
7 regular, $400, 5 hours
8 regular, $400, 3 hours"

Unfortunately no responses are given for this 'thought' experiment, but my choice would be to get a free, frequent-flyer upgrade to the extra-wide seat in first class!

*From http://www.surveysite.com.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

2. Expert-FAQ: Why power statistics look so poor for mixtures

Question
From: Missouri

"I used your Design-Expert software to set up a D-optimal mixture design, but when I performed an evaluation I found that it had no power. Could you tell me what is wrong with the design?"

Answer:
From: Stat-Ease consultant Pat Whitcomb

"Power in a two-level factorial makes perfect sense, i.e., we design to estimate individual effects and test their significance. Since a factorial coefficient is exactly equal to one half the factor effect, power directly measures our ability to do this. However when using response surface methods (process or mixture) we design to estimate a given order polynomial. The emphasis is now on the ability of the design to approximate certain types of behavior (linear, quadratic, etc.) and we are not generally interested in the individual model coefficients. Therefore power is no longer a direct measure of what we are designing for.

For mixtures power calculations are further clouded by collinearity among the coefficients. Recall that in a factorial design the coefficients are orthogonal, i.e., the size of each coefficient is the same regardless of what other coefficients are present in the model. In a mixture this is far from the case, depending on the constraints there is usually extreme dependence. Therefore the power to resolve individual effects is very low, while the ability of the design to adequately model the type of behavior (linear, quadratic, etc.) designed for is high. Unfortunately no one knows how to measure this ability directly.

Thus, power for mixture designs is at best a relative measure for comparing designs over the same mixture space. Even for these direct comparisons the standard error of the coefficients, leverages, determinate and trace are more useful than power."

Pat is working on a technical article that will detail the inherent deficiencies in power for mixture designs. I took the liberty of excerpting key portions of Pat's pending paper and posting it at: http://www.statease.com/pubs/powerinmixsimp.pdf.

(Learn more about power by attending the three-day computer-intensive workshop "Experiment Design Made Easy." See http://www.statease.com/clas_edme.html for a course description. Link from this page to the course outline and schedule. Then, if you like, enroll online.)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

3. Info alert: Stat-Ease featured in cover of "Advance"—a publication for medical laboratory professionals

See the cover of the June 28 issue of "Advance" magazine at: http://makeashorterlink.com/?Z31323AB8. It features an article titled "How Experimental Design Optimizes Assay Automation," co-authored by Stat-Ease consultant Shari Kraber, which describes how "an innovative blend of hardware, software and the right training in statistical know-how supercharges research automation" provided by Beckman Coulter. An unedited version of this article is posted at http://www.statease.com/pubs/optimizeassays.pdf.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

4. Book alert: Recommended summertime reading on statistics

Here's some good summertime reading recommended by Bill Risk—a student of mine at the recent "Experiment Design Made Easy" workshop in San Jose (what a great name for someone interested in statistics for experimenters!): "The Lady Tasting Tea, How Statistics Revolutionized Science in the Twentieth Century." This soft-cover book by David Salsburg is available for purchase at http://makeashorterlink.com/?T11721398. I already had the title story, which was originally told by Sir Ronald Fisher.* As he told it a Lady claims to have the ability to tell which went into her cup first—the tea or the milk. Fisher devised a test whereupon the Lady is presented eight cups in random order, four of which are made one way (tea first) and four the other (milk first). He calculates the odds of correct identification as 1 right way out of 70 possible selections, which falls below the standard 5% probability value generally accepted for statistical significance. Salsburg reveals on good authority (H. Fairfield Smith--a colleague of Fisher) that the Lady identified all eight cups correctly!

Here's another interesting anecdote from Salsburg:
"When I first began to work in the drug industry...one...referred to...uncertainty [as] 'error.' One of the senior executives refused to send such a report to the U.S. Food and Drug Administration [FDA]. 'How can we admit to having error in our data?' he asked [and]...insisted I find some other way to describe it...I contacted H.F. Smith [who] suggested that I call the line 'residual'...I mentioned this to other statisticians...and they began to use it...It seems that no one [in the FDA, at least]...will admit to having error."

*("Mathematics of a Lady Tasting Tea" in Newman’s "The World of Mathematics," Vol. 3, Dover Press, available from http://store.yahoo.com/doverpublications/0486411516.html in paperback.)

PS. A book I recommended last summer, "A Short History of Nearly Everything" by Bill Bryson, just won the Aventis Prize for best science book. The 2004 winner for children's science is a book called "Really Rotten Experiments." That sounds fun! The press releases on these two prize winners and others that won honorable mention are posted at http://news.bbc.co.uk/2/hi/science/nature/3806375.stm.

Original Message
From: Steve Aprahamian

"I just read the May 2004 Stat-Teaser with your daughter's experiment on Coke and Pepsi [posted at
http://www.statease.com/news/news0405.pdf] and I thought I would share an experience I had with the real test done in a mall.

To preface my story, I want to say that, personally, I prefer my soft drinks cold and over ice. I drink them fairly rapidly to keep dilution to a minimum. I also prefer Coke to Pepsi, though I will drink whatever is on sale, my preference for cash overriding the minor preference for taste difference (it is not fine wine we are discussing).

I can tell the difference between Coke and Pepsi, and can also tell "other brands" though I am not always "accurate" in estimation of what the other brand is (RC Cola, Sam's Choice, etc). My kids have taken me up and done blind tests and they haven't stumped me yet on telling the difference between Coke and Pepsi. ("New Coke", really an other, was miscalled as Pepsi on occasion, though I never considered New Coke to be Coke).

I was at the mall a number of years back, when they were doing the "Pepsi Challenge", and I was going to do it and put in my vote for Coke (which is indeed my preference). They put out the two identical cups and I tasted and I noticed that the Coke was (at least to me) noticeably warmer than the Pepsi. I asked: "why is the Coke warmer than the Pepsi?" (My hypothesis being that perhaps this biases the testing, most people I know preferring colder beverages), but they would not answer, they only wanted a response as to what I preferred.

In actuality, I did prefer the cold(er) Pepsi to the cold (yet warmer) Coke (though I would have preferred a cold Coke over a cold Pepsi), but I lied and still chose the Coke over the Pepsi.

I don't know if it was part of the design of the challenge (I would not put it past the designers of the test, since they were trying to get people to "choose Pepsi"), but I did notice it and thought that you might be interested. I could see how this could be a "built-in confounding" to help push a preference to the
"desired response".

I thank you for the sharing of your excellent DOE "experiences". I really like to see the examples of more "everyday" testing that is done as a way of educating. Keep up the good work."

Response:
Thanks for relating another great example of badly executed design (?) of experiments that aliases multiple factors, in your case temperature with cola type.

PS. I am biased to RC Cola because my grandmother always gave it to me and my cousins (our mothers sisters—forbade any soda pop in our respective homes, but they were over-ruled by their mother—our grandmother when on these visits).