Advanced Evaluation
Aliased Models:
The program first determines whether or not the selected design adequately estimates coefficients for the desired model. If the design provides too few points or has the wrong points, the estimated model terms will be aliased. They cannot be separated in the analysis.
Degrees of Freedom:
A good design provides at least 3 lack-of-fit degrees of freedom and 4 pure error degrees of freedom. Larger degrees of freedom increase the discrimination between adequate and inadequate models.
Responses surface designs, that have multi-linear constraints applied, and mixture designs, which have an equality constraint, should be evaluated using Fraction of Design Space (FDS) found under the Graphs tab of the evaluation node rather than the following matrix correlation indices.
Standard Error and Variance Inflation Factors (VIF):
Scroll down to look at the variance inflation factors (VIF) of the design points.
The ideal VIF is 1.0. VIFs above 10 are cause for concern and VIFs above 100 are cause for alarm, indicating coefficients are poorly estimated due to multicollinearity. If the VIFs get above 1000, and there are no built in extra constraints to the design, then it might not be possible to get a useful model.
The VIF measures how much the variance of that model coefficient increases due to the lack of orthogonality in the design. Specifically the standard error of a model coefficient increases in proportion to the square root of the VIF. If a coefficient is orthogonal to the remaining model terms, its VIF is one. The VIF is related to Ri2 by the following:
VIF = 1.0 / (1-Ri2)
More Advanced Details:
Ri2 is the multiple correlation coefficient (also known as the coefficient of determination). It is calculated by regressing the factor in question on all other factors. Considering the factor in question (Xi) as Y: Ri2 = (SYY - RSS)/SYY where SYY is the sum of squares corrected for the mean of Xi and RSS is the sum of squares residuals regressing Xi on all other factors. If the design is orthogonal these two sums of squares are equal and Ri2 is zero. Since the VIF = 1/(1-Ri2), when Ri2 is zero, the VIF is one. If the design is not orthogonal the quantity (SYY - RSS) is not zero and represents the variation in Xi that can be explained by the settings of the other factors. It is a measure of the collinearity of Xi with the other X’s.
Power Calculations:
Power is the probability of detecting an effect of a specific size. The size of the effect is relative to the standard deviation of the process. More runs will provide more power. Power should only be used to judge the capability of a factorial design.
Leverages of Design Points:
Scroll down to look at the leverages of the design points. High leverage points, those very close to 1.0 or those with values twice that of the average, will influence the model fit. The average leverage equals the number of model terms, including the constant and block coefficients, divided by the number of experiments. In this example the average leverage is 0.55, which equals 11 terms divided by 20 experiments.
At the extreme, a leverage of one indicates that the model will fit that particular point no matter what its value. Obviously this should be avoided.
The leverage of a design point can be decreased by replicating it, or by adding design points in close proximity.
Other matrix measures:
Scroll down to look at various measures of the design matrix. These measurements are used as additional measures of design quality.
The scaled D-optimality criterion allows comparison of designs with different number of runs. The smaller the scaled D-optimal criterion the smaller the volume of the joint confidence interval of the model coefficients (this is good!)
The “condition number of the correlation matrix” indicates the degree of multicollinearity present in the design matrix. If the value equals one, there is no multicollinearity and the design is orthogonal. Values between 10 and 100 indicate a slight to moderately serious problem. Values in the range of 100 to 1000 indicate moderate to severe multicollinearity and over 1000 indicates a severe problem.
The determinant, trace and IV are relative measures used to compare designs having the same number of experiments. They are primarily used for algorithmic point selection.
The balance is a measure of how evenly distributed factors settings are throughout the design.
Correlation of Model Coefficients:
In the ideal orthogonal design, none of the coefficients will be correlated; all non-diagonal values in the matrix will equal 0.