Lesson 3: Balanced and orthogonal designs, fitting the model, and center points

Lesson 2 built the 2ᵏ full factorial and showed how to read main effects and interactions off it. This lesson asks why the standard 2ᵏ layout lets you trust those estimates, and what to do once the model is fit. Two structural properties, balance and orthogonality, make a factorial design trustworthy. From there, fitting the model, reading its ANOVA table, checking residuals, and adding center points to test for curvature turn a table of run results into a defensible conclusion about which X's matter.

Balance and orthogonality

A design is balanced when every level of every factor appears the same number of times. In the 2x2 layout above, A is low twice and high twice, and the same is true of B: no level is over-represented.

A design is orthogonal when the coded columns for its factors and interactions are uncorrelated with each other, checked for a two-level design by multiplying corresponding entries and summing: if that dot product is zero for every pair of columns, the design is orthogonal. In the sign table above, the A and B columns are orthogonal to each other, and both are orthogonal to the AB interaction column formed by multiplying them row by row.

Orthogonality lets each effect be estimated independently. In an orthogonal design, A's estimated effect does not shift depending on whether B is in the model, because A and B share no information. In a non-orthogonal design, the estimates become entangled: part of what looks like A's effect may really belong to B. The standard 2ᵏ full factorial is automatically balanced and orthogonal.

Fitting the model and the ANOVA table

Once runs are complete, the data is fit to a model, most simply Y = b0 + b1A + b2B + b12AB, with coefficients equal to one-half the corresponding effect estimates from Lesson 2. Software then produces an ANOVA table partitioning Y's total variation into a piece explained by the model and a piece left as error, exactly as a one-way ANOVA does:

The table's F-ratio (F = MS-model / MS-residual, the same logic as F = MSbetween/MSwithin from Green Belt ANOVA) tests whether the model, or a given term, explains significantly more variation than noise alone would. A small p-value on a term means: reject the null hypothesis that the term has no effect on Y.

Pareto of effects, normal probability plot, and reducing the model

With several factors, a full ANOVA table gets long, so two graphical shortcuts help spot which effects matter:

• A Pareto chart of effects ranks every effect's magnitude largest to smallest against a significance reference line, so the handful that matter stand out from the rest.
• A normal probability plot of effects plots the effect estimates against a normal distribution. Noise-driven effects fall in a straight line near zero; a real effect falls off that line, usually toward one tail.

Once significant effects are identified, the model is reduced by dropping insignificant terms and re-fitting. One rule constrains this: the hierarchy principle. A main effect in a significant interaction is kept even if its own p-value is not significant, because a model that keeps an interaction but drops its parent main effect is hard to interpret and changes if the factors are recoded. Only the reduced, stable model becomes the working equation for Y.

Residual diagnostics

A fitted model is only trustworthy if its residuals, the differences between observed and predicted Y values, behave the way ordinary least squares assumes. Four checks, mirroring Green Belt regression's residual analysis, are standard:

• Normality: a normal probability plot of the residuals should fall roughly on a straight line.
• Constant variance: residuals plotted against predicted values or run order should scatter randomly, with no funnel shape or trend.
• Independence: residuals plotted in run order should show no pattern tied to run sequence, which would suggest an uncontrolled variable drifted through the experiment.
• No unusual points: a residual far larger than the rest deserves investigation before the model is trusted.

For example, if a GCHS team runs a 2² experiment on punch-list rework rate (factors: crew size and inspection timing) and residuals rise steadily across the week the experiment ran, that signals something changed during the week, such as a different inspector, not that the two factors failed to matter.

R-squared, adjusted R-squared, and predicted R-squared

Three statistics summarize model fit. R-squared (R² = SSR/SST, model sum of squares over total sum of squares) is the fraction of variation in Y explained, but it mechanically rises with every added term, even a useless one. Adjusted R-squared corrects for that:

Adjusted R² = 1 - (1 - R²) x (n - 1) / (n - p - 1)

where n is the number of runs and p is the number of model terms (excluding the intercept). Predicted R-squared goes further: it leaves out each run in turn, refits without it, and checks how well the model predicts the left-out value (using the PRESS statistic, prediction error sum of squares). If predicted R-squared is much lower than adjusted R-squared, the model fits these particular runs, not the underlying process, and will predict new runs poorly.

Worked example. A GCHS 2² rework experiment, replicated twice, has n = 8 runs and a model with p = 3 terms (two main effects, one interaction). The ANOVA table gives SST = 100 and model sum of squares SSR = 82, so:

R² = 82 / 100 = 0.82

Adjusted R² = 1 - (1 - 0.82) x (8 - 1) / (8 - 3 - 1) = 1 - 0.18 x 7/4 = 1 - 0.315 = 0.685

The drop from 0.82 to 0.685 reflects the penalty for 3 terms on 8 runs; a predicted R² near 0.68 would mean the model is stable, but one near 0.2 would suggest over-fitting.

Center points: testing for curvature and estimating pure error

A 2ᵏ design only runs each factor at two levels, so alone it cannot tell whether X and Y are truly linear or curve between them. Adding center points, runs with every factor halfway between low and high, coded (0, 0, ...), solves two problems at once:

• It gives a formal test for curvature: if the center points' average is close to the corner runs' average, a straight line adequately fits the range studied. If the center points average meaningfully higher or lower, that is significant curvature, and the team should move to a response surface design with quadratic terms.
• It gives an estimate of pure error, run-to-run noise with no factor changed, since every center point runs under identical conditions, without replicating the whole factorial.

Worked example. A GCHS team adds 3 center points to a 2² design (4 corner runs) on inspection turnaround time. The 4 corner runs average y-bar-F = 45.0 days; the 3 center points average y-bar-C = 51.0 days, with a center-point standard deviation of 1.0 day (2 degrees of freedom). The curvature sum of squares is:

SS-curvature = (nF x nC x (y-bar-F - y-bar-C)²) / (nF + nC) = (4 x 3 x 6.0²) / 7 = 432 / 7 = 61.7

With 1 degree of freedom for curvature and a pure-error mean square of 1.0² = 1.0 (2 degrees of freedom), F = 61.7 / 1.0 = 61.7, far above the F critical value of 18.51 at alpha = 0.05. Curvature is significant: the straight-line model does not fit, and the team should move to a response surface design.