Lesson 1: Designed experiments

Green Belt closed with correlation, regression, and control charts, all tools that observe X and Y as they naturally occur. Black Belt opens with a different move: deliberately changing X to see what happens to Y. The IASSC Certified Lean Six Sigma Black Belt exam is 150 graded questions (some forms add up to 15 ungraded ones), closed book, with a 4 hour time limit and a 70 percent score required to pass, at roughly 30 questions from each DMAIC phase. What Black Belt adds beyond Green Belt is concentrated in two places: a full treatment of designed experiments, starting with this lesson, and a deeper, Black-Belt-level treatment of statistical process control later in this tier. This lesson covers the first half of the Body of Knowledge's Designed Experiments section: what an experiment is trying to accomplish, the vocabulary used to describe one, why changing one factor at a time falls short, and the practical considerations that go into planning a good one.

What Black Belt adds to Green Belt

Green Belt's Analyze and Improve tool set is built almost entirely around observing X's and Y's that already exist in the process: a hypothesis test compares groups that already differ, a regression fits a line through data that was already collected. Black Belt's central addition is the ability to deliberately assign values to the X's and watch what happens, which is a fundamentally stronger form of evidence for cause and effect. The second major addition, covered later in this tier, is a deeper treatment of control charts and control strategy at the level a Black Belt is expected to design and defend, rather than simply run.

Why run an experiment: four objectives

Every designed experiment starts by naming which of four objectives it is after, because the objective drives how many factors, levels, and runs the design needs:

• Screening: with many candidate factors and limited budget, find which few actually matter before investing further. A GCHS example: testing five candidate variables in a concrete cure process (mix ratio, ambient temperature, humidity, cure time, additive dosage) to find the two or three that actually move strength.
• Characterization or modeling: for the factors that matter, build a mathematical model of how they drive the response, so predictions can be made at settings not directly tested.
• Optimization: find the specific factor settings that produce the best response, such as the primer-and-cure-temperature combination that maximizes paint adhesion.
• Robustness: find factor settings where the response stays good even as uncontrollable noise factors, like weather during an exterior paint job, vary.

The vocabulary of an experiment

A short, precise vocabulary describes every experiment, and exam questions frequently test whether these terms are being used correctly:

Why one-factor-at-a-time (OFAT) testing falls short

The intuitive way to test two factors is to hold one fixed and vary the other, find its best level, then hold that fixed and vary the second. This is one-factor-at-a-time testing, and the diagram above shows its structural flaw: OFAT only ever tests along a cross-shaped path through the space of combinations. It never tests the corners, the combinations where both factors are away from the starting point at once, which is exactly where an interaction between the two factors shows up. A GCHS work-order scheduling team testing "batch size" and "dispatch time" one at a time could easily miss that a large batch dispatched early performs far better than either factor's individually best level would predict, because that combination was never run. A full factorial design, testing every combination, is the most direct way to detect an interaction with confidence (a well-chosen fractional design can estimate some interactions too, as Lesson 4 shows).

Randomization, replication, and repetition

Three related but distinct practices protect an experiment's validity. Randomization means running the treatments in a random order rather than a convenient one, so that an unmeasured factor that drifts over time, such as ambient shop temperature rising through the day, does not get confused with a real factor effect. Replication means independently repeating an entire run, including resetting factor levels and any setup steps, which captures the true run-to-run variation and lets the experiment estimate experimental error honestly. Repetition (sometimes called repeated measurement) means taking multiple readings from the same single run without resetting anything, which only captures measurement variation, not the process's true variation between runs. Treating repetitions as if they were replicates understates the real error and is a common design mistake. Blocking groups runs by a known source of variation that is not a factor of interest, such as which crew or which day a batch of concrete panels was poured, so that variation from that source is removed from the comparison between factor levels rather than confused with it.

Design considerations: planning before running

Before any run happens, four practical questions shape the design: choosing which factors and levels are worth testing, based on engineering knowledge and the screening step's results; confirming the measurement system is adequate, since Gage R&R applies here exactly as it did in Measure, because noisy measurements can hide a real factor effect or manufacture a fake one; sizing the sample, meaning the number of replicates, with enough power to detect an effect of practical importance against the response's natural variation; and sequential experimentation, meaning starting with a smaller screening design and adding runs only as needed, rather than committing to one large, expensive design up front before anything is known. A GCHS example: before running a full factorial test of four mailing-response factors (headline, offer, envelope color, send day), the team would first confirm response rate can be measured consistently, then run a smaller screening design to narrow four factors to the two or three worth a fuller study.