Design of Experiments (DoE): A Practical Guide

Most people learn to experiment by changing one thing at a time. It feels rigorous: hold everything else still, vary one setting, see what happens. It is also slow, and it can miss the most important thing in your process. Design of Experiments (DoE) is the alternative: change several factors together, in a deliberate pattern, and let statistics separate their effects.

The vocabulary in one minute

• A factor is an input you control, such as temperature, pressure or concentration.
• A level is a setting of a factor. Two-level designs use a low and a high level.
• The response is the measured outcome you want to improve.
• An interaction means the effect of one factor depends on the level of another.
• A run is one experiment at one combination of settings.

Our glossary defines each term more carefully.

Why not one factor at a time?

Changing one factor at a time cannot detect interactions, and it spends runs repeating information. A factorial design estimates every factor's effect using all the runs at once. We work through a concrete example in why one-factor-at-a-time fails.

The three questions that choose your design

1. Which factors matter? This is screening. With many candidate factors, use a two-level factorial or a fractional factorial to find the vital few cheaply.
2. How do the important factors combine? A full factorial estimates every main effect and interaction.
3. Where is the optimum? This is optimization. Response-surface designs such as the central composite or Box-Behnken fit curvature so you can locate a peak or valley. See central composite vs Box-Behnken.

See the designs for yourself

Pick a number of factors and a design to see the runs. Levels are coded: −1 is the low setting, +1 the high, 0 the midpoint.

Notice how the number of runs grows with the number of factors, and how the response-surface designs add midpoint and axial runs that the two-level factorials lack.

A DoE workflow

1. State the goal and the response. Maximize yield? Hit a target viscosity? Choose a response you can measure reliably.
2. Choose factors and ranges. Ranges should be wide enough to move the response, but safe and realistic. Too narrow and the signal disappears in noise; too wide and the process may fail.
3. Pick a design that matches your question and budget.
4. Randomize the run order. Running experiments in random order stops uncontrolled drift (equipment warm-up, ambient changes, operator fatigue) from being mistaken for a factor effect.
5. Include centre points. Replicated runs at the midpoint provide an estimate of pure error and reveal curvature.
6. Run the experiments and record the responses.
7. Fit a model and check it. Look at the ANOVA table, the lack-of-fit test, and the residuals. A model that fits poorly can mislead.
8. Confirm. Run one or two experiments at the settings the model recommends, and check the prediction.

What the analysis tells you

For a two-level design the model is usually a main-effects-plus-interactions model:

$y = \beta_0 + \sum \beta_i x_i + \sum \beta_{ij} x_i x_j + \varepsilon$

Each coefficient is the effect of a factor, or of a pair of factors together, in coded units. The ANOVA table tests whether the model and each term explain more variation than random error would. Our guide to R², adjusted R² and predicted R² explains how to judge how good a fitted model really is.

Where to go next

• Try the free design generator to build a randomized run sheet and download it as CSV.
• Open the app to describe an experiment in plain English and analyze your results.
• Read how every statistic is computed on the methods page.

Further reading

• D. C. Montgomery, Design and Analysis of Experiments.
• G. E. P. Box, J. S. Hunter and W. G. Hunter, Statistics for Experimenters.
• NIST/SEMATECH e-Handbook of Statistical Methods, chapter on process improvement.