R², Adjusted R² and Predicted R²: Why a High R² Can Mislead

After fitting a model to experimental data, the first number most people look at is R². It is useful, but on its own it can make a poor model look excellent. Three related statistics give a more honest picture.

R²: how well the model fits the data you have

R² is the fraction of the total variation in the response that the model explains:

$R^2 = 1 - SSE / SST$

SSE is the sum of squared residuals (what the model fails to explain) and SST is the total sum of squares about the mean. R² runs from 0 to 1, and it never decreases when you add a term, even a useless one. A model with as many terms as runs passes through every point and has R² = 1.

Adjusted R²: penalizing needless terms

Adjusted R² accounts for the number of terms. With n runs and p fitted coefficients (including the intercept):

$R^2_{adj} = 1 - \frac{SSE / (n - p)}{SST / (n - 1)}$

Adding a term that explains little can lower adjusted R². A big gap between R² and adjusted R² means the model has more terms than the data supports.

Predicted R²: how well it will predict new runs

Predicted R² asks a harder question: if each run were left out in turn, how well would the model predict it? The summary is the PRESS statistic, the sum of squared leave-one-out prediction errors. It can be computed without refitting, using each run's residual e and leverage h:

$PRESS = \sum \left( \frac{e_i}{1 - h_i} \right)^2 \qquad R^2_{pred} = 1 - PRESS / SST$

Predicted R² can be low or even negative while R² looks great. That pattern means the model is describing noise, or that one or two influential runs are driving the fit.

A worked pattern

These values are illustrative, not from a real experiment. The point is the relationship between the three numbers. A common rule of thumb is that adjusted and predicted R² should be within about 0.2 of each other.

What to do when they disagree

1. Look at the residual plots. One extreme point can wreck predicted R².
2. Check lack of fit if you have replicates. A significant result means a term is missing (for instance curvature).
3. Simplify by removing terms that are clearly negligible, then compare again.
4. Add runs where the model is weakest, especially replicates or follow-up points.

For how these checks fit into a whole experiment, read the practical guide to DoE and the glossary entry for predicted R².