Least Squares

Residuals and the Error We Minimize

A residual is the vertical miss between a shown value and a fitted value. Squared error turns those misses into one exact quantity to minimize.

highlighted = computed this step

A trial line

For the trial line y=1 plus 1x, residuals are computed as actual y minus fitted y.

r=y−(1+1x)r=y-(1+{}1x)
Trial residualsThe trial line has exact residuals and a nonzero residual sum.Trial residualsxyfitresid01101321223-13541sum residuals = 1

Why squared error

Squaring makes every miss positive and gives larger misses more weight. The trial residuals are 0, 1, -1, and 1.

SSE=3SSE=3
Trial residualsThe trial line has exact residuals and a nonzero residual sum.Trial residualsxyfitresid01101321223-13541sum residuals = 1

Summary

This trial line has residual sum 1 and SSE 3. Least squares will minimize that exact sum of squares.

∑r=1,SSE=3\sum r=1,\quad SSE=3
Trial residualsThe trial line has exact residuals and a nonzero residual sum.Trial residualsxyfitresid01101321223-13541sum residuals = 1