The Normal Equations - Linear Regression (Least Squares)

Least squares turns a quadratic minimization into linear conditions. The result is a small exact system with no transcendental boundary.

highlighted = computed this step

SSE is a quadratic

SSE is a polynomial in a and b. Minimizing it gives slope conditions that vanish at the minimum.

SSE(a,b)\text{ is quadratic}

Those vanishing conditions are linear in a and b, so the fit becomes an exactly solvable system with no named boundary operation.

X^{T}X\beta=X^{T}y

For the shown data, the normal equations are exactly the displayed augmented matrix.

\begin{bmatrix}4&6&11\\6&14&22\end{bmatrix}