Ridge can be read as minimizing an exact convex quadratic objective on the tiny dataset. This lesson compares candidate weights without importing an optimization method.
highlighted = computed this step
Ridge as an objective
For λ=1, evaluate the exact objective J(w)=sum of squared residuals plus λ*w squared. The table compares finite candidate weights from the same two shown points.
J(w)=∑(y−wx)2+λw2
Three exact candidates
The unregularized candidate w=8/5 has SSE=1/5, penalty=64/25, and J=69/25. The ridge candidate w=4/3 has SSE=5/9, penalty=16/9, and J=7/3. The simpler candidate w=1 has J=3.
J(8/5)=69/25,J(4/3)=7/3,J(1)=3
Where optimization belongs
ML can show this exact toy objective arithmetic. Choosing a general constrained or convex optimization method is the optimization/OR track boundary, not a claim completed by this ridge lesson.
ML shows exact J(w);methods belong to optimization/OR