Ridge, Exactly

Normal-Equation Denominator

The one-weight ridge solve can be seen as a denominator change: least squares uses sum x squared, and ridge uses sum x squared plus λ.

highlighted = computed this step

Start with the least-squares denominator

The same two displayed points give sum xy=8 and sum x squared=5. In this one-weight through-origin toy, least squares uses that denominator.

∑xy=8,∑x2=5\sum xy=8,\quad \sum x^2=5
Normal-equation bridgeLeast squares denominator becomes ridge denominator.one-weight denominator bridgepieceexact arithmeticdata productsΣxy=1*2+2*3=8least-squares denominatorΣx^2=1^2+2^2=5ridge denominatorΣx^2+λ=5+1=6ridge weightw=8/6=4/3one-weight through-origin bridge onlythrough-origin ridge fit; λ is chosen before the solve; shrinkage is mechanical, NOT afuture-data claim

Add λ to that denominator

Choose λ=1 before the solve. Ridge changes the denominator from 5 to 5 + 1 = 6.

∑x2+λ=5+1=6\sum x^2 + \lambda = 5 + 1 = 6
Normal-equation bridgeLeast squares denominator becomes ridge denominator.one-weight denominator bridgepieceexact arithmeticdata productsΣxy=1*2+2*3=8least-squares denominatorΣx^2=1^2+2^2=5ridge denominatorΣx^2+λ=5+1=6ridge weightw=8/6=4/3one-weight through-origin bridge onlythrough-origin ridge fit; λ is chosen before the solve; shrinkage is mechanical, NOT afuture-data claim

Read off the exact weight

The numerator stays sum xy=8. The ridge weight is 8 / 6 = 4/3.

w=86=4/3w={8\over 6}=4/3
Normal-equation bridgeLeast squares denominator becomes ridge denominator.one-weight denominator bridgepieceexact arithmeticdata productsΣxy=1*2+2*3=8least-squares denominatorΣx^2=1^2+2^2=5ridge denominatorΣx^2+λ=5+1=6ridge weightw=8/6=4/3one-weight through-origin bridge onlythrough-origin ridge fit; λ is chosen before the solve; shrinkage is mechanical, NOT afuture-data claim

One-weight bridge only

This is the one-weight version of adding λ to the normal-equation denominator. It is exact arithmetic on the shown rows, not a broader optimization or future-data claim.

one weight: add λ to the denominator\text{one weight: add }\lambda\text{ to the denominator}
Normal-equation bridgeLeast squares denominator becomes ridge denominator.one-weight denominator bridgepieceexact arithmeticdata productsΣxy=1*2+2*3=8least-squares denominatorΣx^2=1^2+2^2=5ridge denominatorΣx^2+λ=5+1=6ridge weightw=8/6=4/3one-weight through-origin bridge onlythrough-origin ridge fit; λ is chosen before the solve; shrinkage is mechanical, NOT afuture-data claim