Ridge, Exactly

What Ridge Is and Is Not

The finale keeps the modeling boundary visible: lambda is chosen, the dataset is tiny, and shrinkage is a mechanical fact of the displayed equation.

highlighted = computed this step

What is exact

The model choice is through-origin, with one weight. The regularized fit is the exact ratio sum xy over sum x squared plus λ.

w(λ)=∑xy∑x2+λw(\lambda)={\sum xy\over \sum x^2+\lambda}
Ridge honesty boundaryExact mechanics with the boundary note.exact shrinkage tableλΣx^2+λw(λ)058/5164/33815104/5|w(λ)| decreases as λ increasesλ=0, w=8/5λ=3, w=1λ=5, w=4/5through-origin ridge fit; λ is chosen before the solve; shrinkage is mechanical, NOT afuture-data claim

What is chosen

λ is chosen before the computation. The diagram states that λ is chosen before the solve, and the arithmetic makes no claim about future data.

λ is chosen before this exact solve\lambda\text{ is chosen before this exact solve}
Ridge honesty boundaryExact mechanics with the boundary note.exact shrinkage tableλΣx^2+λw(λ)058/5164/33815104/5|w(λ)| decreases as λ increasesλ=0, w=8/5λ=3, w=1λ=5, w=4/5through-origin ridge fit; λ is chosen before the solve; shrinkage is mechanical, NOT afuture-data claim

What ridge is and is not

This pins the MECHANICS: the regularized fit is an exact ratio of integers on one tiny dataset. λ is chosen before the solve; shrinkage is mechanical, NOT a future-data claim.

exact ridge mechanics; deferred claims explicit\text{exact ridge mechanics; deferred claims explicit}
Ridge honesty boundaryExact mechanics with the boundary note.exact shrinkage tableλΣx^2+λw(λ)058/5164/33815104/5|w(λ)| decreases as λ increasesλ=0, w=8/5λ=3, w=1λ=5, w=4/5through-origin ridge fit; λ is chosen before the solve; shrinkage is mechanical, NOT afuture-data claim