Exact Sparsity

The Soft-Threshold Rule

The one-weight lasso solution is a soft-thresholded exact ratio. Lambda subtracts from the source correlation, then the numerator floors at zero.

highlighted = computed this step

The soft-threshold rule

Because sum xy is positive here, the closed form is max(sum xy minus λ, zero) divided by sum x squared. With sum xy=28 and sum x squared=14, every row stays exact.

wλ=max⁡(28−λ,0)14w_\lambda={\max(28-\lambda,0)\over 14}
Soft-threshold pathExact lasso weights by lambda.soft-threshold pathλmax(28-λ,0)w7213/2141412800λ=28 gives exact zerothrough-origin lasso fit; λ is chosen before the solve; L1 can hit exact zero; NOTconvergence; NOT general feature selection; NOT generalization

Two shrinkage values

At λ=7, the weight is 3/2. At λ=14, the weight is 1. Both are exact rational shrinkage values.

w(7)=3/2,w(14)=1w(7)=3/2,\quad w(14)=1
Soft-threshold pathExact lasso weights by lambda.soft-threshold pathλmax(28-λ,0)w7213/2141412800λ=28 gives exact zerothrough-origin lasso fit; λ is chosen before the solve; L1 can hit exact zero; NOTconvergence; NOT general feature selection; NOT generalization

Summary

Soft-thresholding subtracts λ from the source correlation and floors at zero. The floor is exact, not approximate.

max⁡(∑xy−λ,0)\max(\sum xy-\lambda,0)
Soft-threshold pathExact lasso weights by lambda.soft-threshold pathλmax(28-λ,0)w7213/2141412800λ=28 gives exact zerothrough-origin lasso fit; λ is chosen before the solve; L1 can hit exact zero; NOTconvergence; NOT general feature selection; NOT generalization