Exact Sparsity

At and Beyond the Threshold

After the coefficient reaches zero, larger lambda values keep the same one-weight solution at the exact floor instead of pushing it negative.

highlighted = computed this step

Choose lambda first

Use the same source sums: sum xy=28 and sum x squared=14. Now compare three chosen lambda values: 21, 28, and 35. Lambda is chosen before the solve.

λ∈{21,28,35};∑xy=28\lambda\in\{21,28,35\};\quad \sum xy=28
At and beyond the thresholdExact floor behavior once lambda reaches the source correlation.soft-threshold pathλmax(28-λ,0)w2171/228003500λ=28 gives exact zerothrough-origin lasso fit; λ is chosen before the solve; L1 can hit exact zero; NOTconvergence; NOT general feature selection; NOT generalization

At the threshold

At λ=28, the numerator is max(28 minus 28, 0)=0. The weight is exactly 0.

max⁡(28−28,0)=0⇒w=0\max(28 - 28,0)=0\Rightarrow w=0
At and beyond the thresholdExact floor behavior once lambda reaches the source correlation.soft-threshold pathλmax(28-λ,0)w2171/228003500λ=28 gives exact zerothrough-origin lasso fit; λ is chosen before the solve; L1 can hit exact zero; NOTconvergence; NOT general feature selection; NOT generalization

Beyond the threshold

At λ=35, max(28 minus 35, 0) is still 0. Once λ reaches the source correlation 28, larger λ values do not make this one-weight coefficient negative; the floor holds it at exact zero.

λ≥28⇒max⁡(28−λ,0)=0⇒w=0\lambda\ge 28\Rightarrow \max(28 - \lambda,0)=0\Rightarrow w=0
At and beyond the thresholdExact floor behavior once lambda reaches the source correlation.soft-threshold pathλmax(28-λ,0)w2171/228003500λ=28 gives exact zerothrough-origin lasso fit; λ is chosen before the solve; L1 can hit exact zero; NOTconvergence; NOT general feature selection; NOT generalization