One Exact Step

The Update Rule

The rule is old value minus a chosen rational step size times the gradient. With eta fixed at one half, every row remains exact rational arithmetic.

highlighted = computed this step

The update rule

Gradient descent uses w_new = w_old - eta*gradient. In this book, eta is the exact rational 1/2.

wnew=wold−η⋅∇w,η=1/2w_{\text{new}}=w_{\text{old}}-\eta\cdot \nabla w,\quad \eta=1/2
The update ruleExact update table with required boundary note.gradient descent update, η=1/2parameteroldgradientη·gradientneww111-2-12w121-4-23b1-1-2-10w211001w22-100-1b20000v11-4-23v21001c0-2-11w_new = w_old - η·gradientone exact step with η=1/2; NOT convergence, NOT the right η, NOT learning

A single parameter row

For w11, the old value is 1 and the gradient is -2. The exact move eta*gradient is -1.

1  −  1/2⋅−2=21\;-\;1/2\cdot-2=2
The update ruleExact update table with required boundary note.gradient descent update, η=1/2parameteroldgradientη·gradientneww111-2-12w121-4-23b1-1-2-10w211001w22-100-1b20000v11-4-23v21001c0-2-11w_new = w_old - η·gradientone exact step with η=1/2; NOT convergence, NOT the right η, NOT learning

Summary

The rule is a signed exact subtraction. Negative gradients increase a parameter here because subtracting a negative amount adds.

1  −  1/2⋅(−2)=21\;-\;1/2\cdot(-2)=2
The update ruleExact update table with required boundary note.gradient descent update, η=1/2parameteroldgradientη·gradientneww111-2-12w121-4-23b1-1-2-10w211001w22-100-1b20000v11-4-23v21001c0-2-11w_new = w_old - η·gradientone exact step with η=1/2; NOT convergence, NOT the right η, NOT learning