One Exact Step

The Updated Network

After the update, the toy network has new exact parameters. The lesson names the changed entries without claiming anything about repeated steps.

highlighted = computed this step

The updated network

After the step, the first hidden unit uses w11=2, w12=3, and b1=0.

(w11,w12,b1)=(2,3,0)(w_{11},w_{12},b_1)=(2,3,0)
Updated networkExact update table with required boundary note.gradient descent update, η=1/2parameteroldgradientη·gradientneww111-2-12w121-4-23b1-1-2-10w211001w22-100-1b20000v11-4-23v21001c0-2-11w_new = w_old - η·gradientone exact step with η=1/2; NOT convergence, NOT the right η, NOT learning

Output layer update

The output layer now has v1=3, v2=1, and c=1.

(v1,v2,c)=(3,1,1)(v_1,v_2,c)=(3,1,1)
Updated networkExact update table with required boundary note.gradient descent update, η=1/2parameteroldgradientη·gradientneww111-2-12w121-4-23b1-1-2-10w211001w22-100-1b20000v11-4-23v21001c0-2-11w_new = w_old - η·gradientone exact step with η=1/2; NOT convergence, NOT the right η, NOT learning

Summary

The network changed by exact rational amounts. This lesson records the new parameters; it does not make a wider claim about repeated steps.

Δw=−η∇w\Delta w=-\eta\nabla w
Updated networkExact update table with required boundary note.gradient descent update, η=1/2parameteroldgradientη·gradientneww111-2-12w121-4-23b1-1-2-10w211001w22-100-1b20000v11-4-23v21001c0-2-11w_new = w_old - η·gradientone exact step with η=1/2; NOT convergence, NOT the right η, NOT learning