The Gradient Question

Backpropagation asks for the loss derivative with respect to each parameter: the exact number that says how the loss moves when that parameter moves. This lesson reuses the exact forward values from the tiny network and sets up the reverse chain rule on the same computation graph. The goal is to make the backward pass feel like graph arithmetic, not a separate trick.

highlighted = computed this step

Backprop asks for dL/d(each weight): the exact rational gradient of the loss with respect to that parameter. It says which way the loss moves if that one weight moves. The chain rule computes those quantities by walking the already-shown graph in reverse.

\frac{dL}{d(\text{parameter})}\quad\text{by the chain rule}

Forward values we reuse

The forward pass ended with yhat=2 and L=1. Those exact node values are the inputs to the backward pass. Backprop attaches a second register to the same nodes, recording how the final loss depends on each one.

\hat y=2,\quad L=1

Summary

The backward pass starts at the loss and moves in reverse. Each node receives an incoming gradient, multiplies by its local derivative, and passes the result to the nodes that fed it. The next step computes dL/dyhat.

L\rightarrow\hat y\rightarrow h\rightarrow z\rightarrow w,b