ReLU is the keystone for exact backprop here. Its derivative is a discrete gate: open for positive pre-activations, closed for negative ones, with the stated convention at zero. Because the gate is exact, no transcendental derivative enters this book, and the reverse pass stays rational.
highlighted = computed this step
ReLU's derivative is exact
ReLU'(z) is 1 when z is positive and 0 otherwise; at z=0 this surface uses the stated convention ReLU'(0)=0. This derivative is a discrete gate, not a curve with a transcendental slope. That is the keystone reason this toy backward pass stays in the exact rational register.
ReLU′(z)={10z>0z≤0
Two derivative gates
Here z1=2 gives ReLU'(z1)=1, while z2=-1 gives ReLU'(z2)=0. The positive pre-activation keeps the incoming gradient alive. The negative pre-activation closes the gate, so the graph records the branch but sends no gradient through it.
ReLU′(z1)=1,ReLU′(z2)=0
The dead branch
Multiplying by the second derivative gate blocks that branch exactly. The result is dL/dz1=-2 and dL/dz2=0. These are gradients with respect to the pre-activation nodes, so they are the precise signals that will feed the hidden weights and biases in the next lesson.