ReLU's Derivative is 0 or 1 - Backpropagation, Exactly

ReLU is the keystone for exact backprop here. Its derivative is an exact gate, so the reverse pass stays rational and the negative branch is blocked.

highlighted = computed this step

ReLU's derivative is exact

ReLU'(z) is 1 when z is positive and 0 otherwise; at z=0 this surface uses the stated convention ReLU'(0)=0.

\operatorname{ReLU}'(z)=\begin{cases}1&z>0\\0&z\le0\end{cases}

Here z1=2 gives ReLU'(z1)=1, while z2=-1 gives ReLU'(z2)=0.

\operatorname{ReLU}'(z_1)=1,\quad \operatorname{ReLU}'(z_2)=0

Multiplying by the second derivative gate blocks that branch exactly. The result is dL/dz1=-2 and dL/dz2=0.

\frac{dL}{dz_1}=-2\cdot1=-2,\quad \frac{dL}{dz_2}=-2\cdot0=0