A single weight gradient is the upstream gradient at the receiving pre-activation times the input activation feeding that weight. This lesson isolates one entry, dw11, so the product can be checked before returning to the full annotated graph.

highlighted = computed this step

Choose one weight

Focus on weight w11. The upstream gradient is dL/dz1=-2, and the input feeding that weight is x1=1. Those two shown values are the source of truth for this one gradient entry.

dLdz1=2,x1=1\frac{dL}{dz_1}=-2,\quad x_1=1
One Weight Gradient ProductOne weight-gradient entry is upstream gradient times the input feeding that weight.One Weight Gradient ProductOne weight-gradient entry is upstream gradient times the input feeding that weight.one exact weight gradient productupstream gradient * input activation = one weight-gradient entryweight dw11upstream dz1=-2input x1=1dL/dw11 = -2*1 = -2gradient table entry: dw11=-2one displayed gradient entry only

Multiply upstream by input

The local rule for this weight is upstream gradient times the input activation. Here dL/dw11=-2*1=-2. No extra displayed number is supplied; the product is recomputed from the same exact inputs.

dLdw11=21=2\frac{dL}{dw_{11}}=-2\cdot1=-2
One Weight Gradient ProductOne weight-gradient entry is upstream gradient times the input feeding that weight.One Weight Gradient ProductOne weight-gradient entry is upstream gradient times the input feeding that weight.one exact weight gradient productupstream gradient * input activation = one weight-gradient entryweight dw11upstream dz1=-2input x1=1dL/dw11 = -2*1 = -2gradient table entry: dw11=-2one displayed gradient entry only

Place the table entry

The single product lands in the gradient table as dw11=-2. This is one displayed gradient entry only: the surrounding table and graph use the same exact reverse-pass register.

dw11=2dw_{11}=-2
One Weight Gradient ProductOne weight-gradient entry is upstream gradient times the input feeding that weight.One Weight Gradient ProductOne weight-gradient entry is upstream gradient times the input feeding that weight.one exact weight gradient productupstream gradient * input activation = one weight-gradient entryweight dw11upstream dz1=-2input x1=1dL/dw11 = -2*1 = -2gradient table entry: dw11=-2one displayed gradient entry only