Residual addition happens before layernorm. This lesson shows one exact vector add, coordinate by coordinate, and then stops before the named layernorm boundary.
highlighted = computed this step
Start with two vectors
The residual path keeps the input vector h=(2,3) and adds the sublayer output s=(1,-1). Both vectors are displayed before the add, so the residual sum has a visible source.
h=(2,3),s=(1,−1)
Add coordinate by coordinate
The first coordinate is 2 + 1 = 3. The second coordinate is 3 + -1 = 2. So the exact residual vector is r=(3,2).
(2+1,3+−1)=(3,2)
Layernorm comes next
The residual result is (3,2). The next block node is layernorm, which is a named boundary here. This lesson stops at the exact residual add and does not compute a layernorm output.