Residual connections add a sublayer output back to its input. The example here is exact vector addition, recomputed from displayed integers and kept separate from named components.

highlighted = computed this step

Residual connections

A residual connection adds a sublayer output back to its input. The rule is exact vector addition: output equals sublayer of x plus x. This direct add is one of the block pieces that stays fully in the exact register.

out=sublayer(x)+x\text{out}=\text{sublayer}(x)+x
Exact residual addThe residual output is recomputed from displayed vectors.Exact residual addThe residual output is recomputed from displayed vectors.exact residual adddisplayed vectors are exact integers; sum is recomputedh=(2,3)+s=(1,-1)=r=(3,2)

Exact residual add

Here the input vector is (2,3) and the sublayer vector is (1,-1). Their exact sum is (3,2). Each coordinate is recomputed from the displayed vectors, so the residual example has no hidden source.

(2,3)+(1,1)=(3,2)(2,3)+(1,-1)=(3,2)
Exact residual addThe residual output is recomputed from displayed vectors.Exact residual addThe residual output is recomputed from displayed vectors.exact residual adddisplayed vectors are exact integers; sum is recomputedh=(2,3)+s=(1,-1)=r=(3,2)

Summary

The residual add keeps a direct path through the block, and the add itself is exact. No transcendental operation appears in this step; the named boundaries come from other components.

residual add is exact\text{residual add is exact}
Exact residual addThe residual output is recomputed from displayed vectors.Exact residual addThe residual output is recomputed from displayed vectors.exact residual adddisplayed vectors are exact integers; sum is recomputedh=(2,3)+s=(1,-1)=r=(3,2)