Wiring the Block

What the Block Is and Is Not

The finale states the boundary: exact wiring and residual addition, with layernorm, positional encoding, and softmax named rather than decimalized.

highlighted = computed this step

What is exact

The block wiring is exact, and each residual add is exact vector addition. The example sum is (3,2). That is the pinned part of this book: component order and residual arithmetic.

residual sum=(3,2)\text{residual sum}=(3,2)
Transformer block boundaryExact residual adds and named boundary nodes.Transformer block boundaryExact residual adds and named boundary nodes.transformer block wiringwiring is exact; named components carry no numeric outputinput hexactdisplayed vector+ positional encodingnamed (sin/cos)attentionnamed (softmax)residual addexactvector additionlayernormnamed (√)MLP-ReLUexactrational ReLU wiringresidual addexactvector additionlayernormnamed (√)outputexactresidual example: h=(2,3) + s=(1,-1) = (3,2)exact residual adddisplayed vectors are exact integers; sum is recomputedh=(2,3)+s=(1,-1)=r=(3,2)

What is not claimed

Layernorm, positional encoding, and softmax are named rather than decimalized. This is not learning, not meaning, and not understanding. The book renders the block structure and the exact residual add, not the broader behavior of a real system.

not learning; not meaning; not understanding\text{not learning; not meaning; not understanding}
Transformer block boundaryExact residual adds and named boundary nodes.Transformer block boundaryExact residual adds and named boundary nodes.transformer block wiringwiring is exact; named components carry no numeric outputinput hexactdisplayed vector+ positional encodingnamed (sin/cos)attentionnamed (softmax)residual addexactvector additionlayernormnamed (√)MLP-ReLUexactrational ReLU wiringresidual addexactvector additionlayernormnamed (√)outputexactresidual example: h=(2,3) + s=(1,-1) = (3,2)exact residual adddisplayed vectors are exact integers; sum is recomputedh=(2,3)+s=(1,-1)=r=(3,2)

Summary

The block wiring and residual adds are exact. Layernorm, positional encoding, and softmax are the named boundary. This pins the block structure; it is not learning, not meaning, and not understanding. The useful distinction is exact wiring plus exact residual addition around named components for normalization, position, and attention weights.

block structure with named boundaries\text{block structure with named boundaries}
Transformer block boundaryExact residual adds and named boundary nodes.Transformer block boundaryExact residual adds and named boundary nodes.transformer block wiringwiring is exact; named components carry no numeric outputinput hexactdisplayed vector+ positional encodingnamed (sin/cos)attentionnamed (softmax)residual addexactvector additionlayernormnamed (√)MLP-ReLUexactrational ReLU wiringresidual addexactvector additionlayernormnamed (√)outputexactresidual example: h=(2,3) + s=(1,-1) = (3,2)exact residual adddisplayed vectors are exact integers; sum is recomputedh=(2,3)+s=(1,-1)=r=(3,2)