Wiring the Block

What the Block Is and Is Not

The finale states the boundary: exact wiring and residual addition, with layernorm, positional encoding, and softmax named.

highlighted = computed this step

What is exact

The block wiring is exact, and each residual add is exact vector addition. The example sum is (3,2).

residual sum=(3,2)\text{residual sum}=(3,2)
Transformer block boundaryExact residual adds and named boundary nodes.Transformer block boundaryExact residual adds and named boundary nodes.transformer block wiringwiring is exact; named components carry no numeric outputinput hexactdisplayed vector+ positional encodingnamed (sin/cos)attentionnamed (softmax)residual addexactvector additionlayernormnamed (√)MLP-ReLUexactrational ReLU wiringresidual addexactvector additionlayernormnamed (√)outputexactresidual example: h=(2,3) + s=(1,-1) = (3,2)exact residual adddisplayed vectors are exact integers; sum is recomputedh=(2,3)+s=(1,-1)=r=(3,2)

What is not claimed

Layernorm, positional encoding, and softmax are named rather than decimalized. This is not learning, not meaning, and not understanding.

not learning; not meaning; not understanding\text{not learning; not meaning; not understanding}
Transformer block boundaryExact residual adds and named boundary nodes.Transformer block boundaryExact residual adds and named boundary nodes.transformer block wiringwiring is exact; named components carry no numeric outputinput hexactdisplayed vector+ positional encodingnamed (sin/cos)attentionnamed (softmax)residual addexactvector additionlayernormnamed (√)MLP-ReLUexactrational ReLU wiringresidual addexactvector additionlayernormnamed (√)outputexactresidual example: h=(2,3) + s=(1,-1) = (3,2)exact residual adddisplayed vectors are exact integers; sum is recomputedh=(2,3)+s=(1,-1)=r=(3,2)

Summary

The block wiring and residual adds are exact. Layernorm, positional encoding, and softmax are the named boundary. This pins the block structure; it is not learning, not meaning, and not understanding.

block structure with named boundaries\text{block structure with named boundaries}
Transformer block boundaryExact residual adds and named boundary nodes.Transformer block boundaryExact residual adds and named boundary nodes.transformer block wiringwiring is exact; named components carry no numeric outputinput hexactdisplayed vector+ positional encodingnamed (sin/cos)attentionnamed (softmax)residual addexactvector additionlayernormnamed (√)MLP-ReLUexactrational ReLU wiringresidual addexactvector additionlayernormnamed (√)outputexactresidual example: h=(2,3) + s=(1,-1) = (3,2)exact residual adddisplayed vectors are exact integers; sum is recomputedh=(2,3)+s=(1,-1)=r=(3,2)