Positional Encoding and the Named Components

Positional encoding uses sine and cosine, and attention uses softmax. This lesson groups those named components and contrasts them with exact wiring.

highlighted = computed this step

Positional encoding

Positional encoding uses sine and cosine, so it is a named component. The flowchart shows the component without pinning a float value.

\text{posenc}=\text{named }(\sin,\cos)

Attention includes the softmax boundary from the previous book. Its wiring is shown, while the softmax weights remain named.

\text{attention}=\text{named }(\operatorname{softmax})

Positional encoding, attention softmax, and layernorm are the named components. Residual adds and the MLP-ReLU box stay in the exact register.

\text{named components plus exact wiring}