Positional encoding uses sine and cosine, and attention uses softmax. This lesson groups those named components and contrasts them with exact wiring and residual addition.
highlighted = computed this step
Positional encoding
Positional encoding uses sine and cosine, so it is a named component. The flowchart shows the component without pinning a float value. The node tells the reader that position information enters here, while the transcendental values are not displayed as exact data.
posenc=named (sin,cos)
Attention's softmax
Attention includes the softmax boundary from the previous book. Its wiring is shown, while the softmax weights remain named. This is the same split as before: exact structure around a named normalization.
attention=named (softmax)
Summary
Positional encoding, attention softmax, and layernorm are the named components. Residual adds and the MLP-ReLU box stay in the exact register. The block is a mixture of exact wiring and named mathematical components.