The Forward Pass

Weighted Sums

Each hidden unit starts with an exact weighted sum plus a bias. The table keeps the weights, inputs, and biases visible so the pre-activations have a shown source.

highlighted = computed this step

First hidden sum

For the first hidden unit, z1 is w11 times x1 plus w12 times x2 plus b1. Here that is 2.

z1=1⋅1  +  1⋅2  −  1=2z_1=1\cdot1\;+\;1\cdot2\;-\;1=2
Weighted sumsHidden pre-activations are recomputed.Weighted sumsHidden pre-activations are recomputed.forward pass (all exact)quantityrulevaluez11*1 + 1*2 - 12z21*1 - 1*2 + 0-1

Term by term

The same first sum is a row dotted with the input column: each weight times its input is one product, and the products accumulate left to right. The two products are 1 and 2, the running total reaches 3, and adding the bias b1=-1 gives z1=2.

z1=(1  +  2)  +  (−1)=2z_1=\big(1\;+\;2\big)\;+\;\big(-1\big)=2
Weighted sum, term by termExact rational row-dot-column walkthrough of the first hidden neuron.weighted sum z = w·x + b (one neuron)Ac1c211·xentry12products1·1=1Σ=1+1·2=2Σ=3result 3 + b-1z = 3 + (-1) = 2one exact weighted sum on a tiny rational MLP; NOT training, NOT learning, NOT ageneralization claim

Second hidden sum

For the second hidden unit, the negative weight on x2 changes the sign of that contribution. The exact result is z2=-1.

z2=1⋅1  −  1⋅2  +  0=−1z_2=1\cdot1\;-\;1\cdot2\;+\;0=-1
Weighted sumsHidden pre-activations are recomputed.Weighted sumsHidden pre-activations are recomputed.forward pass (all exact)quantityrulevaluez11*1 + 1*2 - 12z21*1 - 1*2 + 0-1

Summary

Both pre-activations are exact integers. The visible biases b1=-1 and b2=0 are part of the recomputed source.

z1=2,z2=−1z_1=2,\quad z_2=-1
Weighted sumsHidden pre-activations are recomputed.Weighted sumsHidden pre-activations are recomputed.forward pass (all exact)quantityrulevaluez11*1 + 1*2 - 12z21*1 - 1*2 + 0-1