The Forward Pass

What the Forward Pass Is and Is Not

The finale keeps the exact arithmetic visible while drawing a boundary around broader claims. This book computes one forward pass on a small rational toy network.

highlighted = computed this step

What is exact

The forward pass is exact arithmetic on rational weights and the input (1, 2). The output is yhat=2 and the loss is L=1.

x=(1,2),y^=2,L=1x=(1, 2),\quad \hat y=2,\quad L=1
One exact forward passForward arithmetic is shown without broader claims.One exact forward passForward arithmetic is shown without broader claims.w11=1w12=1w21=1w22=-1ReLUReLUv1=1v2=1target=3x1x=1x2x=2z1val=2z2val=-1h1val=2h2val=0yhatval=2Lval=1b1=-1b2=0c=0z=0 convention: ReLU'(0)=0forward values: z1=2, h1=2, z2=-1, h2=0, yhat=2, L=1

What is not claimed

This is about 10 parameters, not 100B. It is not training, not learning, and no generalization claim is made.

one exact forward pass; no broader claim\text{one exact forward pass; no broader claim}
One exact forward passForward arithmetic is shown without broader claims.One exact forward passForward arithmetic is shown without broader claims.w11=1w12=1w21=1w22=-1ReLUReLUv1=1v2=1target=3x1x=1x2x=2z1val=2z2val=-1h1val=2h2val=0yhatval=2Lval=1b1=-1b2=0c=0z=0 convention: ReLU'(0)=0forward values: z1=2, h1=2, z2=-1, h2=0, yhat=2, L=1

Summary

This is exact arithmetic on a small rational toy network. It is not training, not learning, and no generalization claim; it pins the forward mechanics.

forward mechanics on toy rational data\text{forward mechanics on toy rational data}
One exact forward passForward arithmetic is shown without broader claims.One exact forward passForward arithmetic is shown without broader claims.w11=1w12=1w21=1w22=-1ReLUReLUv1=1v2=1target=3x1x=1x2x=2z1val=2z2val=-1h1val=2h2val=0yhatval=2Lval=1b1=-1b2=0c=0z=0 convention: ReLU'(0)=0forward values: z1=2, h1=2, z2=-1, h2=0, yhat=2, L=1