The Forward Pass

The Computation Graph

The full forward DAG puts every exact node value in one place. It is also the trust boundary: displayed weights and biases drive the recomputation.

highlighted = computed this step

Every node value at once

The full DAG shows the path x to z to h to yhat to L. Each label is recomputed from the displayed weights and biases.

x→z→h→y^→Lx\rightarrow z\rightarrow h\rightarrow \hat y\rightarrow L
Full exact forward DAGEvery node value is recomputed from the shown parameters.Full exact forward DAGEvery node value is recomputed from the shown parameters.w11=1w12=1w21=1w22=-1ReLUReLUv1=1v2=1target=3x1x=1x2x=2z1val=2z2val=-1h1val=2h2val=0yhatval=2Lval=1b1=-1b2=0c=0z=0 convention: ReLU'(0)=0forward values: z1=2, h1=2, z2=-1, h2=0, yhat=2, L=1

The visible source

The edge weights, b1=-1, b2=0, and c=0 are visible, so the diagram can be recomputed.

b1=−1,b2=0,c=0b_1=-1,\quad b_2=0,\quad c=0
Full exact forward DAGEvery node value is recomputed from the shown parameters.Full exact forward DAGEvery node value is recomputed from the shown parameters.w11=1w12=1w21=1w22=-1ReLUReLUv1=1v2=1target=3x1x=1x2x=2z1val=2z2val=-1h1val=2h2val=0yhatval=2Lval=1b1=-1b2=0c=0z=0 convention: ReLU'(0)=0forward values: z1=2, h1=2, z2=-1, h2=0, yhat=2, L=1

Summary

The graph is the trust boundary: stale weights, stale biases, or stale node values are rejected by the renderer.

displayed graph⇒recomputed values\text{displayed graph}\Rightarrow\text{recomputed values}
Full exact forward DAGEvery node value is recomputed from the shown parameters.Full exact forward DAGEvery node value is recomputed from the shown parameters.w11=1w12=1w21=1w22=-1ReLUReLUv1=1v2=1target=3x1x=1x2x=2z1val=2z2val=-1h1val=2h2val=0yhatval=2Lval=1b1=-1b2=0c=0z=0 convention: ReLU'(0)=0forward values: z1=2, h1=2, z2=-1, h2=0, yhat=2, L=1