A multi-channel convolution matches each input channel with a kernel slice, computes each dot product at the same landing, then adds those totals before ReLU.
Each channel has a matching slice
This example has channel A and channel B. Each channel uses its matching given kernel slice at the same valid landing. The focus start is row 0, col 0.
channel A ⋅ kernel A + channel B ⋅ kernel B \text{channel A}\cdot\text{kernel A}\;+\;\text{channel B}\cdot\text{kernel B} channel A ⋅ kernel A + channel B ⋅ kernel B Channels add before ReLU Matching channel and kernel slices are added before ReLU. channel A c1 c2 c3 1 0 2 0 1 0 2 0 1 kernel slice A c1 c2 1 0 0 1 channel B c1 c2 c3 0 1 0 1 0 1 0 1 0 kernel slice B c1 c2 0 1 1 0 channels add before ReLU part exact arithmetic total A dot 1*1 + 0*0 + 0*0 + 1*1 2 B dot 0*0 + 1*1 + 1*1 + 0*0 2 add 2 + 2 4 same landing (0,0); add channel totals before ReLU multi-channel feature map c1 c2 4 0 0 4 valid padding, stride 1 given kernel slices; one exact forward pass; channels add before ReLU; head softmax is NAMED; NOT training; NOT learning; NOT probability; NOT scaling proof kernel is given; valid padding and stride one; one exact forward pass; head softmax is NAMED; NOT training; NOT learning
Compute the two channel dots
At start (0,0), channel A gives 1·1 + 0·0 + 0·0 + 1·1 = 2. Channel B gives 2.
1 ⋅ 1 + 0 ⋅ 0 + 0 ⋅ 0 + 1 ⋅ 1 = 2 1\cdot1 + 0\cdot0 + 0\cdot0 + 1\cdot1 = 2 1 ⋅ 1 + 0 ⋅ 0 + 0 ⋅ 0 + 1 ⋅ 1 = 2 Channels add before ReLU Matching channel and kernel slices are added before ReLU. channel A c1 c2 c3 1 0 2 0 1 0 2 0 1 kernel slice A c1 c2 1 0 0 1 channel B c1 c2 c3 0 1 0 1 0 1 0 1 0 kernel slice B c1 c2 0 1 1 0 channels add before ReLU part exact arithmetic total A dot 1*1 + 0*0 + 0*0 + 1*1 2 B dot 0*0 + 1*1 + 1*1 + 0*0 2 add 2 + 2 4 same landing (0,0); add channel totals before ReLU multi-channel feature map c1 c2 4 0 0 4 valid padding, stride 1 given kernel slices; one exact forward pass; channels add before ReLU; head softmax is NAMED; NOT training; NOT learning; NOT probability; NOT scaling proof kernel is given; valid padding and stride one; one exact forward pass; head softmax is NAMED; NOT training; NOT learning
Add before ReLU
The channel totals are added into one feature-map cell before ReLU: 2 + 2 = 4.
Channels add before ReLU Matching channel and kernel slices are added before ReLU. channel A c1 c2 c3 1 0 2 0 1 0 2 0 1 kernel slice A c1 c2 1 0 0 1 channel B c1 c2 c3 0 1 0 1 0 1 0 1 0 kernel slice B c1 c2 0 1 1 0 channels add before ReLU part exact arithmetic total A dot 1*1 + 0*0 + 0*0 + 1*1 2 B dot 0*0 + 1*1 + 1*1 + 0*0 2 add 2 + 2 4 same landing (0,0); add channel totals before ReLU multi-channel feature map c1 c2 4 0 0 4 valid padding, stride 1 given kernel slices; one exact forward pass; channels add before ReLU; head softmax is NAMED; NOT training; NOT learning; NOT probability; NOT scaling proof kernel is given; valid padding and stride one; one exact forward pass; head softmax is NAMED; NOT training; NOT learning
The multi-channel feature map
Repeating the same rule at every valid start gives [[4, 0], [0, 4]]. This is one exact forward-pass example with given kernels: NOT training, NOT learning, NOT probability, and NOT scaling proof.
[ 4 0 0 4 ] \begin{bmatrix}4&0\\0&4\end{bmatrix} [ 4 0 0 4 ] Channels add before ReLU Matching channel and kernel slices are added before ReLU. channel A c1 c2 c3 1 0 2 0 1 0 2 0 1 kernel slice A c1 c2 1 0 0 1 channel B c1 c2 c3 0 1 0 1 0 1 0 1 0 kernel slice B c1 c2 0 1 1 0 channels add before ReLU part exact arithmetic total A dot 1*1 + 0*0 + 0*0 + 1*1 2 B dot 0*0 + 1*1 + 1*1 + 0*0 2 add 2 + 2 4 same landing (0,0); add channel totals before ReLU multi-channel feature map c1 c2 4 0 0 4 valid padding, stride 1 given kernel slices; one exact forward pass; channels add before ReLU; head softmax is NAMED; NOT training; NOT learning; NOT probability; NOT scaling proof kernel is given; valid padding and stride one; one exact forward pass; head softmax is NAMED; NOT training; NOT learning