The same two stochastic samples can land differently when their order is reversed because each later gradient is recomputed after the previous update.
highlighted = computed this step
Same samples, same eta
Both paths use sample s2 and sample s3 with eta=1/2. The only change is the order.
{s2,s3},η=1/2
Order A: s2 then s3
Order A starts with sample s2 and then sample s3. Because the s3 gradient is recomputed after the first update, the endpoint is (-1/4, 0).
s2→s3:(a,b)=(−1/4,0)
Order B: s3 then s2
Order B starts with sample s3. At the starting line, that row has residual 0 and gradient (0, 0), so the first update does not move. Then sample s2 lands at (1/2, 3/2).
s3→s2:(a,b)=(1/2,3/2)
The paths can differ
The chosen samples are the same, but each second gradient is recomputed at the parameters produced by the first update. This is exact path arithmetic only: NOT convergence, NOT generalization, and NOT learning by itself.