Regression

Least-Squares Line

Fit a least-squares line by computing slope = Σ(xi−mx)(yi−my) / Σ(xi−mx)² and intercept = my − slope×mx. Two loops: the first accumulates sx and sy for the means; the second accumulates cov_sum and var_sum. ddof cancels (n−1 in both numerator and denominator), so unnormalized sums suffice. Library: scipy.stats.linregress(x, y) — RESULT = (slope, intercept) rounded.

By hand

x=[1,2,3,4,5], y=[2,4,4,4,6]. mx=3, my=4. cov_sum=8, var_sum=10. slope=8/10=0.8. intercept=4−0.8×3=1.6. Line: ŷ=0.8x+1.6.

naive.py
Replay: real traced execution (multi-file project)
x = [1, 2, 3, 4, 5]
y = [2, 4, 4, 4, 6]
n = len(x)
sx = 0.0
sy = 0.0
for i in range(n):
    sx = sx + x[i]
    sy = sy + y[i]
mx = sx / n
my = sy / n
cov_sum = 0.0
var_sum = 0.0
for i in range(n):
    cov_sum = cov_sum + (x[i] - mx) * (y[i] - my)
    var_sum = var_sum + (x[i] - mx) ** 2
slope = cov_sum / var_sum
intercept = my - slope * mx
print('RESULT:', (round(slope, 4), round(intercept, 4)))

  1. x ← [1, 2, 3, 4, 5]

    1x = [1, 2, 3, 4, 5]2y = [2, 4, 4, 4, 6]
    values this step[1, 2, 3, 4, 5]x

  2. y ← [2, 4, 4, 4, 6]

    1x = [1, 2, 3, 4, 5]2y = [2, 4, 4, 4, 6]3n = len(x)
    values this step[2, 4, 4, 4, 6]y

  3. n ← 5

    2y = [2, 4, 4, 4, 6]3n = len(x)4sx = 0.0
    values this step5n

  4. sx ← 0.0

    3n = len(x)4sx = 0.05sy = 0.0
    values this step0.0sx

  5. sy ← 0.0

    4sx = 0.05sy = 0.06for i in range(n):
    values this step0.0sy

  6. i ← 0, sx ← 1.0, sy ← 2.0

    pass 1 of 5
    5sy = 0.06for i in range(n):7    sx = sx + x[i]8    sy = sy + y[i]9mx = sx / n
    values this step0i0.0 → 1.0sx0.0 → 2.0sy
    All 5 passes — pass 1 is the card above
    passisxsy
    100.0 → 1.00.0 → 2.0
    20 → 11.0 → 3.02.0 → 6.0
    31 → 23.0 → 6.06.0 → 10.0
    42 → 36.0 → 10.010.0 → 14.0
    53 → 410.0 → 15.014.0 → 20.0

  7. for i in range(n):

    5sy = 0.06for i in range(n):7    sx = sx + x[i]

  8. mx ← 3.0

    8    sy = sy + y[i]9mx = sx / n10my = sy / n
    values this step3.0mx

  9. my ← 4.0

    9mx = sx / n10my = sy / n11cov_sum = 0.0
    values this step4.0my

  10. cov_sum ← 0.0

    10my = sy / n11cov_sum = 0.012var_sum = 0.0
    values this step0.0cov_sum

  11. var_sum ← 0.0

    11cov_sum = 0.012var_sum = 0.013for i in range(n):
    values this step0.0var_sum

  12. i ← 0, cov_sum ← 4.0, var_sum ← 4.0

    pass 1 of 5
    12var_sum = 0.013for i in range(n):14    cov_sum = cov_sum + (x[i] - mx) * (y[i] - my)15    var_sum = var_sum + (x[i] - mx) ** 216slope = cov_sum / var_sum
    values this step4 → 0i0.0 → 4.0cov_sum0.0 → 4.0var_sum
    All 5 passes — pass 1 is the card above
    passicov_sumvar_sum
    14 → 00.0 → 4.00.0 → 4.0
    20 → 1—4.0 → 5.0
    31 → 2——
    42 → 3—5.0 → 6.0
    53 → 44.0 → 8.06.0 → 10.0

  13. for i in range(n):

    12var_sum = 0.013for i in range(n):14    cov_sum = cov_sum + (x[i] - mx) * (y[i] - my)

  14. slope ← 0.8

    15    var_sum = var_sum + (x[i] - mx) ** 216slope = cov_sum / var_sum17intercept = my - slope * mx
    values this step0.8slope

  15. intercept ← 1.5999999999999996

    16slope = cov_sum / var_sum17intercept = my - slope * mx18print('RESULT:', (round(slope, 4), round(intercept, 4)))
    values this step1.5999999999999996intercept

  16. stdout ← RESULT: (0.8, 1.6)

    17intercept = my - slope * mx18print('RESULT:', (round(slope, 4), round(intercept, 4)))
    values this stepRESULT: (0.8, 1.6)stdout

With the library

scipy.stats.linregress(x, y) returns slope, intercept, r (correlation), pvalue, and stderr. The snapshot shows slope, intercept, and r; RESULT is (slope, intercept).

library.py 
from scipy import stats
from dalib.display import set_display
set_display()

x = [1, 2, 3, 4, 5]
y = [2, 4, 4, 4, 6]
fit = stats.linregress(x, y)
print('slope:', round(float(fit.slope), 4))
print('intercept:', round(float(fit.intercept), 4))
print('r:', round(float(fit.rvalue), 4))
print('RESULT:', (round(float(fit.slope), 4), round(float(fit.intercept), 4)))

slope: 0.8
intercept: 1.6
r: 0.8944
RESULT: (0.8, 1.6)

Honesty

This lesson shows the computation of the least-squares line exactly, on a tiny pinned sample. The slope and intercept are arithmetically correct and reproducible, but a line fit to a handful of points is a mechanism demo, not a predictive model — it is not a statistical finding. Reading significance into the slope, or extrapolating the line beyond the observed x-range, is not warranted at this sample size; real inference needs an adequate sample and assumption checks (linearity, independent errors of roughly constant variance).

Implementation notes

  • ddof cancellation: slope = cov_sum/var_sum where both are unnormalized (Σ products, not divided by n−1). The n−1 factors cancel, so the result equals np.cov(x,y)[0,1]/np.var(x,ddof=1) exactly.
  • scipy.stats.linregress uses a numerically stable internal algorithm; parity holds to 4 dp here since the data are small integers.
  • The r value from linregress equals Pearson r from ch04 for the same data.
  • Cross-reference: covariance-by-hand (ch04) for the cov_sum accumulation; pearson-correlation (ch04) for the r value linregress also reports.