Linear Regression

Normal Equation (1D)

Fit a line y=slope·x+intercept using the closed-form 1D normal equation: slope = Σ(xᵢ−x̄)(yᵢ−ȳ) / Σ(xᵢ−x̄)², intercept = ȳ−slope·x̄. A single loop accumulates numerator and denominator simultaneously. Library: sklearn.linear_model.LinearRegression().fit(X, y) with X as a 2D column. RESULT: (slope, intercept) rounded.

By hand

x=[1,2,3,4,5], y=[2,4,4,4,6]. x̄=3, ȳ=4. Numerator: (−2)(−2)+(−1)(0)+(0)(0)+(1)(0)+(2)(2)=8. Denominator: 4+1+0+1+4=10. slope=0.8, intercept=4−0.8·3=1.6.

naive.py
Replay: real traced execution (multi-file project)
x = [1, 2, 3, 4, 5]
y = [2, 4, 4, 4, 6]
n = len(x)
mx = sum(x) / n
my = sum(y) / n
num = 0.0
den = 0.0
for i in range(n):
    dx = x[i] - mx
    num = num + dx * (y[i] - my)
    den = den + dx * dx
slope = num / den
intercept = my - slope * mx
print('RESULT:', (round(slope, 4), round(intercept, 4)))

  1. x ← [1, 2, 3, 4, 5]

    1x = [1, 2, 3, 4, 5]2y = [2, 4, 4, 4, 6]
    values this step[1, 2, 3, 4, 5]x

  2. y ← [2, 4, 4, 4, 6]

    1x = [1, 2, 3, 4, 5]2y = [2, 4, 4, 4, 6]3n = len(x)
    values this step[2, 4, 4, 4, 6]y

  3. n ← 5

    2y = [2, 4, 4, 4, 6]3n = len(x)4mx = sum(x) / n
    values this step5n

  4. mx ← 3.0

    3n = len(x)4mx = sum(x) / n5my = sum(y) / n
    values this step3.0mx

  5. my ← 4.0

    4mx = sum(x) / n5my = sum(y) / n6num = 0.0
    values this step4.0my

  6. num ← 0.0

    5my = sum(y) / n6num = 0.07den = 0.0
    values this step0.0num

  7. den ← 0.0

    6num = 0.07den = 0.08for i in range(n):
    values this step0.0den

  8. i ← 0, dx ← -2.0, num ← 4.0, den ← 4.0

    pass 1 of 5
    7den = 0.08for i in range(n):9    dx = x[i] - mx10    num = num + dx * (y[i] - my)11    den = den + dx * dx12slope = num / den
    values this step0i-2.0dx0.0 → 4.0num0.0 → 4.0den
    All 5 passes — pass 1 is the card above
    passidxnumden
    10-2.00.0 → 4.00.0 → 4.0
    20 → 1-2.0 → -1.0—4.0 → 5.0
    31 → 2-1.0 → 0.0——
    42 → 30.0 → 1.0—5.0 → 6.0
    53 → 41.0 → 2.04.0 → 8.06.0 → 10.0

  9. for i in range(n):

    7den = 0.08for i in range(n):9    dx = x[i] - mx

  10. slope ← 0.8

    11    den = den + dx * dx12slope = num / den13intercept = my - slope * mx
    values this step0.8slope

  11. intercept ← 1.5999999999999996

    12slope = num / den13intercept = my - slope * mx14print('RESULT:', (round(slope, 4), round(intercept, 4)))
    values this step1.5999999999999996intercept

  12. stdout ← RESULT: (0.8, 1.6)

    13intercept = my - slope * mx14print('RESULT:', (round(slope, 4), round(intercept, 4)))
    values this stepRESULT: (0.8, 1.6)stdout

With scikit-learn

LinearRegression().fit(X, y) requires X as a 2D array — one column per feature. coef_[0] is the slope; intercept_ is the intercept.

library.py 
from sklearn.linear_model import LinearRegression
from dalib.display import set_display
set_display()

x = [1, 2, 3, 4, 5]
y = [2, 4, 4, 4, 6]
X = [[v] for v in x]
model = LinearRegression()
model.fit(X, y)
slope = round(float(model.coef_[0]), 4)
intercept = round(float(model.intercept_), 4)
print('coef_:', slope)
print('intercept_:', intercept)
print('RESULT:', (slope, intercept))

coef_: 0.8
intercept_: 1.6
RESULT: (0.8, 1.6)

Honesty

This lesson shows the closed-form fit computed exactly, on a tiny pinned sample. The slope and intercept are arithmetically correct and reproducible, but a fit to a handful of points is a mechanism demo, not a predictive model — it is not a statistical finding. Extrapolating the line beyond the observed x-range, or reading significance into the slope, is unwarranted at this sample size; real inference needs an adequate sample and assumption checks.

Implementation notes

  • The closed-form normal equation finds the exact least-squares solution in one pass — no gradient descent or iteration needed.
  • Cross-reference: least-squares-line (python-stats ch08) derives the same formulas by hand; this lesson wraps them in the sklearn API.
  • LinearRegression expects X as a 2D array (rows=samples, columns=features); a flat 1D list raises a ValueError.
  • RESULT tuple (slope, intercept) matches coef_[0] and intercept_ exactly (both rounded to 4 d.p.).
  • Same data as python-stats ch08 intentionally: slope=0.8, intercept=1.6 confirms consistency across both books.