Linear Regression

Normal Equation (1D)

Fit a line y=slope·x+intercept using the closed-form 1D normal equation: slope = Σ(xᵢ−x̄)(yᵢ−ȳ) / Σ(xᵢ−x̄)², intercept = ȳ−slope·x̄. A single loop accumulates numerator and denominator simultaneously. Library: sklearn.linear_model.LinearRegression().fit(X, y) with X as a 2D column. RESULT: (slope, intercept) rounded.

By hand

With scikit-learn

LinearRegression().fit(X, y) requires X as a 2D array — one column per feature. coef_[0] is the slope; intercept_ is the intercept.

naive.py 
x = [1, 2, 3, 4, 5]
y = [2, 4, 4, 4, 6]
n = len(x)
mx = sum(x) / n
my = sum(y) / n
num = 0.0
den = 0.0
for i in range(n):
    dx = x[i] - mx
    num = num + dx * (y[i] - my)
    den = den + dx * dx
slope = num / den
intercept = my - slope * mx
print('RESULT:', (round(slope, 4), round(intercept, 4)))
library.py 
from sklearn.linear_model import LinearRegression
from dalib.display import set_display
set_display()

x = [1, 2, 3, 4, 5]
y = [2, 4, 4, 4, 6]
X = [[v] for v in x]
model = LinearRegression()
model.fit(X, y)
slope = round(float(model.coef_[0]), 4)
intercept = round(float(model.intercept_), 4)
print('coef_:', slope)
print('intercept_:', intercept)
print('RESULT:', (slope, intercept))

coef_: 0.8
intercept_: 1.6
RESULT: (0.8, 1.6)

Honesty

This lesson shows the closed-form fit computed exactly, on a tiny pinned sample. The slope and intercept are arithmetically correct and reproducible, but a fit to a handful of points is a mechanism demo, not a predictive model — it is not a statistical finding. Extrapolating the line beyond the observed x-range, or reading significance into the slope, is unwarranted at this sample size; real inference needs an adequate sample and assumption checks.

Implementation notes

  • The closed-form normal equation finds the exact least-squares solution in one pass — no gradient descent or iteration needed.
  • Cross-reference: least-squares-line (python-stats ch08) derives the same formulas by hand; this lesson wraps them in the sklearn API.
  • LinearRegression expects X as a 2D array (rows=samples, columns=features); a flat 1D list raises a ValueError.
  • RESULT tuple (slope, intercept) matches coef_[0] and intercept_ exactly (both rounded to 4 d.p.).
  • Same data as python-stats ch08 intentionally: slope=0.8, intercept=1.6 confirms consistency across both books.