Correlation and a Naive Regression on Retail Sales

data/sales.csv is 156 weeks (3 years) of one retail store's data: units sold, price, weekly marketing spend, a holiday-week flag, and marketing spend shifted 2 weeks earlier (marketing_lag2, used in the next lesson). | week | units_sold | price | marketing | holiday | marketing_lag2 | |---|---|---|---|---|---| | 0 | 1111 | 24.91 | 501.37 | 0 | 501.37 | | 1 | 1060 | 24.79 | 555.35 | 0 | 501.37 | | 2 | 1149 | 24.75 | 587.73 | 0 | 501.37 | | 47 | 1619 | 24.76 | 655.7 | 1 | 594.27 | Question: does marketing spend actually move sales, or does any apparent relationship just come from both rising around the holidays?

lmda corr runs Pearson correlation (with two-sided p-values) over every numeric column at once — a first honest look before fitting anything:

lmda corr data/sales.csv --format json

{
  "columns": [
    "week",
    "units_sold",
    "price",
    "marketing",
    "holiday",
    "marketing_lag2"
  ],
  "matrix": [
    [
      1.0,
      0.4624819678943279,
      -0.059953865650270215,
      0.18930308879710447,
      0.17020757459875585,
      0.1721356517070552
    ],
    [
      0.4624819678943279,
      1.0,
      -0.2582387915547417,
      0.5580395333826077,
      0.8793451086879311,
      0.6514872312194679
    ],
    [
      -0.059953865650270215,
      -0.2582387915547417,
      1.0,
      -0.17572072145860895,
      -0.1879852160611071,
      -0.27088211850623833
    ],
    [
      0.18930308879710447,
      0.5580395333826077,
      -0.17572072145860895,
      1.0,
      0.6267025325129743,
      0.6289092748532162
    ],
    [
      0.17020757459875585,
      0.8793451086879311,
      -0.1879852160611071,
      0.6267025325129743,
      1.0,
      0.6618120018565725
    ],
    [
      0.1721356517070552,
      0.6514872312194679,
      -0.27088211850623833,
      0.6289092748532162,
      0.6618120018565725,
      1.0
    ]
  ],
  "p_values": [
    [
      0.0,
      1.215954950750804e-09,
      0.45719702705044496,
      0.017941617654705328,
      0.033643350535673994,
      0.031654378433243084
    ],
    [
      1.215954950750804e-09,
      0.0,
      0.001134522077594747,
      3.783740484145372e-14,
      1.7376645508176486e-51,
      3.289293261023238e-20
    ],
    [
      0.45719702705044496,
      0.001134522077594747,
      0.0,
      0.02822088274609889,
      0.018771271101972326,
      0.0006255633063202704
    ],
    [
      0.017941617654705328,
      3.783740484145372e-14,
      0.02822088274609889,
      0.0,
      2.11433918979253e-18,
      1.481695147104609e-18
    ],
    [
      0.033643350535673994,
      1.7376645508176486e-51,
      0.018771271101972326,
      2.11433918979253e-18,
      0.0,
      5.166826145809926e-21
    ],
    [
      0.031654378433243084,
      3.289293261023238e-20,
      0.0006255633063202704,
      1.481695147104609e-18,
      5.166826145809926e-21,
      0.0
    ]
  ]
}

Reading the units_sold row: marketing correlates at ≈0.558 (p≈3.8e-14, clearly real) and holiday at ≈0.879 (p≈1.7e-51, the strongest relationship in the table). But marketing and holiday also correlate with each other at ≈0.627 — marketing spend rises around holidays too, which is exactly the confound the question above is worried about. Correlation alone can't separate "marketing works" from "marketing and sales both rise together at the holidays."

lmda regress fits units_sold on price, marketing, and holiday together, so each coefficient is marketing's/price's/holiday's effect holding the others fixed — the direct way to try to untangle the correlation above:

lmda regress data/sales.csv --y units_sold --x price,marketing,holiday --format json

{
  "adjusted_r_squared": 0.7779056337089785,
  "coefficients": [
    {
      "estimate": 1785.0329361634,
      "p_value": 2.981387573561904e-13,
      "std_error": 223.18319941571056,
      "t_statistic": 7.998061416973065,
      "term": "intercept"
    },
    {
      "estimate": -21.465805517833886,
      "p_value": 0.013887738723300526,
      "std_error": 8.624246095677488,
      "t_statistic": -2.4890066076143915,
      "term": "price"
    },
    {
      "estimate": 0.0045599407813512955,
      "p_value": 0.9628369294829876,
      "std_error": 0.09770453140030816,
      "t_statistic": 0.046670719525470375,
      "term": "marketing"
    },
    {
      "estimate": 428.5458370997386,
      "p_value": 1.5713570672487912e-38,
      "std_error": 24.336142360242462,
      "t_statistic": 17.609439933251153,
      "term": "holiday"
    }
  ],
  "dropped": 0,
  "method": "ols",
  "n": 156,
  "r_squared": 0.7822042343468693,
  "x": [
    "price",
    "marketing",
    "holiday"
  ],
  "y": "units_sold"
}

The holiday coefficient is large, precise, and highly significant (≈+429 units, p≈1.6e-38) — a large, precisely-estimated holiday association in this fit (not yet a confirmed causal effect — see Honesty). price is negative and significant (≈-21.5 units per dollar, p≈0.014), the expected demand-curve direction. But marketing's coefficient is tiny and its p-value is ≈0.96 — once holiday and price are in the model, this regression finds no evidence marketing spend moves sales at all, despite the raw 0.558 correlation above. That's the honest, if uncomfortable, naive-model answer — and the next lesson (regdiag-marketing-lag-durbin-watson) shows why it isn't the end of the story.

Every number here is a real OLS fit on 156 realistically simulated weeks, not a toy — this is large enough for the coefficient estimates and p-values to be meaningful in the way least-squares-line's 5-point fit cannot be. But "meaningful" is not the same as "final": this is a naive regression that ignores a documented marketing→sales lag (spend this week may move sales 1-3 weeks later, not this week — see the next lesson), and it does not check whether OLS's assumptions (independent, homoscedastic residuals) even hold — weekly retail data is exactly the kind of series where they often don't. Read "marketing shows no effect here" as "this specific, unlagged, undiagnosed model shows no effect" — not as a general claim that marketing doesn't work. The next lesson runs the diagnostics this one skips. Likewise, holiday's large, significant coefficient is a real association in this fit, not a demonstrated causal claim — an OLS coefficient on observational data describes what moves together, not what a controlled intervention would do. As with the rest of this chapter, sales.csv is a realistic simulated retail dataset (lmda's own case-study-4 worked example, seeded RNG), not an actual store's live data — the fit is real, the store is not.

• Both commands are re-run against the committed CSV on every validate_source.py pass and must byte-for-byte match the checked-in outputs/*.json.
• corr reports every numeric column, including week (a trend proxy) and marketing_lag2 — shown in full rather than a hand-picked subset, so nothing is cherry-picked out of the real output.
• Cross-reference: r-squared and least-squares-line (ch08) for the single-predictor mechanics this generalizes; regdiag-marketing-lag-durbin-watson (this chapter) for why this naive model's p-values shouldn't be trusted at face value.