Missing Values

Fill with Column Mean

Replace missing values with the mean of the observed (non-missing) values in the same column. By hand, compute the mean in a first pass (sum and count of non-None values), then fill in a second pass. With pandas, Series.fillna (Series.mean()) combines both steps — mean() skips NaN by default.

By hand

First pass: accumulate total and count over the non-None entries, then mean = round(total / count, 2). Second pass: walk values again and append each element unchanged or substitute mean for None. The trace shows total and count climbing in the first loop and filled growing in the second.

naive.py
Replay: real traced execution (multi-file project)
values = [3.1, None, 7.2, None, 5.0, None, 8.4, 2.9]
total = 0.0
count = 0
for v in values:
    if v is not None:
        total = total + v
        count = count + 1
mean = round(total / count, 2)
filled = []
for v in values:
    filled.append(v if v is not None else mean)
print('RESULT:', [round(v, 2) for v in filled])

  1. values ← [3.1, None, 7.2, None, 5.0, None, 8.4, 2.9]

    1values = [3.1, None, 7.2, None, 5.0, None, 8.4, 2.9]2total = 0.0
    values this step[3.1, None, 7.2, None, 5.0, None, 8.4, 2.9]values

  2. total ← 0.0

    1values = [3.1, None, 7.2, None, 5.0, None, 8.4, 2.9]2total = 0.03count = 0
    values this step0.0total

  3. count ← 0

    2total = 0.03count = 04for v in values:
    values this step0count

  4. v ← None, total ← 3.1, count ← 1

    pass 1 of 4
    3count = 04for v in values:5    if v is not None:6        total = total + v7        count = count + 18mean = round(total / count, 2)
    values this step3.1 → Nonev0.0 → 3.1total0 → 1count
    All 4 passes — pass 1 is the card above
    passvtotalcount
    13.1 → None0.0 → 3.10 → 1
    27.2 → None3.1 → 10.31 → 2
    35.0 → None10.3 → 15.32 → 3
    48.4 → 2.915.3 → 23.7000000000000033 → 4

  5. total ← 26.6

    5if v is not None:6    total = total + v7    count = count + 1
    values this step23.700000000000003 → 26.6total

  6. count ← 5

    6        total = total + v7        count = count + 18mean = round(total / count, 2)
    values this step4 → 5count

  7. for v in values:

    3count = 04for v in values:5    if v is not None:

  8. mean ← 5.32

    7        count = count + 18mean = round(total / count, 2)9filled = []
    values this step5.32mean

  9. filled ← []

    8mean = round(total / count, 2)9filled = []10for v in values:
    values this step[]filled

  10. v ← 3.1, filled ← [3.1]

    pass 1 of 8
    9filled = []10for v in values:11    filled.append(v if v is not None else mean)12print('RESULT:', [round(v, 2) for v in filled])
    values this step2.9 → 3.1v[] → [3.1]filled
    All 8 passes — pass 1 is the card above
    passvfilled
    12.9 → 3.1[] → [3.1]
    23.1 → None[3.1] → [3.1, 5.32]
    3None → 7.2[3.1, 5.32] → [3.1, 5.32, 7.2]
    47.2 → None[3.1, 5.32, 7.2] → [3.1, 5.32, 7.2, 5.32]
    5None → 5.0[3.1, 5.32, 7.2, 5.32] → [3.1, 5.32, 7.2, 5.32, 5.0]
    65.0 → None[3.1, 5.32, 7.2, 5.32, 5.0] → [3.1, 5.32, 7.2, 5.32, 5.0, 5.32]
    7None → 8.4[3.1, 5.32, 7.2, 5.32, 5.0, 5.32] → [3.1, 5.32, 7.2, 5.32, 5.0, 5.32, 8.4]
    88.4 → 2.9[3.1, 5.32, 7.2, 5.32, 5.0, 5.32, 8.4] → [3.1, 5.32, 7.2, 5.32, 5.0, 5.32, 8.4, 2.9]

  11. for v in values:

    9filled = []10for v in values:11    filled.append(v if v is not None else mean)

  12. stdout ← RESULT: [3.1, 5.32, 7.2, 5.32, 5.0, 5.32, 8.4, 2.9]

    11    filled.append(v if v is not None else mean)12print('RESULT:', [round(v, 2) for v in filled])
    values this stepRESULT: [3.1, 5.32, 7.2, 5.32, 5.0, 5.32, 8.4, 2.9]stdout

With pandas

s.mean() returns the mean of all non-NaN entries (skipna is True by default). Passing it straight to fillna substitutes that value at every NaN position. The snapshot shows the original series under before: and the computed mean so both substitution points are visible.

library.py 
import pandas as pd
from dalib.display import set_display
set_display()

values = [3.1, None, 7.2, None, 5.0, None, 8.4, 2.9]
s = pd.Series(values, dtype=float)
m = round(s.mean(), 2)
filled = s.fillna(m)
result = [round(v, 2) for v in filled.tolist()]
print('index:', s.index.tolist())
print('dtype:', s.dtype)
print('before:', s.tolist())
print('mean:', m)
print('RESULT:', result)

index: [0, 1, 2, 3, 4, 5, 6, 7]
dtype: float64
before: [3.1, nan, 7.2, nan, 5.0, nan, 8.4, 2.9]
mean: 5.32
RESULT: [3.1, 5.32, 7.2, 5.32, 5.0, 5.32, 8.4, 2.9]

Implementation notes

  • Series.mean() has skipna=True by default — it ignores NaN when computing the average, which is exactly the behaviour the naive loop replicates with the if v is not None guard.
  • Mean imputation preserves the column mean but reduces variance and can distort correlations. It is most appropriate when data is missing at random and the proportion of missing values is small.
  • For a median or mode fill instead of mean, use s.fillna(s.median()) or s.fillna(s.mode()[0]).
  • Cross-reference: fill-constant (this chapter) for the simpler case of filling with a fixed value rather than a computed statistic.
  • Cross-reference: list-sum-mean (python-data-basics) for the underlying sum-and-divide mean computation this naive half applies.