Clean and Summarize - Data Cleaning Step by Step

Capstone: chain cleaning and aggregation. Given parallel category and value lists with one missing value, drop the row with the missing value, then produce a per-category sum. By hand, skip None values while accumulating into a running total dict. With pandas, chain dropna → groupby → sum in three calls.

By hand

With pandas

df.dropna(subset=['val']) removes the row where val is NaN (one row dropped: shape goes from (6, 2) to (5, 2)). groupby('cat')['val'].sum() aggregates per category. The sum comes back as float64 because the column dtype is float (NaN forces float promotion); int(v) converts each value back to a plain integer to match the naive result.

naive.py

cats   = ['A', 'B', 'A', 'B', 'A', 'B']
values = [10, 20, None, 30, 15, 25]
total = {}
for i in range(len(cats)):
    if values[i] is not None:
        c = cats[i]
        total[c] = total.get(c, 0) + values[i]
result = dict(sorted(total.items()))
print('RESULT:', result)

library.py

import pandas as pd
from dalib.display import set_display
set_display()

cats   = ['A', 'B', 'A', 'B', 'A', 'B']
values = [10, 20, None, 30, 15, 25]
df = pd.DataFrame({'cat': cats, 'val': values})
clean = df.dropna(subset=['val'])
s = clean.groupby('cat')['val'].sum()
result = {k: int(v) for k, v in sorted(s.items())}
print('shape before:', df.shape)
print('shape after:', clean.shape)
print('groups:', s.to_dict())
print('RESULT:', result)

shape before: (6, 2)
shape after: (5, 2)
groups: {'A': 25.0, 'B': 75.0}
RESULT: {'A': 25, 'B': 75}

Implementation notes

A column containing None in a Python list becomes float64 in pandas (NaN forces float promotion). That is why sum() returns floats here even though all real values are integers; int() converts them back for a clean result.
Cross-reference: drop-missing-rows (ch01) for the dropna step that removes the None row before aggregation.