Data Pipeline Patterns

Reduce Totals

Reduction combines many values into a single result.

reducing values A reduce stage carries an accumulator forward as it visits each value.

Reduce Totals

include_shipping
reduce_totals.py
Replay: real traced execution (multi-file project)
amounts = [12, 8, 15]

include_shipping = False

total = 0
for amount in amounts:
    total += amount

if include_shipping:
    total += 5

print(f"total={total}")

amounts = [12, 8, 15]

include_shipping = True

total = 0
for amount in amounts:
    total += amount

if include_shipping:
    total += 5

print(f"total={total}")

  1. amounts ← [12, 8, 15], include_shipping ← False, total ← 0

    1amounts→ [12, 8, 15] = [12, 8, 15]23include_shipping→ False = False  #@include_shipping=True45total→ 0 = 06for amount in amounts:

  2. total ← 12

    pass 1 of 3
    5total = 06for amount12 in amounts[12, 8, 15]:7    total→ 12 += amount12
    All 3 passes — pass 1 is the card above
    passamounttotal
    1120 → 12
    2812 → 20
    31520 → 35

  3. print(f"total={total}")

    12print(f"total={total35}")
    outputtotal=35

  1. amounts ← [12, 8, 15], include_shipping ← True, total ← 0

    1amounts→ [12, 8, 15] = [12, 8, 15]23include_shipping→ True = True45total→ 0 = 06for amount in amounts:

  2. total ← 12

    pass 1 of 3
    5total = 06for amount12 in amounts[12, 8, 15]:7    total→ 12 += amount12
    All 3 passes — pass 1 is the card above
    passamountinclude_shippingtotal
    112—0 → 12
    28—12 → 20
    315True20 → 35

  3. total ← 40

    9if include_shippingTrue:10    total→ 40 += 5

  4. print(f"total={total}")

    12print(f"total={total40}")
    outputtotal=40