Outliers and Ranges

Range Flag

Flag values that fall outside an allowed band. Given a list of numbers and bounds [lo, hi], mark each value True if it is below lo or above hi, False otherwise. By hand, evaluate the two-sided condition with or in a loop. With pandas, combine two boolean Series with | to produce the same flags in one expression.

By hand

Loop over the values and append v < lo or v > hi to result. The condition is True for any value that violates either bound. Python evaluates the left side first and short-circuits if it is already True, so under-range values (3 and 2) never test the upper bound.

naive.py
Replay: real traced execution (multi-file project)
values = [3, 15, 7, 42, 11, 2, 28, 9]
lo = 5
hi = 25
result = []
for v in values:
    result.append(v < lo or v > hi)
print('RESULT:', result)

  1. values ← [3, 15, 7, 42, 11, 2, 28, 9]

    1values = [3, 15, 7, 42, 11, 2, 28, 9]2lo = 5
    values this step[3, 15, 7, 42, 11, 2, 28, 9]values

  2. lo ← 5

    1values = [3, 15, 7, 42, 11, 2, 28, 9]2lo = 53hi = 25
    values this step5lo

  3. hi ← 25

    2lo = 53hi = 254result = []
    values this step25hi

  4. result ← []

    3hi = 254result = []5for v in values:
    values this step[]result

  5. v ← 3

    4result = []5for v in values:6    result.append(v < lo or v > hi)
    values this step3v

  6. result ← [True]

    5for v in values:6    result.append(v < lo or v > hi)7print('RESULT:', result)
    values this step[] → [True]result

  7. v ← 15

    4result = []5for v in values:6    result.append(v < lo or v > hi)
    values this step3 → 15v

  8. result ← [True, False]

    5for v in values:6    result.append(v < lo or v > hi)7print('RESULT:', result)
    values this step[True] → [True, False]result

  9. v ← 7

    4result = []5for v in values:6    result.append(v < lo or v > hi)
    values this step15 → 7v

  10. result ← [True, False, False]

    5for v in values:6    result.append(v < lo or v > hi)7print('RESULT:', result)
    values this step[True, False] → [True, False, False]result

  11. v ← 42

    4result = []5for v in values:6    result.append(v < lo or v > hi)
    values this step7 → 42v

  12. result ← [True, False, False, True]

    5for v in values:6    result.append(v < lo or v > hi)7print('RESULT:', result)
    values this step[True, False, False] → [True, False, False, True]result

  13. v ← 11

    4result = []5for v in values:6    result.append(v < lo or v > hi)
    values this step42 → 11v

  14. result ← [True, False, False, True, False]

    5for v in values:6    result.append(v < lo or v > hi)7print('RESULT:', result)
    values this step[True, False, False, True] → [True, False, False, True, False]result

  15. v ← 2

    4result = []5for v in values:6    result.append(v < lo or v > hi)
    values this step11 → 2v

  16. result ← [True, False, False, True, False, True]

    5for v in values:6    result.append(v < lo or v > hi)7print('RESULT:', result)
    values this step[True, False, False, True, False] → [True, False, False, True, False, True]result

  17. v ← 28

    4result = []5for v in values:6    result.append(v < lo or v > hi)
    values this step2 → 28v

  18. result ← [True, False, False, True, False, True, True]

    5for v in values:6    result.append(v < lo or v > hi)7print('RESULT:', result)
    values this step[True, False, False, True, False, True] → [True, False, False, True, False, True, True]result

  19. v ← 9

    4result = []5for v in values:6    result.append(v < lo or v > hi)
    values this step28 → 9v

  20. result ← [True, False, False, True, False, True, True, False]

    5for v in values:6    result.append(v < lo or v > hi)7print('RESULT:', result)
    values this step[True, False, False, True, False, True, True] → [True, False, False, True, False, True, True, False]result

  21. for v in values:

    4result = []5for v in values:6    result.append(v < lo or v > hi)

  22. stdout ← RESULT: [True, False, False, True, False, True, True, False]

    6    result.append(v < lo or v > hi)7print('RESULT:', result)
    values this stepRESULT: [True, False, False, True, False, True, True, False]stdout

With pandas

Build two boolean Series — df['x'] < lo flags values below the lower bound, df['x'] > hi flags values above the upper bound — then combine them with |. Parentheses around each comparison are required because | has higher operator precedence than < and >.

library.py 
import pandas as pd
from dalib.display import set_display
set_display()

values = [3, 15, 7, 42, 11, 2, 28, 9]
lo = 5
hi = 25
df = pd.DataFrame({'x': values})
flags = (df['x'] < lo) | (df['x'] > hi)
result = flags.tolist()
print('index:', flags.index.tolist())
print('dtype:', flags.dtype)
print('values:', flags.tolist())
print('RESULT:', result)

index: [0, 1, 2, 3, 4, 5, 6, 7]
dtype: bool
values: [True, False, False, True, False, True, True, False]
RESULT: [True, False, False, True, False, True, True, False]

Implementation notes

  • Parentheses around each comparison are mandatory: (df['x'] < lo) | (df['x'] > hi). Without them, Python parses the expression incorrectly because | binds tighter than < and >.
  • To flag values within the band instead, negate with ~flags, or use (df['x'] >= lo) & (df['x'] <= hi) directly.
  • Cross-reference: combine-two-masks (numpy chapter) for the equivalent np.logical_or approach.