Selecting and Filtering

Filter with Multiple Conditions

Combine two per-row boolean tests with Python's and in a loop to build a single mask, then keep matching rows. With pandas, element-wise & combines two boolean Series before indexing the DataFrame — and and or are not valid for Series and will raise an error.

By hand

Loop over records, apply both conditions with and, and append the combined bool to mask. Loop over indices to collect names where mask[i] is True.

naive.py 
# trace: ignore records
records = [
    {'name': 'al', 'age': 20, 'score': 55},
    {'name': 'bo', 'age': 25, 'score': 82},
    {'name': 'cy', 'age': 22, 'score': 71},
    {'name': 'di', 'age': 28, 'score': 90},
]
lo, hi = 21, 88
mask = []
for r in records:
    mask.append(r['age'] > lo and r['score'] < hi)
names = []
for i in range(len(records)):
    if mask[i]:
        names.append(records[i]['name'])
print('RESULT:', names)

With pandas

(df['age'] > lo) & (df['score'] < hi) combines two boolean Series elementwise. Parentheses around each condition are required because & has higher operator precedence than > and <.

library.py 
import pandas as pd
from dalib.display import set_display
set_display()

records = [
    {'name': 'al', 'age': 20, 'score': 55},
    {'name': 'bo', 'age': 25, 'score': 82},
    {'name': 'cy', 'age': 22, 'score': 71},
    {'name': 'di', 'age': 28, 'score': 90},
]
lo, hi = 21, 88
df = pd.DataFrame(records)
mask = (df['age'] > lo) & (df['score'] < hi)
filtered = df[mask]
print('mask:', mask.tolist())
print('names:', filtered['name'].tolist())
print('shape:', filtered.shape)
print('RESULT:', filtered['name'].tolist())

mask: [False, True, True, False]
names: ['bo', 'cy']
shape: (2, 3)
RESULT: ['bo', 'cy']

Implementation notes

  • Use & (bitwise AND) and | (bitwise OR) to combine boolean Series — NOT Python's and / or. and/or call bool() on the whole Series, which raises ValueError: The truth value of a Series is ambiguous.
  • Parentheses around each condition are mandatory: (df['a'] > lo) & (df['b'] < hi). Without them, & binds tighter than > and the expression parses incorrectly.
  • This is the same operator rule as NumPy: (a > lo) & (a < hi) — see combine-two-masks (python-numpy ch06).
  • For more than two conditions, chain & or |: (c1) & (c2) & (c3). Alternatively, df.query("age > 21 and score < 88") allows Python-style and/or syntax inside a string.