Selecting and Filtering

Filter with isin

Keep rows whose category field appears in an allowed-values list by testing cat in allowed for each record in a loop. With pandas, df['cat'].isin(allowed) tests membership elementwise and returns a boolean Series, replacing a chain of == comparisons with a single readable call.

By hand

Loop over records. Test r['cat'] in allowed and append the name to names when the membership test passes.

naive.py
Replay: real traced execution (multi-file project)
# trace: ignore records
records = [
    {'name': 'al', 'cat': 'A'},
    {'name': 'bo', 'cat': 'B'},
    {'name': 'cy', 'cat': 'A'},
    {'name': 'di', 'cat': 'C'},
]
allowed = ['A', 'C']
names = []
for r in records:
    if r['cat'] in allowed:
        names.append(r['name'])
print('RESULT:', names)

  1. {'name': 'al', 'cat': 'A'},

    2records = [3    {'name': 'al', 'cat': 'A'},4    {'name': 'bo', 'cat': 'B'},

  2. {'name': 'bo', 'cat': 'B'},

    3{'name': 'al', 'cat': 'A'},4{'name': 'bo', 'cat': 'B'},5{'name': 'cy', 'cat': 'A'},

  3. {'name': 'cy', 'cat': 'A'},

    4{'name': 'bo', 'cat': 'B'},5{'name': 'cy', 'cat': 'A'},6{'name': 'di', 'cat': 'C'},

  4. {'name': 'di', 'cat': 'C'},

    5    {'name': 'cy', 'cat': 'A'},6    {'name': 'di', 'cat': 'C'},7]

  5. records = [

    1# trace: ignore records2records = [3    {'name': 'al', 'cat': 'A'},

  6. allowed ← ['A', 'C']

    7]8allowed = ['A', 'C']9names = []
    values this step['A', 'C']allowed

  7. names ← []

    8allowed = ['A', 'C']9names = []10for r in records:
    values this step[]names

  8. r ← {'name': 'al', 'cat': 'A'}

    9names = []10for r in records:11    if r['cat'] in allowed:
    values this step{'name': 'al', 'cat': 'A'}r

  9. if r['cat'] in allowed:

    10for r in records:11    if r['cat'] in allowed:12        names.append(r['name'])

  10. names ← ['al']

    11    if r['cat'] in allowed:12        names.append(r['name'])13print('RESULT:', names)
    values this step[] → ['al']names

  11. r ← {'name': 'bo', 'cat': 'B'}

    9names = []10for r in records:11    if r['cat'] in allowed:
    values this step{'name': 'al', 'cat': 'A'} → {'name': 'bo', 'cat': 'B'}r

  12. if r['cat'] in allowed:

    10for r in records:11    if r['cat'] in allowed:12        names.append(r['name'])

  13. r ← {'name': 'cy', 'cat': 'A'}

    9names = []10for r in records:11    if r['cat'] in allowed:
    values this step{'name': 'bo', 'cat': 'B'} → {'name': 'cy', 'cat': 'A'}r

  14. if r['cat'] in allowed:

    10for r in records:11    if r['cat'] in allowed:12        names.append(r['name'])

  15. names ← ['al', 'cy']

    11    if r['cat'] in allowed:12        names.append(r['name'])13print('RESULT:', names)
    values this step['al'] → ['al', 'cy']names

  16. r ← {'name': 'di', 'cat': 'C'}

    9names = []10for r in records:11    if r['cat'] in allowed:
    values this step{'name': 'cy', 'cat': 'A'} → {'name': 'di', 'cat': 'C'}r

  17. if r['cat'] in allowed:

    10for r in records:11    if r['cat'] in allowed:12        names.append(r['name'])

  18. names ← ['al', 'cy', 'di']

    11    if r['cat'] in allowed:12        names.append(r['name'])13print('RESULT:', names)
    values this step['al', 'cy'] → ['al', 'cy', 'di']names

  19. for r in records:

    9names = []10for r in records:11    if r['cat'] in allowed:

  20. stdout ← RESULT: ['al', 'cy', 'di']

    12        names.append(r['name'])13print('RESULT:', names)
    values this stepRESULT: ['al', 'cy', 'di']stdout

With pandas

df['cat'].isin(allowed) produces a boolean Series — True where the column value is in allowed, False elsewhere. Indexing the DataFrame with it keeps only the matching rows.

library.py 
import pandas as pd
from dalib.display import set_display
set_display()

records = [
    {'name': 'al', 'cat': 'A'},
    {'name': 'bo', 'cat': 'B'},
    {'name': 'cy', 'cat': 'A'},
    {'name': 'di', 'cat': 'C'},
]
allowed = ['A', 'C']
df = pd.DataFrame(records)
mask = df['cat'].isin(allowed)
filtered = df[mask]
print('mask:', mask.tolist())
print('names:', filtered['name'].tolist())
print('shape:', filtered.shape)
print('RESULT:', filtered['name'].tolist())

mask: [True, False, True, True]
names: ['al', 'cy', 'di']
shape: (3, 2)
RESULT: ['al', 'cy', 'di']

Implementation notes

  • .isin(values) accepts any iterable — list, set, or another Series. Pandas converts the input to an internal lookup structure regardless, so the choice of list vs set does not affect .isin performance. (In the naive loop, using a set for allowed would make each in test O(1) instead of O(n) — that speedup belongs to the pure-Python half, not to pandas.)
  • The alternative (df['cat'] == 'A') | (df['cat'] == 'C') is equivalent but verbose; .isin scales cleanly to any number of allowed values.
  • To exclude a set of values instead of including them, negate the mask: df[~df['cat'].isin(excluded)]. The ~ operator inverts a boolean Series elementwise.