Data Validation Pipelines

Missing Code Filter

Count Usable Values

Scientific data often uses sentinel values before a richer missing-data format is introduced. A validation pass can count the usable readings.

Program

Play the program to choose the sentinel code and see how the valid-count summary changes.

missing_code
missing_code_filter.f90
Replay: real traced execution (multi-file project)
program missing_code_filter_demo
    implicit none
    integer :: readings(5)
    integer :: missing_code
    integer :: usable_count
    integer :: usable_total
    logical :: usable(5)

    readings = [12, -99, 15, 18, -99]
    missing_code = -99
    usable = readings /= missing_code
    usable_count = count(usable)
    usable_total = sum(readings, mask=usable)
    print '(I0, 1X, I0)', usable_count, usable_total
end program missing_code_filter_demo

program missing_code_filter_demo
    implicit none
    integer :: readings(5)
    integer :: missing_code
    integer :: usable_count
    integer :: usable_total
    logical :: usable(5)

    readings = [12, -99, 15, 18, -99]
    missing_code = -1
    usable = readings /= missing_code
    usable_count = count(usable)
    usable_total = sum(readings, mask=usable)
    print '(I0, 1X, I0)', usable_count, usable_total
end program missing_code_filter_demo

program missing_code_filter_demo
    implicit none
    integer :: readings(5)
    integer :: missing_code
    integer :: usable_count
    integer :: usable_total
    logical :: usable(5)

    readings = [12, -99, 15, 18, -99]
    missing_code = 15
    usable = readings /= missing_code
    usable_count = count(usable)
    usable_total = sum(readings, mask=usable)
    print '(I0, 1X, I0)', usable_count, usable_total
end program missing_code_filter_demo

  1. readings ← [12, -99, 15, 18, -99]

    9readings = [12, -99, 15, 18, -99]10missing_code = -99
    values this step[12, -99, 15, 18, -99]readings

  2. missing_code ← -99

    9readings = [12, -99, 15, 18, -99]10missing_code = -9911usable = readings /= missing_code
    values this step-99missing_code

  3. usable ← [T, F, T, T, F]

    10missing_code = -9911usable = readings /= missing_code12usable_count = count(usable)
    values this step[T, F, T, T, F]usable[12, -99, 15, 18, -99]readings-99missing_code

  4. usable_count ← 3

    11usable = readings /= missing_code12usable_count = count(usable)13usable_total = sum(readings, mask=usable)
    values this step3usable_count[T, F, T, T, F]usable

  5. usable_total ← 45

    12usable_count = count(usable)13usable_total = sum(readings, mask=usable)14print '(I0, 1X, I0)', usable_count, usable_total
    values this step45usable_total[12, -99, 15, 18, -99]readings[T, F, T, T, F]usable

  6. print '(I0, 1X, I0)', usable_count, usable_total

    13    usable_total = sum(readings, mask=usable)14    print '(I0, 1X, I0)', usable_count, usable_total15end program missing_code_filter_demo
    output3 45
    values this step3usable_count45usable_total

  1. readings ← [12, -99, 15, 18, -99]

    9readings = [12, -99, 15, 18, -99]10missing_code = -1
    values this step[12, -99, 15, 18, -99]readings

  2. missing_code ← -1

    9readings = [12, -99, 15, 18, -99]10missing_code = -111usable = readings /= missing_code
    values this step-1missing_code

  3. usable ← [T, T, T, T, T]

    10missing_code = -111usable = readings /= missing_code12usable_count = count(usable)
    values this step[T, T, T, T, T]usable[12, -99, 15, 18, -99]readings-1missing_code

  4. usable_count ← 5

    11usable = readings /= missing_code12usable_count = count(usable)13usable_total = sum(readings, mask=usable)
    values this step5usable_count[T, T, T, T, T]usable

  5. usable_total ← -153

    12usable_count = count(usable)13usable_total = sum(readings, mask=usable)14print '(I0, 1X, I0)', usable_count, usable_total
    values this step-153usable_total[12, -99, 15, 18, -99]readings[T, T, T, T, T]usable

  6. print '(I0, 1X, I0)', usable_count, usable_total

    13    usable_total = sum(readings, mask=usable)14    print '(I0, 1X, I0)', usable_count, usable_total15end program missing_code_filter_demo
    output5 -153
    values this step5usable_count-153usable_total

  1. readings ← [12, -99, 15, 18, -99]

    9readings = [12, -99, 15, 18, -99]10missing_code = 15
    values this step[12, -99, 15, 18, -99]readings

  2. missing_code ← 15

    9readings = [12, -99, 15, 18, -99]10missing_code = 1511usable = readings /= missing_code
    values this step15missing_code

  3. usable ← [T, T, F, T, T]

    10missing_code = 1511usable = readings /= missing_code12usable_count = count(usable)
    values this step[T, T, F, T, T]usable[12, -99, 15, 18, -99]readings15missing_code

  4. usable_count ← 4

    11usable = readings /= missing_code12usable_count = count(usable)13usable_total = sum(readings, mask=usable)
    values this step4usable_count[T, T, F, T, T]usable

  5. usable_total ← -168

    12usable_count = count(usable)13usable_total = sum(readings, mask=usable)14print '(I0, 1X, I0)', usable_count, usable_total
    values this step-168usable_total[12, -99, 15, 18, -99]readings[T, T, F, T, T]usable

  6. print '(I0, 1X, I0)', usable_count, usable_total

    13    usable_total = sum(readings, mask=usable)14    print '(I0, 1X, I0)', usable_count, usable_total15end program missing_code_filter_demo
    output4 -168
    values this step4usable_count-168usable_total
sentinel A sentinel value marks readings that should be excluded.
logical mask `readings /= missing_code` creates a mask of usable positions.
masked sum `sum(readings, mask=usable)` ignores values that are not usable.