Data Pipeline Case Studies
Normalize Input
Normalize small input labels before counting useful records.
Normalize Input
normalize_input.lua
local min_length =
local kept = 0
local labels = ""
for _, raw in ipairs({" Ada ", "Lua", "Trace"}) do
local name = raw:lower():gsub("^%s+", ""):gsub("%s+$", "")
if #name >= min_length then
kept = kept + 1
labels = labels .. name .. ";"
end
end
print("min_length=" .. min_length)
print("kept=" .. kept)
print("labels=" .. labels)
local min_length =
local kept = 0
local labels = ""
for _, raw in ipairs({" Ada ", "Lua", "Trace"}) do
local name = raw:lower():gsub("^%s+", ""):gsub("%s+$", "")
if #name >= min_length then
kept = kept + 1
labels = labels .. name .. ";"
end
end
print("min_length=" .. min_length)
print("kept=" .. kept)
print("labels=" .. labels)
local min_length =
local kept = 0
local labels = ""
for _, raw in ipairs({" Ada ", "Lua", "Trace"}) do
local name = raw:lower():gsub("^%s+", ""):gsub("%s+$", "")
if #name >= min_length then
kept = kept + 1
labels = labels .. name .. ";"
end
end
print("min_length=" .. min_length)
print("kept=" .. kept)
print("labels=" .. labels)
normalization
Pipeline code often trims or lowers values before comparing them.
counting
The final count stays scalar so replay can show the pipeline result clearly.