Data Pipeline Patterns
Input Normalization
A data pipeline often starts by trimming and reshaping input into predictable fields.
Input Normalization
InputNormalization.java
public class InputNormalization {
record UserLabel(String displayName, String slug) {}
static UserLabel normalize(String rawName) {
String trimmed = rawName.trim();
String displayName = trimmed.replaceAll("\\s+", " ");
String slug = displayName.toLowerCase().replaceAll("\\s+", "-");
return new UserLabel(displayName, slug);
}
public static void main(String[] args) {
String rawName = ;
UserLabel label = normalize(rawName);
System.out.println("name=" + label.displayName());
System.out.println("slug=" + label.slug());
}
}
public class InputNormalization {
record UserLabel(String displayName, String slug) {}
static UserLabel normalize(String rawName) {
String trimmed = rawName.trim();
String displayName = trimmed.replaceAll("\\s+", " ");
String slug = displayName.toLowerCase().replaceAll("\\s+", "-");
return new UserLabel(displayName, slug);
}
public static void main(String[] args) {
String rawName = ;
UserLabel label = normalize(rawName);
System.out.println("name=" + label.displayName());
System.out.println("slug=" + label.slug());
}
}
public class InputNormalization {
record UserLabel(String displayName, String slug) {}
static UserLabel normalize(String rawName) {
String trimmed = rawName.trim();
String displayName = trimmed.replaceAll("\\s+", " ");
String slug = displayName.toLowerCase().replaceAll("\\s+", "-");
return new UserLabel(displayName, slug);
}
public static void main(String[] args) {
String rawName = ;
UserLabel label = normalize(rawName);
System.out.println("name=" + label.displayName());
System.out.println("slug=" + label.slug());
}
}
normalization
Normalizing input once gives later pipeline stages a cleaner value to use.
derived field
The slug is derived from the display name, so it stays consistent with the cleaned input.