Ale CabreraClearbit · 2023
Data Guardrails
Preventive guardrails in the data pipeline, moving quality from fixing records after a customer complained to catching whole classes of errors before delivery.
Outcome
- ↓ 70%customer-reported data issues, measured over 3 months
- 1 → 5attributes covered by guardrails
- ~15validation rules, grown from a handful as quarantine surfaced new issues
- A field could be populated and still be obviously wrong. A global consumer brand listed as a 10-employee company.
- Too much quality detection happened after customers ran into the issue.
- Customer-reported problems kept showing the same failure patterns.
- 859+ flip-flops a month in employee count alone.
- A wrong value can be worse than no value. Coverage alone was the wrong goal.
- Found the patterns in support tickets, SQL analysis, dashboards and calls, and grouped them into classes of errors.
- Talked to engineering, support and sales before designing anything.
- Decided to prevent classes of errors instead of fixing single records.
- Started small: employee count only, and a few rules. Let quarantine show which rule to add next.
- Introduced validation rules in the pipeline, before data reached customers.
- A value that failed went to quarantine until it was validated.
- Rejected values were kept in a "deleted" table, available for investigation.
- Grew from 1 attribute to all 5, and to around 15 rules.
My Role
Defined quality from the customer's perspective. Analyzed the failure patterns, wrote the guardrail design and decided which attribute to start with and which rules to add next.
Engineering Collaboration
Worked with engineering inside the data pipeline instead of treating quality as a downstream QA step. Rules were added in small steps, each one coming from an issue found in quarantine.
Problem & Risk
Customers were finding the errors before the company did. Every wrong value that reached a customer cost trust, and fixing records one by one did not stop the next one. The feasibility question was where to hold a suspicious value and how to check it without slowing delivery.
Customer Discovery & Validation
- Recurring patterns across support tickets, SQL analysis, dashboards and calls.
- Conversations with engineering, support and sales.
- 859+ monthly flip-flops in employee count data.
- Customers catching errors on well-known companies themselves.
Context
Clearbit enriched company data for customers at scale. Having a value was being treated as having good data, and quality problems were handled after delivery.
Timeline
What We Did
- Defined classes of "obviously wrong" data from customer reports.
- Added validation rules before delivery, starting with employee count.
- Built a quarantine step: failed values wait there until validated.
- Kept rejected values in a "deleted" table for investigation.
- Expanded to all 5 attributes and around 15 rules.
validation rules, before delivery
employee count first, then all 5 attributes
passed, sent to production
failed, held in quarantine
rejected, kept in the "deleted" table
Who Benefits
Customers get fewer wrong values and fewer flip-flops. Support and sales stop being the ones who hear about bad data first. Engineering gets a place to investigate errors instead of chasing single records.
Next Steps
The pattern is reusable: new rules can be added as new classes of errors show up in quarantine.