Case Studies

Client Context

Series B SaaS company ($22M ARR, 3 data engineers). Product and growth teams disputed weekly KPI reports — churn numbers from the warehouse didn't match billing system exports. Leadership lost confidence in the analytics team.

What Was Broken

Event telemetry had silent schema drift: a frontend deploy renamed two event properties without updating the warehouse ETL. Downstream churn and activation metrics used stale column references, producing numbers that looked plausible but were wrong by 15–30% depending on the week.

What BayesIQ Found

• 5 metrics recomputed from raw events disagreed with reported dashboard values
• 2 event properties renamed upstream but never updated in ETL (schema drift)
• ~18% of session records had null user_id due to a race condition in the event logger
• 3 near-duplicate event types (e.g., signup_complete vs sign_up_completed) feeding separate pipelines

Business Impact

The churn metric was overstated by 22% in Q3 board reporting. A planned pricing experiment was designed around incorrect activation numbers. Two analysts spent ~10 hours/week manually reconciling reports.

Remediation

BayesIQ mapped all event-to-metric lineage, flagged the stale column references, and delivered a dbt project with staging models that normalize event names and enforce not-null constraints on user_id. Schema tests now catch property renames before they reach production dashboards.

Deliverables

• Scored audit report (38/100 — Critical)
• dbt project: 6 staging models, 3 mart models, 42 schema tests
• Streamlit dashboard with corrected metrics and data quality summary
• ASSUMPTIONS.md documenting 11 data contracts
• METRICS.md with canonical definitions for 5 KPIs

Client Context

Mid-market payments processor ($45M revenue, 2 data engineers, 1 analytics manager). Preparing for a SOC 2 audit and needed to demonstrate data pipeline reliability. Internal team suspected issues but lacked tooling to quantify them.

What Was Broken

Transaction event pipeline had timestamp gaps during peak processing windows. Settlement reconciliation ran on a T+1 batch job that silently dropped records when the source schema changed — which happened twice in the prior quarter during API version upgrades.

What BayesIQ Found

• 4 timestamp gaps > 15 minutes in the prior 90 days, each during peak settlement windows
• ~1,200 transaction records dropped by the T+1 batch job due to unhandled schema changes
• Revenue metric definition used gross_amount in one pipeline and net_amount in another — $340K annual discrepancy
• Column naming inconsistencies: transaction_id vs txn_id vs trans_id across 3 source tables

Business Impact

Dropped records meant settlement reports underreported daily volume by 0.3–0.8%. The revenue metric discrepancy showed up in board materials as an unexplained variance. SOC 2 auditors flagged the timestamp gaps as a control weakness.

Remediation

BayesIQ delivered a canonical column mapping that unified naming across all source tables, added dbt tests for timestamp continuity and record completeness, and documented the revenue metric definition so both pipelines use net_amount consistently. The batch job was patched to handle schema evolution gracefully.

Deliverables

• Scored audit report (52/100 — Needs Attention)
• dbt project: 8 staging models, 4 mart models, 56 schema tests
• Streamlit dashboard with settlement reconciliation and gap detection
• ASSUMPTIONS.md documenting 14 data contracts
• METRICS.md with canonical revenue and volume definitions

Client Context

Regional health system (12 clinics, 4-person data team). Building a clinical analytics platform to track patient outcomes and operational metrics. Data sourced from EHR exports, claims feeds, and manual spreadsheet uploads. No existing data quality framework.

What Was Broken

Patient outcome metrics were computed from EHR exports that arrived in inconsistent formats across clinics. Some clinics exported dates as MM/DD/YYYY, others as YYYY-MM-DD. Readmission rate calculations double-counted patients who appeared in multiple clinic feeds with different patient ID formats.

What BayesIQ Found

• 3 date format variations across clinic EHR exports, causing ~6% of records to parse incorrectly
• ~340 patients double-counted in readmission metrics due to inconsistent patient ID formatting across clinics
• 2 clinics submitted CSV exports with trailing whitespace in diagnosis codes, causing join failures with the claims feed
• Null rate on discharge_date was 12% — far above the expected <1% — due to a misconfigured EHR export filter

Business Impact

Readmission rate was reported as 14.2% when the actual rate was 11.8% after deduplication. Quality reporting to CMS was at risk of inaccuracy. The data team spent an estimated 20 hours/month on manual data cleaning that could have been automated.

Remediation

BayesIQ standardized date parsing and patient ID canonicalization across all clinic feeds, added null-rate monitoring on critical fields, and delivered dbt models with referential integrity tests between EHR and claims data. Whitespace trimming was added to the staging layer.

Deliverables

• Scored audit report (44/100 — Critical)
• dbt project: 10 staging models, 5 mart models, 48 schema tests
• Streamlit dashboard with patient metric reconciliation and clinic-level quality scores
• ASSUMPTIONS.md documenting 9 data contracts
• METRICS.md with canonical definitions for readmission rate, length of stay, and 3 operational KPIs