Reporting data quality — the metrics worth tracking
Most reporting dashboards measure whether returns went out on time, which is the one thing the supervisor can already see. Timeliness is necessary and almost never the problem. The metrics that predict a supervisory finding measure something harder: whether the process is drifting away from the business it is supposed to describe. Four of them are cheap to produce and genuinely leading.
1. The four leading metrics
| Metric | What it detects | Bad sign |
|---|---|---|
| Rejection rate, by rule | Where your data model and the framework disagree | Repeat rejections on the same rule — a mapping problem being manually patched each cycle |
| Resubmission rate | Errors surviving to submission | Any upward trend; resubmissions should be rare and falling |
| Reconciling-item drift | Scope errors | A definitional difference growing beyond its expected size |
| Manual interventions per cycle | Fragility | Adjustments applied at the file level rather than fixed at source |
2. Rejections tell you where, not just how many
A rejection rate on its own is nearly useless. The same rate can mean a healthy process catching a one-off or a chronic mismatch being patched every period. What makes it diagnostic is breaking it down by rule and tracking recurrence.
Three patterns are worth alerting on:
- The same rule failing every cycle. This is a mapping defect with a manual workaround, and it will eventually be missed.
- A new rule failing. Usually a validation-rule package change rather than a data change — worth confirming, because published packages are updated quarterly and independently of framework versions.
- Cross-template rules failing. Almost always one data point sourced twice by different queries, which is a structural problem rather than a value problem.
3. Drift is the metric nobody has
The most valuable measure is also the least common: the size of each reconciling item between returns, tracked over time against its expected size.
A definitional difference between two frameworks has a predictable magnitude. Once quantified, it becomes a control: if it stays where expected, both pipelines are behaving; if it grows, something entered one and not the other. That is how a scope error — a product or entity added to one pipeline only — becomes visible before a supervisor’s cross-check finds it.
Producing it costs almost nothing once the reconciliation exists, because it is simply the reconciliation’s own output plotted over time rather than reviewed and discarded each cycle.
4. The lagging metrics, and their proper use
Timeliness, acknowledgement receipt and query volume are worth tracking, but as confirmation rather than as early warning:
- Timeliness is binary and visible to the supervisor anyway. Its only diagnostic value is the margin — a return that is always filed on the last possible day has no resilience.
- Unmatched acknowledgements should be zero, and any non-zero value is an incident, not a metric.
- Supervisory query volume tells you how the supervisor sees you, with a long lag.
5. Using them
Facts: a reporting pack shows 100% timeliness for six consecutive quarters, and the team reports the function as healthy.
What the leading metrics show: manual interventions have risen from two per cycle to eleven; the same three validation rules fail every quarter and are corrected by hand; one reconciling item has grown by half without explanation.
What that means: the function is not healthy, it is being held together by effort. Timeliness is intact precisely because people are compensating — and the compensation is invisible in the metric being reported. The reconciling-item growth is the one to investigate first, because it is the only one of the three that indicates the numbers may be wrong rather than the process being strained.
What the practitioner does: adds the four leading metrics to the pack, sets a target of reducing manual interventions rather than maintaining timeliness, and treats each recurring rejection as a mapping defect with an owner and a fix date rather than as a routine step in the cycle.
FAQ
Why is timeliness a poor primary metric?
Because the supervisor already knows it, and because it can be maintained indefinitely by manual effort that hides a deteriorating process.
What is the best single leading indicator?
The count of manual interventions per cycle. Each one is a defect that was worked around rather than fixed.
What does a growing reconciling item indicate?
Usually a scope error — a product or entity that reached one pipeline and not the other — rather than a valuation problem.
How often should these be produced?
Every cycle, as a by-product of the process rather than as a separate exercise. All four fall out of artefacts a well-designed pipeline already generates.
6. Why these are cheap
The argument against reporting metrics is usually effort, and it does not survive contact with these four. Rejection data comes from the validation output the process already produces. Resubmission counts come from the archive. Reconciling-item sizes come from the reconciliation. Manual-intervention counts require the only new discipline on the list — logging an adjustment when it is made, which takes seconds and is worth doing for its own sake.
None of them requires new tooling, a data warehouse or a project. They require that the artefacts a controlled process already generates are retained and plotted rather than reviewed once and discarded — which is the difference between a process that produces evidence and one that produces files.
Related: Reconciling returns · Validation rules · Reporting governance
