Exception Reports: The Origin Story
This article kicks off a three-part series on “Exception Reporting,” my counterintuitive approach to improving data quality. I also plan to share some fun case studies along the way, so stay tuned.
Tim Keller defined wisdom as the ability to make the right decision in a situation, even when there are many acceptable options. He's right. Real life is situational, and you can't write a manual big enough to tell you what to do in every circumstance. You need to develop “competence with regard to the complex realities of life…[how to understand] a particular situation and then know the right thing to do.”
A wise mentor is much better than any instruction manual. That's what makes mentors so valuable; you can always ask yourself, "What would they do in this situation?" instead of just looking for a set of rules.
I watched my boss, Ray Vincent, in action for about twelve years. He was a math major at USC and worked his whole career in Accounting and Finance. I wondered why anyone would want to spend their college years in the abstract world of math only to spend their corporate career in something as mundane as accounting.
I admired this, but I didn't understand it. That came later.
People who work in data eventually discover that learning to apply data to business decisions is much more complicated than moving and modifying data with software. Companies fill their teams with technical experts and expect them to know how to connect data to decision-making intuitively. Wisdom like this is not intuitive to them, and no amount of technical expertise will close that gap.
Closing that gap was easy for Ray; his thought process and work ethic naturally connected the data to decisions. Looking back now, I realize that he passed that on to me. He didn’t tell me how to connect data to decision-making; he showed me.
When he asked me for an “exception report,” I didn’t know he was about to show me a critical tool for improving how an entire company makes decisions.
Reversing the Flow
Ray came to my office one day in 2001 with an idea. I was managing the reporting systems at Broadcom, and Ray noticed incorrect data on the daily management report. He knew why the data was bad, and every week, he faithfully logged into the business systems to fix it.
"Instead of waiting to see the bad data on your report, I'll tell you the rule to find all the errors. All you need to do is find a way to make the system send the list of exceptions to me every day," he told me.
That was a simple but incredibly brilliant vision: Ray wanted to reverse the decision flow. Instead of waiting for a report to see the bad data, he wanted to fix it proactively.
But he didn't tell me how to make this happen; I had to figure that out. The first version of that application was a complete hack job, but it worked. We called the system “Exception Reports.” You’ll hear a lot of talk in the industry these days about data quality, but that name reveals something completely different about our solution: the system doesn’t fix your data, it just measures its quality.
My team turned his idea into a system that anyone in the company could use to correct their data. All they had to do was tell us their business rule in plain English, and the very next day, they'd start receiving an email with the data records they needed to fix. The system tracked their progress and helped them uncover the broken business processes that led to the bad data in the first place.
Ray’s vision revolutionized my thinking, and it’s still ahead of its time.
These days, vendors sell “master data management (MDM)” software solutions that promise to fix your data. They override mistakes, add layers of logic, and centralize everything. Exception Reports do the opposite: they help people solve problems without IT help. They don’t fix data, they just measure the errors.
The idea grew as others saw the power of Exception Reports. Maxim used them to synchronize entire enterprise systems. Broadcom and Maxim's accountants used them to support their compliance audits. IT teams used them to make better decisions about which system fixes mattered most. People at those companies experienced the power of Exception Reports to harmonize all their data and decisions.
Correcting and Connecting Everything
What made Ray different was the way he understood business data as a system. He wanted to close the loop between the inputs from other systems and the outputs on the report. That's how a mathematician thinks. Ray was willing to do the work, and he didn't shift responsibility because he only cared about one thing: connecting the data to decisions.
Fix data in the right place, and you heal the whole system; fix it in the wrong place, and you might make the system more fragile. Wisdom like this doesn’t come from a training manual. As Ray showed me, seeing good data management in practice makes all the difference.
I’m looking forward to developing this idea with you much more, by explaining the systems and processes that support it and sharing some great case studies to help you catch the vision.
Awesome! This is such a great article. It's very clear and simple. For each subject or item, decide what is the dominating system of record and where other systems don't agree with it, the discrepancy automatically shows up in the exception report for the owner of the item to fix according to the system of record. This usually does not usually require the addition of an expensive system. It usually works with some SQL and a little automation resulting in an email to the owner. As you mention in the article it can resolve just about any issue in any system that has discrepancies.
Very well said! I especially agree with the framing of knowing WHERE to correct data or a process is the most undervalued skill in my industry. Fix the process and watch the data fix itself over time, that’s one of the most exciting truths of operations work!