How to Guarantee Integrity in Large Datasets
- Frictionless Decisions brings you counterintuitive, original, jargon-free ideas for connecting data to decisions. This article is part of my series, The 00001010 Commandments of Data.
All the buzz around analytics today might have you convinced that there’s gold buried somewhere in your data. As Billy Beane famously said in Moneyball (the story that started the buzz), “We are card counters at the blackjack table. And we're gonna turn the odds on the casino.” But everyone saw the movie, so it’s really a race and you’re worried you might fall behind.
I’ve got news: if you think analytics is mainly about finding "hidden insights," you're wrong.
Business analysts spend a lot more time explaining what happened and why the results don’t match people's expectations rather than finding brilliant ideas. Questions like “We had 1,000 students last year, now we have 1,100. What changed?” or “We booked some big orders this quarter, but our backlog only went up by $100k. What happened?” consume most of their time.
I’ve always used this “Sixth Commandment of Data” to help analysts see internal integrity in the numbers. By measuring the grand totals before measuring the different parts of a set of data, you can ensure that the beginning and end points of the numbers agree with (“reconcile”) all the changes in between.
Defining "The Whole"
There's no way to explain what changed in your business unless you know how much business you had at the beginning and what you now have at the end. That's what I mean by measuring the "whole": knowing the grand totals of your most important data sets.
For a manufacturing company, it's your customers. What's the value of all the orders on the backlog?
- For a university, it's your students. What's the total number of enrolled students?
- For a SaaS software company, it's your subscribers. What's the value of their active monthly subscriptions?
- If you're managing a warehouse, it's your inventory. What's the value of all the items in stock?
- For any company, it's your people. Measure the total count of employees (headcount).
I call this the "book of business": knowing the total amount (value and quantity) of all the assets you manage. Most analytic questions try to explain what happened between the beginning and ending values of these totals. You can apply this approach to any set of data for your company; the more broadly you define the “whole”, the more questions analysts can answer with the data.
Measuring the Change
When you measure the whole book of business, you can help analysts answer questions about what changed. You can calculate the amount of the overall change before measuring all the different reasons the numbers changed.
- Let's say your organization is a university, and you started the year with 1,000 students and ended with 1,100. Take these steps to measure the whole change:
First, calculate the total change: the difference between the total student count at the beginning (1,000) and what you have at the end (in my example, an overall increase of 100).
- Once you know the overall change in the data, you can start to measure the parts and calculate easily defined changes that everyone understands. You added 267 students through new enrollments but lost 128 students through graduation. Everyone agrees on how to measure those numbers.
- But that doesn't explain all the changes; you've got an additional, unexplained drop of 39 students. Maybe that was students who unexpectedly dropped out?
By measuring the whole before measuring the parts, you've "cornered" the unknown changes; you haven’t explained all the changes until all the explanations equal the total change amount.
What I’m showing here is just the normal way analysts think. But embedding that thought process into all the data is how a great data strategy accelerates their work. What's the best explanation for the unknowns? Ask the analysts. Measure the changes like this and they'll figure out what's really happening in the business.
Different industries use different names for the activities that normally change their book of business. In manufacturing, it's "bookings". In SaaS software, it's "churn". In retail, it's customer “retention”. How you measure the "whole" depends on your business model, and my simple examples might not describe your industry; a manufacturer can easily measure customer value from hard orders in their system, but measuring customer value for software subscriptions requires more estimation. In any case, following the definitions your company uses to report the financial results usually makes sense.
Defining this "whole" for your company sets the cornerstone for most analytics, and you can apply this approach to almost any type of data. That's why I'm giving you this sixth commandment:
Always measure the whole before measuring the parts.
To remind you of this week’s data concept, enjoy Head over Heels, by The GoGo’s, from the Frictionless Data Spotify playlist.
For the full story about making data flow faster and better, check out Frictionless Data on Amazon.
