Why you, as an auditor, need to think a bit differently to the rest of the organization, and specifically govern the data that you use for audits.
In your audits, you will regularly combine data from different domains to test your hypotheses.
Most data analysis focuses on a single subject matter or a single system.
Your ability, and need, to combine data from multiple sources enables you to obtain and provide a different view of risks and opportunities.
However, the joined data is often more sensitive; and can reveal new, potentially unexpected, insights that are also sensitive. With this comes a different, more stringent, set of data governance and management expectations.
Let’s consider a real-life case study and how this played out.
The project: Revenue assurance for a health service provider
This organization provides services to the public, under a subsidized fee-for-service model. The objective of the audit was to determine whether fees were charged appropriately, particularly the application of subsidies.
Customers undergo an annual needs assessment, using a well-defined and controlled process, leading to a determination as to what the level of subsidy (or discount) will be for each household.
The CRM system has two levels – household and customer. Each customer is assigned to a household. Subsidy rates are captured here at the household level, which then automatically filters down to individual customers in real-time. The customer numbers and subsidy rates are then sent to the billing system via an automatic overnight update.
The subsidies are applied automatically within the billing system, based on the subsidy rating, and the remainder is billed to the customer. For example, if the service fee was $100 and the customer subsidy level was 45%, the customer would be billed $55. The invoice does not reflect the subsidy rating.
Because the subsidy is applied to a household, and there are some special circumstances under which an individual member of a household might have a higher level of financial need, there is some potential to override the “discount” that is applied. This happens via a manual update of both systems, because the automatic update cannot handle this special circumstance.
Manual subsidy adjustments are appropriate.
- Billing transactions
- Billing system master file data
- Billing system master file audit trail (changes)
The challenge that we encountered halfway through the audit
The initial audit steps included a review of the master file and the audit trail. Unfortunately, neither the master file itself, nor the audit trail, captured the name of the user account that was used to make the change.
This made it difficult to determine who made the change – in turn, making it difficult to determine whether the change was appropriate (e.g., authority to make the change). It did, however, capture the date and time of the change.
How we sought to overcome the challenge
In particular, the audit trail from the CRM system. This showed the manual changes made at customer level, along with the usernames, dates and times of those changes.
Joined the audit trails from the two systems to:
- Correlate the changes that were made
- Determine who made each of those changes
- Determine whether the changes were made within a reasonable time frame of each other
(e.g. CRM system updated and then billing system updated within a couple of hours)
- Identify a set of targeted samples for further review.
What does this have to do with data governance and management?
1. If we had billing system information alone, or CRM system information only:
That wouldn’t be more sensitive than existing reports and outputs.
Because we had to also obtain CRM data and combine it with the billing system data, we had personal information about customers from the CRM system and medical history information from the billing system.
Classifying this joined data, in accordance with the organization’s data classification scheme, meant that we now had data that was more sensitive than any existing data extracts.
We needed to protect and monitor accordingly. This required a bit more thought and effort, but under an established methodology, so fairly simple.
2. There was another data sensitivity matter, though.
And this is one that you probably face, as an auditor, all the time.
We found that some billing changes were made independently of CRM changes – manually – and that these were then reversed within a short time window. Conducting further analysis, we then found that the time windows related exactly to when bills were being sent out.
Uh, oh. Was this fraud?
Whether it was, or wasn’t, there was suspicion. Enough to further raise the level of sensitivity of the data that we now had.
The organizational data governance framework did not cater for how to protect this data. So, we had to think about it, work out what we needed to do and then obtain approval for the approach.
Not insurmountable. But bad timing. We had the investigation to focus on.
It would have been easier if we had worked out what to do if faced with a scenario like this. For example, in developing the overall audit plan, in refreshing our methodology or during planning for this specific audit. We may not have explored this specific situation. Planning for every eventuality isn’t an efficient approach. Exploring exceptional circumstances and what process to follow in such circumstances would have helped.
Develop your audit specific data governance and management regime.
Work out, in advance, what you need to do in these sorts of situations.
Go through the approval process.
Then you don’t have to rush around at the last minute, when there are pressing matters to deal with – like the investigation into potential fraud that we had to undertake at the same time as working out what we needed to do to govern and manage the new sensitive data.
[You’re probably wondering – it turned out not to be fraud, of course, or it wouldn’t be here for public consumption].