In this episode we discuss a few key matters that auditors need to know relating to the use of AI and auditing AI.
3 key objectives:
3 key considerations:
Yusuf: The discussion that we're having today is auditing AI. So, auditing artificial intelligence.
And just to set the scene for this early on, we are talking primarily about the use of machine learning. The reason for that is that the majority of auditing work that relates to AI will be in the form of auditing machine learning algorithms, as opposed to most other types of AI. As an auditing fraternity overall, the majority of what we've been seeing has been in the field of machine learning. There are a range of things we'll be talking about today that will cover AI more broadly, but the examples will be machine learning focused.
So, the first thing is, why is this topic important to auditors?
Conor: It's incumbent on us to actually understand how they work and factor that into our audit work program and how we actually look at that as performance auditors to make sure that these new techniques are delivering as intended.
Yusuf: Within internal audit, the reason for understanding AI, and then auditing it would be fairly similar. And we'd have a combination of auditing AI directly, so, audit topics that focus on auditing the use of AI. But then there's also other audit topics where, an understanding of AI will be useful. Either because it is being used and we need to understand what that means for audit risks, or where it isn't being used and our recommendations might go to using it more to achieve a particular purpose that we've identified. Interestingly, a lot of robotic process automation while it is in the field of or could easily be called within the field of AI, the majority of projects haven't yet got to intelligent process automation or intelligent automation. Where the RPA is combined with machine learning to let the machine do more than individuals were doing before the bots were put in. So, most of those bots just replace what humans would do as opposed to going beyond that.
Conor: Today we're going to have three key objectives and three key considerations we'll discuss. First objective is bias and ethics. The second will be providing sound decision-making support, and thirdly, we'll talk about the use of AI and mostly machine learning, where we're not maximizing the opportunity.
And then we'll move on to some considerations. We'll talk about things like model accuracy.
First objective. Bias and ethics. What are some of the main things we need to be thinking about there when we're auditing AI?
Yusuf: Bias , is quite a broad topic and , it's quite an important consideration for auditors, but in brief, one of the first things that we want to understand is are the algorithms that are being used, creating any bias in our decision making. an example of that would be where we use a machine learning algorithm to, segment our customers. Or to, conduct some initial triaging. Let's talk about, the use of a supervised machine learning model, we might have a set of transactions that have already been labeled as either fraudulent or not fraudulent. When new transactions come in, we then want to understand whether those new transactions are also fraudulent. Because we already know what the different, features were of the old transactions that resulted in them being labeled as fraudulent. We can apply that same thinking and process to the new transactions automatically through use of a machine learning algorithm to also label those.
Sometimes the transactions that we have may not be representative of everything that we expect to see. And when we have a set of transactions, that are not representative of everything that we expect to see, those labels, the fraudulent or not fraudulent outcome labels, can be based on personal characteristics of individuals. If we don't have enough information to understand how the decision was made, we can end up with a situation where the machine thinks that the decisions were made based on certain personal characteristics of individuals. And what you then doing is you're not looking at the specifics of the transaction and so that's an example of where you have bias in a machine learning algorithm. Bias is a subset of the ethical considerations in deploying a machine learning model.
Conor: At least in the early stages of deployment, have some human interaction to check that the machine isn't creating any bias. And that's your safeguard, somebody who understands the process makes sure that the machine is not weighting too heavily things that were more correlative rather than causative. From a government perspective, one of the fundamentals of public service delivery is do no harm. So make sure that the vulnerable in society are not harmed or any members of the public are not harmed.
So, you've got to be really careful that no complacency exists with how you implement and assure that your algorithms are actually doing what they need to do as opposed to creating any adverse consequences.
Yusuf: So, the second one is providing sound decision-making support, and this is fairly similar to what we spoke about in terms of bias. So, where we are using a machine learning algorithm to enable decision making and supporting decision-making. We want to make sure that those decisions are based on a sound set of inputs and a sound set of data and an algorithm that actually makes sense and works for the particular purposes we have. If the wrong data is going in, or we're using the wrong features to create the outcomes that we're looking for, the decision making may be flawed. For example, where we have, performance indicators: there's more and more use within talent management of automating some of the decisions around individual performance. And so you want to make sure that the decision making is based on sound data, and if you don't have all of them, the right data that you need, you might be using a smaller set of data to understand what it looks like. Let's say you're using it, learning to understand revenue and you using it to forecast your revenue over next little while. There are several situations in which you want to make sure that you to include all of what, you know, in terms of how the business operates into the support that is being provided for decision making. For example, if you were in the tourism industry, seasonality is very important for revenue. If you don't include seasonality, time-related information in the decision making, you then definitely won’t have a good forecast. So, you need to make sure that your AI provides for sound decision making support by including all of the relevant indicators that you would normally have as a business.
Conor: So you talked their about still a great need to gather all the relevant inputs into your machine learning, what are some of the practical ways that you can get a handle on those early on?
Yusuf: Most important thing is to understand two things. You need to understand the business. And you need to understand the data. and as many commentators have been discussing recently that usually doesn't sit within one individual, they’d be across a number of individuals. If you've got somebody sitting in the corner, developing a machine learning algorithm and not talking to anybody else then you’re probably going to get a bad outcome. You need to understand the business, that you are in, and the function that you are looking at, and then you need to understand the data that you have and what can be done with it.
So, it's top down, bottom up. What does the business look like? What are the indicators that you would normally look at? So, if a human were, because remember that, machine learning really is an artificial intelligence. It's taking what a human would normally do and making it “artificial”. Getting a computer to do what a human would do. So, you need to think about if there were no computers, what would I look at in determining what the relevant business indicators would be? What are the decisions that we need to make? And what would factor into the decisions that we need to make? So that's the first thing top down. What would we do?
And then the next thing is what is the data that we have to be able to answer those questions? Do we have all of the data that we need to answer those questions? is the data accurate? Is it reasonable? Is it timely? What is the level of integrity of the data, but then also, is there any data that we don't have and do we need to fill in some of those gaps in order to be able to get to those decisions?
Those are the two things that you need to do to ensure that you have the data that you need to enable sound decision making.
Conor: When you're attempting to understand the business, and the data, and you have all those stakeholders, gathered together, it's really important for them to have a common understanding of the objective that you're trying to achieve with a machine learning approach.
Yusuf: You almost only take the hype out of it and think about it in terms of what you would automatically do. Yes, of course. machine learning and, related automation, enables more than what human would ordinarily be able to do, but you do want to start with what would a human actually do, and take the hype out of it so that you can think clearly without being muddled by what the potential with an AI solution might be.
Conor: And by taking that hype out of it, you remove the suggestion that some individuals may want to get certain things for their part of the business side of it, where it becomes more of a collective shared responsibility as this is what the organization is going to get out of it, because we all have the shared understanding of what we're trying to achieve so you minimize the risk of pet projects.
Yusuf: Usually the pet projects would come from a technology group or a particular group that understands the technology is so well that they able to bypass what would normally be, you know, good decision making.
The last one, is where as audit is, we've identified that, there is potential for the use of AI of some sort, but the business is not taking the opportunity, and not, using the opportunities that they have, to enable more efficient, more effective outcomes.
This can come out in two ways. So the first is where you are conducting an audit and you find that there's a significant set of duplication or a significant set of human involvement in a process that makes it a lot less efficient than it could be.
So, what you then want to be able to identify is, is there potential for that efficiency through some level of automation? The second is as an auditor where you identify a range of control weaknesses, and the control weaknesses are significant, but almost feels as though it's too difficult or too expensive to remediate those, given human interaction in remediating. So, what you then want to do is look for ways. And I know as auditors we don't always provide recommendations, but we do need to think broadly about what the potential is and whether finding that we have is actionable.
So, think about what the opportunities might be to remediate an issue that we've identified through the use of more efficient approaches like using a machine as opposed to putting more humans in which may make the cost benefit analysis not stack up.
Conor: Some of those repetitive processes lend themselves more readily to an automated or machine learning approach, whether it be, the performance of controls checking or whether it be a transactional environment where there's lots of repetitive activities that are carried out by one or more individuals, then quite obviously there's opportunities for efficiency there.
Yusuf: Examples of that that have been coming up quite often, recently are in the cybersecurity domain - information security. Cyber is just a new term for that really, but in the information security domain, where. you just don't have enough people. You can put more people on, and you just don't have enough money to put more people on and that's where you want to then get the business to think about using automated approaches to evaluate logs, etc., in order to be able to reduce your risk around security.
Conor: As internal auditors or performance auditors, just because you identify through your audit activity, the opportunity for potential use of machine learning or other AI technology, you don't need to be an expert in that space. You just need to recognize through your work that the opportunity exists.
Yusuf: That's right. And then there's a whole bunch more work that will need to then be done in terms of sizing the opportunity up and creating a business case, etc. It's about that high-level initial thinking around, is this actually possible?
Can this be done? What is the cost? Does the cost exceed the benefit?
Conor: And so internal auditors and performance auditors need to be really alive to that sort of thinking as they go through all their work, it really should be part of where their heads out for each audit project.
Yusuf: So those were the three objectives. The three considerations that we have are model accuracy, data quality and then deployment control, security and change.
So, we'll talk about model accuracy first. So this is the actual model that is being used and the determination by the data scientist, or whoever's putting the model in as to whether the model that is being used is the best fit for the purpose that you have.
There's obviously a range of different types of models that are available to be used in machine learning projects.
What you need to understand is that the model that is being used, first of all, in most cases, the model likely does not need to be created from scratch. There are so many, available models that exist already that people don't need to be coding these things from scratch. You really want to, in most cases, use something that exists already, but then the way in which you evaluate the accuracy of the model is important as well.
We've seen examples of where single digit model accuracy levels, single digit percentage model accuracy levels, or even low double digits appear to be reasonable. In reality, if you ask most people involved in creating and deploying machine learning algorithms, they're usually looking for 90% or above.
And we've had a few where we got to 70% and we were really happy, but there's reasons for that. But usually you want to get to a fairly high level of double digits in terms of the accuracy of a model. And so, you don't just want to deploy the first one that you get your hands on, because it sounds interesting.
Random forest sounds really interesting. Tree ensembles sound really interesting. Those actually are some of the better ones for particular purposes. But you want to have the right model for the right purpose and you want to ensure that the accuracy level makes sense and that the accuracy level hopefully is going up not down over time as you get more data and better understanding. So that's accuracy of a model just to understand what's going in and what's coming out.
The second one is data quality. this phrase has been used for years - garbage in, garbage out - and it holds true and probably even more so for the use of machine learning. If you have poor data quality, you are simply not going to have a good result. It may look right. So, you may say 95% accurate or whatever, but with data quality, you're not going to have a realistically correct outcome. So, it might sound technically correct, but it doesn't actually make business sense. And so, you really want to make sure that the data that is being used is the right data and that it's at the right level of quality before it goes into the model.
So, how have you decided what data to select? What data not to include. And then what the level of quality is that you would accept for data that is going into a model? So that's quality.
The last consideration is how is the model, and associated workflows or automations -how is that being deployed? How is it being controlled? How is it being secured and what is the level of change management over that. So, this is a combination of, technical deployment controls, you know, traditionally with any IT type project or ITBAU activity. We think about the way in which something is controlled.
It's usually a very important topic. so technical change control, but then also in terms of the business, how is, is the, entity or the function. being prepared for and responding to the change that exists as a result of using a new algorithm or the change that exists as a result of a tweak to an existing algorithm.
So, a combination of technical change control, deployment control, and security considerations, but also, human related change thinking, how is that actually being put together.
Conor: To summarize we had three main objectives. The first one was bias and ethics and how important that is as with all of our audits and making sure that in developing these machine learning models, that there is no bias Inherent within how they’re put together. The second one is how machine learning can provide sound decision-making support. So, you spoke there quite a bit about making sure that the inputs were right and understanding the business properly and having some sort of fundamental basis upon which we would design the machine learning or the algorithm. Third objective we talked about was where we've actually identified through our audit work, where AI or machine learning in particular could provide some efficiencies or help the business in terms of overall performance, but that those opportunities are not being taken up.
Some of the broader considerations were model accuracy, the old favorite of data quality and then lastly, once these models are deployed, what is the change that needs to happen.
All up there six key matters that every auditor should be looking at when they're thinking about audit and AI.
Bias and ethics, decision-making support, opportunities for the use of AI, making sure that there's accuracy in the models, good data going in and then how is your internal controls environment impacted on or how has the business changed as a result of deployment?
Yusuf: We're recording this in late May of 2020. We expect that a lot of this will hold true for some time. There will obviously be some tweaks. What we spoke about were basics. There's a lot more as auditors use AI more themselves or as auditors audit AI more, over time. So, both performance auditors and internal auditors, there’s a lot more detail that we'll get into and in five years’ time or in 10 years’ time, some of this might've changed. But, for now this is what we have, and this is what we need to use.