Episode 24 | Discovery exercises can help de-risk internal audits and performance audits

The Assurance Show
Episode 24 | Discovery exercises can help de-risk internal audits and performance audits
/

 

Show Notes

In this episode we discuss why discovery exercises are becoming very useful for audits.
They help de-risk the conduct of audits, and the three main scenarios we use them for:

  1. Audit topic validation and overall audit planning
  2. Scoping, depth and deliverable
  3. Planning individual audits

Transcript

Narrator: 

Welcome to the assurance show. This podcast is for internal auditors and performance auditors. We discuss risk and data focused ideas that are relevant to assurance professionals. Your hosts are Conor McGarrity and Yusuf Moolla.

Yusuf: 

Okay. So today we’re talking about discovery exercises. My understanding is that performance audit teams are not really doing this yet. Or there aren’t that many that have maturity in the way in which they do this yet.

Conor: 

We are aware of performance audit teams experimenting with this approach. It hasn’t been embedded to any great degree as far as we’ve seen.

Yusuf: 

A few internal audit teams have been doing this already. This is not a mature field. This is an emerging field. And so, what we talking about here will change over the next few years.

Conor: 

We’re going to be talking about the importance of taking a discovery approach to internal audits and performance audits , t he purpose of a discovery approach, why they’re useful and some of the benefits.

Yusuf: 

What is a discovery approach?

Conor: 

So a discovery exercise is spending time upfront trying to explore what’s possible with the audit and how you should conduct itT.

Yusuf: 

The reason that you’re doing it will dictate the focus of the discovery exercise.

Conor: 

There are three main scenarios or purposes where you might find it useful to take a discovery approach. The first one is audit topic validation: in that scenario, you may not be fully confident that this audit is going to deliver enough value. The second one would be where you want to understand the level of depth that’s required to conduct the audit, and as part of that you might also want to figure out what the deliverable looks like. And then the third scenario is where you need to understand how you’re going to undertake the audit, what are the lines of inquiry, what’s the information and evidence available to you and how far that can take you? They’re the three main scenarios. There may be one or more of those reasons for why you decide to take a discovery approach.

Yusuf: 

What you’re saying then is that you could, for any given audit, direct your discovery exercise at one of those scenarios, or you could be covering off two or even all three of those scenarios

Conor: 

That’s why they’re so useful. They can enable you to de-risk your audit program or a particular audit.

Yusuf: 

The first one you said was topic validation. What is that? And why would we need to do it?

Conor: 

Every internal auditor and performance auditor will have their work plan. Three months, six months, a year, in some exceptional circumstances may be more than a year. As part of developing that work plan, they would have gone through an exercise to generate ideas about a range of topics. As part of the evaluation process they will decide, yes, this is a group that we are definitely doing because, there’s a potential for great risk or it may be really important to stakeholders or for a few other reasons. So those are ones that we said, yes, we’re definitely doing those. On the other end of the scale you’ll have some of those topics, when you came to evaluate them, the team identified they’re definitely not worthwhile. not going to deliver impact or it’s addressing an issue that has, to some extent being mitigated or some other reasons.

Yusuf: 

And we know that for certain.

Conor: 

You at least have done enough, asked the right questions to determine that those are topics that we’re definitely not going to include in our plan. Between the definites and the definitely not topics, we have this group of maybes. These are topics we’ve identified, we’ve done the evaluation process and we think they may be useful. We’re just not sure how useful they are going to be. And we need to ask are they going to stay on our plan or do they come off? A way to assess that need. So to validate each of those maybe topics is to do a discovery exercise. So with all those maybes, what we find, through our work is that it’s useful to take a discovery approach to them to answer that question, will they stay on my plan or will they come off?

Yusuf: 

The idea is that you have either an external to the audit team party that needs some more information. Or you as the internal audit team need more information in order to determine whether this audit will go ahead or not. So either you’re not sure. And you need to, do some more work to convince yourself, or you have a gut feel about it, but the auditor committee or management or the CEO have looked at it and thought, I’m not sure. Can you tell me why we should do this and you have a gut feel why, but you don’t have a definite why? So you’re going to go with a discovery approach to move from gutfeel to a definite yes or a definite no. Is that what we’re talking about there?

Conor: 

That’s spot on because you need to be accountable for how resources are prioritized and what they do. So you need to give comfort to those stakeholders. If it’s external to your unit. that you’ve done enough work for those topics to stay on, or indeed you need to do enough for yourself to be able to defend those topics that you’re keeping on your plan.

Yusuf: 

Or you have the potential to do 20 audits and you’ve got 10 yeses and you’ve got 20 maybes and you need to now determine off the 20 maybes, which 10 are going to go to a yes. And which 10 are going to go to? No. So let’s take some sort of structured process to get to the answer. it may end up being 8 yes and 12 no for various reasons,

Conor: 

The second scenario is where you’re trying to understand the level of depth you need to go to within your audit.

Yusuf: 

So this is now for both yeses and maybes.

Conor: 

Correct

Yusuf: 

Okay.

Conor: 

So in the performance audit domain, You generally start out with quite a broad audit objective, which is absolutely fine. And your performance audit may look at the delivery of a service or a government program that incorporates multiple entities. However, as an example on your discovery exercise, you may find that, the greatest risks that you’re trying to address a touch to just one of those entities. you’re then enabled to. Refine your approach, to focus on not one particular entity, okay. So that’s different to, preliminary scoping activity. And the reason that it’s different to preliminary scoping activity is that you would go to a button more depth and you would do work that would be able to be reused as part of the audit. And you would do enough work that enables you to produce a deliverable at the end of that discovery exercise that says, yes, we’re going to go ahead or no, we’re not going to go ahead. Or yes, we’re going to go ahead at this level and we’re going to go ahead with this particular sample of entities and how does that differ then to, current performance audit methods, once you’ve decided on the topic and started, the preliminary scoping for that topic, you would then go in to determine what your sample of entities is going to be in your sample of entities, ideally should be risk-based. So how is the discovery exercise different to that preliminary scoping in respect of collecting a sample? Well, firstly, the discovery exercise is in much greater depth. Then you would, generally go to, as part of your original scoping of the audit, it may well be that as part of the original scoping, you didn’t have proprietary information available to you that you can only get through a more in depth exercise. So what we’re saying is as part of your initial scoping, you may have looked at some sort of really high level data, some high level risks. Do you have an actually. Drill down into, those particular risk areas.

Yusuf: 

Okay, that makes sense. there’s a bit of overlap between preliminary scoping and a discovery exercise. But what we’re saying then is that the discovery exercise would go into more depth and would cover some level of field work that you then able to reuse. So it’s a little bit different. and produces a different deliverable in itself to what the preliminary scoping exercise would be.

Conor: 

when we’re talking about level of depth here, too, of course, the discovery exercise, allows you to determine what your deliverable might be. So at the end of that discovery exercise, you’ll have more clarity around what the deliverable looks like. And that’s really useful to bring back to stakeholders.

Yusuf: 

That in the case of internal audit, you conduct a full internal audit. As in you conduct an internal audit all the way to the end and produce, a standard order report or whatever other communication is that you’re going to be putting out. whereas you could decide as part of your discovery exercise, that what you’ve identified as part of your discovery size is sufficient and stop and report there. Or you could go so somewhere in between. So if you would normally take, four to six months to conduct an audit, you’re doing a discovery access for one month. You could say I’m either going to exit after my discovery exercise. Cause I’ve got enough information to report at this point. Based on how much I want to report on, or I’m going to go all the way to the end and do a full blown audit and report is normal and communicate is normal. Well, I’m going to go somewhere halfway and that means instead, I’m actually able to deliver to the next audit committee as opposed to two away.

Conor: 

regardless of, those three outcomes that you have, you’re always going to have some sort of product. So you’re delivering value across that chain, regardless of where you end up.

Yusuf: 

Fantastic. Okay. and then the third one

Conor: 

so, what we’re talking about here is where the discovery exercise, can Help me understand the ways in which you get the information to deliver the audit

Yusuf: 

Okay, so you’ve decided on a topic and it’s either a yes or maybe. you then determined how deep you’re going to go and what the deliverable exactly is going to be. And you may have done some of that simultaneously, but let’s assume a particular scenario, you’ve decided on an audit topic. So the answer to it at the beginning was yes. So you’re not going to go and validate the topic. You knew how much depth you going to go to. So you know that you need to do a full audit because it’s really important to really risky. What then does this mean this approach? Does this mean you can then determine how it is that you will go about executing on that based on some preliminary information gathering?

Conor: 

Yeah, that’s certainly right, but probably even more beneficial is it, allows you to identify those areas where you see limited return in how you deliver your audit. in practical terms, for example, where you might find out through the discovery exercise that there’s not much documented information, or there’s not a lot of data around, so we may need to Conduct a lot of interviews or find some other means of gathering the information we need to deliver the audit we wouldn’t have no, not from the outset, unless we’d taken this discovery approach to try and understand what’s possible in terms of being able to identify information, data, or evidence, to be able to deliver

Yusuf: 

So we’ve done some scoping work. We know broadly what it is that we are going to be doing. When we do the planning work, we decide what the audit approach is going to be, to be able to conclude on the lines of inquiry. And we may have combined interviews and documentation and data evaluation and analysis, et cetera. Now, if we do the discovery exercise, we might be able to fine tune that approach early on so that we don’t go down the path of lots of documentation around what the interviews are going to be, what the surveys are going to be we know already. But we try to determine at the discovery step, what exactly we can and can’t use. in the data world we’d look at. What data is usable, what data we can get, whether we can get that data reasonably. What the quality of the data is to enable us to come to conclusions.

, Conor: 

no overall it’s about managing the engagement risk, making sure that we are focusing our efforts on the information or the evidence that’s going to give us the greatest value. Rather than try to spread ourselves too thinly. discovery can help you zone in and what’s going to be most useful to deliver so that they’re not. Wasting time on things that are either going to be irrelevant or not that much value and how you’re going to use that information to, draw your conclusions or findings.

Yusuf: 

it sounds then like a shortcut method. So a discovery that is focused on topic validation or shortcut Overall annual planning the level of depth, discovery helps do shortcut door scoping exercise, and the approach discovery helps to shortcut your planning exercise.

Conor: 

the important thing to understand is that just because they’re shortcuts. Doesn’t mean they’re creating risk for you because they’re actually minimizing your overall risk of not being able to deliver on your mandate or on your function or what you’ve been set up to do. So they’re really, strategically important in that sense. So while they are shortcuts and they allow you to prioritize your resources, focus on greatest impact, do the right audits at the right time and the right way. overall discovery exercise is a really good way to manage your strategic risk for under delivery.

Yusuf: 

you can use a discovery exercise for one or more scenarios, would you plan your discovery exercise differently based on the purpose of the exercise. So if you were looking at topic validation only, or you were looking at topic validation, plus confirming the approach together, would that discovery exercise look different?

Conor: 

Yes, if you’re purely doing discovery for topic validation, you may be able to do that really quickly in a short period Now, obviously if you’re combining other purposes or scenarios onto that, whether it be level of depth or your. Ultimate deliverable you’re trying to identify, or if you’re really trying to get granularity on the approach, that’s going to deliver most value. Then that could take a little bit more time and more planning so that the discovery exercise is done properly to tick off in those things.

Yusuf: 

we started a couple of years ago with these discovery approaches and we weren’t very clear on what the purpose was then, and that created difficulty. We weren’t sure what exactly the objective was going to be. in coming up with, the different scenarios, the different purposes and focusing the discovery exercises on that purpose or that set of purposes made a world of difference. It’s all about what is the objective of that exercise? And then focusing your effort on ensuring that you achieve that objective by following the right steps.

Conor: 

answering those critical questions. And trying to determine purpose. it takes a little bit of practice and you’re not going to get it right first time.

Yusuf: 

internal audit performance audit have been going on for decades. but it took quite a while to get to how exactly do you go about conducting a performance or how exactly do you go about conducting an internal audit? And then there were all sorts of standards that came out and people, I think still today, still try to create their own individual methodologies within individual teams, which is ridiculous, but. Regardless of that. the discovery approach form spot of what people are starting to call agile, auditing agile, internal auditing, or agile performance sort of thing. a newer concept. We were always used to waterfall style approach, project management, and, will take a while to bet down. So I don’t think there’s one exact answer to this. but it does appear as though the approaches are gathering steam, first of all, but also getting to a point where we very quickly understanding how we need to do this, to be able to achieve the result better.

Conor: 

the most important note of caution is answer that critical question about what is the purpose of this discovery exercise or this proof of concept? has been a few occasions now where we’ve, seen that term used, but it hasn’t been really clear about why that approach was being taken in the first instance.

Yusuf: 

Okay. And there probably are other scenarios or other purposes. We tend to focus on these three because they make the most sense to us.

Conor: 

we’ve had traditional approaches to audits for a long time now. And you mentioned waterfall, project methodology and so forth. One of the things that we’d say about performance auditing, is that traditionally there have been, an all or nothing project we say, we’re going to audit this. This is the objective and the topic let’s just go and do it. And the range of value. Presented at the end of the nine months or 12 months or six months or a significant period before you get to an outcome, differs from a lot of impact and a lot of value to perhaps something that didn’t deliver very much. So we’re slowly moving away from the all or nothing approach NAI and, discovery helps you move away from that, risky scenario.

Yusuf: 

the other risk that it helps you mitigate is we’ve seen February, March, April, this year. and this has been recorded in 2020 where internal audit plans in the internal audit world, and performance order plans as well. But internal audit plans largely were chucked out, right? People are to throw them out because they just weren’t relevant anymore. and there’ll be a few. Compliance focused internal audit teams. That will be a little bit different to that, but generally we’re not talking about those entities because they tend to be more compliance functions than actual internal audit functions. So most internal auditor’s that deliver real value, their plans were just eliminated and they had to go back to the drawing board. if you’re thinking about doing an audit and it’s going to take you three to six months to complete that audit, you may get through a small portion of that audit and be able to deliver nothing because an event takes place or the strategy changes. and so because there’s so much of movement in terms of external factors, if you are taking shorter view, using shorter cycles,, you then able to get to that result faster and actually deliver something, for which there’s less risk of being disrupted by an external event.

Conor: 

from you your first work paper or whoever you’re conducting your audit to your end product or deliverable, whatever that may be. You want to compact that timeframe as much as possible, but still delivering value. it could be. Just an insights paper at the end of your discovery exercise. So you spent four to six weeks, let’s say you’ve determined the risks weren’t as significant in the performance of the activity, as you thought. So you say we’re going to finish up here. we’ll deliver some other product, and not a full, reasonable assurance report. None of your time or effort or analysis has been wasted. And in fact, you’ve saved the rest of that time. That would have potentially been spent delivering an old traditional end to end performance audit that may have taken a long time.

Yusuf: 

In the performance audit world, you need to deliver something on paper in the internal audit world. It may not even be a report. insights that you generate, you may be able to deliver that verbally. provide an update to the audit committee or, talk to management, talk to the CEO, talk to the head of internal audit. If they making the decision. often you want to document something, but it could be like a one page memo. It could be an email depending on how quickly you go. how that ends up will depend on what level of formality you need within your team, how long it takes will depend on How long things take within your particular organization within performance audit, they would usually go for somewhere between three to six weeks within internal audit. They can go anywhere from a week to three weeks and sometimes a little bit more. it really does depend smaller organizations. You can do it within a week, but usually you want to give yourself a little bit more time. You could go to the around about the three week Mark in order to be able to deliver that. so it does vary. There’s no rule around how long it’s going to take and what exactly you’re going to deliver, but the rule, if you want to call it, that is that it will be significantly shorter and far more valuable than trying to deliver something big bang upfront.

Conor: 

And because it’s got a shorter focus. It’s a sprint really, I guess, to get to where you need to be at the end of the discovery exercise, you can do multiple of these at the same time within larger teams. And you don’t have all your resources focused on one particular, large audit consuming all of your attention that may ultimately not give you the impact that you need.

Yusuf: 

Wrapping up. We had three purposes for conducting a discovery exercise. The first is to validate the topics that you’ve selected The second is determining how far you’re going to go. What the level of depth is for an audit the specific deliverable or the set of deliverables And then the last thing is how you’re going to approach the audit. And that’s validating through determining what data and information you can get. if you haven’t started doing this yet, it’s worth considering because it can significantly. Change your life Thanks Conor.

Conor: 

Thanks Yusuf. See ya.

Yusuf: 

See ya.

Narrator: 

If you enjoyed this podcast, please share it with a friend and rate us in your podcast app. For immediate notification of new episodes, you can subscribe at assuranceshow.com. The link is in the show notes.