Risk Insights Logo

Data warehouse (DW) projects - as an auditor, have you been asked to get involved in a big bang DW project?

March 28, 2019

During the late 90's, the strategic planners at an African university decided to improve access to information.

So they created a data warehouse.

At the time, financials and payroll were delivered by an aging mainframe based GL system. At month-end, we'd receive piles of paper containing various reports. Two of us would go through them, manually separate, place in envelopes, label the envelopes and send them off via internal mail. This took 2-3 days. It was painful.

The team chose the tech, created the data extracts, built the models and trained a range of users. It took a few months to build the reports - we had to painstakingly replicate the exact format that people were used to. It was worth it. Within weeks of the reports being made available, reporting had shifted - from paper based, once a month, to electronic, on-demand. A significantly better solution.

And the self-service solution freed us up - the humans that were originally involved in manual report distribution now had time to focus on higher value activity.

But that was two decades ago.


Today, it is interesting to see the same, waterfall style, large data warehouse projects being established.

Many fail to deliver positive ROI and some fail completely.

As an auditor, you may be involved in such a project - ideally at inception. 

If you see a project like this, ask whether it is necessary to embark on a large project for this purpose.


Can smaller projects, driven by use cases, deliver faster incremental value?

Definitely, maybe:

  1. Definitely: Large projects continue to fail - including some that claim to be "agile" but are really just waterfall with a few "agile" bolt-ons
  2. Definitely: Sophisticated solutions for data prep and advanced analytics, including open source options (like KNIME), are mature and widely available
  3. Definitely: Visualisation tools have come a long way too - e.g. we use Tableau and Power BI - and these are only two among various solid options
  4. Maybe: you still need a clean, well governed set of data - garbage in, garbage out. So you still need an overarching data management /governance approach.
  5. Maybe: reporting, particularly when consolidating across disparate systems, is usually more efficient via a data warehouse (speed, accuracy, load on production systems, etc). It's not dead - but perhaps just not created in one big bang
  6. Maybe: new solutions & approaches - e.g. Snowflake (cloud, fast), ELT (rather than ETL) are helping to enable better ROI.


For some industries e.g.,financial services, the level of complexity and the compliance expectations may mean that a larger project is required. However, a different approach (e.g. agile) is still valuable.

Share this article

Get more insights like this

Blog Post
The Assurance Blog
March 3, 2022

Data in Audit Guide

Read article
Blog Post
The Assurance Blog
December 16, 2021

The Data-Confident Internal Auditor: Software

Read article

Subscribe to our mailing list

Get notified by email about new blog posts and podcast episodes by the Risk Insights Team.