TIAPS ALB_Module 2E. Data Analytics for Internal Auditing
- No tags were found...
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
<strong>2E</strong>. <strong>Data</strong> <strong>Analytics</strong> <strong>for</strong> <strong>Internal</strong> <strong>Auditing</strong><br />
<strong>2E</strong> Learning Outcomes<br />
On completion of this section, students will be better able to:<br />
• Describe tools and methods used in data analytics.<br />
• Select appropriate data analytics techniques.<br />
<strong>2E</strong>.1 <strong>Data</strong> <strong>Analytics</strong> and <strong>Internal</strong> <strong>Auditing</strong><br />
IIA <strong>Internal</strong> Audit Competency Framework: In<strong>for</strong>mation Technology<br />
General Awareness: Describe the basic concepts of IT and data analytics. Describe the<br />
various risks related to IT, in<strong>for</strong>mation security, and data privacy. Recognize the purpose and<br />
applications of IT control frameworks and basic IT controls.<br />
Applied Knowledge: Apply data analytics and IT in auditing. Identify and assess various risks<br />
related to IT, in<strong>for</strong>mation security, and data privacy. Apply IT control frameworks.<br />
Expert: Evaluate the use of data analytics and IT in auditing. Recommend actions to address<br />
IT risks, in<strong>for</strong>mation security, and data privacy. Evaluate the use of IT control frameworks. 62<br />
The opportunity to access and analyze large amounts of data has been considerably<br />
enhanced through technology. This is true <strong>for</strong> organizational managers as well as the<br />
internal audit function. There are many potential benefits, including:<br />
• Improving risk identification, analysis, and control.<br />
• Increasing the level of assurance auditors can provide.<br />
• Enhancing efficiency.<br />
• Providing clearer reporting.<br />
• Delivering greater internal audit quality. 63<br />
The ability to be agile and responsive in the face of an ever-changing risk landscape<br />
requires the application of sophisticated tools. Often when we refer to data analytics we<br />
consider a broad-range of tools and methods that are automated and intelligent. However,<br />
many analytical processes and data visualizations can be per<strong>for</strong>med manually (with a<br />
calculator or basic spreadsheet functions), albeit with a high degree of human input. Such<br />
techniques and tools can be taken together or used in isolation. The role of the human<br />
internal auditor is still essential to provide insight, creativity, and judgment. The technology<br />
opens previously unimaginable opportunities. This includes:<br />
• Accessing, recording, and analyzing large amounts of data very quickly.<br />
• Combining internal and external data sets.<br />
• Benchmarking per<strong>for</strong>mance.<br />
• Identifying and reporting unusual transactions, irregularities, and anomalies in real<br />
time through continuous monitoring and continuous auditing.<br />
• Anticipating issues and addressing them be<strong>for</strong>e they occur.<br />
62<br />
<strong>Internal</strong> Audit Competency Framework, The IIA, 2022.<br />
63<br />
See Wolters Kluwer “Five benefits of data analytics <strong>for</strong> internal audit.”<br />
70
The purpose remains to identify weaknesses in risk management and control and to<br />
implement improvements. Like any tool, their utilization is dependent on the intelligence and<br />
creativity of the user. Application of technology does not guarantee better outcomes. Poorly<br />
used tools that are not well understood by the auditor may produce confusing, misleading, or<br />
inaccurate findings.<br />
<strong>2E</strong>.1: Reflection<br />
To what extent does your internal audit unit use data analytics as part of its work?<br />
What expectations do your clients and stakeholders have regarding the use of data<br />
analytics?<br />
What is the greatest barrier to greater use of data analytics?<br />
71
<strong>2E</strong>.2 <strong>Data</strong> <strong>Analytics</strong> Methods<br />
Analytical methods can be grouped according to their main purpose.<br />
• Descriptive methods are designed to report activity and often includes aggregating<br />
and summarizing large amounts of data using averaging and other techniques <strong>for</strong><br />
making comparisons.<br />
• Diagnostic methods are used to interpret in<strong>for</strong>mation and identify likely causal<br />
relationships and trends.<br />
• Predictive methods are used to make <strong>for</strong>ecasts by extrapolating known data and<br />
creating models based on trends and known interdependencies and correlations.<br />
• Prescriptive methods go one step further than predictive methods and suggest<br />
actions to optimize future per<strong>for</strong>mance.<br />
Prior to applying any analytical technique it will be important to validate the data and apply<br />
data hygiene techniques, removing duplicates and inconsistencies. Unstructured data (like<br />
emails, social media posts, contracts, and recordings of phone calls) must be organized and<br />
structured be<strong>for</strong>e it is possible to process it. The analysis can only be as good as the data<br />
you start with. Common types of data validation checks include:<br />
• <strong>Data</strong> type check, confirming the entry of data in a data field is consistent.<br />
• Code check, confirming data con<strong>for</strong>ms to valid values according to set rules.<br />
• Range check, confirming data falls within any set parameters.<br />
• Format check, confirming data consistently matches defined <strong>for</strong>mats.<br />
• Consistency check, confirming logical consistency that matches the process or<br />
activity recorded.<br />
• Uniqueness check, confirming identifiers such as IDs or emails are unique. 64<br />
The following analytical methods are described below and may be utilized manually or by<br />
applying technological tools:<br />
• Variance Analysis.<br />
• Trend Analysis.<br />
• Reasonableness Testing.<br />
• Ratio Estimation.<br />
• Benchmarking.<br />
Many other methods (e.g., decision trees, time series, fuzzy logic) are also available.<br />
<strong>2E</strong>.2.1 Variance Analysis<br />
Variance analysis involves comparing two similar sets of data and attempting to find reasons<br />
<strong>for</strong> any differences. Typically the comparison is between actual outcomes and one or more<br />
of the following:<br />
• Expected or desired outcomes.<br />
• Predicted or <strong>for</strong>ecast outcomes.<br />
• Budgeted outcomes.<br />
• Historical outcomes.<br />
• Comparable benchmarks.<br />
64<br />
See <strong>Data</strong> Validation, Corporate Finance Institute, 2023.<br />
72
This can help managers identify and react to problems or opportunities. Some causes of<br />
variances are purely random and can be eliminated. The comparison of two sets of data can<br />
help determine the extent to which they are correlated.<br />
Variance analysis is discussed further in <strong>Module</strong> 3.<br />
<strong>2E</strong>.2.2 Trend Analysis<br />
Trends are changes (or variances) in data over time. The changes observed may be:<br />
• Random, to be identified and eliminated or ignored.<br />
• Cyclical, recurring over short cycles, such as higher demands <strong>for</strong> customer services<br />
at certain times of the day or week.<br />
• Seasonal, recurring over longer cycles, such as peaks and troughs in sales of ice<br />
cream over a year.<br />
• Underlying trends, being the true long-term pattern, often over multiple years, having<br />
isolated random, cyclical, and seasonal factors. Often underlying trends are most<br />
apparent when we can compare data from the some point in a cycle, season, or year<br />
over multiple cycles, seasons, or years.<br />
<strong>2E</strong>.2.3 Reasonableness Testing<br />
Reasonableness testing is another <strong>for</strong>m of variance or trend analysis in which reported or<br />
apparent per<strong>for</strong>mance is compared with what might reasonably have been expected, once<br />
the <strong>for</strong>ecast is adjusted to take account of everything that is known. Variances may highlight<br />
errors or deliberate misstatements. When there seems to be no reasonable explanation,<br />
when things look too good or too bad to be true, then it deserves further investigation.<br />
<strong>2E</strong>.2.4 Ratio Estimation<br />
Findings based on a sample of data can be extrapolated to make assumptions about the<br />
remaining data or used as the basis <strong>for</strong> <strong>for</strong>ecasting. Larger samples help reduce the<br />
likelihood of bias but there is no guarantee the sample is representative of the whole<br />
population. Statistical modeling is used to calculate the degree of confidence in the analysis.<br />
<strong>2E</strong>.2.5 Benchmarking<br />
Comparing actual per<strong>for</strong>mance with benchmarking data can help identify errors,<br />
weaknesses, and opportunities <strong>for</strong> improvement to align more closely with best practice.<br />
<strong>2E</strong>.2: Reflection<br />
Consider these commonly used methods <strong>for</strong> analysis of data:<br />
Variance analysis<br />
Trend analysis<br />
Reasonable testing<br />
Ratio estimation<br />
Benchmarking<br />
Which of these do you commonly use?<br />
Which of these do you feel you need more help in developing your competency?<br />
73
<strong>2E</strong>.3 <strong>Data</strong> <strong>Analytics</strong> Tools<br />
To help with the heavy lifting, auditors can take advantage of a wide array of technological<br />
solutions all of which are being used increasingly in service delivery, marketing, sales,<br />
human resources, finance, budgeting, and accounting. There are audit specific applications<br />
– both customized and off-the-shelf – <strong>for</strong>:<br />
• Smart apps.<br />
• Utilization of big data.<br />
• Artificial intelligence.<br />
• Machine learning.<br />
• Natural Language Processing NLP.<br />
• Robotic process automation.<br />
• Drones.<br />
• Artificial reality.<br />
Many proprietary tools are available. Vendors often provide consultation, technical support,<br />
and training to aid the adoption of solutions. Comparison websites are useful to help identify<br />
suitable options depending on organizational need and available resources. 65<br />
For the internal audit function, such enablers may be used to support the following:<br />
• Remote auditing, using drones, artificial reality, robots, teleconferencing, and other<br />
similar tools.<br />
• Automated audit management, using audit software <strong>for</strong> planning, communication,<br />
storage and access, reporting, supervision, monitoring, and open issue tracking.<br />
• Advanced data analytics, using number-crunching, machine learning, artificial<br />
intelligence, and data visualization tools.<br />
• <strong>Internal</strong> cooperation and collaboration with other functions, especially assurance<br />
providers, through alignment with governance, risk management, and internal control<br />
plat<strong>for</strong>ms.<br />
• Significant reduction of manual and repetitive tasks and unnecessary duplication of<br />
data sets through robot process automation and reconciliation.<br />
• Continuous auditing through artificial intelligence.<br />
Artificial intelligence (AI) is a “hot topic” which much discussion about apps such as Chat<br />
GPT which may revolutionize many activities. It is not possible to predict exactly how AI will<br />
impact internal auditing but it is clear it creates new opportunities. Manual tasks such as data<br />
extraction can be automated much more quickly and without errors. Rather than relying on a<br />
sample, AI can per<strong>for</strong>m analytics on complete population sets and in real time to identify<br />
trends, anomalies, errors, and potential fraud. AI can also anticipate new and emerging risks<br />
through rigorous interrogation of historical and recent data.<br />
<strong>Internal</strong> audit functions are often criticized <strong>for</strong> being slow to adopt technology in comparison<br />
with other functions. A recent article recounted the excuses commonly given by auditors:<br />
• Don’t have the budget.<br />
• Don’t have the time.<br />
• Don’t have the right people.<br />
65<br />
See <strong>for</strong> example, The Best <strong>Data</strong> <strong>Analytics</strong> Tools & Software of 2023, Forbes, 2023.<br />
74
• Don’t have the right leadership.<br />
• Don’t have proper support.<br />
• Don’t have the right knowledge on the team.<br />
• Inertia and complacency. 66<br />
What can internal audit do about these obstacles? A lack of the right people and sufficient<br />
budget can be significant inhibitors which is why it is necessary to persuade senior<br />
management and the governing body. <strong>Internal</strong> audit managers should develop a robust<br />
strategic plan aligned with organizational priorities and based on a clear vision of an<br />
advanced, agile, and responsive function in which utilization of technology plays a central<br />
role. The mission of helping management and the board make timely interventions to<br />
support organizational success is the justification needed <strong>for</strong> investing in internal audit digital<br />
trans<strong>for</strong>mation. To attract the best talent and to deliver the maximum value requires audit<br />
functions to be leaders of innovation. It can be easy to be complacent about the current state<br />
and essential to break that mindset in favor of continuous positive momentum, even if taken<br />
incrementally. Audit leaders can start small and build on success. Senior management and<br />
the governing body may not be pressing the audit function to change but it should be part of<br />
every auditor’s DNA to strive <strong>for</strong> continual improvement. Sometimes an external quality<br />
review can help provide added weight to the case <strong>for</strong> investment in technology. As the<br />
advocate <strong>for</strong> continuous improvement, audit managers need to implement processes <strong>for</strong><br />
identifying and evaluating opportunities <strong>for</strong> improving the delivery of service excellence.<br />
In addition to the use the internal audit function may make of artificial intelligence and other<br />
technological innovations, they also represent potential opportunities and threats <strong>for</strong> other<br />
parts of an organization. <strong>Internal</strong> auditors need to be well in<strong>for</strong>med to be able to offer<br />
meaningful assurance, insight, and advice. The IIA has produced a three part series to help<br />
internal auditors. 67 The guidance references four different kinds of AI:<br />
Type I. Reactive machines: This is AI at its simplest. Reactive machines respond to<br />
the same situation in exactly the same way, every time. An example of this is a<br />
machine that can beat world-class chess players because it has been programmed<br />
to recognize the chess pieces, know how each moves, and can predict the next<br />
move of both players.<br />
Type II. Limited memory: Limited memory AI machines can look to the past, but the<br />
memories are not saved. Limited memory machines cannot build memories or “learn”<br />
from past experiences. An example is a self-driving vehicle that can decide to change<br />
lanes because a moment ago it noted an obstacle in its path.<br />
Type III. Theory of mind: Theory of mind refers to the idea that a machine could<br />
recognize that others it interacts with have thoughts, feelings, and expectations. A<br />
machine embedded with Type III AI would be able to understand others’ thoughts,<br />
feelings, and expectations, and be able to adjust its own behavior accordingly.<br />
Type IV. Self-awareness: A machine embedded with Type IV AI would be selfaware.<br />
An extension of “theory of mind,” a conscious or self-aware machine would be<br />
66<br />
Garyn, Hal, “Why Is <strong>Internal</strong> Audit Often A Tech Laggard?” <strong>Internal</strong> Audit 360, 2022.<br />
67<br />
Global Perspectives & Insights, The IIA, various.<br />
75
aware of itself, know about its internal states, and be able to predict the feelings of<br />
others. 68<br />
The guidance also highlights potential opportunities and threats.<br />
Opportunities<br />
• The ability to compress the data processing cycle.<br />
• The ability to reduce errors by replacing human actions with perfectly repeatable<br />
machine actions.<br />
• The ability to replace time-intensive activities with time-efficient activities (process<br />
automation), reducing labor time and costs.<br />
• The ability to have robots or drones replace humans in potentially dangerous<br />
situations.<br />
• The ability to make better predictions, <strong>for</strong> everything from predicting sales of certain<br />
goods in particular markets to predicting epidemics and natural catastrophes.<br />
• The ability to drive revenue and grow market share through AI initiatives.<br />
Threats<br />
• Unidentified human biases are imbedded in the AI technology.<br />
• Human logic errors are imbedded in the AI technology.<br />
• Inadequate testing and oversight of AI results in ethically questionable results.<br />
• AI products and services cause harm, resulting in financial and/or reputational<br />
damage.<br />
• Customers or other stakeholders do not accept or adopt the organization’s AI<br />
initiatives.<br />
• The organization is left behind by competitors if it does not invest in AI.<br />
• Investment in AI (infrastructure, research and development, and talent acquisition)<br />
does not yield an acceptable ROI. 69<br />
Which of these apply to your situation?<br />
<strong>2E</strong>.3: Reflection<br />
Don’t have the budget.<br />
Don’t have the time.<br />
Don’t have the right people.<br />
Don’t have the right leadership.<br />
Don’t have proper support.<br />
Don’t have the right knowledge on the team.<br />
Inertia and complacency (we don’t need to do more – clients and stakeholders are happy).<br />
68<br />
Artificial Intelligence, <strong>Internal</strong> Audit’s Role, and Introducing a New Framework Part 1, The IIA, 2017.<br />
69<br />
Artificial Intelligence, <strong>Internal</strong> Audit’s Role, and Introducing a New Framework Part 1, The IIA, 2017.<br />
76
<strong>2E</strong>.4 <strong>Data</strong> Visualization<br />
Auditors gather data from many sources and in different ways. Examples include:<br />
• Interviewing people or conducting focus groups within or outside of the areas being<br />
audited.<br />
• Using questionnaires or checklists to collect in<strong>for</strong>mation, including observations and<br />
opinions from people who work in or deal with the business area being audited.<br />
• Observing the workings within a business area over a period of time to spot issues or<br />
inconsistencies.<br />
• Vertical auditing, in which the auditor monitors one process from beginning to end to<br />
identify any issues.<br />
• Documenting <strong>for</strong>mal practices and procedures within a business area.<br />
• Accessing in<strong>for</strong>mal documentation that may provide insights into ad hoc processes<br />
and procedures. 70<br />
<strong>Data</strong> must be validated by:<br />
• Evaluating whether the data has come from a reliable source and makes sense in<br />
context with the auditors’ overall understanding of the business area.<br />
• Considering how many sources the data are derived from, as well as how long they<br />
took to obtain, to determine if these factors raise risks to data integrity. 71<br />
<strong>Data</strong> visualization can be defined as<br />
the practice of translating in<strong>for</strong>mation into a visual context, such as a map or graph, to<br />
make data easier <strong>for</strong> the human brain to understand and pull insights from. The main<br />
goal of data visualization is to make it easier to identify patterns, trends and outliers in<br />
large data sets. The term is often used interchangeably with others, including in<strong>for</strong>mation<br />
graphics, in<strong>for</strong>mation visualization and statistical graphics. 72<br />
Software makes it easy to create appealing graphics to communicate key in<strong>for</strong>mation<br />
succinctly and powerfully. It can also be a temptation <strong>for</strong> overelaborate and complex pictures<br />
that obscure more than they reveal. Choice of colors, 3-D effects, font size, animation, and<br />
other graphical elements combined with decisions regarding what data to include and to<br />
what level of detail create challenges <strong>for</strong> an auditor completing a report or preparing a<br />
presentation. 73 Auditors should be familiar with the best way to use pie charts, bar charts,<br />
and so on. Some common techniques are described below, based on a Harvard Business<br />
School list. 74<br />
70<br />
<strong>Data</strong> <strong>Analytics</strong> Part 2: Gathering, Understanding, and Visualizing <strong>Data</strong>, The IIA, 2022.<br />
71<br />
<strong>Data</strong> <strong>Analytics</strong> Part 2: Gathering, Understanding, and Visualizing <strong>Data</strong>, The IIA, 2022.<br />
72<br />
<strong>Data</strong> Visualization, TechTarget, Business <strong>Analytics</strong>, 2022.<br />
73<br />
Examples of data visualization tools can be found at The Best <strong>Data</strong> Visualization Tools of 2023, Forbes, 2023.<br />
74<br />
See Harvard Business School online, https://online.hbs.edu/blog/post/data-visualization-techniques<br />
77
Technique and Description<br />
A pie chart is a segmented circle depicting relative<br />
proportions of categories. Used <strong>for</strong>. An exploded pie<br />
chart highlights a segment or segments of particular<br />
relevance.<br />
A bar graph compare multiple categories of data.<br />
Variants include vertical, horizontal, segmented,<br />
and others.<br />
A histogram looks similar to a bar chart, but it<br />
illustrates how a variable changes.<br />
A Gantt chart organizes events in relation to each<br />
other to show their relative timing, how they are<br />
linked, and their current status.<br />
A heat map organizes data with different colors<br />
(often using a spectrum from cold to hot) to indicate<br />
relative size and significance.<br />
A box and whisker diagram illustrates frequency<br />
and marks the upper and lower quartiles in the <strong>for</strong>m<br />
of a box and the median as a line inside the box.<br />
The “whiskers” show the range from the highest and<br />
lowest values.<br />
A waterfall chart shows a stepwise progress of a<br />
variable in relation to another factor.<br />
An area chart is a line graph under which the areas<br />
are shaded.<br />
A scatter plot illustrates the relationship between<br />
two variables.<br />
A pictogram uses a relevant icon or picture to show<br />
relative size or changes in a variable. The picture<br />
may be repeated or scaled in proportion.<br />
A timeline depicts related in<strong>for</strong>mation arranged in<br />
chronological order.<br />
A highlight table deploys colors within a data grid<br />
to show areas of significance.<br />
A bullet graph variation of a simple bar graph to<br />
show live data variances with benchmarks, targets,<br />
or expected values.<br />
A choropleth map is a shaded geographical map.<br />
A word cloud is an array of words using font size to<br />
show relative frequency.<br />
A network diagram comprises nodes and links that<br />
illustrate connections between different points.<br />
A correlation matrix uses colors to emphasize<br />
emphasis and appeal.<br />
Usefulness<br />
Useful <strong>for</strong> conveying simple messages<br />
about comparative weightings<br />
Useful <strong>for</strong> quick comparison of size.<br />
Useful <strong>for</strong> illustrating changes over<br />
time or in relation to another variable.<br />
Useful <strong>for</strong> project management.<br />
Among other applications, heat maps<br />
are often used to show risk data.<br />
Useful <strong>for</strong> communicating frequency<br />
data and its relative spread.<br />
Useful <strong>for</strong> communicating significant<br />
moments of change in values.<br />
Provides a visual cue of relative<br />
proportions.<br />
Useful <strong>for</strong> a quick visual determination<br />
if there is a direct, indirect, or negligible<br />
relationship between two variables.<br />
Useful <strong>for</strong> a quick and impactful<br />
illustration of relative proportions.<br />
Useful <strong>for</strong> illustrating developments<br />
over time.<br />
Useful <strong>for</strong> communicating such<br />
features about the data as<br />
highest/lowest, trends, anomalies, etc.<br />
Useful <strong>for</strong> showing current activity in<br />
comparison with desired outcomes and<br />
<strong>for</strong> creating dashboards.<br />
Useful <strong>for</strong> depicting regional variations.<br />
Useful <strong>for</strong> showing the most frequently<br />
occurring topics or words from<br />
unstructured in<strong>for</strong>mation.<br />
Useful <strong>for</strong> identifying relationships<br />
among individuals, teams,<br />
organizations, geographical locations,<br />
centers of activity, and more<br />
Useful <strong>for</strong> communicating correlation<br />
coefficients easily.<br />
78
The secrets to successful visualized reporting are:<br />
• Using the appropriate technique to communicate what you are trying to show.<br />
• Keeping it simple.<br />
• Avoiding excessive use of colors, animation, special effects, etc.<br />
• Including only what is relevant.<br />
• Avoiding inclusion of every data point but be careful not to create misleading<br />
graphics.<br />
• Helping the intended audience focus on what is most important.<br />
<strong>2E</strong>.4: Reflection<br />
From recent internal and external reports you have seen, identify examples of good and bad<br />
practice with respect to presenting data, and share these examples with your fellow<br />
students.<br />
What is it about the examples you have that make the data presentation effective or<br />
ineffective?<br />
Do you feel confident using graphics to represent data in your audit reports and<br />
presentations?<br />
Have you asked your clients whether they find your reports (especially tables and<br />
graphics) clear and helpful?<br />
What more can you do to improve the user-friendliness of data as presented in your<br />
reports?<br />
79
References and Additional Reading<br />
7 Ways to Win the <strong>Internal</strong> Audit Budget Argument with your CFO, AuditBoard, 2019.<br />
https://www.auditboard.com/blog/7-ways-to-win-the-budget-argument-with-your-cfo/<br />
Artificial Intelligence, <strong>Internal</strong> Audit’s Role, and Introducing a New Framework Part 1<br />
Becoming agile: A guide to elevating internal audit’s per<strong>for</strong>mance and value, Deloitte, 2017.<br />
https://www2.deloitte.com/content/dam/Deloitte/global/Documents/Finance/gx-fa-agileinternal-audit-introduction-elevating-per<strong>for</strong>mance.pdf<br />
The Best <strong>Data</strong> <strong>Analytics</strong> Tools & Software 2023, Forbes, 2023.<br />
https://www.<strong>for</strong>bes.com/advisor/business/software/best-data-analytics-tools/<br />
The Best <strong>Data</strong> Visualization Tools of 2023, Forbes, 2023.<br />
https://www.<strong>for</strong>bes.com/advisor/business/software/best-data-visualization-tools/Brown,<br />
Brené, Dare To Lead: Brave Work, Tough Conversations, Whole Hearts, 2018.<br />
Collins, Jim, Good To Great: Why Some Companies Make the Leap…And Others Don’t,<br />
Harper Collins, 2001.<br />
Computer Assisted Audit Techniques (CAATS): Definition, types, advantages and<br />
disadvantages, Accounting Hub. https://www.accountinghub-online.com/computerassisted-audit-techniques/<br />
COSO <strong>Internal</strong> Control – Integrated Framework, COSO, 2017.<br />
COSO Enterprise Risk Management – Integrating with Strategy and Per<strong>for</strong>mance, COSO,<br />
2017.<br />
<strong>Data</strong> Validation, Corporate Finance Institute, 2023.<br />
https://corporatefinanceinstitute.com/resources/data-science/data-validation/<br />
<strong>Data</strong> Visualization, TechTarget, Business <strong>Analytics</strong>, 2022,<br />
https://www.techtarget.com/searchbusinessanalytics/definition/data-visualization<br />
<strong>Data</strong> <strong>Analytics</strong> Part 2: Gathering, Understanding, and Visualizing <strong>Data</strong>, The IIA, 2022.<br />
https://www.theiia.org/globalassets/site/content/articles/global-knowledgebrief/2022/data-analytics-part-2-gathering-understanding-and-visualizing-data/dataanalytics-part-2--final.pd<br />
Garyn, Hal, “Why Is <strong>Internal</strong> Audit Often A Tech Laggard?” <strong>Internal</strong> Audit 360, 2022.<br />
https://internalaudit360.com/why-internal-audit-is-a-tech-laggard-and-how-to-fix-it/<br />
Global Knowledge Brief: Audit Team Trans<strong>for</strong>mation, The IIA, 2019.<br />
https://www.theiia.org/globalassets/site/content/articles/global-knowledgebrief/2019/april/audit-team-trans<strong>for</strong>mation/audit-team-trans<strong>for</strong>mation_iia_global_kb.pdf<br />
Global Perspectives & Insights, From Con<strong>for</strong>mance to Ambition: Applying the <strong>Internal</strong> Audit<br />
Ambition Model, The IIA, 2020.<br />
https://www.theiia.org/globalassets/documents/content/articles/gpi/2020/august/globalperspectives-and-insights---ambition-model_final-1.pdf<br />
80
Global Risks Report 2023, World Economic Forum, 2023.<br />
https://www3.we<strong>for</strong>um.org/docs/WEF_Global_Risks_Report_2023.pdf<br />
Guidelines <strong>for</strong> Conducting a Quality Assessment (QA), Deutsches Institut für Interne<br />
Revision e.V. (IIA–Germany), September 2007.<br />
https://www.diir.de/fileadmin/fachwissen/standards/downloads/DIIR_Revisionsstandard_<br />
Nr_3_englisch_V1.1.pdf<br />
IIA Global Perspectives and Insights Artificial Intelligence three parts, The IIA, 2017-18.<br />
IIA Competency Framework, The IIA, 2022.<br />
https://www.theiia.org/globalassets/documents/standards/ia-competencyframework/2022-4103-sem-competency-framework-graphics-table_fnl.pdf<br />
https://www.theiia.org/en/content/articles/global-perspectives-and-insights/2018/artificialintelligence-internal-audits-role-and-introducing-a-new-framework-part-1/<br />
https://www.theiia.org/globalassets/documents/content/articles/gpi/2017/december/gpiartificial-intelligence-part-ii.pdf<br />
https://www.theiia.org/en/content/articles/global-perspectives-and-insights/2018/artificialintelligence-part-iii/<br />
IIA Position Paper: Staffing/Resourcing Considerations <strong>for</strong> <strong>Internal</strong> Audit Activity, The IIA,<br />
2018. https://www.theiia.org/globalassets/documents/resources/staffing-considerations<strong>for</strong>-internal-audit-activity-may-2018/staffing-considerations-<strong>for</strong>-internal-audit-activity.pdf<br />
The Institute of <strong>Internal</strong> Auditors’ Quality Assessment Manual <strong>for</strong> the <strong>Internal</strong> Audit Activity,<br />
6th Edition. https://www.theiia.org/en/products/bookstore/quality-assessment-manual-<strong>for</strong>the-internal-audit-activity-qa-manual/<br />
<strong>Internal</strong> Audit Capability Model (IA-CM) <strong>for</strong> the Public Sector, The IIA, 2023.<br />
https://www.theiia.org/en/promotions/bookstore/IA-CM/<br />
<strong>Internal</strong> Audit’s Role in ESG Reporting, The IIA, 2021.<br />
https://www.theiia.org/globalassets/documents/standards/implement-thestandards/2021/internal-audits-role-in-esg-reporting-independent-assurance-is-critical-toeffective-sustainability-reporting/white-paper-internal-audits-role-in-esg-reporting.pdf<br />
International Professional Practices Framework, The IIA, 2016.<br />
https://www.theiia.org/en/standards/international-professional-practicesframework/?gclid=CjwKCAiA0JKfBhBIEiwAPhZXDwsJCKbFTF34RJmKqKME7VhbLi9b0<br />
s_GYUUIcz-ae9H7y8VAyBYe-RoCuH8QAvD_BwE<br />
ISO 31000: Risk Management, ISO, 2018.<br />
Landry, Lauren, “How to Delegate Effectively: 9 Tips For Managers,” Harvard Business<br />
School, 2020. https://online.hbs.edu/blog/post/how-to-delegate-effectively<br />
Lean <strong>Auditing</strong> (Part 1): Rethinking <strong>Internal</strong> Audit Using Lean Techniques to Enhance Value<br />
and Improve Productivity, efficientlearning.com,<br />
https://www.efficientlearning.com/blog/lean-auditing-part-one/<br />
81
Lean <strong>Auditing</strong> (Part 2): Rethinking <strong>Internal</strong> Audit Using Lean Techniques to Enhance Value<br />
and Improve Productivity, efficientlearning.com<br />
https://www.efficientlearning.com/blog/lean-auditing-part-two/<br />
Lean <strong>Auditing</strong> (Part 3): Rethinking <strong>Internal</strong> Audit Using Lean Techniques to Enhance Value<br />
and Improve Productivity, efficientlearning.com<br />
https://www.efficientlearning.com/blog/lean-auditing-part-three/<br />
Local Management Guide: The Practice of <strong>Internal</strong> Controls, Office o the New York State<br />
Comptroller, 2010. https://www.osc.state.ny.us/files/localgovernment/publications/pdf/the-practice-of-internal-controls.pdf<br />
Mintzberg, Henry, Mintzberg on Management, Free Press, Simon & Schuster, Inc., 1989.<br />
“One page audit report: maximizing efficiency, elevating impact,” AuditBoard, 2022.<br />
https://www.auditboard.com/blog/one-page-audit-report-maximizing-efficiency-elevatingimpact/<br />
Paterson, James, Lean <strong>Auditing</strong>: Driving Added Value and Efficiency in <strong>Internal</strong> Audit, Wiley,<br />
2015.<br />
“Planning <strong>for</strong> uncertainty: the rise of the flexible audit plan,” AuditBoard, 2022.<br />
https://www.auditboard.com/blog/planning-<strong>for</strong>-uncertainty-the-rise-of-the-flexible-auditplan/<br />
Practice Guide: <strong>Auditing</strong> Culture, The IIA, 2019.<br />
https://www.theiia.org/globalassets/documents/content/articles/guidance/practiceguides/auditing-culture/pg-auditing-culture.pdf<br />
Practice Guide: <strong>Auditing</strong> Third Party Risk Management, The IIA, 2018.<br />
https://www.theiia.org/globalassets/documents/content/articles/guidance/practiceguides/auditing-third-party-risk-management/pg-auditing-third-party-riskmanagement.pdf<br />
Practice Guide: Developing the <strong>Internal</strong> Audit Strategic Plan, The IIA, 2012.<br />
https://www.theiia.org/globalassets/documents/content/articles/guidance/practiceguides/developing-the-internal-audit-strategic-plan/pg-developing-the-internal-auditstrategic-plan.pdf<br />
The Agile <strong>Internal</strong> Audit Journey Series Part 1: Trans<strong>for</strong>ming <strong>Internal</strong> Audit To Add Greater<br />
Value, BakerTilly, 2019. https://www.bakertilly.com/insights/the-agile-internal-auditjourney-series-part-i--trans<strong>for</strong>ming-internal-audit-to-add-greater-value<br />
Practice Guide: Quality Assurance and Improvement Program, The IIA, 2012.<br />
https://www.theiia.org/globalassets/documents/content/articles/guidance/practiceguides/quality-assurance-and-improvement-program/ippf-pg---quality-assurance-andimprovement-program.pdf<br />
Risk Appetite: Critical to Success, COSO, 2020.<br />
https://www.coso.org/Shared%20Documents/COSO-Guidance-Risk-Appetite-Critical-to-<br />
Success.pdf<br />
82
Risk Assessment in Practice, COSO, 2012.<br />
https://www.coso.org/Shared%20Documents/COSO-ERM-Risk-Assessment-in-Practice-<br />
Thought-Paper-October-2012.pdf<br />
Risk in focus 2023: more risky, uncertain, and volatile times ahead, ECIIA, 2023.<br />
https://www.eciia.eu/2022/09/risk-in-focus-2023-more-risky-uncertain-and-volatile-timesahead/<br />
Wolters Kluwer “Five benefits of data analytics <strong>for</strong> internal audit.”<br />
https://www.wolterskluwer.com/en/expert-insights/5-benefits-of-da ta-analytics-<strong>for</strong>internal-audit<br />
83
CIPFA: 77 Mansell Street, London E1 8AN<br />
+44 20 7543 5600<br />
cipfa.org<br />
CEF: Cankarjeva cesta 18, 1000 Ljubljana, Slovenia<br />
+386 1 369 61 90<br />
cef-see.org<br />
The Chartered Institute of Public Finance and Accountancy. Registered with the Charity<br />
Commissioners of England and Wales No 231060. Registered with the Office of the<br />
Scottish Charity Regulator No SCO37963.<br />
84