10.07.2015 Views

From Data to Decisions - FIVE STEPS TO EVIDENCE BASED MANAGEMENT

From Data to Decisions - FIVE STEPS TO EVIDENCE BASED MANAGEMENT

From Data to Decisions - FIVE STEPS TO EVIDENCE BASED MANAGEMENT

SHOW MORE
SHOW LESS
  • No tags were found...

Create successful ePaper yourself

Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.

<strong>From</strong> <strong>Data</strong><strong>to</strong> <strong>Decisions</strong><strong>FIVE</strong> <strong>STEPS</strong> <strong>TO</strong> <strong>EVIDENCE</strong>-<strong>BASED</strong> <strong>MANAGEMENT</strong>Bernard MarrWhat is the issue?Most organizations are drowning in data whilethirsting for good fact-based insights that cansupport decision making.What can be done?Evidence-based management provides astructured approach for turning data in<strong>to</strong>critical management decisions.Why is it important?At no point in his<strong>to</strong>ry have organizations hadaccess <strong>to</strong> more data, be it financial or non-financial.Today, an organization’s success depends on itsability <strong>to</strong> gain fact-based insights faster than thecompetition, and <strong>to</strong> turn those insights in<strong>to</strong> gooddecision making.GUIDANCE


The material contained in this Guidance is designed <strong>to</strong> provide illustrative information of thesubject matter covered. It does not establish standards or preferred practices. This material has notbeen considered or acted upon by any senior or technical committees or the board of direc<strong>to</strong>rs ofCPA Canada and does not represent an official opinion or position of CPA Canada.© 2013 Chartered Professional Accountants of CanadaAll rights reserved. This publication is protected by copyright and written permission is required<strong>to</strong> reproduce, s<strong>to</strong>re in a retrieval system or transmit in any form or by any means (electronic,mechanical, pho<strong>to</strong>copying, recording, or otherwise).For information regarding permission, please contact permissions@cpacanada.caCPA Canada277 Welling<strong>to</strong>n Street WestToron<strong>to</strong>, ON Canada M5V 3H2T. 416 977.3222 F. 416 977.8585www.cpacanada.ca


<strong>From</strong> <strong>Data</strong> <strong>to</strong> <strong>Decisions</strong>: GUIDANCE1<strong>From</strong> data <strong>to</strong> decisionsA cursory glance in<strong>to</strong> the operations of most organizationsshows a common challenge, irrespective of theirsize, industry or sec<strong>to</strong>r. Most struggle <strong>to</strong> turn the massof data now available <strong>to</strong> them in<strong>to</strong> the critical knowledgerequired <strong>to</strong> win in <strong>to</strong>day’s fiercely competitiveand highly unpredictable markets.At the same time there are pioneering organizationsthroughout the world that are using an emerging disciplinecalled evidence-based management (EBM) as away <strong>to</strong> improve their competitive positions. ThroughEBM, organizations explicitly use the best and mostappropriate data <strong>to</strong> guide the decision-making processes.Crucially, however, EBM involves much morethan just the collection and s<strong>to</strong>rage of data and informationin large quantities — it also requires buildingcompetitive strategies around data-driven insights(see Text Box 1 for an overview of key terms).Robert Sut<strong>to</strong>n, a professor at Stanford University,argues: “Evidence-based management is a simple idea.It just means finding the best evidence you can, facingthose facts, and acting on those facts — rather thandoing what everyone else does, what you have alwaysdone, or what you thought was true.”


2 <strong>From</strong> <strong>Data</strong> <strong>to</strong> <strong>Decisions</strong>: GUIDANCETEXT BOX 1 Defining some key terms The five-step EBM model<strong>Data</strong> comes in a myriad of forms. It includes numbers,words, sounds, or pictures, but without context (e.g.,15/3, 5, 68).Evidence is any data or information that might be used<strong>to</strong> determine the truth of an assertion.Information includes a collection of words, numbers,sounds, or pictures that have meaning (e.g., on the 15thof March at 5 p.m., we were all at no. 68 Vic<strong>to</strong>ria St.).Knowledge is acquired when we take in and understandinformation about a subject, which then allows us <strong>to</strong> formjudgments <strong>to</strong> support decision making, and then act onthe decision. We do this by using rules about how theworld works based on information we have gained frompast experience.Business intelligence (BI) refers <strong>to</strong> technologies, applications,and practices for collecting, integrating, analyzing,and presenting business information.Analytics refers <strong>to</strong> the use of (a) data and evidence,(b) statistical, quantitative, and qualitative analysis,(c) explana<strong>to</strong>ry and predictive models, and (d) factbasedmanagement <strong>to</strong> drive decision making.Big data analytics refers <strong>to</strong> the analysis of data thatcome in vast volumes, is often fast-moving and usuallyvaried in formats (structured and unstructured). Big datais <strong>to</strong>o messy for traditional business intelligence andanalytics approaches.There are five steps for the effective deployment ofEBM (Figure 1). This begins with Step 1 — Definingobjectives and information needs. During this step,these questions are asked: “What are our strategicaims?” and “Based on those aims, what do we need<strong>to</strong> know?” This vital first step ensures we clearlyarticulate the real information needs, and clarifywho needs <strong>to</strong> know what, when, and why. Step 2 —Collecting data — calls for gathering and organizingthe right data. The emphasis here is on meaningfuland relevant data <strong>to</strong> meet the information needsidentified in Step 1. Organizations need <strong>to</strong> (a) assesswhether the needed data is already held somewherein the organization, or (b) know the best way <strong>to</strong> collectthe data. Step 3 — Analyzing data — focuses on turningdata in<strong>to</strong> relevant insights. <strong>Data</strong> has <strong>to</strong> be analyzedand put in<strong>to</strong> context <strong>to</strong> extract information. Step 4 —Presenting information — focuses on communicatingthe information and insights extracted in Step 3. Themain focus here is <strong>to</strong> get the information, in its mostappropriate form, <strong>to</strong> the decision makers.Step 5 — Making evidence-based decisions — is concernedwith turning information in<strong>to</strong> knowledge anddecisions. The emphasis here is on making sure theavailable evidence is used <strong>to</strong> make the best decisions.Here, it is important <strong>to</strong> create a knowledge-<strong>to</strong>-actionculture and avoid the knowing-doing gap so prevalentin many organizations <strong>to</strong>day.In addition <strong>to</strong> the five steps there is a feedback loopbetween the last and the first step — after learninghas taken place and decisions have been made, theyin turn inform future informational needs.As one can see from the framework, there is a sixthbox — IT infrastructure and business intelligence (BI)applications as enablers. Even though it is not a step inits own right, IT and BI play a crucial role in evidencebasedmanagement. They are critical enablers of thedata collection process, data analysis, and the presentationand dissemination of information.


<strong>From</strong> <strong>Data</strong> <strong>to</strong> <strong>Decisions</strong>: GUIDANCE3FIGURE 1EBM framework1 2 3 4 5DEFININGOBJECTIVES ANDINFORMATIONNEEDSCOLLECTINGDATAANALYZINGDATAPRESENTINGINFORMATIONMAKING<strong>EVIDENCE</strong>-<strong>BASED</strong>DECISIONSWhat are ourstrategic aims?Based on thoseaims, what do weneed <strong>to</strong> know?Can we clearlyarticulate ourinformation needs?Who needs <strong>to</strong>know what, when,and why?Do we have orcan we collectmeaningful andrelevant data<strong>to</strong> meet ourinformation needs?How can we turnthe data in<strong>to</strong>relevant insights?How can we putthe data in<strong>to</strong>context and extractinformation?How can webest present andcommunicatethe insights andinformation <strong>to</strong>inform decisionmakers?IT INFRASTRUCTURE AND BUSINESS INTELLIGENCEAPPLICATIONS AS ENABLERSHow do we best leverage our information technologyinfrastructure and our business intelligence applications<strong>to</strong> support evidence-based decision making?How do we ensurethat the availableevidence is used<strong>to</strong> make the bestdecisions?How do we createa knowledge-<strong>to</strong>actionculture?How do we avoidthe knowing–doinggap?FEEDBACK LOOP


4 <strong>From</strong> <strong>Data</strong> <strong>to</strong> <strong>Decisions</strong>: GUIDANCESTEP 1Defining objectives andinformation needsSuccessfully negotiating Step 1 requires careful answering of one key question:“What do you need <strong>to</strong> know?” In most organizations, the use of BI and analyticsis driven more by the information that is available than by the information needed<strong>to</strong> make essential organization decisions. This is clearly back-<strong>to</strong>-front. EffectiveEBM should be driven by the needs of the decision makers. In essence, identify theinformation needs first and apply BI and analytical capabilities accordingly.1 2 3 4 5DEFININGOBJECTIVES ANDINFORMATIONNEEDSCOLLECTINGDATAANALYZINGDATAPRESENTINGINFORMATIONMAKING<strong>EVIDENCE</strong>-<strong>BASED</strong>DECISIONSWhat are ourstrategic aims?Based on thoseaims, what do weneed <strong>to</strong> know?Can we clearlyarticulate ourinformation needs?Who needs <strong>to</strong>know what, when,and why?Do we have orcan we collectmeaningful andrelevant data<strong>to</strong> meet ourinformation needs?How can we turnthe data in<strong>to</strong>relevant insights?How can we putthe data in<strong>to</strong>context and extractinformation?How can webest present andcommunicatethe insights andinformation <strong>to</strong>inform decisionmakers?IT INFRASTRUCTURE AND BUSINESS INTELLIGENCEAPPLICATIONS AS ENABLERSHow do we best leverage our information technologyinfrastructure and our business intelligence applications<strong>to</strong> support evidence-based decision making?How do we ensurethat the availableevidence is used<strong>to</strong> make the bestdecisions?How do we createa knowledge-<strong>to</strong>actionculture?How do we avoidthe knowing–doinggap?FEEDBACK LOOP


<strong>From</strong> <strong>Data</strong> <strong>to</strong> <strong>Decisions</strong>: GUIDANCE5Identify the strategic objective/information needFirst, it is critical we link the data that organizationscollect <strong>to</strong> the strategy and the key drivers of value andperformance. By doing so, we ensure the analytics wegenerate (a) are relevant <strong>to</strong> the organization’s competitivepositioning, (b) support its greatest informationneeds, and (c) are not wasted on irrelevant “interesting<strong>to</strong> know” issues. One very effective way <strong>to</strong> articulatea strategy is through the use of a strategy map, whichallows organizations <strong>to</strong> express their strategy ona simple one-page document that can then be used<strong>to</strong> anchor any future data requirements.Identify who has the information needThe second phase is <strong>to</strong> identify who needs the information.Here it is important <strong>to</strong> define the target audience(information cus<strong>to</strong>mers). Information cus<strong>to</strong>mers canbe (a) groups of people such as the board of direc<strong>to</strong>rs,senior managers, the HR department, the marketingmanagers, or (b) a single person. It is critical <strong>to</strong> clarifywho requires the information, because different audienceshave vastly different needs, even in relation <strong>to</strong>a single strategic objective.Clarify what questions they wantansweredNext you want <strong>to</strong> identify exactly what questionsthe target audience wants answered. Often, however,recipients of information don’t fully know their exactrequirements. A powerful <strong>to</strong>ol for guiding audiences<strong>to</strong> identifying their specific requirements is <strong>to</strong> formulatekey analytics questions (KAQs). In essence, aKAQ makes sure we know what it is that we want <strong>to</strong>know — that we fully appreciate the exact performanceissue we are grappling with.allows us <strong>to</strong> “do” something about the future. We thenlook at data in a different light, trying <strong>to</strong> understandwhat the data and management information meansfor the future. This helps with data interpretation, andensures we collect data that helps <strong>to</strong> inform our decisionmaking (see Text Box 2 for examples).TEXT BOX 2Examples of key analytics questions• To what extent are we growing profitably?• To what extent are we retaining our most profitablecus<strong>to</strong>mers?• How well are we promoting our services?• How do our cus<strong>to</strong>mers perceive our service?• How effective are we in managing our relationshipswith key suppliers?• How well are we communicating within ourorganization?• How well are we building our new competencies in X?• To what extent do people feel passionate about workingfor our organization?Clarify what decisions need <strong>to</strong> be takenAlthough KAQs narrow the possible data that can beused, it still leaves many possible data sets <strong>to</strong> choosefrom. Another question can be used <strong>to</strong> narrow therange of possible indica<strong>to</strong>rs even further. This questionseeks <strong>to</strong> clearly identify any important decisions thedata would support (See Text Box 3 for examples). Byarticulating the question and the decisions performancedata will possibly help <strong>to</strong> address, it is possible <strong>to</strong> reducethe potential number of indica<strong>to</strong>rs from an almost endlessnumber <strong>to</strong> a smaller and more focused set.TEXT BOX 3Examples of possible questions• Which cus<strong>to</strong>mers <strong>to</strong> target.• How best <strong>to</strong> redesign our website.• The best route for our delivery trucks.• In which part of our branding should we invest?• How best <strong>to</strong> package our service offerings.• Which people should we recruit?• Which part of our production process should wefurther optimize?KAQs should focus on the future. For example, ask“How effective are our attempts <strong>to</strong> increase our marketshare?” instead of “Has our market share increased?”By focusing on the future, we open up a dialogue that


6 <strong>From</strong> <strong>Data</strong> <strong>to</strong> <strong>Decisions</strong>: GUIDANCESTEP 2Collecting dataOnce the strategic information needs are clear the right evidence (right dataand data of the right quality) needs <strong>to</strong> be collected <strong>to</strong> support decision making.1 2 3 4 5DEFININGOBJECTIVES ANDINFORMATIONNEEDSCOLLECTINGDATAANALYZINGDATAPRESENTINGINFORMATIONMAKING<strong>EVIDENCE</strong>-<strong>BASED</strong>DECISIONSWhat are ourstrategic aims?Based on thoseaims, what do weneed <strong>to</strong> know?Can we clearlyarticulate ourinformation needs?Who needs <strong>to</strong>know what, when,and why?Do we have orcan we collectmeaningful andrelevant data<strong>to</strong> meet ourinformation needs?How can we turnthe data in<strong>to</strong>relevant insights?How can we putthe data in<strong>to</strong>context and extractinformation?How can webest present andcommunicatethe insights andinformation <strong>to</strong>inform decisionmakers?IT INFRASTRUCTURE AND BUSINESS INTELLIGENCEAPPLICATIONS AS ENABLERSHow do we best leverage our information technologyinfrastructure and our business intelligence applications<strong>to</strong> support evidence-based decision making?How do we ensurethat the availableevidence is used<strong>to</strong> make the bestdecisions?How do we createa knowledge-<strong>to</strong>actionculture?How do we avoidthe knowing–doinggap?FEEDBACK LOOP


<strong>From</strong> <strong>Data</strong> <strong>to</strong> <strong>Decisions</strong>: GUIDANCE7In its broadest sense, evidence includes any data orinformation that might be used <strong>to</strong> determine the truthof an assertion. Building evidence requires the carefulcollection of the right data. And yet our understandingof the word “data” is confused. People often wronglybelieve the word “data” has a narrow numeric definition.This is incorrect. <strong>Data</strong> comes in myriad forms —sounds, text, graphics, and pictures are as much dataas are numbers.Consequently, it is important <strong>to</strong> become familiar withthe available data collection methodologies. Theseapproaches are usually described as either quantitative(being concerned with the collection of numericaldata) or qualitative (concerned with the collectionof non-numerical data). Both approaches have differentpurposes, and each has identifiable strengthsand weaknesses. What is important is that we get thebalance right between the quantitative and qualitativedata. This can be achieved, for example, by conductinga survey that asked cus<strong>to</strong>mers <strong>to</strong> score their likelihood<strong>to</strong> recommend your company on a scale from one <strong>to</strong>ten (quantitative) and asking additional open questionssuch as “What do you really like about this company?”and “What could we do better?” (qualitative).Qualitative vs. quantitative datacollection methodsThe aim of quantitative data collection methodologiesis <strong>to</strong> classify features, count them, and then constructstatistical models in an attempt <strong>to</strong> explain whatis observed. Quantitative data is usually collectedau<strong>to</strong>matically from operations, or through structuredquestionnaires that incorporate mainly closed questions,with specified answer choices.Collecting evidence and dataBy collecting both quantitative and qualitative data, weare then able <strong>to</strong> begin assigning meaning <strong>to</strong> the data.<strong>Data</strong> can be collected au<strong>to</strong>matically (e.g., web logs,sensor data, etc.) or through surveys (e.g., cus<strong>to</strong>merfeedback questionnaire), focus groups (e.g., employeefeedback sessions), interviews, observations, assessments,etc.When collecting data we have <strong>to</strong> ensure it is reliableand valid. Reliability and validity can be substantiallyheightened through applying the idea of “triangulation”— collecting data using various techniques (e.g.,interviews with board members, middle managers,and front-line workers) and methodologies (e.g.,survey 70% of your suppliers and interview 30%).This allows organizations <strong>to</strong> contrast and comparethe information gathered from use of the differenttechniques. The rationale behind this is the moreinformation we have from as many possible sources,the greater the likelihood that it is reliable.Planning data collectionWhen planning your data collection the followingsteps are recommended:• Decide on the data collection method: Beforedeciding how <strong>to</strong> collect the data, it is important <strong>to</strong>establish whether or not existing data can be used.It is important, though, <strong>to</strong> make sure the existingdata is of the appropriate quality. If appropriatedata is not available or needs <strong>to</strong> be supplementedwith more evidence, new data has <strong>to</strong> be collected.• Decide on the source of the data: At this stage, itis crucially important <strong>to</strong> think about access <strong>to</strong> dataand answer questions such as: “Is the data readilyavailable?” “Is it feasible <strong>to</strong> collect it?” “Will thedata collection method, for example interviewswith senior managers, provide honest information?”If not, it might be appropriate <strong>to</strong> combinevarious data collection methods.• Decide when the data will be collected, and inwhat sequence and frequency: Here, one needs<strong>to</strong> determine when and how often the data forthat indica<strong>to</strong>r should be collected. Some data setsare collected continuously, others hourly, daily,


8 <strong>From</strong> <strong>Data</strong> <strong>to</strong> <strong>Decisions</strong>: GUIDANCEmonthly, or even annually. It is important <strong>to</strong>decide what frequency provides sufficient data<strong>to</strong> answer the key performance questions andhelps <strong>to</strong> support decision making.• Decide on who is responsible for collecting thedata: Here we identify the person, function, orexternal agency responsible for data collection anddata updates. The person responsible for measuringcould be an internal person or function withinyour organization or, increasingly, it can be externalagencies, since many organizations outsourcethe collection of specific data such as cus<strong>to</strong>mer oremployee surveys.


<strong>From</strong> <strong>Data</strong> <strong>to</strong> <strong>Decisions</strong>: GUIDANCE9STEP 3Analyzing dataAfter ensuring we are collecting the right data, we need <strong>to</strong> turn this data in<strong>to</strong>insights and information. In the context of EBM, data analysis is the applicationof analytical <strong>to</strong>ols (e.g., statistical analysis or qualitative text analysis) <strong>to</strong> gainorganizational insights. <strong>Data</strong> analysis is a core requirement in creating evidenceused for decision making. Yet repeated research shows most organizations are stillmore focused on simply collecting and distributing data than they are in doing anymeaningful analysis.1 2 3 4 5DEFININGOBJECTIVES ANDINFORMATIONNEEDSCOLLECTINGDATAANALYZINGDATAPRESENTINGINFORMATIONMAKING<strong>EVIDENCE</strong>-<strong>BASED</strong>DECISIONSWhat are ourstrategic aims?Based on thoseaims, what do weneed <strong>to</strong> know?Can we clearlyarticulate ourinformation needs?Who needs <strong>to</strong>know what, when,and why?Do we have orcan we collectmeaningful andrelevant data<strong>to</strong> meet ourinformation needs?How can we turnthe data in<strong>to</strong>relevant insights?How can we putthe data in<strong>to</strong>context and extractinformation?How can webest present andcommunicatethe insights andinformation <strong>to</strong>inform decisionmakers?IT INFRASTRUCTURE AND BUSINESS INTELLIGENCEAPPLICATIONS AS ENABLERSHow do we best leverage our information technologyinfrastructure and our business intelligence applications<strong>to</strong> support evidence-based decision making?How do we ensurethat the availableevidence is used<strong>to</strong> make the bestdecisions?How do we createa knowledge-<strong>to</strong>actionculture?How do we avoidthe knowing–doinggap?FEEDBACK LOOP


<strong>From</strong> <strong>Data</strong> <strong>to</strong> <strong>Decisions</strong>: GUIDANCE11STEP 4Presenting informationIt is crucial, when analyzing data, <strong>to</strong> keep the target audiences and their specificneeds in mind. After all, organizations gain competitive advantage when the rightinformation is delivered <strong>to</strong> the right people at the right time.1 2 3 4 5DEFININGOBJECTIVES ANDINFORMATIONNEEDSCOLLECTINGDATAANALYZINGDATAPRESENTINGINFORMATIONMAKING<strong>EVIDENCE</strong>-<strong>BASED</strong>DECISIONSWhat are ourstrategic aims?Based on thoseaims, what do weneed <strong>to</strong> know?Can we clearlyarticulate ourinformation needs?Do we have orcan we collectmeaningful andrelevant data<strong>to</strong> meet ourinformation needs?How can we turnthe data in<strong>to</strong>relevant insights?How can we putthe data in<strong>to</strong>context and extractinformation?How can webest present andcommunicatethe insights andinformation <strong>to</strong>inform decisionmakers?How do we ensurethat the availableevidence is used<strong>to</strong> make the bestdecisions?How do we createa knowledge-<strong>to</strong>actionculture?Who needs <strong>to</strong>know what, when,and why?IT INFRASTRUCTURE AND BUSINESS INTELLIGENCEAPPLICATIONS AS ENABLERSHow do we best leverage our information technologyinfrastructure and our business intelligence applications<strong>to</strong> support evidence-based decision making?How do we avoidthe knowing–doinggap?FEEDBACK LOOP


12 <strong>From</strong> <strong>Data</strong> <strong>to</strong> <strong>Decisions</strong>: GUIDANCEBut it is crucial the information presented is relevantand meaningful <strong>to</strong> that audience. Earlier, we outlinedthe importance of the right reporting frequency; agreat indica<strong>to</strong>r is of little use if that information gets<strong>to</strong> its audience <strong>to</strong>o late for timely decision making.The designer of the indica<strong>to</strong>r must also considerreporting channels — that is, what outlets or reportswill be used <strong>to</strong> communicate the data. An indica<strong>to</strong>rcan, for example, be included in the monthly performancereport <strong>to</strong> the executive management committee,or included in the quarterly performance report<strong>to</strong> the board. It might be required for the weekly performancereports <strong>to</strong> heads of service, reported on theorganizational Intranet, or made available <strong>to</strong> externalstakeholders through external reports or the website.The designers of data reports must also considerreporting formats, thus deciding how best <strong>to</strong> presentthe data. As examples, data can be shown as a number,a narrative, a table, a graph, or a chart.Charting dataLet’s look in more detail at presentation formats. Wewill stress that, in engaging the minds of the targetaudience, it is crucial the visual presentation <strong>to</strong>olsare clear, informative and compelling.In displaying information, we would recommendalways starting with the KAQ the data/informationsets out <strong>to</strong> answer. This provides context <strong>to</strong> whatwill follow. It should also ensure the report is focusedsquarely on meeting a critical information need of thetarget audience, thus avoiding any inclination <strong>to</strong> focuson “interesting” rather than “valuable” information.The KAQ should be followed by meaningful graphsand charts. Graphs are the most widely used visualdisplay <strong>to</strong>ols in organizations. Many different typesof graphs can be deployed <strong>to</strong> convey information.These include, for example, pic<strong>to</strong>graphs, tally charts,bar graphs, his<strong>to</strong>grams, scatter plots, line graphs,and pie charts.Each chart has a different purpose, and should thereforebe used appropriately.Graphs provide many benefits for conveying information.They are quick and direct, highlight the mostimportant facts, facilitate an easy understanding ofthe data, and can be easily remembered. Here aresome more generic tips for producing graphs:• Keep them simple and focus on the message theuser needs <strong>to</strong> receive;• Try <strong>to</strong> avoid three-dimensional graphs — they areharder <strong>to</strong> read;• Rarely use emphasis colours (e.g., bright red,yellow, orange, or green), and only where youwant <strong>to</strong> highlight specific issues;• Don’t use <strong>to</strong>o many different varieties of graphs,because an analysis across different graphs isdifficult; and• Try <strong>to</strong> avoid any unnecessary decorations, backgroundcolours, etc. Any additional and unnecessaryelements just distract us and make it harder<strong>to</strong> extract the insights.Placing a graph directly after a KAQ is a great of wayof quickly showing progress in answering that question.TEXT BOX 4When <strong>to</strong> use which graphBar graphs, which can display multiple instances, providefor easy comparison between adjacent values. They areparticularly good for nominal or ordinal scales.Line graphs can best display time series data, e.g., theshare price or quality fluctuations over a given period.However, a line graph is not suitable for data in nominalor ordinal scales. What it does do well is show trends,fluctuations, cycles, rates of change, and comparing twodata sets over time.Pie charts highlight various data as a percentage ofthe <strong>to</strong>tal data, each segment representing a particularcategory. They are generally not suitable for more thansix components, or when the values of each componentare similar, because it makes it <strong>to</strong>o difficult <strong>to</strong> distinguishbetween the values.Scatter plots are useful for depicting the correlationbetween two sets of data and showing the strengthand direction of that relationship.


<strong>From</strong> <strong>Data</strong> <strong>to</strong> <strong>Decisions</strong>: GUIDANCE13Headlines and narrativesFlowing down from the graphs, a good report (ordashboard) should then use narratives and “headlines.”A headline summarizes the main finding fromthe data, whereas the narrative provides context andmeaning. Using graphs and narrative <strong>to</strong>gether enablethe telling of the s<strong>to</strong>ry, which neither can fully do inisolation. For instance, a graph containing past performanceis extremely useful for analyzing trends overtime, but a narrative can put the graphical informationin<strong>to</strong> context — explaining why the trend is as it is.TEXT BOX 5Information dashboard design mistakesIn his book, Information Dashboard Design, StevenFew outlines the following 13 common dashboarddesign mistakes:1. Exceeding the boundaries of a single screen;2. Supplying inadequate context for the data;3. Displaying excessive detail or precision;4. Choosing a deficient measure;5. Choosing inappropriate display media;6. Introducing meaningless variety;7. Using poorly designed display media;8. Encoding quantitative data inaccurately;9. Arranging the data poorly;10. Highlighting important data ineffectively or not at all;11. Cluttering the display with useless decorations;12. Misusing or overusing colour; and13. Designing an unattractive visual display.


14 <strong>From</strong> <strong>Data</strong> <strong>to</strong> <strong>Decisions</strong>: GUIDANCESTEP 5Making evidence-based decisionsThis final step looks at how <strong>to</strong> turn the information in<strong>to</strong> (a) knowledge, and(b) better decisions <strong>to</strong> act on.1 2 3 4 5DEFININGOBJECTIVES ANDINFORMATIONNEEDSCOLLECTINGDATAANALYZINGDATAPRESENTINGINFORMATIONMAKING<strong>EVIDENCE</strong>-<strong>BASED</strong>DECISIONSWhat are ourstrategic aims?Based on thoseaims, what do weneed <strong>to</strong> know?Can we clearlyarticulate ourinformation needs?Who needs <strong>to</strong>know what, when,and why?Do we have orcan we collectmeaningful andrelevant data<strong>to</strong> meet ourinformation needs?How can we turnthe data in<strong>to</strong>relevant insights?How can we putthe data in<strong>to</strong>context and extractinformation?How can webest present andcommunicatethe insights andinformation <strong>to</strong>inform decisionmakers?IT INFRASTRUCTURE AND BUSINESS INTELLIGENCEAPPLICATIONS AS ENABLERSHow do we best leverage our information technologyinfrastructure and our business intelligence applications<strong>to</strong> support evidence-based decision making?How do we ensurethat the availableevidence is used<strong>to</strong> make the bestdecisions?How do we createa knowledge-<strong>to</strong>actionculture?How do we avoidthe knowing–doinggap?FEEDBACK LOOP


<strong>From</strong> <strong>Data</strong> <strong>to</strong> <strong>Decisions</strong>: GUIDANCE15Yet, just as we caution care in how organizationsanalyze data sources (for example, that they shouldnot be overly concerned with quantitative data only),we also suggest a certain circumspection in the useof information for decision making. We often find apredisposition <strong>to</strong> make important decisions based ona very narrow information set. Consequently, erroneousdecisions can be made that have damaging, andsometimes catastrophic, consequences. Often, managersare in such a rush <strong>to</strong> gain performance advantagesfrom “proven” approaches they fail <strong>to</strong> ensure considerationof other information when making decisions.Knowledge must be drawn from the best availableinformation, which will likely come from manysources. But amassing knowledge, however insightfulor compelling in and of itself, is of little value unlessit is turned in<strong>to</strong> action. Put in stark terms, if knowledgeis not turned in<strong>to</strong> action, then the entire effortexpended in sequencing through the previous stepsin the EBM framework would have been a pointlessexercise and a waste of resources. <strong>Decisions</strong> have <strong>to</strong>be made and acted upon.The knowing–doing gapThe book The Knowing–Doing Gap — How SmartCompanies Turn Knowledge in<strong>to</strong> Action, explains whymany organizations that possess plentiful knowledgefail <strong>to</strong> turn that knowledge in<strong>to</strong> action. The authorsargue the knowing–doing gap (where knowledge isnot implemented) is the most menacing phenomenonmost organizations face <strong>to</strong>day. This phenomenon, theyrightly claim, costs organizations billions of dollarsand leads <strong>to</strong> a wide array of failures in strategic implementationand other failures.The most destructive aspect of the knowing–doinggap, the authors argue, is what they call the “smart talktrap,” where talk becomes a substitution for action,and where myriad members of the organizations makedecisions that change nothing. Other reasons for thegap are (a) entrenched and outdated culture, (b) fearof change, (c) internal competition, and (d) measurementsthat lead nowhere.Consequently, closing this knowledge–doing gapoften requires a wholesale reworking of the processfor turning knowledge in<strong>to</strong> action — a reworking thathas cultural as well as process, structural, and technologicalcomponents.For creating a culture that is conducive <strong>to</strong> transformingknowledge in<strong>to</strong> action, we recommend organizationsfollow the following seven steps:1. Have passion for learning and improvement. Themost important ingredient, which is why it is thefirst on the list, is <strong>to</strong> create an organization-widepassion for learning and improvement.2. Ensure leadership buy-in. To make EBM a reality,senior level buy-in and support is important. TomDavenport and Jeanne Harris argue in their bookCompeting on Analytics: “If the CEO or a significantfaction of the senior executive team doesn’tunderstand or appreciate at least the outputs ofquantitative analysis or the process of fact-baseddecision making, analysts are going <strong>to</strong> be relegated<strong>to</strong> the back office, and competition will be based onguesswork and gut feel, not analytics.”3. Develop widespread analytical capabilities throughoutthe organization. Without the competenciesand skills <strong>to</strong> turn data in<strong>to</strong> insights EBM won’twork. Most organizations have a big training needin business analytics and EBM.4. Use judgment. In making analytics work, employees(at all levels) must balance facts and judgment.5. Share information. For EBM <strong>to</strong> be effective, themessage has <strong>to</strong> get out, loud and clear, that informationbelongs <strong>to</strong> the organization, and that allemployees should be focused not on its ownership


16 <strong>From</strong> <strong>Data</strong> <strong>to</strong> <strong>Decisions</strong>: GUIDANCEbut on working <strong>to</strong>gether <strong>to</strong> create the richness ofdifferent perspectives that can turn this informationin<strong>to</strong> golden nuggets of actionable knowledge.6. Reward and recognize fact-based decision-making.In planning their strategy for implementing the EBMframework, organizations should look <strong>to</strong> weavingin some form of supporting reward strategy, as it isimportant <strong>to</strong> recognize and reward EBM attempts.This will show that organizations take the approachseriously, and value those trying <strong>to</strong> make it a practicalreality. This can start with a simple thank you andsharing of success s<strong>to</strong>ries.7. Build appropriate IT infrastructure. You can havea wealth of analytical intentions and skills, butyou also need the <strong>to</strong>ols <strong>to</strong> put them in<strong>to</strong> practice.Organizations need the right IT Infrastructure.Essentially, this comprises (a) databases, datawarehouses, data marts, etc. <strong>to</strong> s<strong>to</strong>re the data;(b) networks and connections <strong>to</strong> share the informationand <strong>to</strong> make it accessible; and (c) the software<strong>to</strong> analyze and share the data.What makes organizations succeed in <strong>to</strong>day’s competitiveand unpredictable world is the ability <strong>to</strong> learnfaster than the competition, and the ability <strong>to</strong> identifyand act on facts faster than the competition. ThisGuidance outlines how EBM can enable organizations<strong>to</strong> do exactly that. Any organization can boostits competitive position by aligning the data collection<strong>to</strong> the strategic value drivers, and collecting the bestavailable evidence, by using this evidence <strong>to</strong> extractvaluable insights and by communicating the informationin a way that allows acting on those insights.The tips, <strong>to</strong>ols, and templates presented as part of thefive-part EBM model should enable organizations <strong>to</strong>become more evidence-based in their decision making,and avoid the traps of making decisions basedon anecdotal data or dangerous half-truths. ☐This publication is one in a series on <strong>From</strong> <strong>Data</strong> <strong>to</strong> <strong>Decisions</strong>. An Overview and Case Studies arealso available on our website. For additional information, please contact Carol Raven, Principal,Strategic Management Accounting & Finance at 416-204-3489 or email craven@cpacanada.ca


<strong>From</strong> <strong>Data</strong> <strong>to</strong> <strong>Decisions</strong>: GUIDANCE17Additional sources of informationAlexander, Jack. Performance Dashboards andAnalysis for Value Creation. Hoboken, NJ: JohnWiley & Sons, 2007.Ayres, Ian. Super Crunchers. New York: BantamDell, 2007.Bensoussan, Babette E., and Fleisher, Craig. AnalysisWithout Paralysis: 10 Tools <strong>to</strong> Make Better Strategic<strong>Decisions</strong>. Saddle River, NJ: FT Prentice Hall, 2008.Few, Stephen. Information Dashboard Design: TheEffective Visual Communication of <strong>Data</strong>. Sebas<strong>to</strong>pol,CA: O’Reilly Media, 2006.Few, Stephen. Show Me the Numbers: Designing Tablesand Graphs <strong>to</strong> Enlighten. Oakland, CA: AnalyticsPress, 2004.Marr, Bernard. Key Performance Indica<strong>to</strong>rs: The 75+Measures Every Manager Needs <strong>to</strong> Know. Harlow, CM:Financial Times Prentice Hall, 2012.Marr, Bernard. The Intelligent Company: Five steps <strong>to</strong>Success with Evidence-Based Management. Chichester,UK: Wiley 2010.Marr, Bernard. Strategic Performance Management:Leveraging and Measuring your Intangible ValueDrivers. Oxford, UK: Elsevier Ltd., 2006.Redman, Thomas C. <strong>Data</strong> Driven: Profiting <strong>From</strong>Your Most Important Business Assets. Bos<strong>to</strong>n: HarvardBusiness School Press, 2008.About the authorBernard Marr is the founder and CEO of theAdvanced Performance Institute. He is acknowledgedas a leading global authority and bestsellingauthor on organizational performance andorganization success. In this capacity he regularlyadvises leading companies, organizations and governmentsacross the globe, and he is an acclaimedand award-winning keynote speaker, researcher,consultant and teacher. His latest books include:The Intelligent Company: Five Steps To SuccessWith Evidence-Based Management and KeyPerformance Indica<strong>to</strong>rs: The 75+ Measures EveryManager Needs To Know. For more informationvisit www.ap-institute.com or contact Bernard atbernard.marr@ap-institute.com.Marr, Bernard. Managing and Delivering Performance.Oxford, UK: Elsevier Ltd., 2008.


277 WELLING<strong>TO</strong>N STREET WEST<strong>TO</strong>RON<strong>TO</strong>, ON CANADA M5V 3H2T. 416 977.3222 F. 416 977.8585WWW.CPACANADA.CA

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!