From Hindsight to Foresight: Applying Delphi, Cross-Impact Analysis ...

Techniques for Modeling the 

Future 

Delphi, Cross-Impact, Scenarios 

Paul Gray 

Anat Hovav 

Tutorial Presented at AMCIS 2007, Keystone CO 

1

Outline 

• ABOUT STUDYING THE FUTURE 

• Assumptions 

• Dealing with Uncertainty 

• Using Experts 

• Delphi 

• Cross-Impact 

• Scenarios 

• Discussion 

A handout shows a typical scenario 

2

Futures Research Assumptions 

• The future is not prewritten nor wholly determined 

by what we know now 

• Present conditions and trends will be evolutionary 

unless natural limits or external changes occur 

• External changes are inherently uncertain 

• Some depend on human choice (policy) 

• Future conditions can be altered by social 

resources 

3

Dealing with Uncertainty 

• By fiat (I say, you do) 

• Extrapolate historical data. Implies 

assuming all developments are 

foreseeable 

• Elicit and evaluate judgments about the 

future from groups of experts (wisdom 

of crowds) 

• Synthesis of methods 2 and 3 

4

Who Are Experts? When 

Should You Use Them? 

• Defn: People qualified to explore answers 

from a relevant disciplinary perspective. 

Includes practitioners and users 

• Use when: little or no source of facts 

• A basis for opinion exists 

• Info. is likely to affect decision 

• Expert opinion is an acceptable source 

5

Types of Issues for Experts 

• Tech forecasts 

• Social forecasts 

• Resolving uncertainty 

• Determining consumer needs 

• Desirability of change 

• Lack of documentation of current state 

• Interpreting history 

6

Role of Experts 

• Offer critique of ideas 

• Interpret unintended consequences 

• Evaluate quality of new concepts 

• Imagine what can be done 

• Estimate how well (poorly) idea will work 

• Invent new ideas beyond those of the 

group 

7

Outline 

• About Studying the Future 

• DELPHI 

• Procedures 

• Events and Trends 

• Research Results 


• Scenarios 



8

Delphi 

• Provides estimates of p (events) and 

evolution of trends over time 

• Use when there is little factual data and 

a basis for opinion exists, and decisions 

and policies are required 

• History of Delphi. Inventors 

9

Delphi Procedure 

• Multistage (= Rounds) 

• Present history and ask questions to expert 

panel 

• Provide anonymity to respondents 

• Panelists make estimates, give reasons 

• Analyze data from a round. Feed it back to 

the panel, including rationale for extreme 

values 

• Repeat rounds with re-estimates until 

consensus or dissensus is reached. 

10

Candidate Events and Trends 

• Event: when will a virus be created that 

cannot be contained or countered? 

(E,ML,L) 

• Trend: $ loss from phishing 

• Social acceptance: Fingerprint ID of PC 

users; cell phone locators 

REMINDER: Creating any questionnaire 

(including Delphi) requires care and pretesting 

11

Sample Answers to 

Questionnaire 

Question 2008 2012 2016 2020 Never 

LINUX market 

share >Microsoft 

An unfixable virus 

is created 

Annual $ loss 

from phishing 

0.02 0.05 0.10 0.20 

E M L 

50000 30000 20000 50000 -- 

Loss from Phishing ($M) 

40000 

30000 

20000 

10000 

0 

1 2 3 4 5 6 

Years 

12

Research Results on Delphi 

• Average error declines with size of group 

• Accuracy of group better on 2 nd but not later 

iteration 

• Smaller dispersion more accurate result 

• Higher self-confidence greater accuracy 

13

Delphi Advantages 

• Provides a framework in which to work 

• Anonymity minimizes follow the leader, 

specious persuasion, other psych barriers 

• Allows everyone to be heard w/o labels 

• Provides precise documented records 

14

Delphi Disadvantages 

• Participants lose stimulation of face-to-face 

contact 

• Panelists can (and do) drop out between 

rounds even in computer-based Delphi 

15

Conclusions on Delphi 

• Now well established 

• An input to cross-impact and scenarios 

• By using wisdom of crowds when the 

crowd knows something about subject 

commends its use where other 

approaches fail 

16

Outline 




• Why cross impact 

• Cross-impact table 

• Mechanics 

• Outcomes 

• Scenarios 



17

Cross Impact 

• Delphi gives individual, independent forecasts 

• Assumes “all other things being equal”, which 

they are not. 

• The order in which things happen can change 

and affect outcomes. 

• Example from security: advanced preventives, 

secure operating system, uncontainable virus 

• Role of cross-impact is to model these effects 

• History of cross-impact 

18

Cross-Impact Table 

If event: 

A 

B 

C 

XI on A XI on B XI on C XI on 

trend 

1 2 3 1.05 

0.5 1 1 1.2 

1 0.25 1 0.9 

19

Mechanics of Cross Impact 

• Define time interval and scene length 

2008 2012 2016 2020 2024 

• Inputs: probability, event x-impact, trends, 

trend x-impact 

• Run multi-period simulation. Outputs of one 

scene are inputs to the next 

20

Mechanics of Cross-Impact 

• In a scene, decide which event occurs 

• Update values in next scene depending 

on whether event occurred or not 

• Include both recurring and nonrecurring 

events and all trends 

• Repeat until you reach end of last scene 

• RESULT: A Scenario 

21

Expected Outcomes 

• Create a large number of scenarios 

• Determine the mean and variance and 

time distribution of the outcomes 

• Use extreme scenarios to help 

understand the range of outcomes that 

can be encountered 

• Use result to help set policies for 

expected outcomes and contingencies. 

22

Conclusions on Cross-Impact 

• Cross-impacts exist . They allow modeling 

disruptive changes and interdependencies 

• No methodology is standard; Deals with 

second order effects but not third order 

order effects 

• Although out of fashion, cross impact is a 

useful analytic tool that merits further 

development 

23

OUTLINE 




• SCENARIOS 

• What are scenarios 

• Assumptions about scenarios 

• Development principles 

• Characteristics 

• Scenario space and developing scenarios 

• Pitfalls to avoid 

• Conclusions 



24

Scenarios 

• Stories about alternative futures 

• A way of communicating with managers 

and non-experts 

• Describe some future or plausible state 

of the firm at a specific time 

• Used to stimulate future plans, 

strategies and business choices 

25

Assumptions about Scenarios 

• To avoid information overload, a set of 

assumptions and constraints need to be 

established: 

• Purpose 

• Domain 

• Geopolitics and economics 

• Time frame 

• Unit of Analysis 

26

Scenario Development 

Principles 

• Identify the critical (or possible) choices 

that the subject is likely to face 

• Identify the key drivers 

• Model the scenario space 

• Use the scenarios to determine strength 

and weaknesses of the organization and 

to create alternative contingency 

policies 

27

Characteristics of Scenarios 

• Possible 

• Technically and/or socially feasible 

• Plausible 

• The reader of the scenario would believe 

that the events could occur 

• Internally consistent 

• All parts of the scenario are consistent with 

one another 

28

Scenario Space (example) 

High technical 

security level 

A highly trusted 

environment 

A “Big Brother” 

environment * 

Low technical 

security level 

Status-quo 

Chaos 

Low identity theft 

level 

High identity theft 

level 

* See handout for a sample scenario based on the H-H dimensions 

29

Developing scenarios 

• Select system variables of interest 

• Forecast system parameters 

• Select scenario space 

• Perform a consistency check 

• Develop measurements 

• Write the scenarios 

• Analysis and policy development 

30

Pitfalls to Avoid 

• Scenarios are not the outcome 

• They do not define the future, only describe 

possible futures 

• Scenarios are not a forecast 

• Users should not assume that any one of the 

scenarios describes the future 

• Scenarios are not static 

• Assumptions change rapidly 

• Scenarios must be re-evaluated periodically 

31

Conclusions about Scenarios 

• Scenario building is both an art and a 

science 

• The process has to be replicable 

• The requirements for internal 

consistency lead to the creation of pure 

scenarios 

• There are no guidelines for building 

mixed scenarios 

32

Conclusions about Scenarios 

• Scenarios are most valuable when 

• The external environment is highly 

uncertain 

• The environment might take plausible 

alternative trajectories 

• A specific set of indicators can be 

developed to measure the driving forces 

and their likely impacts if they were to 

occur 

33

Outline 




• Scenarios 



34

Working Paper 

• A draft of the complete article is 

available at: 

http://biz.korea.ac.kr/~anat/Workinpro 

gress.htm 

35

Essential Delphi Features 

• Anonymity 

• Feedback between rounds 

• Re-estimate based on feedback 

• Use a facilitator 

• Input on history of events, trends 

36

Updating Probabilities 

• Probability of A in Scene i+1 is affected by occurrence or 

non-occurrence of B in scene i 

• Done in terms of odds. 

• Odds=p/(1-p); e.g., p=.6, odds =.6/.4=1.5 

• Find new odds by multiplying by x-impact 

• E.g., if XI=2, p=.6, odds= 1.5 New odds =2*1.5= 3, New 

p = .75 

• More complex to get new p if non-occurrence. In this case, 

if P(B) = 0.3, new P = .536 

37

From Hindsight to Foresight: Applying Delphi, Cross-Impact Analysis ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?