# Emulating expensive decision choices with application to ... - MUCM

Emulating expensive decision choices with application to ... - MUCM

Emulating expensive decision choices with

application to computer models of complex physical

systems

Jonathan A Cumming and Michael Goldstein

Department of Mathematical Sciences, Durham University

July 2012

1 / 23

Outline

1 Motivation

Overview

2 General Problem Description

General Problem Description

Influence Diagram

Decision Tree

Methodology

3 Application: Well Test Analysis

Overview

Results

4 Concluding Remarks

Remarks

2 / 23

Motivation

Motivation

3 / 23

Well test analysis

Motivation Overview

Well test analysis is a technique used to reveal information about the

geometry and geological features of an underground hydrocarbon

reservoir

In practice, a well test involves producing fluid from a reservoir at a known

and controlled rate and observing the change in pressure during and

after this period of production

4 / 23

Well test analysis

3800 4400 5000

Pressure

0 400 800

Rate

Motivation Overview

0 10 20 30 40 50 60 70

Time

4 / 23

Well test analysis

Motivation Overview

4 / 23

Well test analysis

Motivation Overview

Well test analysis is a technique used to reveal information about the

geometry and geological features of an underground hydrocarbon

reservoir

In practice, a well test involves producing fluid from a reservoir at a known

and controlled rate and observing the change in pressure during and

after this period of production

Pressure and rates are linked by a convolution relationship with a third

function, usually called the deconvolved derivative

The behaviour of this derivative function follows common patterns which

are attributed to particular reservoir characteristics

4 / 23

Well test analysis

Deconvoled Derivative

−1.5 −1.0 −0.5

Motivation Overview

−2 −1 0 1 2

log 10(T)

Infinite

Fault

Channel

Closed

4 / 23

Well test analysis

Motivation Overview

Well test analysis is a technique used to reveal information about the

geometry and geological features of an underground hydrocarbon

reservoir

In practice, a well test involves producing fluid from a reservoir at a known

and controlled rate and observing the change in pressure during and

after this period of production

Pressure and rates are linked by a convolution relationship with a third

function, usually called the deconvolved derivative

The behaviour of this derivative function follows common patterns which

are attributed to particular reservoir characteristics

The object of the well test is to make an inference about the class of the

reservoir

4 / 23

The Problem

Motivation Overview

We have a reservoir of interest, which can belong to one of four different

classes

We have to make a decision about how to perform the well test – what

rate to produce, how long for, how long to observe pressure

We then enact that decision (under error) and observe the pressures and

rates, and estimate the deconvolved derivative function

We then make an inference about the class of the reservoir using the

observations

On the basis of this inference, we then take a further action about

whether to develop or abandon this reservoir

Depending on the decisions we made and the nature of the reservoir, we

will receive some reward (or penalty)

We have a computer model which simulates the results of the well test

Are there any generally good well test regimes, and if so what are they?

5 / 23

General Problem Description

General Problem Description

6 / 23

General Problem

General Problem Description General Problem Description

We are interested in some complex physical system

Denote the portion of the system that is observable (and interesting) as y,

and let the state of the complex system be θ

We want to make a decision, δ, that may or may not affect the system

We enact that decision, and take observations of the system z

Using what we learn from z, we take a further decision (or action) α

Depending on δ, α, and the system θ, we receive a reward expressed as

a utility U[δ, α, θ]

We want to find good choices of δ

7 / 23

General Problem Description General Problem Description

Well Testing as a Decision Problem

The system is the reservoir under study

y is the behaviour of the true pressures and rates during the well test

θ indicates to which of four potential classes the reservoir belongs; in this

case there are only four possibilities: Infinite System, Single Fault,

Channel, Closed System

The parameters of the well test form the decision δ, which governs the

amount and quality of data we gather to learn about θ via three variables:

Duration, Rate, and Buildup.

The decision δ does not uniquely determine the actual well test conditions

z is then the observations of y made during the well test

Given the observed well test, we estimate the deconvolved derivative and

make an inference about what class θ of reservoir this is

We have three different actions, α: Abandon, Small Production, Large

Production

The reward from performing a given action depends on the class of the

reservoir as given by a payoff matrix

8 / 23

General Problem Description Influence Diagram

Our First Decision and the System

In the real world, we have a complex system.

9 / 23

General Problem Description Influence Diagram

Our First Decision and the System

Real World

In the real world, we have a complex system.

θ

Denote the state of the complex system as θ, and the portion of the

system that is observable (and interesting) as y

y

9 / 23

General Problem Description Influence Diagram

Our First Decision and the System

Real World

In the real world, we have a complex system.

θ

δ

Denote the state of the complex system as θ, and the portion of the

system that is observable (and interesting) as y

We make the decision δ and do it

y

9 / 23

General Problem Description Influence Diagram

Our First Decision and the System

Real World

In the real world, we have a complex system.

θ

δ

Denote the state of the complex system as θ, and the portion of the

system that is observable (and interesting) as y

We make the decision δ and do it

The decision δ could be active (an intervention) or passive (an

observation), and so δ may or may not affect y

y

9 / 23

General Problem Description Influence Diagram

Our First Decision and the System

Real World

In the real world, we have a complex system.

θ

δ

Denote the state of the complex system as θ, and the portion of the

system that is observable (and interesting) as y

We make the decision δ and do it

The decision δ could be active (an intervention) or passive (an

observation), and so δ may or may not affect y

We expect that our decision will not be carried out exactly as desired, as

so there will be decision execution uncertainty in the real world

y

9 / 23

Our Second Decision

General Problem Description Influence Diagram

θ

Real World Observations

δ y z

Having made our decision δ we make observations of the system, z

10 / 23

Our Second Decision

θ

General Problem Description Influence Diagram

Real World Observations Real Decision

δ y z

Having made our decision δ we make observations of the system, z

Depending what we learn from the observations, we take the action α

α

10 / 23

Our Second Decision

θ

General Problem Description Influence Diagram

Real World Observations Real Decision

δ y z

α U(δ, α, θ)

Having made our decision δ we make observations of the system, z

Depending what we learn from the observations, we take the action α

Action α will give us utility U(δ, α, θ) – a combination of the costs of

performing the decisions we chose, and the rewards for doing so

10 / 23

Our Second Decision

θ

General Problem Description Influence Diagram

Real World Observations Real Decision

δ y z

α U(δ, α, θ) U(δ) U ∗

Having made our decision δ we make observations of the system, z

Depending what we learn from the observations, we take the action α

Action α will give us utility U(δ, α, θ) – a combination of the costs of

performing the decisions we chose, and the rewards for doing so

By taking expectations over α and θ, we obtain U(δ) which we can then

optimise to find our good decisions δ

10 / 23

Enter the simulator

θ

General Problem Description Influence Diagram

Real World Observations Real Decision

δ y z

f (x, d)

Computer Model

α U(δ, α, θ) U(δ) U ∗

The computer model f will mimic the behaviour of y

11 / 23

Enter the simulator

θ

General Problem Description Influence Diagram

Real World Observations Real Decision

δ y z

f (x, d) z(x, d)

Computer Model Synthetic Observations

α U(δ, α, θ) U(δ) U ∗

The computer model f will mimic the behaviour of y, and by adding some

noise z

11 / 23

Enter the simulator

θ

General Problem Description Influence Diagram

Real World Observations Real Decision

δ y z

x

f (x, d) z(x, d)

Computer Model Synthetic Observations

α U(δ, α, θ) U(δ) U ∗

The computer model f will mimic the behaviour of y, and by adding some

noise z

The simulator takes two sets of parameters

1 x, the usual input parameters – corresponding in some way to θ

11 / 23

Enter the simulator

θ

General Problem Description Influence Diagram

Real World Observations Real Decision

δ y z

d

x

f (x, d) z(x, d)

Computer Model Synthetic Observations

α U(δ, α, θ) U(δ) U ∗

The computer model f will mimic the behaviour of y, and by adding some

noise z

The simulator takes two sets of parameters

1 x, the usual input parameters – corresponding in some way to θ

2 d, the decision as enacted in the model – corresponding in some way to δ

11 / 23

Emulation

θ

General Problem Description Influence Diagram

Real World Observations Real Decision

δ y z

d

x

f (x, d) z(x, d)

Computer Model Synthetic Observations

α U(δ, α, θ) U(δ) U ∗

We can explore the behaviour of f (or z) over d and x by making many

evaluations of the model

12 / 23

Emulation

θ

General Problem Description Influence Diagram

Real World Observations Real Decision

δ y z

d

x

f (x, d) z(x, d)

Computer Model Synthetic Observations

α U(δ, α, θ) U(δ) U ∗

We can explore the behaviour of f (or z) over d and x by making many

evaluations of the model – but learning about f (or z) isn’t really the goal

12 / 23

Emulation

θ

General Problem Description Influence Diagram

Real World Observations Real Decision

δ y z

d

x

f (x, d) z(x, d)

Computer Model Synthetic Observations Decision Model

α U(δ, α, θ) U(δ) U ∗

a

U(δ, α, θ)

We can explore the behaviour of f (or z) over d and x by making many

evaluations of the model – but learning about f (or z) isn’t really the goal

Instead, we want to propagate our model runs through the decision

12 / 23

Emulation

θ

General Problem Description Influence Diagram

Real World Observations Real Decision

δ y z

d

x

f (x, d) z(x, d)

Computer Model Synthetic Observations Decision Model

α U(δ, α, θ) U(δ) U ∗

a

U(δ, α, θ)

So we need many evaluations over d and x (corresponding to δ, θ) and a

mechanism for propagating them through the decision process

12 / 23

Emulation

θ

General Problem Description Influence Diagram

Real World Observations Real Decision

δ y z

d

x

f (x, d) z(x, d)

Computer Model Synthetic Observations Decision Model

α U(δ, α, θ) U(δ) U ∗

a

U(δ, α, θ)

So we need many evaluations over d and x (corresponding to δ, θ) and a

mechanism for propagating them through the decision process

At the end, we get values of the complex utility function U(δ, α, θ) over δ,

α, and θ

12 / 23

The Big Picture

θ

General Problem Description Influence Diagram

Real World Observations Real Decision

δ y z

d

x

f (x, d) z(x, d)

Computer Model Synthetic Observations Decision Model

α U(δ, α, θ) U(δ) U ∗

a

U(δ, α, θ)

13 / 23

General Problem Description Decision Tree

The problem as a decision tree

Decision

δ

Observation

z

P [z|δ]

Action

α

State of

the World

θ

P [θ|δ, z, α]

The role of the computer model is to probabilise the tree

U[δ, α, θ]

14 / 23

General Problem Description Methodology

Key features and methodology

We have a (2-stage) decision problem in the world of computer models

15 / 23

General Problem Description Methodology

Key features and methodology

We have a (2-stage) decision problem in the world of computer models

There are all the usual uncertainties, as well as uncertainty on the

decision enacted given our decision choice

15 / 23

General Problem Description Methodology

Key features and methodology

We have a (2-stage) decision problem in the world of computer models

There are all the usual uncertainties, as well as uncertainty on the

decision enacted given our decision choice

The nature of the decision problem shifts the focus of the analysis from f

to U

15 / 23

General Problem Description Methodology

Key features and methodology

We have a (2-stage) decision problem in the world of computer models

There are all the usual uncertainties, as well as uncertainty on the

decision enacted given our decision choice

The nature of the decision problem shifts the focus of the analysis from f

to U

Our general approach to this type of problem will be as follows:

1 Construct a design over (x, δ, θ) space

15 / 23

General Problem Description Methodology

Key features and methodology

We have a (2-stage) decision problem in the world of computer models

There are all the usual uncertainties, as well as uncertainty on the

decision enacted given our decision choice

The nature of the decision problem shifts the focus of the analysis from f

to U

Our general approach to this type of problem will be as follows:

1 Construct a design over (x, δ, θ) space

2 At every point in the design, we add noise to δ to give us a d and we then

evaluate the simulator at f (x, d)

15 / 23

General Problem Description Methodology

Key features and methodology

We have a (2-stage) decision problem in the world of computer models

There are all the usual uncertainties, as well as uncertainty on the

decision enacted given our decision choice

The nature of the decision problem shifts the focus of the analysis from f

to U

Our general approach to this type of problem will be as follows:

1 Construct a design over (x, δ, θ) space

2 At every point in the design, we add noise to δ to give us a d and we then

evaluate the simulator at f (x, d)

3 Introduce observational error and discrepancy to generate z(x, d) for every

run of the model and therefore evaluate the joint for (z, θ) for selected δ

15 / 23

General Problem Description Methodology

Key features and methodology

We have a (2-stage) decision problem in the world of computer models

There are all the usual uncertainties, as well as uncertainty on the

decision enacted given our decision choice

The nature of the decision problem shifts the focus of the analysis from f

to U

Our general approach to this type of problem will be as follows:

1 Construct a design over (x, δ, θ) space

2 At every point in the design, we add noise to δ to give us a d and we then

evaluate the simulator at f (x, d)

3 Introduce observational error and discrepancy to generate z(x, d) for every

run of the model and therefore evaluate the joint for (z, θ) for selected δ

4 Assign probabilities over the possible states θ, and hence determine what

action a to take for each z

15 / 23

General Problem Description Methodology

Key features and methodology

We have a (2-stage) decision problem in the world of computer models

There are all the usual uncertainties, as well as uncertainty on the

decision enacted given our decision choice

The nature of the decision problem shifts the focus of the analysis from f

to U

Our general approach to this type of problem will be as follows:

1 Construct a design over (x, δ, θ) space

2 At every point in the design, we add noise to δ to give us a d and we then

evaluate the simulator at f (x, d)

3 Introduce observational error and discrepancy to generate z(x, d) for every

run of the model and therefore evaluate the joint for (z, θ) for selected δ

4 Assign probabilities over the possible states θ, and hence determine what

action a to take for each z

5 For each run, evaluate the utility value U(δ, α)

15 / 23

General Problem Description Methodology

Key features and methodology

We have a (2-stage) decision problem in the world of computer models

There are all the usual uncertainties, as well as uncertainty on the

decision enacted given our decision choice

The nature of the decision problem shifts the focus of the analysis from f

to U

Our general approach to this type of problem will be as follows:

1 Construct a design over (x, δ, θ) space

2 At every point in the design, we add noise to δ to give us a d and we then

evaluate the simulator at f (x, d)

3 Introduce observational error and discrepancy to generate z(x, d) for every

run of the model and therefore evaluate the joint for (z, θ) for selected δ

4 Assign probabilities over the possible states θ, and hence determine what

action a to take for each z

5 For each run, evaluate the utility value U(δ, α)

6 Use this information to collapse the decision tree and obtain U(δ)

15 / 23

General Problem Description Methodology

Key features and methodology

We have a (2-stage) decision problem in the world of computer models

There are all the usual uncertainties, as well as uncertainty on the

decision enacted given our decision choice

The nature of the decision problem shifts the focus of the analysis from f

to U

Our general approach to this type of problem will be as follows:

1 Construct a design over (x, δ, θ) space

2 At every point in the design, we add noise to δ to give us a d and we then

evaluate the simulator at f (x, d)

3 Introduce observational error and discrepancy to generate z(x, d) for every

run of the model and therefore evaluate the joint for (z, θ) for selected δ

4 Assign probabilities over the possible states θ, and hence determine what

action a to take for each z

5 For each run, evaluate the utility value U(δ, α)

6 Use this information to collapse the decision tree and obtain U(δ)

7 Emulate and optimise U(δ) over the decision space as the ‘output’ of the

model

15 / 23

Application: Well Test Analysis

Application: Well Test Analysis

16 / 23

Application: Well Test Analysis Results

Design and Evaluations

We constructed a design of 40,000 runs of the computer model

composed of:

Each of 4 possible reservoir classes θ

For each θ, we choose 1000 values of δ in a Latin hypercube over the

3-dimensional decision space

For each (θ, δ), we choose 10 different values of the simulator input x in a

further Latin-hypercube over the 9-dimensional x space

17 / 23

Application: Well Test Analysis Results

Design and Evaluations

We constructed a design of 40,000 runs of the computer model

composed of:

Each of 4 possible reservoir classes θ

For each θ, we choose 1000 values of δ in a Latin hypercube over the

3-dimensional decision space

For each (θ, δ), we choose 10 different values of the simulator input x in a

further Latin-hypercube over the 9-dimensional x space

This gave 40,000 runs of the computer model requiring 11.52 hours to

complete.

17 / 23

Application: Well Test Analysis Results

Design and Evaluations

17 / 23

Application: Well Test Analysis Results

Design and Evaluations

We constructed a design of 40,000 runs of the computer model

composed of:

Each of 4 possible reservoir classes θ

For each θ, we choose 1000 values of δ in a Latin hypercube over the

3-dimensional decision space

For each (θ, δ), we choose 10 different values of the simulator input x in a

further Latin-hypercube over the 9-dimensional x space

This gave 40,000 runs of the computer model requiring 11.52 hours to

complete.

Given each run of the computer model, we made an inference about the

value of θ for that particular run

17 / 23

Application: Well Test Analysis Results

Design and Evaluations

We constructed a design of 40,000 runs of the computer model

composed of:

Each of 4 possible reservoir classes θ

For each θ, we choose 1000 values of δ in a Latin hypercube over the

3-dimensional decision space

For each (θ, δ), we choose 10 different values of the simulator input x in a

further Latin-hypercube over the 9-dimensional x space

This gave 40,000 runs of the computer model requiring 11.52 hours to

complete.

Given each run of the computer model, we made an inference about the

value of θ for that particular run

This gave a vector of classification probabilities pik that the ith run derived

from reservoir class θk, k = 1, . . . , 4

17 / 23

Application: Well Test Analysis Results

Emulating expected utilities

By combining these probabilities with a payoff matrix for the possible

actions and states, we can identify the action which maximises our

expected utility

Large Small Abandon

Infinite 100 75 0

Fault 50 75 0

Channel 50 75 0

Closed -10 -5 100

18 / 23

Application: Well Test Analysis Results

Emulating expected utilities

By combining these probabilities with a payoff matrix for the possible

actions and states, we can identify the action which maximises our

expected utility

Large Small Abandon

Infinite 100 75 0

Fault 50 75 0

Channel 50 75 0

Closed -10 -5 100

Given the possible states θ and the probabilities that a particular run

belongs to that state, we can collapse the last node of the decision tree

18 / 23

Application: Well Test Analysis Results

Emulating expected utilities

By combining these probabilities with a payoff matrix for the possible

actions and states, we can identify the action which maximises our

expected utility

Large Small Abandon

Infinite 100 75 0

Fault 50 75 0

Channel 50 75 0

Closed -10 -5 100

Given the possible states θ and the probabilities that a particular run

belongs to that state, we can collapse the last node of the decision tree

For every decision value δ in the design, we can then take expectations

over the 40 associated runs of the model over different choices of

simulator parameters

18 / 23

Application: Well Test Analysis Results

Emulating expected utilities

By combining these probabilities with a payoff matrix for the possible

actions and states, we can identify the action which maximises our

expected utility

Large Small Abandon

Infinite 100 75 0

Fault 50 75 0

Channel 50 75 0

Closed -10 -5 100

Given the possible states θ and the probabilities that a particular run

belongs to that state, we can collapse the last node of the decision tree

For every decision value δ in the design, we can then take expectations

over the 40 associated runs of the model over different choices of

simulator parameters

We now have a value of U[δ] for every choice of δ in our original design.

18 / 23

U(δ)

Application: Well Test Analysis Results

Since U[δ] is a complex function about which we only have partial

knowledge over the space of δ, we apply standard emulation techniques

to approximate its behaviour.

19 / 23

U(δ)

Application: Well Test Analysis Results

Emulation revealed that the Rate decision variable was not active, and so

did not contribute to variation in the U[δ] surface

19 / 23

U(δ)

Application: Well Test Analysis Results

This left Duration and Buildup as the driving variables for the U[δ] surface

19 / 23

Application: Well Test Analysis Results

Factoring in decision costs

So far we our choices of decision have been costless

We introduce a simple linear cost function over the decisions δ with

coefficients

Intercept 50

Duration 0.035

Buildup 0.03

Rate 0.003

20 / 23

Application: Well Test Analysis Results

Factoring in decision costs

This gives the following expected utility surface factoring in decision cost

20 / 23

Application: Well Test Analysis Results

Factoring in decision costs

The unique maximum of the emulated utility surface occurs at

Duration= 65.7hrs and Buildup= 139.6hrs.

20 / 23

Application: Well Test Analysis Results

Factoring in decision costs

However there is a region of (almost equally) good decisions when

surrounding this point.

20 / 23

Concluding Remarks

Concluding Remarks

21 / 23

Remarks

Concluding Remarks Remarks

We can apply emulation technology to solve complex decision problems

We can use emulators to model the behaviour of the (expected) utilities in

the decision tree

If the simulator itself was more expensive, we would emulate f (x, d)

Note we need enough realisations for each branch of the tree for our

approximations to have small error

Optimisation was straightforward here; if δ were higher-dimensional we

would require a more sophisticated approach to maximise the utility

Given a problem with many sequential decisions, we would emulate

nodes recursively from the end of the tree

22 / 23

References

Concluding Remarks Remarks

J Sacks, W J Welch, T J Mitchell, and H P Wynn. Design and analysis of

computer experiments. Statistical Science, 4(4):409–435, 1989.

T von Schroeter, F Hollaender, and A C Gringarten. Deconvolution of well

test data as a nonlinear total least squares problem. Society of Petroleum

Engineers Journal, 9(4):375-390, 2004. SPE-77688-PA.

J A Cumming, D A Wooff, T Whittle, A C Gringarten. Assessing the

Non-Uniqueness of the Well Test Interpretation Model Using

Deconvolution. 2012. In submission

D Williamson and M Goldstein. Bayesian policy support for adaptive

strategies using computer models for complex physical systems. Journal

of the Operational Research Society, 2011.

23 / 23

More magazines by this user
Similar magazines