Building a Treeage Markov Model

EP550 Lecture/Demonstration 16 and Homework 10, Due Friday, March 25
The purpose of this exercise is to teach you how to create Markov models using the TreeAge
2011 software program and to use what you’ve created to illustrate additional features of costeffectiveness
analysis. The exercise uses the lupus example from class.
Create the model
1. Open TreeAge, clear any material from the center panel where decision trees are created and
modified, and maximize this panel. Create a decision node by clicking “File/New.../Blank Tree
Diagram/OK.” A square decision node should appear. If not, in the “Tree Nodes” box to the right, double
click “Decision.” Label the decision node “Lupus.” Right click on the decision node, and add two chance
nodes. Label the upper chance node “Usual Care.” Label the lower chance node “Intervention.”
2. Change the Usual Care chance node to a Markov node: right click the node and click on “Change
type/Markov.” (Note the difference between selecting the node and selecting the subtree attached to the
node.) Make the display simpler: click on “Edit/Tree Preferences/Display/Variables/Markov info,” select
“Hide Definitions,” and click “OK.”
3. Add four branches to the Markov node. (“Node/Add branch” or “right click/Add branch.” The Lupus
node must be selected each time you repeat this process.)
4. Click on the Markov node, select its subtree (“Subtree/Select Subtree”), copy the subtree (“Edit/Copy”),
select the first of the four branches of the Markov node, and paste the selected subtree to this branch
(“Edit/Paste”).
5. Label the four branches of the Markov node (above the line): “Remission, Active, Flair, Death.”
6. Select the first branch of the “Remission” node. Change this node to a terminal node (right-click the
node, click “Change Type” and select “Terminal”), when the box pops up select the “Remission” state for
this node to jump to, and click “OK.” Change the other three nodes to terminal nodes and select the other
three states for them to jump to.
7. Click on the “Remission” chance node that immediately follows the Markov node, select its subtree
(“Subtree/Select Subtree”), copy the subtree (“Edit/Copy”), and paste the subtree to the second and third
branches of the Markov node (select the terminal node and then select “Edit/Paste Subtree”).
8. Change the fourth branch of the Markov node (“Death”) to a terminal node.
Your tree should look like the tree on the following page.

Add information about the transition probabilities using the following table from the lecture notes:
9. Start by clicking immediately underneath the”Active” terminal branch of the “Remission” branch of the
Markov node, typing in “0.41,” and then “Enter.” Type in “0.00" for the transition probabilities for the
terminal branches “Flare” and “Death.” Type in “#” for the transition probability for the terminal node
“Remission.” The reason for this way of indicating probabilities will become obvious later in the exercise.
Add the other transition probabilities to the other terminal nodes. Use the pound sign (#) for the transition
probabilities for “Death” in the other terminal nodes.
Add information about the initial distribution of patients
The following table is from The lecture notes:
10. To start, click underneath the “Remission” branch of the Markov node and type in “0.10.” Add
numbers that correspond to the “Active” and “Flare”initial states, and use “#’ to indicate the value for
“Dead.”
Add values for the outcome variables
11. In this example, we will use two outcome variables. The first outcome variable will be a cost variable.
That variable is created using the probability of hospitalization, which is described in the following table
from the lecture notes:
3

To create the outcome variable, multiply the mean number of hospitalizations times the average cost of
hospitalization (cHosp = $10,000). To add the dollar cost of transitioning from remission to remission,
select the “Remission” terminal node of the “Remission” subtree coming out of the Markov node. Select
Node/Values List/Markov Info, or go to the top of your screen, click on the symbol that represents Markov
information (includes an “M” with a circle around it). Maximize the box that opens, and click on the row
that corresponds to Transition Reward 1, which is the row labeled “Trans Rwd” that follows the row labeled
“All Sets (Unused+Active).” In the column labeled “Value” type in (cHosp*.05)/((1+r)^ stage). Note that
dividing by (1+r)^_stage discounts the outcome. Push “Enter,” and a box will pop up. Select “General.”
W hen asked for descriptions, cHosp is the “Cost of a hospitalization, and r is the “Discount rate” and
stage is “Cycle number.” For each variable make sure the button for “Show definitions in tree” is
selected.
The second outcome variable measures the years of life. One year in any state except death will equal
one, and one year in the state of death will equal 0.5, because we assume death at midyear.
To recognize that years of life have different qualities, use the hypothetical quality data from the following
table in the lecture notes:
4

To add the number of QALYs associated with transitioning from remission to remission, select the
“Remission” terminal node of the “Remission” subtree coming out of the Markov node. Select
Node/Values List/Markov Info, or go to the top of your screen, click on the symbol that represents Markov
information (includes an “M” with a circle around it). In the box that opens, click on the row that
corresponds to Transition Reward 2, which is the row labeled “Trans 2” that follows the row labeled “All
Sets (Unused+Active).” In the column labeled “Value” type in 0.9. To discount this value, divide by (1+r)^_
stage, so the whole expression is: 0.9/((1+r)^ stage). Push “Enter.”
To display outcome information in the tree, go to “Edit/TreePreferences/Display/Variables/Markov Info and
select “Show Definitions.”
To prepare the tree for a cost-effectiveness analysis, go to “Edit/TreePreferences/Calculation/Calculation
Method/Cost-Effectiveness, and make sure “Cost payoff” is number 1 and “Effectiveness payoff” is
number 2. If the program says “C-E calculation method is not active. Do you want to activate it now?”
click “Yes.” Click on “Numeric Formatting,” and three boxes should appear. In all three, select 2 “Decimal
places,” “Add trailing zeros,” “Use thousands separator,” and “Show numbers” exactly. In the top most
box select “Units : Currency.” In the middle box, select “Units: Custom suffix” and “Prefix/suffix: QALYs.”
In the lowest box select “Units: None.”
Save your partially completed Markov model in *.trex format (“File/
save As ...”) in a file labeled with your last name followed by the number 8, for example, Williams8.
Repeat these steps to add discounted values for cost as Transition Reward 1 and discounted values for
Quality Adjusted Life Years as Transition Reward 2 to other terminal nodes, including the nodes for
“Death,” even when those values are zero.
If you want to avoid the tedium of adding all these values, download “Lupus Markov model 1" from the
assignments section of Blackboard, which has these values already added.
Even though you are not going to use stage rewards in this exercise, the program requires that there be
stage rewards. Check the stage rewards in the tree (they are located underneath the four branches of the
Markov node) to make sure all of them have values of zero. If any do not have the value of zero, go to the
top of your screen, click on the symbol that represents Markov information (includes an “M” with a circle
around it), and put zeros in the locations for stage rewards.
Tell the program how many cycles it should run
12. Select the Markov node, go to the top of your screen, select Node/Values List/Markov Info, or click on
the symbol that represents Markov information (includes an “M” with a circle around it). A panel should
appear at the bottom of your screen with the name “Termination conditions.” In the row labeled “Term” in
the column labeled “Value” type in STAGE>1999.
Define values for the variables you have created
Select the “Lupus” node, go to the top of your screen, select Node/Define Variable, or click on the symbol
at the top (and the bottom right) of your screen that includes “V=” twice, one over the top of the other. A
panel should appear with the names of the variables you have created. Select “cHosp” and assign it the
value of 10000 in the box that pops up. Repeat the process to assign the value of 0.03 to “r.” Repeat the
process and select stage. In the box that pops up there is a section labeled “Groups:” which is
immediately below the section labeled “_stage =”. In this section, select “Keywords.” Doing this activates
a list of words in the section labeled “Element:,” which is to the right. In this section, double click “_stage.”
Check to make sure that at the top of the box you have “ stage `= stage.”
Your tree should look like the following tree:
5

Analyze the model to make sure it works
13. To determine the average number of QALYs that would be expected for a population of lupus patients
who started with this distribution of states, go to “Edit/Tree Preferences .../Calculation Method/Active
Method/Simple/Active Payoff: 2/OK.” Select the Markov node and then go to “Analysis/Markov
Cohort/Markov Cohort (Full)/Decimal probability/ Don’t collapse subtrees/Show both stage and cumulative
rewards/Stages to include: 0 to 100/OK.”
Use the fourth icon of the nine icons at the upper right of this panel to save the text report as a
.pdf file. Label the file with your last name followed by 9, for example, Williams9.
Conduct a cost-effectiveness analysis comparing the Usual Care of SLE with a hypothetical
medical intervention.
14. Change the “Intervention” chance node into a Markov node, select the “Usual Care” subtree, copy it,
and paste it onto the “Intervention” Markov node.
Change the transition probability of the “Active” terminal node of the “Remission” branch of the
“Intervention” Markov node from 0.41 to rr*0.41.
Select the Lupus decision node, right click, select “Define variables” and assign the value 0.8537 to rr.
Assign values to the other variables at this node (cHosp = 10000, r = 0.03, and _stage = _stage).
Because the treatment costs one dollar a day, you should change the cost of the outcomes to reflect that
additional cost. For example, for the transition from Remission to Remission in the Intervention Markov
node change the value of Transition Reward 1 from “(cHosp*.05)/(1+r)^ stage” to
“((cHosp*.05)+cInterv)/(1+r)^ stage,” where cInterv is a variable representing the cost of the intervention
and has the value $365 (select the terminal node, click Node/Value Lists/Markov Info, and type in the new
information when the transition reward box pops up). Select the Lupus decision node, right click, select
“Define variables” and assign the value 365 to cInterv.
Save your partially completed Markov model in *trex format (“File/save As ...”) in a file labeled with
your last name followed by the number 10, for example, Williams10.
Make similar changes to the other transition rewards for cost (except those with zero values, because no
one made these transitions).
If you want to avoid the tedium of adding all these values, download “Lupus Markov model 2" from the
assignments section of Blackboard, which has these values already added.
Select the “Intervention” Markov node and go to “Node/Value Lists/Markov Info.” Type in “_stage > 1999”
for the termination rule. Check to make sure you have the same values for Termination in the boxes under
the two Markov nodes.
Select the “Lupus” node and use “Node/Value Lists/Variable Definitions to insert 0.03 to define the value
of r and 10000 to indicate the value of cHosp. To avoid potential conflicts between the definitions for
these variables at the “Lupus” node versus the definitions for these variables at the 2 Markov nodes, get
rid of definitions for these variables at the 2 Markov nodes by selecting each node and using
“Node/Values List/Variable Definition to bring up a list of variables. Highlight each variable you want to
delete and click on the red X to delete each one. Do not delete the variable definitions for the “Lupus”
node, which also are listed here.
Click on the “Lupus” node, and prepare for a cost-effectiveness analysis by clicking on “Edit/Tree
7

Preferences/Calculation/Calculation Method/Cost Effectiveness/ Cost Payoff 1, Effectiveness Payoff
2/OK.”
15. Perform a cost-effectiveness analysis. Click on the node “Lupus,” and click “Analysis/Cost-
Effectiveness...” Select “Yes” to the question “Draw chart with cost on the vertical axis?” W ait – this could
take a few seconds. On the right-hand side, select “Text Report.” If “Text Report” is not visible, minimize
the material in the extreme right side of this panel.
Use the fourth icon of the nine icons at the upper right of this panel to save the text report as a
.pdf file. Label the file with your last name followed by 11, for example, Williams11.
Email your saved files (file 8 through file 11) to sankey@wharton.upenn.edu.
Prepare the tree for a probabilistic sensitivity analysis.
The material that follows is optional, in part because it refers to last year’s information, which
included a model that was based on TreeAge 2009. If you decide to use this material anyway, when
you’re finished, you will have a model that will allow you to do probabilistic sensitivity analyses. If
you want to do probabilistic sensitivity analyses without doing all the things described in the
following material, download “Lupus Markov Model 3” from the assignments section of
Blackboard, and go to item 21 “Conduct a probabilistic sensitivity analysis.”
16. Start by creating Dirichlet distributions for the transition probabilities and the initial distribution. For the
transition probabilities of the Active subtree in the Usual Care Markov node, click
“Values/Distributions/New/Dirichlet,” type in the box labeled Alphas list the following material
“List(66;806;56;9),” which comes from the slide below, click OK, name the distribution “ddA,” describe it as
“Dirichlet distribution of the Active subtree,” click “Close.”
Go to the Remission terminal node of the Active subtree, and click underneath the branch next to 0.07 in
the grey box. Click “Insert Distribution,” click on “ddA,” click “Use,” and make sure the material in the white
box reads “Dist(1;1).” If it doesn’t, type the material in the white box. Click “OK.” Use similar steps to label
the Active terminal node “Dist(1;2),” the Flare terminal node “Dist(1;3),” and the Death terminal node
“Dist(1:4)” (or, alternatively, leave the pound sign in place).
8

Use similar steps to create an additional Dirichlet distribution for the transition probabilities of the flare
subtree, and put the transition probabilities in the tree, except in the terminal node Remission because we
have no observations that this transition takes place. Use similar steps to create an additional Dirichlet
distribution for the Remission subtree. Put zeros as the transition probabilities into the Flare and Death
terminal nodes of the Remission subtree (because we have no observations that these transitions take
place). Put a pound sign for the transitional probability of the Remission terminal node and “Dist(3;2)” for
the transitional probability for the Active terminal node. Create an additional Dirichlet distribution for the
initial distribution and put the probabilities into the tree below the branches leading from the Markov node.
Create distributions for cost and QALYs using the following information.
Table. Hypothetical mean number of hospitalizations for those who begin in state i and end in state j.
Remis. Active Flare Death
Remission 0.05 0.25 0.00 0.00
Active 0.10 0.20 1.00 0.50
Flare 0.00 0.25 1.25 0.75
9

QALY Distributions
Assume normal distribution (mean, SE)
Assume SD = 0.1
Assume N for calculating SE is same as the number of
individuals observed making the transitions reported in
the Table on slide 24
RtoR: 0.9, 0.0130; RtoA: 0.8, 0.0156
AtoR: 0.8, 0.0123; AtoA: 0.7, 0.0035
AtoF: 0.6, 0.0134; AtoD: 0.35, 0.0333
FtoA: 0.6, 0.0213; FtoF: 0.5, 0.0236
FtoD: 0.25, 0.0158
17. For example, to create outcomes for the Remission terminal node of the Remission subtree, first
create the distributions for the outcomes: click “Values/Distributions/new/Poisson”(Poisson because the
slide above says to use Poisson distributions for the number of hospitalizations), type “0.05" in the white
box labeled lambda (from the hypothetical table above of the number of hospitalizations for going from
Remission to Remission), click “OK,” name the distribution “dhRR,” describe the distribution as
“Distribution of number of hospitalizations, Remission to Remission,” and click “OK.” For the next
distribution click “Values/Distributions/new/Normal,” type “10,000" in the white box labeled Mean and 100
in the white box labeled Std Dev (from the slide above describing the cost of hospitalization), click “OK,”
name the distribution “dcHosp,” describe the distribution as “Distribution of the cost of hospitalization,” and
click “OK.” Use similar steps and the information in the slide immediately above to create a distribution for
the number of QALYs gained for going from Remission to Remission and name this distribution dqRR. To
put these distributions in the Markov model, “select the first terminal node, right click the terminal node,
select “Markov Transition Rewards,” type in the box for Transition Rwd 1 (which is labeled “Cost:”)
“(dhRR*dcHosp)/(1+r)^t,” type in Transition Rwd 2 (which is labeled “Effect;) (dqRR/(1+r)^t,” and click
“OK.”
Create and insert other distributions for the transition rewards for the remaining terminal nodes in the
Usual Care Markov node.
Transfer information from the Usual Care Markov node to the Intervention Markov node, and
modify it.
18. Start by deleting the subtree following the existing Intervention Markov node: right click the
Intervention Markov node, click “Select Subtree,” right click again, and click “Cut Subtree.” Right click the
Usual Care Markov node, click “Select Subtree,” right click again, click “Copy Subtree,” right click the
Intervention Markov node, and click “Paste nodes.”
19. There are two differences between the Usual Care and the Intervention Markov nodes. The transition
probabilities in the Remission subtree are different and the cost of the intervention must be added to the
outcomes in the Intervention model. To make these changes, start by creating a distribution for the
10

elative risk of fewer transitions from Remission to Active in the Intervention model (see slide above).To
create this distribution, click “Values/Distributions/New/LogNormal,” type “-.1582" in the white box labeled
mu (mean of logs) and .1816 in the white box labeled sigma (std dev of logs), click “OK,” name the
distribution “drr,” descibe the distribution as “Distribution of the relative risk of fewer transitions with
intervention,” click “OK, and click “Close.” Insert this distribution in the tree: click the Remission to Active
branch of the Intervention Markov node, change the transition probability from “Dist(3;2)” to “drr*Dist(3;2).”
Do not insert drr in any other transition nodes.
20. To create the distribution for the cost of the intervention, click “Values/Distributions/New/Normal,” type
“365" in the white box labeled mean and “50" in the white box labeled Std Dev, click “OK,” name the
distribution “dcDrug,” descibe the distribution as “Distribution of the cost of the intervention,” click “OK, and
click “Close.” Insert this distribution into the tree: for each terminal node in the Intervention model, right
click the node, and add dcDrug to the cost outcome. For example, change the cost outcome of the first
terminal node from “(dhRR*dcHosp)/(1+r)^t” to “((dhRR*dcHosp)+dcDrug)/(1+r)^t.”
Conduct a probabilistic sensitivity analysis.
21. Highlight the Lupus decision node. Click “Analysis/Monte Carlo Simulation/Sampling (Probabilistic
Sensitivity). . ./Sample all distributions.” Type 1000 in the white box labeled “Number of samples. . . .”
Click “More options/Seed random number generator,” make sure 1 is in the white box, and click
“OK/Begin.” W hen the program has finished running, click “Stats Format/Effectiveness,” put 4 in the white
box labeled Decimal places, click “OK/Close.” Compare your output with the output in the following 3
slides. To compare your output with the output in the first slide, click the rectangle at the bottom of the
page labeled “Stats shown for” to toggle between output for Usual Care and Intervention. To compare
your output with the output in the second slide, click “Graph/Distribution of Incrementals/Incremental
Cost/OK” and then “Graph/Distribution of Incrementals/Incremental Effectiveness/OK.” To compare your
output with the output in the third slide, repeat what you did to display the graphs and click on
11

“Actions/Distribution Statistics.”
12