Filetype PDF | Posted on 30 Jan 2023 | 3 years ago
Authentication
digital resources
352x Filetype PDF File size 1.44 MB Source: www.med.mcgill.ca
Markov Models in Medical Decision Making:
A Practical Guide
FRANK A. J. ROBERT MD
SONNENBERG, MD, BECK,
when a
are useful decision involves risk
Markov models problem that is continuous over
of events is and
when the when events more
time, timing important, important may happen
than once. such clinical with conventional decision trees is difficult
Representing settings
Markov
and unrealistic assumptions. models assume that a
may require simplifying patient
number
is in of a finite of health called
always one discrete states, Markov states. All events
are as transitions from one state to another. A Markov
model be
represented may evaluated
a
matrix as a cohort or as Monte Carlo simulation. A newer
by algebra, simulation, repre-
sentation of Markov the uses a tree of clinical
models, tree, representation
Markov-cycle
either as a cohort simulation or as a
events and be evaluated Monte Carlo
may simulation.
to events and
The of the Markov model represent repetitive the time of
ability dependence
both and utilities allows for more accurate of clinical that
probabilities representation settings
Markov
involve these issues. words: models; decision decision mak-
Key Markov-cycle tree;
Decis 1993;13:322-338)
ing. (Med Making
A decision
tree models the of a sub- of survival’ or from
prognosis patient standard life tables.’ This
paper
the
to choice of a For another method for life
sequent management
strategy. explores estimating expec-
a model the Markov
example, strategy involving surgery may the tancy, model.
events of and In Beck and Pauker described the
use of
death, 1983, Mar-
surgical surgical complications,
various outcomes of the treatment itself. For kov models for in
surgical determining medical
prognosis ap-
the must be
restricted to a Since that Markov
reasons, introduction, models
practical analysis plications.’
finite time often
referred to as the
time have been
frame, horizon with in
applied increasing frequency pub-
of the This
means aside from the lished decision
analysis. that, death, analyses.’-9 software
Microcomputer
outcomes chosen to be terminal
represented nodes has been to and eval-
by developed permit
constructing
of the tree not be
final but Markov models more For these
may outcomes, may reasons,
simply uating easily.
convenient for
represent stopping points the of a revisit of the Markov model is This
scope timely. paper
the tree
Thus, contains terminal nodes serves both as a review of the behind the
analysis. every theory Mar-
that for a kov model of and as a for
represent &dquo;subsequent
prognosis&dquo; particular prognosis practical guide
combination
of characteristics
and events. the construction of Markov
patient models microcom-
using
There are various in which a
ways decision software.
analyst puter
decision-analytic
can values to these
terminal nodes of the de- Markov models
are useful
assign particularly when a de-
cision tree. In some cases the outcome
measure
is a cision involves a risk that is over
problem ongoing time.
crude life in
expectancy; others it is a Some clinical are the risk of
quality-adjusted examples hemorrhage
life One method
expectancy.’ for life ex- while on the risk of of
estimating anticoagulant
therapy, rupture
is the an
pectancy declining exponential abdominal aortic and the risk of mor-
approximation aneurysm,
of
life which
(DEALE),2 calculates a in whether sick or There
expectancy patient- tality person,
any healthy.
rate for a combination of are two of events that
specific mortality given pa- important have
consequences
tient characteristics and comorbid
diseases. Life ex- risk. the
times at
which the will
ongoing First, events
also be obtained from models occur
are uncertain.
pectancies may Gompertz This has
important
implications
because the of an
outcome
utility often on
depends
when it occurs. For a
stroke
that im-
example, occurs
have a
mediately may different on the
Received from Division of impact patient
23, 1993, the General Internal
February than one
that occurs ten later. For economic
of Robert Wood Johnson years
Medicine, UMDNJ
Department Medicine,
both
costs and
Medical New the utilities are discounted&dquo;,&dquo;
School, Brunswick, New (FAS) and Infor- analyses,
Jersey
mation of
Medicine, such that later events have
Program, Houston, less than earlier
Technology Baylor College impact
Texas in
(JRB). Grant LM05266 from the National
Supported part
by ones. The
second is that a event
of Medicine and consequence given
Grant from the for Health
Library HS06396 Agency occur more
than once. As
may the
Care and Research. following example
Policy events that
shows, are or that
Address and to Dr. representing repetitive
correspondence reprint
requests Sonnenberg: occur with
Division of General Internal UMDNJ Robert Wood John- uncertain is difficult a
Medicine, timing using simple
son Medical 97 Paterson New NJ 08903.
School, tree
Street, Brunswick, model.
322
Downloaded from http://mdm.sagepub.com at National Institutes of Health Library on January 21, 2009
323
A Specific Example
has heart
a who a valve
Consider patient prosthetic
is Such a
and anticoagulant therapy. patient
receiving
embolic or event at time.
have an hemorrhagic any
may
Either kind of event causes and/
(short-term
morbidity
in the death. The
or and result
chronic) may patient’s
in 1 shows
tree one of
decision fragment figure way
such a The
the for patient. first
representing prognosis
chance labelled has three
node, ANTICOAG, branches,
labelled BLEED, and NO EVENT. Both BLEED and
EMBOLUS,
FATAL or NON-FATAL. If NO EVENT
EMBOLUS be either
may
the remains WELL.
occurs, patient
with
There are several this model. FIGURE 1. tree of
First, Simple fragment modeling antico-
shortcomings complications
therapy.
does not when events occur. Sec- agulant
the model specify
the structure that either or
ond, implies hemorrhage
In either
embolus occur once. fact, oc- out this for even five would
may only may carrying analysis years
more than once. at the terminal nodes
cur with hundreds of
Finally, result in a tree terminal branches.
POST POST and the
labelled EMBOLUS, BLEED, WELL, analyst Thus, a recursive model is tractable for a
only very
still is faced with the of utilities, a horizon.
problem assigning short time
task to the for each of
equivalent specifying prognosis
outcomes.
non-fatal
these The Markov Model
The first problem, specifying when events occur,
be addressed the tree structure in The Markov model a far more
may by using figure provides convenient
1 and the that either BLEED or of for
making assumption way modelling prognosis clinical with
problems
EMBOLUS occurs at the time consistent with risk. The
average ongoing model assumes that the is
patient
rate of each For if
known
the in one of a finite number of
states of
complication. example, always health
rate of is a constant 0.05
the referred to as
Markov states. All events of interest
hemorrhage per person are
the time before the occurrence
then modelled as transitions from one
state to
another.
per year, average Each
or the event
of a is 1/0.05 20 state is a and the
Thus, contribution of this
hemorrhage years. assigned utility,
of a fatal will be associated with to the overall on the
having hemorrhage utility prognosis depends length
a of 20 of survival. of
time in the In
utility years normal-quality However, spent state. our of a
example patient
the normal life be less than with a heart valve, these states are WELL,
patient’s expectancy may prosthetic
of a stroke would have
20 the occurrence and
DEAD. For the sake
Thus, DISABLED, of in this
years. simplicity
the effect of the life we assume that either a
bleed or a
paradoxical improving patient’s example, non-fatal
Other such as that embolus will result in the same
expectancy. approaches, assuming state (DISABLED) and
the stroke occurs the nor- that the is
through patient’s
halfway disability permanent.
mal life are and lessen the The time
expectancy, arbitrary may horizon of the is divided into
analysis equal
the
of increments of referred
fidelity analysis. time, to as Markov Dur-
cycles.
and the of
Both the of events each the make a
timing representation ing cycle, patient transition from
may
more than once can be ad- one
occur state
events that may to another. 3 shows a used
Figure commonly
dressed a recursive decision tree.12 In a re- of Markov called a state-
using representation
by processes,
that have
have branches
some nodes transition in
cursive which each state is
tree, ap- diagram, represented
in the tree. Each of the a circle. Arrows different
two states
previously repetition by in-
peared connecting
a convenient of time dicate
tree structure allowed transitions. Arrows from a
represents length leading state
event be considered A re- to itself indicate the remain
and that in that
any may repeatedly. may
patient
that
tree models the state in
cursive anticoagulation problem consecutive certain transitions
cycles. Only
is in 2. are allowed. For a in the WELL
depicted figure example, person state
the nodes the terminal make a transition to the DISABLED but a
Here, representing previous may state,
nodes and No EVENT transition
POST-BLEED, are re- from DISABLED to WELL is not A
POST-EMBOLUS, allowed. per-
the chance node which son in either WELL
the state or the DISABLED
by ANTICOAG, state
placed appeared may
at the
root of the tree. Each die
occurrence of and thus make a transition
to the DEAD
previously state. How-
or EMBOLUS
BLEED a distinct time so ever, a who is in the DEAD state,
represents period, person obviously,
recursive model when events
the can occur. cannot make
a transition to other state.
represent any Therefore,
this model a
and car- arrow emanates from
the
However, DEAD
despite simple state,
relatively single leading
out the recursion for two time the back to itself. It is assumed
rying only periods, that a in a
patient given
in 2 is
tree with 17 terminal state can make
branches. a state
figure &dquo;bushy,&dquo; only single transition a
during
If each level of recursion represents one then
year, cycle.
Downloaded from http://mdm.sagepub.com at National Institutes of Health Library on January 21, 2009
324
FIGURE 2. Recursive tree mod-
of
eling complications antico-
agulant therapy.
data. For
The of the is chosen to a determined the available ex-
length cycle represent by probability
time interval. For a model that if only yearly probabilities are available, there
clinically meaningful ample,
the entire life of a and is little to a
spans history patient relatively advantage using monthly cycle length.
rare events the can be one On the
length
cycle year.
other if the time frame is shorter and
hand, models
events that occur much
more the
may frequently, cycle UTILITY
INCREMENTAL
time must be for or even
shorter, example monthly
Markov the
The time Evaluation of a
weekly. cycle also must be shorter if a rate process yields average
over of the amount
time. An is the risk of number (or
changes rapidly example cycles analogously, average
each state. Seen the
of in another
infarction time)
perioperative myocardial (MI) spent way,
following pre-
to a is credit&dquo; for the time in each
vious MI that declines stable value over six months.&dquo; patient &dquo;given spent
attribute of of
The of this in state. If the interest is duration
rapidity change risk dictates a only
monthly
time. Often the choice of a time will be then one need add the
cycle cycle survival, only together average
Downloaded from http://mdm.sagepub.com at National Institutes of Health Library on January 21, 2009
325
When cost-effectiveness a
performing analyses,
incremental be for each
separate utility may specified
state, the financial cost of in that
representing being
is
state for The model evaluated
one
cycle. separately
for cost and survival. Cost-effectiveness ratios
are cal-
culated as for a standard decision tree.10,11
MARKOV
TYPES OF PROCESSES
Markov are to
processes categorized according
whether the state-transition are constant
probabilities
over In the most
time or not. general of Markov
type
the transition over
process, probabilities
may
change
time. For the transition for the
FIGURE 3. Markov-state Each circle a Markov example, probability
diagram. represents DEAD
transition from WELL to consists of two
allowed transitions. compo-
state. Arrows indicate
nents. The first is the of
component probability dying
from unrelated causes. In general, this probability
times spent in the individual states to arrive at an
as
over time the
because, older,
survival for the changes patient gets
expected process. the of from unrelated causes will
probability dying
increase The second is the
continuously. component
n of a fatal or
Expected utility = ~ ts probability suffering hemorrhage embolus
the This be
not constant
s=l i during cycle. may or may
over time.
A of Markov in
where is the time in state s. special type process which the tran-
ts spent
sition are constant over time
of probabilities is called a
the survival is consid-
Usually, however, quality
Markov chain. If it has an its
state, behavior
ered Each state is associated with a absorbing
important. quality
over
time can be determined as an exact
solution
factor the of life in that state rel- by
representing quality
matrix as discussed below.
ative to health. The that is associated simple algebra, The DEALE
perfect utility
with in a state is can be used to derive the constant mortality rates
one referred
spending cycle particular
needed to a Markov chain. the
to as the incremental Consider the Markov implement However,
utility. pro-
of software to evaluate
cess in 3. If the incremental of availability specialized Markov
depicted figure utility
and the afforded
the DISABLED state is 0.7, then the in processes greater accuracy by age-
spending cycle
specific rates have resulted in reli-
the DISABLED mortality
state contributes 0.7 greater
quality-adjusted cycles
ance on Markov with
to the expected utility. Utility accrued for the entire processes time-variant proba-
Markov is the total number of in bilities.
process cycles spent
The net of a transition from
the making one
each each incremental probability
state, multiplied by utility
state
to another a is called a
for single tran-
that state. during cycle
sition The Markov is
probability. process completely
n defined by the probability distribution among the
X the
ts Us states and the for individual
utility = ~
Expected starting probabilities
s=l i of n
allowed transitions. For a Markov model states,
there n2 When
DEAD state has an incremen- will be transition probabilities. these
Let us assume that the with to
probabilities are constant respect time, they
WELL
tal of and that the state has an in-
utility zero,* x n as shown
can be an n matrix, in
that represented by
cremental 1.0. This means for
of
utility every cycle table 1. Probabilities representing disallowed transi-
is credited
in the WELL state the with a
spent patient tions of be zero. This called the P
of to the duration of a will, course, matrix,
quantity utility equal single matrix, forms the basis for the fundamental matrix
Markov If the spends, on 2.5
cycle. patient average, described in detail
solution of Markov chains Beck
in the WELL state and 1.25 in the DISABLED by
cycles cycles and Pauker.’
DEAD the
state before the state,
entering utility assigned
would be X + X or 3.9
(2.5 1) (1.25 0.7), quality-ad-
justed This number is the life
cycles. quality-adjusted TaMe 1 . P Matrix
of the
expectancy patient.
* medical the incremental of the
For examples, utility absorbing
must
DEAD state be because the will an infinite
zero, patient spend
time in the DEAD if the
of state and incremental were
amount utility
the net for the Markov would
non-zero, utility process be infinite.
Downloaded from http://mdm.sagepub.com at National Institutes of Health Library on January 21, 2009
no reviews yet
Please Login to review.
