The Effect of Learning and Fatigue on Preferences and WTP in a ...

<strong>The</strong> <strong>Effect</strong> <strong>of</strong> <strong>Learning</strong> <strong>and</strong> <strong>Fatigue</strong> on Preferences <strong>and</strong> WTP
in a Choice Experiment
Fredrik Carlsson 1 , Morten Raun Mørkbak 2 , <strong>and</strong> Søren Bøye Olsen 2
1 University <strong>of</strong> Gothenburg, Department <strong>of</strong> Economics, Box 640, SE-405 30 Gothenburg, Sweden.
2 Institute <strong>of</strong> Food <strong>and</strong> Resource Economics, Faculty <strong>of</strong> Life Sciences, University <strong>of</strong> Copenhagen.
Rolighedsvej 25, 1958 Frederiksberg C, Denmark.
Abstract
This paper addresses the issues <strong>of</strong> how learning <strong>and</strong> fatigue effects related to the repeated choices
potentially affect respondents’ stated preferences in a Choice Experiment setting. In a case study
concerning food quality attributes in chicken breast filet, the results show that a learning effect is
indeed present in a sequence <strong>of</strong> 16 choice sets, whereas there is no evidence <strong>of</strong> fatigue effects. This
conclusion is based on an observed reduction in the error variance for the last 8 choice sets relative
to the first 8 choice sets. An analysis incorporating choice set specific scale factors reveals that the
main part <strong>of</strong> the observed learning process takes place in the first choice set. <strong>The</strong> overall preference
structure is found to differ significantly between the two sequences <strong>of</strong> choice sets. This suggests that
the learning effect leads to changes in overall preference structure. Turning to a comparison <strong>of</strong>
attribute WTP estimates, the results indicate that respondents refining <strong>and</strong> significantly adjusting
their stated values upwardly for two <strong>of</strong> the four quality attributes considered in the survey. If the
main objective <strong>of</strong> a Choice Experiment survey is to obtain policy advice in terms <strong>of</strong> WTP, then
allowing for learning effects can be essential.
1

1. Introduction
Ample experimental <strong>and</strong> stated preference evidence suggests that people do make incoherent
choices, that they are affected by for example the context, <strong>and</strong> various cues. This is clearly at odds
with st<strong>and</strong>ard assumptions in economics, <strong>and</strong> also with the assumptions we make when analysing
stated preference responses. One hypothesis set forth explaining some <strong>of</strong> these anomalies is the
Discovered Preference Hypothesis (DPH) (Braga & Starmer 2005, Plott 1996). <strong>The</strong> DPH states that
when respondents are faced with new decisions in unfamiliar environments, initial decisions will be
incoherent <strong>and</strong> exhibit large r<strong>and</strong>omness. However, as choices are repeated <strong>and</strong> respondents gain
familiarity with the decision environment, decisions will progressively be more coherent <strong>and</strong> less
r<strong>and</strong>om. It is thus argued that by “training” respondents their preferences become more stable,
which is also denoted as a learning effect (Cummings et al. 1986). However, a counteracting effect
in the setting <strong>of</strong> a stated preference survey is fatigue. Respondents could get tired <strong>of</strong> the task, <strong>and</strong>
thus making more r<strong>and</strong>om choices over the sequence <strong>of</strong> choice tasks.
<strong>Learning</strong> effects has been discussed widely in the economic valuation literature – initially in the
context <strong>of</strong> contingent valuation surveys (CVM), see e.g. Bateman et al. (2008a), Bergstrom & Stoll
(1989), or Bjornstad et al. (1997). <strong>The</strong> issue <strong>of</strong> learning <strong>and</strong> fatigue is, however, <strong>of</strong> greater interest
when subjects hare making repeated choices such as in double-bounded CVM <strong>and</strong> in particular
choice experiments (CEs). Consequently, these issues have received a fair amount <strong>of</strong> attention in
the CE literature (Bateman et al. 2008b, Brouwer et al. 2009, Carlsson & Martinsson 2001, Dellaert
et al. 1999, DeShazo & Fermo 2002, Hanley et al. 2002, Holmes & Boyle 2005, Johnson &
Bingham 2001, Ladenburg & Olsen 2008, Lagerkvist et al. 2006, Phillips et al. 2002, Savage &
2

Waldman 2008). However, results are ambiguous. <strong>The</strong> majority <strong>of</strong> these papers have examined
learning <strong>and</strong> fatigue effects by comparing the choices made in identical choice sets presented at
different positions in a sequence <strong>of</strong> choice sets. <strong>The</strong> traditional assumption <strong>of</strong> stable preferences
entails that the choice should be the same regardless <strong>of</strong> the location in the sequence <strong>of</strong> choices.
<strong>Learning</strong> <strong>and</strong> fatigue effects would violate this assumption <strong>and</strong> could result in choices being
dependent on the location in the choice set sequence.
From an econometrics point <strong>of</strong> view <strong>of</strong> the r<strong>and</strong>om utility framework, identical repeated choice
tasks should result in identical choices (Bateman et al. 2008b). However, from the psychology point
<strong>of</strong> view there is a true r<strong>and</strong>om component in the r<strong>and</strong>om utility framework, so it would be expected
that some deviations would exist (Bateman et al. 2008b). Because <strong>of</strong> this Bateman et al. (2008b)
also argue that from an economic point <strong>of</strong> view, a test <strong>of</strong> whether or not preferences have changed
would be a more relevant test than whether or not choices have changed. In the present paper we
add to the existing literature by providing an internal test for potential learning <strong>and</strong>/or fatigue
effects. Specifically, we do not only examine the potential changes in the preference structure
throughout the choice set sequence as has been done in the majority <strong>of</strong> the previous studies on this
subject; to our knowledge we are the first to provide an internal test for potential changes in the
willingness-to-pay (WTP) estimates caused by the choice set sequence, i.e. the impact <strong>of</strong> learning
<strong>and</strong>/or fatigue effects on obtained policy advice. In a case considering food safety attributes <strong>of</strong>
chicken breast filet, we apply an experimental setup using 16 CS per respondent where the last
sequence <strong>of</strong> 8 CS is an exact copy <strong>of</strong> the first sequence <strong>of</strong> 8 CS. <strong>The</strong> advantage <strong>of</strong> repeating exactly
the same choice sets is that it limits the influence <strong>of</strong> sequencing <strong>of</strong> the choice sets. On the other
h<strong>and</strong>, there is an additional counter acting effect to learning with our design <strong>and</strong> that is respondents
desire to act in a coherent way. This is what Ariely et al. (2003) call coherent arbitrariness, i.e. an
3

individual’s choices can be internally coherent, but at the same time they can be anchored on the
first choice or some initial starting point.
Our results suggest that a learning effect is present. This is observed through a reduction in the error
variance for the last 8 CS relative to the first 8 CS. Moreover the results show that the overall
preference structure differs between the two sequences <strong>of</strong> choice sets, <strong>and</strong> further analysis showed
that the WTP estimates were affected to some degree. We find not evidence <strong>of</strong> fatigue effects in the
applied sequence <strong>of</strong> 16 CS. While our results imply that learning effects can indeed be <strong>of</strong> significant
structural importance when conducting CE surveys, our results also underline that if focus is solely
on obtaining policy advice in terms <strong>of</strong> WTP estimates, learning effects are not negligible.
<strong>The</strong> paper is organized as follows. <strong>The</strong> following section provides an overview <strong>of</strong> the existing
literature dealing with learning <strong>and</strong> fatigue effects in CE. Section 3 briefly describes the method
used in the survey <strong>and</strong> the theory behind the method. In section 4 the design <strong>of</strong> the survey <strong>and</strong> the
data are described, followed by presentation <strong>of</strong> the results in section 5. Finally we discuss <strong>and</strong>
conclude in section 6.
2. Existing evidence <strong>of</strong> learning <strong>and</strong> fatigue effects in CE
In a split sample CE study, Ladenburg & Olsen (2008) find evidence <strong>of</strong> starting point bias, but the
impact <strong>of</strong> this anomaly decays as more choice sets are evaluated. <strong>The</strong>y conclude that a learning
effect effectively reduces the bias. This is clearly in accordance with the DPH. Previous studies
concerning the DPH interpretation <strong>of</strong> decaying anomalies have found similar results. In a literature
review, Braga & Starmer (2005) find some, though not unequivocal, support for the DPH. Cherry et
al. (2003) <strong>and</strong> List (2003) find that people generally become more rational through refining values,
4

not by changing preferences as have been suggested from a constructed preference or prospect
theory view (Slovic 1995, Kahneman & Tversky 1979, Tversky & Kahneman 1986). In other terms,
the stated values become more consistent with the true preferences as market experience increases
through a process <strong>of</strong> repetition <strong>and</strong> learning.
Dellaert et al. (1999) <strong>and</strong> DeShazo & Fermo (2002) both examine the effect <strong>of</strong> different choice set
designs (complexity) on consumer choice consistency <strong>and</strong> find that increased cognitive burden are
associated with an increase in the error variance, which they interpret as decrease in choice
consistency. Lagerkvist et al. (2006) use a binary heteroskedastic logit model to account for
differences in the error variance between the first 3 choice sets (CS) <strong>and</strong> the last 3 CS in a CE
survey. <strong>The</strong>y find a higher variance for the second half <strong>of</strong> the experiment <strong>and</strong> conclude that this is
due to respondents getting fatigued <strong>and</strong> lose interest in the end <strong>of</strong> the experiment. This finding is
supported by the findings in Savage & Waldman (2008). <strong>The</strong>y also find that respondents suffer
fatigue or boredom when they compare the error variance in the first 4 CS with the error variance in
the last 4 CS. Holmes & Boyle (2005) use an attribute-based referenda approach, which is a
combination <strong>of</strong> a choice experiment <strong>and</strong> a dichotomous choice question with a sequence <strong>of</strong> four
“choice sets” presented to each respondent. <strong>The</strong>y test the error variance <strong>of</strong> the first valuation
question against the error variance <strong>of</strong> the second, third <strong>and</strong> fourth valuation question, respectively,
<strong>and</strong> find only the error variance <strong>of</strong> the first <strong>and</strong> fourth valuation question to differ, with the variance
in the latter being the smallest, suggesting a learning effect. Carlsson & Martinsson (2001) examine
learning in a CE conducted both on hypothetical market <strong>and</strong> on an actual market, by dividing the 16
CS into two sequences <strong>of</strong> 8 (A, B). <strong>The</strong>y then compare model results <strong>of</strong> sequence B data in a split
sample design in which respondents have received sequence B as the first <strong>and</strong> as the last 8 CS
respectively. <strong>The</strong>y find no differences between the scale parameters in the two models, neither in
5

the hypothetical market case nor in the actual market case, suggesting no learning or fatigue effect
being present. Comparing a sequence <strong>of</strong> four CS with a sequence <strong>of</strong> eight CS, Hanley et al. (2002)
find only very weak evidence that the number <strong>of</strong> CS affects preferences in any way. Further, they
find no significant impact on WTP measures, again suggesting that no learning or fatigue effect is
present. Similarly, Johnson & Bingham (2001) compare WTP estimates obtained from the
beginning <strong>and</strong> last end <strong>of</strong> a sequence <strong>of</strong> SP questions <strong>and</strong> find no significant differences.
In a repeated choice experiment Brouwer et al. (2009) examine learning over five CS with the first
<strong>and</strong> the last CS being identical. <strong>The</strong>y also examine a learning effect through the size <strong>of</strong> the error
variance, <strong>and</strong> find no differences in the variance throughout the five CS, <strong>and</strong> hence no learning or
fatigue effect. In addition, they test the stability <strong>of</strong> preferences by comparing the first <strong>and</strong> the fifth
CS which is identical, <strong>and</strong> find no differences in the error variance here as well, although 27 % <strong>of</strong>
the respondents did chose another policy alternative in the fifth CS compared to the first CS.
Phillips et al. (2002) use a similar approach, also adding an identical first CS at the end <strong>of</strong> the CE.
<strong>The</strong>y only examine the rate <strong>of</strong> the same alternative being chosen in the last CS, <strong>and</strong> they find that 75
% <strong>of</strong> the respondents choose the same alternative in the last CS as they did in the first CS. Bateman
et al. (2008b) also analyses the effect <strong>of</strong> adding in an identical first CS as the last choice, resulting
in 17 CS in total for each respondent. <strong>The</strong>y find that respondents are less likely to choose an
identical alternative when it is placed in the end <strong>of</strong> a choice sequence, <strong>and</strong> furthermore that the
effect on the marginal utility <strong>of</strong> a given alternative is lower when placed in the end <strong>of</strong> the choice set
sequence. Finally they examine a heteroskedastic multinomial logit model in which 16 orderspecific
scale parameters were estimated. <strong>The</strong> results <strong>of</strong> this showed that only the scale factors for
choice tasks 3, 9, <strong>and</strong> 16 were statistically significantly different from one at the usual significance
level (0.05). On the basis <strong>of</strong> this they conclude that the most reliable estimates are found in the
6

eginning <strong>and</strong> in the end <strong>of</strong> the choice task sequence, so the approach <strong>of</strong> using order-specific scale
parameters to support a learning hypothesis failed.
3. Method
<strong>The</strong> underlying theory <strong>of</strong> CE is based on Lancaster’s Consumer <strong>The</strong>ory (LCT) (Lancaster 1966) <strong>and</strong>
r<strong>and</strong>om utility theory (Gravelle & Rees 1992, Luce 1959, McFadden 1974). According to
Lancaster, the (indirect) utility V ij that individual i achieves from good j is the sum <strong>of</strong> the utilities
obtained from each <strong>of</strong> the K characteristics s kij . Assuming linearity in the valuation <strong>of</strong> attributes, the
utility V ij can be written as follows:
V = β1s1 + β2 s2 + .......... + β s
ij i ij i ij Ki Kij
(1)
<strong>The</strong> parameter β ki represents the weight by which attribute k is valued by individual i 1 . In r<strong>and</strong>om
utility theory, it is assumed that individuals make choices according to a deterministic part along
with some degree <strong>of</strong> r<strong>and</strong>omness. Allowing U ij to represent the r<strong>and</strong>om utility that individual i
places on alternative j, V ij now represents the deterministic component <strong>of</strong> the utility function <strong>and</strong> ε ij
is a r<strong>and</strong>om variable that captures the unsystematic <strong>and</strong> unobserved r<strong>and</strong>om element <strong>of</strong> individual
i’s choice (Hanley et al. 2002, Holmes & Adamowicz 2003). We will assume throughout the paper
that the error terms are independent Gumbel distributions. Hence, the r<strong>and</strong>om utility U ij can be
represented as follows:
U
ij
= β .......... +
is
ij
+ β
is
ij
+ + β
KisKij
ε
(2)
1 1 2 2
ij
1 Sometimes, the attribute weights differ between alternatives β kj but in the present study it was not meaningful to
operate with different attribute weights for the hypothetical alternatives.
7

In a st<strong>and</strong>ard logit specification, all parameter coefficients are fixed whereas a r<strong>and</strong>om parameter
logit (RPL) model, as we apply in this paper allows for variations in personal tastes. <strong>The</strong> researcher
cannot observe β i for each individual so it is not possible to condition on β i . However, by assuming
that the β i coefficients vary across respondents with the density f(β i ), it is possible to obtain the
unconditional probability P ij that individual i chooses alternative j, as the integral <strong>of</strong> the conditional
probability on β i over all possible values <strong>of</strong> β i :
⎛ βi'
S ⎞
ij
e
Pij =
⎜ ⎟
f
'
( βi ) dβ
β i
i Sin
e
⎜ ⎟
⎝ n ⎠
∫ ∑ (3)
We assume that all non-price attributes are normally distributed thereby allowing consumers to
place positive as well as negative values on the non-price attributes as well as the alternative
specific constant, which has been shown to work well in earlier studies. <strong>The</strong> price coefficient is
being held constant because such an assumption <strong>of</strong> a constant price parameter allows straight
forward calculations <strong>of</strong> the distribution <strong>of</strong> WTP.
In testing for a potential learning or fatigue effects we apply the Likelihood Ratio test procedure
suggested by Swait & Louviere (1993) <strong>and</strong> draw on the notion used by e.g. Holmes & Boyle (2005)
<strong>and</strong> Savage & Waldman (2008) that larger error variance is caused by fatigue, while smaller error
variance is caused by learning. In this test we compare a model based on the first 8 CS with a model
based on the last 8 CS, but the test could easily be conducted on larger number <strong>of</strong> models within the
16 CS (see below for further details on the design). In the context <strong>of</strong> learning, it makes sense to
assume that learning processes will lead to choices that exhibit a relatively lower degree <strong>of</strong>
8

unobserved variability, <strong>and</strong> thus a higher degree <strong>of</strong> estimation precision (Hole 2006). It makes
similarly good sense to assume that fatigue would have the opposite effect.
Recall that the scale parameter is the inverse <strong>of</strong> the st<strong>and</strong>ard deviation <strong>of</strong> the error term (Swait &
Louviere 1993). This implies that the variance <strong>of</strong> the error term or “noise” can be estimated as 1/σ 2 .
<strong>The</strong> coefficients (β) in the econometric models are usually expressed in their scaled form
*
( β β σ )
= , where the scale parameter σ <strong>and</strong> the ‘original’ coefficients β * cannot be separately
identified (Train 2003). <strong>The</strong> estimated parameter β indicate the effect <strong>of</strong> each observed variable
relative to the variance <strong>of</strong> the unobserved factors (Train 2003). To test for differences in scale, the
differences have to be isolated before comparing the parameters. <strong>The</strong> problem is though, that the
scaling factor cannot be identified in any particular set <strong>of</strong> empirical data. But instead the ratio <strong>of</strong> the
scale factor <strong>of</strong> one data set relative to another can be identified by normalizing one <strong>of</strong> them to the
value <strong>of</strong> 1 <strong>and</strong> then defining a range <strong>of</strong> values <strong>of</strong> the other scale factor, within which we expect the
log likelihood function to be maximized (Swait & Louviere 1993).
In estimating the models we have used the s<strong>of</strong>tware package Biogeme (Bierlaire 2003), which
allows for a direct estimation <strong>of</strong> the ratio <strong>of</strong> scale factors. <strong>The</strong> models are estimated with simulated
maximum likelihood using Halton draws with 300 replications; see Train (2003) for details on
simulated maximum likelihood <strong>and</strong> Halton draws.
4. Design
9

Prior to the design <strong>of</strong> the CE, three focus group interviews were performed 2 . In the focus groups the
following attributes were identified as being important in relation to choice <strong>of</strong> chicken breast fillets:
Type <strong>of</strong> production, country <strong>of</strong> origin <strong>and</strong>, to some extent, food safety (mainly Salmonella). Even
though food safety did not appear to be <strong>of</strong> great concern to consumers, the original purpose <strong>of</strong> the
present study was to elicit the relative weighting <strong>of</strong> food safety. Consequently, we included two
food safety attributes associated with the chicken products: “Salmonella-free” <strong>and</strong> “Campylobacterfree”.
<strong>The</strong> two specific food safety attributes were chosen because <strong>of</strong> their relevance to each <strong>of</strong> the
products <strong>and</strong> also because they were judged as representing an increasingly important issue from a
scientific as well as a political perspective. <strong>The</strong> attributes <strong>and</strong> their associated levels are presented in
table 1.
Table 1: <strong>The</strong> attributes <strong>and</strong> their levels in the Choice Experiment for the chicken breast fillet.
Attributes
Type <strong>of</strong> production
Country <strong>of</strong> origin
Campylobacter-free
Salmonella-free
Levels
Conventional (indoor), organic (outdoor)
Domestic (Danish), Non-domestic
Not labelled Campylobacter-free, Campylobacter-free
Not labelled Salmonella-free, Salmonella-free
Price (DKK) 25, 28, 33, 40, 50, 65, 85, 115
Note: DKK 10 ~ EUR 1.34.
<strong>The</strong> two food safety attributes in the survey, Salmonella-free <strong>and</strong> Campylobacter-free are quite
similar. <strong>The</strong>y both exhibit private good characteristic to a large extent, <strong>and</strong> both give rise to more or
less the same course <strong>of</strong> illness. <strong>The</strong> main difference is that the current risk <strong>of</strong> getting infected by
Campylobacter is much higher than the risk <strong>of</strong> a Salmonella infection. Consequently, our a priori
2 <strong>The</strong> focus groups were headed by the sociologist Korzen, S., Department <strong>of</strong> Human Nutrition, University <strong>of</strong>
Copenhagen, who works with qualitative studies on food perceptions.
10

expectation <strong>of</strong> the value <strong>of</strong> a Salmonella-free chicken is that respondents will not value it as highly
as the Campylobacter-free characteristic.
A D-optimal fractional factorial design resulting in a sequence <strong>of</strong> 8 CS was used for the
experimental design. At the end <strong>of</strong> this sequence, the entire sequence <strong>of</strong> 8 CS was repeated,
resulting in a total <strong>of</strong> 16 choice sets per respondent. Hence the fractional factorial design <strong>of</strong> 8 CS
was actually presented twice to each respondent. Respondents were not made explicitly aware <strong>of</strong>
this feature <strong>of</strong> the design. In each choice set, the respondents were faced with two alternative
chicken breasts fillet products plus a third status quo alternative (all specified as packages <strong>of</strong> 500
grams). <strong>The</strong> latter characterised the respondents’ usual purchase, which was identified earlier in the
questionnaire. This approach <strong>of</strong> using the respondents’ ”own” status quo values has been
recommended <strong>and</strong> used in other studies to mimic the actual purchasing situation as closely as
possible (Kontoleon & Yabe 2003, Ruby et al. 1998). <strong>The</strong> use <strong>of</strong> individual specific status quo
alternatives in the design procedure <strong>of</strong> CE have been further developed by Rose et al. (2008), in
what they refer to as segment-specific efficient designs, two-stage process designs <strong>and</strong> individual
efficient designs.
5. Results <strong>and</strong> discussion
<strong>The</strong> CE survey was conducted using an internet panel. <strong>The</strong> sample was obtained from Nielsen’s
online database. In Denmark, there are approximately 2.4 million private households, <strong>of</strong> whom 87%
are online. <strong>The</strong> panel members are all aged 15 years or above, <strong>and</strong> they all reside in a household
with internet access, but in the present survey only respondents above age 18 were allowed to
participate. <strong>The</strong> final sample consisted <strong>of</strong> 389 respondents, each answering 16 CS resulting in a
total <strong>of</strong> 6224 choice observations. <strong>The</strong> response rate was 26 %.
11

Initially, four RPL models are estimated:
Model (i): based on the first 8 CS;
Model (ii): based on the last 8 CS;
Model (iii): based on all 16 CS, not accounting for potential difference in scale
between the first <strong>and</strong> last 8 CS;
Model (iv): based on all 16 CS, allowing <strong>and</strong> accounting for potential difference in
scale between the first <strong>and</strong> last 8 CS.
Table 2 displays the results obtained in the four different models.
Table 2: RPL models for the first 8 <strong>and</strong> the last 8 CS, <strong>and</strong> models for including all 16 CS both with
<strong>and</strong> without corrections for differences in scale.
Variable
Campylobacter-free label
Salmonella-free label
Domestic produce
Organic production
ASC (Status quo)
Price
St<strong>and</strong>ard Deviation
Campylobacter-free label
Model (i)
First 8 CS
Coefficients
(Std. Err.)
0.81
(0.0668)
0.62
(0.081)
1.192
(0.0992)
0.454
(0.0952)
0.142
(0.13)
-0.0375
(0.00267)
t-value
12.13
7.66
12.01
4.77
1.09
-14.07
Model (ii)
Last 8 CS
Coefficients
(Std. Err.)
0.916
(0.0886)
0.95
(0.1224)
1.338
(0.1604)
0.764
(0.1378)
0.32
(0.139)
-0.0441
(0.00368)
t-value
10.35
7.76
8.34
5.55
2.29
-11.96
Model (iii)
All 16 CS - not
corrected for scale
Coefficients
(Std. Err.) t-value
1.252
(0.103)
12.16
1.142
(0.1338)
8.55
1.906
(0.1778)
10.73
1.214
(0.1732)
7.01
-0.36
(0.165)
-2.18
-0.0549
(0.00394)
-13.92
Model (iv)
All 16 CS - corrected
for scale
Coefficients
(Std. Err.)
t-value
0.432
(0.0358)
12.05
0.396
(0.0458)
8.63
0.658
(0.0624)
10.52
0.422
(0.059)
7.13
-0.123
-2.12
(0.058)
-0.0189
(0.00135)
0.0746
0.22
1.174
0.406
0.35
0.76
8.72
(0.212)
(0.29)
(0.1346)
(0.046)
Salmonella-free label
0.54
0.84
1.552
0.534
-3.84
-4.29
-10.56
(0.1404)
(0.1962)
(0.1468)
(0.0506)
Domestic produce
0.676
1.038
1.736
0.598
-4.53
-5.9
-10.84
(0.1496)
(0.176)
(0.16)
(0.0556)
Organic production
1.046
1.236
2.38
0.818
5.42
-5.52
-9.36
(0.193)
(0.224)
(0.254)
(0.0874)
ASC (Status quo)
1.93
2.11
3.39
1.17
13.93
10.95
13.93
(0.138)
(0.193)
(0.243)
(0.0854)
Scale factor (first 8 CS = 1)
1.14
(0.0491)
LL -2534 -2415 -4304 -4300
Adj. Rho-square 0.256 0.291 0.369 0.369
-13.99
8.8
-10.54
-10.73
-9.37
13.65
2.85 a
12

Note: ASC is the alternative specific constant for the status quo alternative. LL refers to the final log-likelihood <strong>and</strong> adj.
Rho-square refers to the adjusted Likelihood Ratio Index. Because the discrete variables are effect coded, the non-price
attribute-coefficients have been multiplied by two, to account for the fact that the reference level is assigned a value <strong>of</strong>
-1 instead <strong>of</strong> 0 (Bech & Gyrd-Hansen 2005).
a <strong>The</strong> t-value for the scale factor is a t-value tested against the null hypothesis H 0 : σ =1.
Almost all attribute variables show a statistical significant effect on the choice probabilities, with
the exception <strong>of</strong> the ASC in model (i) <strong>and</strong> the st<strong>and</strong>ard deviation <strong>of</strong> the Campylobacter-free label in
models (i) <strong>and</strong> (ii). With adjusted rho-square values <strong>of</strong> above 0.256, all models describe the
variation in the data very well (Domencich & McFadden 1975; Louviere et al. 2000). It is however
clear from the adjusted rho-square measures that the models based on all 16 CS perform much
better than the models based on only 8 CS. Interestingly, Model (ii) obtains a markedly higher
adjusted rho-square than Model (i). <strong>The</strong> fact that the model based on the last 8 CS obtains a higher
degree <strong>of</strong> estimation precision than the model based on the first 8 CS provides a first indication <strong>of</strong>
learning effects taking place.
<strong>The</strong> effect <strong>of</strong> repeating the choice set sequence is initially examined through a likelihood ratio test
for equality <strong>of</strong> model parameters (Louviere & Swait 1993). <strong>The</strong> test results are displayed in table 3.
<strong>The</strong> null hypothesis is that scale adjusted parameters are stable across the first 8 CS <strong>and</strong> the last 8
CS. First we look at the two models based on the first 8 CS <strong>and</strong> the last 8 CS, <strong>and</strong> a pooled model,
which is not corrected for the scaling effect. <strong>The</strong> test returns a test value at 1290, which clearly
suggests that the hypothesis <strong>of</strong> equal parameters in the two models can be rejected at the st<strong>and</strong>ard
5% level <strong>of</strong> statistical significance. <strong>The</strong> differences in parameter values between the two models
might be caused by differences in the underlying scale <strong>of</strong> utility <strong>and</strong> not due to differing preference
structures (Swait & Louviere 1993, Train 2003). Thus, we estimate a pooled model accounting for
different scales <strong>of</strong> utility across the two models <strong>and</strong> run the test again.
13

Table 3: Likelihood ratio tests for equality <strong>of</strong> model parameters.
Model
Likelihood Sum <strong>of</strong> sample 2 x (sum <strong>of</strong> sample A Critical values for chisquare
statistics at 5%
value A <strong>and</strong> B <strong>and</strong> B - pooled)
First 8 CS -2533.9
Last 8 CS -2414.6 -4948.6
Pooled -4303.5 -1290.13 18.31
Pooled with scaling -4299.7 -1297.81 16.92
This test also shows that the hypothesis <strong>of</strong> equal parameters in the two models can be rejected at the
5% level (with a test value at 1298), suggesting that the preference structure is clearly different in
the last 8 CS, even after an adjustment for differences in the error variance. Recall that the scale
parameter is the inverse <strong>of</strong> the st<strong>and</strong>ard deviation <strong>of</strong> the error term (Swait & Louviere 1993). <strong>The</strong>
scale factor <strong>of</strong> 1.14 identified in table 2 implies that the variance <strong>of</strong> the error term or “noise” in the
model based on the last 8 CS is 23% smaller than in model based on the first 8 CS 3 . <strong>The</strong> appearance
<strong>of</strong> smaller error variance in the model based on the last 8 CS indicates that respondents’ preferences
have settled <strong>and</strong> that a learning effect has taken place during the first 8 CS. This corresponds to the
comparison <strong>of</strong> model fit between the two models (i) <strong>and</strong> (ii), where the model based on the last 8
CS clearly provides a better fit to the data. This is equivalent to the findings by Holmes & Boyle
(2005).
<strong>The</strong> above tests are joint tests that demonstrate a change in the overall preference structure. In a
more detailed analysis <strong>of</strong> how the preferences change, we compare the relative size <strong>of</strong> the attribute
coefficients across sub-samples. In order to avoid the confounding <strong>of</strong> scale <strong>of</strong> the Gumbel error <strong>and</strong>
structure <strong>of</strong> utility across the two sub-samples used in Models (i) <strong>and</strong> (ii) (Hausman & Ruud 1987,
3 (1/1.14 2 ) = 0.7695, hence there is a 23% difference in the variance between the two models.
14

Scarpa et al. 2008, Swait & Louviere 1993), we focus on the marginal rate <strong>of</strong> substitution between
price <strong>and</strong> attributes, which is equivalent to the WTP for each attribute.
Table 4: WTP estimates, variance <strong>of</strong> WTP <strong>and</strong> t-values based on Models (i) <strong>and</strong> (ii).
Model (i): First 8 CS Model (ii): Last 8 CS
WTP Var(WTP) 1 WTP Var(WTP) 1 |t-value|
Campylobacter-free label 21.60 1.96 20.77 1.98 0.418
Salmonella-free label 16.53 2.01 21.54 2.65 2.318
Domestic produce 31.79 3.37 30.34 4.98 0.500
Organic production 12.11 4.11 17.32 5.28 1.703
ASC (Status quo) 3.79 12.57 7.26 11.12 0.713
Price
St<strong>and</strong>ard Deviation
Campylobacter-free label 1.99 15.97 4.99 20.75 0.495
Salmonella-free label 14.40 8.71 19.05 14.51 0.964
Domestic produce 18.03 10.10 23.54 13.04 1.146
Outdoor production 27.89 12.86 28.03 20.39 0.023
ASC (Status quo) 51.47 21.66 47.85 20.97 0.555
Note: <strong>The</strong> WTP estimates are calculated as marginal rates <strong>of</strong> substitution between the given non-price variable <strong>and</strong> the
price variable. <strong>The</strong> estimates are presented in DKK (DKK 10 ~ EUR 1.34). 1) <strong>The</strong> variation <strong>of</strong> the WTP is estimated
with the Delta method (Greene 2003).
<strong>The</strong> WTP estimates reported in table 4 reveals that the learning effect found when looking at the
error variance between the first 8 <strong>and</strong> the last 8 CS as well as the LR test for identical preferences
translates into some significant differences in the WTP estimates. However, only for the
Salmonella-free attribute do we see a significant change in terms <strong>of</strong> an increase in WTP going from
the first 8 CS to the last 8 CS. Relaxing the cut-<strong>of</strong>f level for statistical significance to a 0.10 level,
15

the results also show that the value <strong>of</strong> organic production increases significantly. Overall, this
suggests that the learning process makes respondents refine their stated preferences especially for
these two attributes whereas the others are not significantly affected. Importantly, this implies that
the policy advice obtained from the present survey is to some extent dependent on whether we take
learning into account or not.
Assuming that the preferences stated in the last 8 CS are the ones closest to the true preferences 4 ,
then basing policy advice on the first 8 CS will significantly under-estimate the value <strong>of</strong> two <strong>of</strong> the
quality attributes that we are interested in. While the majority <strong>of</strong> the previous studies considering
learning <strong>and</strong> fatigue effects in CE have focused on establishing whether these effects take place,
very few have actually considered the potential impacts on WTP estimates. Hanley et al. (2002)
find only weak evidence <strong>of</strong> learning <strong>and</strong> no significant impact on WTP estimates. This is however
not directly comparable to the internal test we provide here as the results reported in Hanley et al.
(2002) are based on an external test comparing two split samples with respondents answering four
<strong>and</strong> eight CS, respectively. Furthermore, the comparison <strong>of</strong> WTP in their paper represents only a
minor objective in their study <strong>and</strong>, hence, relatively limited information about the actual test is
reported. Another related paper is Johnson & Bingham (2001). In a health economics paper they
report findings from an internal test probably somewhat similar to ours where they compare VTP 5
estimates based on the beginning <strong>and</strong> the end <strong>of</strong> a SP question sequence, respectively. <strong>The</strong>y find no
significant differences. Unfortunately, they report only very limited information about the actual
experimental setup <strong>and</strong> test procedure, so it is difficult to assess the validity <strong>of</strong> their result. Based on
Hanley et al.’s (2002) <strong>and</strong> Johnson & Bingham’s (2001) results researchers might reach the
conclusion that WTP estimates will not be affected by learning <strong>and</strong> fatigue effects, <strong>and</strong>, hence,
4 Even though we do not have real market data available to support this assumption, it seems like a reasonable
assumption considering the fact that preferences have stabilized in accordance with the DPH.
5 VTP is an abbreviation <strong>of</strong> Value To Patients, a concept quite similar to WTP.
16

testing for impacts on WTP can be neglected when testing for learning effects. Our results which
are based on a rigorous experimental setup intended to test for learning <strong>and</strong> fatigue effects show that
testing for impacts on WTP is indeed an important extension to make, especially considering the
fact that policy advice in terms <strong>of</strong> WTP estimates is the major aim in most economic valuation
surveys.
5.1 Testing for dynamics <strong>of</strong> learning
<strong>The</strong> above tests show what takes place at one time during the sequence <strong>of</strong> choices, but it does not
show the dynamic changes which may take place as the respondents move through the choice tasks
as noted by Bateman et al. (2008b). In the following model we estimate separate scale factors for
CS 2 to 16 relative to the normalised scale factor in CS number 1, which set to one. This done in
order to capture the dynamic effect mentioned above.
Table 5: Estimation <strong>and</strong> comparison <strong>of</strong> CS specific scale factors in a RPL model.
CS (First 8 CS) 1 2 3 4 5 6 7 8
Scale factor 1 1.37 1.72 1.61 1.68 1.27 1.16 2.22
Std. Err. 0 0.17 0.20 0.21 0.23 0.17 0.23 0.23
|t-values| (relative to one) 3.53 4.57 3.64 2.79 2.45 0.08 5.36
CS (Last 8 CS) 9 10 11 12 13 14 15 16
Scale factor 1.54 1.65 1.91 1.80 1.60 1.46 0.98 1.98
Std. Err. 0.13 0.18 0.20 0.22 0.22 0.19 0.22 0.18
|t-values| (relative to one) 2.18 3.69 2.92 2.94 1.58 0.70 5.33 4.32
|t-value| (CS(X First 8 ) vs CS(X Last 8 )) 4.32 1.12 0.68 0.63 0.25 0.75 0.56 0.82
Note: <strong>The</strong> vertical aligment <strong>of</strong> CS in the table correspond to the CS that are identical.
As can be seen from table 5, almost all <strong>of</strong> the estimated scale factors for CS 2 to 16 differ
significantly from the scale factor in the first CS, which is fixed to one. Furthermore it shows that
17

they all are larger than one, <strong>and</strong> hence the error variance is smaller than one, suggestive <strong>of</strong> the
already established learning effect. What is also important to note is that it appears, from the bottom
row <strong>of</strong> the table, that the learning effect only takes place primarily in the first CS, <strong>and</strong> not as much
successively through the following CS as can be seen from the insignificant differences between the
scale factors. This is in contrast to the findings by Bateman et al. (2008b), where the most reliable
estimates are found during the first 4 <strong>and</strong> last 4 CS, whereas the middle 8 CS somewhat surprisingly
are the least reliable estimates. <strong>The</strong> results are however largely in accordance with Ladenburg &
Olsen (2008) who find evidence that preferences become stable after the first three CS, supporting
the notion <strong>of</strong> learning effects taking place.
6. Conclusion
This paper addresses the issues <strong>of</strong> how learning <strong>and</strong> fatigue effects related to the repeated choices
potentially affect respondents’ stated preferences in a Choice Experiment setting. <strong>The</strong> main focus is
on changes in preference structure, error variance <strong>and</strong> willingness-to-pay. To the best <strong>of</strong> the
authors’ knowledge this paper is the first to provide an internal test across all these three measures
in one study. In a case study concerning food quality attributes in chicken breast filet, the results
show that a learning effect is indeed present in a sequence <strong>of</strong> 16 choice sets, whereas there is no
evidence <strong>of</strong> fatigue effects. This conclusion is based on an observed reduction in the error variance
for the last 8 choice sets relative to the first 8 choice sets – a result implying that preferences have
become relatively more stable in the last half <strong>of</strong> the choice set sequence which is in accordance with
the Discovered Preference Hypothesis <strong>and</strong> the concept <strong>of</strong> learning. An analysis incorporating choice
set specific scale factors reveals that the main part <strong>of</strong> the observed learning process takes place in
the first choice set. <strong>The</strong> overall preference structure is found to differ significantly between the two
sequences <strong>of</strong> choice sets. This suggests that the learning effect leads to changes in overall
18

preference structure. Turning to a comparison <strong>of</strong> attribute WTP estimates based on the first <strong>and</strong> the
last half <strong>of</strong> the choice set sequence, the results indicate that the change in overall preference
structure is mainly driven by respondents refining <strong>and</strong> significantly adjusting their stated values
upwardly for two <strong>of</strong> the four quality attributes considered in the survey. <strong>The</strong> implication is that if
the main objective <strong>of</strong> a Choice Experiment survey is to obtain policy advice in terms <strong>of</strong> WTP, then
allowing for learning effects can be essential. Of course, generalisations based on a single survey
are inappropriate, so more research is clearly needed. Specifically, bearing in mind that most <strong>of</strong>ten
the purpose <strong>of</strong> economic valuation is to obtain WTP estimates, more investigations considering the
impact <strong>of</strong> learning on WTP estimates are necessary to further assess the importance <strong>of</strong> allowing
respondents to engage in learning prior to the actual preference eliciting valuation questions.
19

References
Ariely, D.; Loewenstein, G. & Prelec, D. (2003): Coherent arbitrariness: Stable dem<strong>and</strong> curves
without stable preferences, <strong>The</strong> Quarterly Journal <strong>of</strong> Economics, vol. 118, no. 1, pp. 73-105.
Bateman, I. J., Burgess, D., Hutchinson, W. G., & Matthews, D. I. (2008a): <strong>Learning</strong> design
contingent valuation (LDCV): NOAA guidelines, preference learning <strong>and</strong> coherent arbitrariness,
Journal <strong>of</strong> Environmental Economics <strong>and</strong> Management, vol. 55, no. 2, pp. 127-141
Bateman, I.; Carson, R. T.; Day, B.; Dupont, D.; Louviere, J.; Morimoto, S.; Scarpa, R. &
Wang, P. (2008b): Choice Set Awareness <strong>and</strong> Ordering <strong>Effect</strong>s in Discrete Choice Experiments
CSERGE Working Paper EDM 08-01.
Bergstrom & Stoll (1989): Application <strong>of</strong> Experimental Economics Concepts <strong>and</strong> Precepts to
CVM Field Survey Procedures. Western Journal <strong>of</strong> Agricultural Economics, vol. 14, no. 1 (July),
pp. 98-109.
Bjornstad, D., Cummings, R., & Osborne, L. (1997): A <strong>Learning</strong> Design for Reducing
Hypothetical Bias in the Contingent Valuation Method, Environmental <strong>and</strong> Resource Economics,
vol. 10, no. 3, pp. 207-221
Bech, M. & Gyrd-Hansen, D. (2005): <strong>Effect</strong> coding in discrete choice experiments, Health
Economics, vol. 14, pp. 1079-1083.
Bierlaire, M. (2003): BIOGEME: A free package for the estimation <strong>of</strong> discrete choice models,
Proceedings <strong>of</strong> the 3rd Swiss Transportation Research Conference, Ascona, Switzerl<strong>and</strong>.
Braga, J. & Starmer, C. (2005): Preference Anomalies, Preference Elicitation <strong>and</strong> the Discovered
Preference Hypothesis, Environmental <strong>and</strong> Resource Economics, vol. 32, no. 1, pp. 55-89.
Brouwer, R.; Dekker, T.; Rolfe, J. & Windle, J. (2009): Choice Certainty <strong>and</strong> Consistency in
Repeated Choice Experiments, Environmental <strong>and</strong> Resource Economics, vol. DOI 10.1007/s10640-
009-9337-x.
Carlsson, F. & Martinsson, P. (2001): Do Hypothetical <strong>and</strong> Actual Marginal Willingness to Pay
Differ in Choice Experiments, Journal <strong>of</strong> Environmental Economics <strong>and</strong> Management, vol. 41, pp.
179-192.
Carlsson, F. & Martinsson, P. (2003): Design techniques for stated preference methods in health
economics, Health Economics, vol. 12, no. 4, pp. 281-294.
Cherry, T. L.; Crocker, T. D. & Shogren, J. F. (2003): Rationality spillovers, Journal <strong>of</strong>
Environmental Economics <strong>and</strong> Management, vol. 45, no. 1, pp. 63-84.
Cummings, R. G.; Brookshire, D. S. & Schulze, W. D. (1986): Valuing Environmental Goods:
An Assessment <strong>of</strong> the Contingent Valuation Method Rowman <strong>and</strong> Allenheld, Totowa, NJ.
20

Dellaert, B. G. C.; Brazell, J. D. & Louviere, J. (1999): <strong>The</strong> <strong>Effect</strong> <strong>of</strong> Attribute Variation on
Consumer Choice Consistency, Marketing Letters, vol. 10, no. 2, pp. 139-147.
DeShazo, J. R. & Fermo, G. (2002): Designing Choice Sets for Stated Preference Methods: <strong>The</strong>
<strong>Effect</strong>s <strong>of</strong> Complexity on Choice Consistency, Journal <strong>of</strong> Environmental Economics <strong>and</strong>
Management, vol. 44, pp. 123-143.
Gravelle, H. & Rees, R. (1992): Microeconomics 2nd edn, Pearson Education Ltd., Harlow,
Engl<strong>and</strong>.
Greene, W. H. (2003): Econometric Analysis 5 edn, Prentice-Hall International, Inc., New York.
Hanley, N.; Wright, R. E. & Koop, G. (2002): Modelling Recreation Dem<strong>and</strong> Using Choice
Experiments: Climbing in Scotl<strong>and</strong>, Environmental <strong>and</strong> Resource Economics, vol. 22, pp. 449-466.
Hanley, N. & Shogren, J. F. (2005): Is Cost−Benefit Analysis Anomaly-Pro<strong>of</strong>, Environmental
<strong>and</strong> Resource Economics, vol. 32, no. 1, pp. 13-24
Hausman, J. A. & Ruud, P. A. (1987): Specifying <strong>and</strong> testing econometric models for rankordered
data, Journal <strong>of</strong> Econometrics, vol. 34, no. 1-2, pp. 83-104.
Hole, A. R., (2007): A comparison <strong>of</strong> approaches to estimating confidence intervals for willingness
to pay measures. Health Economics, vol. 16, pp. 827-840.
Holmes, T. & Adamowicz, W. L. (2003): Attribute Based Methods. In A Primer on the Economic
Valuation <strong>of</strong> the Environment Kluwer.
Holmes, T. & Boyle, K. J. (2005): <strong>Learning</strong> <strong>and</strong> Context-Dependence in Sequential, Attribute-
Based, Stated-Preference Valuation Questions, L<strong>and</strong> Economics, vol. 81, no. 1, pp. 114-126.
Johnson, F. R. & Bingham, M. F. (2001): Evaluating the validity <strong>of</strong> stated-preference estimates <strong>of</strong>
health values, Swiss Journal <strong>of</strong> Economics <strong>and</strong> Statistics (SJES), vol. 137, pp. 49-63
Kahneman, D. & Tversky, A. (1979): Prospect <strong>The</strong>ory: An analysis <strong>of</strong> Decision under Risk,
Econometrica, vol. 47, no. 2, pp. 263-291
Kontoleon, A. & Yabe, M. (2003): Assessing the Impacts <strong>of</strong> Alternative 'Opt-out' Formats in
Choice Experiment Studies: Consumer Preferences for Genetically Modified Content <strong>and</strong>
Production Information in Food, Journal <strong>of</strong> Agricultural Policy Research, vol. 5, pp. 1-43.
Ladenburg, J. & Olsen, S. B. (2008): Gender-specific starting point bias in choice experiments:
Evidence from an empirical study, Journal <strong>of</strong> Environmental Economics <strong>and</strong> Management no.
Forthcoming.
Lagerkvist, C.-J.; Carlsson, F. & Viske, D. (2006): Swedish Consumer Preferences for Animal
Welfare <strong>and</strong> Biotech: A Choice Experiment, AgBioForum, vol. 9, no. 1, pp. 51-58.
Lancaster, K. J. (1966): A New Approach to Consumer <strong>The</strong>ory, <strong>The</strong> Journal <strong>of</strong> Political
Economy, vol. 74 , no. 2, pp. 132-157.
21

List, J. A. (2003): Does Market Experience Eliminate Market Anomalies, Quaterly journal <strong>of</strong>
Economics, vol. 118, no. 1, pp. 41-71.
Luce, R. D. (1959): Individual choice behaviour Wiley, New York.
McFadden, D. (1974): Conditional Logit Analysis <strong>of</strong> Qualitative Choice Behavior In: Frontiers in
Econometrics, P. Zarembka, ed., (eds.) (1974): Academic, New York, pp. 105-142.
Phillips, K. A.; Johnson, F. R. & Maddala, T. (2002): Measuring What People Value: A
Comparison <strong>of</strong> "Attitude" <strong>and</strong> "Preference" Surveys, Health Services Research, vol. 37, no. 6, pp.
1659-1679.
Plott, C. R. (1996): Rational Individual Behaviour in Markets <strong>and</strong> Social Choice Processes: <strong>The</strong>
Discovered Preference Hypothesis In: Rational Foundations <strong>of</strong> Economic Behaviour, K. Arrow et
al., eds., (eds.) (1996): Macmillan Press Ltd., London.
Rose, J. M.; Bliemer, M. C. J.; Hensher, D. A. & Collins, A. T. (2008): Designing efficient
stated choice experiments in the presence <strong>of</strong> reference alternatives, Transportation Research Part B,
vol. 42, pp. 395-406.
Ruby, M. C.; Johnson, F. R. & Mathews, K. E. (1998): Just say no: Opt-out alternatives <strong>and</strong>
anglers' stated preferences, TER General Working Paper No.T-9801R.
Savage, S. J. & Waldman, D. (2008): <strong>Learning</strong> <strong>and</strong> <strong>Fatigue</strong> During Choice Experiments: A
Comparison <strong>of</strong> Online <strong>and</strong> Mail Survey Modes, Journal <strong>of</strong> Applied Econometrics, vol. 23, pp. 351-
371.
Scarpa, R.; Thiene, M. & Train, K. (2008): Utility in Willingness to Pay Space: A Tool to
Address Confounding R<strong>and</strong>om Scale <strong>Effect</strong>s in Destination Choice to the Alps, American Journal
<strong>of</strong> Agricultural Economics, vol. 90, no. 4, pp. 994-1010.
Slovic, P. (1995): <strong>The</strong> Construction <strong>of</strong> Preferences, American Psychologist, vol. 50, pp. 364-371
Swait, J. & Louviere, J. (1993): <strong>The</strong> Role <strong>of</strong> the Scale Paramter in the Estimation <strong>and</strong> Comparison
<strong>of</strong> Multinomial Logit Models, Journal <strong>of</strong> Marketing Research pp. 305-314.
Train, K. (2003): Discrete Choice Methods with Simulation Cambridge University Press,
Cambridge.
Tversky, A. & Kahneman, D. (1986): Rational Choice <strong>and</strong> the Framing <strong>of</strong> Decisions., <strong>The</strong>
Journal <strong>of</strong> Business, vol. 59, no. 4, part 2, p. s251-s278
22