Bernoulli bandits with covariates - Department of Statistics ...

More documents

Recommendations

Info

However, if a a < a', the integrand in (2,4) is nonnegative, and if a 9 a c a' fajls, the integrand is zero.//// An immediate consequence of Lemma 2.1 and Theorem 2.3 is the following. i Theorem 2.4: For a1 1 n,R,c,t, and i, each of Wn(aSR,c), W,( t, oSF!,c), and ~,[t,o~R,c) are nonincreasing in s and each of Wn(mSR,c), wA(t,$,~.c), and W (t,$,R,c) are nondecreasing in s. n Theorem 2.4 and parts (i)- (iv) of Proposf tion 2.1 below are related to the "informationn obtained by a particular pull. A success observed on arm 1 when s is large is relatively uninformative, since lim ~ ( s ) = 1 for any a. However, a S'=" success observed when s is large in magnitude and negative is potentially quite informative: it suggests that a itself I s large, The reverse sf tuation arises when we observe a failure on arm 1. Bounds on the "information" available in a pull are given by parts (ii) and (iii) of the Proposition. Part (v) of Proposition 2.1 suggests that as the current covariate value grows large fn magnitude, we become tndif ferent to the choice of arms for the next pull, do,R ,a dh+J -a Proposition 2.1: Define cr_-R by 7 = 7 and R by aR = - Ee ~ e ' - ~ For all n, for = 1 and 2, for all R and for a17 c, (i) lirn W (0,R.c) = lim wn(rn,R,c) = w,(R,c) S+do s+-' (ii) lim M,( oSR,c) = W,( q,R,c) s+-m
1 2 (v) l i m Wn(t.R,c) = Tim W,(t,R,c) t+s- t+fm Proof: We prove some parts of the proposition. The remainder follow similarly. First we prove the flrst half of part (iv), using induction. The result is easy when n = I. Note that By the induction hypothesis, l im W,_l( S show that Tim ECp(t)l~~R1=Ep(t). SO S +- O~O~R,C)=W,-~( otR, C) . A1 so it is easy to To prove part (i), use part (lv), noting that lim W,(o,R,c) S -bm = lim j Wn(t.o,R,c)dt s+* The second equal i ty above fol 1 ows from the dominated convergence theorem and equatlon (2.1). To prove part (v), note that
Page 1 and 2: . DEPARTMENT OF STATISTICS --I-----
Page 3 and 4: 1, Introduction A bandit problem in
Page 5 and 6: observed val ues of the covaria te.
Page 7 and 8: t, then we are faced with a (t,n-l,
Page 9 and 10: In words, W,(R,c) is bounded above
Page 11 and 12: Proof: Suppose to the contrary, tha
Page 13: Since R' is to the rlght of R, E(p(
Page 17 and 18: Remark: If R is degenerate at c, th
Page 19 and 20: If Wl( t,R,c) = Ep(t), then we note
Page 21 and 22: 2 2 1 if Hz(-1,R) = W2(-l,R), then
Page 23 and 24: Remark: The Rypothesfs of the propo
Page 25 and 26: REFERENCES Berry, D.A. (19721, A be

Bernoulli bandits with covariates - Department of Statistics ...

Create successful ePaper yourself

Delete template?

Save as template?