Relax and Randomize: From Value to Algorithms

However this argmin calculation is identical to the one in the proof of Proposition 4 (with C = 1 and
T − t = 0) and the solution is given by
Thus we conclude the proof.
f ∗ t = f t =
− ∑ t−1
i=1 x i + 4 √ 2 ∑ T i=t+1 x i
√
∥− ∑ t−1
i=1 x i + 4 √ 2 ∑ T i=t+1 ɛ i x i ∥ 2 + 1 2
Proof of Lemma 11. We shall start by showing that the relaxation is admissible for the game where
we pick prediction ŷ t and the adversary then directly picks the gradient ∂l(ŷ t , y t ). To this end note
that
inf
ŷ t
sup
∂l(ŷ t,y t)
{∂l(ŷ t , y t ) ⋅ ŷ t + Rel T (F∣∂l(ŷ 1 , y 1 ), . . . , ∂l(ŷ t , y t ))}
= inf
ŷ t
≤ inf
ŷ t
sup
∂l(ŷ t,y t)
sup
r t∈[−L,L]
{∂l(ŷ t , y t ) ⋅ ŷ t + E
ɛ
{r t ⋅ ŷ t + E
ɛ
[sup
f∈F
2L
[sup
f∈F
T
∑
i=t+1
2L
T
∑
i=t+1
t
ɛ i f[t] − ∑ ∂l(ŷ i , y i ) ⋅ f[i]]}
i=1
ɛ i f[t] − L t−1 (f) − r t ⋅ f[t]]}
Let us use the notation L t−1 (f) = ∑ t−1
i=1 ∂l(ŷ i , y i ) ⋅ f[i] for the present proof. The supremum over
r t ∈ [−L, L] is achieved at the endpoints since the expression is convex in r t . Therefore, the last
expression is equal to
inf
ŷ t
= inf
ŷ t
sup
r t∈{−L,L}
sup
p t∈∆({−L,L})
= sup inf
p t∈∆({−L,L}) ŷ t
{r t ⋅ ŷ t + E ɛ sup
f∈F
[2L
T
∑
i=t+1
E [r t ⋅ ŷ t + E ɛ sup [2L
r t∼p t
f∈F
E [r t ⋅ ŷ t + E ɛ sup [2L
r t∼p t
f∈F
ɛ i f[t] − L t−1 (f) − r t ⋅ f[t]]}
T
∑
i=t+1
T
∑
i=t+1
ɛ i f[t] − L t−1 (f) − r t ⋅ f[t]]]
ɛ i f[t] − L t−1 (f) − r t ⋅ f[t]]]
where the last step is due to the minimax theorem. The last quantity is equal to
sup
p t∈∆({−L,L})
≤
≤
sup
p t∈∆({−L,L})
sup
p t∈∆({−L,L})
= sup
p t∈∆({−L,L})
E [ E
ɛ
[inf E [r t ] ⋅ ŷ t + sup (2L
r t∼p t r t∼p t
ŷ t
E [ E [sup (2L
ɛ r t∼p t
f∈F
E [E ɛ sup [2L
r t,r t ′ ∼pt f∈F
E [E ɛ sup [2L
r t,r t ′ ∼pt f∈F
T
∑
i=t+1
T
∑
i=t+1
T
∑
i=t+1
f∈F
T
∑
i=t+1
ɛ i f[t] − L t−1 (f) − r t ⋅ f[t])]]
ɛ i f[t] − L t−1 (f) + ( E
r t∼p t
[r t ] − r t ) ⋅ f[t])]]
ɛ i f[t] − L t−1 (f) + (r ′ t − r t ) ⋅ f[t]]]
ɛ i f[t] − L t−1 (f) + ɛ t (r ′ t − r t ) ⋅ f[t]]]
By passing to the worst-case choice of r t , r ′ t (which is achieved at the endpoints because of convexity),
we obtain a further upper bound
sup E ɛ sup [2L
r t,r t ′ ∈{L,−L} f∈F
≤
sup
r t∈{L,−L}
E ɛ sup
f∈F
[2L
T
∑
i=t+1
T
∑
i=t+1
ɛ i f[t] − L t−1 (f) + ɛ t (r ′ t − r t ) ⋅ f[t]]
ɛ i f[t] − L t−1 (f) + 2ɛ t r t ⋅ f[t]]
= sup E ɛ sup [2L ∑ ɛ i f[t] − L t−1 (f)]
r t∈{L,−L} f∈F
T
i=t
= Rel T (F∣∂l(ŷ 1 , y 1 ), . . . , ∂l(ŷ t−1 , y t−1 ))
22