Wireless Network Design: Optimization Models and Solution ...

14 Cross Layer Scheduling in Wireless Networks 337
Let cp(Sn,Un) = P(Xn,Un) be the ‘immediate’ cost in terms of power required
in transmitting Un packets when the state is Sn. Let cq(Sn,Un) ∆ = Qn denote the
‘immediate’ cost due to buffering. Let µ = {µ1, µ2,...} be the control policy. We
would like to determine the policy µ that minimizes
subject to
¯P = limsup
N→∞
¯Q = limsup
N→∞
1
N E
1
N E
N
∑
n=1
N
∑
n=1
cp(Sn,Un), (14.44)
cq(Sn,Un) ≤ ¯ δ. (14.45)
It can be easily argued that this is a CMDP with average cost and finite state and
action spaces. For average cost CMDP with finite state and action space, it is well
known that a optimal stationary randomized policy exists. Let ¯P ∗ denote the optimal
cost i.e.,
¯P ∗ = min
µ
¯P µ , (14.46)
where ¯P µ is the cost (14.44) under policy µ.
Let µ(·|s) : s ∈ F be the probability measure on U. For each state s, µ(·|s) specifies
the distribution with which the control in that state is applied. We assume that
{Sn} is an ergodic Markov chain under such policies and thus has a unique stationary
distribution ρ µ .
Let E µ denote the expectation with respect to (w.r.t.) ρ µ . Under a randomized
policy µ, the costs in (14.44) can be expressed as:
and,
¯P µ ∆ = E µ �
cp(Sn, µ(Sn))
�
= ∑ρ u,s
µ (s)µ(u|s)cp(s, µ(s)), (14.47)
¯Q µ ∆ µ
= E �
�
cq(Sn, µ(Sn)) = ∑ρ u,s
µ (s)µ(u|s)cq(s, µ(s)), (14.48)
respectively. Then the scheduler objective can be stated as:
Minimize ¯P µ subject to ¯Q µ ≤ δ. (14.49)
We now demonstrate that the optimal average cost and policy can be determined
using an unconstrained Markov Decision Process (MDP) problem and Lagrangian
approach.
14.5.1.1 The Lagrangian Approach
Let λ ≥ 0 be a real number. Define c : R + × S × U → R as follows,
c(λ,s,u) = cp(s,u) + λ(cq(s,u) − δ). (14.50)