Wireless Network Design: Optimization Models and Solution ...

14 Cross Layer Scheduling in Wireless Networks 339
Theorem 14.1. The optimal policy µ ∗ (s) = µ ∗ (q,x) is non decreasing in channel
state x and non-decreasing in queue length q.
The proof of this theorem is based on supermodularity and increasing differences
properties of the value function. These properties establish the monotonicity of the
optimal policy in channel state and buffer occupancy.
The structural results imply that the optimal decision is to transmit a certain number
of packets in a given slot where this number is an increasing function of the
current queue length and channel state. Thus for a fixed channel state, the greater
the queue length, the more will be the number of packets that will be transmitted.
Similarly, for a fixed queue length, the better the channel, the more will be the number
of packet transmissions. Thus, the optimal policy always transmits at the highest
rate when channel condition is the most favorable and queue length is the largest.
Similar structural results also hold for Markovian arrivals and Markovian channel
state and continuous state processes. However, for these general state spaces,
the dynamic programming equation (14.52) for the long run average cost problem
requires to be rigorously justified. See bibliographic notes and [1] for a discussion
of this.
14.5.1.3 Learning Algorithm for Scheduling
In this section, we discuss the computation of optimal policy discussed in the preceding
section. For a fixed λ, the Relative Value Iteration Algorithm (RVIA) can
be used for solving the dynamic programming equation in an iterative fashion. The
average cost RVIA for determining the value function such that (14.52) is satisfied
can be written as:
Vn+1(s) = min
u∈U(s) [c(λ,s,u) +∑ s ′
p(s,u,s ′ )Vn(s ′ )] −Vn(s 0 ), (14.56)
where s,s ′ ,s 0 ∈ S and s 0 is any fixed state. Vn(·) is an estimate of the value function
after n iterations for a fixed LM λ.
Due to the large size of the state space S, solution of this algorithm is computationally
expensive. One approach for addressing this issue is to utilize the structural
properties of the optimal policy outlined above to develop efficient heuristics that
are computationally less expensive and hence can be implemented in a practical system.
In [54], one such approach has been discussed. Techniques based on function
approximation can also be employed for dimensionality reduction. The structural
results of the policy may be used to choose the basis functions in function approximation.
Another technique is to use ‘primal-dual’ type approach with conventional iteration
schemes for the value function (primal) and Lagrange multiplier (dual) [13].
However, even with this approach (and for that matter, with all other approaches
discussed above for computing (14.56)), one major issue is that it requires the
knowledge of transition probabilities p(s,u,s ′ ). Note that p(s,u,s ′ ) depends upon