Wireless Network Design: Optimization Models and Solution ...

338 Nitin Salodkar and Abhay Karandikar
Note that the function c(·,·,u) is a strictly convex function of u (as the power required
to transmit u packets is a strictly convex function of u). The unconstrained
problem is to determine an optimal stationary policy µ ∗ (·) that minimizes
L(µ,λ) = E µ�
�
c(λ,Sn, µ(Sn)) , (14.51)
for a particular value of λ called the Lagrange Multiplier (LM). L(·,·) is called the
Lagrangian.
Let p(s,u,s ′ ) be the probability of reaching state s ′ upon taking action u in state
s. Let V (s) denote the optimal value function (i.e. expected cost) for a state s. The
following dynamic programming equation provides the necessary condition for op-
timality of the policy.
V (s) = min
u
�
c(λ,s,u) − β +∑ s ′
p(s,u,s ′ )V (s ′ �
) , s ′ ∈ S, (14.52)
where β ∈ R is uniquely characterized as the corresponding optimal cost (power)
per stage. If we impose V (s0 ) = 0 for any pre-designated state s0 ∈ S, then V is
unique. Furthermore, an optimal policy µ ∗ must satisfy,
support(µ ∗ �
(·|s)) ⊆ argmin c(λ,s,u) − β +∑ p(s,u,s
s ′
′ )V (s ′ �
) ∀s ∈ S.(14.53)
It follows that the constrained problem has a stationary optimal policy which is also
optimal for the unconstrained problem considered in (14.51) for a particular choice
of λ = λ ∗ (say). In general, this optimal policy may be a randomized policy. In fact,
it can be shown that the optimal stationary policy is deterministic for all states but
at most one s, i.e., there exists a unique u ∗ (s) such that µ ∗ (u ∗ (s)|s) = 1 and u ∗ is the
solution to the following equation,
u ∗ (s) = argmin
�
c(λ ∗ ,s,u) − β +∑ s ′
p(s,u,s ′ )V (s ′ �
)
∀s ∈ S. (14.54)
Furthermore, for the single (if any) state s for which this fails, µ(·|s) is supported on
exactly two points. The optimal average cost β gives the minimum power consumed
¯P ∗ subject to the specified queue length constraint δ. Moreover, the following saddle
point condition holds:
L(µ ∗ ,λ) ≤ L(µ ∗ ,λ ∗ ) ≤ L(µ,λ ∗ ). (14.55)
14.5.1.2 Structural Properties of the Optimal Policy
Before we discuss the computational issues in determining the optimal policy
through dynamic programming equation (14.52), we discuss some structural properties
of the optimal policy. The result for i.i.d. arrival and channel state processes
can be stated as: