Belief-Assisted Pricing for Dynamic Spectrum Allocation in Wireless ...

sig.umd.edu

Belief-Assisted Pricing for Dynamic Spectrum Allocation in Wireless ...

This full text paper was peer reviewed at the direction of IEEE Communications Society subject matter experts for publication in the IEEE SECON 2006 proceedings.Belief-Assisted Pricing for Dynamic Spectrum Allocationin Wireless Networks with Selfish UsersZhu Ji and K. J. Ray LiuElectrical and Computer Engineering Department and Institute for Systems ResearchUniversity of Maryland, College Park, MD 20742email: zhuji, kjrliu@umd.eduAbstract— In order to fully utilize the scarce spectrumresources, with the development of cognitive radio technologies,dynamic spectrum allocation becomes a promisingapproach to increase the efficiency of spectrum usage.In this paper, we consider the spectrum allocation in wirelessnetworks with multiple selfish legacy spectrum holdersand unlicensed users as multi-stage dynamic games. Abelief-assisted dynamic pricing approach is proposed tooptimize overall spectrum efficiency while keeping theparticipating incentives of the users based on doubleauction rules. Moreover, considering the budget constraintsof the unlicensed users, a dynamic programming approachis further developed to optimize the spectrum allocationover time. The simulation results show that our proposedscheme not only approaches optimal outcomes with lowoverhead compared to general continuous double auctionmechanisms, but also fully exploits the time diversity ofspectrum resources when budget constraints exist.I. INTRODUCTIONRecently, regulatory bodies like the Federal CommunicationsCommission (FCC) in the United States arerecognizing that current static spectrum allocation canbe very inefficient considering the bandwidth demandsmay vary highly along the time dimension or the spacedimension. In order to fully utilize the scarce spectrumresources, with the development of cognitive radio technologies,dynamic spectrum access becomes a promisingapproach to increase the efficiency of spectrum usage,which allows unlicensed wireless users to dynamicallyaccess the licensed bands from legacy spectrum holdersbased on leasing agreements.Cognitive radio technologies have the potential toprovide the wireless devices with various capabilities,such as frequency agility, adaptive modulation, transmitpower control and localization. The advances ofcognitive radio technologies make more efficient andintensive spectrum access possible on a negotiated or anopportunistic basis. The FCC began to consider moreflexible and comprehensive use of available spectrumin [1], [2]. Then, great attentions have been drawn toexplore the open spectrum systems [3], [4] for dynamicspectrum sharing. Traditionally, network-wide spectrumassignment is carried out by a central server, namely,spectrum broker [5], [6]. Recently, distributed spectrumallocation approaches [7], [8] have been well studied toenable efficient spectrum sharing only based on localobservations. In [7], local bargaining mechanism wasintroduced to distributively optimize the efficiency ofspectrum allocation and maintain bargaining fairnessamong secondary users. In [8], the authors proposeda repeated game approach to increase the achievablerate region of spectrum sharing, in which the spectrumsharing strategy can be enforced by the Nash Equilibriumof dynamic games. Moreover, efficient spectrumsharing has also been studied from a practical point ofview, such as in [9] and [10], which analyzed spectrumsharing games for WiFi networks and cellular networks,respectively.Although the existing dynamic spectrum accessschemes have achieved some success on enhancing thespectrum efficiency and distributive design, most of themfocus on efficient spectrum allocation given fixed topologiesand cannot quickly adapt to the dynamics of wirelessnetworks due to node mobility, channel variations orvarying wireless traffic. Furthermore, existing cognitivespectrum sharing approaches generally assume that thenetwork users will act cooperatively to maximize theoverall system performance, which is a reasonable assumptionfor traditional emergency or military situations.However, with the emerging applications of mobile adhoc networks envisioned in civilian usage, the users maynot serve a common goal or belong to a single authority,which requires that the network functions can be carriedout in a self-organized way to combat the selfish behaviors.In dynamic spectrum allocation scenarios, theusers’ selfishness causes more challenges for efficientmechanism design, such as incentive-stimulation andprice of anarchy [9], [11]. Therefore, novel spectrumallocation approaches need to be developed consideringthe dynamic nature of wireless networks and users’selfish behaviors.Considering a general network scenario in whichmultiple primary users (legacy spectrum holders) and1-4244-0626-9/06/$20.00 (c) 2006 IEEE.119


secondary users (unlicensed users) coexist, primary usersattempt to sell unused spectrum resources to secondaryusers for monetary gains while secondary users try toacquire spectrum usage permissions from primary usersto achieve certain communication goals, which generallyintroduces reward payoffs for them. In order to solvethe above issues, we consider the spectrum sharingas multistage dynamic games and propose a dynamicpricing approach to optimize the overall spectrum efficiency,meanwhile, keeping the participating incentivesof the users based on double-auction rules and copingwith the budget constraints by dynamic programming.The main contributions of this paper are multi-fold.First, by modeling the spectrum sharing as a dynamicpricing game, we are able to quickly and accuratelycoordinate the spectrum allocation among primary andsecondary users through a trading process to maximizethe payoffs of both primary and secondary users. Further,we develop a belief system to assist greedy users updatetheir strategies adaptive to the spectrum demand and supplychanges, which not only approaches the theoreticaloptimal outcomes of the spectrum allocation problem butalso substantially decreases the pricing overhead due tofrequent bid/ask updates and message exchange. Third,by considering the budget constraints of the secondaryusers, the proposed dynamic pricing approach is able tofurther exploit the time diversity of spectrum resources.The reminder of this paper is organized as follows:The system model of dynamic spectrum allocation isdescribed in Section II. In Section III, we formulatethe spectrum allocation as pricing games based on thesystem model. In Section IV, the belief-based dynamicpricing approach is proposed for the optimal spectrumallocation. The simulation studies are provided in SectionV. Finally, Section VI concludes this paper.II. SYSTEM MODELWe consider the wireless networks where multipleprimary users and secondary users operate simultaneouslyin a wireless network, which may represent variousnetwork scenarios. For instance, the primary users can bethe spectrum broker connected to the core network andthe secondary users are the base stations equipped withcognitive radio technologies; or the primary users arethe access points of a mesh network and the secondaryusers are the mobile devices. On one hand, every primaryuser has the license of using a certain spectrum range,which can be divided into non-overlapping orthogonalchannels. Considering that the authorized spectrum ofprimary users may not be fully utilized over time, theyprefer to lease the unused channels to the secondaryusers for monetary gains. On the other hand, since theunlicensed spectrums become more and more crowded,the secondary users may try to lease some unusedchannels from primary users for more communicationgains by providing leasing payments.In our system model, we assume all users are selfishand rational, that is, their objectives are to maximizetheir own payoffs, not to cause damage to other users.However, users are allowed to cheat whenever theybelieve cheating behaviors can help them to increasetheir payoffs. Generally speaking, in order to acquire thespectrum licenses from regulatory bodies such as FCC,the primary users have certain operating costs. Withregard to secondary users, in order to have the rewardsof achieving certain communication goals, they want toutilize more spectrum resources. The selfishness of bothprimary and secondary users will prevent them from revealingtheir private information such as acquisition costsor reward payoffs, which makes traditional spectrumallocation approaches not applicable under this scenario.Therefore, novel spectrum allocation approaches needto be developed which not only optimize the spectrumefficiency but also extract the private information fromthe selfish parties through certain mechanisms to assistthe optimization of spectrum allocation.Specifically, we consider the collection of the availablespectrums from all primary users as a spectrum pool,which totally consists of N non-overlapping channels.Assume there are J primary users and K secondaryusers, indicated by the set P = {p 1 ,p 2 , ..., p J } andS = {s 1 ,s 2 , ..., s K }, respectively. We represent thechannels authorized to primary user p i using a vectorA i = {a j i } j∈{1,2,...,n i}, wherea j irepresents the channelindex in the spectrum pool and n i is the total number ofchannels which belong to user p i .DefineA as the set ofall the channels in the spectrum pool. Moreover, denotethe acquisition costs of user p i ’s channels as the vectorC i = {c aj ii} j∈{1,2,...,ni}, where the jth element representsthe acquisition cost of the jth channel in A i . Forsimplicity, we write c aj iias c j i . As for secondary user s i,we define her/his payoff vector as V i = {v j i } j∈{1,2,...,N},where the jth element is the reward payoff if this usersuccessfully leases the jth channel in the spectrum pool.III. PRICING GAME MODELIn this paper, we model the dynamic spectrum allocationproblem as a pricing game to study the interactionsamong the players, i.e., the primary and secondary users.Based on the discussion in the previous section, we are120


able to have the payoff functions of the players in ourdynamic game. Specifically, if primary user p i reachesagreements of leasing all or part of her/his channels tosecondary users, the payoff function of this primary usercan be written as follows.U pi (φ Ai ,α Aii)=∑n ij=1(φ aji− c j i )αaj ii, (1)where φ Ai = {φ aj} i j∈{1,2,...,ni} and φ aj is the paymentithat user p i obtains from the secondary user by leasingthe channel a j iin the spectrum pool. Note that α Aii={α aj ii} j∈{1,2,...,ni} and α aj ii∈ {0, 1} which indicates ifthe jth channel of user p i has been allocated to asecondary user or not. For simplicity, we denote α aj iiasα j i . Similarly, the payoff function of secondary user s ican be modeled as follows.U si (φ A ,β A i )=N∑(v j i − φ j)β j i , (2)j=1where φ A = {φ j } j∈{1,2,...,N} , βiA = {β j i } j∈{1,2,...,N}.Note that β j i∈ {0, 1} illustrates if secondary user s isuccessfully leases the jth channel in the spectrum poolor not. Hence, the strategies of the primary users andsecondary users are actually defined by α Aiiand βi A,respectively.Since the players may have conflict interests witheach other, our dynamic spectrum sharing game canbe modeled as a multi-stage non-cooperation game. Tobe specific, from the primary users’ point of view,they want to earn the payments by leasing the unusedchannels which not only cover their spectrum acquisitioncosts but also gain as much extra payments as possible;from the secondary users’ point of view, they aim toaccomplish their communication goals by providing theleast possible payments to lease the channels; whilefrom the network designers’ point of view, they attemptto maximize the network performance, which in ourcase is the spectrum efficiency. Therefore, the spectrumusers involved in the spectrum sharing process constructa non-cooperative pricing game [11], [12]. Since theselfish users are their own authorities, they will notreveal their private information to others unless somemechanisms have been applied to guarantee that it is notharmful to disclose the private information. Generally,such non-cooperative game with incomplete informationis complex and difficult to study as the players do notknow the perfect strategy profile of others. But basedon our game setting, the well-developed auction theory[13] can be applied to formulate and analyze the pricinggame.In auction games [13], according to an explicit setof rules, the principles (auctioneers) determine resourceallocation and prices on the basis of bids from the agents(bidders). In our spectrum allocation pricing game, theprimary users can be viewed as the principles, whoattempts to sell the unused channels to the secondaryusers. The secondary users are the bidders who competewith each other to buy the permission of usingprimary users’ channels, by which they may gain extrapayoffs for future use. In our pricing game, multiplesellers and buyers coexist, which indicates the doubleauction scenario. It means that not only the secondaryusers but also the primary users need to compete witheach other to make the beneficial transactions possibleby eliciting their willingness of the payments in theforms of bids or asks. Specifically, the double auctionis one of the most common exchange mechanisms, usedextensively in stock markets such as the New YorkStock Exchange (NYSE) or commodity markets suchas Chicago Merchandize Exchange (CME). The mostimportant property of double auction mechanism is itshigh efficiency, which is still not fully understood ineconomic theory. Moreover, it can respond quickly tochanging conditions of auction participants. However, inorder to achieve the full efficiency of the double auctionmechanism, a lot of messages need to be exchangedamong the auction participants, which can be easilyimplemented by powerful central authorities in stockor commodity markets. It is worth noticing that in autonomouswireless networks either central authorities canbe pre-assumed or the bandwidth of control channels isvery limited. Therefore, we aims to develop an efficientpricing approach for spectrum allocation, which not onlyhas the prevalence of the double auction mechanismbut also uses simple message exchanges to quickly andaccurately coordinate the spectrum sharing.IV. BELIEF-ASSISTED DYNAMIC PRICING FOREFFICIENT SPECTRUM ALLOCATIONA. Static Pricing Game and Competitive EquilibriumAssume that the available channels from the primaryusers are leased for usage of certain time period T .Also, we assume that the cost of the primary users andreward payoffs of the secondary users remain unchangedover this period. Before this spectrum sharing period,we define a trading period τ, within which the usersexchange their information of bids and asks to achieveagreements of spectrum usage. The time period T + τ is121


considered as one stage in our pricing game. We firststudy the interactions of the players in static pricinggames. Note that the users’ goals are to maximize theirown payoff functions. As for the primary users, theoptimization problem can be written as follows.O(p i )=max U pi (φ Ai ,α Aiφ Ai ,α A i), ∀i ∈{1, 2, ..., J} (3)iis.t. Uŝaji({φ −aj,φ i aji},β A i ) ≥ Uŝaji({φ −aj, ˜φi aj},βi A ),iŝ aj ≠0,a jii ∈ A i. (4)where ˜φ aj is any feasible payment and φi−aj is theipayment vector excluding the element of the paymentfor the channel a j i . Note that ŝ ais defined as follows.jiŝ aj =i{s k if β aj ik =1,0 if β aj ik =0, ∀k ∈{1, 2, ..., K}. (5)Thus, (4) is the incentive compatible constraint [13]. Itmeans that the secondary users have incentives to providethe optimal payment because they cannot have extragains by cheating on the primary users. Similarly, theoptimization problem can be written for the secondaryusers as follows.O(s i )= maxφ A,β A iU si (φ A ,β A i ), ∀i ∈{1, 2, ..., K} (6)s.t. Uˆpj ({φ −j ,φ j },β A i ) ≥ Uˆpj ({φ −j , ˜φ j },β A i ),ˆp j ≠0,β j iwhere ˆp j is defined as{pk if β j ˆp j =i =1,j ∈ A k,α j k =10 otherwise, ∀k ∈{1, 2, ..., J}.=1. (7)(8)Similarly, (7) is the incentive compatible constraint forthe primary users, which guarantees that the primaryuser will give the usage permission of their channels tothe secondary users so that they can receive the optimalpayments.From (3) and (6), we can see that in order to obtainthe optimal allocation and payments, a multi-objectiveoptimization problem needs to be solved, which becomesextremely complicated due to our game setting thatonly involves incomplete information. Thus, in orderto make this problem tangible, we analyze it from thegame theory point of view. Generally speaking, gametheory provides well-developed equilibrium concepts oroptimality criteria to study the outcomes of games.For instance, Nash Equilibrium [12] is an importantFig. 1: Illustration of supply and demand functions.concept to measure the outcome of a non-cooperationgame, which is a set of strategies, one for each player,such that no selfish player has incentive to unilaterallychange his/her action. In order to further measure theefficiency of game outcomes, Pareto Optimality [11]is defined such that a Pareto optimal outcome cannot beimproved upon without hurting at least one player. Often,a Nash equilibrium is not Pareto optimal while Paretooptimal outcomes may not be sustained consideringthe selfishness of the players. Further, considering thedouble auction scenarios of our pricing game, CompetitiveEquilibrium (CE) [14] is a well-known theoreticalprediction of the outcomes. It is the price at which thenumber of buyers willing to buy is equal to the numberof sellers willing to sell. Alternatively, CE can also beinterpreted as where the supply and demand match [13].The supply function can be defined as the relationshipbetween the acquisition costs of primary users and thenumber of corresponding channels; the demand functioncan be defined as the relationship between the rewardpayoffs of secondary users and the number of correspondingchannels. We describe the supply and demandfunctions in Figure 1. Note that CE is also proved tobe Pareto optimal in stationary double auction scenarios[15]. It is worth noting that in order to achieve the CE thetraditional continuous bid/ask interactions among playerswill involve a great amount of message exchanges andrequire powerful centralized control, which may not beapplicable to wireless networking scenarios due to thelimited bandwidth of control channels.B. Belief-Assisted Dynamic PricingConsidering network dynamics due to mobility, channelvariations or wireless traffic variations, the secondaryusers may have different reward payoffs of acquiring122


certain channels from primary users at different timestages. Specifically, since the secondary users can bemobile devices, they may move out the access rangeof certain channels and hence the corresponding rewardpayoffs v j iare regarded as 0. Or, the secondary users mayface various channel fading conditions within differentspectrum ranges or during different time periods, whichchanges their payoff values v j iat different time stages.Moreover, the costs of primary users will also changeover time due to network dynamics. For instance, if thelegacy users themselves have larger spectrum demands,some legacy channels may not be available for leasinganymore, which actually indicates an infinite leasing costof those channels in our pricing model. In brief, c j i and vj ineed to be considered as random variables in dynamicscenarios, which we assume to satisfy the probabilitydensity functions (PDF) f c (c) and f v (v), respectively.Therefore, considering dynamic network conditions, wefurther model the spectrum sharing as a multi-stagedynamic pricing game. Let γ be the discount factor ofthe multi-stage game. Based on (3) and (6), the objectivefunctions for the primary users and secondary users canbe rewritten as follows.Õ(p i )=Õ(s i )=max Eφ Ai ,t,α A i cj [i ,vj ii,tmax E cj [φ A,t,βi,tA i ,vj i∞∑t=1γ t · U pi,t(φ Ai,t,α Aii,t)], (9)∞∑γ t · U si,t(φ A,t ,βi,t)], A (10)t=1where the subscript t indicates the tth stage of themulti-stage game. Generally speaking, there may existsome overall constraints of spectrum sharing such aseach secondary user’s total budget for leasing spectrumresources or each primary user’s total available spectrumsupply. Under these constraints, the above problem needto be further modeled as a dynamic programming process[16], [17] to obtain optimal sequential strategies byconsidering some state parameters such as the numberof channels to be allocated at every stage or the residualmonetary budget. However, the major difficulty ofdynamic spectrum sharing lies in that how to efficientlyand quickly update the spectrum sharing strategies adaptto the changing network conditions only based on localinformation. Therefore, in the following parts, we firstfocus on developing a belief-assisted dynamic pricingapproach, which can not only approach CE outcomesbut also responds quickly to networking dynamics whileonly introducing limited overhead. Then, the total budgetconstraint is taken into consideration and a dynamicprogramming approach is further proposed to obtain theoptimal sequential strategies.1) Belief-Assisted Dynamic Pricing: Since our pricinggame belongs to the non-cooperation games withincomplete information [12], the players need to build upcertain beliefs of other players’ future possible strategiesto assist their decision making. Considering that there aremultiple players with private information in the pricinggame and what directly affect the outcome of the gameare the bid/ask prices, it is more efficient to define onecommon belief function based on the publicly observedbid/ask prices than generating specific belief of everyother player’s private information. Hence, enlightened by[14], we consider the primary/secondary users’ beliefsas the ratio their bid/ask being accepted at differentprice levels. At each time during the dynamic spectrumsharing, the ratio of asks from primary users at x thathave been accepted can be written as follows.˜r p (x) = µ A(x)µ(x) , (11)where µ(x) and µ A (x) are the number of asks at x andthe number of accepted asks at x, respectively. Similarly,at each time during the dynamic spectrum sharing, theratio of bids from secondary users at y that have beenaccepted is˜r s (y) = η A(y)η(y) , (12)where η(y) and η A (y) are the number of bids at y andthe number of accepted bids at y, respectively. Usually,˜r p (x) and ˜r s (y) can be accurately estimated if a greatnumber of buyers and sellers are participating in thepricing at the same time. However, in our pricing game,only a relatively small number of players are involvedin the spectrum sharing at the specific time. The beliefs,namely, ˜r p (x) and ˜r s (y) cannot be practically obtainedso that we need to further consider using the historicalbid/ask information to build up empirical belief values.Considering the characteristics of double auction, wehave the following observations:• If an ask ˜x xis accepted, the ask at x will alsobe accepted;• If a bid ỹ>xis made, the ask at x will also beaccepted.Based on the above observations, the players’ beliefscan be further defined as follows using the past bid/askinformation.123


Definition 1: Primary users’ beliefs: for each potentialask at x, define⎧⎪⎨ 1 x =0∑w≥xˆr p(x) =µ A(w)+ ∑ ∑ w≥x η(w)w≥x ⎪⎩µ A(w)+ ∑ x ∈ (0,M)w≥x η(w)+∑ w≤x µ R(w)0 x ≥ M(13)where µ R (w) is the number of asks at w that has beenrejected, M is a large enough value so that the asksgreater than M won’t be accepted. Also, it is intuitivethat the ask at 0 will be definitely accepted as no cost isintroduced.Definition 2: Secondary users’ beliefs: for each potentialbid at y, define⎧⎪⎨ˆr s(x) =⎪⎩0 y =0∑w≤y η A(w)+ ∑ w≤y µ(w)∑w≤y η A(w)+ ∑ w≤y µ(w)+∑ w≥y η R(w)y ∈ (0,M)1 y ≥ M(14)where η R (w) is the number of bids at w that has beenrejected. And, it is intuitive that the bid at 0 will not beaccepted by any primary users.Noting that it is too costly to build up beliefs on everypossible bid or ask price, we can update the beliefs onlyat some fixed prices and use interpolation to obtain thebelief function over the price space. Then, it is worthdiscussing the effect of the available public informationon the efficiency of the above belief system. First, in thescenario that only local information is available to eachuser, the user updates the belief based on her/his ownobserved past bid/ask information, which results in moremessage exchanges to achieve the equilibrium price.Second, considering the broadcast nature of wirelesschannels, the neighbors’ bid/ask information may beobserved by the users, which can also be utilized toupdate the beliefs. In this scenario, the users may havepart of the public information besides of their privateinformation, which may accelerate their belief-updatingpace and result in more efficient pricing process. Moreover,if the users have the access to all the publicinformation such as ask/bid interactions through somecentralized point, the above belief function is able toquickly reflect current supply and demand relationships.Before using our defined belief functions to assist thestrategy decisions, we first look at the Spread ReductionRule (SRR) of double auction mechanisms. Generally,before the double auction pricing game converges toCE, there may exist a gap between the highest bidand lowest ask, which is called the spread of doubleauction. The SRR states that any ask that is permissiblemust be lower than current lowest ask, i.e., outstandingask [14], and then either each new ask results in anTABLE I: Belief-assisted dynamic spectrum allocation1. Initialize the users’ beliefs and bids/asks⋄ The primary users initialize their asks as large values close to Mand their beliefs as small positive values less than 1;⋄ The secondary users initialize their bids as small values close to 0and their beliefs as small positive values less than 1.2. Belief update based on local information:Update primary and secondary users’ beliefsusing (13) and (14), respectively3. Optimal bid/ask update:⋄ Obtain the optimal ask for each primary user by solving (16);⋄ Obtain the optimal bid for each secondary user by solving (17).4. Update leasing agreement and spectrum pool:⋄ If the outstanding bid is greater than or equal to the outstanding ask,the leasing agreement will be signed between the corresponding users;⋄ Update the spectrum pool by removing the assigned channel.5. Iteration:If the spectrum pool is not empty, go back to Step 2.agreed transaction or it becomes the new outstandingask. A similar argument can be applied to bids. Bydefining current outstanding ask and bid as ox and oy,respectively, we let ¯r p (x) =ˆr p (x) · I [0,ox) (x) for eachx and ¯r s (y) =ˆr s (x) · I (oy,M](y) for each y, which aremodified belief function considering the SRR. Note thatI (a,b) (x) is defined as{ 1 if x ∈ (a, b);I (a,b) (x) =(15)0 otherwise.By using the belief function ¯r p (x), the payoff maximizationof selling the ith primary user’s jth channel can bewritten asmax E[U p i(x, j)], (16)x∈(oy,ox)where U pi (x, j) represents the payoff introduced byallocating the jth channel when the ask is x, andthenE[U pi (x, j)] = (x − c j i ) · ¯r p(x). Similarly, as for thesecondary user s i , the payoff maximization of leasingthe jth channel in the spectrum pool can be written asmax E[U s i(y, j)], (17)y∈(oy,ox)where U si (y, j) represents the payoff introduced byleasing the jth channel in the spectrum pool when the bidis y, andthenE[U si (y, j)] = (v j i − y) · ¯r s(y). Therefore,by solving the optimization problem for each primaryand secondary user using (16) and (17), respectively,primary and secondary users can make the optimaldecision of spectrum allocation at every stage conditionalon dynamic spectrum demand and supply. Based onthe above discussions, we illustrate our belief-assisteddynamic pricing algorithm for spectrum allocation inTable I.124


2) Dynamic Pricing with Budget Constraints: Basedon the belief-assisted dynamic pricing algorithm developedabove, in this part we further consider the optimalspectrum allocation when each secondary user is constrainedby a total monetary budget for leasing spectrumusage. Note that the spectrum allocation problem can besimilarly solved when the overall constraints exist forprimary users.Considering the budget constraints of secondary users,we rewrite their optimization objectives as follows.Ô(s i )=max E cj [φ A,t,βi,t A ,ψi i ,vj i∞∑γ t · U si,t(φ A,t ,βi,t, A ˜ψ i,t )],t=1(18)s.t. Uˆpj,t({φ −j,t ,φ j,t }) ≥ Uˆpj,t({φ −j,t , ˜φ j,t }),(19)∞∑ψ t ≤ B i . (20)t=1where ψ i = {ψ i,t } t∈{1,2,...,∞} and ψ i,t is the totalmonetary payment used during the tth stage for the ithsecondary user leasing the channels. Moreover, B i isthe ith secondary user’ total budget. Note that ˜ψi,t =B i − ∑ τ=t−1τ=1ψ i,τ , which is the residual budget at the tthstage and can be considered as a state parameter. Hence,(19) and (20) are the incentive compatible constraint andtotal budget constraint, respectively. As it is difficult todirectly solve (18), we study the dynamic programmingapproach to simplify the multistage optimization problem.Define the value function Q si,t( ˜ψ i ) as the ith secondaryuser’s maximum expected payoff obtainable fromperiods t, t+1, ..., ∞ given that the monetary budget leftis ˜ψ i . Simplifying (18) using the Bellman equation [16],we have the maximal expected payoff Q si,t( ˜ψ i ) writtenas follows.Q si,t( ˜ψ i )=max {E cj [Uφ A,t,βi,t A ,ψi i ,vj i si,t(φ A,t ,βi,t, A ˜ψ i )+γ · Q si,t+1( ˜ψ i − ψ i,t )]}, (21)s.t. Uˆpj,t({φ −j,t ,φ j,t }) ≥ Uˆpj,t({φ −j,t , ˜φ j,t }). (22)The boundary conditions for the above dynamic programmingproblem areQ si,∞( ˜ψ i )=0, ˜ψi ∈ (0,B i ]. (23)Note that the first term on the right hand side (RHS)of (21) represents the payoff at current stage and thesecond term on the RHS of (21) represents the futurepayoff obtained after the tth stage give the budget state˜ψ i −ψ i,t . Further, applying the principle of optimality in[16], the spectrum sharing configuration {φ A,t ,βi,t A,ψ i}that achieves the maximum in (21) given ˜ψ i , t and thestatistics of c j i ,vj iis also the optimal solution for theoverall optimization problem (18).In order to obtain Q si,t( ˜ψ i ), the maximal payoff ofone stage needs to be first derived for different residualbudget values ˜ψ i . The difference of the current payofffunction in (18) and the one-stage payoff function in(6) lies in that the applied budget constraint affects theoutcomes of the pricing game. For instance, even thoughboth the primary users and secondary users can achievehigher payoffs by assigning a channel to user s i , the users i may not have enough budgets to lease this channel.Thus, the algorithm in Table I cannot be directly appliedhere for optimal spectrum sharing. We need to modifythe bid update step as follows: user s i updates his/her bidby min{ ˜ψ i ,y}, wherey is obtained from (17). Note thatit is highly complicated to derive the close-form solutionfor the one-stage payoff function in (18) [13], [15].Thus,we use simulation to approximate it for different residualbudget values, which proceeds as follows: Generate alarge number of samples of the secondary and primaryusers with reward payoffs and costs satisfying f v (v) andf c (c), respectively. Using the above modified version ofthe algorithm in Table I, calculate the average one-stagepayoffs given different ˜ψ based on the outcomes of thespectrum allocation samples.By using the numerical results of the one-stage payofffunction, we then derive Q si,t( ˜ψ i ) using dynamicprogramming methods. Considering infinite spectrumallocation stages, the maximum payoff Q si,t( ˜ψ i ) in (21)can be written as follows.Q ∗ s i( ˜ψ i )=max {E cj [Uφ A,t,βi,t A ,ψi i ,vj i si,t(φ A,t ,βi,t, A ˜ψ i )+γ · Q ∗ s i( ˜ψ i − ψ i,t )]}, (24)or, equivalently, Q ∗ s i= T Q ∗ s i,whereT is the operatorupdating Q ∗ s iusing (24). Let S be the feasible set ofthe state parameter. The convergence proposition of thedynamic programming algorithm [16] can be appliedhere, which states that: for any bounded function Q :S→R, the optimal payoff function satisfies Q ∗ (x) =lim p→∞ (T p Q)(x), ∀x ∈S.AsQ si ( ˜ψ i ) is bounded inour algorithm, we are able to apply the value iterationmethod to approximate the optimal Q si ( ˜ψ i ), which proceedsas follows: Start from some initial function forQ si ( ˜ψ i ) as Q 0 s i( ˜ψ i )=g(x), where the superscript standsfor the iteration number. Then, iteratively update Q si ( ˜ψ i )by letting Q p+1s i( ˜ψ i )=(T Q p s i)( ˜ψ i ). The iteration process125


Fig. 2: Comparison of the total payoff for the proposed scheme andtheoretical Competitive Equilibrium.ends until |Q p+1s i( ˜ψ i ) − Q p s i( ˜ψ i )|≤ɛ, forall ˜ψi ∈S,where ɛ is the error bound for Q ∗ s i( ˜ψ i ).Intuitively, the basic idea behind our dynamic pricingapproach for spectrum allocation with budget constraintscan be explained as follows: Considering the overallbudget constraints, the users make their spectrum sharingdecisions not only based on their current payoffs butalso based on expected future payoffs. Specifically, ifthe competition for spectrum resources is high at currentstage, the users prefer to save their monetary budgetsfor future usage, which will yield higher overall payoffsfor the users. Therefore, by using our proposed dynamicpricing approach, the spectrum allocation can be optimizednot only in the space and frequency domains butalso in the the time domain.V. SIMULATION RESULTSIn this section, we evaluate the performance of theproposed belief-assisted dynamic spectrum sharing approachin wireless networks. Considering a wirelessnetwork covering 100 × 100 area, we simulate J primaryusers by randomly placing them in the network.These primary users can be the base stations serving fordifferent wireless network operators or different accesspoints in a mesh network. Here we assume the primaryusers’ locations are fixed and their unused channels areavailable to the secondary users within the distance of50. Then, we randomly deploy K secondary users inthe network, which are assumed to be mobile devices.The mobility of the secondary users is modeled usinga simplified random waypoint model [18], where weassume the “thinking time” at each waypoint is closeto the effective duration of one channel-leasing agreement,the waypoints are uniformly distributed within theFig. 3: Comparison of the overhead between the proposed schemeand continuous double auction scheme.distance of 10, and the traveling time is much smallerthan the “thinking time”. Let the cost of an availablechannel in the spectrum pool be uniformly distributedin [10, 30], the reward payoff of leasing one channelbe uniformly distributed in [20, 40]. If a channel is notavailable to some secondary users, let the correspondingreward payoffs of this channel be 0. Note that J =5and 10 3 pricing stages have been simulated. Let n i =4, ∀i ∈{1, 2, ..., J} and γ =0.99.We first focus our simulation studies on dynamicspectrum sharing without budget constraints, which canbe used to illustrate the efficiency of the proposedbelief-assisted pricing algorithm for spectrum allocation.In our simulation, the local bid/ask information withinthe transmission range of each node is used for beliefconstruction and update. In Figure 2, we compare thetotal payoff of all users of our proposed approach withthat of the theoretical CE outcomes for different numberof secondary users. It can be seen from this figure thatthe performance loss of our approach is very limitedcompared to that of the theoretical optimal solutions.Moreover, when the number of secondary users increases,our approach is able to approach the optimal CE.It is because that the belief function reflects the spectrumdemand and supply more accurately when more users areinvolved in spectrum sharing.Now we study the overhead of our pricing approach.Here we measure the pricing overhead by showingthe average number of bids and asks for each stage.In Figure 3, the overhead of our pricing approach iscompared to that of the traditional continuous doubleauction when the same total payoff is achieved. Assumethe minimal bid/ask step δ of the continuous doubleauction to be 0.01. It can be seen from the figure126


There are several avenues for future research. Weintend to further perform the equilibrium analysis ofthe solution obtained by the proposed dynamic spectrumallocation scheme. Also, we would like to study howto adapt our scheme to mesh, cellular or combinednetworks under practical constraints caused by the natureof wireless networking. Further, security issues needto be extensively investigated for dynamic spectrumallocation. Not only the security threats caused by thedesigned mechanism itself such as bidding ring problems[13] need to be considered, but also the threats due tothe limitations of wireless systems and radio interface.Fig. 4: Comparison of the total payoffs of the proposed schemewith those of the static scheme.that our approach substantially decreases the pricingcommunication overhead. Note that when decreasing theoverhead, our proposed approach may introduce extracomplexity to update the beliefs.Then, we study the dynamic spectrum allocation wheneach secondary user is constrained by his/her monetarybudget. For comparison, we define a static scheme inwhich the secondary users make their spectrum-leasingdecisions without considering their budget limits. Withoutloss of generality, we assume that the budget constraintsfor the secondary users are the same. In Figure 4,we compare the total payoffs of our proposed dynamicprogramming scheme with those of the static schemefor different budget constraints. It can be seen from thefigure that our proposed scheme achieves significant performancegains over the static scheme when the budgetconstraints are taken into consideration. Also, when thebudget limits increase, the proposed scheme achieveshigher gain by further exploiting the time diversity.VI. CONCLUSIONS AND FUTURE WORKIn this paper, we have studied dynamic pricing forefficient spectrum allocation in wireless networks withselfish users. We model the dynamic spectrum allocationas a multi-stage game and propose a belief-assisteddynamic pricing approach to maximize the users’ payoffswhile providing them the participating incentives viadouble auction rules. Further, the dynamic pricing underthe budget constraints of secondary users is analyzedusing dynamic programming. Simulation results showthat the proposed scheme can approach the optimalperformances by only using limited overhead. Moreover,the time diversity of spectrum resources can be fullyexploited when budget constraints exist.REFERENCES[1] FCC, “Spectrum policy task force report,” FCC Document ETDocket No. 02-135, November 2002.[2] FCC, “Facilitating opportunities for flexible, efficient, andreliable spectrum use employing cognitive radio technologies:notice of proposed rule making and order,” FCC Document ETDocket No. 03-108, December 2003.[3] R. J. Berger, “Open spectrum: a path to ubiquitous connectivity,”FCC ACM Queue 1, 3, May 2003.[4] J. M. Peha, “Approaches to spectrum sharing,” IEEE CommunicationsMagazine, vol. 43, pp. 10–12, Feburary 2005.[5] M. M. Buddhikot, “Dimsumnet: new directions in wirelessnetworking using coordinated dynamic spectrum access,” inProc. of IEEE WoWMoM’05, 2005.[6] C. Peng, H. Zheng, and B. Y. Zhao, “Utilization and fairnessin spectrum assignment for opportunistic spectrum access,” toappear in Mobile Networks and Applications (MONET), 2006.[7] L. Cao and H. Zheng, “Distributied spectrum allocation vialocal bargaining,” in Proc. of IEEE DySpan, 2005.[8] R. Etkin, A. Parekh, and D. Tse, “Spectrum sharing forunlicensed bands,” in Proc. of IEEE DySpan, 2005.[9] M. H. Halldorson, J. H. Halpern, L. Li, V. S. Mirrokni, “Onspectrum sharing games,” in Proc. of ACM Symposium onPrinciple of Distributed Computing (PODC), 2004.[10] M. Felegyhazi and J. P. Hubaux, “Wireless operators in a sharedspectrum,” in Proc. of IEEE INFOCOM’06, 2006.[11] D. Fudenberg and J. Tirole, Game Theory, The MIT Press,Cambridge, Massachusetts, 1991.[12] M. J. Osborne and A. Rubinstein, A Course in Game Theory,The MIT Press, Cambridge, Massachusetts, 1994.[13] V. Krishna, Auction Theory, Academic Press, 2002.[14] S. Gjerstad and J. Dickhaut, “Price formation in doubleauctions,” Games and Economic Behavior, vol. 22, pp. 1–29,1998.[15] L. Hurwicz, R. Radner, and S. Reiter, “A stochastic decentralizedresource allocation process: Part i,” Econometrica, vol. 43,pp. 363–393, 1975.[16] D. Bertsekas, Dynamic Programming and Optimal Control, vol.1,2, Athena Scientific, Belmont, MA, Second edition, 2001.[17] Z. Ji, W. Yu, and K. J. R. Liu, “An optimal dynamic pricingframework for autonomous mobile ad hoc networks,” in Proc.of IEEE INFOCOM’06, 2006.[18] D. B. Johnson and D. A. Maltz, “Dynamic source routing in adhoc wireless networks, mobile computing,” IEEE Transactionson Mobile Computing, pp. 153–181, 2000.127

More magazines by this user
Similar magazines