step-size optimization of the bndr-lms algorithm - LPS/UFRJ

Table 1: The Binormalized Data-Reusing LMS Algorithm[4].BNDR-LMS = small valuefor each kf x 1 = x(k)x 2 = x(k 1)d 1 = d(k)d 2 = d(k 1)a = x T 1 x 2b = x T 1 x 1c = x T 2 x 2d = x T 1 w(k)if a 2 == bcf w(k +1)=w(k)+(d 1 d)x 1 =(b+)gelsef e = x T 2 w(k)den = bc a 2A =(d 1 c+ea dc d 2 a)=denB =(d 2 b+da eb d 1 a)=denw(k +1)=w(k)+(Ax 1 +Bx 2 )ggto be equal to unity. This expression for the excess in theMSE [1] was obtained with the help of a simple model[6] for the input signal vector given byx(k) =s k r k V k (2)where s k is 1 with probability 1=2, r k has the samedistribution as k x(k) k or E[rk 2] = (N +1)2 x , andV k is one of the N + 1 orthonormal eigenvectors of theautocorrelation matrix R of the input signal. It wasassumed in the analysis that each vector V i occurs withan equal probability of 1=(N +1) which means whitenoise type input.3 THE OPTIMAL STEP-SIZE SEQUENCEIn this section the optimal step-size sequence for thegiven problem is derived. From the expression of (k+1), we will follow an approach similar to that used in [6]and rewrite (1) assuming that up to time k we have theoptimal sequence (0) to (k 1) already availableand also the optimal quantities (k) and (k 1).(k)((k) 2)(k +1) =1+ (k)N +1+ N(k)(1 (k))2 ((k) 2)(N +1) 2 (k 1)+ (1 + N((k) 2)2 )(k) 2(N +1) 2 2 n (3)If we now compute the derivative of (k +1) withrespect to (k) and make it equal to zero, we obtainafter some algebraic manipulation (k) = 1= 1ss11 (k)+ (k 1)2( (k 1) + 2 n ) (k)+ (k 1) 2n22 (k 1)(4)It is worth mentioning that (4) is in accordance with thesituation when the convergence is reached; in that casewe have (k)= (k 1) = n 2 and therefore we have (k)=0as expected. Moreover, if we have n 2 = 0the value of (k) will be close to one (admitting that (k) (k 1)) even after convergence, whichmeans that we should have maximum speed of convergencewith minimum misadjustment if the noise is zero.For the normalized LMS (NLMS) algorithm, a recursiveformula for (k) in terms of (k 1) and theorder N was obtained in [6]. Unfortunately, a similarexpression could not be obtained for the BNDR-LMSalgorithm. Instead, a simple algorithm was used to producethe optimal step-size sequence. This algorithm ispresented in Table 2 1 and has one important initializationparameter with a strong inuence on the behaviorof (k). This parameter is the ratio 2 dwhere the numeratoris the variance of the referencen2 signal.Table 2: Algorithm for computing the optimal step-sizesequence.(k) of the BNDR-LMS algorithm(0)=( 1) = d2n 2 = noise varianceN = adaptive lter order(0)=1for each k(k)+(k 1)f (k) =1q1aa =h1+(k)((k) 2)N+1i2((k 1)+ 2 n )bb = N(k)(1 (k))2 ((k) 2)(N+1) 2cc = (1+N((k) 2)2 )(k) 2(N+1) 2 n2(k +1)=aa(k)+bb(kg1) + ccWe next present in Fig. 1 the curves of (k) for dierentvalues of what should be called in this case (desired)signal to noise ratio or SNR =10log 2 dfrom 0 to 40 dB.n2Note that for n 2 = 0 (noiseless case), the SNR goes toinnity and the step-size would remain xed at unity.1 Note that the asterisk (*) was dropped out from the optimalvalues for simplicity only.

20LEARNING CURVES FOR TIME−VARYING STEP−SIZE5LEARNING CURVES FOR COLORED SIGNAL1510fixed step−sizeoptimal step−sizeNLMS approximation1/k approximation0fixed step−size sequenceoptimal sequence5MSE in dB0−5−10MSE in dB−5−10−15−20−15−250 50 100 150 200 250k−200 50 100 150 200 250kFigure 3: Learning curves for the xed step-size, theoptimal step-size and its two approximations.Figure 4: Comparing the learning curves for the case ofcolored input signal.x1+ p 1 xapproach is the possibility of fast tracking of suddenand strong changes in the environment. In this case,the instantaneous error becomes high and the estimated(k + 1) is increased such that the value of approachesthe unity again and a fast re-adaptation starts.When using this approach, it is worth rememberingpthat, since equation (4) is of the type 1 1 x, thestep-size (k) can be written as which isanumericallyless sensitive expression. Equation (8) showsthis expression.(k) =1+q15 CONCLUSIONS(k)+(k 1) 2 2 n2(k 1)(k)+(k 1) 2 2 n2(k 1)(8)This paper addressed the optimization of the step-sizeof the BNDR-LMS algorithm when the input signalis uncorrelated. An optimal sequence was proposedand a simple algorithm to nd this sequence was introduced.Alternative approximation sequences were alsopresented and their initialization parameters compared.Simulations carried out in a system identication problemshowed the good performance of the optimal stepsizesequence as well as the possibility of using alternativessequences obtained with less eort and with similareciency. In another simulation it was possible to observethat the same step-size sequence, optimal for thewhite noise input, can also be used in applications wherea highly correlated input signal is present.References[1] P. S. R. Diniz, Adaptive Filtering: Algorithms andPractical Implementation. Kluwer Academic Press,Boston, MA, USA, 1997.[2] S. Haykin, Adaptive Filter Theory. Prentice Hall,Englewood Clis, NJ, USA, 3rd edition, 1996.[3] J. A. Apolinario Jr., M. L. R. de Campos andP. S. R. Diniz, \The Binormalized Data-ReusingLMS Algorithm," Proceedings of the XV SimposioBrasileiro de Telecomunicac~oes, Recife, Brazil,1997.[4] J. A. Apolinario Jr., M. L. R. de Campos and P. S.R. Diniz, \Convergence analysis of the BinormalizedData-Reusing LMS Algorithm," Proceedingsof the European Conference onCircuit Theory andDesign, Budapest, Hungary, 1997.[5] M. L. R. de Campos, J. A. Apolinario Jr. and P.S. R. Diniz, \Mean-squared error analysis of theBinormalized Data-Reusing LMS algorithm," Proceedingsof the International Conference onAcoustics,Speech, and Signal Processing, Seattle, USA,1998.[6] D. Slock, \On the convergence behavior of the LMSand the normalized LMS algorithms," IEEE Transactionson Signal Processing, vol. 41, pp. 2811-2825, Sept. 1993.