Rate-compatible distributed arithmetic coding - Politecnico di Torino

1Rate-compatible distributed arithmetic codingMarco Grangetto, Member, IEEE, Enrico Magli, Senior Member, IEEE, Roberto Tron, Student Member, IEEE,Gabriella Olmo, Senior Member, IEEEAbstract—Distributed arithmetic coding has been shown to beeffective for Slepian-Wolf coding with side information. In thisletter, we extend it to rate-compatible coding, which is useful inpresence of a feedback channel between encoder and decoder. Theperformance loss with respect to the original version is negligible.Index Terms—Distributed source coding, arithmetic coding,Slepian-Wolf coding, Wyner-Ziv coding, compression.I. INTRODUCTIONThe theory of distributed source coding (DSC) provesthat separate lossless encoding of two or more correlateddata sources is optimal, provided that the sources are jointlydecoded. A classical situation encompasses a first source, orside information, which is subject to standard encoding, anda second one that is conditionally encoded at a rate lowerthan its entropy. DSC coders typically represent the sourceusing the syndrome or parity bits of a suitable channel codeof given rate, e.g. trellis codes, turbo codes and low-densityparity-check codes.In [1] a Slepian-Wolf coder is proposed based on a modificationof the arithmetic coder (AC). The scheme, nameddistributed arithmetic coder (DAC), encodes a binary sourceat a rate smaller than its entropy by allowing the intervals tooverlap and generating an ambiguous codeword. A soft jointdecoder exploits knowledge of the side information to decodethe source. DAC has been shown to work well also for shortblock lengths, representing a good solution for image or videocoding.To take advantage of a feedback channel between theencoder and the decoder, the DAC principle has to be revisitedto provide a rate-compatible system, similarly to what canbe done with turbo [2] and low-density parity-check (LDPC)codes [3]. The algorithm proposed in this letter builds upon theDAC, and proposes a rate-compatible DAC-based algorithm.In particular, the DAC decoding procedure is modified to allowthe transmission on the feedback channel of some extra bitsupon decoder request, in case this latter encounters a decodingambiguity. This allows to reduce the residual bit-error rate(BER), up to error-free decoding.Marco Grangetto is with the Department of Informatics - Universitàdegli Studi di Torino, C.So Svizzera, 185 - 10149 Torino -Italy - Ph.: +39-011-6706847 - FAX: +39-011-011751603 - Email:marco.grangetto@unito.it. Enrico Magli and Gabriella Olmo arewith the Department of Electronics - Politecnico di Torino, Corso Duca degliAbruzzi 24 - 10129 Torino - Italy - Ph.: +39-011-5644094 - FAX: +39-011-5644149 - E-mail: enrico.magli(gabriella.olmo)@polito.it.Roberto Tron is with the Vision Lab Center for Imaging Science - JohnsHopkins University, 3400 N. Charles Street - Baltimore, MD 21218 - USA -Ph.: 410-516-6461 - FAX: 410-516-4594 -E-mail: tron@cis.jhu.edu.II. DISTRIBUTED ARITHMETIC CODINGLet ¼ ½ ¡¡¡ Æ ½℄ be a length-Æ binary sequenceto be encoded, with probability distribution Ô ¼ È ´ ¼µand Ô ½ È ´ ½µ, ¼ ¡¡¡Æ ½, and let ¼ ½ ¡¡¡ Æ ½℄ be a correlated side information binarysymbol sequence. In classical AC, the source symbols aremapped onto sub-intervals of ¼ ½µ. For each input symbol ,the current interval (initially set to ¼ ½µ) is partitioned intotwo adjacent sub-intervals whose lengths are proportional toÔ ¼ and Ô ½ . The sub-interval representing is selected as thenext current interval, and this procedure is iterated. After allÆ symbols have been processed, the sequence is representedby codeword , consisting in a binary number lying in thefinal interval Á, whose average length is Ä ÐÓ ¾ Á bits,where Á is the length of Á.In DAC [1], a modified interval subdivision strategy isadopted, and the interval lengths are taken proportionally to themodified probabilities Ô ¼ Ô ¼ and Ô ½ Ô ½ . The sub-intervalsare allowed to partially overlap in order to fit the ¼ ½µ interval.As a consequence, the decoder will typically be unable todecode the source unambiguously without knowledge of theside information. The overlapping leads to a larger final interval,and hence a shorter codeword. More details on the DACencoding procedure, e.g. the policy of interval overlapping andthe related coding rate, can be found in [1], [4].The DAC decoding process can be formulated as a symboldrivensequential search along a proper decoding tree, whereeach node represents a state of the sequential arithmeticdecoder. In [1], the Maximum A Posteriori (MAP) metricÈ ´ µ is employed. Whenever lies in an overlappedregion, the decoder performs a branching. Let ¾´¼ ½µ, ¼ ¡¡¡Æ ½ be a ternary symbol that representsthe decoding of the –th symbol, where represents a decodingambiguity. If , since could be equal to either 0 or1, two alternative paths are stored in the decoding memory,corresponding to the two alternative symbols ¼ and ½ that can be decoded at this step. Then, their MAPmetrics are updated and the corresponding interval is selectedfor next iteration.III. RATE-COMPATIBLE DACDue to the limited available memory, at each step the DACdecoder is able to store only a limited number Å of candidatesequences. As a consequence, it may happen that the correctpath is discarded, as it exhibits a partial metric that is worsethan that of other paths; in this case, decoding errors mayoccur. In particular, errors may stem from erroneous decisionsregarding decoded bits, when the received codeword lies in anambiguous probability interval (i.e. ) in one of the Å

2active paths. In this situation, the decoding process may behelped if the encoder passes on to the decoder those bits thatturn out to be ambiguous during the decoding process (namedfeedback bits - FB). Upon detection of decoding failure, e.g.by using a cyclic redundancy check code, the decoder can haveone or more FBs sent over the feedback channel, and refineits first decoding attempt, possibly correcting the erroneouschoices. To this end, it is necessary that the encoder andthe decoder agree on a list of ambiguous symbols for whichFBs may be needed. The ambiguous symbols depend on both and . An explicit list would require a large amount ofextra rate, as well as explicit communication between and. On the other hand, the encoder can mimic the decoderoperations and create a list of the input symbols that willbe recognized as ambiguous by the decoder. In doing so, itexploits its knowledge of the original sequence, always takingcorrect decisions at branching nodes. This can be done inseveral ways; three options have been considered in this letter,namely: direct ordering, inverse ordering, hybrid ordering.A. Direct ordering: encoderIn direct ordering, the FBs are inserted in a list (called theFB list) following the normal forward ordering of the encodingand decoding procedures. The generation of the FB list ´µat the encoder, containing Ä bits, can be summarized asfollows.FB list generation (direct ordering)- Encode the input sequence using DAC- Decode the obtained encoded sequencetaking correct decisions at branching points, as described below:- ¼- for ¼to Æ ½if ·½append to direct FB list: ´µ end forIt is worth noticing that only the values of the FBs, not theirpositions, need to be transmitted to the decoder.B. Direct ordering: decoderWhen the decoder is unable to reconstruct the source usingthe received DAC codeword, FBs are extracted from the listand sent to the decoder upon request. With direct ordering,the decoding operations are very simple. DAC decoding isperformed as usual; every time an ambiguous symbol isencountered, a FB is used to drive the correct decoding branch,until all the available FBs have been used; then, usual DACdecoding is performed to deal with further branchings.However, it has been found that decoding errors tend tooccur mostly at the end of the sequence [4]. In fact, correctdecoding happens because wrong paths that diverge from thecorrect one at the beginning of the sequence, are likely tobe discarded, as the metric of the correct path asymptoticallytends to get dominant. Therefore, the direct ordering is suboptimalin terms of additional rate, in that the first transmittedFBs are less useful, as they do not refer to the most criticalbranches.C. Inverse ordering: encoderSending the FBs in inverse order (i.e., starting from thosereferring to the last branching events) is more efficient if notall the FBs are transmitted, as these latter are more proneto decoding errors. The generation of the FB list in inverseorder Á ´µ, follows a procedure identical to direct ordering,except that the bits are transmitted in inverse order, i.e. Á´µ ´Ä µ.D. Inverse ordering: decoderInverse ordering makes the decoder design more complex.In fact, it is possible that not all the Å sequences that areretained by the decoder stem from decoder branching decisionson ambiguous bits that match the transmitted FBs. In case asequence (i.e. a path) is not congruent with the FBs, it shouldbe removed from the decoder list, and another sequence,previously disregarded due to the lack of decoding memory,should be resumed. In order to keep track of which sequenceshave been discarded by the Å-algorithm and which ones donot match the FBs, it is necessary to store the whole decodingtree in the decoder memory. The modified decoding processcan then be summarized as follows:1) Decode the whole sequence and store the whole decodingtree. At the end of the decoding procedure, the Åoutmost leaves of the decoding tree represent the set ofcandidate sequences, among which the one with the bestmetric will be selected.2) Backward-check the decoding tree using the FBs.3) Clear the decoding sequences that do not match the FBs.4) Revisit the decoding tree and resume already discardedbranches.5) Select the sequence with the best decoding metric.The backward-check verifies if the sequences selected bythe decoder match the FBs at the branching points. As the FBsare transmitted in inverse order, it is necessary to scan thedecoding tree from the leaves towards the root. Backwardcheckis implemented by means of a recursive procedure.Starting from a leaf of the tree, it is verified whether the branchleading to that leaf matches the FBs. In case it does not, thatleaf must be pruned; otherwise, the control step moves onto the parent node, and so forth until all the FBs have beenscanned. In case a sequence is not compliant with the FBs, itis removed from the decoding memory, and the related leaf ispruned from the decoding tree.In order to make the memory management more efficient,and to speed up the algorithm execution, it is necessary tocancel those paths that have already been completely explored,and that cannot lead to decoding solutions compatible with theinformation contained in the FB list. This is done by means ofthe clearing step. It is a recursive procedure that visits eachnode in the tree. If both children nodes have already beenchecked, and do not match the FBs, that node is pruned fromthe tree. The branches are pruned progressively from the leavestowards the root, avoiding to prune any node that can lead tosolutions not already explored.In case one or more sequences have been cleared, sequencespreviously discarded due to finite Å can be resumed. The

3restarting step visits the decoding tree in depth, and restoressome of those branches that had already been discarded by theÅ-algorithm. The selection of the visiting order is done in agreedy manner, starting from the children nodes with bettermetric. When visiting a node whose children have not beenexplored yet, the decoding and the clearing passes are repeatedstarting from that node. After that, the visit of nodes for therestarting procedure continues recursively on the just createdpath. The procedure stops when all possible sequences havebeen verified, or the decoding memory is full.In order to terminate the decoding, all the solutions foundbefore and after the restarting step are compared, and that withthe best accumulated metric is selected as the final solution.As a final comment, it should be noted that, for both directand inverse ordering, correct decoding of the whole sequenceis ensured if all the FBs are transmitted and Ä is known atthe decoder.E. Hybrid orderingIn order to blend the advantages of the two discussedapproaches, a hybrid ordering has also been considered. Thefirst à FBs are transmitted in inverse order, whereas the othersare transmitted in direct order. In this way, a trade off betweenrate performance and decoding complexity can be achieved.The decoder starts with inverse ordering as long as it hasused the first à FBs. This yields a set of decoder statesthat are compatible with those FBs. Then, it uses the directordering decoding procedure, which selects one out of theavailable candidates, identifying the decoder best path andhence allowing to output the decoded sequence.IV. EXPERIMENTAL RESULTSÆ samples of a binary memoryless source have beengenerated. The side information is obtained as the outputof a binary symmetric channel with input and transitionprobability Ô. In each simulation and for each data block, thesource data are compressed using DAC at a selected initialbase bit-rate, without using the FBs. Then, the FBs are usedone at a time, until the number of decoding errors for thatblock is zero. In all simulations the hybrid algorithm withà ½has been employed.Simulations have been run using 1000 blocks with Æ ½¼¼, Ô ¼ ¼, Ô ¼½½ (corresponding to À´µ ¼¼).Tab. I reports the average bit-rate for error-free decoding; notethat, unlike fixed-rate coding, these variable-rate simulationsconverge very quickly with a small number of trials. As canbe seen, it is convenient to take a base bit-rate that slightly exceedsthe Slepian-Wolf bound; in this case, error-free decodingis achieved at a rate about 0.07 bps larger than the bound. Forcomparison, a non rate-compatible DAC achieves error-freedecoding at a rate equal to 0.58 bps. This is obtained averaging1000 trials in which each sequence is reencoded with DAC atincreasing rate up to error-free decoding; while this proceduredoes not represent a practical solution, it is a good benchmarkfor the rate-compatible algorithm. These results show that theperformance loss incurred for rate compatibility is negligible.Another simulation has been performed using 100 datablocks with Æ ¾¼¼, Ô ¼ ¼ and Ô ¼¾¿ (correspondingto À´µ ¼). Error-free decoding is achieved at a rateabout 0.06 bps larger than the bound. For comparison, the nonrate-compatible DAC achieves a rate of 0.61 bps.For the sake of comparison, we have also simulated a systembased on rate-compatible LDPC codes such as in [3], and onturbo codes such as in [2]. For the LDPC code, the only blocklength available in the software implementing [3] is Æ ¿.For the turbo code, we use block lengths of Æ ½¼¼ and Æ ¾¼¼ with S-random interleaving. For Ô ¼ ¼ and Ô ¼½½,the average bit-rate for error-free decoding turns out to be0.59 bps for irregular LDPC codes and 0.62 for regular codes.Under the same conditions, the turbo code with Æ ½¼¼achieves 0.73 bps. For Æ ¾¼¼, Ô ¼ ¼ and Ô ¼¾¿, theirregular and regular LDPC codes achieve 0.67 and 0.69 bpsrespectively, while the turbo code achieves 0.99 bps; the poorperformance of turbo codes for asymmetric input distributionhas also been noted in [4]. Hence, for small block lengths,the rate-compatible DAC outperforms turbo and LDPC codes.However, as noted in [4], unlike turbo and LDPC codes, theDAC performance does not improve significantly increasingthe block length.Table IAVERAGE BIT-RATE (BPS) NECESSARY FOR ERROR-FREE DECODINGBase bit-rate Average bit-rateÔ ¼ ¼, Ô ¼½½¼¾, Æ ½¼¼0.53 0.580.56 0.590.59 0.61Ô ¼ ¼, Ô ¼¾¿, Æ ¾¼¼0.57 0.640.61 0.62V. CONCLUSIONSIn this letter we have introduced a rate-compatible versionof DAC. It has been shown that the proposed algorithmoutperforms LDPC and turbo codes, and that the performanceloss of the rate-compatible with respect to the standard DACis negligible. This makes DAC suitable for applications wherea feedback channel is available for incremental transmissionof a source with side information is available at the decoder.REFERENCES[1] M. Grangetto, E. Magli, and G. Olmo, “Distributed arithmetic coding,”IEEE Communications Letters, vol. 11, no. 11, pp. 883–885, Nov. 2007.[2] B. Girod, A. Aaron, S. Rane, and D. Rebollo-Monedero, “Distributedvideo coding,” Proceedings of the IEEE, vol. Special Issue on Advancesin Video Coding and Delivery, no. 1, pp. 71–83, Jan. 2005.[3] D. Varodayan, A. Aaron, and B. Girod, “Rate adaptive codes fordistributed source coding,” Signal Processing, vol. 86, no. 11, pp. 3123–3130, Nov. 2006.[4] M. Grangetto, E. Magli, and G. Olmo, “Distributed arithmeticcoding for the asymmetric slepian-wolf problem,” IEEE Transactionson Signal Processing (submitted), preprint available at URLwww1.tlc.polito.it/sas-ipl/Magli/index.php.