Trellis-coded quadrature amplitude modulation with ... - IEEE Xplore

IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 48, NO. 5, SEPTEMBER 1999 1475 

Trellis-Coded Quadrature Amplitude 

Modulation with -Dimensional 

Constellations for Mobile Radio Channels 

Corneliu Eugen D. Sterian, Senior Member, IEEE, Frank Laue, and Matthias Pätzold, Senior Member, IEEE 

Abstract— Using a modified Wei method, originally designed 

for additive white Gaussian noise (AWGN) channels, we have 

constructed four-dimensional (4-D) and six-dimensional (6-D) 

trellis codes with rectangular signal constellations for frequencynonselective 

mobile radio channels. Applying a novel way of 

partitioning the two-dimensional (2-D) constituent constellations, 

both into subsets with enlarged minimum Euclidean distance 

and subrings including equal energy signal points, we have 

obtained partitions of the 2N-D signal sets into subsets with a 

Hamming distance between signal points which equals N. This is 

fundamental for constructing good trellis codes to transmit data 

over flat fading channels. 

Index Terms— Constellation shaping, fading channel models, 

frequency-nonselective mobile radio channels, multidimensional 

trellis-coded modulation, quadrature amplitude modulation, set 

partitioning, shell mapping. 

I. INTRODUCTION 

MOBILE radio channels exhibit a time-varying behavior 

in the received signal envelope, which is called fading. 

This is caused if the receiving antenna, used in mobile radio 

links, picks up multipath reflections. While there are other 

degradations like additive white Gaussian noise (AWGN), the 

fading is by far the main impairment encountered on this 

type of channel. Trellis-coded modulation (TCM) is a standard 

technique used to improve the performance of a digital 

transmission system. Originally, TCM has been introduced 

by Ungerboeck in a seminal paper [1] for AWGN channels. 

The scheme proposed by Ungerboeck uses an expanded onedimensional 

(1-D) or two-dimensional (2-D) constellation with 

2 b+1 signals to transmit b information bits per signaling 

interval without increasing the bandwidth or the transmitted 

power. The constellation is partitioned into 2 m+1 subsets with 

enlarged intrasubset minimum Euclidean distance. Of the b 

information bits that arrive in each signaling interval, m enter 

a rate-m=m +1 convolutional encoder, and the resulting m+1 

coded bits specify which subset is to be used. The remaining 

b 0 m information bits specify which point from the selected 

subset is to be transmitted. In the receiver, a soft-decision 

maximum likelihood decoder attempts to recover the original 

information from the channel output. 

Manuscript received September 28, 1998; revised January 25, 1999. 

C. E. D. Sterian was supported by a scholarship from the German Service for 

Academic Exchanges DAAD (Deutscher Akademischer Austauschdienst). 

The authors are with the Technische Universitat Hamburg-Harburg, 21071 

Hamburg, Germany. 

Publisher Item Identifier S 0018-9545(99)07380-6. 

The first important application of TCM has been a 2-D 

eight-state nonlinear trellis code with 4-dB coding gain designed 

by Wei [2] which was adopted in the Recommendations 

V.32, V.32 bis, and V.33 of ITU-T (formerly CCITT) for 

data transmission over voice-band telephone channels. Three 

four-dimensional (4-D) trellis codes have been adopted in the 

Recommendation V.34 of ITU-T for 33.6-kb/s transmission 

over the switched telephone network [3]. These codes have 

been designed using a method invented by Wei [4]. A 2Ndimensional 

signal point is the concatenation of N 2-D points. 

In Wei’s method, the 2-D constituent constellation has 2 b 

inner points which is the size of the constellation used by the 

reference system, and 2 b =N outer points which provide the 

redundancy necessary for the error control. Since the number 

2 b =N of the outer points must be a positive integer, N must be 

a power of two. This limitation has been removed in [5] which 

allows us to construct six-dimensional (6-D) trellis codes. 

The beginning of TCM for mobile radio channels is related 

to the pioneering work of Divsalar and Simon [6], [7]. 

Many years, only TCM with constant-amplitude modulation 

schemes, like phase shift keying (PSK) and continuous phase 

modulation (CPM), have been considered [8], [9]. The reason 

for this is that, for efficiency, the high-power amplifier (HPA) 

of the transmitter antenna was operated in a very nonlinear 

region. However, a number of important papers appeared 

recently in which quadrature amplitude modulation (QAM) 

is used [10]–[13]. Since QAM is more bandwidth-efficient 

than PSK, for which also the data rate is limited to b = 3 

bit/signaling interval, we consider it in this paper. 

In contradistinction to TCM for AWGN channels, where 

the primary objective is to maximize the Euclidean distance 

between symbol sequences, in designing TCM for fading 

channels, the main task is to maximize the smallest Hamming 

distance of the trellis code. Remember that the Hamming 

distance between two sequences of symbols is defined as 

the number of positions where the symbols are different. 

A secondary objective is to maximize the product distance, 

defined as the product of the nonzero-squared Euclidean 

distances between the symbols in the same position of two 

sequences having the same beginning and the same end [21], 

[22]. 

The concept of time diversity plays a crucial role in the 

performance of coded modulation for fading channels. Independent 

fading in the different symbols is established by means 

of interleaving. Full interleaving can greatly reduce the re- 

0018–9545/99$10.00 © 1999 IEEE

1476 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 48, NO. 5, SEPTEMBER 1999 

quired transmit power on fading channels. A block interleaver 

can be regarded as a buffer with d rows which represent the 

depth of interleaving, and s columns which represent the span 

of interleaving. In this paper, we do not address the problem of 

interleaver design. However, we suppose that the transmission 

chain includes an interleaver/deinterleaver. 

Our work has been motivated by the need of digital mobile 

communication systems having higher transmission rates by 

keeping the bandwidth as low as possible. One way to achieve 

this goal is to use trellis-coded QAM instead of constantamplitude 

modulation schemes. When using QAM, it is a 

well-established fact that larger Hamming distances and lower 

average energy of the signal constellations can be obtained 

going from 2-D to a higher dimension (e.g., 4-D or more). 

In this paper, we propose a novel way of partitioning the 2-D 

constituent QAM constellations, both in subsets with enlarged 

minimum Euclidean distance and subrings including equal 

energy points. We have thus obtained partitions of the 2N- Fig. 1. Twelve-point 2-D constellation partitioned into four subsets 

D signal sets into subsets with a Hamming distance between (A; B; C; D) and into three rings (R 0 ;R 1 ;R 2 ). 

points which equals N. According to Divsalar and Simon 

[7], this is fundamental for constructing good trellis codes to Sixteen 4-D types may then be defined, each corresponding 

to a concatenation of two 2-D subsets, and denoted 

transmit data over flat fading channels. 

The paper is organized as follows. In Section II, we consider as (A; A); (A; B); 111; and (D; D). The 16 4-D types are 

4-D rectangular signal constellations which are also used grouped into four subsets with Hamming distance between 

for AWGN channels, but partition them in a novel way in types d H = 2in two different ways. 

order to maximize the Hamming distance d H between the Partition I: The first partition is performed as follows: 

points of the same subset. We then design TCM schemes 

SS 

to transmit b = 3 and 4 information bits per signaling 

0 =(A; A) [ (B; B) [ (C; C) [ (D; D) (2a) 

interval using QAM. While for b = 3, there exist schemes 

SS 1 =(A; C) [ (B; D) [ (C; B) [ (D; A) (2b) 

for both PSK and QAM, the data rate b = 4bits/signaling 

SS 2 =(A; B) [ (B; A) [ (C; D) [ (D; C) (2c) 

interval can only be obtained with QAM. Section III describes 

SS 3 =(A; D) [ (B; C) [ (C; A) [ (D; B): (2d) 

the transmission chain including the interleaver/deinterleaver 

and some considerations are given to the decoding strategy. Note with reference to Fig. 1 that these four subsets are 

In this section, we also present an efficient computer-based invariant under 90 , 180 , and 270 

rotation. 

technique to simulate realistic mobile radio channel scenarios. Partition II: The even-indexed subsets are the same as 

The performance of the proposed trellis-coded QAM system is before, but the odd-indexed ones are replaced by 

then investigated in Section IV. Finally, Section V concludes 

SS 0 1 =(A; C) [ (B; D) [ (C; A) [ (D; B) 

our paper. 

(2b’) 

SS 0 3 =(A; D) [ (B; C) [ (C; B) [ (D; A): (2d’) 

II. 4-D TRELLIS-CODED MODULATION 

Note that these two subsets are invariant under 180 , but they 

are not under 90 

In this section, we will consider 4-D rectangular signal 

and 270 rotation. 

constellations to transmit b = 3 and 4 bits per signaling 

Let us consider two generic 4-D points of coordinates 

(x 

interval using QAM. The points of the 2-D constituent signal n ;y n ;x n+1 ;y n+1 ) and (x 0 n ;y0 n ;x0 n+1 ;y0 n+1 ). Define the 

product distance (PD) between these two points as 

constellation belong to a rectangular grid and have odd integer 

coordinates. In other words, if Z is the set of integers, then 

PD = [(x n 0 x 0 

the coordinates of the 2-D points belong to the set f2Z +1g 2 n )2 +(y n 0 y 0 n )2 ] 

. 

2 [(x n+1 0 x 0 n+1 

Then, following Wei [4] and with reference to Fig. 1, we 

)2 +(y n+1 0 y 0 n+1 )2 ]: (3) 

partition this infinite set into four 2-D subsets A; B; C; and It can be verified by looking at Fig. 1 that for the Partition 

D according to 

I the intrasubset minimum product distance (MPD) is 16 for 

four subsets S 

A = f4Z +1g 2 

0 ;S 1 ;S 2 ; and S 3 , the other twelve having an 

(1a) MPD of 64. For Partition II, only six subsets S 4 ; S 6 ; S 8 ; 

B = f4Z +3g 2 

(1b) S 10 ;S 12 , and S 14 have an MPD of 64, the other ten subsets 

C = f4Z +1gf4Z +3g 

(1c) having an MPD of 16. Taking into consideration the less good 

rotational and product distance properties of the Partition II, 

D = fZ +3gf4Z +1g: 

(1d) 

we will not use it. 

If we denote the minimum-squared Euclidean distance Two coded bits, let us say Z0 p and Z1 p , where p = n and 

(MSED) of the set f2Z +1g 2 as 2 0 

, then the MSED of n +1, are used to select one out of the four 2-D subsets as 

every subset A; B; C; and D is 4 2. 0 

shown in Table I.

STERIAN et al.: TRELLIS-CODED QUADRATURE AMPLITUDE MODULATION 1477 

TABLE I 

CORRESPONDENCE BETWEEN Z1 pZ0 p AND THE FOUR 2-D SUBSETS 

TABLE II 

CORRESPONDENCE BETWEEN Z3 pZ2 p AND THE THREE RINGS 

A. Partition of the 4-D Signal Constellation for 

Transmitting Three Bits per Signaling Interval 

Let us define the norm or energy of a 2N-D point of 

coordinates (x 1 ;y 1 ; 111;x N ;y N ) as the squared distance to 

the origin, i.e., 

E = 

NX 

i=1 

0 

x 

2 

i + y2 i1 

: (4) 

Let us furthermore define a length N frame as the concatenation 

of N 2-D points in the signal constellation. Use now 

a technique called shell mapping which is applied in the 

V.34 voice-band high-speed modem [3]. For N an integer 

power of two, partition the (1+1=N )2 b point 2-D constituent 

constellation of the 2N-D signal set into N inner rings and 

one outer ring of equal size 2 b =N . In our case, where we 

have N =2, the first inner ring R 0 contains 2 b01 2-D points 

of least norm, and the second inner ring R 1 contains the next 

2 b01 points such that, taken together, ring R 0 and ring R 1 

form the 2 b point signal constellation which would be used by 

the uncoded reference system to transmit b bits per signaling 

interval. The outer ring R 2 contains 2 b01 redundant points 

chosen in ascending order of the norm. The 2 2b+1 point 4- 

D constellation is the union of the Cartesian products of the 

rings (R i ;R j ) such that at most one ring R i or R j is the outer 

ring. Define a 4-D inner point as a 4-D point which belongs 

to an inner subset (R 0 ;R 0 ); (R 0 ;R 1 ); (R 1 ;R 0 ) or (R 1 ;R 1 ) 

and a 4-D outer point as one which belongs to the subsets 

(R 0 ;R 2 ); (R 1 ;R 2 ); (R 2 ;R 0 ) and (R 2 ;R 1 ). The probability 

of sending a 4-D inner point equals the probability of sending 

a 4-D outer point and is 1/2. However, for each constituent 

2-D constellation, an inner ring is used three times (generally 

2N 0 1 times) as often as the outer ring R 2 . This produces 

the well-known effect of shaping the constellation and results 

in a small gain of the signal-to-noise ratio [3]. 

Note that, when applying this known method, we have 

further partitioned each of the (N +1)rings into 2 b03 fourpoint 

subrings such that any 2-D point may be obtained by 

rotating any other 2-D point within a given subring by 90 , 

180 , or 270 . 

Two coded bits, let us say Z2 p and Z3 p , where p = n and 

n +1, are used to select one out of three rings as shown in 

Table II. 

Using these rings, partition the 4-D signal constellation into 

four subsets called shells as follows: 

SH 0 =(R 0 ;R 0 ) [ (R 1 ;R 1 ) 

SH 1 =(R 0 ;R 1 ) [ (R 1 ;R 0 ) 

SH 2 =(R 0 ;R 2 ) [ (R 2 ;R 0 ) 

SH 3 =(R 1 ;R 2 ) [ (R 2 ;R 1 ): 

(5) 

The partition has been done in such a way that the Hamming 

distance between the ring types inside a given shell is equal 

to d H = 2. 

Combining the four subsets SS i defined by (2.a)–(2.d) with 

the four shells SH j , we obtain 16 4-D subsets S k (k = 

0; 1; 111; 15) as shown in Table III. Note that the index k of 

the subset S k is given by the relation 

k =4j + i: (6) 

The decimal values of the indices i and j are given by (see 

Fig. 2) 

and 

i =2I1 n + Y 0 n (7) 

j =2I3 n + I2 n (8) 

respectively. The 16 subsets S k are numbered from 0 to 15. 

Group the 16 subsets into two families F 0 and F 1 as follows: 

F 0 = 

F 1 = 

[ 

[ 

k=even 

k=odd 

Sk 

Sk: 

(9a) 

(9b) 

As it may be seen from Table III, every one of the 16 subsets 

contains eight 4-D points which are different of any other one 

in both the first and the second 2-D component of it. Use now 

this partition for TCM with b = 3. 

B. Design of 4-D Trellis Codes for Transmitting 

Three Bits per Signaling Interval 

The 12-point 2-D constituent constellation of the 4-D signal 

set is shown in Fig. 1. For each point, the capital letter 

indicates the subset A; B; C; or D and the number refers to 

the ring 0, 1, or 2. To send b =3information bits per signaling 

interval using a rate 3/4 trellis code with a 4-D constellation 

partitioned into 2 4 =16subsets, three of the six information 

bits (I1n;I2n;I3n) arriving in each block of two signaling 

intervals enter the trellis encoder, and the resulting four coded 

bits specify which 4-D subset is to be used. The remaining 

three information bits (I1n+1;I2n+1;I3n+1) specify which 

point from the selected 4-D subset is to be transmitted. 

Denote the six information bits gathered at the input of the 

trellis-coded modulator in two successive signaling intervals 

n and n +1as I1n; I2n; I3n; I1n+1; I2n+1, and I3n+1. As 

shown in Fig. 2, the first three bits enter a rate 3/4 systematic 

convolutional encoder which outputs the coded bit Y 0n. 

In order to make the scheme transparent to all the phase 

ambiguities of the constellation, we choose the bit pair 

I3n+1I2n+1 and differentially encode it in such a way that if 

we translate a sequence of this bit pair by the same number


TABLE III 

PARTITION OF THE 128-POINT 4-D SIGNAL SET INTO 16 SUBSETS Sk (k =0; 1; 111; 15) 

of positions, one, two, or three, in a circular sequence, 00, 01, 

10, 11, then the sequence of 2-D points produced by the 4-D 

constellation mapping procedure will be rotated by 90 , 180 , 

and 270 clockwise, respectively. Therefore, a differential 

encoder of the form 

I3 0 n I20 n =(I30 n02 I20 n02 + I3 nI2 n ) mod 100 base2 (10) 

shown in Fig. 2 and a corresponding differential decoder of 

the form 

I3 n I2 n =(I3 0 n I20 n 

0 I3 0 n02 I20 n02 ) mod 100 base2 (11) 

at the output of the trellis decoder will remove all the phase 

ambiguities of the constellation [2], [4], [5]. 

A bit converter (see Fig. 2) converts the four bits Y 0 n ; 

I1 n ; I2 0 n+1 ; and I30 n+1 

into two pairs of selection bits 

Z0 n Z1 n and Z0 n+1 Z1 n+1 , which are used to select the 

pair of 2-D subsets corresponding to the 4-D type. With the 

correspondence between the bit pair Z0 p Z1 p and the 2-D 

subsets A; B; C; and D as shown in Table I, the operation of 

the bit converter for the Partition I is as shown in Table IV. 

A 4-D block encoder then takes three input information bits 

I2 n ;I3 n ; and I1 n+1 and generates two pairs of selection bits, 

Z2 n Z3 n and Z2 n+1 Z3 n+1 , in accordance with Table V. Each 

of the bit pairs can assume any of the values given in Table II, 

but they cannot both assume the value ten which corresponds 

to the outer ring R 2 . 

Fig. 2. General structure for rotationally invariant TCM with 4-D QAM to 

send b =3bits per signaling interval. 

We will design now two convolutional encoders which fit 

in the general diagram shown in Fig. 2. Note that every one 

of the 16 4-D subsets contains eight 4-D points which are 

different from each other in both the first and the second 

2-D component. Therefore, the intraset Hamming distance 

of the 16 subsets is maximized to d H = 2. However, the 

interset Hamming distance only equals one. Recall that our 

aim is not to maximize the Euclidean distance between allowed 

sequences, but the Hamming distance. 

1) Eight-State Convolutional Encoder: Denote the current 

and the next states of the trellis encoder as W 1 p W 2 p W 3 p ; 

p = n and n +2. Let us number the states from 0 to 7 by 

using the relation 

W p =4W 1 p +2W 2 p + W 3 p : (12)


(a) 

Fig. 3. 

(b) 

(a) Trellis diagram of eight-state code of Figs. 2 and 4(a) and (b) trellis diagram of 16-state code of Figs. 2 and 4(b). 

The trellis diagram is as shown in Fig. 3(a). It is fully 

connected and we may express the mapping Wn ! Wn+2 

in algebraic form as 

f0; 1; 2; 3; 4; 5; 6; 7g !f0; 1; 2;3; 4; 5; 6;7g: 

The association of 4-D subsets with the state transitions 

satisfies the following requirement. 

Rule 1: The 4-D subsets associated with the transitions 

originating from a state are different from each other and 

belong to the same 4-D family F 0 or F 1 ; the 4-D subsets 

associated with the transitions leading to a state are different 

from each other, but may belong to both families F 0 and F 1 . 

The logic diagram of the eight-state convolutional encoder is 

given in Fig. 4(a). 

The shortest error event path is given by parallel paths 

between successive states of the convolutional encoder. Indeed, 

although drawn as a single one, there are eight parallel 

transitions between two successive states in the trellis diagram


(a) 

Fig. 4. 

Trellis encoders of Fig. 2: (a) eight-state and (b) 16-state. 

(b) 

TABLE IV 

PARTITION OF 4-D 128 POINT RECTANGULAR CONSTELLATION INTO 16 TYPES 

TABLE V 

THE 4-D BLOCK ENCODER FOR b =3 

shown in Fig. 3. The Hamming distance between these parallel 

transitions is d H =2. However, the two transitions error event 

paths also have d H = 2and the same multiplicity as single 

transition error event paths. This is since the trellis diagram 

in Fig. 3 has full connectivity. To improve the performance, a 

convolutional encoder with a larger number of states must be 

used in the general structure shown in Fig. 2. 

2) 16-State Convolutional Encoder: Denote the current 

and the next states of the trellis encoder as 

W 1 p W 2 p W 3 p W 4 p ; p = n and n + 2. Number the 

states from 0 to 15 using the relation 

W p =8W 1 p +4W 2 p +2W 3 p + W 4 p : (13) 

The trellis diagram is as shown in Fig. 3(b) and may be also 

expressed in algebraic form as 

f0; 2; 4; 6; 8; 10; 12; 14g !f0; 1; 2; 3; 4; 5; 6; 7g 

f1; 3; 5; 7; 9; 11; 13; 15g !f8; 9; 10; 11; 12; 13; 14; 15g:


Fig. 5. Twenty-four-point 2-D constellation partitioned into four subsets (A; B; C; D) and into six rings (R 0 ;R 1 ;R 2 ;R 3 ;R 4 ;R 5 ). 

There are eight parallel transitions between any current state 

i and a successive state j. Therefore, in this case also, the 

shortest error event path has a length equal to one transition. 

However, the multiplicity of two transitions error event paths 

has been halved. 

The association of 4-D subsets with the state transitions 

should satisfy the following requirement. 

Rule 2: The 4-D subsets associated with the transitions 

originating from a state are different from each other and 

belong to the same 4-D family F 0 or F 1 and likewise for 

the 4-D subsets associated with the transitions leading to a 

state. The logic diagram of the 16-state convolutional encoder 

is given in Fig. 4(b). 

C. Partition of the 4-D Signal Constellation and Design of 

Trellis Codes for Transmitting Four Bits per Signaling Interval 

The transmission rate of b = 4bits per signaling interval 

is clearly not possible using TCM with a PSK constellation 

(32-PSK has unacceptably small MSED). The 24-point 2-D 

constituent constellation of the 4-D signal set is shown in 

Fig. 5. 

As for b =3, the 2-D constellation is partitioned into four 

subsets A; B; C; and D, but the number of subrings is six, 

numbered from 0 to 5 in ascending order of the norm. Three 

bits, let us say Z2 p ;Z3 p ; and Z4 p ; where p = n and n +1, 

are used to select one out of these six subrings as shown in 

Table VI. 

TABLE VI 

CORRESPONDENCE BETWEEN Z4 pZ3pZ2p AND THE SIX SUBRINGS 

Using these subrings, partition the 4-D signal constellation 

into eight subsets called shells as follows: 

SH 0 =(R 0 ;R 0 ) [ (R 1 ;R 1 ) [ (R 2 ;R 2 ) [ (R 3 ;R 3 ) 

SH 1 =(R 0 ;R 1 ) [ (R 1 ;R 0 ) [ (R 2 ;R 3 ) [ (R 3 ;R 2 ) 

SH 2 =(R 0 ;R 2 ) [ (R 2 ;R 0 ) [ (R 1 ;R 3 ) [ (R 3 ;R 1 ) 

SH 3 =(R 0 ;R 3 ) [ (R 3 ;R 0 ) [ (R 1 ;R 2 ) [ (R 2 ;R 1 ) 

SH 4 =(R 0 ;R 4 ) [ (R 4 ;R 0 ) [ (R 1 ;R 5 ) [ (R 5 ;R 1 ) 

SH 5 =(R 1 ;R 4 ) [ (R 4 ;R 1 ) [ (R 0 ;R 5 ) [ (R 5 ;R 0 ) 

SH 6 =(R 2 ;R 4 ) [ (R 4 ;R 2 ) [ (R 3 ;R 5 ) [ (R 5 ;R 3 ) 

SH 7 =(R 3 ;R 4 ) [ (R 4 ;R 3 ) [ (R 2 ;R 5 ) [ (R 5 ;R 2 ): 

(14) 

The partition has been done in such a way that the Hamming 

distance between the subring types inside a given shells is 

equal to d H = 2. 

We combine the four subsets SS i defined as before with 

the eight shells SH j defined by (14) to obtain 32 4-D subsets 

such that the relation (6) still holds, but in this case j goes 

from 0 to 7. For instance, the subset S 0 =SS 0 2 SH 0 contains


TABLE VII 

THE 4-D BLOCK ENCODER FOR b =4 

16 4-D points as follows: (A n ;A n ); (B n ;B n ); (C n ;C n ); and 

(D n ;D n ), where n =0; 1; 2; 3. The 32 subsets are numbered 

from 0 to 31 and grouped into two families F 0 and F 1 as in (9). 

Denote the eight bits gathered at the input of the trellis-coded 

modulator in two successive signaling intervals n and n +1 

as I1n; I2n; I3n; I4n; I1n+1; I2n+1; I3n+1; and I4n+1. As 

shown in Fig. 6, the first four bits enter a rate 4=5 systematic 

convolutional encoder which outputs the coded bit Y 0n. In 

order to make the scheme rotationally invariant, differentially 

encode the bit pair I3n+1I2n+1 as for the case b = 3.A 

bit converter converts the four bits Y 0n; I1n; I2 0 n+1; and 

I3 0 n+1 

into two pairs of selection bits as for the case b =3. 

A 4-D block encoder then takes five input information bits 

I2n; I3n; I4n; I1n+1; and I4n+1 and generates two groups 

of selection bits, Z2nZ3nZ4n and Z2n+1Z3n+1Z4n+1 in 

accordance with Table VII. Each of the bit groups can assume 

any of the values given in Table VI, but they cannot both 

assume the values 100 and 101 which correspond to the outer 

subrings R 4 and R 5 , respectively. 

The general diagram of the TCM scheme for b =4is shown 

in Fig. 6. The convolutional encoder should have at least 16 

states 


and the next states of the trellis encoder as in the case of 

the 16-state convolutional encoder in Fig. 4(b). However, 

the trellis has full connectivity, i.e., from any of the 16 

originating states, 16 groups of transitions lead to any of the 

Fig. 6. General structure for rotationally invariant TCM with 4-D QAM to 

send b =4bits per signaling interval. 

16 next states. The association of 4-D subsets with the state 

transitions satisfies the Rule 1 as given for the case b =3. The 

logical diagram of the convolutional encoder which is part of 

the general structure in Fig. 6 is shown in Fig. 7(a). 

The shortest error event path is given by parallel transitions 

between successive states of the convolutional encoder. The 

Hamming distance between the 16 parallel transitions is d H = 

2. Note that the two transitions error event paths also have 

d H = 2and the same multiplicity as single transitions error 

event paths. 


and the next states of the trellis encoder as 

W 1 p W 2 p W 3 p W 4 p W 5 p ;p = n and n + 2. Number the


(a) 

Fig. 7. 

Trellis encoders of Fig. 6: (a) 16-state and (b) 32-state. 

(b) 

states from 0 to 31 using the decimal representation 

W p =16W 1 p +8W 2 p +4W 3 p +2W 4 p + W 5 p : (15) 

The trellis diagram may be expressed in algebraic form as 

and 

f0; 2; 4; 6; 8; 10; 12; 14; 16; 18; 20; 22; 24; 26; 28; 30g 

!f0; 1; 2; 3; 4; 5; 6; 7; 8; 9; 10; 11; 12; 13; 14; 15g 

f1; 3; 5; 7; 9; 11; 13; 15; 17; 19; 21; 23; 25; 27; 29; 31g 

!f16; 17; 18; 19; 20; 21; 22; 23; 24; 25; 

26; 27; 28; 29; 30; 31g: 

There are 16 parallel transitions between any current state i and 

a successive state j. Therefore, in this case also, the shortest 

event path has a length equal to one transition. However, the 

multiplicity of two transitions error event path is only half of 

that for the 16-state convolutional encoder. The association 

of 4-D subsets with the state transitions satisfies the Rule 

2 as given for the case b = 3. The logical diagram of the 

convolutional encoder is shown in Fig. 7(b). 

The authors have also designed 4-D trellis codes for transmitting 

b = 5 bits per signaling interval and 6-D trellis 

codes based on signal constellations partitioned such that the 

Hamming distance between the points within a subset equals 

three. These codes have not been included in this paper by 

considerations of typographical space. The interested readers 

are kindly invited to contact the authors. 

III. TRANSMISSION SYSTEM AND CHANNEL MODEL 

In this section, we describe the transmission system and 

the channel model by making use of the equivalent complex 

baseband notation. 

A. The Transmission System 

The transmission system we consider is presented in Fig. 8. 

A data source generates a random information bit stream that 

enters the foregoing described trellis encoder. The encoded 

2-D output symbols are then interleaved by a block interleaver 

which can be regarded as a rectangular buffer with d rows 

and s columns representing the interleaving depth and span, 

respectively. The M-QAM modulator maps the block interleaved 

2-D symbols to the signal points of the signal space 

diagram for the M-QAM constellation shown in Fig. 1. 

The transmitted signal is impaired first by a complex multiplicative 

stochastic process describing the fading behavior of 

the frequency-nonselective mobile radio channel and second 

by a complex AWGN process. In the receiver, the received 

signal is demodulated and deinterleaved before feeding into the 

trellis decoder. The trellis decoder is based on the maximum 

likelihood sequence decoding principle by using the classical 

Viterbi algorithm. It is well known that the performance of 

the trellis decoder can be considerably improved if channel 

state information (CSI) is available. To obtain CSI from the 

received signal a channel estimator is required in the receiver. 

B. The Channel Model 

An often used statistical model for modeling various types 

of terrestrial mobile radio channels and especially for land 

mobile satellite channels is the well-known Suzuki process 

[14]. Such a process is defined as a product process of a 

Rayleigh process with uncorrelated underlying in-phase and 

quadrature components and a lognormal process. Recently, 

two different modified versions of the classical Suzuki process 

have been introduced in [15] and [16] which are called 

extended Suzuki processes of Types I and II, respectively. 

Moreover, it has been shown [17] that both types of extended 

Suzuki processes can be combined to a joint statistical channel 

model called generalized Suzuki process. The main advantage


Fig. 8. 

Equivalent complex baseband model of the trellis-coded M-QAM transmission system. 

of extended and in particular generalized Suzuki processes is 

that their statistical properties are more flexible than those 

of the original Suzuki process. Thus, the former processes 

allow in general a much better fitting of the statistics of the 

channel model to real-world measurements and that not only 

with respect to different kinds of measured probability density 

functions of the received envelope, but also with respect to 

the corresponding higher order statistical properties like the 

level-crossing rate and the average duration of fades. 

In this paper, we use for the channel model the extended 

Suzuki process of Type I which is briefly reviewed in the 

following. For a detailed description of that process, we are 

referring the interested reader to [15]. The extended Suzuki 

process (of Type I), denoted henceforth by (t), is defined 

as product process of a Rice process (t) with given crosscorrelation 

properties between the underlying in-phase and 

quadrature components and a lognormal process (t), i.e., 

(t) =(t) 1 (t): (16) 

The Rice process (t) is obtained from a zero-mean complex 

Gaussian noise process 

(t) = 1 (t) +j 2 (t) (17) 

representing the scattered (diffuse) component and a complex 

line-of-sight (LOS) component 

as follows: 

m(t) = 1 expfj(2f t + )g (18) 

(t) =j(t) +m(t)j: (19) 

Thereby, the parameters ; f ; and appearing in (18) are 

the amplitude, Doppler frequency, and phase of the LOS 

component, respectively. The Rice process (t) is used to 

model the short-term fading effects, whereas the lognormal 

process (t) models the long-term fading variations due to 

shadowing. The lognormal process (t) can be derived from 

a nonlinear transformation of a further real Gaussian noise 

process 3 (t) having zero mean and unit variance according 

to 

(t) = expf 3 3 (t) +m 3 g (20) 

where 3 and m 3 are parameters introduced to control the 

statistics of (t). 

The second-order statistical properties of the extended 

Suzuki process (t) are strongly influenced by the Doppler 

power spectral density (PSD) S (f) of the complex Gaussian 

noise process (t) introduced by (17). Typical for the extended 

Suzuki model is that the complex Gaussian noise process (t) 

has cross-correlated in-phase and quadrature components. 

A cross-correlation between the generating components can 

easily be achieved by using an asymmetrical Doppler PSD 

S (f), e.g., the left-sided restricted Jakes PSD which is 

defined by [15] 

S (f) = 

8 

< 

: 

20 

2 

f max 

p1 0 (f=f ; 0 0f max f f max 

max ) 2 

0; else 

(21) 

where f max denotes the maximum Doppler frequency, the 

parameter 0 is within the interval [0; 1], and 20 2 defines 

the maximum mean power (variance) of (17) obtained for 

0 = 1. Note that for the special case where 0 = 1, the 

relation (21) results in the classical Jakes PSD [20] which 

has a symmetrical shape, and, consequently, the in-phase and 

quadrature components of the complex Gaussian noise process 

(t) are in this case uncorrelated. On the other hand, if 0 =0, 

then (21) results in an asymmetrical right-sided Jakes PSD 

and thus the in-phase and quadrature components of (t) are 

strongly correlated. It is also important to note that the fading 

rate of the channel model can easily be reduced (without 

changing the maximum Doppler frequency f max ) by reducing 

the quantity 0 . This is important because the fading rate of 

real-world mobile radio channels is often much lower than the 

theoretical expected fading rate. 

The above parameters 0 2 ; 0;f max ;; 3 ;m 3 ; and f are 

the primary model parameters of the extended Suzuki process.


TABLE VIII 

OPTIMIZED PARAMETERS OF THE EXTENDED SUZUKI CHANNEL MODEL [15] 

Fig. 9. Structure of an efficient simulation model for extended Suzuki processes of Type I. 

These parameters have been optimized successfully in [15] in 

such a way that not only the cumulative distribution function 

of (t) but also the level-crossing rate and average duration 

of fades are very close to measured data of a real-word 

land mobile satellite channel in different (light and heavy) 

shadowing environments. The optimized primary parameters 

of the channel model are listed in Table VIII. 

The above-described extended Suzuki process (t) is an 

analytical (mathematical) model that cannot be implemented 

exactly on a computer. In order to enable the simulation of 

such processes, an efficient simulation model was also derived 

in [15] (see Fig. 9) by applying the concept of deterministic 

channel modeling (e.g., [18] and [19]). The parameters ; 3 ; 

m 3 , and f appearing in Fig. 9 are obtained directly from 

Table VIII, whereas the remaining parameters of the simulation 

model (f i;n ;c i;n ; i;n ) have to be determined according 

to the procedure described in [15]. 

IV. PERFORMANCE 

The discussion of the performance of the proposed 2N-D 

trellis-coded M-QAM system is restricted for short to the case 

N =2 and M =12. A 4-D trellis code for transmitting 

b =3bits per signaling interval was designed according to 

the procedure described in Section II-B. For the convolutional 

encoder (see Fig. 2) the 16-states trellis encoder shown in 

Fig. 4(b) has been selected. For the depth d and span s 

describing the interleaver (deinterleaver) the moderate values 

d =16and s =16have been chosen. Hence, due to the finite 

interleaver (deinterleaver) size, the digital channel between 

the interleaver input and the deinterleaver output is nonideally 

interleaved (correlated fading). Furthermore, we have assumed 

that ideal CSI is available within the decoding unit. The trellis 

decoder operates on optimum (unquantized) soft decisions 

made by the 12-QAM demodulator; and finally, the decoding 

depth L of the Viterbi algorithm is finite (L = 16). The 

simulation of the complete trellis-coded 12-QAM transmission 

system shown in Fig. 8 has been performed by choosing a 

symbol rate to sampling rate ratio of f S =f A =1=8 and symbol 

rate to maximum Doppler frequency ratio of f S =f max =0:02. 

The resulting bit error rate (BER) of the trellis-coded 12- 

QAM transmission system by using the extended Suzuki 

channel model with two realistic scenarios (light and heavy 

shadowing) are presented in Fig. 10. For reasons of comparison, 

we have also shown in this figure the simulation results 

for the BER by using an AWGN channel and a Rayleigh


The obvious disadvantage of these schemes is that the trellis 

encoders have rather a large number of states. 

When moving to higher dimensions (the next step is 6-D), 

the benefits diminish quickly and the complexity grows unacceptably 

high. 

Fig. 10. BER of the trellis-coded 12-QAM transmission system with b =3 

bits per signaling interval. 

channel in conjunction with the classical Jakes PSD. It should 

be mentioned that the extended Suzuki process includes the 

Rayleigh process as a special case. The results in Fig. 10 show 

us that, for a large range of the Eb 

=N 0 ratio, the standard 

Rayleigh channel model does not represent the worst case 

condition, but the extended Suzuki process on the heavy 

shadowing condition does it. On the other side, the BER 

determined for the light shadowing situation is for a given 

E b =N 0 ratio always below the corresponding results obtained 

for the Rayleigh channel, as it was intuitively expected. 

A profound insight into the performance of the proposed 

scheme is obtained by comparing the BER of our b =34-D 

TCM system with the BER of an appropriate reference system. 

As reference system we used an uncoded 8-PSK system which 

has the same information bit rate as the proposed trellis-coded 

12-QAM transmission system with b =3bits per signaling 

interval. The resulting BER of the reference system is also 

depicted in Fig. 10. The presented results show us that the 

coding gain ranges from 2.0 dB (AWGN) up to 5.4 dB (heavy 

shadowing) at a BER of 10 03 . Note that the proposed TCM 

scheme has especially been designed for fading channels, what 

explains the fact that the achieved coding gain is higher for 

the heavy shadowing condition than for the AWGN channel. 

V. CONCLUSION 

A novel way of designing TCM codes for radio mobile fading 

channels using rectangular signal constellations has been 

demonstrated. In order to obtain large minimum intrasubset 

Hamming distance, the signal constellation is partitioned both 

into subsets with enlarged minimum Euclidean distance and 

into shells. It was inspired by the methods applied in the very 

performant V.34 modem used to transmit data over the voiceband 

telephone channel, which of course is not affected by 

fading. However, our aim in so doing was different. 

For 4-D, this resulted in rather simple TCM schemes which 

have the advantage of maximizing the minimum intrasubset 

Hamming distance without neglecting the Euclidean distance. 

ACKNOWLEDGMENT 

Dr. Sterian would like to thank the German Service for 

Academic Exchanges DAAD (Deutscher Akademischer Austauschdienst) 

which provided him with the opportunity to 

spend a fruitful month in the Department of Digital Communications 

Systems of the Technical University Hamburg- 

Harburg, working in the research group of Prof. U. Killat. 

This international collaboration has been kindly encouraged 

by S. Pantis, Minister, and D. Chirondojan, Secretary of State, 

both with the Ministry of Communications of Romania. 

REFERENCES 

[1] G. Ungerboeck, “Channel coding with multilevel/phase signals,” IEEE 

Trans. Inform. Theory, vol. IT-28, pp. 55–67, Jan. 1982. 

[2] L.-F. Wei, “Rotationally invariant convolutional channel coding with 

expanded signal space—Part I: 180 degrees and Part II: Nonlinear 

codes,” IEEE J. Select. Areas Commun., vol. SAC-2, pp. 659–686, Sept. 

1984. 

[3] ITU-T, “V.34—A modem operating at data signalling rates of up to 

33600 bit/s for use on the general switched telephone network and on 

leased point-to-point 2-wire telephone-type circuits,” Sept. 1994. 

[4] L.-F. Wei, “Trellis-coded modulation with multidimensional constellations,” 

IEEE Trans. Inform. Theory, vol. IT-33, pp. 483–501, July 

1987. 

[5] C. E. D. Sterian, “Wei-type trellis-coded modulation with 2Ndimensional 

rectangular constellation for N not a power of two,” 

IEEE Trans. Inform. Theory, vol. 43, pp. 750–758, Mar. 1997. 

[6] D. Divsalar and M. K. Simon, “Trellis coded modulation for 4800–9600 

bits/s transmission over a fading mobile satellite channel,” IEEE J. 

Select. Areas Commun., vol. SAC-5, pp. 162–175, Feb. 1987. 

[7] , “Multiple trellis coded modulation (MTCM),” IEEE Trans. 

Commun., vol. 36, pp. 410–419, Apr. 1988. 

[8] C.-E. W. Sundberg and N. Seshadri, “Coded modulations for fading 

channels: An overview,” European Trans. Telecommun. (ETT), vol. 4, 

pp. 309–324, May–June 1993. 

[9] C. Schlegel, “Trellis coded modulation on time-selective fading channels,” 

IEEE Trans. Commun., vol. 42, pp. 1617–1627, Feb./Mar./Apr. 

1994. 

[10] D. Subasinghe-Dias and K. Feher, “A coded 16 QAM scheme for fast 

fading mobile radio channels,” IEEE Trans. Commun., vol. 43, pp. 

1906–1916, May 1995. 

[11] X. Giraud and J. C. Belfiore, “Constellations matched to the Rayleigh 

fading channels,” IEEE Trans. Inform. Theory, vol. IT-42, pp. 106–115, 

Jan. 1996. 

[12] J. Boutos, E. Viterbo, C. Rastello, and J. C. Belfiore, “Good lattice 

constellations for both Rayleigh and Gaussian channels,” IEEE Trans. 

Inform. Theory, vol. 42, pp. 502–518, Mar. 1996. 

[13] X. Giraud and E. Boutillon, “Algebraic tools to build modulation 

schemes for fading channels,” IEEE Trans. Inform. Theory, vol. 43, 

pp. 938–952, May 1997. 

[14] H. Suzuki, “A statistical model for urban radio propagation,” IEEE 

Trans. Commun., vol. 25, pp. 673–680, July 1977. 

[15] M. Pätzold, U. Killat, and F. Laue, “An extended Suzuki model for land 

mobile satellite channels and its statistical properties,” IEEE Trans. Veh. 

Technol., vol. 47, pp. 254–269, Feb. 1998. 

[16] M. Pätzold, U. Killat, Y. Li, and F. Laue, “Modeling, analysis, and 

simulation of nonfrequency-selective mobile radio channels with asymmetrical 

Doppler power spectral density shapes,” IEEE Trans. Veh. 

Technol., vol. 46, pp. 494–507, May 1997. 

[17] M. Pätzold and F. Laue, “Generalized Rice processes and generalized 

Suzuki processes for modeling frequency-nonselective mobile radio 

channels,” to be published. 

[18] M. Pätzold, U. Killat, and F. Laue, “A deterministic digital simulation 

model for Suzuki processes with application to a shadowed Rayleigh


land mobile radio channel,” IEEE Trans. Veh. Technol., vol. 45, pp. 

318–331, May 1996. 

[19] M. Pätzold, U. Killat, F. Laue, and Y. Li, “On the statistical properties 

of deterministic simulation model for mobile fading channels,” IEEE 

Trans. Veh. Technol., vol. 47, pp. 617–630, May 1998. 

[20] W. C. Jakes (Ed.), Microwave Mobile Communications. New York: 

IEEE Press, 1993. 

[21] D. Divsalar and M. K. Simon, “The design of trellis coded MPSK for 

fading channels: Performance criteria,” IEEE Trans. Commun., vol. 36, 

pp. 1004–1012, Sept. 1988. 

[22] , “The design of trellis coded MPSK for fading channels: Set 

partitioning for optimum code design,” IEEE Trans. Commun., vol. 36, 

pp. 1004–1012, Sept. 1988. 

[23] L.-F. Wei, “Coded M-DPSK with built-in time diversity for fading 

channels,” IEEE Trans. Inform. Theory, vol. 39, pp. 1820–1839, Nov. 

1993. 

Corneliu Eugen D. Sterian (M’96–SM’98) was 

born in Bucharest, Romania, on April 30, 1947. He 

received the Diploma Engineer and Ph.D. degrees 

in electronics and telecommunications in 1970 and 

1994, respectively, from the Polytechnical University 

of Bucharest, Bucharest. 

He has been active for many years in scientific 

research. He also taught data transmission courses at 

the Faculty of Electronics and Telecommunications, 

Polytechnical University of Bucharest. From 1997 

to 1999, he was with the Ministry of Communications 

as Director General. He authored the book Echo Cancelation 

in Telecommunications (in Romanian) and some articles published in the 

IEEE TRANSACTIONS ON INFORMATION THEORY, European Transactions on 

Telecommunications, and others. His research interests include information 

theory and more particularly channel coding. 

Frank Laue was born in Hamburg, Germany, in 

1961. He received the Dipl.-Ing. (FH) degree in 

electrical engineering from the Fachhochschule 

Hamburg, Hamburg, Germany, in 1992. 

Since 1993, he has been a CAE Engineer at the 

Digital Communication Systems Department, Technical 

University of Hamburg-Harburg, Hamburg, 

where he is involved in mobile communications. 

Matthias Pätzold (M’94–SM’98) was born in Engelsbach, 

Germany, in 1958. He received the Dipl.- 

Ing. and Dr.-Ing. degrees in electrical engineering 

from Ruhr University Bochum, Bochum, Germany, 

in 1985 and 1989, respectively. 

From 1990 to 1992, he was with ANT Nachrichtentechnik 

GmbH, Backnang, where he was engaged 

in digital satellite communications. Since 1992, he 

has been Chief Engineer at the Digital Communication 

Systems Department, Technical University 

Hamburg-Harburg, Hamburg, Germany. His current 

research interests include mobile communications, especially multipath fading 

channel modeling, channel parameter estimation, and channel coding theory.

Trellis-coded quadrature amplitude modulation with ... - IEEE Xplore

Transform your PDFs into Flipbooks and boost your revenue!

Delete template?

Save as template ?