Error Control - From Theory to Near-Capacity Implementation

Error Control - 

From Theory to Near-Capacity 

Implementation 

Pål Orten 

Nera Research 

Outline 

• Why error control? 

• What is possible with error control 

coding? 

• Traditional Coding Schemes 

• New Capacity Approaching Schemes 

• Coding for High Spectral Efficiency 

• Example applications - Nera Products 

© Nera Research/P. Orten 

1

Why Error Control Coding 

Transmitter 

Channel 

Receiver 

• Thermal noise 

• Channel: Fading and inter-symbol interference 

• Interference: from system itself and other 

systems 

• May be combated by channel coding (there are 

other ways also) 


Interference 

Noise 

Principle of error control 

• Add redundancy to enable 

– Error detection 

– Error correction 

– or both 

• d min =10, correct 4 errors, detect d min -1 errors 

C1 

d min 

C2 


2

Some Theory 


Shannon Limit 

⎛ S ⎞ 

C = W ⎜1 

+ ⎟ 

⎝ N ⎠ 

⇔ 

E 

N 

b 

0 

log 2 

C 

( 2 1) 

/ W 

− 

= 

W 

C 

W → ∞ ⇒ E 

/ N0 → −1.6 dB 

No limit on input, AWGN channel 

b 


3

W/C 

8 

6 

4 

2 

W/C versus E b /N 0 

Shannon Limit 

Asymptote -1.6 dB 

0 

-2 0 2 4 6 

© Nera Research/P. Orten E b 

/N 0 

[dB] 

• No limit on 

input 

• AWGN 

channel 

• To reach the 

limit: 

– infinite 

bandwidth 

– infinite code 

word length 

I praksis? 

• For praktiske systemer: 

– koderate R c > 0 

– begrensninger på input (gitt av modulasjon) 

– endelig kodeordslengde 

⇒ 

-1.6 dB grensen vil ikke nås 

• Praktisk grense for 


– koderate eller bit/s/Hz 

– modulasjonsformat 

– kodeordslengde 

4

Effect of Coding 


E b /N 0 

Capacity for given Code Rate 

C 

= 

1 

2 

log 

2 

(1 + 

2RcE 

N 

0 

b 

) 

R 

c 


C 

≤ 

1− 

H ( e) 

b 

Find (P b , E b /N 0 ) pair 

⇒ 

From the converse 

to the coding theorem 

Limit for given code rate 

5

Capacity for different code rates 

10 0 E b 

/N 0 

[dB] 

BER 

10 -2 

R c =0.75 

R c 

=0.5 

R c =0.33 

R c =0.1 

Unconstrained 

AWGN 

Channel 

10 -4 

10 -6 

-2 -1 0 1 2 


Capacity as function of code 

rate - binary input (1) 

Example: Binary-input (+/- A), AWGN channel 

∞ 

1 

p( 

y | A) 

1 

p( 

y | −A) 

C = 

∫ 

p( 

y | A) log 

2 

dy + 

∫ 

p( 

y | −A) log 

2 

dy 

2 

p( 

y) 

2 

p( 

y) 

−∞ 

∞ 

−∞ 

Constrained 

AWGN Channel, 

(From Proakis) 


6

Capacity as function of code 

rate - binary input (2) 

Code Rate 

1 

0.9 

0.8 

0.7 

0.6 

0.5 

0.4 

0.3 

Unconstrained Input 

Binary Input 


0.2 

-1 0 1 2 3 4 5 6 

Capacity Limit E /N [dB] b 0 

Theoretical limit for given 

spectral efficiency 

•Limited bandwidth, W 

•Data rate R (bits/s) 

•Shannons capasity can be expressed as: 

R 

η = 

W 


E b 

N 

0 

η 

2 −1 

≥ 

η 

is spektral-efficiency in bit/s/Hz 

7

Theoretical limit for given 

spectral efficiency (plot) 

Spectral Effeciency [bit/s/Hz] 

5 

4 

3 

2 

1 

0 

-2 0 2 4 6 8 

Capacity Limit E /N (dB) b 0 


Unconstrained 

AWGN 

Channel 

Larger Modulation Alphabets- 

Higher Spectral Efficiency 

Capacity Limit E b 

/N 0 

(dB) 

10 

8 

6 

4 

2 

0 

QPSK 

16-QAM 

64-QAM 

Constrained 

Additive Gaussian 

Channel 

0.3 0.4 0.5 0.6 0.7 0.8 0.9 

Code Rate 


8

Capacity as function of 

spectral efficiency 

Spectral Effeciency [bit/s/Hz] 

6 

5 

4 

3 

2 

1 

0 

-2 0 2 4 6 8 10 

Capacity Limit E /N (dB) b 0 


Unconstrained 

QPSK 

16-QAM 

64-QAM 

Constrained 

Additive Gaussian 

Channel 

Calculation: 

η = log 2 (M)*R c 

Capacity Limit E b 

/N 0 

(dB) 

Capacity as function of 

Packet Length 

Based on paper: 

3 

2 

1 

0 

-1 

R c 

=0.1 

R c 

=0.33 

R c 

=0.5 

R c 

=0.75 

10 2 10 3 10 4 10 5 10 6 

Block length N information bits 


Code Performance as 

Function of Block 

Size, av Dolinar, 

Divsalar & Simon 

Expression given as 

function of N, R c and 

P w (Word Error 

Probability) 

9

Traditional Coding Schemes 


Basic Terms 

• Code rate 

k 

R = = 

n 

• Soft Decisions 

# Information 

# Code 

word 

symbols 

symbols 

– Principle idea (more or less correct/wrong) 

– Erasures decoding (3 level soft decision - very simple) 

– 8 level soft decision - often used in many coding 

schemes 

– Soft decision gain: ca 2 dB for AWGN channels 

– Typically 7-9 dB possible for fading channels 


10

Traditional Coding Schemes 

● 

Block Codes 

– Long Codes 

● 

Convolutional Codes 

– Short Codes 

– Hard desisjon 

– Soft desisjon (Viterbi) 

– Good at low bit error 

rates (10 -4 and down) 

– Good at high bit-error 

rates (10 -3 - 10 -4 ) 

– Many types 

• Ex. Hamming, Golay, 

Reed-Muller, Reed 

Solomon, BCH, etc 

– Used for instance in CD 

players (RS), Intelsat 

(RS), Voyager (Golay) 

– Used in Inmarsat, 3G and 

various wireless systems 

– Long constraint length 

codes with sequential 

decoding 


Basic Block Code 

• Operates on one block at a time 

c = mG 

• Algebraic decoding (often) 

• Soft decision decoding - high complexity 

• Typically good performance at high code 

rates 


11

Performance Golay Code 

• Extended Golay 

Code (24,12) 

• Hard decision 

• Erasure decoding 

• Soft decoding using 

Chase algorithm 

• Used in WLL 

• Low complexity 

decoding 


Convolutional Codes 

– Described by trellis 

– Maximum Likelihood Decoding (ML) by Viterbi 

algorithm 

– Soft Decision decoding easily implemented 

– Used in very many applications, Deep Space, SatCom, 

various wireless systems like 3G 

– Rate 1/(n+1) encoder: 

1 

2 

3 

K-2 K-1 K 

g 0 


g n 

12

Convolutional Code - BER 

• K=7 Feed 

Forward Code 

• MFD (and ODS) 

• Viterbi Decoding 

• Rate 3/4 

obtained by 

puncturing rate 

1/2 

BER 

BER 

10 0 1/2-rate 

3/4-rate 

10 -2 

10 -4 

10 -6 

10 -8 

0 1 2 3 4 5 6 

EbNo (dB) 


E b /N 0 

Foldningskode - Tap 

Koderate 

Yting ved Kapasitet 

10 -6 Constrained 

Binary Input 

Tap pga 

avgrensa 

blokklengde 

½ 5.0 0.2 ?? 4.8 

¾ 6.0 1.6 ?? 4.4 

Tap til 

Constrained 

input 

Grense 

•Implementert i ASIC 

•Relativt stort tap til kapasitetsgrensa 

•Blokklengde - vanskelig å definere 


13

Concatenated Coding 

RS 

encoder 

Convolutional 

Encoder 

Viterbi 

Decoder 

RS 

decoder 

• Traditional scheme for very low error rate 

• Typically RS + Convolutional 

• Viterbi - soft decisions 

• RS burst error correction 

• Viterbi gives moderate BER where RS 

takes over and brings it further down 

• Long Code with practical complexity 


Alternative to Concatenation 

• Use a very long 

constraint length (K) 

convolutional code 

• Typically - K=30-50 

• Error rate decreases 

exponentially with 

constraint length 

• Viterbi decoding: 

– 2 30 ~ 10 9 states 

• NOT IMPLEMENTABLE 

• Solution: Sequential 

Decoding 

– Limited but smart 

search 

– Search the most likely 

paths 

– Complexity depends 

on channel conditions 

– Can calculate 

conditions for when 

decoding is possible 


14

Sekvensiell dekoding 

Koderate 

Pareto 

eksponent 

lik 1 ved: 

Kapasitet 

Constrained 

Binary Input 

Tap pga 

avgrensa 

blokklengde 

Tap til 

Constrained 

Grense 

½ 2.2 dB 0.2 dB ?? 2 dB 

¾ 3.7 dB 1.6 dB ?? 2.1 dB 

• K=36, R=1/2 Implementert i DSP 

• Problem: Overflyt i buffere 



Eksempel frå Nera Produkt 

• Inmarsat M&B 

– Foldningskode K=7 (1/2 og 3/4-rate) 

– Sekvensiell dekoding K=36 (1/2 rate) 

• Nera World Communicator 

– Turbo 1/2-rate med 16-QAM 

• DVB-RCS 

– Foldningskode og Reed-Solomon (255,239,8) 

– Turbo 

•BGAN (next generation mobile terminals) 

– Turbo 16 tilstander 

15

New iterative capacity 

approaching schemes 


Turbo Codes 

ICC 1993 - Presentation of Turbo Codes 

- a huge step towards the Shannon limit 

• Convolutional Turbo: 

– Parallel Concatenated 

Convolutional (PCC) codes 

– Serially Concatenated 

Convolutional (SCC) Codes 

– Separated by interleaver 

– Soft in and soft out, iterative 

decoding 

• Block based Turbo: 

– Turbo Product Codes 

(TPC) 

– PA-Codes 

– LDPC Codes 

– Soft decision, iterative 

decoding 

Long codes, soft decision, iterative decoding 


16

PCC Turbo Principle 

• Encoder 

– Recursive convolutional 

encoders 

– Interleaver essential 

X m 

• Decoder 

– Soft in soft out 

– Many iterations 

Xd 

KODER 

I 

X s 

Ys 

Y m 

DEKODER 

I 

Le 

(Soft info) 

INTER- 

LEAVER 

INTER- 

LEAVER 

DEINTER- 

LEAVER 

KODER 

II 

Zs 

Zm 

DEKODER 

II 

L e 

(Soft info) 

X'd 


PCC codes - properties 


– The ”original” Turbo code as presented by 

Berrou in 1993 

– Good overall performance, also at 10 -3 

– Good convergence of iterative decoder (few 

iterations necessary) 

– Possible candidates: 

• BGAN (16-states) 

• UMTS (8-states) 

• DVB-RCS (8-states, duo-binary) 

– The above codes must be punctured to get 

high rate => affects BER performance 

17

Performance Example 

BER 

10 -1 Capacity Limit 

Uncoded 

10 -2 

Sequential limit 

Convolutional Coding 

Concatenated Coding 

DVB-RCS Turbo 

10 -3 

10 -4 

Convolutional 

• MPEG block 

lengths (188 

bytes) 

• R=1/2 

10 -5 

10 -6 

Turbo 

Concatenated 

RS+Convolutional 

10 -7 

0 2 4 6 8 10 12 14 


E b /N 0 

Code Rate 

DVB-RCS 

1 

0.9 

0.8 

0.7 

0.6 

0.5 

0.4 

0.3 

-1 0 1 2 3 4 5 6 

Required E b /N 0 

Code Performance Comparison for MPEG (1 packet) and PER=10 -5 


Required Eb/No to get 1e-5 (MPEG) 

Capacity Binary Input 

Capacity MPEG Block 

Turbo Codes 

Concatenated Codes 

18

Iterative decoding - Block Codes 

• Turbo Product Codes (TPC) 

• Product Accumulate Codes 

• Low Density Parity Check Codes (LDPC) 

• Nera: Theoretical Studies and Simulations 

• Interesting because: 


– Typically better than convolutional based Turbo 

codes at high code rates 

– Fewer patents 

– TPC included in IEEE802.16 standard 

– PA codes - low complexity 

– LDPC has “World Record” 0.05 dB from Shannon 

– Complexity -> high-capacity systems 

Turbo Product Codes (TPC) 

• Two (or more) block codes 

concatenated 

• Soft in soft out, iterative decoding 

• Component codes - e.g. BCH 

d 11 

d 12 

d 13 

d 14 

d 15 

p 16 

p 17 

d 21 

d 22 

d 23 

d 24 

d 25 

p 26 

p 27 

d 31 d 32 

d 33 

d 34 

d 35 

p 36 

p 37 

p 41 p 42 

p 43 

p 44 

p 45 

p 46 p 47 


19

Ytelse TPC 

• Simulert med AWGN, QPSK and 16QAM 

• Ytelse ved 10 -5 BER 

(TPC) 

Koderate 

Ytelse ved Kapasitet 


Binary Input 

Blokklengde 

(kodet) 

Ekstra tap 

Blokklengde 

Tap til 

Praktisk 

Grense 

0.66 3.0 dB 1.2 dB 1024 (32x32) 1.0 dB 0.8 dB 

0.79 3.3 dB 2.0 dB 4096 (64x64) 0.5 dB 0.8 dB 

16-QAM 

0.79 6.8 dB 5.1 dB 4096 (64x64) 0.5 dB 1.3 dB 


TPC Performance vs TCM 

TCM and TPC with 64 QAM, comparable 

spectral efficiency ((64,57) TPC code) 

BER 

10 -2 Capacity Limit 

TPC and 64QAM 

64-TCM 

10 -4 

10 -6 

• Good at high code 

rates 

• Interesting for radio 

link systems 

• In IEEE802.16 

standard 

10 -8 

4 6 8 10 12 14 16 18 


E b /N 0 

20

Product Accumulate Codes 

PA-I 

SPC 

π 1 

π 2 

+ 

T 

SPC 

PA-II 

TPC 

TPC 

TPC 

/SPC 

/SPC 

/SPC 

π 2 

+ 

T 


PA code -properties 

– A new low complexity scheme, presented in 

2001 

– PA-codes have good overall performance 

– No puncturing, assumed to have better 

performance at low BER 

– Few/no patents 

– Slower convergence than PCC 

• PA-I 

• PA-II 

• Generalised PA- codes (for lower rates) 


21

Low Density Parity Check 

(LDPC) Codes 

• Block Code 

• Iterative decoding 

c=mG GH T =0 

1 1 

1 

1 

© Nera Research/P. Orten 1 

1 

H 

1 

1 

1 

1 

LDPC codes -properties 

– Proposed by Gallager i 1963 

– Good performance, especially for high code 

rates and long block lengths 

– 0.005 dB from Shannon limit!! 

– Low decoder complexity 

– Few (no?) patents 

– Disadvantages: 

• Some error flattening for some codes 

• Not ready for implementation, much ongoing 

research 


22

Ytelse - LDPC koder 

• Simulert ytelse, BPSK/QPSK, AWGN 

• ATM og MPEG blokker 

• Regulære LDPC koder, BER=10 -5 

(LDPC) 

Koderate 

Ytelse ved Kapasitet 


Binary Input 

Blokklengde 

(kodet) 

Ekstra tap 

Blokklengde 

Tap til 

Praktisk 

Grense 

1/2 1.8 dB 0.2 dB 1504 MPEG 0.7 dB 0.9 dB 

4/5 3.7 dB 1.6 dB 1504 MPEG 0.7 dB 1.4 dB 

1/2 2.8 dB 0.2 dB 424 ATM 1.2 dB 1.4 dB 

4/5 4.5 dB 1.6 dB 424 ATM 1.2 dB 1.7 dB 


Performance Example LDPC Codes 

Spectral 

Efficiency 

1 

0.8 

0.6 

0.4 

•PER=10 -5 

• BER=10 -5 

for LDPC 

• MPEG packets 

Capacity 

0.2 

Concatenated 

PCC 

TPC 

LDPC 

0 

-1 0 1 2 3 4 5 6 7 


Required E b /N 0 

23

Mapping to higher order 

constellations 

TCM 

x n 

x m+1 

n 

: : 

n-m 

: : 

MAPPING 

Select signal 

from subset 

a i 

xm 

x1 

: : 

Convolutional 

Encoder 

Rate m/(m+1) 

z m 

: : 

z1 

Select 

Subset 

z0 

• Optimised mapping necessary in order to 

achieve coding gain 

• Effective “block length” of code is short 


Mapping to higher order 

constellations 

Turbo-TCM 

/ TCM 

x 1 

Turbo-TCM / TCM 

Encoder together with 

optimised mapping 

ai 

Encoder + 

Gray mapping 


MAPPING 

x1 y 1 

Turbo 

Simple 

Encoder 

Gray 

Mapping 

ai 

24

Mapping 

• Turbo + Gray mapping works well, 

because 


– Code is long and give good coding gain 

anyway 

– Distance properties of individual symbols in 

the code is less important 

– Also more flexible with respect to 

• Modulation schemes 

• Code rate 

– Confirmed by several papers 

Code Comparison 

• Block lengths K=1000 and 4000 

– With 2.5*N total delay16µs and 64 µs 

• Modulation/Code rate Schemes 

• 64-QAM Code rate ≈ 0.85 

• 128-QAM Code rate ≈0.93 


25

BER Performance 64-QAM 

Short Delay (K=1000) 


BER Performance 64-QAM 

Long Delay (K=4000) 


26

Evaluation of Iterative Decoding 

BER 

BER 

Flattening 

is possible Bad implemenaton of 

iterative algorithm 

Given by 

minimum distance 

of code 

Eb/N 0 


1000 

Decoding Complexity of Coding 

Schemes 

900 

800 

700 

600 

500 

400 

300 

200 

100 

0 

LDPC (tree) PCC-BGAN PCC-UMTS PA-I PA-II PA-II (tree) TCM 

R=0.84 R=0.93 

Complexity of coding schemes in # of max-log or min-sum operations pr decoded bit 


27

Conclusion 

• For practical systems we can not achieve 

the -1.6 dB limit 

• “Practical limit” - depend on rate, 

constellation, block length 

• New techniques with iterative decoding 

approach capacity 

• For High Data Rates - Complexity is a 

consern 

• Mapping not critical for long codes 


28

Error Control - From Theory to Near-Capacity Implementation

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?