Scalar and Vector Quantization
Scalar and Vector Quantization
Scalar and Vector Quantization
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
<strong>Scalar</strong> <strong>and</strong> <strong>Vector</strong> <strong>Quantization</strong><br />
National Chiao Tung University<br />
Chun-Jen Tsai<br />
4/26/2012
Basic Concept of <strong>Quantization</strong><br />
<strong>Quantization</strong> is the process of representing a large,<br />
possibly infinite, set of values with a smaller set<br />
Example: real-to-integer conversion<br />
Source: real numbers in the range [–10.0, 10.0]<br />
Quantizer: Q(x) = ⎣x + 0.5⎦<br />
[–10.0, –10.0] → { –10, –9, …, –1, 0, 1, 2, …, 9, 10}<br />
The set of inputs <strong>and</strong> outputs of a quantizer can be<br />
scalars (scalar quantizer) or vectors (vector quantizer)<br />
2/55
The <strong>Quantization</strong> Problem<br />
Encoder mapping<br />
<br />
<br />
<br />
Map a range of values to a codeword<br />
Irreversible mapping<br />
If source is analog → A/D converter<br />
Decoder mapping<br />
<br />
<br />
<br />
Map the codeword to a (fixed) value representing the range<br />
Knowledge of the source distribution can help us pick a<br />
better value representing each range<br />
If output is analog → D/A converter<br />
Informally, the encoder mapping is called the<br />
quantization process, <strong>and</strong> the decoder mapping is<br />
called the inverse quantization process<br />
3/55
<strong>Quantization</strong> Examples<br />
3-bit Quantizer<br />
<br />
Encoder (A/D) Decoder (D/A)<br />
Digitizing a sine wave<br />
4/55
<strong>Quantization</strong> Function<br />
A quantizer describes the relation between the<br />
encoder input values <strong>and</strong> the decoder output values<br />
<br />
Example of a quantization function:<br />
5/55
<strong>Quantization</strong> Problem Formulation<br />
Input:<br />
<br />
<br />
Output:<br />
<br />
<br />
X – r<strong>and</strong>om variable<br />
f X (x) – probability density function (pdf)<br />
{b i } i = 0..M decision boundaries<br />
{y i } i = 1..M reconstruction levels<br />
Discrete processes are often approximated by<br />
continuous distributions<br />
<br />
<br />
E.g.: Laplacian model of pixel differences<br />
If source is unbounded, then first/last decision<br />
boundaries = ±∞ (they are often called “saturation” values)<br />
6/55
<strong>Quantization</strong> Error<br />
If the quantization operation is denoted by Q(·), then<br />
Q(x) = y i iff b i–1 < x ≤ b i .<br />
The mean squared quantization error (MSQE) is then<br />
σ<br />
2<br />
q<br />
=<br />
=<br />
∫<br />
∞<br />
−∞<br />
M<br />
∑∫<br />
i=<br />
1<br />
( x − Q(<br />
x)<br />
)<br />
b<br />
b<br />
i<br />
i−1<br />
( x − y )<br />
<strong>Quantization</strong> error is also called quantization noise or<br />
quantizer distortion, e.g., additive noise model:<br />
i<br />
2<br />
<strong>Quantization</strong> noise<br />
2<br />
f<br />
f<br />
X<br />
X<br />
dx<br />
dx<br />
Quantizer input<br />
+<br />
Quantizer output<br />
7/55
Quantized Bitrate with FLC<br />
If the number of quantizer output is M, then the<br />
bitrate of the quantizer output is R = ⎡log 2 M⎤<br />
Eample: M = 8 → R = 3<br />
Quantizer design problem:<br />
<br />
Given an input pdf f X (x) <strong>and</strong> the number of levels M in the<br />
quantizer, find the decision boundaries {b i } <strong>and</strong> the<br />
reconstruction levels {y i } so as to minimize the mean<br />
squared quantization error<br />
8/55
Quantized Bitrate with VLC<br />
For VLC representation of quantization intervals, the<br />
bitrate depends on decision boundary selection<br />
Example: eight-level quantizer:<br />
R<br />
=<br />
=<br />
M<br />
∑<br />
i=<br />
1<br />
M<br />
∑<br />
i=<br />
1<br />
l P(<br />
y )<br />
l<br />
i<br />
i<br />
⎡<br />
⎢⎣<br />
∫<br />
b<br />
b<br />
i<br />
i−1<br />
i<br />
f<br />
X<br />
( x)<br />
dx⎤<br />
⎥⎦<br />
9/55
Optimization of <strong>Quantization</strong><br />
Rate-optimized quantization<br />
Given: Distortion constraint σ q2 ≤ D*<br />
Find: { b i }, { y i } binary codes<br />
<br />
Such that: R is minimized<br />
Distortion-optimized quantization<br />
Given: Rate constraint R ≤ R*<br />
<br />
<br />
Find: { b i }, { y i } binary codes<br />
Such that: σ q2 is minimized<br />
10/55
Uniform Quantizer<br />
All intervals are of the same size<br />
<br />
Boundaries are evenly spaced (step size:Δ), except for outmost<br />
intervals<br />
Reconstruction<br />
<br />
Usually the midpoint is selected as the representing value<br />
Quantizer types:<br />
<br />
<br />
Midrise quantizer: zero is not an output level<br />
Midtread quantizer: zero is an output level<br />
11/55
Midrise vs. Midtread Quantizer<br />
Midrise Midtread<br />
12/55
Uniform <strong>Quantization</strong> of Uniform Source<br />
If the source is uniformly distributed in [–X max , X max ],<br />
the output is quantized by an M-level uniform<br />
quantizer, then the quantization step size is<br />
<strong>and</strong> the distortion is<br />
∆ =<br />
2X<br />
max<br />
M<br />
,<br />
σ<br />
M / 2<br />
2<br />
2<br />
i∆<br />
2 ⎛ 2i<br />
−1<br />
⎞ 1 ∆<br />
= 2∑∫<br />
q<br />
⎜ x − ∆⎟<br />
dx =<br />
( i−1)<br />
∆<br />
i=<br />
1 ⎝ 2 ⎠ 2X<br />
max<br />
12<br />
.<br />
This is quite difficult to evaluate!<br />
13/55
Alternative MSQE Derivation<br />
We can also compute the “power” of quantization<br />
error q = x – Q(x), q ∈ [–Δ/2, Δ/2] by:<br />
σ q<br />
2<br />
1 / 2<br />
2 2 ∆<br />
= ∫ ∆ q dq =<br />
∆<br />
−∆<br />
/ 2<br />
12<br />
.<br />
14/55
The SNR of <strong>Quantization</strong><br />
For n-bit uniform quantization of an uniform source of<br />
[–X max , X max ], the SNR is 6.02n dB, where n = log 2 M:<br />
10log<br />
10<br />
⎛<br />
2<br />
⎜<br />
σ<br />
s<br />
2<br />
⎝ σ<br />
q<br />
⎞<br />
⎟<br />
⎠<br />
= 10log<br />
10<br />
⎛ (2X<br />
⎜<br />
⎝ 12<br />
max<br />
)<br />
2<br />
12<br />
⋅<br />
2<br />
∆<br />
⎞<br />
⎟<br />
⎠<br />
= 10log<br />
=<br />
20log<br />
10<br />
10<br />
⎛<br />
⎜<br />
⎜ (2X<br />
⎜ 12<br />
⎜<br />
⎝<br />
2<br />
n<br />
max<br />
)<br />
2<br />
⎛<br />
⎜<br />
⎝<br />
12<br />
2X<br />
M<br />
= 6.02n<br />
dB.<br />
max<br />
⎞<br />
⎟<br />
⎠<br />
2<br />
⎞<br />
⎟<br />
⎟<br />
⎟<br />
⎟<br />
⎠<br />
= 10log<br />
10<br />
M<br />
2<br />
15/55
Example: <strong>Quantization</strong> of Sena<br />
Darkening <strong>and</strong> contouring effects of quantization<br />
8 bits / pixel<br />
1 bits / pixel<br />
2 bits / pixel<br />
3 bits / pixel<br />
16/55
<strong>Quantization</strong> of Non-uniform Sources<br />
Given a non-uniform source, x ∈[–100, 100],<br />
P(x∈[–1, 1]) = 0.95, <strong>and</strong> we want to design an 8-level<br />
(3-bit) quantizer.<br />
A naïve approach uses uniform quantizer (∆ = 25):<br />
<br />
95% of sample values represented by only two numbers:<br />
–12.5 <strong>and</strong> 12.5, with a maximal quantization error of 12.5<br />
<strong>and</strong> minimal error of 11.5<br />
If we use ∆ = 0.3 (two end-intervals would be huge)<br />
<br />
Max error is now 98.95 (i.e. 100 – 1.05), however, 95% of<br />
the time the error is less than 0.15<br />
17/55
Optimal ∆ that minimizes MSQE<br />
Given pdf f X (x) of the source, let’s design an M-level<br />
mid-rise uniform quantizer that minimizes MSQE:<br />
σ<br />
2<br />
q<br />
=<br />
2<br />
−1<br />
M<br />
2<br />
∑∫<br />
i=<br />
1<br />
i∆<br />
( i−1)<br />
∆<br />
⎛<br />
⎜<br />
⎝<br />
x −<br />
2i<br />
−1<br />
⎞<br />
∆⎟<br />
2 ⎠<br />
2<br />
f<br />
X<br />
( x)<br />
dx<br />
→ Granular error<br />
+ 2<br />
∫<br />
∞<br />
⎛ M ⎞<br />
⎜ −1⎟∆<br />
⎝ 2 ⎠<br />
⎛<br />
⎜<br />
⎝<br />
x −<br />
M −1<br />
⎞<br />
∆⎟<br />
2 ⎠<br />
2<br />
f<br />
X<br />
( x)<br />
dx.<br />
→ Overload error<br />
x – Q(x)<br />
Overload error<br />
Granular error<br />
18/55
Solving for Optimum Step Sizes<br />
Given an f X (x) <strong>and</strong> M, we can solve for ∆ numerically:<br />
dσ<br />
2<br />
q<br />
d∆<br />
= −<br />
−1<br />
M<br />
2<br />
∑<br />
i=<br />
1<br />
− ( M<br />
(2i<br />
−1)<br />
−1)<br />
∫<br />
∞<br />
∫<br />
i∆<br />
( i−1)<br />
∆<br />
⎛ M ⎞<br />
⎜ −1⎟∆<br />
⎝ 2 ⎠<br />
⎛<br />
⎜<br />
⎝<br />
⎛<br />
⎜<br />
⎝<br />
2i<br />
−1<br />
⎞<br />
x − ∆⎟<br />
f<br />
2 ⎠<br />
M −1<br />
⎞<br />
x − ∆⎟<br />
f<br />
2 ⎠<br />
( x)<br />
dx<br />
( x)<br />
dx =<br />
Optimal uniform quantizer ∆ for different sources:<br />
X<br />
X<br />
0.<br />
19/55
Overload/Granular Regions<br />
Selection of the step size must trade off between<br />
overload noise <strong>and</strong> granular noise<br />
20/55
Variance Mismatch Effects (1/2)<br />
Effect of variance mismatch on the performance of a<br />
4–bit uniform quantizer<br />
21/55
Variance Mismatch Effects (2/2)<br />
The MSQE as a function of variance mismatch with a<br />
4–bit uniform quantizer.<br />
22/55
Distribution Mismatch Effects<br />
Given 3-bit quantizer, the effect of distribution<br />
mismatch for different sources (SNR errors in dB):<br />
<br />
<br />
Form left-to-right, we assume that the sources are uniform,<br />
Gaussian, Laplacian, <strong>and</strong> Gamma, <strong>and</strong> compute the<br />
optimum MSQE step size for uniform quantizer<br />
The resulting ∆ gets larger from left-to-right<br />
→ if there is a mismatch, larger than “optimum” ∆ gives better performance<br />
→ 3-bit quantizer is too coarse<br />
23/55
Adaptive <strong>Quantization</strong><br />
We can adapt the quantizer to the statistics of the<br />
input (mean, variance, pdf)<br />
Forward adaptive (encoder-side analysis)<br />
<br />
<br />
<br />
<br />
Divide input source in blocks<br />
Analyze block statistics<br />
Set quantization scheme<br />
Send the scheme to the decoder via side channel<br />
Backward adaptive (decoder-side analysis)<br />
<br />
<br />
<br />
Adaptation based on quantizer output only<br />
Adjust ∆ accordingly (encoder-decoder in sync)<br />
No side channel necessary<br />
24/55
Forward Adaptive <strong>Quantization</strong> (FAQ)<br />
Choosing analysis block size is a major issue<br />
Block size too large<br />
<br />
<br />
Not enough resolution<br />
Increased latency<br />
Block size too small<br />
<br />
More side channel information<br />
Assuming a mean of zero, signal variance is<br />
estimated by<br />
N 1<br />
2 1 2<br />
ˆ<br />
−<br />
σ = ∑ q<br />
x n + i.<br />
N<br />
i=<br />
0<br />
25/55
Speech <strong>Quantization</strong> Example (1/2)<br />
16-bit speech samples → 3-bit fixed quantization<br />
26/55
Speech <strong>Quantization</strong> Example (2/2)<br />
16-bit speech samples → 3-bit FAQ<br />
<br />
<br />
Block = 128 samples<br />
8-bit variance quantization<br />
27/55
FAQ Refinement<br />
So far, we assumed uniform pdf over maximal ranges,<br />
we can refine it by computing the range of distribution<br />
adaptively for each block<br />
Example: Sena image, 8x8 blocks, 2x8-bit for range<br />
per block, 3-bit quantizer<br />
Original 8 bits/pixel<br />
Quantized 3.25 bits/pixel<br />
28/55
Backward Adaptive <strong>Quantization</strong> (BAQ)<br />
Key idea: only encoder sees input source, if we do<br />
not want to use side channel to tell the decoder how<br />
to adapt the quantizer, we can only use quantized<br />
output to adapt the quantizer<br />
Possible solution:<br />
<br />
<br />
<br />
Observe the number of output values that falls in outer levels<br />
<strong>and</strong> inner levels<br />
If they match the assumed pdf, ∆ is good<br />
If too many values fall in outer levels, ∆ should be enlarged,<br />
otherwise, ∆ should be reduced<br />
Issue: estimation of pdf requires large observations?<br />
29/55
Jayant Quantizer<br />
N. S. Jayant showed in 1973 that ∆ adjustment based<br />
on few observations still works fine:<br />
<br />
<br />
If current input falls in the outer levels, exp<strong>and</strong> step size<br />
If current input falls in the inner levels, contract step size<br />
The total product of expansions <strong>and</strong> contraction should be 1<br />
Each decision interval k has a multiplier M k<br />
If input s n–1 falls in the k th interval, step size is multiplied by M k<br />
Inner-level M k < 1, outer-level M k > 1<br />
<br />
Step size adaptation rule:<br />
∆<br />
=<br />
n<br />
M l ( n−1)<br />
n−<br />
where l(n–1) is the quantization interval at time n–1.<br />
∆<br />
1,<br />
30/55
Output Levels of 3-bit Jayant Quantizer<br />
The multipliers are symmetric:<br />
M 0 = M 4 , M 1 = M 5 , M 2 = M 6 , M 3 = M 7<br />
31/55
Example: Jayant Quantizer<br />
M 0 = M 4 = 0.8, M 1 = M 5 = 0.9<br />
M 2 = M 6 = 1.0, M 3 = M 7 = 1.2, Δ 0 = 0.5<br />
Input: 0.1, –0.2, 0.2, 0.1, –0.3, 0.1, 0.2, 0.5, 0.9, 1.5<br />
32/55
Picking Jayant Multipliers<br />
We must select Δ min <strong>and</strong> Δ max to prevent underflow<br />
<strong>and</strong> overflow of step sizes<br />
Selection of multipliers<br />
Total production of expansion/contractions should be 1<br />
<br />
<br />
Π<br />
k = 0<br />
n k<br />
k<br />
= 1.<br />
Scaled to probability of events in each interval, we have<br />
Pick γ > 1, <strong>and</strong> let M k = γ l k, we have<br />
M<br />
M<br />
n<br />
N<br />
M<br />
P<br />
k<br />
Π = 1, where = , = ∑<br />
M<br />
k<br />
Pk<br />
N<br />
k =0 k =0<br />
M<br />
∑<br />
k = 0<br />
M k<br />
l k<br />
P k<br />
= 0,<br />
→ γ <strong>and</strong> l k are chosen, P k is known.<br />
n<br />
k<br />
.<br />
33/55
Example: Ringing Problem<br />
Use a 2-bit Jayant quantizer to quantize a square<br />
wave<br />
P 0 = 0.8, P 1 = 0.2 → pick l 0 = –1, l 1 = 4, γ ~ 1.<br />
34/55
Jayant Quantizer Performance<br />
To avoid overload errors, we should exp<strong>and</strong> ∆ rapidly<br />
<strong>and</strong> contracts ∆ moderately<br />
Robustness over changing input statistics<br />
Jayant is about 1dB lower<br />
than fixed quantizer<br />
35/55
Non-uniform <strong>Quantization</strong><br />
For uniform quantizer,<br />
decision boundaries<br />
are determined by a<br />
single parameter ∆.<br />
We can certainly reduce<br />
quantization errors<br />
further if each decision<br />
boundaries can be<br />
selected freely<br />
36/55
pdf-optimized <strong>Quantization</strong><br />
Given f X (x), we can try to minimize MSQE:<br />
σ<br />
2<br />
q<br />
=<br />
M<br />
∑∫<br />
i=<br />
1<br />
b<br />
b<br />
i<br />
i−1<br />
2<br />
( x − y ) f ( x)<br />
dx.<br />
i<br />
X<br />
Set derivative of σ q2 w.r.t. y j to zero <strong>and</strong> solve for y j ,<br />
we have:<br />
b<br />
∫<br />
j<br />
xf<br />
X<br />
( x)<br />
dx<br />
b j−1<br />
y<br />
y<br />
j<br />
=<br />
.<br />
j<br />
is the center of mass of<br />
b j<br />
f X<br />
in [b j–1<br />
, b j<br />
)<br />
f ( x)<br />
dx<br />
∫<br />
b<br />
j−1<br />
X<br />
If y j are determined, the b j ’s can be selected as:<br />
j<br />
( y y )/<br />
2.<br />
b +<br />
=<br />
j+<br />
1 j<br />
37/55
Lloyd-Max Algorithm (1/3)<br />
Lloyd-Max algorithm solves y j <strong>and</strong> b j iteratively until<br />
an acceptable solution is found<br />
Example:<br />
For midrise quantizer,<br />
b 0 = 0,<br />
b M/2 is the largest input,<br />
we only have to find<br />
{ b 1 , b 2 , …, b M/2–1 } <strong>and</strong><br />
{ y 1 , y 2 , …, y M/2–1 }.<br />
38/55
Lloyd-Max Algorithm (2/3)<br />
Begin with j = 1, we want to find b 1 <strong>and</strong> y 1 by<br />
y<br />
b1<br />
b<br />
1<br />
= ∫ xf ( ) ∫<br />
X<br />
x dx<br />
b<br />
b<br />
0<br />
( x)<br />
dx.<br />
Pick a value for y 1 (e.g. y 1 = 1), solve for b 1 <strong>and</strong><br />
compute y 2 by<br />
y 2 = 2b 1 + y 1 ,<br />
<strong>and</strong> b 2 by<br />
y<br />
b2<br />
b<br />
2<br />
= ∫ xf ( ) ∫<br />
X<br />
x dx<br />
b<br />
b<br />
1<br />
0<br />
1<br />
Continue the process until all {b j } <strong>and</strong> {y j } are found<br />
1<br />
f<br />
2<br />
X<br />
f<br />
X<br />
( x)<br />
dx.<br />
39/55
Lloyd-Max Algorithm (3/3)<br />
If the initial guess of y 1 does not fulfills the termination<br />
condition:<br />
y M<br />
ˆ<br />
/ 2<br />
− y M / 2<br />
≤ ε,<br />
where<br />
yˆ = M 2<br />
2bM<br />
/ 2 1<br />
yM<br />
/ 2 1,<br />
y<br />
+<br />
/ − −<br />
b<br />
∫<br />
M / 2<br />
b<br />
/ 2<br />
= ( ) ∫<br />
M<br />
xf<br />
X<br />
x dx<br />
b<br />
b<br />
M / 2−1<br />
( x)<br />
dx.<br />
we must pick a different y 1 <strong>and</strong> repeat the process.<br />
M<br />
/ 2<br />
M / 2−1<br />
f<br />
X<br />
40/55
Example: pdf-Optimized Quantizers<br />
We can achieve gain over the uniform quantizer<br />
(9.24) (7.05)<br />
(12.18) (9.56)<br />
(14.27) (11.39)<br />
41/55
Mismatch Effects<br />
Non-uniform quantizers also suffer mismatch effects.<br />
To reduce the effect, one can use an adaptive nonuniform<br />
quantizer, or an adaptive uniform quantizer<br />
plus comp<strong>and</strong>ed quantization techniques<br />
Variance mismatch on a 4-bit Laplacian non-uniform quantizer.<br />
42/55
Comp<strong>and</strong>ed <strong>Quantization</strong> (CQ)<br />
In comp<strong>and</strong>ed quantization, we adjust (i.e. re-scale)<br />
the intervals so that the size of each interval is in<br />
proportion to the probability of inputs in each interval<br />
equivalent to a non-uniform quantizer<br />
43/55
Example: CQ (1/2)<br />
The compressor function:<br />
c(<br />
x)<br />
=<br />
⎧<br />
⎪<br />
⎨<br />
⎪<br />
⎩<br />
2x<br />
3<br />
2x<br />
3<br />
2x<br />
+<br />
−<br />
4<br />
3<br />
4<br />
3<br />
if<br />
−1<br />
≤ x ≤1<br />
x > 1 .<br />
x < −1<br />
The uniform quantizer:<br />
step size ∆ = 1.0<br />
44/55
Example: CQ (2/2)<br />
The exp<strong>and</strong>ar function:<br />
c<br />
−1<br />
( x)<br />
=<br />
⎧<br />
⎪<br />
⎨<br />
⎪<br />
⎩<br />
3x<br />
2<br />
3x<br />
2<br />
x<br />
2<br />
− 2<br />
+ 2<br />
if<br />
− 2 ≤ x ≤<br />
x > 2<br />
x < −2<br />
2<br />
.<br />
The equivalent non-uniform<br />
quantizer<br />
45/55
Remarks on CQ<br />
If the level of quantizer is large <strong>and</strong> the input is<br />
bounded by x max , it is possible to choose a c(x) such<br />
that the SNR of CQ is independent to input pdf:<br />
SNR = 10 log 10 (3M 2 ) – 20log 10 α,<br />
where c′(x) = x max / (α |x|) <strong>and</strong> a is a constant.<br />
Two popular CQ for telephones: µ-law <strong>and</strong> A-law<br />
<br />
µ-law compressor<br />
A-law compressor<br />
c(<br />
x)<br />
=<br />
⎧<br />
⎪<br />
⎨<br />
⎪⎩<br />
x<br />
max<br />
A x<br />
1+<br />
ln A<br />
⋅<br />
ln(1 + µ<br />
x<br />
)<br />
max<br />
c( x)<br />
= x max<br />
sgn( x).<br />
ln(1 + µ )<br />
1+<br />
ln<br />
sgn( x),<br />
A x<br />
xmax<br />
1+<br />
ln A<br />
sgn( x),<br />
0 ≤<br />
1<br />
A<br />
≤<br />
x<br />
x<br />
x<br />
max<br />
x<br />
x<br />
max<br />
≤<br />
≤<br />
1<br />
A<br />
.<br />
1<br />
46/55
Entropy Coding of Quantizer Outputs<br />
The levels of the quantizer is the alphabet of entropy<br />
coders, for M-level quantizer, FLC needs log 2 M bits<br />
per output<br />
Example of VLC coded output of minimum MSQE<br />
quantizers:<br />
<br />
Note: non-uniform quantizer has higher entropies since high<br />
probability regions uses smaller step sizes<br />
47/55
<strong>Vector</strong> <strong>Quantization</strong><br />
<strong>Vector</strong> quantization groups source data into vectors<br />
<br />
At both the encoder <strong>and</strong> decoder, we have a set of vectors<br />
called the codebook. Each code-vector in the codebook is<br />
assigned a binary index.<br />
48/55
Why <strong>Vector</strong> <strong>Quantization</strong> (1/2)?<br />
Correlated multi-dimensional data have limited valid<br />
ranges<br />
useless<br />
useless<br />
49/55
Why <strong>Vector</strong> <strong>Quantization</strong> (2/2)?<br />
Looking at the data from a higher dimension allow us<br />
to better fit the quantizer structure to the joint pdf<br />
<br />
Example: quantize the Laplacian source data two at a time:<br />
11.44 dB 11.73 dB<br />
50/55
<strong>Vector</strong> <strong>Quantization</strong> Rule<br />
<strong>Vector</strong> quantization (VQ) of X may be viewed as the<br />
classification of the outcomes of X into a discrete<br />
number of sets or cells in N-space<br />
<br />
Each cell is represented by a vector output Y j<br />
Given a distance measure d(x, y), we have<br />
VQ output: Q(X) = Y j iff d(X, Y j ) < d(X, Y i ), ∀ i ≠ j.<br />
<br />
<strong>Quantization</strong> region: V j = { X: d(X, Y j ) < d(X, Y i ), ∀ i ≠ j}.<br />
x 2<br />
V j<br />
Y j<br />
x 1<br />
51/55
Codebook Design<br />
The set of quantizer output points in VQ is called the codebook<br />
of the quantizer, <strong>and</strong> the process of placing these output points<br />
is often referred to as codebook design<br />
The k-means algorithm is often used to classify the outputs<br />
<br />
<br />
<br />
<br />
Given a large set of output vectors from the source, known as the<br />
training set, <strong>and</strong> an initial set of k representative patterns<br />
Assign each element of the training set to the closest<br />
representative pattern<br />
After an element is assigned, the representative pattern is updated<br />
by computing the centroid of the training set vectors assigned to it<br />
When the assignment process is complete, we will have k groups of<br />
vectors clustered around each of the output points<br />
52/55
The Linde-Buzo-Gray Algorithm<br />
1. Start with an initial set of reconstruction values<br />
{Y i<br />
(0)<br />
} i=1..M <strong>and</strong> a set of training vectors {X n } n=1..N .<br />
Set k = 0, D (0) = 0. Select threshold ε.<br />
2. The quantization regions {V i<br />
(k)<br />
} i=1..M are given by<br />
V j<br />
(k)<br />
= {X n : d(X n , Y i ) < d(X n , Y j ), ∀ j ≠ i}, i = 1, 2, …, M.<br />
3. Compute the average distortion D (k) between the<br />
training vectors <strong>and</strong> the representative value<br />
4. If (D (k) – D (k–1) )/D (k) < ε, stop; otherwise, continue<br />
5. Let k = k + 1. Update {Y i<br />
(k)<br />
} i=1..M with the average<br />
value of each quantization region V i<br />
(k–1)<br />
. Go to step 2.<br />
53/55
Example: Codebook Design<br />
Initial state<br />
Final state<br />
54/55
Impact of Training Set<br />
The training sets used to construct the codebook<br />
have significant impact on the performance of VQ<br />
Images quantized at 0.5<br />
bits/pixel, codebook size 256<br />
55/55