# 1 jeudi 20 juin 13

1 jeudi 20 juin 13

Introduction to

Signal Processing

jeudi 20 juin 13

1

Introduction

jeudi 20 juin 13

2

What is a signal

As far as this course is concerned, a

signal is a mathematical function that

makes sense in the physical world.

jeudi 20 juin 13

3

Why sense

Let us consider a popular signal

processing application: music

compression.

The obvious answer to pure compression

is to replace the signal by one zero scalar.

But this is not music!

jeudi 20 juin 13

4

What is signal

processing

Signal processing transforms signals for

various purposes while essentially

keeping the sense of it.

It may also consist in extracting and/or

manipulating features that are

meaningful in the signal.

jeudi 20 juin 13

5

General mathematical

representations of signals

Signals can be represented as functions

in the continuous or in the discrete

domain.

jeudi 20 juin 13

6

30

20

10

0

−10

20

−30

0 500 1000 1500 2000 2500

300

200

Example of

continuous

100

0

−100

0 500 1000 1500 2000 2500

30

20

10

0

−10

20

0 500 1000 1500 2000 2500

time data:

EEG

measures

(source:

30

20

10

0

−10

State

20

−30

0 500 1000 1500 2000 2500

University)

150

100

50

0

−50

−100

0 500 1000 1500 2000 2500

jeudi 20 juin 13

7

4500

Reported Number of Cases of Measles, Monthly, Jan. 1939− June 1972 Baltimore

4000

3500

3000

2500

2000

1500

1000

Example of

(monthly)disc

rete time data

(Source: Time

Series Data

Library)

500

0

1935 1940 1945 1950 1955 1960 1965 1970 1975

jeudi 20 juin 13

8

Food for thought

Both data sets consisted in a finite series of

numbers. In the second set, we felt that the

«sampling rate» of the data could have been

increased, so it was considered discrete.

In the first case, the sampling rate was high

compared to the time scales of the visible

patterns, so we called the signal continuous.

jeudi 20 juin 13

9

General mathematical

representations of signals

Another distinction is whether the signal

is periodic or not.

In practice, this is used to distinguish

whether the data is available on a finite

amount of time or on the real axis.

jeudi 20 juin 13

10

15 Monthly GLOBAL Temperature in degrees Celsius, year 1880−2010, base period: 1951−1980 = 14°C 11

14.8

14.6

14.4

14.2

14

13.8

13.6

13.4

13.2

Some data may have

periodic pattern

(seasonal variations

within the year),

while displaying

long term non

periodic trends.

(Source: NASA)

13

1880 1900 1920 1940 1960 1980 2000 2020

jeudi 20 juin 13

Some music to end

First 24 bars of «When A Guitar Plays The Blues»

(Roy Buchanan)

jeudi 20 juin 13

12

Some music to end

First 24 bars of «When A Guitar Plays The Blues»

(Roy Buchanan)

jeudi 20 juin 13

12

Linear Time

Invariant Systems

and Related Transforms

jeudi 20 juin 13

13

The How and the Why

Linear Time Invariant (LTI) filters have

been hugely successful because:

• they are easy to understand and to

implement

• they are related, via Fourier related

transforms, to important features of

signals

jeudi 20 juin 13

14

Notes

Because we have (at least) four

representations of signals (continuous/

discrete combined with finite/infinite

support), we will have as many forms of

LTI filters and Fourier transforms.

jeudi 20 juin 13

15

Notes

We shall first concentrate on continuous LTI

filters defined on the real axis, beginning with

a loose definition.

jeudi 20 juin 13

15

Definition of an LTI

system (or filter)

• It linearly maps an input signal into an

output signal

• if the input is shifted in time by an

amount T, then the corresponding

output is also shifted by T

jeudi 20 juin 13

16

Impulse response

• The impulse response h is the output of

the Dirac distribution

• In the general case, the output y is

obtained by convolving the input u with

the impulse response h:

Z +1

Z +1

y(t) =

h(t s)u(s)ds =

u(t

s)h(s)ds

1

1

jeudi 20 juin 13

17

Some mathematical

results

done in discrete time. The continuous

time settings is interesting because it taps

into the rich literature devoted to the

topic of functional analysis. To use this,

we must refine the previous definition.

jeudi 20 juin 13

18

Signal spaces

For convenience sake, we shall assume

that the signals belong to the space of

square integrable functions. This will

simplify the manipulation of the Fourier

Transform.

jeudi 20 juin 13

19

convolution formula

We give here a simple proof for the

convolution formula. Let L a LTI filter and t

any scalar. We assume that, for any square

integrable signal f, the linear form

[L t ]f =[Lf](t)

is defined for all t and is continuous with

respect to f. Then there exists K 0 in L 2 such that

[L 0 ]f =

Z +1

1

K 0 (s)f( s)ds

which , together with the shift invariance,

proves the convolution formula

jeudi 20 juin 13

20

The Fourier integral

and transform

If f is an integrable signal, we can define its

Fourier integral by

Z +1

ˆf(!) =

1

f(t)e i!t dt

After some additional work, it can be

extended to L 2 , and the transform is square

integrable. After a normalization, we define

the Fourier transform by

ˆF (!) = p 1 Z +1

f(t)e i!t dt

2⇡

1

jeudi 20 juin 13

21

Warning

• ! is often called the frequency in the

textbooks; it is actually an angular velocity

• the frequency f is related to the angular

velocity by f = !/2". f is measured in Herz.

• most of the time we will follow the

textbooks and call ! a frequency; but in

applications we will use actual frequencies.

jeudi 20 juin 13

22

Properties

The Fourier integral and the Fourier

transform have many properties

• The Fourier transform is an isometry from

L 2 into itself, and the inverse formula is

obtained by replacing ! by -!.

• If f and g are integrable (resp. square

integrable), then their convolution f * g is

integrable (resp. square integrable) with

[f ⇤ g = ˆfĝ

jeudi 20 juin 13

23

Transfer function

• Assume that the Fourier integral of the

impulse response of a LTI filter is defined;

it is then called the transfer of the filter

• The previous result shows that, provided

that the input is in a suitable space, that

the Fourier integral of the output is the

product of the Fourier integral of the input

with the transfer.

jeudi 20 juin 13

24

Sobolev Regularity

• A signal f is said to have a Sobolev

regularity of order N if it has N

derivatives in the sense of the

distributions, and that all are square

integrable.

• This is equivalent to the condition

Z +1

1

|û(!)! ↵ | 2 < 18↵ apple N

jeudi 20 juin 13

25

Sobolev Regularity

Z +1

1

|û(!)! ↵ | 2 < 18↵ apple N

jeudi 20 juin 13

25

Sobolev Regularity

In fact, if f has a Sobolev derivative of order

N, its Fourier integral is given by

f d (N) (!) =(i!) N ˆf(!)

25

jeudi 20 juin 13

The why of low pass

filters

• Regular signals can better be

anticipated because they are

differentiable

• A low pass filter is a filter whose

transfer decays sharply at the infinity

• As a consequence, the output of a low

pass filter is more regular than its input:

it is better conditioned!

jeudi 20 juin 13

26

Regularization by

Convolution

The previous result on low pass filtering

is connected to a well known result about

convolution:

If the impulse response is N times

continuously differentiable, then any

output is also N times continuously

differentiable

jeudi 20 juin 13

27

100

80

60

40

20

0

20

−40

−60

−80

−100

0 200 400 600 800 1000 1200 1400 1600

Application

Here are the previously viewed

temperature variations

jeudi 20 juin 13

28

50

40

30

20

10

0

−10

20

−30

−40

0 200 400 600 800 1000 1200 1400 1600

Application

And here is the signal filtered by an order 20 low pass

Butterworth filter.

Observe that, although we use an analog rational causal

filter, a delay occurs because of the low pass property.

jeudi 20 juin 13

29

Various Fourier

Transform

• If the signal is in continuous time over the

real axis, we use the Fourier Transform

• If the signal is in continuous time over an

interval, we use the Fourier Series

• If the signal is discrete and spans all integers,

we use the Discrete Fourier Transform (DFT)

• If the signal is discrete and has a finite

number of non zero values, we use the

Discrete Time Fourier Series (DTFS)

jeudi 20 juin 13

30

Nature of the Various

Transforms

• The Fourier Transform maps L 2 (R) into itself

• The Discrete Fourier Transform maps discrete

signals into trigonometric series

• The Fourier Series decomposes signals in L 2 ([0,1])

into a Hilbert basis whose elements are sinusoids

• The Discrete Time Fourier Series maps finite

signals into a combination of complex numbers

on the unit circle (it can be seen as a simple

change of basis in a finite dimensional space).

This is what the FFT computes.

jeudi 20 juin 13

31

What’s the Catch

• In practice, if we use a computer (or an

embedded digital system), we

manipulate discrete signals

• But if the «sampling rate» is fast

enough, we can pretend the signal is in

continuous time, and use results from

functional analysis

jeudi 20 juin 13

32

Discretizations

jeudi 20 juin 13

33

Think Analog,

Act Digital

• As we said, we shall do computations

digitally while using some results from

the continuous time theory

• This motivates the study of the

correspondences between continuous

and discrete models

jeudi 20 juin 13

34

Sampling

• We present here the classical sampling

theory which leads to the Shannon

theorem

• Other methods use the decomposition

on a basis to discretize the signal (see

the section on wavelets).

• We start by sampling a filter:

jeudi 20 juin 13

35

Filter Sampling

• Let L a LTI filter with a continuous

impulse response h

• For a given sampling rate T, the

corresponding sampled filter L T is

defined by its impulse response h T

• h T is the product of h with the Dirac

comb

h T (t) =h(t)C(t) =h(t)

n=+1

X

n= 1

(t nT )

jeudi 20 juin 13

36

Filter Sampling

• Since the Diracs distributions are

pointwise supported, h T can also be

written as

h T (t) =

n=+1

X

n= 1

h(nT ) (t nT )

• The output of x can be written as

(h T x)(t) =

n=+1

X

n= 1

h(nT )x(t nT )

jeudi 20 juin 13

37

Signal sampling

• Since the signal and the impulse response

play a symmetrical role a convolution, we see

that the sampling of a signal x is given by

x T (t) =

n=+1

X

n= 1

• Its Fourier Transform is

x(nT ) (t nT )

cx T (!) =

n=+1

X

n= 1

x(nT )e inT ! 38

jeudi 20 juin 13

Shannon Theorem

• It can be shown that the Fourier transform

of the sampled data is the 2"/T periodization

of the Fourier transform of the original data.

• If the support of the Fourier transform is

within [-"/T,"/T], then the periodization

does not cause overlapping. Therefore the

original Fourier transform can be recovered

by restriction to [-"/T,"/T].

• This leads to the following theorem:

jeudi 20 juin 13

39

Shannon Theorem

If the support of the Fourier Transform of

x lies in the band [-"/T,"/T], then x can

be recovered from it samples x(nT) by the

formula

with

x(t) =

n=+1

X

n= 1

sinc(t) = sin t

x(nT )sinc

t

= 1 2

⇣ ⌘

T (t nT )

Z +1

1

e i!t d!

jeudi 20 juin 13

40

Relation with basis

coordinates

• In the case of band limited signals, we

see that sampling is equivalent to finding

the coordinates of the signal on a basis of

shifted sinc functions.

• This relates the sampling to more general

methods of discretization by

decomposition on a basis.

jeudi 20 juin 13

41

Discrete Fourier

Transform (DFT)

If we take T=1 in the Fourier transform of

the sampled data, we can use it to define

the discrete Fourier transform of a

sequence x:

bx(!) =

n=+1

X

n= 1

which is a 2" -periodic function.

x[n]e in! 42

jeudi 20 juin 13

z-Transform

• In many cases the previous expression is

holomorphic and we can expand it to the

whole complex plane by replacing the complex

sinusoid by a general complex variable z. This

yields the z-transform:

X(z) =

n=+1

X

x[n]z n 43

n= 1

• z stands here for the advance operator, which is

not causal. Its value is e i! in the Fourier domain

jeudi 20 juin 13

Fourier Series

• It can be shown that the family

e ikt k2Z

is a basis of L 2 [-",+ "]. Therefore the DFT

maps square integrable discrete sequences to

signals in L 2 [-",+ "].

• Conversely, the computation of the

coordinates of elements of L 2 [-",+ "] in the

previous basis maps a functional space into a

space of sequence. This transform is called

the Fourier Series of the signal.

jeudi 20 juin 13

44

Fourier Series

Specifically, the Fourier series of a signal

x in L 2 [-",+ "] is given by

X[k] = 1

2⇡

Z ⇡

x(t)e ikt dt

with the inverse formula

x(t) =

k=+1

X

k= 1

X[k]e ikt 45

jeudi 20 juin 13

Circular convolution

We define the circular convolution of x

and y in L 2 [-",+ "] by

x ~ y(t) =

Z ⇡

x(t

s)y(s)ds

Then the Fourier Series of

x ~ y

is 2 " times the product of the Fourier

series of x and y

jeudi 20 juin 13

46

Periodic signals

All that was said about the Fourier series

can be extended to periodic signals, and

the circular convolution is the same as

the ordinary convolution if the two

signals have the same period.

jeudi 20 juin 13

47

Discrete Time

Fourier Series

• Naturally, periodic signals can be

sampled, provided the sampling rate

divides the period

• Discrete circular convolutions are

defined, and Discrete Time Fourier

Series are defined accordingly, with the

usual product property

jeudi 20 juin 13

48

A finite number of

«frequencies»

• A brutal applications of the Fourier series on

N-periodic sampled X data yields

X[k] = x[n]e ikn 2⇡ N

n2Z

• (N does not appear in the Fourier series

because the period has been normalized)

• Sampled data Fourier transform are periodic.

Specifically, the transform above is N

periodic. As a consequence, we only need its

values for n = 0 … N-1.

jeudi 20 juin 13

49

Discrete Time

Fourier Series

Therefore, the DTFS of a signal of length

N is given by

X[k] = 1 N

n=N 1

X

n=0

x[n]e ikn 2⇡ N

with the inverse formula

x[n] =

k=N 1

X

k=0

X[k]e ikn 2⇡ N

jeudi 20 juin 13

50

Discrete Time

Fourier Series

• Technically, the previous formulas are

just a change of coordinates between

two bases of a finite dimensional space.

As such, they involve only a finite

number of elementary operations.

• But the important point is that they can

be interpreted as Fourier series of

sampled data.

jeudi 20 juin 13

51

Fast Fourier transform

(FFT)

• The Fast Fourier Transform is a, well, fast implementation

of the Discrete Time Fourier Series

• The observation is that, if a signal has length M=2N, the

usual formula involves M 2 operations. However, a smart

formula shows that it is equivalent to the computation of

2 DTFS of length M, which, in addition, are related.

• If M is a power of 2, this can be repeated, and the

resulting algorithm involves a number of operations

which of the order of M log 2 (M)

But my signal is not

periodic!

• If the important features are persistent,

then the FFT will nonetheless provide

good results

• If the signal present important

transients (whether in the time of

frequency domain) as time varies, then

other transforms, described later in this

course, should be considered.

jeudi 20 juin 13

53

Frequency

Analysis

Some examples

jeudi 20 juin 13

54

2

Sum of two sinusoids at 50 Hz and 120 Hz

1.5

1

0.5

0

−0.5

−1

−1.5

−2

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2

time in seconds

A numerical

example

We consider a signal with 1000 samples on

[0,1]. It is the sum of two sinusoids at 50

and 120Hz with amplitude 0.7 and 1

respectively. The signal is almost periodic.

jeudi 20 juin 13

55

Single−Sided Amplitude Spectrum of y(t)

0.9

0.8

0.7

0.6

|Y(f)|

0.5

0.4

0.3

0.2

0.1

0 50 100 150 200 250 300 350 400 450 500

Frequency (Hz)

A numerical

example

And here is the FFT. Observe the two peaks

at 50 and 120 Hz.

jeudi 20 juin 13

56

6

Signal Corrupted with Zero−Mean Random Noise

4

2

0

−2

−4

−6

−8

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2

time in seconds

A numerical

example

The sinusoids are now perturbed additively

by the double of a normally distributed

pseudo noise.

jeudi 20 juin 13

57

1

Single−Sided Amplitude Spectrum of y(t)

0.9

0.8

0.7

0.6

|Y(f)|

0.5

0.4

0.3

0.2

0.1

0 50 100 150 200 250 300 350 400 450 500

Frequency (Hz)

A numerical

example

This the FFT of the noisy signal. Observe

that the peaks at 50 and 120 Hz dominate

the FFT.

jeudi 20 juin 13

58

1

Signal Corrupted with Zero−Mean Random Noise

0.8

0.6

0.4

0.2

0

−0.2

−0.4

−0.6

−0.8

−1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

time in seconds

A numerical

example

We now use the two sinusoids in

succession.

jeudi 20 juin 13

59

Single−Sided Amplitude Spectrum of y(t)

0.45

0.4

0.35

0.3

|Y(f)|

0.25

0.2

0.15

0.1

0.05

0 50 100 150 200 250 300 350 400 450 500

Frequency (Hz)

A numerical

example

Here is the FFT. The non periodicity make

the FFT a little dirty, but we can see the

same peaks (with half magnitude).

jeudi 20 juin 13

60

1

Signal Corrupted with Zero−Mean Random Noise

0.8

0.6

0.4

0.2

0

−0.2

−0.4

−0.6

−0.8

−1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

time in seconds

A numerical

example

We now switch the two sinusoids.

jeudi 20 juin 13

61

0.5

Single−Sided Amplitude Spectrum of y(t)

0.45

0.4

0.35

0.3

|Y(f)|

0.25

0.2

0.15

0.1

0.05

0 50 100 150 200 250 300 350 400 450 500

Frequency (Hz)

A numerical

example

The FFT is visually unchanged, despite the

radical change in the time localization of

the frequencies.

jeudi 20 juin 13

62

2

Sum of two linear chirps

1.5

1

0.5

0

−0.5

−1

−1.5

−2

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2

time in seconds

Another numerical

example

This is almost the same signal as in the first

example, except that, in the sinus formulas, we

have replaced time by it square. The two modified

sinusoid are called linear chirps, because their

«frequency» varies linearly with time.

jeudi 20 juin 13

63

0.14

Single−Sided Amplitude Spectrum of y(t)

0.12

0.1

0.08

|Y(f)|

0.06

0.04

0.02

0 50 100 150 200 250 300 350 400 450 500

Frequency (Hz)

Another numerical

example

And here is the FFT. The time varying

«frequency» makes the FFT go awry. Note

that, since the time is smaller than 1,

squaring it decreases the «frequency».

jeudi 20 juin 13

64

Fourier

Transforms:

Warnings

jeudi 20 juin 13

65

Gibbs Oscilations

The Fourier series may not converge

pointwise. This called the Gibbs phenomenon.

Here we sum the Fourier series of a ramp.

However, all the terms of the series vanish at t

= 0 and t = "

jeudi 20 juin 13

66

Compact support

• A signal and its Fourier transform

cannot be simultaneously compactly

supported

• As a consequence, a band limited signal

does not have a bounded support, and

a compactly supported signal cannot be

band limited

jeudi 20 juin 13

67

Time-Frequency Centers

If a signal x is square integrable, we can

define its center u by

u = 1

kxk 2 ZR

t|x(t)| 2 dt

and its frequency center # by

⇠ =

1

2⇡kxk 2 ZR

!|ˆx(!)| 2 d!

(2" is here because we use a Fourier

integral)

jeudi 20 juin 13

68

We then define the time spread \$ t by

2

t = 1

ZR kxk 2 (t u) 2 |x(t)| 2 dt

and the frequency spread \$ ! by

2

! =

ZR

1

2⇡kxk 2 (! ⇠) 2 |ˆx(!)| 2 d!

jeudi 20 juin 13

69

Heisenberg Theorem

Then we have

t !

1

2

In other words, if we generalize the

width of a signal to the L 2 sense, the

spreads cannot be arbitrarily small both

in the time and the frequency domain.

The rectangle with center (u, #) and dimensions \$ t , \$ ! is called the

Heisenberg box of the signal x.

jeudi 20 juin 13

70

Analog IIR

Filter Design

jeudi 20 juin 13

71

IIR Filter Design

• Not exhaustively, we present two examples of

analog filter design. Both have a rational transfer,

and thus have an Infinite Impulse Response (IIR).

• FIR filter design will be presented using

windowing. It is generally used in discrete time

and will be presented later.

• Here both IIR filters are low pass filter; we shall

see later how to discretize these filters and

modify their passing band.

jeudi 20 juin 13

72

Analog Filter Design:

the Butterworth Filter

Assume, to simplify, that the cutting

frequency of the filter that we search is 1.

Then the Butterworth filter of order N has

no zero, a static gain that is equal to 1, and

its poles are conjugate, stable, complex roots

of 1.

Renormalization of the poles and of the gain

allows the choice of any cutting frequency.

jeudi 20 juin 13

73

1.4

1.2

1

0.8

0.6

0.4

0.2

0

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

Butterworth filter

of order 10

Amplitude of the transfer. The cutting

jeudi 20 juin 13

74

1.4

Step Response

1.2

1

Amplitude

0.8

0.6

0.4

0.2

0

0 5 10 15 20 25 30 35

Time (sec)

Butterworth filter

of order 10

Step response of the filter. Observe the

delay of 4.

jeudi 20 juin 13

75

1.4

1.2

1

0.8

0.6

0.4

0.2

0

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

Butterworth filter

of order 50

The transfer if much closer to an ideal low

pass filter.

jeudi 20 juin 13

76

1.4

Step Response

1.2

1

Amplitude

0.8

0.6

0.4

0.2

0

0 20 40 60 80 100 120 140

Time (sec)

Butterworth filter

of order 50

Step response of the Butterworth filter of

order 50. By comparison to the order 10, the

delay has been multiplied by 6, and the

stabilization is much slower.

jeudi 20 juin 13

77

Analog Filter Design

Chebychev filters

The magnitude of the transfer of the

Butterworth filters have a simple behavior:

they are almost equal to 1 until they reach

the cutting frequency, then decay in a band

which includes the cutting frequency, then

decay rapidly to 0.

For general low pass filters, things are a

little more complicated. We shall take the

example of Chebychev filters.

jeudi 20 juin 13

78

1

0.9

0.8

1

1+✏ 2

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0

0 0.5 1 1.5 2 2.5 3

1

A 2

⌦ p

⌦ s

Ripples and

transition band

Starting at the zero frequency, the transfer stays

between 1 and 1/(1/% 2 ). & p is the frequency where

the transfer decreases until & s where it stays under

the value 1/A 2 ; ripples can also occur thereafter.

The cutoff frequency is between & p and & s .

jeudi 20 juin 13

79

Chebychev low pass

filters

• The Chebychev filters are called thus because their

poles are roots of a Chebychev polynomial

• Chebychev filters display ripples is the pass band

but are monotone in the stop band

• The gain (by comparison to the Butterworth filters)

is that the same behavior in the transition band is

achieved with much lower orders.

• There are two type of Chebychev filters: type I

exhibit ripples in the pass band, type II exhibit

ripples in the stop band.

jeudi 20 juin 13

80

1

Chebychev filter of order 5 with ripple parm epsilon = 0.50885

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0

0 0.5 1 1.5 2 2.5 3

Order 5 Chebychev

filter: transfer

While the pass band behavior is worse than

the order, the transition and stop band

behaviors are similar.

jeudi 20 juin 13

81

1.4

1.2

1

0.8

0.6

0.4

0.2

0

0 20 40 60 80 100 120 140

Order 5 Chebychev

filter: step response

The delay is equal to 5 and similar to what

we have on the Butterworth filter.

jeudi 20 juin 13

82

1

Chebychev filter of order 20 with ripple parm epsilon = 0.50885

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0

0 0.5 1 1.5 2 2.5 3

Order 20 Chebychev

filter: transfer

The transition and stop band behaviors are

very good, and close to what is obtained

with the order 50 Butterworth filter. Here

the order is 20.

jeudi 20 juin 13

83

1.4

1.2

1

0.8

0.6

0.4

0.2

0

0 100 200 300 400 500 600

Order 20 Chebichev

filter: step response

The delay is roughly equal to 50 whereas it

is less than 30 for the order 50 Butterworth

filter.

jeudi 20 juin 13

84

Analog Low Pass to High

Pass Conversion

• The following method can be used to design an

analog high pass filter with cutting frequency

! c :

• design a low pass filter with cutting frequency 1

(prototype filter)

• replace the Laplace variable s by

This is used in the temperature processing

example below.

s

! ! c

s

jeudi 20 juin 13

85

Analog Low Pass to

Band Pass Conversion

• Assume that we have a prototype

(normalized) low pass filter

• We wish to transform it into a band pass filter,

with midband frequency ! 0 and bandwidth B.

• A solution consists in replacing the Laplace

variable s by

s ! s2 + ! 2 0

Bs

jeudi 20 juin 13

86

Analog to discrete

Filter Conversion

jeudi 20 juin 13

87

Introduction

• According what was exposed before,

there is one canonical way to discretize

an analog signal (or filter): use a Dirac

comb

• Unfortunately (Shannon) this only

works for band limited filters

• For general filters there is a better

method.

jeudi 20 juin 13

88

The Bilinear Transform

It uses the following transform which

maps the Laplace variable s into the z

variable of the z-transform:

s = 2 T

1 z 1

1+z 1 89

z standing for the advance operator (and s

for the derivative operator).

jeudi 20 juin 13

Properties of the

Bilinear Transform

• It maps the analog stable half plane to

the discrete stable disk.

• It is a one to one mapping between the

imaginary axis and the unit circle. T is a

scaling factor between the Fourier

variable ! and the angle of the z variable

• It maps the unstable half plane into the

complement of the stable disk.

jeudi 20 juin 13

90

Frequency mapping

• we assume that the Laplace variable s is

purely imaginary, i.e. s=i". Then the z

variable has a unit modulus, ✓ z=e ◆

i! , with

! = 2 tan 1 ⌦T

2

• we see that, if we sample " linearly, then ! is

sampled nonlinearly, ranging from -"/2 to "/

2. The benefit is that to each " corresponds a

unique !. If T is small, then ! is more dense.

The mapping is nearly linear for small ".

jeudi 20 juin 13

91

Warning

• The bilinear transform is reliable if you

work in zero/pole transfer form

• If your output is in rational form, and if the

order of the transfer is large with typically a

small cutting frequency (or a small T) a

stable analog transfer may numerically

produce an unstable discrete transfer.

• When this happens, try to reduce the order of

the analog filter.

jeudi 20 juin 13

92

Fiddling

with passing bands

• We start here with discrete low pass

filters. They may have been discretized

as in the previous slides, or designed

directly. Their cutoff frequency is !’ c

• in what follows, z is the complex

variable that stands for the advance

operator. Frequencies range from 0 to "

jeudi 20 juin 13

93

From low pass to

low pass

• This is essentially a change in the cutoff

frequency, the new one being ! c

• Let

↵ = sin[(!0 c ! c )/2]

sin[(! 0 c + ! c )/2]

• Then substitute

z ! z 1

1 ↵z 1 94

jeudi 20 juin 13

From low pass to

high pass

• ! c is the new cutting frequency

• Let

↵ = cos[(!0 c + ! c )/2]

cos[(! 0 c ! c )/2]

• Substitute

z ! z 1 + ↵

1+↵z 1 95

jeudi 20 juin 13

From low pass to

band pass

• ! l is the lower cutting frequency, ! u is the

upper cutting frequency

• Define

= cos[(! u + ! l )/2]

cos[(! u ! l )/2]

K = cot ! u

tan !0 c

2 2

↵ 1 = 2 K/(K + 1),↵ 2 =(K 1)(K + 1)

! l

• substitute

z ! z 2 ↵ 1 z 1 + ↵ 2

↵ 2 z 2 ↵ 1 z 1 +1

jeudi 20 juin 13

96

From low pass to

stop band

• The formulas are the same as for the

band pass, except that

• in the definition of K, a tangent replaces

the cotangent

factor disappears.

jeudi 20 juin 13

97

The bottom line

If your original low pass filter has a

rational transfer (like a Butterworth or

Chebychev filter) then the final filters

have a rational transfer.

jeudi 20 juin 13

98

Example

Application

jeudi 20 juin 13

99

25

Centered annual temperature

20

15

10

5

0

−5

−10

−15

20

0 50 100 150 200 250 300 350 400

Days

Annual

Temperature

Here is the temperature somewhere in

France, with one-hour sample rate over 365

days. (The average temperature has been

set to zero).

jeudi 20 juin 13

100

Power spectrum of the anuual temperature; spike is at period 23.9945 hours

14 x 104 Frequency (Hz)

12

10

|hat(temp)| 2

8

6

4

2

0

0 0.2 0.4 0.6 0.8 1 1.2 1.4

x 10 −4

And here is its FFT.

Observe the spike.

We can detect the spike by sorting the

values of the spectrum by decreasing order.

The relevant spike comes at position 3, for a

period of 23,9945 hours.

jeudi 20 juin 13

101

• Although the signal is far from being

periodic from day to day, there is a

strong and isolated component at 24

hours in the FFT.

• This is due to the persistence of the day/

night cycle.

• Let us try to separate the intra day from

the inter day phenomena by using

Butterworth filters.

jeudi 20 juin 13

102

To do so, we use

Simulink allows us to use continuous time

We use Butterworth low pass and high pass

filters to separate the intraday from the

interday phenomena.

jeudi 20 juin 13

103

1.4

1.2

1

abs(transfer)

0.8

0.6

0.4

0.2

0

0 0.01 0.02 0.03 0.04 0.05 0.06 0.07

Hz

We use a Butterworth

of order 50 for the

lowpass

The cutting frequency corresponds to a

period slightly greater than 24 hours, to

accomodate the transition band. The

frequency for a period of 24 hours is 0.0417.

jeudi 20 juin 13

104

1.4

1.2

1

abs(transfer)

0.8

0.6

0.4

0.2

0

0 0.01 0.02 0.03 0.04 0.05 0.06 0.07

Hz

We use an order 30

Butterworth high pass filter

designed in the analog domain

The cutting frequency is the same. We have

taken an order 30 prototype low pass filter

and replaced s by ! c /s.

jeudi 20 juin 13

105

20

Day to day low pass component of the temperature

15

10

5

0

−5

−10

−15

20

0 1 2 3 4 5 6 7

x 10 4

Here is the output of

the temperature

through the low pass

We now see only the variations from day to

day.

jeudi 20 juin 13

106

10

Intra day high pass component of the temperature

8

6

4

2

0

−2

−4

−6

−8

−10

0 1 2 3 4 5 6 7

x 10 4

And here is the output

of the High Pass

We only see the variations within each day.

jeudi 20 juin 13

107

Discrete FIR

Filter Design

jeudi 20 juin 13

108

Introduction

• Finite Impulse Response (FIR) filters are

filters whose impulse response, by

contrast to IIR filters, have a bounded

support

• Continuous time FIR filters are not easy to

implement, so we shall concentrate on

Discrete FIR filters, which are very easy to

implement by a finite discrete

convolution.

jeudi 20 juin 13

109

Windowing

• We concentrate on low pass filter

design. Other band pass filters can be

built from here using previously

described methods.

• The idea is to take a ideal discrete low

pass filter and multiply its impulse

response by a window with bounded

support

jeudi 20 juin 13

110

Transfer

• In the Fourier domain, and since the data is

discrete, the windowing appears as the circular

convolution of a rectangular window with the

DFT of the window.

• If the window has M samples and M is large,

then the main lobe of its DFT will be narrow

and the convolution with the rectangular

window will not distort it much. The

downside is that the processing will take

longer.

jeudi 20 juin 13

111

ŵ !

2

Δω

-ω 0

ω 0

A

^w(ω)

A

2

|ŵ(0)| 2 = 1 2

A = 10 log 10

|ŵ(! 0 )| 2

|ŵ(0)| 2

|ŵ(!)| = O(! p 1 )

|!| !1

t

Window Parameters

We plot here the Fourier integral of the

continuous time window w. Three important

parameters: '!, A and p, help determine the

effect on the convolution with the rectangle.

jeudi 20 juin 13

112

Outline of the Transfer

• We consider a discrete ideal low pass

filter with cutting frequency ! c not to

close to 0 or to ". Moreover we assume

that the width of the main lobe of the

window %! is smaller than 2! c .

• The window is assumed to be

symmetric with parameters %!, A and

p.

jeudi 20 juin 13

113

Outline of the Transfer

(in amplitude)

• The transfer has roughly the shape of a low pass

transfer, so we can use the corresponding terms.

• The transition band is centered at ! c and has a

width slightly smaller than %!.

• The ripple height has value A in decibels, i.e. the

main ripple is due to the convolution with the

first side lobe.

• If p is large, i.e. if w is quite smooth, then its

transfer decays fast and the window sampling

does not create significant aliasing.

jeudi 20 juin 13

114

Common Windows

Windows with unit norm and support [-1/2,1/2]

Name w(t) '! A p

Rectangle 1 0.89 -13db 0

Hamming 0.54+0.46cos(2"t) 1.36 -43db 0

Gaussian exp(-18t 2 ) 1.55 -55db 0

Hann(ing) cos 2 ("t) 1.44 -32db 2

Blackman

0.42+0.5cos(2"t)

+0.08cos (4"t)

1.68 -58db 2

Kaiser windows are effective

and are computed with Bessel functions

jeudi 20 juin 13

115

Sampling Rate

Conversion

jeudi 20 juin 13

116

Introduction

• It is common to deal with signals which have

different sampling rates. For practical reasons, it

is often necessary to synchronize them, which

implies sampling rate conversions.

• We consider here the multiplication of the

sampling rate by a rational number N/D, N and

D being coprime.

• The material explained here will be re-used when

implementing the Discrete Wavelet Transform.

jeudi 20 juin 13

117

Downsampling

• We start our sampling rate conversion

by downsampling the signal by a factor

D.

• This can be naively done by

periodically picking 1 value out of D

consecutive ones (this is called

decimation)

jeudi 20 juin 13

118

Downsampling

• However, we must try to avoid aliasing.

• Imagine that the original samples correspond

to a band limited signal and that the Shannon

theorem applies.

• If we wish to keep the Shannon criterion

valid, then the pass band of the original

signal must be narrowed by a factor D.

• To do so, we pre-process the signal by a low

pass filter before decimation.

jeudi 20 juin 13

119

Ideal Decimator

Ideal Decimator

x

LP

D

y

Ideal Low!

Pass Filter

"Lazy"!

Decimator

• The low pass filter narrows the pass

band by to [-"/D,"/D]

• The «lazy» decimator keeps one value

out of D.

jeudi 20 juin 13

120

Upsampling

• We wish now to create more samples

than available, by a factor I.

• To do so, we insert (I-1) zeros between

each sample

jeudi 20 juin 13

121

Post processing

• If we convert the z-transform to a DFT

using the new sampling rate, we see that

the DFT has been compressed by a factor I.

This causes aliasing, as periodic replicas

of the original DFT appear in [-",+ "].

• To avoid this, we post process the signal

by a low pass filter of pass band [-"/I,+

"/I].

jeudi 20 juin 13

122

Ideal Interpolator

Ideal Interpolator

x

I

LP

y

"Lazy"!

Upsampler

Ideal Low!

Pass Filter

• The «lazy» upsampler simply inserts

zeros

• The low pass filter removes aliasing

jeudi 20 juin 13

123

Sampling Rate Conversion

by a factor I/D

• Observe that the interpolator preserves the

shape of the DFT (on the pass band), while

the decimator does not.

• This is why is preferred to interpolate first

by a factor I, then decimate by a factor D.

• Observe that the two low pass filters

operate in cascade at the same rate, and can

be combined into a single low pass filter

with a suitable pass band.

jeudi 20 juin 13

124

The Discrete Cosine

Transform and JPEG

jeudi 20 juin 13

125

Introduction

• As we have seen, the DTFS (and hence the

FFT) can be interpreted as the Discrete

Fourier Transform of a N-periodic signal.

• As such, if we perform an FFT of a finite

set of data where the first value is not

close to the end value, we have large

Fourier coefficients due to the

discontinuity at the boundary.

jeudi 20 juin 13

126

The folding trick

• To make the periodic signal continuous,

we first make it symmetric by folding

f(t)

0

N

• This way, the periodized signal is

continuous.

jeudi 20 juin 13

127

The new DTFS

• Because of the symmetry, the new

signal’s DTFS can be written as

ˆX k = c k e ki⇡

2N

• with

c k =2

m=N 1

X

m=0

x m cos

✓ k⇡(m +1/2)

N

• observe that, if we know all the c k , we

can recover the DTFS and hence, x.

jeudi 20 juin 13

128

The DCT

• The Direct Cosine Transform (DCT) (of type II) of x

r

is defined by

m=N

1 X

1

c 0 =

N

c k =

r

2

N

for 1 ≤ k ≤ N-1.

m=N

X

1

m=0

x m cos

m=0

✓ ◆

k⇡(m +1/2)

• Observe that the DCT is real if x is real.

• as k increases, c k represents the high frequency

component of the DTFS of the folded signal.

N

x m

129

jeudi 20 juin 13

Matrix form

• The DCT is a linear operator from R N

into R N . As such, it can be represented

by a matrix C N of size NxN.

• Careful investigation of C N shows that

it is orthogonal, i.e. C

-1 N = C

T N

• In the coding and decoding of JPEG,

N=8

jeudi 20 juin 13

130

The DCT in dimension 2

• If A is an mxn matrix, its DCT is

defined by

Â = C m AC T n

• This is equivalent to performing a DCT

row by row then column by column (or

in the other order).

jeudi 20 juin 13

131

Theoretical Benefits

• The DCT is not sensitive to the non

periodicity of the signal

• It is a real transform

• it retains some frequency meaning on

the folded signal

jeudi 20 juin 13

132

JPEG

step 1

• Color management

• A digital image consists generally, for each

pixel, in a triplet of 8-bit values

• It maybe wise at this stage to change the

color space representation (for instance, hue,

saturation and value)

• we shall consider in what follows that we

have a grayscale image, or more generally,

that we work on the value channel.

jeudi 20 juin 13

133

JPEG

Step 2: DCT

• It the number of column or rows is not a

multiple of 8, the image is padded with values

(depending of the algorithm).

• A DCT is performed on each bloc of 8x8 pixels.

The DCT is not affected by the non periodicity of

each small block.

• It may be efficient to represent the DCT

coefficient at the zero frequency (average) by it

difference with the average of the previous block

jeudi 20 juin 13

134

JPEG

Step 3: quantization

• Let b k,m the values of the DCT on a block,

e k,m an 8x8 table of quantization integers;

additionally, a real number r can be used

to influence the quality of the quantization

• The DCT is quantized by the formula

q(b k,m ) = round

bk,m

e k,m r

• Examples of tables for e can be found in

textbooks

jeudi 20 juin 13

135

JPEG

Step 4: compression

• The combined effect of the DCT and of the

quantization is to replace a large number of

coefficients by 0. This why JPEG is called a lossy

encoding.

• The large number of zeros can be efficiently

represented by using Run Length Encoding (RLE)

• Standard computer compression algorithms can

then be used to further reduce the size of the

transformed image

• Note: the last two steps are lossless

jeudi 20 juin 13

136

JPEG

Decompression

• The computer compression and RLE are

easily inverted

• The DCT coefficients can only be

recovered approximatively because of

the quantization; we get

˜bk,m = q(b k,m )e k,m r

• Then the image can be recovered by

inverting the DCT

jeudi 20 juin 13

137

JPEG

why it works

• In many parts of the image, the 8x8

blocks contains «smooth» image parts

• as a result, the DCT coefficients at high

frequencies (k and m large) are small

and quantized into 0. The result is a

block with many zeros in the lower

right part of the quantized DCT.

jeudi 20 juin 13

138

Windowed

Fourier Transform

and Instantaneous Frequency

jeudi 20 juin 13

139

Introduction

• We have seen on an example that the

Fourier transform is unable to localize

in time two successive sinusoids.

• If we know the support of the first

sinusoid of the example, we can first

multiply the signal by the characteristic

function of the support of the first

sinusoid.

jeudi 20 juin 13

140

Definition

• The previous example is particularly

simple because the restriction of the

signal is exactly a sinusoid

• For more complex signals, we need a

more general definition

• The main purpose is to detect «local»

frequencies

jeudi 20 juin 13

141

Definition

• Let w a real, symmetric function with

unit square integral.

• The Windowed Fourier Transform

(WFT) of x is Zdefined by

Sx(t, !) =

R

x(s)w(s t)e i!s ds

• It is the Fourier integral of x multiplied

by a translate of the window w.

The S in Sx stands for «Short Time Fourier Transform».

jeudi 20 juin 13

142

example

• Let us consider the case where x is a sinusoid

• Then its WFT is

x(t) =e i! 0t

Sx(t, !) =e i(! 0 !)tŵ(!

! 0 )

• Its frequency spread is centered at ! 0 and its width

is the same as the frequency width of the window

• As consequence, we see that the choice of a

rectangular window may not be the best, as its

Fourier transform (a sinc function) has a wide

jeudi 20 juin 13

143

Spectrogram

• The spectrogram of x is the square of the

magnitude of its WFT

• On the sinusoid example, we see that the

spectrogram is centered at ! 0 , and is a translation

of the square of the magnitude of the Fourier

transform of the window

• If the Fourier transform of w is narrow, then the

frequency ! 0 is well located by the spectrogram.

Observe that the Fourier transform of w decays

fast if w is regular.

jeudi 20 juin 13

144

Inversion

• The WFT can be inverted by the

following formula

x(t) = 1 Z

Sx(s, !)w(t s)e i!t d!ds

2⇡ R 2

• By contrast to the Fourier transform, not

all functions of two variables are the

WFT of a signal. This is because the WFT

is a highly redundant transform.

jeudi 20 juin 13

145

Instantaneous

Frequency

• We wish to use the WFT to study the

frequency content of a signal locally in

time

• To do so, we introduce the

instantaneous frequency of the

analytical part of a signal.

jeudi 20 juin 13

146

Caveat

• We know, from the Heisenberg theorem,

that the frequency spread of a signal is 0

if and only if its time spread is infinite;

so it is (in L 2 ) hopeless to define the

frequency of a signal locally in time

• A definition of the instantaneous

frequency can be given is the signal is

«analytic»

jeudi 20 juin 13

147

«Analytic» Signal

• The «analytic» part f a of a signal f is

defined by

ˆf a (!) = 2ˆf(!) if! 0

ˆf a (!) = 0 if !

Instantaneous Frequency

of an «analytic» signal

If f is an «analytic» signal, then it is a

complex signal which can be uniquely

decomposed into its modulus and phase

f(t)=a(t)e i#(t)

Its instantaneous frequency is the derivative

'’(t) of the phase. For non «analytic» signals

the definition is extended by taking the

instantaneous frequency of their «analytic»

part.

jeudi 20 juin 13

149

Single Frequency

• assume x(t)=a(t)cos(! 0 t+#)

• If we assume that a is band limited with

band in [-! 0 ,+! 0 ], then the «analytical»

part of x is x a (t) = a(t)e i(! 0 t+#) and its

instantaneous frequency is ! 0 , so all is

well.

jeudi 20 juin 13

150

Multiple Frequencies

• If we consider now the sum of cosine signals

(the amplitude is constant), then one easily

see that the instantaneous frequency of the

sum is the average of the two frequencies,

which is not intuitive; we would rather have

two instantaneous frequencies.

• This can be obtained by the WFT and its

spectrogram. To do so, we have to chose a

window.

jeudi 20 juin 13

151

Window Choice

support equal to [-1/2,1/2]

• As a consequence, its Fourier transform has

infinite support. We shall assume that its main

component is a lobe centered at 0.

• Its frequency parameters %!, A and p have been

defined in the section about FIR filter design.

• Standard windows (rectangle, Hamming,

Gaussian, Hann(ing), Blackman…) have been

described in the same section.

jeudi 20 juin 13

152

Scaling

• A scaling parameter s can be introduced

to transform w into w s with

✓ t

s◆

w s (t) =

1 p s

w

• The time spread of w is multiplied by s,

and its frequency spread is divided by s.

jeudi 20 juin 13

153

Assumptions on the

signals

Let x(t) = a(t)cos'(t). We assume that a(t)

and '’(t) have small variations over

intervals that have the size of the scaling

parameter s, and that

'’(t)≥ %!/s

the right hand side being the mean

bandwidth of the scaled window.

jeudi 20 juin 13

154

Ridge Points

• Points (t,!) in the time frequency plane where the

spectrogram is locally maximum with respect to ! for

a fixed t are called ridge points.

• If x is a signal that satisfies the assumptions of the

previous slide, then its spectrogram has ridge points

which are located, at the first approximation, at

(t,#’(t)) where #’(t) is the instantaneous frequency of x

at time t. The amplitude can the be recovered from the

value of the spectrogram.

• Under these assumptions, Fourier ridges recovers

instantaneous frequencies.

jeudi 20 juin 13

155

Multiple Frequencies

from Ridge Points

• Let x = x1+x2 such that x1 and x2 satisfy the

previous assumptions with respect to the

window. Then, at the first approximations, the

spectrogram of x has ridge points at the locations

(t,#’ 1 (t)) and (t,#’ 2 (t)), where #’ 1 (t) and #’ 2 (t) are

the instantaneous frequencies of x1 and x2, if

|' 0 1(t) ' 0 2(t)|

• If this not the case, the two ridges overlap

s

!

jeudi 20 juin 13

156

0.5

0

−0.5

100 200 300 400 500 600 700 800 900 1000

500

400

Frequency

300

200

100

Example: sum of two

linear chirps (From S.

Mallat’s book)

A linear chirp is a sinusoid whose frequency

increases linearly with time, for instance

x=sin(!t 2 ).

We have tried to analyze such a signal with a

classical Fourier transform, and it was a disaster.

jeudi 20 juin 13

157

0.5

0

−0.5

100 200 300 400 500 600 700 800 900 1000

500

400

Frequency

300

200

100

0

0 100 200 300 400 500 600 700 800 900 1000

500

400

Example:

300

sum of two

linear chirps (From S.

200

Frequency

Mallat’s book)

100

Here is a spectrogram, computed with a

Gaussian window and a scaling factor s =

50.

jeudi 20 juin 13

158

100

0

0 100 200 300 400 500 600 700 800 900 1000

500

400

Frequency

300

200

100

0

100 200 300 400 500 600 700 800 900 1000

Example: sum of two

linear chirps (From S.

Mallat’s book)

Here are the ridge points. The instantaneous

frequencies are well discriminated thanks to

the right choice of the scaling factor with

respect to the signal’s characteristics.

jeudi 20 juin 13

159

100

0

0 100 200 300 400 500 600 700 800 900 1000

500

400

Frequency

300

200

100

0

100 200 300 400 500 600 700 800 900 1000

Example: sum of two

hyperbolic chirps (From

S. Mallat’s book)

A hyperbolic chirp is a signal of the form acos(%/(&-t)). Here is the

plot of the ridges of the sum of two hyperbolic chirps. As time

increases, the variations of the instantaneous frequencies increase

and after some time the assumption on the signal with respect to

the window cannot be satisfied. This explains why the Fourier

ridges fail to discriminate the frequencies when they get high.

jeudi 20 juin 13

160

Discrete Windowed

Fourier Transform

• The formula for the DWFT is obtained by

taking the formula of the DTFS and replacing

x[n] by x[n]w[n-m] where m is the running

index that stands for the time argument in the

WFT. The window w is a discrete symmetric

sequence of period N with unit norm.

• As a consequence, it can be computed by a

sequence of FFTs. The previous figures have

been computed this way. Be careful in the

choice of N and s.

jeudi 20 juin 13

161

More magazines by this user
Similar magazines