Spectral Analysis and Time Series - max planck research school ...

solar.system.school.de

Spectral Analysis and Time Series - max planck research school ...

A. Lagg – Spectral Analysis

Spectral Analysis and Time Series

Andreas Lagg

Part I: fundamentals

on time series

classification

prob. density func.

auto­correlation

power spectral density

cross­correlation

applications

pre­processing

sampling

trend removal

Part II: Fourier series

definition

method

properties

convolution

correlations

leakage / windowing

irregular grid

noise removal

Part III: Wavelets

why wavelet

transforms?

fundamentals:

FT, STFT and

resolution problems

multiresolution

analysis: CWT

DWT

Exercises


d

e

t

e

r

m

A. Lagg – Spectral Analysis

Basic description of physical data

deterministic: described by explicit mathematical relation

xt=X cos

k t t

n

non o deterministic: no way to predict an exact value at a future instant of time

n


A. Lagg – Spectral Analysis

Classifications of deterministic data

Deterministic

Periodic

Nonperiodic

Sinusoidal

Complex Periodic

Almost periodic

Transient


A. Lagg – Spectral Analysis

Sinusoidal data

xt=X sin2 f 0

t

T =1/ f 0

time history

frequency spectrogram


A. Lagg – Spectral Analysis

Complex periodic data

xt=xt±nT

n=1,2,3,...

xt = a 0

2 ∑ a n

cos 2n f 1

t b n

sin 2 n f 1

t

(T = fundamental period)

time history

frequency spectrogram


A. Lagg – Spectral Analysis

Almost periodic data

xt=X 1

sin2 t 1

X 2

sin3t 2

X 3

sin50 t 3


no highest common divisor ­> infinitely long period T

time history

frequency spectrogram


A. Lagg – Spectral Analysis

Transient non­periodic data

all non­periodic data other than almost periodic data

xt={ Ae−at t≥0

0 t0

xt={ Ae−at cos bt t≥0

0 t0

xt={ A c≥t≥0

0 ct0


A. Lagg – Spectral Analysis

Classification of random data

Random Data

Stationary

Nonstationary

Ergodic

Nonergodic

Special classifications

of nonstationarity


A. Lagg – Spectral Analysis

stationary / non stationary

collection of sample functions = ensemble

data can be (hypothetically) described by

computing ensemble averages (averaging over

multiple measurements / sample functions)

mean value (first moment):

x

t 1

= lim

N ∞

1

∑ N

N k=1

autocorrelation function (joint moment):

R x

t 1,

t 1

= lim

N ∞

x k

t 1


1

∑ N

N k=1

x k

t 1

x k

t 1


stationary: x

t 1

= x

, R x

t 1,

t 1

=R x

weakly stationary: x

t 1

= x

, R x

t 1,

t 1

=R x


A. Lagg – Spectral Analysis

ergodic / non ergodic

Ergodic random process:

properties of a stationary random process

described by computing averages over only one

single sample function in the ensemble

mean value of k­th sample function:

x

k=lim

T ∞

autocorrelation function (joint moment):

T

1

∫ T

0

x k

tdt

R x

, k=lim

T ∞

T

1

∫ T

0

x k

t x k

tdt

ergodic:

x

k= x

, R x

, k=R x


Basic descriptive properties of random data

mean square values

propability density function

autocorrelation functions

power spectral density functions

(from now on: assume random data to be stationary and ergodic)

A. Lagg – Spectral Analysis


A. Lagg – Spectral Analysis

Mean square values

(mean values and variances)

describes general intensity of random data: 2 x

= lim

T ∞

rout mean square value:

x

rms

= x

2

T

1

∫ T

0

x 2 tdt

often convenient:

static component described by mean value:

x

= lim

T ∞

T

1

∫ T

0

xtdt

dynamic component described by variance:

x

2

= lim

T ∞

1

∫ T

T

0

[ xt− x

] 2 dt

2

standard deviation: x

= x

= x 2 − x

2


A. Lagg – Spectral Analysis

Probability density functions

describes the probability that the data will assume a value within some defined

range at any instant of time

for small x :

Prob[ xxt≤x x] = lim

T ∞

Prob[ xxt≤x x] ≈ p x x

probability density function

px =

lim

x 0

probability distribution function

P x

= Prob[xt≤x]

x

= ∫ pd

−∞

Prob[ xxt≤x x]

x

T x

= lim

x 0

k

T , T x = ∑

i=1

1

x [ lim

T ∞

t i

T x

T ]


Illustration: probability density function

sample time histories:

sine wave (a)

sine wave + random noise

narrow­band random noise

wide­band random noise

all 4 cases: mean value x

=0

A. Lagg – Spectral Analysis


A. Lagg – Spectral Analysis

Illustration: probability density function

probability density function


A. Lagg – Spectral Analysis

Autocorrelation functions

describes the general dependence of the data values at one time on the

values at another time.

R x

= lim

T ∞

T

1

∫ T

0

xt xtdt

x

= R x

∞ 2 x

= R x

0 (not for special cases like sine waves)


A. Lagg – Spectral Analysis

Illustration: ACF

autocorrelation functions (autocorrelogram)


A. Lagg – Spectral Analysis

Illustrations

autocorrelation function of a rectangular pulse

xt xt−

autocorrelation function


A. Lagg – Spectral Analysis

Power spectral density functions

(also called autospectral density functions)

describe the general frequency composition of the data in terms

of the spectral density of its mean square value

f ,

f f

mean square value in frequency range :

x 2 f , f = lim

T ∞

T

1

∫ T

0

definition of power spectral density function:

G x

f =

lim

f 0

x 2 f , f

f

xt , f , f

2 dt

portion of x(t) in (f,f+∆f)

f)

= lim

f 0

x 2 f , f ≈ G x

f f

1

f [ lim

T ∞

T

1

∫ T

0

important property: spectral density function is related to the

autocorrelation function by a Fourier transform:

G x

f = 2∫

−∞


R x

e −i 2 f d = 4∫

0


xt , f , f 2 dt]

R x

cos 2 f d


A. Lagg – Spectral Analysis

Illustration: PSD

power spectral density functions

Dirac delta function at f=f 0

sine wave

sine wave

+ random noise

narrowband

noise

“white” noise:

spectrum is uniform over

all frequencies

broadband

noise


A. Lagg – Spectral Analysis

Joint properties of random data

until now: described properties of an

individual random process

Joint probability density functions

joint properties in the amplitude domain

Cross­correlation functions

joint properties in the time domain

joint probability measurement

Cross­spectral density functions

joint properties in the frequency domain


A. Lagg – Spectral Analysis

Cross­correlation function

describes the general dependence of

one data set to another

R xy

= lim

T ∞

T

1

∫ T

0

xt ytdt

similar to autocorrelation function

R xy

=0

functions are

uncorrelated

cross­correlation measurement

typical cross­correlation plot (cross­correlogram):

sharp peaks indicate the existence of a correlation

between x(t) and y(t) for specific time displacements


A. Lagg – Spectral Analysis

Applications

Measurement of time delays

2 signals:

different offset

different S/N

time delay 4s

often used:

'discrete' cross correlation coefficient

lag =l, for l >=0:

R xy

l =

N −l


k=1

∑ N

k=1

x k

−xy kl

−y

N

x k

−x 2 ∑ y k

−y 2

k=1


A. Lagg – Spectral Analysis

Applications

Detection and recovery from

signals in noise

3 signals:

noise free replica of the signal

(e.g. model)

2 noisy signals

cross correlation can be used to

determine if theoretical signal is

present in data


A. Lagg – Spectral Analysis

Pre­processing Operations

sampling considerations

trend removal

filtering methods

sampling

cutoff frequency (=Nyquist

frequency or folding frequency)

f c

= 1

2 h


A. Lagg – Spectral Analysis

Trend removal

often desirable before performing a

spectral analysis

Least­square method:

time series: ut

desired fit

(e.g. polynomial):

K

u=∑

k=0

N

Lsq­Fit: minimize Qb=∑

set partial

derivatives to 0:

∂Q

n=1

∂ b l

=∑

n=1

K

K+1 equations: ∑

k=0

N

bk nh k

u n

−u n

2

n=1,2,... , N

2u n

−u n

[−nh l ]

N

b k ∑ nh kl

n=1

N

= ∑

n=1

u n

nh l


Digital filtering

A. Lagg – Spectral Analysis


end of part I ...

A. Lagg – Spectral Analysis


A. Lagg – Spectral Analysis

Spectral Analysis and Time Series

Andreas Lagg

Part I: fundamentals

on time series

classification

prob. density func.

auto­correlation

power spectral density

cross­correlation

applications

pre­processing

sampling

trend removal

Part II: Fourier series

definition

method

properties

convolution

correlations

leakage / windowing

irregular grid

noise removal

Part III: Wavelets

why wavelet

transforms?

fundamentals:

FT, STFT and

resolution problems

multiresolution

analysis: CWT

DWT

Exercises


A. Lagg – Spectral Analysis

Fourier Series and Fast Fourier Transforms

Standard Fourier series procedure:

if a transformed sample record x(t) is periodic with a period T p

(fundamental

frequency f 1

=1/T p

), then x(t) can be represented by the Fourier series:

where


xt = a 0

2 ∑ a q

cos 2q f 1

t b q

sin 2q f 1

t

T

q=1

a q

= 2 ∫ xtcos 2q f

T 1

t dt q=0,1,2,...

0

T

b q

= 2 T ∫ 0

xtsin 2q f 1

t dt

q=1,2,3,...


A. Lagg – Spectral Analysis

Fourier series procedure ­ method

sample record of finite length, equally spaced sampled:

x n

=xnh

n=1,2,... , N

Fourier series passing through these N data values:

xt = A 0

N /2


q=1

A q

cos 2q t

T p


N /2−1


q=1

B q

sin 2qt

T p

Fill in particular points: t=nh , n=1,2,... , N , T p

=Nh , x n

=xnh = ...

coefficients A q

and B q

: A 0

= 1 ∑ N n=1

N

x n

= x A N /2

= 1 N ∑ n=1


N

x n

cos n

N

A q

= 2 N ∑ n=1

N

B q

= 2 N ∑ n=1

x n

cos 2q n

N

x n

sin 2q n

N

inefficient & slow => Fast Fourier Trafos developed

q=1,2,... , N 2 −1

q=1,2,... , N 2 −1


A. Lagg – Spectral Analysis

Fourier Transforms ­ Properties

Linearity

DFT

{x n

} ⇔

DFT

{y n

} ⇔

a{x n

}b{y n

}

DFT


{X k

}

{Y k

}

a{X k

}b{Y k

}

Symmetry {X k

} = {X −k

R{X k

} is even

∗ }

I{X k

} is odd

Circular time shift

DFT

{x n−n0

} ⇔

{e i k n DFT

0

y n

} ⇔

{e −i k n 0

X k

}

{Y k−k0

}


A. Lagg – Spectral Analysis

Using FFT for Convolution

r * s ≡ ∫

−∞


r st−d

Convolution Theorem:

r * s ⇔ FT R f S f

Fourier transform of the convolution

is product of the individual Fourier

transforms

discrete case:

N /2

r * s j

≡ ∑

k=−N /21

Convolution Theorem:

N /2


k=−N /21

s j−k

r k

s j−k

r k

⇔ FT R n

S n

(note how the response function for negative times is wrapped

around and stored at the extreme right end of the array)

constraints:

duration of r and s are not the same

signal is not periodic

original data

response function

convolved data


A. Lagg – Spectral Analysis

Treatment of end effects by zero padding

constraint 1:

constraint 2:

simply expand response function to length N by padding it with zeros

extend data at one end with a number of zeros equal to the max.

positive / negative duration of r (whichever is larger)


A. Lagg – Spectral Analysis

FFT for Convolution

1. zero­pad data

2. zero­pad response function

(­> data and response function have N elements)

3. calculate FFT of data and response function

4. multiply FFT of data with FFT of response function

5. calculate inverse FFT for this product

Deconvolution

­> undo smearing caused by a response function

use steps (1­3), and then:

4. divide FFT of convolved data with FFT of response

function

5. calculate inverse FFT for this product


A. Lagg – Spectral Analysis

Correlation / Autocorrelation with FFT

definition of correlation / autocorrelation see first lecture

Corr g , h = g* h = ∫

−∞


Correlation Theorem:

Corr g , h ⇔ FT

Auto­Correlation:

G f H * f

Corr g , g ⇔ FT ∣G f ∣ 2

discrete correlation theorem:

Corr g , h j


N −1


k=0

⇔ FT G k

H k

*

g jk

h k

gthd


A. Lagg – Spectral Analysis

Fourier Transform ­ problems

spectral leakage

T =nT p

T ≠nT p


A. Lagg – Spectral Analysis

reducing leakage by windowing (1)

Applying windowing (apodizing) function to data record:

xt = xtwt (original data record x windowing function)

x n

= x n

w n

No Window:

Bartlett Window:

Hamming Window:

wt=1−

∣t−T /2∣

T /2

wt=0.540.46∗cost

Blackman Window:

Hann Window:

Gaussian Window:

wt =

0.420.5∗cost

0.08 cos2 t

wt=cos 2 t

2 wt=exp−0.5a t 2


A. Lagg – Spectral Analysis

reducing leakage by windowing (2)

without windowing

with Gaussian windowing


A. Lagg – Spectral Analysis

No constant sampling frequency

Fourier transformation requires constant sampling (data points at

equal distances)

­> not the case for most physical data

Solution: Interpolation

linear:

inear interpolation between y and y k k+1

IDL> idata=interpol(data,t,t_reg)

quadratic:

quadratic interpolation using y k­1 , y k

and y k+1

IDL> idata=interpol(data,t,t_reg,/quadratic)

least­square quadratic

least­square quadratic fit using y k­1 , y k

, y k+1 and y k+2

IDL> idata=interpol(data,t,t_reg,/lsq)

spline

IDL> idata=interpol(data,t,t_reg,/spline)

IDL> idata=spline(t,data,t_reg[,tension])

important:

interpolation changes

sampling rate!

­> careful choice of

new (regular) time grid

necessary!


Fourier Transorm on irregular gridded data ­ Interpolation

original data: sine wave + noise

FT of original data

irregular sampling of data (measurement)

interpolation: linear, lsq, spline, quadratic

're­sampling'

FT of interpolated data

A. Lagg – Spectral Analysis


A. Lagg – Spectral Analysis

Noise removal

Frequency threshold (lowpass)

make FT of data

set high frequencies to 0

transorm back to time

domain


A. Lagg – Spectral Analysis

Noise removal

signal threshold for

weak frequencies (dB­threshold)

make FT of data

set frequencies with amplitudes

below a given threshold to 0

transorm back to time domain


A. Lagg – Spectral Analysis

Optimal Filtering with FFT

normal situation with measured data:

underlying, uncorrupted signal u(t)

+ response function of measurement r(t)

= smeared signal s(t)

+ noise n(t)

= smeared, noisy signal c(t)

st = ∫

−∞


r t−ud

ct = stnt

estimate true signal u(t) with:

U f = C f f

R f

f ,t = optimal filter

(Wiener filter)


A. Lagg – Spectral Analysis

Calculation of optimal filter

reconstructed signal and uncorrupted signal should be close in least­square sense:

­> minimize

­> minimize

∫−∞


∣ ut−ut∣ 2 dt = ∫

−∞


∣ U f −U f ∣ 2 df



∣ ∂ f

⇒ f =

[S f N f ] f

R f

∣S f ∣ 2

∣S f ∣ 2 ∣N f ∣ 2

− S f

R f ∣2

= 0

additional information:

power spectral density can often

be used to disentangle noise

function N(f) from smeared

signal S(f)


A. Lagg – Spectral Analysis

Using FFT for Power Spectrum Estimation

discrete Fourier transform of c(t)

­> Fourier coefficients:

N−1

C k

=∑

j=0

c j

e 2 i j k / N

k=0,... , N−1

­> periodogram estimate of

power spectrum:

P0 = P f 0

= 1 N 2 ∣C 0 ∣ 2

Periodogram

P f k

=

1

N 2 [∣C k ∣ 2 ∣C N−k ∣ 2 ]

P f c

= P f N /2

= 1 N 2 ∣C N /2 ∣ 2


end of FT

A. Lagg – Spectral Analysis


A. Lagg – Spectral Analysis

Spectral Analysis and Time Series

Andreas Lagg

Part I: fundamentals

on time series

classification

prob. density func.

auto­correlation

power spectral density

cross­correlation

applications

pre­processing

sampling

trend removal

Part II: Fourier series

definition

method

properties

convolution

correlations

leakage / windowing

irregular grid

noise removal

Part III: Wavelets

why wavelet

transforms?

fundamentals:

FT, STFT and

resolution problems

multiresolution

analysis: CWT

DWT

Exercises


A. Lagg – Spectral Analysis

Introduction to Wavelets

why wavelet transforms?

fundamentals: FT, short term FT and resolution

problems

multiresolution analysis:

continous wavelet transform

multiresolution analysis:

discrete wavelet transform


A. Lagg – Spectral Analysis

Fourier: lost time information

6 Hz, 4 Hz, 2 Hz, 1 Hz 6 Hz + 4 Hz + 2 Hz + 1 Hz


Solution: Short Time Fourier Transform

(STFT)

perform FT on 'windowed' function:

example: rectangular window

move window in small steps over data

perform FT for every time step

STFT f ,t ' = ∫[ xtt−t ']e −i 2 f t dt

t

A. Lagg – Spectral Analysis


A. Lagg – Spectral Analysis

Short Time Fourier Transform

STFT

STFT­spectrogram shows both time and

frequency information!


A. Lagg – Spectral Analysis

Short Time Fourier Transform: Problem

narrow window function ­> good time resolution

wide window function ­> good frequency resolution

Gauss­functions as

windows


A. Lagg – Spectral Analysis

Solution: Wavelet Transformation

time vs. frequency resolution is intrinsic problem (Heisenberg Uncertainty Principle)

approach: analyze the signal at different frequencies with different resolutions

­> multiresolution analysis (MRA)

Continuous Wavelet Transform

similar to STFT:

signal is multiplied with a

function (the wavelet)

transform is calculated

separately for different segments

of the time domain

but:

the FT of the windowed signals are

not taken

(no negative frequencies)

The width of the window is changed

as the transform is computed for

every single spectral component


A. Lagg – Spectral Analysis

Continuous Wavelet Transform

CWT x = x , s = 1

∣s∣ ∫ t

... translation parameter , s ... scale parameter

t ... mother wavelet = small wave

*

t−


xt

s

dt

mother wavelet:

finite length (compactly

supported) ­>'let'

oscillatory ­>'wave'

functions for different

regions are derived from

this function ­> 'mother'

scale parameter s replaces frequency in STFT


A. Lagg – Spectral Analysis

The Scale

similar to scales used in maps:

high scale = non detailed global view (of the signal)

low scale = detailed view

in practical applications:

low scales (= high frequencies)

appear usually as short bursts or

spikes

high scales (= low frequencies) last

for entire signal

scaling dilates (stretches out) or

compresses a signal:

s > 1 ­> dilation

s < 1 ­> compression


A. Lagg – Spectral Analysis

Computation of the CWT

signal to be analyzed: x(t), mother wavelet: Morlet or Mexican Hat

start with scale s=1 (lowest scale,

highest frequency)

­> most compressed wavelet

shift wavelet in time from t 0

to t 1

increase s by small value

shift dilated wavelet from t 0

to t 1

repeat steps for all scales


A. Lagg – Spectral Analysis

CWT ­ Example

signal x(t)

axes of CWT: translation and

scale (not time and frequency)

translation ­> time

scale ­> 1/frequency


A. Lagg – Spectral Analysis

Time and Frequency Resolution

every box corresponds to a value of the wavelet

transform in the time frequency plane

all boxes have constant area

Δf Δt t = const.

low frequencies: high resolution

in f, low time resolution

high frequencies: good time

resolution

STFT: time and frequency

resolution is constant (all boxes

are the same)


A. Lagg – Spectral Analysis

Wavelets: Mathematical Approach

WL­transform:

CWT x = x , s = 1

∣s∣ ∫ t

*

t−


xt

s

dt

Mexican Hat wavelet:

t =

−t 2

1

s e

2 2

3

t2

2 −1

Morlet wavelet:

t = e i a t e − t2

2

inverse WL­transform:

xt =

1 2 ∫∫ x

, s 1

c s s t−

2 s

d ds

admissibility

condition:

c

={2∫

−∞∞

∣∣ 2

∣∣

1/2

d } ∞ with

FT


t


A. Lagg – Spectral Analysis

Discretization of CWT: Wavelet Series

­> sampling the time – frequency (or scale) plane

advantage:

sampling high for high frequencies (low scales)

scale s and 1 rate N 1

sampling rate can be decreased for low

frequencies (high scales)

scale s 2

and rate N 2

N 2

= s 1

s 2

N 1

N 2

= f 2

f 1

N 1

continuous wavelet

discrete wavelet

, s

= 1 s t−

s t = s − j/2 −

j , k 0

s j 0

t−k 0

, j , k

x

j , k =∫

t

xt = c ∑

j

*

xt j , k

tdt


k

x

j , k j , k

t

orthonormal

WL­transformation

reconstruction of

signal


A. Lagg – Spectral Analysis

Discrete Wavelet Transform

(DWT)

discretized continuous wavelet transform is only a sampled version of the CWT

The discrete wavelet transform (DWT) has significant advantages for

implementation in computers.

excellent tutorial:

http://users.rowan.edu/~polikar/WAVELETS/WTtutorial.html

IDL­Wavelet Tools:

IDL> wv_applet

Wavelet expert at MPS:

Rajat Thomas


end of Wavelets

A. Lagg – Spectral Analysis


A. Lagg – Spectral Analysis

Exercises

Part I: Fourier Analysis

(Andreas Lagg)

Instructions:

http://www.linmpi.mpg.de/~lagg

Part II: Wavelets

(Rajat Thomas)

Seminar room

Time: 15:00


A. Lagg – Spectral Analysis

Exercise: Galileo magnetic field

data set from Galileo magnetometer

(synthesized)

file: gll_data.sav, contains:

total magnetic field

radial distance

time in seconds

your tasks:

Which ions are present?

Is the time resolution of

the magnetometer

sufficient to detect

electrons or protons?

Background:

Tips:

restore,'gll_data.sav'

use IDL­FFT

remember basic plasma

physics formula for the

ion cyclotron wave:

gyro

= q B

m ,

f gyro = gyro

2

If the density of ions is high enough they will excite ion cyclotron waves

during gyration around the magnetic field lines. This gyration frequency

only depends on mass per charge and on the magnitude of the magnetic

field.

In a low­beta plasma the magnetic field dominates over plasma effects.

The magnetic field shows only very little influence from the plasma and

can be considered as a magnetic dipole.

http://www.sciencemag.org/cgi/content/full/274/5286/396


A. Lagg – Spectral Analysis

Literature

Random Data: Analysis and Measurement Procedures

Bendat and Piersol, Wiley Interscience, 1971

The Analysis of Time Series: An Introduction

Chris Chatfield, Chapman and Hall / CRC, 2004

Time Series Analysis and Its Applications

Shumway and Stoffer, Springer, 2000

Numerical Recipies in C

Cambridge University Press, 1988­1992

http://www.nr.com/

The Wavelet Tutorial

Robi Polikar, 2001

http://users.rowan.edu/~polikar/WAVELETS/WTtutorial.html

More magazines by this user
Similar magazines