WAVELETS: Theory and Applications An Introduction Willy Hereman ...

.WAVELETS: Theory and ApplicationsAn IntroductionWilly HeremanDept. of Mathematical and Computer SciencesColorado School of MinesGolden, ColoradoUSAPh.D. Program, Department of PhysicsUniversity of AntwerpAntwerp, BelgiumDecember 4-15, 20001

.Introduction to WAVELETS: Theory and ApplicationsLECTURE NOTESHOMEWORK ASSIGNMENT & EXAMBOOKS ON WAVELETSProf. Willy HeremanDept. of Mathematical and Computer SciencesColorado School of MinesGolden, ColoradoUSAHonors Course–Numerical AlgorithmsDepartment of Applied MathematicsUniversity of StellenboschStellenbosch, South AfricaMarch 9-23, 2001Wed.: 10-10:50, Fri.: 10-10:50, 11-11:502

Topics—Outline• Introduction: History, motivation, applications• Key ideas and definitions• Haar and Daubechies wavelets• Scaling and wavelet functions• Multi-resolution analysis• Wavelet analysis: decomposition and reconstruction• Fast Fourier Transform (FFT) versus Fast Wavelet Transform (FWT)• Vanishing moments, smoothness, approximation• Low and high pass filters• Quadrature Mirror Filters (QMF)• Construction of Daubechies’ wavelets• Construction of scaling and wavelet functions• Selected applications• Software demonstration3

• INTRODUCTION (André Weideman)• BACKGROUND AND EXPERIENCE WITH WAVELETS* 1993- Work with Gregory Beylkin at CU - Boulder.* 1993 - ’94 Sabbatical year devoted to wavelets and applications.* 1993 - Short course in Ghent, Belgium (my alma mater).* 1994- Work on coiflets (with Monzon and Beylkin),- work on Dubuc-Deslauriers’ subdivision scheme and wavelets,- work on Battle-Lemarié spline based wavelets.* Course on wavelets at CSM-Golden, CO (1995).* Short course on wavelets in Antwerp. Fall 2000 (Also, May 1995).* 1998 Paper on coiflets.* Currently wavelet issues related to applications (facial recognition, fingerprints,etc.).* Belgian connection with Ingrid Daubechies and Wim Sweldens.• PLAN AT STELLENBOSCH* Gentle introduction to wavelets (concepts).* American style: ask questions.* Mediao Lecture notes in library.o Copies of slides in library.o Reference books in library.o Review papers in office.o Wavelet Explorer (Mathematica).• WHO HAS EXPERIENCE WITH WAVELETS?* o Theory.o Applications? Which?* Who has heard about wavelets and where, what context?* Relation to field of study.4

* Background in mathematics.* Reasons to take course?• GRADE FOR WAVELET SHORT COURSE* One assignment 4 or 5 problems.* One examination question (open book).• INFO ABOUT WAVELETSo long list of books (copy in library).o thousands of papers.o two dozen good review papers.o Wavelet Digest (Wavelets for Kids) http://www.wavelet.org/wavelet.o Electronic resources.o Wavelet Software.o Info on WEB (see course Weideman).To get to Wavelet Explorer.- Start Mathematica.- Click on Help.- Click on Add-ons.- Click on Wavelet Explorer.DEMO OF WAVELET EXPLORERTo get intro to Wavelet ExplorerFrom wavelet Explorer Pick Fundamentals of WaveletsIn [1]In [2]In [3]In [4]To use it in your own notebook in MathematicaNeeds [“Wavelets‘ Wavelets‘”].? *aub*.DaubechiesFilter[n] (brief description)??DaubechiesFilter[n] (for full description with all options)DaubechiesFilter[3].For filtertaps of Daubechies averaging filter with 3 vanishing moments.5

GENERAL IDEAS ABOUT WAVELETS* Arena (setting) Mathematics behind data, signal, image analysis or processing(Engr.)and numerical analysis (Mathematics/Applied Mathematics).* Wavelets is at interface of engineering and mathematics (compare with mathematicalphysics, mathematical biology, mathematical finance, ....)* Counterpart to Fourier analysisFFT ←→ FWT (or discrete DWF)Fast Fourier transform Fast wavelet transformGreat Discovery of Jean-Baptiste Fourier (1768-1830).A periodic signal (sound, function) can be decomposed in harmonics (sinesor cosines, or complex exponentials).* Shortcomings of Fourier analysis (weaknesses)- sines and cosines wiggle (extend) infinitely far,- does not handle gaps or localized signals well, or faulty data (too manycoefficients needed to make cancellations to zero),- a transient sharp “blip” needs many Fourier components. Huge numberof non-zero coefficients,- does not handle fast variations (highly oscillatory signal) well,- trouble at discontinuities (overshoot, undershoot, Gibbs phenomena).* Strong points of Fourier analysis- functions are analytic and simple,- simple (sines and cosines) orthogonal functions,- fast FFT is n log n algorithm - due to trick invented in 1965 by J. Tuckeyand J. Cooley (same trick applies for wavelets),- few coefficients in series are needed to represent a smooth periodic function,- very applicable recursive algorithms,- caused a revolution in scientific computing,- continuous and discrete transforms are available,6

→ Try to keep what’s good and try to improve on bad properties =⇒Wavelets.* Key requirements* Cost:o a few functions (no longer sines/cosines) should give good approximation,o orthogonal functions,o we want localization in both time (via translations) and (space) frequency(wavelength) via dilations and contractions,o we want fast, recursive algorithm,o zoom in - zoom out property,o something highly applicable,o mathematically on solid foundation (requires new mathematical concepts,ideas, ....construction),o continuous and discrete transforms.=⇒ no longer analytical functions,=⇒ not as simple as Fourier series or Fourier transform.WHAT IS A WAVELET?“a little wiggle, ripple”French: ondeletteHow many types of wavelets are there? - About 2 dozen practical types.(Daubechies’ wavelets, Meyer wavelets, biorthogonal wavelets, Battle-Lemariéwavelets, ...)HOW DOES ONE GET TO WAVELET BASIS?In Fourier analysis:Take sinusoids (or complex exponentials) as basis functions and then study theproperties of the Fourier series (or Fourier integral)In wavelet analysis:One poses the desired properties and then derives the resulting basis functions.7

[For instance, multiresolution is a key property, orthogonality also, and perhapssymmetry, vanishing moments property].Key idea:You want to construct a basis for IL 2 (IR) (= collection/space of all square integrablefunctions)f ∈ IL 2 (IR) if∫ +∞−∞ f(x)2 dx < ∞CONCEPTUAL ANALOGIES (OTHER PERSPECTIVES)1. The game of averages and differences and vanishing momentsStudent #1 has X 1 amount of money in bank accountStudent #2 has X 2 amount of money in bank accountYou can either give X 1 , X 2 itself or give theaveragedifferenceAdvantage ifS 1 = X 1 + X 22D 1 = X 1 − X 22⇐⇒ X 1 = S 1 + D 1⇐⇒ X 2 = S 2 − D 1X 1 = X 2 =⇒ D 1 = 0.If X 1 = X 2 follow the same constant trendthen simple “differences” creates a zero student number“he same amount in account” −→ difference = zero, −→ average = how mucheach has.Question: Is it possible to generalise this simple idea? YES.Look at it from a “signal data” processing point of view8

X 1 X 2} {{ } −→ X 3 X 4} {{ }move over in steps of two.X 5 . . .for averagesfor differences(1•2)(1•2)(1= h 0 •2)= h 1(1= g 0 • = g 12)ChoicesS 1 = 1 2 X 1 + 1 2 X 2 = h 0 X 1 + h 1 X 2D 1 = 1 2 X 1 − 1 2 X 2 = g 0 X 1 + g 1 X 2(12 , 1 2)(12 , −1 2)= h 0 , h 1= (g 0 , g 1 )or (1, 1) = (h 0 , h 1 )orLeaping ahead(1, −1) = (g 0 , g 1 )( )1 1 √2 , √ = (h 0 , h 1 )2(1√2 , −√ 1)2= (g 0 , g 1 ) Haar caseSimilarly S 2 = h 0 X 3 + h 1 X 4D 2 = g 0 X 3 + g 1 X 4 , etc.If X 1 , X 2 , X 3 , X 4 , . . . lie on line can we find h 0 , h 1 and g 0 , g 1 so that D 1 , D 2etc. are zero? No.We need four numbersh 0 , h 1 , h 2 , h 3 for averaging (low pass filter)g 0 , g 1 , g 2 , g 3 for differencing so that (high pass filter)D 1 = g 0 X 1 + g 1 X 2 + g 2 X 3 + g 4 X 4 = 0ifX 1 , X 2 , X 3 , X 4 , . . . are on lineAlso,9D 2 = 0 etc.

If X i are on parabola we will needWe will need six numbers.Etc. can be done for x n also.(h 0 , . . . , h 5 ) for averaging(g 0 , . . . , g 5 ) for differencing, sothat g 0 X 1 + . . . + g 5 X 6 = D 1 = 0if X i follow quadratic trenda + bX + cX 2↓ ↓ ↓.constant linear and quadratic combinedIf X i follow on a + bX + cX 2 + . . . + fX nthen you need 2n “magical” numbers with special properties so thatExamples of data following trendsD 1 = 0 ,D 2 = 0 , etc.* Pictures: cone and ball on tiled floor. Show and explain image compression.* Cartoon: how are the Disney cartoons made? Game for kids: picturebook with slightly different (shifted figures) on frozen background.* Video: TV broadcasting. Transmit only the differences.−→ Life is full of redundancy!2. Vanishing momentsIf you could construct a wavelet function ψ(x) which corresponds to “differencing”then you could require that ψ(x) annihilates constant, linear,quadratic, cubic, ..... trends . . . up to degree say x M−1so∫x l ψ(x) dx = 0 for l = 0, 1, 2, . . . M − 1} {{ }M moments10

Since a function f(x) can be written in McLaurin seriesf(x) =you would havef(0)} {{ }constant+ f ′ (0)x +} {{ }linearquadraticf ′′ (0)2 x2 + . . . + f (M−1)} {{ }(M − 1)! xM−1 + . . .degree M − 1∫f(x)ψ(x)dx = 0 + 0 + 0 . . . + 0 +∫x MM! ψ(x)dx + . . .} {{ }highly oscillating part−→ Wavelets would be sensitive to fast oscillating pieces of f(x) and annihilateslower varying pieces of f(x).3. Fingerprint storyPopularized wavelets(FBI competition for fingerprint compression)(Old systems for identification was based on facial features. Bertillion 1884).- 37 million fingerprint cards for criminals (some in duplicate),- 250 million fingerprint cards in total (job applications, immigration,background checks for employment ..., weapon purchases, ... ),- 60 km of “thin” cardboard,- each card has 10 rolled imprints of fingers, 2 unrolled thumb prints, twofull hands,- cataloguing is several million cards behind,- request for identity check may take between 100 - 140 days,- enormous office building full of these cards.−→ Need for automation and electronic storage (via scanning), saves space.Allows for on-line verification (in shortest possible time, while suspect isbeing held).Should be accessible from every police car (digitizing tablet, modem, ...).- FBI has approximately 350 employees to handle the verification requests,cataloguing, removing duplicates, etc. ...11

- Automatic system: IAFIS (Automatic Fingerprint Identification System).* cards are digitized and stored on 2 computers in a nuclear attackproof building (cellar 20 m deep in Clarksburg, West Virginia),* old card collection is moved to Federicksburg,* scanning all cards took 5 years,* cost: $640 million,* every police car has a scanner and they can fully automatically checkthe enormous database.- FBI get 45,000 requests per day for verification.- FBI adds 5,000 fingerprints to database per day.- Average checking time (for routine stuff, like for application for weaponspermit) takes 30 seconds.- Computer database is 40 terabytes.1 byte = 8 bids (zero or one)40 × 10 12 bytes (= 40,000 PC with harddrive of 10 gigabytes). Evencompressed!How does it get that large?Each square inch fingerprint image is broken into 500 × 500 grid of smallboxes, called pixels.Each pixel is given a gray-scale from 0 to 255↓white↓black−→ darkness of one pixel requires 8 bits = 1 byte.Start multiplying: 1 whole card ∼ = 10 MB.(Mega byte = 10 6 bytesGiga byte = 10 9 bytesTera byte = 10 12 bytes)=⇒ 2500 Tera byte = 2500 10 12 12

250 million cards ∼ = 250010 6 10 6}{{}MB∼ = 2500 × 10 2 ×For ≈ 37 million criminals alone10GB} {{ }harddrive∼ = 250, 000PC’s. (quarter million PC’s.)=⇒ 37,000 PC’s. 1,000 cards fill up one PC!Impossible without serious data compression for storage and transmission(via JPEG (Joint Photographic Experts Group) or something else.−→ FBI competition won by Ron Coifman (Yale) and Bradley and Brislawn,at Los Alamos National Laboratory (biorthogonal wavelets).DATA COMPRESSION:Sending 1 card without compression via 56K modem would take ≈ 20 minutesStandard compression ratios 3/1 or 2/1Wavelet compression ratios 15/1 or 20/1−→ with 20/1 compressiontransmission time = 1 minute.Space savings also factor 15 or 20−→ 1/20 of space(5% kept)(For cirminals’ card storage −→ less than 2,000 PC’s)Wavelet algorithms closer to home:* wavelet chip in modems (Aware Corp.)* JPEG 2000 standard for internet pictures (you see the details come in).13

APPLICATIONS OF WAVELETSSignal and Data processing:- analysis (eg. detection of edges, detection of faults, abrupt changes,defects like “cracks” in materials, . . . ),- compression (reduction of storage space, fast transmission, as in fingerprintapplication),- smoothing (attenuation of noise, denoising),- synthesis (reconstruction after compression or/and smoothening, orother types of modification).Types of signals1D : sound, speech, time-series (e.g. history of a stock in financial markets),2D : pictures, maps, . . . ,3D : spatial diffusion (e.g. of heat of body, dopamine injected in brain, . . . ).SAMPLE APPLICATIONS−→ Reconstruction of Johannes Brahms’ early recording (1889) of First HungarianDance (story in Scientific American 1993).Music speech−→ Medical applications: analysis of– MRI scan,– electro cardiogram,– ultrasonic images,– mammograms (tumor detection),– various other modern scanning, visualization methods in medicine.Medical imaging−→ Military applications:formation of images based on partial radar data (distinguish ambulancefrom tank).Military imaging−→ High speed data transmission:- single pictures (come gradually in focus),14

- video (explain principle).Digital communication−→ Denoising technology:(project for ADA Technologies: detection of metals like vanadium, mercury,lead in smoke, laser spectroscopy −→ least squares method andwavelets).Data analysis−→ Mathematical, engineering applications:fast PDE solvers, Ax = b solvers, numerical analysisFFT vs FWT−→ Geophysical applicationsseismic methods for detection of rock formations, pockets of gas, detectionof oil fields, . . . .CSMMassive data−→ Finance:analysis of time-series (prediction of behaviour of stocks, other financialintruments . . . ) .Money matters−→ Metereology:analysis of weather data, climate, temperature in oceans,. . . .Massive dataSHORT HISTORY“Wavelets come from a synthesis of ideas”.• First wavelet: Haar 1910 (box functions).• Engineering experiments in 1960’s −→ 1980’s.• Experiments with windowed Fourier Transforms.• 1980: Jean Morlet (geophysicist)- wavelets for analysis of seismic data.• 1980 Alex Grossmann: theoretical study of wavelets and wavelet transform.• 1985: Yves Meyer (harmonic analyst)15

- connections with other fields in maths,- connection with singular integral operators.• 1986 - 1988: Ingrid Daubechies (VUB colleague)- frames (generalized basis for Hilbert space),- 1988 - major breakthrough,- “Daubechies” family of orthonormal wavelets with compact support,- connection with coherent states.- Inspired by work of Mallat and Meyer on multi-resolution analysisand applications in decomposition and reconstruction of images (computergraphics).- “Ten lectures on Wavelets”.• Beyond 1988: exponential growth in theoretical developments.(5,000 papers in Science Citation Index).Applications in physics (coherent states), quantum field theory, electricalengineering, computer science, statistics, etc., etc., . . . .“SYNTHESIS OF IDEAS”16

SCALING AND WAVELET FUNCTIONS* Wavelet is little ripple, a little wave.∫ψ(x), not symmetric, such thatψ(x)dx = 0, and such that(admissibility condition)ψ(x) will have special properties.* Start with “mother” wavelet, concentrated, say, in interval [0,1].* Generate other wavelet by moving the mother wavelet left or right in unitsteps, and by dilating or compressing the mother wavelet by repeated factorsof 2.Compare with images (zoom into a picture)zoom outdilate (stretch) −→ lower resolution details of picture,zoom incompress (squeeze) −→ higher resolution details of picture.Look at TV-screen:large screen (blurred image) expanded imagesmall screen (sharper image) compressed image.Definition (Mathematical)Let IL 2 IR be the Hilbert space of square integrable functions on real line.A wavelet is a funtion ψ ∈ IL 2 (IR) such thatis an orthonormal basis of IL 2 (IR)2 j/2 ψ(2 j x − k) j, k ∈ ZThere are two families in the game:ϕ(x) scaling function −→ ϕ j,k (x)↓father where ϕ j,k (x) = 2 j/2 ϕ(2 j x − k) = 2 j/2 ϕ(2 j (x − k/2 j ))↑ ↑2 parameter family kids (boys)ψ(x) wavelet function −→ ψ j,k (x) −→ kids(girls)↓mother where ψ j,k (x) = 2 j/2 ψ(2 j x − k) = 2 j/2 ψ(2 j (x − k/2 j ))17

↑2 parameter familyHaar waveletDaubechies wavelet(DAUB4)Picture motherPicture fatherPictures of scaling and wavelet functions2 vanishing momentsKey features (desired properties).* The basis should be orthonormal “vectors” (here functions) should be mutuallyorthogonal, and properly normalized−→ fast algorithms to compute coefficients.* The basis functions (wavelets) should have vanishing moments−→ sparsity (lots of zeros) everywhere.* The algorithms should be recursive. True due to the multi-resolution propertiesof the basis (always devide or multiply by factor 2).* There should be no need for the functions themselvesso, the family {2 j/2 ψ(2 j x − k)} j,k is not needed.Instead one uses “filter coefficients or taps” for averaging, differencing (youwant to maintain a constant norm, independent of j).* NormalizationIf ψ j,k (x) = cψ(2 j x − k)Require ‖ψ j,k (x)‖ 2 = ‖ψ 0,0 (x)‖ 2 = ‖ψ‖ 2∫ψj,k (x) 2 dx = c 2 ∫ ψ 2 (2 j x − k)dx, set z = 2 j x − k= c 2 2 −j ∫ ψ 2 (z)dz = c 2 2 −j ‖ψ‖ 2 = 1.‖ψ‖ 2so we need c = 2 j/2 .Note: Daubechies waveletsIf ψ is a Daubechies wavelet with M vanishing moments that generates an orthonormalbasis of IL 2 (IR) thenψ is supported on [0, 2M − 1] or [−M + 1, M] if translated to leftover M − 118

Vanishing momentsϕ is supported on [0, 2M − 1].2 vanishing moments means4 vanishing momentsM vanishing moments∫ψ(x) dx = 0,∫x ψ(x) dx = 0∫x l ψ(x) dx = 0 for l = 0, 1, 2, 3∫x l ψ(x) dx = 0 l = 0, 1, 2, . . . , M − 1.Wavelets with vanishing moments will annihilate (ignore, or be blind to) certaintrends; linear (including const), quadratic, cubic, quartic, quintic, etc. and onlybe sensitive ( pick out) higher degree oscillations.Expanding and contracting wavelets.Compare with music.ψ quarter note at middle C (where your start)1V 2 ψ(x 2) half note a lower octave (stretch)2ψ(4x) 16 th note at 2 octaves higher (contract)THE REFINEMENT OR TWO-SCALE DIFFERENCE EQUATIONHaar caseϕ(x) =ϕ(x) = characteristic function on (0,1)⎧⎨⎩1 , 0 ≤ x < 10 , elsewhere (x < 0 or x ≥ 1).Observe ∫ 1∫ ϕ(x)dx = 1 = 10 0 ϕ2 (x)dx.Also, for scaling functionϕ(x) = ϕ(2x) + ϕ(2x − 1).Refinement equation for Haar case.Other names: dilation equation; two-scale difference equation.19

Verify0 ≤ x < 1/2 1 = 1 + 01/2 ≤ x ≤ 1 1 = 0 + 1x < 0 or x ≥ 1 0 = 0 + 0.Also, for wavelet functionψ(x) = ϕ(2x) − ϕ(2x − 1).Note: wavelet function can be defined in terms of scaling function.IN GENERAL (DAUBECHIES WAVELET AND OTHERS)Equivalentlyϕ(x) = √ 2 L−1 ∑k=0ψ(x) = √ 2 L−1 ∑k=0h k ϕ(2x − k)g k ϕ(2x − k)For Haar caseϕ(x/2) = √ 2 L−1 ∑k=0L = 2h k ϕ(x − k), similar for ψ(x/2)(length of filter)ϕ(x) = ϕ(2x) + ϕ(2x − 1)= √ 2(1 √2 ϕ(2x) + 1 √2ϕ(2x − 1)ψ(x) = ϕ(2x) − ϕ(2x − 1)= √ 1(√2 12ϕ(2x) − √ 12ϕ(2x − 1)−→ (h 0 , h 1 ) =−→ (g 0 , g 1 ) =(√2 1, 1)√2= H(√2 1, −√ 1)2= G))Filters for averaging and differencing.20

In generalAlways come in pairsH = (h 0 , h 1 , h 2 , . . . , h L−1 ) low pass filterG = (g 0 , g 1 , g 2 , . . . , g L−1 ) high pass filter.Note: As vectors −→ H ⊥ −→ Gi.e. ϕ⊥ψ as functionsH −→ G reverse the order and switch every other signH = (h 0 , h 1 , . . . , h L−1 ) −→ G = (h L−1 , −h L−2 , . . . , −h 0 ).Reverse (of refinement equation)Later, general:ϕ(2x) = 1 (ϕ(x) + ψ(x))2ϕ(2x − 1) = 1 (ϕ(x) − ψ(x))2ϕ(2x − l) = ∑ k(a l−2k ϕ(x − k) + b l−k ψ(x − k))Remarks1. In general there is no (explicit) analytical form for ϕ(x) or ψ(x)2. ϕ(x) is evaluated via the refinement equation at dyadic points x = k 2 jj, k ∈ Z−→ (Assignment plus old notes. Eigenvalue problem).Example:Set x = 1/2 ϕ(1/2) =Also0{ }} {1{ }} {ϕ(1) + ϕ(0) = 1ϕ(−1/2) = ϕ(−1) + ϕ(−2)= 0 + 0 = 0So, if ϕ is known at integers then ϕ is known at the halvesRecursive!ϕ(1/4) = ϕ(1/2) + ϕ(−1/2) = 121

General ϕ(2 j−1 n) = ϕ(2 j n) + ϕ(2 j n − 1)3. To be able to use {ϕ(x − k)} k ∈ Z to approximate even simple functionwe assumethat ϕ(x) and its translates form a partition of unity.∑k∈ Zϕ(x − k) = 1,∀x ∈ IRProof in Fourier domainˆϕ(0) = 1 =⇒ ∑ ϕ(x − k) = 1.kMore general:f(t) = ∑ nf n (t) where f n (t) = ϕ(t − n)f(t) −→ localized function.For Haar case:ϕ(x) = ϕ(2x) + ϕ(2x − 1)4. Refinement equations were known in context of splines.Examples* Piecewise linear spline (hat function)SatisfiesN 2 =* Piecewise quadratic splineSatisfies⎧⎪⎨⎪⎩x , 0 ≤ x < 12 − x , 1 ≤ x < 20 , elsewhereN 2 (x) = 1 2 N 2(2x) + N 2 (2x − 1) + 1 2 N 2(2x − 2)N 3 (x) =⎧⎪⎨⎪⎩1/2x 2 , 0 ≤ x ≤ 13/4 − (x − 3/2) 2 , 1 ≤ x < 21/2(x − 3) 2 , 2 ≤ x ≤ 30 , elsewhereN 3 (x) = 1 4 N 3(2x) + 3 4 N 3(2x − 1) + 3 4 N 3(2x − 2) + 1 4 N 3(2x − 3)22

5. In generalϕ(x) = √ 2 L−1 ∑k=0h k ϕ(2x − k)ψ(x) = √ 2 ∑ g k ϕ(2x − k)Proof - Integrate both sides1 =∫ϕ(x)dx = √ 2 L−1 ∑=√22L−1 ∑∫ϕ(x)dx=1=⇒ L−1 ∑k=0∫ψ(x)dx=0=⇒ L−1 ∑∫h kk=0∫h kk=0= √ 1 L−1 ∑2−→ L−1 ∑k=0Verify for Haar case √ 12+ √ 12= √ 22= √ 2∫ ∑ψ(x)dx = 0 =⇒ k g k = 0∑Similar proof fork g k = 0Verify for Haar case1 √2 − 1 √2= 0k=0h k = √ 2g k = 0z{ }} {ϕ (2x − k) dxϕ(z)dz} {{ }1h kk=0h k = √ 2linearlinear6. Orthogonality propertiesϕ(x) ⊥ ϕ(x − k), ∀k ∈ Z 0 (k ≠ 0)Inner product in IL 2 (IR)< f, g >=∫ +∞−∞f(t) g(t) dtSo < ϕ, ϕ n >=< ϕ, ϕ(x − n) >= δ noOne can show thatϕ(x)⊥ϕ(x − n), if n ≠ 0andϕ 2 (x) normalized (n = 0)∫ϕ 2 (x)dx = 1leads to (No proof here - proof in extended notes)23

(i)(ii)L−1 ∑k=0h 2 k = 1 ←− quadratic (n = 0) case aboveL−1−2m ∑l=0Verify for Haar caseh 2m+l h l = 0 m = 1, 2, . . . , L 2− 1 (n ≠ 0 )aboveShow in detail via shifts of twoh 0 h 1 h 2 h 3 . . .h 0 h 1 . . .(i) ( 1 √2) 2 + ( 1 √2) 2 = 1 2 + 1 2 = 1L = 2 m = 0 - no condition of type (ii)ϕ ⊥ ψ (h 0 , h 1 ) ⊥ (g 0 , g 1 ) = (h 1 , −h 0 )so, h 0 h 1 − h 1 h 0 = 0.ForL = 8⎧⎪⎨⎪⎩h 0 h 2 + h 1 h 3 + h 2 h 4 + h 3 h 5 + h 4 h 6 + h 5 h 7 = 0h 0 h 4 + h 1 h 5 + h 2 h 6 + h 3 h 7 = 0h 0 h 6 + h 1 h 7 = 07. Admissibility ConditionIn general,L−1 ∑k=0∫h 2k = L−1 ∑k=0∫ψ(x)dx = 0ψ(x) = h 0 ϕ(2x) − h 1 ϕ(2x − 1)1∫{ }} {ϕ(z)dz − h 02ψ(x)dx = 0 = h 12=⇒ h 0 = h 1h 2k+1 = 1 √2,1{ }} {∫ϕ(z)dxand also L−1 ∑(−1) k h k = 0 (H ⊥ G)k=024

REPRESENTING FUNCTIONS IN A WAVELET BASISMulti-resolutionFor Haar case (1910){ψ j,k (x)} k,j∈ Z is a fully orthonormal basis for IL 2 (IR).So for f(x) ∈ IL 2 (IR)f(x) =In practice, Haar also used+∞ ∑j=−∞= +∞ ∑j=−∞+∞ ∑k=−∞+∞ ∑k=−∞a jk ψ j,k (x)a jk 2 j/2 ψ(2 j x − k)↑ waveletsf(x) = +∞ ∑c k ϕ(x − k) + +∞ ∑k=−∞k=−∞+∞ ∑j=0d jk ψ j,k (x)Effect of jx ←→ 2 j xt ←→ 2 j tEffect of k↓↓translates at scale adding weightedj = 0wavelets at differentscales, starting from j = 0and allowing translates also.space ←→ scaletime ←→ frequency (via dilation)localization in space (or time)(via translation)25

A few examplesRESOLUTION - MULTI-RESOLUTION1. Our number system has multi-resolution.Compare with computation of circumference circle2 radius= ππ = 3.1415926535j’aime a faire connaitre le nombre utile aux sages.3 = 3. 110 0 −→ what scaling function shows at a given resolution3.1 = 3. 110 0 + 1. 1 10} {{ }−→ better resolutionwavelet contribution - encodes the detail3.14 = 3. 110 0 + 1. 1 10} {{ }+ 4. 1 + . . .} {{10 2}Other direction: stretching (adding more and more detail), the scalingfunction too much we end up seeing nothing at all. Represent 3.14159 in10 = 10 1 or 100 = 10 2 scales.2. Picture of Dali’s painting.“Gala naked watching the sea”.(Salvador Dali, 1976, Dali Museum, Figueres, Spain).Zoom in: Look at detail (close-up) −→ Gala (Dali’s wife).Zoom out: Look at coarse picture (from afar) −→ Abraham Lincoln.Dali was aware of images at different scales of resolution.26

Basic notions (First for Haar case)Inspiration for multi-resolution properties.1. Consider the scaling function ϕ and the associated 2 parameter scaling familyϕ j,k (x) = 2 j/2 ϕ(2 j x − k)j, k ∈ ZFor fixed j : {ϕ j,k (x)} k∈ Zspans a subspace V j of IL 2 (IR).More precise: We define the subspace −→ closure.V j= span k∈ Z{ϕj,k (x) } j fixedand ϕ j,k (x) is a basis for V j , the space of piecewise constant functions (withdyastic points k 2 j ).Example:If f(x) ∈ V 0 then f(x) = ∑a kk∈ ZV 0 = space of piecewise constant functions over integers.V 1 = space of ... over halves = larger space than V 0 ,because the functions ϕ 1,k (x) = √ 2ϕ(2x − k)= √ 2ϕ(2(x − k/2))are narrower and translated in smaller steps.can represent finer detailSo V 0 ⊂ V 1In general . . . ⊂ V −2 ⊂ V −1 ⊂ V 0 ⊂ V 1 ⊂ V 2 ⊂ . . .V j ⊂ V j+1 −→ Question: what “lies” between them?ϕ(x−k){ }} {ϕ 0,k (x)2. Consider the wavelet function ψ and the associated 2 parameter familyψ j,k (x) = 2 j/2 ψ(2 j x − k) j, k ∈ Z.For fixed j : {ψ j,k (x)} k∈ Z spans W j27

{ψ j,k } k∈ Z is basis for W j , again with piecewise functions ondyadic grid {k/2 j }is a fully orthonormal basis.3. {ψ j,k } (j,k∈ Zbasis for IL 2 IR. This basis is complete.It is an infinite basis in terms of wavelet functions.4. Practical alternativeTake the setwhich also forms a basis for IL 2 (IR).{ϕ j0 ,k(x) = j 0 /2ϕ(2 j 0 x − k),ψ j,k = 2 j/2 ψ(2 j x − k) } j≥0, k∈ Z.In practice, one quite often takes j 0 = 0and therefore takes{ϕ(x), ϕ(x ± 1), ϕ(x ± 2), . . . , . . .√ψ(x), ψ(x ± 1),√ψ(x ± 2), . . .2ψ(2x ± 1), 2ψ(2x ± 2), . . ..2 j/2 ψ(2 j x−k)} as the basisNote: The set {ϕ j0 ,k(x)} k∈ Z spans the same subspace as {ψ j,k (x)} j≤j0 , k∈ Z.5. Visualization of Haar’s multi-resolutionV 0coarseV 1 finer V 0 ⊂ V 1 ⊂ V 2V 2V j.still finerby having j sufficiently large the subspace V j is quite large.Using the wavelets ψ j,k (x) takes care of the differences.The precise relationship between V j , W j and IL 2 (IR) is given via the MultiresolutionAnalysis (MRA).28

6. Beyond the Haar caseThe same construction is possible, the same concepts and ideas hold for manytypes of wavelet families (Daubechies wavelets in particular).Advantage: We will be able to use more regular (smoother) functions ϕ andψ at the cost of a more elaborate construction and loss of analytic or closedforms, longer filters, etc.Multi-resolution analysis (Meyer and Mallat - 1986)Multi-resolution analysis of IL 2 (IR) is an increasing sequence of closed (nested)subspacesSuch that(i) ⋂k∈ Z(ii) ⋃k∈ Z{V j } j∈ Z of IL 2 (IR),. . . ⊂ V −2 ⊂ V −1 ⊂ V 0 ⊂ V 1 ⊂ V 2 ⊂ . . . ⊂ V j ⊂ V j+1 ⊂ . . .V j = {0} orlim V j = {0} (no overlap, separate spaces)j→−∞V j = IL 2 (IR) or lim V j = IL 2 (IR)j→∞(iii) For any f ∈ IL 2 (IR), for any j ∈ Z⎛⎝ ⋃ jV j is dense in IL 2 (IR)f(x) ∈ V j iff f(2x) ∈ V j+1(Also f(x) ∈ V0 ⇐⇒ f(2 j x) ∈ V j)(iv) For any f ∈ IL 2 (IR), for any k ∈ Zf(x) ∈ V 0 iff f(x − k) ∈ V 0⎞⎠(v) ∃ϕ ∈ V 0such that{ϕ(x − k)} k∈ Z is an orthonormal basis of V 0.Notes:(1) ϕ is the scaling function.(2) ψ will come as a by-product.29

(3) Riesz basis: A countable set {f n } of a Hilbert space IL 2 (IR) is a Riesz basisif ∀f ∈ IL 2 (IR) it can be uniquely written asf = ∑n∈ INc n f n and ∃A, B > 0 such thatA‖f‖ 2 ≤ ∑ n|c n | 2 ≤ B‖f‖ 2Now define W j as the orthogonal complement of V j in V j+1 ,i.e. V j ⊕ W j = V j+1 , j ∈ Z.Then IL 2 (IR) is represented as the direct sumIn practiceIL 2 (IR) = ⊕W jj∈ Z1. There is a coarsest scale “n” andV −n ⊂ . . . ⊂ V −1 ⊂ V 0 ⊂ V 1 ⊂ . . .−→ IL 2 (IR) = V −n ⊕ W j,j≥−n low resolution2. There is a finite number of scales, select j = 0 to be the coarsest scaleV 0 ⊂ V 1 ⊂ . . . ⊂ V n , V 0 ⊂ IL 2 (IR)↓ ↓coarsest finest3. In numerical implementations the subspace V 0 is finite dimensional.Be aware of other choices in the literature where ϕ j,k (x) = 2 −j/2 ϕ(2 −j x − k)ThenV j = V j+1 ⊕ W j+1j −→ −jand⊃ V −2 ⊃ V −1 ⊃ V 0 ⊃ V 1 ⊃ . . .30

EXPLANATION OF MULTI-RESOLUTION ANALYSISHaar basis on [0,1]V 0 is spanned by ϕ(x)W 0 is spanned by ψ(x)V 0 ⊥W 0 and V 1 = V 0 ⊕ W 0V 1 = space of all piecewise constant functions on half intervals.Basis for V 1 consists of √ 2ϕ(2x) and √ 2ϕ(2x − 1) −→ √ 2ϕ(2x − k), k = 0, 1.√√ 22ϕ(2x) = (ϕ(x) + ψ(x))2√√ 22ϕ(2x − 1) = (ϕ(x) − ψ(x))2↑ ↑∈ V 0 ∈ W 0Since V 0 ⊂ V 1 −→ ϕ(x) can be written in terms of√2ϕ(2x) and√2ϕ(2x − 1)IndeedSineϕ(x) = 1 √2(√2ϕ(2x) +√2ϕ(2x − 1))W 0 ⊂ V 1 −→ ψ(x) = ϕ(2x) − ϕ(2x − 1)= √ 1 (√ √ )2ϕ(2x) − 2ϕ(2x − 1)2Basis for W 1 consists of √ 2ψ(2x) and √ 2ψ(2x − 1) −→ √ 2ψ(2x − k), k = 0, 1.V 2 is spanned by 2ϕ(2 2 x − k) = 2ϕ(4x − k), k = 0, 1, 2, 3−→ space of piecewise functions on quarter interval.W 2 is spanned by 2ψ(4x − k), k = 0, 1, 2, 3V 2 = V 1 ⊕ W 1 = V 0 ⊕ W 0 ⊕ W 131

So, a function f ∈ V 2 can be written in terms ofϕ(x), ψ(x), 2ψ(2x) and √ 2ψ(2x − 1)But V 2 is also spanned by2ϕ(4x), 2ϕ(4x − 1), 2ϕ(4x − 2) and 2ϕ(4x − 3).In general,V j is spanned by 2 j/2 ϕ(2 j x − k) for fixed j and k = 0, 1, . . . , 2 j − 1W j is spanned by 2 j/2 ψ(2 j x − k), and k = 0, 1, . . . , 2 j − 1V j+1= V j ⊕ W j= V j−1 ⊕ W j−1 ⊕ W j= · · ·= · · ·V j+1= V 0 ⊕ W 0 ⊕ W 1 ⊕ · · · ⊕ W jand the translates of wavelets on the right are also translates of scaling functionson the left.Next step, move from [0,1] to IR by allowing all translations (no restrictions onindex k).A basis for IL 2 IR consists of ϕ(x − k) k∈ Z together withAnother basis for IL 2 IR consists ofOrthogonality Issuesψ j,k = 2 j/2 ψ(2 j x − k) with j ≥ 0, k ∈ Zψ j,k (x) = 2 j/2 ψ(2 j x − k) ∀j ∈ Z , ∀k ∈ Z.1. For fixed j V j ⊥W j∫ϕj,k ψ j,l dx = 0 ∀k, l2. Since W j ⊂ V j+1 and W j+1 ⊥V j+1∫ψj,k (x)ψ j+1,l dx = 0=⇒ W j ⊥W j+1Then W j+1 ⊂ V j+2 and W j+2 ⊥V j+2 =⇒ W j+1 ⊥W j+2All W j are mutually orthogonal.32

3. Since V j ⊂ V j+1 and W j+1 ⊥V j+1∫ψp,k ψ q,l dx = 0 if p ≠ q=⇒ V j ⊥W j+1∫ϕj,k ψ j+1,l dx = 0.4. It is not true that V j ⊥V j+1 , but ϕ⊥ its translates.Representing a function in a wavelet basisThere are 5 different approaches (points of view).Each one leads to a key interpretation, to key properties and to an algorithmto perform wavelet decomposition.Approaches:1. inner product method,2. matrix method (change of basis),3. convolution method,4. FWT method,5. projection matrices method.Example to illustrate the methodsTake f(x) = 9ϕ(4x) + 1.ϕ(4x − 1) + 2ϕ(4x − 2) + 0.ϕ(4x − 3).Rewrite in basis 2 j/2 ϕ(2 j x − k) on V 2here j = 2 , k = 0, 1, 2, 3f(x) = 9 22ϕ(4x − 0)} {{ } +1 22ϕ(4x − 1)} {{ }= 9 2 ϕ 2,0(x) + 1 2 ϕ 2,1(x) + 1ϕ 2,2 (x) + 0ϕ 2,3 (x)+1 2ϕ(4x − 2) +0 ϕ(4x − 3)} {{ } } {{ }We want to writebasis vectors for V 2f(x) = aϕ(x) + bψ(x) + c √ 2ψ(2x) + d √ 2ψ(2x − 1)↓ ↓ ↓ ↘ ↙∈ V 2 ∈ V 0 ∈ W 0 ∈ W 133

There are 5 different methods to compute a, b, c and d.METHOD #1:Inner product method (not efficient)a = < f(x), ϕ(x) >= 9 2 < ϕ 2,0, ϕ > + 1 2 < ϕ 2,1′ϕ > +1 < ϕ 2,2′ ϕ > +0= 9 2 < 2ϕ(4x), ϕ(x) > +1 2< 2ϕ(4x − 1), ϕ(x) > + < 2ϕ(4x − 2), ϕ(x) >= 9 2 2. 1 4 + 1 2 .2. 1 4 + 2.1 4 + 0= 9 4 + 1 4 + 2 4 = 3.Similarlyb = < f(x), ψ(x) >= · · · = 2c = < f(x), √ 2ψ(2x) >= · · · = 4/ √ 2d = < f(x), √ 2ψ(2x − 1) >= · · · = 1/ √ 2−→ (x) = 3.ϕ(x) + 2ψ(x) + 4 √2. √ 2ψ(2x) + 1 √2. √ 2ψ(x − 1).So,f(x) = 3ϕ(x) + 2ψ(x) + 4 √2√2ψ2x +1 √2√2ψ(2x − 1)f (2) (x) = f (0) + g (0) + g (1)↓ ↓ ↓ ↓∈ V 2 ∈ V 0 ∈ W 0 ∈ W 11 coeff 1 coeff 2 coeffsIn general g (j) is made up with functions on mesh 1 2 j .Full decompositionf (n) (x) = f (0) + g (0) + g (1) + g (2) + · · · + g (n−1)# coefficients 1 + 1 + 2 + 4 + 8 + · · · + 2 n−1} {{ }2 n coefficientsAfter J steps of decompositionf (n) (x) =f (n−J) (x) + g (n−J) (x) + · · · + g (n−1) (x)# coefficients 2 n−J + · · · 2 n−2 + 2 n−1 = 2 n coefficients34

METHOD #2:Matrix MethodOld basis B = {2ϕ(4x), 2ϕ(4x − 1), 2ϕ(4x − 2), 2ϕ(4x − 3)}New basis B ′ = {ϕ(x), ψ(x), √ 2ψ(2x), √ 2ψ(2x − 1)}⎡⎢⎣9/21/210Column #1 :because⎤⎥⎦=⎡⎢⎣↑old coordinates1/2 1/2 1/ √ 2 01/2 1/2 −1/ √ 2 01/2 −1/2 0 1/ √ 21/2 −1/2 0 −1/ √ 2⎤ ⎡⎥ ⎢⎦ ⎣abcd⎤⎥⎦⇐⇒ → f = R → b↑ R = reconstruction matrixIf matrix with as columns the coordinatesof the new basis vectors in terms of theold basis.⎡⎢⎣1/21/21/21/2ϕ(x) = 1 2 2ϕ(4x) + 1 2 2ϕ(4x − 1) + 1 2 2ϕ(4x − 2) + 1 2ϕ(4x − 3)2Similarly, for columns 2, 3 and 4ψ(x) = 1 2 2ϕ(4x) + 1 2 2ϕ(4x − 1) − 1 2 2ϕ(4x − 2) − 1 2ϕ(4x − 3)2√ 12ψ(2x) = √2 2ϕ(4x) − √ 1 2ϕ(4x − 1) + 0.2ϕ(4x − 2) + 0.2ϕ(4x − 3)2√2ψ(2x − 1) = 0.2ϕ(4x) − 0.2ϕ(4x − 1) +√22ϕ(4x − 2) −1 √2 2ϕ(4x − 3)⎤⎥⎦→b = D → fD = decomposition matrixWe need→b = R −1 → f = R ⊤ → fbecause the columns of R are orthonormal vectors.35

−→⎡⎢⎣abcd⎤⎥⎦=⎡⎢⎣1/2 1/2 1/2 1/21/2 1/2 −1/2 −1/21/ √ 2 −1/ √ 2 0 00 0 1/ √ 2 −1/ √ 2⎤ ⎡⎥ ⎢⎦ ⎣9/21/210⎤⎥⎦=⎡⎢⎣324/ √ 21/ √ 2⎤⎥⎦.So, f (2) (x) = f(x) = 3ϕ(x) + 2ψ(x) + 4 √2√2ψ(2x) +1 √2√2ψ(2x − 1)= 3ϕ(x) + 2ψ(x) + 4ψ(2x) + 1ψ(2x − 1)PictureMETHOD #3:❀ 9ψ(4x) + ϕ(4x − 1) + 2ϕ(4x − 2) + 0ϕ(4x − 3)Convolution MethodFor Haar case H = ( 1 √2,1 √2 ), G = ( 1 √2, − 1 √2)→s(n)−→ H → s (n−1) −→ H → s (n−2) · · · −→ → s (0)G ↓ G ↓ G ↓ ↓ Gzero.→d(n−1) →d(n−2) →d(n−3) →d(0)} {{ }set asidestop at level⎡⎢⎣→s (0). . . . .→d(0). . . . .→d(1). . . . .→d(2). . . . ... . . . .→d(n−1)⎤⎥⎦.→s(2)= 9/2 1/2 1 0↓ GH −→→s (1) = 5/ √ 2 1/ √ 2↓ G→d(1)= 4/√2 1/√2 d (0) = 2H−→→s (0) = 3⎡ ⎤→(0)s . . .→(0)d ⎢ ⎥⎣ ⎦. . .→d(1)36

Assemble the result:⎡⎢⎣9/21/210⎤⎥⎦D=⇒⎡⎢⎣3. . . . .2. . . . .4/ √ 2√2⎤⎥⎦In general⎡→n ⎢s =⎢⎣.....⎤⎫⎪⎬⎥⎦⎪⎭2 n data −→⎡⎢⎣→s(0). . . .→d(0). . . .→d(1). . . .→d(2). . . .... . . .→dn−1⎤⎥⎦112⎫⎪ ⎬24n data8.2 n−1 ⎪ ⎭Convolution9/2 1/2 1 0 −→ 5/ √ 2 1/ √ 2↘ ↙ ↘ ↙ ↘ ↙Average: 5/ √ 2 1/ √ 2 Average: 3Difference: 4/ √ 2 1/ √ 2 Difference: 237

METHOD #4: Fast Wavelet TransformDecompose the matrix D into a product of 3 sparser matrices.Decompose at level 1: V 2 = V 1 ⊕ W 1h 0 h 1 −→g 0 g 1 −→⎡⎢⎣1 √21√2 0 0√20 00 0 √2 1 √2 1√12− 10 01 √2 − 1⎤ ⎡⎥ ⎢⎦ ⎣√29/21/210⎤⎥⎦=⎡⎢⎣5/ √ 24/ √ 21/ √ 21/ √ 2⎤⎥⎦− low frequency− high frequency− low frequency− high frequencybutterfly (structured) matrixorM 1→f =→b1Bring high frequencies to bottom via a permutation⎡⎢⎣1 0 0 00 0 1 00 1 0 00 0 0 1⎤ ⎡⎥ ⎢⎦ ⎣5/ √ 24/ √ 21/ √ 21/ √ 2⎤⎥⎦=⎡⎢⎣5/ √ 21/ √ 24/ √ 21/ √ 2⎤⎥⎦− low frequencies− low frequencies− high frequencies− high frequenciesvery sparseor M 2→b1 = → b 2Decompose at level 2: V 1 = V 0 ⊕ W 0⎡⎢⎣1 √21√2 0 0√20 00 0 1 00 0 0 1√12− 1⎤ ⎡⎥ ⎢⎦ ⎣5/ √ 21/ √ 24/ √ 21/ √ 2⎤⎥⎦=⎡⎢⎣324/ √ 21/ √ 2⎤⎥⎦SoorM 3→b2 = → b→ →M 1 f = b1→⎫⎪ →→ ↙ → ⎬ M 3 M 2 M 1 f = bM 2 b1 = b2→ ↙ →M 3 b2 =⎪= D f → = →b⎭bD = M 3 M 2 M 138

Count number of operationsForD → f = → b −→∼ N 2 operations (not counting additions)D is a dense N × N matrix12 multiplicationsForM 1→f=→b1M 2→b1 = → b 2M 3→b2In general= → b8 multiplicationsno cost, only shuffling4 multiplications⎫⎪⎬⎪⎭∼ 12 multiplicationsN log 2 N algorithm instead of N 2 algorithm.Compare with FFT Tukey, Cooley (1965)(See slides)* Same idea: write dense matrix as product of sparser “butterfly” and permutationmatrices.* Computation count N log 2 N versus N 2If N = 1024 N 2 = 1, 048576N log 2 N = 10240.• Show M 1 for Daubechies case (DAUB4)METHOD #5Projection matrices MethodSee e.g. Daubechies’ Ten Lectures on Wavelets.• Comparison with FFT.QUADRATURE MIRROR FILTERS(A look in the Fourier domain)The Fourier transform.There are 3 equivalent choices, depending on where one puts the “2π”.39

Each choice exists of two versions, depending on whether the minus is put inthe exponent of the direct or the inverse transform.So, we have1. (Ff)(ω) =ˆf(ω)def= 1 √2π∫ +∞−∞ f(x)e−iωx dx→ ω is in radians per second (1 radian = 12π circle)∫ +∞2. ˆf(ω)def=−∞ e−2πiωt dt→ ω in Hertz∫ +∞3. ˆf(ω) =−∞ f(x)e−iωx dx→ in radians.For these notes:∫ +∞Direct Transformation: ˆf(ω) = (Ff)(ω) =−∞ e−2πiωt f(t) dtInverse Transformation: f(t) = (F −1 ∫ +∞ˆf)(t) =−∞ e2πiωt f(ω) dω1. Apply the Fourier Transformation to the refinement equation(in real space)ϕ(x) = √ 2 L−1 ∑ˆϕ(ω) =h k ϕ(2x − k)k=0∫ +∞−∞ e−2πiωx ϕ(x)dx= √ 2 L−1 ∑k=0h k∫ +∞−∞ e−2πiωx ϕ(2x − k)dxSet u = 2x − k −→ x = u + k2−→ dx = du 2=√2 L−1 ∑∫ +∞u+kh k e−2πiω[ 2 ] ϕ(u)du2 k=0−∞= √ 1 L−1 ∑∫ +∞h k2 −∞ e−2πi ω 2 k e −2πi ω 2 u ϕ(u)duk=040

= √ 1 L−1 ∑h k e −2πi ω 2 k ∫ +∞2 k=0−∞ e−2πi( ω 2 )u ϕ(u)du= √ 1 L−1( )∑ωh k e −2πi ω 2 k ˆϕ2 2k−0DefineThenm 0 (ω) = √ 1 L−1 ∑h k e −2πikω2k=0ˆϕ(ω) = m 0(ω2)( )ωˆϕ2or equivalently,ˆϕ(2ω) = m 0 (ω) ˆϕ(ω)refinement equation in Fourier domain!Notes:m 0 (ω) is a trigonometric polynomialm 0 (ω + 1) = m 0 (ω)If we know m 0 (ω) =⇒ we know h k .2. Keep refiningSinceˆϕ(ω) = m 0(ω2)( )ωˆϕ2then ω → ω/2 ˆϕ ( ) ( ) ( )ω2 = ω m04 ˆϕ ω4then ω → ω/2 ˆϕ ( ) ( ) ( )ω4 = ω m08 ˆϕ ω8We getˆϕ(ω) = m 0(ω2) ( ) ( ) ( )ω ω ωm 0 m 0 · · · m 04 8 32(ωˆϕ32).Keep going ω −→ ω/2ˆϕ(ω) = N ∏j=1m 0(ω2 j )( )ωˆϕ2 N41

Take limit forButSo,N −→ ∞∏ˆϕ(ω) = ∞ ( )ωm 0 ˆϕ(0)2 jˆϕ(0) =∫j=1e −2πi0t ϕ(t)dt =∏ˆϕ(ω) = ∞ ( )ωm 02 jj=1∫ϕ(t)dt = 1Infinite refinement of the two-scale difference relation in the Fourier domain.Similarly,withorˆψ(ω) = m 1 (ω/2) ˆϕ(ω/2)ˆψ(2ω) = m1 (ω) ˆϕ(ω)m 1 (ω) def = √ 1 L−1 ∑2Also, by continuing the refinement processˆψ(ω) = m 1 (ω/2) ˆϕ(ω/2)k=0= m 1 (ω/2)m 0 (ω/4) ˆϕ(ω/4)g k e −2πikω= m 1 (ω/2)m 0 (ω/4)m 0 (ω/8) ˆϕ(ω/8)=⇒ ˆψ(ω) = m 1(ω2) ∞∏j=2Since G ⊥ H so g k = (−1) k h L−1−kone has m 1 (ω) = e −2πi(1−L)ω m 0(ω +12.m 0(ω2 j ))42

3. Change of notationSet z = e −2πiωThen m 0 (ω) −→ H(z) = √ 2m 0 (ω(z))AndH(z) = L−1 ∑k=0h k z km 1 (ω) → G(z) = √ 2m 1 (ω(z))G(z) = L−1 ∑k=0g k z kAlso, m 1 (ω) is related to m 0 (ω + 1/2) =⇒ G(z) = −z L−1 H(− 1 z )DERIVATION OF THE QMF CONDITION(Quadrature mirror filters)Tools neededParseval’s Identity∫ +∞∫< f, g >= f(x)g(x)dx = +∞ˆf(ω)g(ω)dω =< ˆf, ĝ >−∞ −∞Consequence (Plancherel’s formula)‖ f ‖ 2 ∫∫= ffdx = |f| 2 dx∫∫= ˆf(ω)f(ω)dω = | ˆf(ω)| 2 dω =‖ ˆf ‖ 2(F is an isometric isomorphism)First, we prove that the following statements are equivalent(i)∫ +∞< ϕ(x − k), ϕ(x − l) >= ϕ(x − k)ϕ(x − l)dx = δ kl−∞i.e. ϕ ⊥ ϕ translates.43

(ii)(iii)Proof:ˆϕ(ω) satisfies+∞ ∑k=−∞∫ +∞−∞ e−2πikω | ˆϕ(ω)| 2 dω = δ k0| ˆϕ(ω + k)| 2 = 1Define the function K(ω) = +∞ ∑| ˆϕ(ω + k)| 2k=−∞Then(1) K(ω) is 1 periodic(2)K(ω + 1) = ∑∫ 1= ∑ kk∈ Z| ˆϕ(ω + k + 1) | 2 = ∑∫ K(ω)dω = 10 0∫ k+1k= ∑ k∫ 1Set∑kl∈ Z| ˆϕ(ω + k) | 2 dω0 | ˆϕ(ω + k) |2 dωz = ω + k| ˆϕ(z) | 2 dz =∫ +∞∫ +∞−∞| ˆϕ(ω + l) | 2 = K(ω)| ˆϕ(z) |2 dzPlancherel= | ϕ(z) −∞ |2 dz = 1.So, K(ω) is 1 periodic and integrable (integrates to 1)So, K(ω) has a Fourier seriesWhere c n ==K(ω) =∫ 1+∞ ∑n=−∞c n e 2πinω0 e−2πinω K(ω)dω∫ 1+∞ ∑0 e−2πinωk=−∞| ˆϕ(ω + k) | 2 dω44

Setω + k = zConsequences= +∞ ∑∫ k+1k=−∞k= +∞ ∑∫ k+1==k=−∞ ∫ +∞ke −2πin(z−k) |ˆ ϕ(z) | 2 dze −2πinz | ˆϕ(z) | 2 dz−∞ e−2πinz ˆϕ(z) ˆϕ(z)dzˆϕ(z − n) ˆϕ(z)dz∫ +∞−∞Parseval=∫ +∞−∞ ϕ(x − n)ϕ(x)dx = δ noSo, c n = δ n0Hence, K(ω) = +∞ ∑=⇒ +∞ ∑k=−∞n=−∞δ no e 2πinω = 1| ˆϕ(ω + k) 2 |= 1.(1) If ˆϕ(0) = 1Then from+∞ ∑k=−∞| ˆϕ(ω + k) | 2 =|=⇒ ˆϕ(k) = δ k0(2)One can show that+∞ ∑k=−∞ϕ(x − k) = 1ˆϕ(k) = δ k0 ⇐⇒ +∞ ∑Proof: Similar to the above by settingk=−∞Partition of unity| ˆϕ(ω + k) | 2 = 1.P (x) = ∑ kϕ(x − k)showing that P (x + 1) = P (x) and P (x) integrableCompute c n =∫ 10 e−2πinx ∑ kϕ(x − k)dx= · · · = δ n0 , etc.45

(3)ˆϕ(k) = δ k0 ⇐⇒ ∑k∈ Zϕ(x − k) = 1Compare with Poisson summation formula∀xNow we can show that+∞ ∑n=−∞ˆf(n) = +∞ ∑k=−∞f(k)or, equivalently,| m 0 (ω) | 2 + | m 0 (ω + 1/2) | 2 = 1 QMF conditionH(z)H(1/z) + H(−z)H(−1/z) = 2Proofuse ˆϕ(2ω) = m 0 (ω) ˆϕ(ω) (1)DenoteK(ω) = ∑ | ˆϕ(ω + k) | 2 = 1kK(2ω) = ∑ | ˆϕ(2ω + k) | 2 = 1k(1)= ∑ k| ˆϕ(ω + k 2 ) |2 | m 0 (ω + k 2 ) |2Split in even and odd terms= ∑+l (k=2l)∑l (k=2l+1)m 0 (ω) is 1-periodic| ˆϕ(ω + l) | 2 | m 0 (ω + l) | 2| ˆϕ(ω + l + 1 2 ) |2 | m 0 (ω + l + 1 2 ) |2= ∑ | m 0 (ω) | 2 | ˆϕ(ω + l) | 2 + ∑ | m 0 (ω + 1ll 2 ) |2 | ˆϕ(ω + l + 1 2 ) |2= | m 0 (ω) | 2 ∑ (| ˆϕ(ω + l) | 2 + | m 0 ω + 1 )| 2 ∑ (| ˆϕ ω + l + 1l2 l2} {{ }1= | m 0 (ω) | 2 + | m 0 (ω + 1 2 ) |2 = 1 QMF} {{ }1)| 246

Rewrite in terms of H(z)H(z)H(1/z) + H(−z)H(−1/z) = 2Proofm 0 (ω) = 1 √2H(z) since z = e −2πiωm 0 (ω) = √ 1 ∑hk e −2πikω = 1 ∑√2 2= 1k√2H(1/z)m 0 (ω + 1/2) = √ 1 ∑h k e −2πik(ω+1/2)2k= √ 1 ∑h k (−1) k e −2πikω2k= √ 1 ∑2m 0 (ω + 1/2) = √ 12H(−1/z)kh k e 2πiωkh k (−z) k = 1 √2H(−z)Soorm 0 (ω)m 0 (ω) + m 0 (ω + 1/2)m 0 (ω + 1/2) = 1−→ H(z) √2H(1/z)√2+ H(−z) H(−1/z)√ √ = 12 2H(z)H(1/z) + H(−z)H(−1/z) = 2Expresses that ϕ ⊥ ϕ translates.Similarly,For reversalϕ ⊥ ψ G(z)H(1/z) + G(−z)H(−1/z) = 0ψ ⊥ ψ G(z)G(1/z) + G(−z)G(−1/z) = 2H ←→ GG(z) = −z L−1 H(−1/z)47

Vanishing moments for ψ(x)Theorem.The following statements are equivalent.(i) The wavelet function ψ has M vanishing momentsi.e.∫x m ψ(x)dx = 0 , m = 0, 1, 2, · · · , M − 1(ii) The filter coefficients h k satisfym = 0, 1, 2, · · · , M − 1L−1 ∑k=0(−1) k k m h k = 0,(iii) H(z) has z = −1 as a zero with multiplicity Mi.e. H(−1) = H ′ (−1) = · · · = H (M−1) (−1) = 0or H (m) (−1) = 0 , m = 0, 1, 2, · · · , M − 1Hence, H(z) = √ 2( 1+z2 )M Q(z)(iv) The matrix which determines the values of ϕ(x) at integers has eigenvalues1, 1 2 , (1 2 )2 , · · · , ( 1 2 )M−1(v) The Fourier transform of ψ , i.e. ˆψ(ω), has a M-fold zero at ω = 0(vi) The Fourier transform of ϕ, i.e. ˆϕ(ω), has a M-fold zero at ω = k, exceptat 0 where ˆϕ(0) = 1.Proof(some samples, to get the flavour)ˆψ(ω) =ˆψ(0) =ψ ′ (0) =So,∫∫ +∞−∞ ψ(x)e−2πiωx dx∫ +∞ψ(x)dx = 0 −→ ˆψ(0) = 0−∞ d ˆψ ∫dω | +∞ω=0 =−∞ ψ(x)(−2πix)e−2πiωx dxxψ(x)dx = 0 ←→ ˆψ ′ (0) = 0, etc.ˆψ(ω) = m 1 (ω/2) ˆϕ(ω/2) −→ ˆψ(0) = 0 =⇒ m 1 (0) = 0 −→ H(−1) = 0, etc.and also ˆϕ(0) = 1.48

Example: DAUB6. (3 vanishing moments)On the other hand−→ Q(z) is degree 2∫x m ψ(x)dx = 0 m = 0, 1, 2So, H(z) = √ ( )1 + z32 Q(z) (1)2∑H(z) = 5 h k z k = h 0 + h 1 z + · · · + h 5 z 5 (2)k=0So, Q(z) = a 0 + a 1 z + a 2 z 2H(1) = √ ∑2Q(1) = 5 h k = √ 2k=0( due to normalization)So, Q(1) = 1 =⇒ a 0 + a 1 + a 2 = 1Put H(z) = √ ( )1 + z32 Q(z) = √ ( )1 + z32 (a 0 + a 1 z + a 2 z 2 ) (3)22into QMFand take coeff of z 0 , z or 1 z , z2 or 1 z 2Solve=⇒⎧⎪⎨⎪⎩=⇒10(a 2 0 + a 2 1 + a 2 2) + 15(a 0 a 1 + a 1 a 2 ) + 6a 0 a 2 = 163(a 2 0 + a 2 1 + a 2 2) + 8(a 0 a 1 + a 1 a 2 ) + 10a 0 a 2 = 0a 0 a 1 + a 1 a 2 + 6a 0 a 2 = 0⎧⎪⎨⎪⎩a 2 0 + a 2 1 + a 2 2 = 19/4a 0 a 1 + a 1 a 2 = −9/4a 0 a 2 = 3/8Substitute in (3), compare with (2)(a 0 = 1 41 + √ 10 + 5 + 2 √ 10a 1 = 1 ( √ ) ( √ )4 2 − 2 10 =12 1 − 10a 2 = 1 (41 + √ √10 − 5 + 2 √ )10√)49

=⇒⎧⎪⎨⎪⎩h 0 = 1(16 √ 1 + √ √10 + 5 + 2 √ )102. .h 5 = 116 √ 2(1 + √ √10 − 5 + 2 √ )10DAUBECHIES WAVELETS: SOLVING THE QMFWe will solveSetthen| m 0 (ω) | 2 + | m 0 (ω + 1/2) | 2 = 1 QMF| m 0 (ω) | 2 = M 0 (ω)| m 0 (ω + 1/2) | 2 = M 0 (ω + 1/2)QMF : M 0 (ω) + M 0 (ω + 1/2) = 1For wavelets with M vanishing moments we haveM 0 (ω) =m 0 (ω) =⎡⎛So we will assume the form⎣⎛⎝ 1 + e−2πiω2⎞M⎠L(ω)→ trigonometric polynomial in e −2πiω1 + ⎝ e−2πiω2⎞ ⎛⎠where⎝ 1 + e2πiω2= cos 2M (πω)IL(ω)⎞⎤M⎠⎦IL(ω)IL(ω) = L(ω)L(ω)= |L(ω)| 2→ is trigonometric polynomial in e −2πiω→ can be written in sin 2 πω→ say P (sin 2 πω)ThenM 0 (ω) = cos 2M (πω)P (sin 2 πω)M 0 (ω + 1/2) = sin 2M (πω)P (cos 2 πω)50

Set y = sin 2 πω M 0 (ω) = (1 − y) M P (y)Then 1 − y = cos 2 πω M 0 (ω + 1/2) = y M P (1 − y)So we need to find P (y) such that(1 − y) M P (y) + y M P (1 − y) = 1Note:Doing this all for z = e −2πiω would give(z + 1) 2M(4z) M} {{ }(1−y) MStep 1: Find P (y)Most general solutionP (y){ }} {M (z − 1)2MQ(z)Q(1/z) + (−1)(4z) MP (y) = M−1 ∑k=0⎛} {{ }y M⎞Q(−z)Q(−1/z) = 1} {{ }P (1−y)M + k + 1( )⎝ 1⎠y k + y M Rk2 − ywhere R(y) is an odd polynomial chosen so that P (y) ≥ 0 for 0 ≤ y ≤ 1Let’s consider the case R = 0 ⇐⇒ minimum length IL where L = 2MArgument: P (y) polynomial in yP (1 − y) ∼ y −M(If R ≠ 0 then M > 2L , more freedom).P (y) ∼ (1 − y) −MTaylor⎛∞∑k=0⎞M + k + 1⎝ ⎠y kk−→ must be truncated at degree M − 1CandidateorP (y) = M−1 ∑k=0⎛⎞M + k + 1⎝ ⎠y kkQ(z)Q(1/z) = M−1 ∑(−1) k (1 − z) 2k −k (M + k − 1)!(4z)k!(M − 1)!k=051

Example: DAUB6 caseLHSRHSQ(z) = a 0 + a 1 z + a 2 z 2)= a0 + a 1z+ a 2z 2Q ( 1z( )1Q(z)Q = a 2 0 + a 2 1 + a 2 2 + (a 0 a 1 + a 1 a 2 )z2∑k=0⎛k (1 − z)2k(−1) 2 + k(4z) k⎝k⎞⎠ =⎛⎝ 2 0⎞⎠ −⎛= 194 − 9 (z + 1 )+ 3 4 z 8=⇒⎧⎪⎨⎪⎩⎞⎫⎬⎭(z + 1 za 2 0 + a 2 1 + a 2 2 = 19/4a 0 a 1 + a 1 a 2 = −9/4a 0 a 2 = 3/83 (1 − z)2⎝ ⎠ +4 ⎝1 4z 2(z 2 + 1 )z 2a 0 + a 1 + a 2 = 1 since Q(1) = 1Solvea⎧⎪ 0 = 1 (⎨ 41 + √ 10 +a 1 = 1 ( √ )2 1 − 10⎪ ⎩ a 2 = 1 4√5 + 2 √ )10(1 + √ √10 − 5 + 2 √ )10−→ H(z) = √ 2 ( )1+z 3 (2 a0 + a 1 z + a 2 z 2)≡ h 0 + h 1 z + · · · + h 5 z 5−→⎧⎪⎨⎪⎩h 0 = 1(16 √ 1 + √ √10 + 5 + 2 √ )102. ..h 5 = 116 √ 2.(1 + √ 10 −)+ a 0 a 2(z 2 + 1 z 2 )⎛√5 + 2 √ )10⎞⎠(1 − z)4(4z) 252

GENERAL POLYNOMIAL SOLUTION OF QMF(1 − y) M P (y) + y M P (1 − y) = 1Use Theorem of Bezout (gives existence and uniqueness of polynomial solutions).Theorem: If p 1 (x) and p 2 (x) are two polynomials of degree n 1 and n 2 with nocommon zero, then there exists two unique polynomials q 1 (x) and q 2 (x) of degree≤ n 2 − 1 and ≤ n 1 − 1 so thatApply Bezout’s theoremNow substitute 1 − y for yp 1 (x)q 1 (x) + p 2 (x)q 2 (x) = 1p 1 (y) = (1 − y) Mp 2 (y) = y MdegreeMdegreeM∃ q 1 (y) and q 2 (y) so that(1 − y) M q 1 (y) + y M q 2 (y) = 1−→ y M q 1 (1 − y) + (1 − y) M q 2 (1 − y) = 1Since q 1 (y) and q 2 (y) are unique =⇒ q 1 (y) = q 2 (1 − y)SoSo, (1 − y) M q 1 (y) + y M q 1 (1 − y) = 1and q 1 (y) is of degree ≤ M − 1P (y) = P M (y) (the polynomial we sought)Exists and is unique−→ P M (y) = M−1 ∑k=0⎛⎞M + k − 1⎝ ⎠y kkStep 2 How do we get Q(z) or m 0 (ω) now that we have M 0 (ω) =| m 0 (ω) | 2or P (y) = Q(z)Q(1/z)=⇒ Use spectral Factorization Theorem of Riesz53

Let A(ξ) be a positive trigonometric polynomial invariant under the symmetryξ −→ −ξthen A(ξ) is of the formA(ξ) = M ∑m=0a m cos(mξ) , a m ∈ IRand there then exists a trigonometric polynomial B(ξ) of degree M, i.e.such that | B(ξ) | 2 = A(ξ)B(ξ) is “root”.B(ξ) = M ∑m=0b m e imξ , b m ∈ IRIn practice: Factor A(ξ) and investigate the complex (or real) root structure.Example: DAUB 64 rootsA(ξ) = Q(z)Q(1/z) =38 − 9 4 z + 19 4 z2 − 9 4 z3 + 3 8 z4z 2⎫z ⎬ 1z ⎭ 2 inside unit circle |z 1 | = 0.3254061 Re(z 1 ) = 0.28725z −11z 1−1⎫⎬⎭2 outside unit circleQ(z) = cte ( z 2 − 2z Re(z 1 )+ | z 1 | 2)Q(1) = 1 −→ Q(z) = 1.8818688(z 2 − 0.57450z + 0.1058894)Then H(z) = √ 2( 1+z2 )3 Q(z) = h 0 + h 1 z + . . . + h 5 z 5−→ h 1 , h 2 , · · · , h 5 valuesDoing so one gets the numerical values of the DAUB6 filter taps.For the roots outside the unit circle in the Gauss plane one gets54

h 0 =1(16 √ 2(h 1 = 116 √ 2h 2 =18 √ 2h 3 =18 √ 2h 4 = 116 √ 2h 5 =116 √ 2(((1 + √ 10 +√√5 + 2 √ 10) ∼= 0.3326705529500825 · · ·5 + √ 10 + 3 5 + 2 √ 10) ∼= 0.8068915093110924 · · ·5 − √ √10 + 5 + 2 √ )10 ∼ = 0.4598775021184914 · · ·5 − √ √10 − 5 + 2 √ )10 ∼ = −0.1350110200102545 · · ·5 + √ 10 − 3(1 + √ 10 −√5 + 2 √ 10) ∼= −0.0854412738820267 · · ·√5 + 2 √ 10) ∼= 0.03522629188570 · · ·Using the roots inside the unit circle one gets the filter in reversed order.Construction of scaling and wavelet filtersEigenvalue-Eigenvector ProblemExample: DAUB6 (L = 6)ϕ(x) = √ 2 l−1 ∑k−0h k ϕ(2x − k)Assume that ϕ is bounded to an interval [a, b].Then[a, b] matching ←→[ ]a2 , b2,[a + 12, b + 1] [a + L + 1, · · · ,22, b + L − 1]2Soa ≡ a 2 −→ a = 0b ≡So ϕ(x) lives on interval [0, L − 1]Same for ϕ(x)b+L−12−→ ̸ 2b = ̸ b + L − 1b = L − 1Supp ϕ = Supp ψ = [0, L − 1]55

For L = 6 Set c k = √ 2h kϕ(0) = c 0 ϕ(0) + 0 (Since ϕ(−1) = 0, etc.)ϕ(1) = c 0 ϕ(2) + c 1 ϕ(1) + c 2 ϕ(0) + 0ϕ(2) = c 0 ϕ(4) + c 1 ϕ(3) + c 2 ϕ(2) + c 3 ϕ(1) + c 4 ϕ(0)ϕ(3) = c 0 ϕ(6) + c 1 ϕ(5) + c 2 ϕ(4) + c 3 ϕ(3) + c 4 ϕ(2) + c 5 ϕ(1)ϕ(4) = c 3 ϕ(5) + c 4 ϕ(4) + c 5 ϕ(3)ϕ(5) = c 5 ϕ(5)Since all c i ≠ 0, i = 0, · · · , 5 =⇒ ϕ(0) = ϕ(5) = 0⎧⎪⎨⎪⎩ϕ(0) = 0ϕ(1) = c 0 ϕ(2) + c 1 ϕ(1)ϕ(2) = c 0 ϕ(4) + c 1 ϕ(3) + c 2 ϕ(2) + c 3 ϕ(1)ϕ(3) = c 2 ϕ(4) + c 3 ϕ(3) + c 4 ϕ(2) + c 5 ϕ(1)ϕ(4) = c 4 ϕ(4) + c 5 ϕ(3)ϕ(5) = 0Write non-trivial equations in matrix form.⎡⎢⎣ϕ(1)ϕ(2)ϕ(3)ϕ(4)⎤⎥⎦=⎡⎢⎣Eigenvalue - Eigenvector Problem⎤c 1 c 0 0 0c 3 c 2 c 1 c 0c 5 c 4 c 3 c 2 ⎥⎦0 0 c 5 c 4→ϕ = A → ϕ⎡⎢⎣ϕ(1)ϕ(2)ϕ(3)ϕ(4)⎤⎥⎦λ → v = A → v where λ = 1→v unknown.Due to partition of unity ∑ v i = 1, so,ϕ(1) + ϕ(2) + ϕ(3) + ϕ(4) = 1(∗)Eigenvalues of matrix A are 1 4 , 1 2 , 1, 1−√ 10856

If you usec 0 = 1(161 + √ 10 +c 1 = 1(5 + √ 10 + 3c 2 = 1 8c 3 = 1 816(5 − √ 10 +(5 − √ 10 −√√5 + 2 √ 10√)5 + 2 √ 10)5 + 2 √ 10√5 + 2 √ 10Using (∗) you get for the eigenvector components))ϕ(1) = 1.28634ϕ(2) = −0.385837ϕ(3) = 0.0952675ϕ(4) = 0.00423435−→allows one to computeϕ , ψ at dyadicpoints . . .57

Reference Books on Wavelets available at Engr. Library:• Ingrid Daubechies, Ten Lectures on Wavelets, CBMS-NSF Regional ConferenceSeries in Applied Mathematics, vol. 61, SIAM, Philadelphia, 1992• Michael Frazier, An Introduction to Wavelets through Linear Algebra, SpringerVerlag, New York, 1998• Eugene Hernandez and Guido Weiss, A First Course on Wavelets, CRC Press,Boca Raton, Florida, 1996• Gerald Kaiser, A Friendly Guide to Wavelets, Birkhauser Press, Boston, 1994• Yves Meyer (transl: Robert D. Ryan), Wavelets, Algorithms and Applications,SIAM, Philadelphia, 1993• Yves Nievergelt, Wavelets Made Easy, Birkhauser, Boston, 1999Review Papers available from Instructor’s Office (220)• Adhemar Bultheel, “Learning to swim in a sea of wavelets,” Bull.Math. Soc. vol. 2 (1995) 1-44.Belg.• Pieter W. Hemker, Tom H. Koornwinder, and Nico M. Temme, “Wavelets:Mathematical preliminaries,” from: T.H. Koornwinder (ed.), Wavelets: AnElementary Treatment of Theory and Applications, World Scientific, Singapore,1993, pp. 13-26.• Peter Meuller and Brani Vidakovic, “Wavelets for Kids: tutorial paper for thestatistical community,” (with Mathematical notebook, part B), ISDS, DukeUniversity. FTP to ftp.isds.duk.edu in directory /pub/brani/wav4Kids[A-B].ps.Z.uu• Peter Nacken, “Image compression using wavelets,” from: T.H. Koornwinder(ed.), Wavelets: An Elementary Treatment of Theory and Applications,World Scientific, Singapore, 1993, pp. 81-92.• Adri B. Olde Daalhuis, “Computing with Daubechies’ wavelets,” from: T.H.Koornwinder (ed.), Wavelets: An Elementary Treatment of Theory and Applications,World Scientific, Singapore, 1993, pp. 93-105.58

• T.K. Sarkar, et al. “A tutorial on wavelets from an electrical engineeringperspective: Part 1: Discrete wavelet techniques”, IEEE Antennas and PropagationMagazine, vol. 40, no.5, 1998, 49-70.• T.K. Sarkar, et al. “A tutorial on wavelets from an electrical engineeringperspective: Part 2: “The continuous case,” IEEE Antennas and PropagationMagazine, vol. 40, no. 6, 1998, 36-49.• Nico M. Temme, “Wavelets: First steps,” from: T.H. Koornwinder (ed.),Wavelets: An Elementary Treatment of Theory and Applications, WorldScientific, Singapore, 1993, pp. 1-12.Books available at Instructor’s Office 220 Engr. Bldg.:• C. Sidney Burrus, Ramesh A. Gopinath and Haitao Guo, Introduction toWavelets and Wavelet Transforms: A Primer, Prentice Hall, Upper SaddleRiver, New Jersey, 1998• Charles K. Chui, Wavelets: A Mathematical Tool for Signal Analysis, SIAM,Philadelphia, 1997• Ingrid Daubechies, Ten Lectures on Wavelets, CBMS-NSF Regional ConferenceSeries in Applied Mathematics, vol. 61, SIAM, Philadelphia, 1992• Barbara Burke Hubbard, The World According to Wavelets: The Story of aMathematical Technique in the Making (2nd edition), A.K. Peters, Natick,Massachusetts, 1998• Charles K. Chui, Wavelets: A Mathematical Tool for Signal Analysis, SIAM,Philadelphia, 1997• Stephane Mallat, A Wavelet Tour of Signal Processing (2nd edition), AcademicPress, New York, 1999• Yves Meyer (transl: Robert D. Ryan), Wavelets, Algorithms and Applications,SIAM, Philadelphia, 1993• Yves Nievergelt, Wavelets Made Easy, Birkhauser, Boston, 1999• Gilbert Strang and Truong Nguyen, Wavelets and Filter Banks, Wellesley-Cambridge Press, Wellesley, Massachusetts, 1996• M. Victor Wickerhauser, Wavelet Packet Laboratory, A.K. Peters, Wellesley,Massachusetts, 199459

TW Honns: Numeriese Algoritmes Huiswerk # 4(Vir 3/4/01) 2001INSTRUCTIONS:(i) General instructions (a) and (b) from the previous assignments apply.(ii) The computational part of the assignment should be done with software ofyour choice.Recommended: MATLAB, Mathematica, Fortran or C.Task 1: Wavelet Decomposition & Reconstruction with Haar WaveletsConsider a function f(x), that is piecewise constant on quarter-intervals:f(x) =⎧⎪⎨⎪⎩2, 0 ≤ x < 1 40,14 ≤ x < 1 2(1)16,2 ≤ x < 3 434,4 ≤ x < 1(i) Express f(x) in terms of the scaling function φ 2,0 (x) = 2φ(4x) and its translatesφ 2,1 (x) = 2φ(4x − 1), φ 2,2 (x) = 2φ(4x − 2), φ 2,3 (x) = 2φ(4x − 3).Let ⃗ f B denote the coordinate vector of f(x) with respect to the (old) basis B,made up with the above 4 functions belonging to the scaling family φ j,k (x) =2 j 2φ(2 j x − k). What is the explicit form of ⃗ f B ?(ii) Express f(x) in terms of the wavelet (new) basis B ′ = {φ 0,0 (x) = φ(x), ψ 0,0 (x) =ψ(x), ψ 1,0 (x) = √ 2ψ(2x), ψ 1,1 (x) = √ 2ψ(2x − 1)}.Let ⃗ f B′ denote the coordinate vector of f(x) with respect to the (new) basis B ′ ,made up with the above 4 functions. What is the explicit form of ⃗ f B ′?(iii) Find the explicit form of the transformation matrix D such that ⃗ f B′ = D ⃗ f B .The matrix D is called the decomposition matrix.(iv) Find the explicit form of the transformation matrix R such that ⃗ f B = R ⃗ f B ′.The matrix R is called the reconstruction matrix.(v) Show that the matrices D and R are both orthogonal matrices.(vi) Show that D can be written as the product of three matrices, i.e. D =D 2 P 1 D 1 , where D 1 is the matrix for averaging and differencing at level 1, D 260

is the matrix for averaging and differencing at level 2, and P 1 is a permutationmatrix which brings averages together (at the top) and differences together (atthe bottom).(vii) Verify your analytic work on the computer. Define the matrices D 1 , D 2 andP 1 and compute their product to obtain D. Compute the matrix R explicitly.Verify the formulas in (iii) and (iv).(viii) Compute f ⃗ B′ via repeated convolutions with the low-pass filter H = (h 0 , h 1 ) =( √ 12, √ 12) and the high-pass filter G = (g 0 , g 1 ) = (h 1 , −h 0 ) = ( √ 12, − 1How may times can you apply each filter if you start with the data defined byf(x)?(ix) If there are 4 data entries (the four values of f(x)) then all the matrices aresize 4 × 4.Generalize the above problem to sizes 8 × 8 and 16 × 16.For instance, for the 8 × 8 case, B = {2 √ 2φ(8x − k), k = 0, 1, 2, ..., 7} andB ′ = {φ(x), ψ(x), √ 2ψ(2x), √ 2ψ(2x − 1), 2ψ(4x), 2ψ(4x − 1), 2ψ(4x − 2), 2ψ(4x −3)}.Set up the matrices D and R for each case.(x) Show that for the 8×8 case, the matrix D can be written as D = D 3 P 2 D 2 P 1 D 1 .The explicit forms of all the decomposition and permutation matrices should bein your computer program.(xi) Show that for the 16 × 16 case, the matrix D can be written as D =D 4 P 3 D 3 P 2 D 2 P 1 D 1 . The explicit forms of all the decomposition and permutationmatrices should be in your computer program.(xii) Apply the Haar decomposition and reconstruction algorithm to the followingdata sets⃗f B = 2 √ 2[1, 3, 0, 5, 7, 4, 2, 10] T (2)√2).andDetermine ⃗ f B′⃗f B = [4, 20, 0, 8, 4, 12, 12, 16, 24, 28, 4, 8, 0, 12, 36, 24] T (3)for both cases. Have the computer do the computations!Task 2: Construction of Daubechies Filter(s) (DAUB4) with 2 vanishingmomentsThis task involves the computation of the Daubechies filter(s) (DAUB4), H andG, each with 4 filter taps, corresponding to wavelets with two vanishing moments61

(M = 2).The goal is to find the real numbers (preferably 16 digits beyond the decimalpoint) h 0 , h 1 , h 2 and h 3 such that(i) G(g 0 , g 1 , g 2 , g 3 ) is a high-pass filter which annihilates both constant and lineartrends (or signals). Thus, we require two vanishing moments for ψ : i.e.∫ ∞∫ ψ(x)dx = ∞xψ(x)dx = 0. (4)−∞−∞(ii) H(h 0 , h 1 , h 2 , h 3 ) is normalized in the usual way:∫ ∞φ(x)dx = 1; (5)−∞(iii) The scaling function φ(x) is orthogonal to its translates, i.e.∫ ∞−∞ φ(x)φ(x − n)dx = δ n0, (6)where the Kronecker δ n0 equals 1 when n = 0 and zero otherwise.First, express the conditions in (i), (ii) and (iii) in terms of the filter taps of thelow-pass filter H. Second, solve that system of five(*) equations for h 0 , h 1 , h 2 andh 3 . How many solutions?(*) Remarks:(a) The quadratic condition coming from n = 0 in (iii) is actually redundant,but can and should be used to verify the results. So, one has to solve four (threelinear, one quadratic) equations.(b) The filter taps can be expressed in closed (irrational) form. Compare theclosed forms with the numerical values. Do they match?Task 3: Construction of the DAUB4 Scaling and Wavelets FunctionsThis task involves the construction of the scaling and wavelets functions, φ(x) andψ(x), for the Daubechies wavelets with two vanishing moments (DAUB4), wherethe low-pass and high-pass filters, H and G, have 4 filter taps. The computationscan be done by hand or (easier) with a symbolic manipulation program such asMathematica or Maple.One of the solutions obtained in Task 2 readsh0 = 0.4829629131445343h1 = 0.836516303737808h2 = 0.2241438680420134h3 = -0.129409522551260462

(i) Use the exact, closed form irrational representation corresponding to the abovenumerical values of the filter H = {h 0 , h 1 , h 2 , h 3 }, of length L = 4, and the twoscaledifference (refinement or dilation) equations for φ(x) and ψ(x) :φ(x) = √ 2 L−1 ∑k=0ψ(x) = √ 2 L−1 ∑k=0h k φ(2x − k) = L−1 ∑k=0g k φ(2x − k) = L−1 ∑k=0c k φ(2x − k) with c k = √ 2h k , (7)d k φ(2x − k) with d k = √ 2g k , (8)to determine the values of the scaling function φ(x) at the integers. To do this,go through the following steps:(a) Prove that the scaling function φ(x) is compactly supported on the interval[0,L-1] = [0,3].(b) Prove that φ(0) = φ(3) = 0.(c) Set up the eigenvalue-eigenvector problem A⃗v = λ⃗v to determine φ(1) andφ(2).What is the matrix A? What is the vector ⃗v? What are the eigenvalues of A? Doyou notice anything special about these eigenvalues? What are the eigenvectors ofA? Which eigenvalue and eigenvector will allow you to determine φ(1) and φ(2)?(d) Use the resolution of the identity (also called partition of unity) property ofthe scaling function,∑φ(x − k) = 1 (9)k∈ Zto normalize φ(1) and φ(2). Compute the closed forms of the irrational numbersφ(1) and φ(2)?(ii) What is the compact support of the wavelet function ψ(x)? Why?(iii) Again, use the refinement equations for φ(x) and ψ(x) to compute the valuesof these functions at the so-called diadic (dyadic) rational (x = k 2 −j ), k and jinteger). Once the values of φ(x) and ψ(x) at integers are computed, (7) and (8)determine the values at the halves, quarters, eights, etc. In general, by repeatedapplications of the refinement equations, you can obtain the values of φ(x) andψ(x) at diadic points x = k 2 −j for k = ..., −2, −1, 0, 1, 2, ...; j = 1, 2, 3, .... Fordecent resolution of the pictures, it suffices to run this iteration process till j = 4or 5. Determine explicitly the (closed-form irrational or numerical) values of φ(x), and 3.(iv) Graph the various iterations of φ(x) and ψ(x). Make sure that the horizontaland vertical axes are properly labelled.and ψ(x) at x = 0, 132 , 3 16 , 1 2 , 5 8 , 3 2 , 11 463

Task 4: Scaling and Wavelet Functions via a Cascade AlgorithmConsider the following cascade (iterative) scheme based on the refinement equationfor φ(x) :φ (i+1) (x) = √ 2 L−1 ∑h k φ (i) (2x − k), i ≥ 0, (10)k=0where the initial φ (0) (x) must be given. We use the characteristic function on theunit interval⎧1, 0 ≤ x < 1,φ (0) ⎪⎨(x) =(11)⎪⎩ 0, elsewhere.(i) For the Haar case (L = 2), take h 0 = h 1 = 1 √2, and implement the cascadealgorithm to compute the values of φ (1) (x), φ (4) (x) and φ 12) (x).(ii) Make pictures of these three scaling functions over their intervals of support.1For a decent resolution of the graphs, use steps of64. Do not go beyond theboundaries of the supporting intervals.Note: If you use Mathematica in order to efficiently compute the values of φ (i) (x),use a construction likephi[0,x_] := 0 /;phi[0,x_] := 0 /;phi[0,x_] := 1 /;x=1.0(x=0.0)h0=1/Sqrt[2];h1=1/Sqrt[2];phi[i_,x_] := phi[i,x] =N[Sqrt[2]*(h0*phi[i-1,2*x]+h1*phi[i-1,2*x-1]),8] /; i >= 1(iii) Still for the Haar case, use the relation (8) to compute the values of ψ (12) (x).(iv) Make a picture of this wavelet function.(v) What is the function corresponding to φ (∞) (x)? Does it have an analyticalform? What is it?(vi) What is the function corresponding to ψ (∞) (x)? Is there an analytical form?What is it?64

(vii) Repeat steps (i)-(iv) for the case of Daubechies wavelets (DAUB4) with twovanishing moments, whereh0 = 0.4829629131445343,h1 = 0.836516303737808,h2 = 0.2241438680420134,h3 = -0.1294095225512604.(viii) Answer the corresponding questions for the DAUB4 case: Can you still use(11) as the starting point? [Hint: Experiment with different initial conditions].What could one use instead?(ix) Assuming that (10) has a ‘fixed point’ for any choice of a low-pass filter,prove that the cascade algorithm always converges to the true scaling functioncorresponding to that filter.(x) Prove or disprove that the ‘fixed point’ φ(x) depends on the initial φ (0) (x).[Hint: for (ix) and (x) work with the equivalent of (10) in the Fourier domain].65

TW Honns: Numeriese Algoritmes Exam Question # 4(Vir Vrydae 6/4/01) 2001Question 4: Wavelets: solving the Quadrature Mirror Filter conditionBased on the Quadrature Mirror Filter (QMF) condition it is possible to computeFinite Impulse Response (FIR) low-pass and high-pass filters, (h 0 , h 1 , h 2 , ..., h L−1 )and (g 0 , g 1 , g 2 , ..., g L−1 ), for the Daubechies wavelets. The QMF condition readsforH(z)H( 1 z ) + H(−z)H(−1 ) = 2, ∀z, (12)zH(z) = L−1 ∑k=0h k z k , (13)where L is the length of the filter and h k are the low-pass filter taps.The conditionnormalizes the filter (and the scaling function).For a wavelet ψ(x) with M vanishing moments, i.e.∫ ∞H(1) = √ 2 (14)−∞ xm ψ(x)dx = 0, m = 0, 1, 2, ..., M − 1, (15)the polynomial H(z) and its derivatives up to order M −1 must be zero at z = −1.Thus,H(−1) = H ′ (−1) = H ′′ (−1) = ...... = H (M−1) (−1) = 0. (16)(i) Use the normalization condition H(1) = √ 2 and QMF (12) to show thatH(−1) = 0.What does the latter condition imply in terms of the filters (h 0 , h 1 , h 2 ,..., h L−1 )and (g 0 , g 1 , g 2 ,..., g L−1 )? What is relevance in signal processing?(ii) Show that (14) and (16) imply that H(z) is of the formwhere Q(z) is polynomial and Q(1) = 1.H(z) = √ ( )1 + zM2 Q(z), (17)2(iii) Show that for a FIR filter with M vanishing moments Q(z) has degree M −1.(iv) Consider the Haar case (L = 2, M = 1). What is Q(z)? Determine H(z) andthe filter coefficients from (17) by comparing with (13).66

(v) If the QMF condition is rewritten in terms of Q(z), one can show (see LectureNotes, no need to prove this) that Q(z) must satisfyQ(z)Q( 1 z ) = M−1 ∑k=0What does (18) reduce to for the Haar case?(−1) k (M − 1 + k)!(1 − z) 2k (4z) −k . (18)k!(M − 1)!(vi) Substitute (13) directly into (12) to compute the conditions for h 0 , h 1 for theHaar case. How many conditions do you get? How many are trivial? How manyare nontrivial? What do these conditions express? Compute h 0 and h 1 explicitly,using (14) to normalize.(vii) Consider DAUB4, where L = 4, M = 2, corresponding to compactly supportedwavelets with two vanishing moments. What is the degree of Q(z)? Use(18) and Q(1) = 1 to determine Q(z) explicitly. Then, continuing with one ofthe (two possible) solutions, use (17) to compute H(z). Finally, use (13) to findclosed-form irrational expressions for the filter coefficients h 0 , h 1 , h 2 , and h 3 .(viii) Substitute (13) directly into (12) to compute the conditions for the h k forthe DAUB4 low-pass filter (L = 4). How many conditions do you get? How manyare trivial? How many are nontrivial? What do these conditions actually express?Verify that low-pass filter (h 0 , h 1 , h 2 , h 3 ) computed in (vii) satisfies all conditions,including (14).67

Books, Review Papers, Info about WaveletsCompiled by Willy Hereman—–Updated: February 9, 2001Books on Wavelets1. Ali N. Akansu and Richard A. Haddad, Multi-resolution Signal Decomposition:Transforms, Subbands, and Wavelets, Academic Press, New York,19922. Ali N. Akansu and M. Medley (eds.), Wavelet, Subband and Block Transformsin Communications and Multimedia, Kluwer Academic Press, Norwood,Massachusetts, 19983. Ali N. Akansu and Mark J.T. Smith, Subband and Wavelet Transforms,Design and Applications, Kluwer Academic Press, Boston, 19954. Akram Aldroubi and Michael Unser (eds.), Wavelets in Medicine and Biology,CRC Press, Boca Raton, Florida, 19965. Anestis Antoniadis and G. Oppenheim (eds.), Wavelets and Statistics, SpringerVerlag, Berlin, 19956. M. Barlaud (ed.), Wavelets in Image Communication, Elsevier, Dublin, 19957. John J. Benedetto and Michael W. Frazier (eds.), Wavelets: Mathematicsand Applications, CRC Press, Boca Raton, Florida, 19938. Andrew Bruce and Hong-Ye Gao, Applied Wavelet Analysis with S-Plus,Spinger Verlag, Berlin, New York, 19969. C. Sidney Burrus, Ramesh A. Gopinath and Haitao Guo, Introduction toWavelets and Wavelet Transforms: A Primer, Prentice Hall, Upper SaddleRiver, New Jersey, 199810. Y.T. Chan, Wavelet Basics, Kluwer Academic Publishers, Norwood, Massachusetts,199511. Charles K. Chui, An Introduction to Wavelets. Vol. 1 series: Wavelet Analysisand its Applications, Academic Press, New York, 199212. Charles K. Chui (ed.), Wavelets: A Tutorial in Theory and Applications.Vol. 2 series: Wavelet Analysis and its Applications, Academic Press, NewYork, 199268

13. Charles K. Chui, Laura Montefusco and Luigia Puccio (eds.), Wavelets: Theory,Algorithms and Applications. Vol. 5 series: Wavelet Analysis and itsApplications, Academic Press, Orlando, Florida, 199514. Charles K. Chui, Wavelets: A Mathematical Tool for Signal Analysis, SIAM,Philadelphia, 199715. Albert Cohen and Robert Ryan, Wavelets and Multiscale Signal Processing,Chapman & Hall, London, 199516. J.M. Combes, A. Grossmann and Ph. Tchamitchian (eds.), Wavelets: Time-Frequency Methods and Phase Space, Proc. Int. Conf. Marseille, France,December 1987, Springer Verlag, New York, 199017. Wolfgang Dahmen, Andrew Kurdila and Peter Oswald (eds.), MultiscaleWavelet Methods for Partial Differential Equations. Vol. 6. series: WaveletAnalysis and Applications, Academic Press, San Diego, 199718. Ingrid Daubechies, Ten Lectures on Wavelets, CBMS-NSF Regional ConferenceSeries in Applied Mathematics, 61, SIAM, Philadelphia, 199219. I. Daubechies, S. Mallat and A. Willsky (eds.), Special Issue on WaveletTransforms and Multiresolution Signal Analysis, IEEE Transactions on InformationTheory 38 199220. Ingrid Daubechies (ed.), Different Perspectives on Wavelets, Proc. of Symposiain Applied Mathematics 47, Lecture Notes of Short Course, San Antonio,Texas, January 1993, AMS, Providence, Rhode Island, 199321. Guy David, Wavelets and Singular Integrals on Curves and Surfaces, LectureNotes Nankai Institute of Mathematics, Tianjin, People Republic of China,June 1988, Springer Verlag, Berlin, 199122. Marie Farge, J.C.R. Hunt and J.C. Vassilicos (eds.), Wavelets, Fractals andFourier Transforms, Clarendon Press (Oxford University Press), Oxford, U.K.,199323. F.J. Fliege, Multirate Digital Signal Processing: Multirate Systems, FilterBanks and Wavelets, Wiley & Sons, New York, 199424. Efi Foufoula-Georgiou and Praveen Kumar (eds.), Wavelets in Geophysics,Academic Press, San Diego, 199469

25. Michael Frazier, An Introduction to Wavelets through Linear Algebra, SpringerVerlag, New York, 199826. Eugene Hernández and Guido Weiss, A First Course on Wavelets, CRC Press,Boca Raton, Florida, 199627. Matthius Holschneider, Wavelets: An Analysis Tool, Clarendon Press (OxfordUniversity Press), Oxford, U.K., 199528. Barbara Burke Hubbard, The World According to Wavelets: The Story of aMathematical Technique in the Making (2nd edition), A.K. Peters, Natick,Massachusetts, 199829. S. Jaffard and Yves Meyer, Wavelet Methods for Pointwise Regularity andLocal Oscillations of Functions, AMS, Providence, Rhode Island, 199630. Kurt Jetter and Florencio I. Utreras (eds.), Multivariate Approximation:From CAGD to Wavelets, Proc. Int. Workshop, Santiago, Chili, September1992, World Scientific, Singapore, 199331. Jean-Pierre Kahane and P.G. Lemarié-Rieusset, Fourier Series and Wavelets,Gorden & Breach, London, 199532. Gerald Kaiser, A Friendly Guide to Wavelets, Birkhäuser Press, Boston, 199433. T.H. Koornwinder (ed.), Wavelets: An Elementary Treatment of Theory andApplications, World Scientific, Singapore, 199334. Andrew F. Laine (ed.), Mathematical Imaging: Wavelet Applications in Signaland Image Processing, Proc. Meeting, San Diego, CA, July 1993, SPIE2034, Bellingham, Washington, 199435. Andrew F. Laine, Wavelet Theory and Applications, Kluwer Academic Publishers,Dordrecht, The Netherlands 199536. P.G. Lemarié (ed.), Les Ondelettes en 1989, Lecture Notes in Mathematics,1438, Springer Verlag, Berlin, 199037. LeMehaute, Laurent and L.L. Schumaker (eds.), Wavelets, Images and SurfaceFitting, A.K. Peters, Wellesley, Massachusetts, 199438. Alfred K. Louis, Wavelets: Theory and Applications, John Wiley, New York;Teubner, Germany, February 199770

39. Stéphane Mallat, A Wavelet Tour of Signal Processing (2nd edition), AcademicPress, New York, 199940. Peter R. Massopust, Fractal Functions, Fractal Surfaces and Wavelets, AcademicPress, Orlando, Florida, 199441. Yves Meyer, Ondelettes et Opérateurs I, II et III, Hermann, Paris, 199042. Yves Meyer, Wavelets and Applications, Research Notes in Applied Mathematics,20, Proc. Int. Conf. Marseille, France, May 1989, Masson, Milan andSpringer Verlag, Berlin, 199143. Yves Meyer, Wavelets and Operators. Vol. 1, Cambridge Studies in AdvancedMathematics 37, Cambridge University Press, Cambridge, 199244. Yves Meyer, Les Ondelettes, Algorithmes et Applications, Armand Colin,199245. Yves Meyer (transl: Robert D. Ryan), Wavelets, Algorithms and Applications,SIAM, Philadelphia, 199346. Yves Meyer, Wavelets and Operators. Vol. 2: Calderón-Zygmund Operatorsand Multilinear Operators, Cambridge Studies in Advanced Mathematics 48,Yves Meyer, Cambridge University Press, Cambridge, 199547. Yves Meyer and S. Roques (eds.), Progress in Wavelet Analysis and Applications,Proc. Int. Conf. on Wavelets and Applications, Toulouse, France, June1992.48. Michel Misiti, Yves Misiti, Georges Oppenheim and Jean-Michel Poggi, WaveletToolbox User’s Guide, The MathWorks Inc., Natick, Massachusetts, 199649. Marius Mitrea, Clifford Wavelets, Singular Integrals and Hardy Spaces, LectureNotes in Mathematics 1575, Springer Verlag, Berlin, 199450. Rodolphe L. Motard and Babu Joseph (eds.), Wavelet Applications in ChemicalEngineering, Kluwer Academic Publishers, Boston, 199451. David E. Newland, An Introduction to Random Vibrations, Spectral andWavelet Analysis (3rd edition), John Wiley, New York; Longman, U.K., 199352. Yves Nievergelt, Wavelets Made Easy, Birkhäuser, Boston, 199971

53. R. Todd Ogden, Essential Wavelets for Statistical Applications and DataAnalysis, Birkhäuser Press, Boston, 199654. Mary B. Ruskai, Gregory Beylkin, Ronald Coifman and Ingrid Daubechies,Wavelets and Their Applications, Stephane Mallat, Yves Meyer and LouiseRaphael, Jones and Bartlett, Boston, Massachusetts, 199255. Larry L. Schumaker and Glenn Webb (eds.), Recent Advances in WaveletAnalysis, Wavelet Analysis and its Applications 3, Academic Press, NewYork, 199356. Eric J. Stollnitz, Tony D. DeRose and David H. Salesin, Morgan Kaufmann,San Francisco, 199657. Gilbert Strang and Truong Nguyen, Wavelets and Filter Banks, Wellesley-Cambridge Press, Wellesley, Massachusetts, 199658. Harold H. Szu (ed.), Wavelet Applications, Proc. Meeting Orland, Florida,April 1994, SPIE 2242, Bellingham, Washington, 199459. C. Taswell, Handbook of Wavelet Transform Algorithms, Birkhäuser Press,Boston, 199660. Anthony Teolis, Computational Signal Processing with Wavelets, BirkhäuserPress, Boston, 199661. Ahmed H. Tewfik, Wavelets and Multiscale Signal Processing Techniques:Theory and Applications, 199762. Bruno Torrésani, Analyse Continue par Ondelettes, Savoirs Actuels, CNRSEditions, Paris, 199563. J.C. van den Berg (ed.), Wavelets in Physics, Cambridge University Press,Cambridge, 199864. Martin Vetterli and J. Kovacevic, Wavelets and Subband Coding, PrenticeHall, Des Moines, Iowa, 199565. Gilbert G. Walter, Wavelets and Other Orthogonal Systems with Applications,CRC Press, Boca Raton, Florida, 199466. Mladen Victor Wickerhauser, Adapted Wavelet Analysis from Theory to Software,A.K. Peters, Wellesley, Massachusetts, 199472

67. M. Victor Wickerhauser, Wavelet Packet Laboratory, A.K. Peters, Wellesley,Massachusetts, 199468. Gregory W. Wornell, Signal Processing with Fractals: A Wavelet-based Approach,Prentice Hall, Upper Saddle River, New York, 199669. P. Wojtaszczyk, A Mathematical Introduction to Wavelets, London MathematicalSociety Graduate Texts, London, U.K., 199770. Randy K. Young, Wavelet Theory and Its Applications, Series in Engineeringand Computer Science. VLSI, computer architecture and digital signalprocessing, Kluwer Academic Publishers, Dordrecht, 1993Some Review Papers on Wavelets1. Marc A. Berger, “Lectures on Wavelets and Lectures on Products of RandomMatrices,” School of Mathematics, Georgia Institute of Technology, Atlanta,GA 30332 (1991) 72 pages2. Gregory Beylkin, “Wavelets and Fast Numerical Algorithm,” Proc. Symposiain Appl. Math. 47 (1993) 89-1773. Gregory Beylkin, “Wavelets, Multiresolution Analysis and Fast NumericalAlgorithm,” Draft of INRIA Lectures, 19914. Adhemar Bultheel, “Learning to swim in a sea of wavelets,” Bull. Belg. Math.Soc. 2 (1995) 1-44.5. Ingrid Daubechies, “Orthonormal Bases of Compactly Supported Wavelets,”Comm. Pure Appl. Math. 41 (1988) 909-9966. Christopher E. Heil and David F. Walnut, “Continuous and Discrete WaveletTransforms, SIAM Review 31 (1989) 628-6667. Björn Jawerth and Wim Sweldens, “An Overview of Wavelet Based MultiresolutionAnalyses,” SIAM Review 36 (1994) 377-4128. Don Lancaster, “Hardware Hacker: Understanding transforms, video compressionsecrets, ... and more wavelet breakthroughs,” Radio-ElectronicsMagazine, July 1991, 68-?.9. Stephane G. Mallat, “A Theory for Multiresolution Signal Decomposition:The Wavelet Representation,” IEEE Trans. Patt. Anal. Mach. Intell. 11(1989) 674-69373

10. Stephane G. Mallat, “Multiresolution Approximations and Wavelet OrthonormalBases of L 2 (R), Trans. AMS 315 (1989) 69-8711. Stephane G. Mallat, “Multifrequency Channel Decomposition of Images andWavelet Model, IEEE Trans. Acoust. Speech Signal Process. 37 (1989) 2091-211012. Peter Meuller and Brani Vidakovic, Wavelets for Kids: tutorial paper for thestatistical community (with Mathematical notebook, part B), ISDS, DukeUniversity. FTP to ftp.isds.duk.edu in directory /pub/brani/wav4Kids[A-B].ps.Z.uu13. Yves Meyer, Book Reviews, Bulletin (New Series) of the AMS 28 (1993)350-360. Review of the books “An introduction to wavelets,” by C. Chui,and “Ten Lectures on Wavelets,” by I. Daubechies14. Olivier Rioul and Martin Vetterli, “Wavelets and Signal Processing,” IEEESignal Proc. Mag. 8 (1991) 14-3815. Alistair C.H. Rowe and Paul C. Abbott, “Daubechies wavelets and Mathematica,”Computers in Physics 9(6) (1995) 635-64816. Gilbert Strang, “Wavelet Transforms Versus Fourier Transforms,” Bulletin(New Series) AMS 28 (1993) 288-30517. Gilbert Strang, “Wavelets and Dilation Equations: A Brief Introduction,”SIAM Review 31 (1989) 614-62718. Gilbert Strang, “Wavelets,” American Scientist 82 (1994) 250-25519. Martin Vetterli, “Filter Banks Allowing Perfect Reconstruction,” Signal Processing10 (1986) 219-24420. Martin Vetterli and Cormac Herley, “Wavelets and Filter Banks: Theory andDesign,” IEEE Trans. Acoust. Speech Signal Process. 40 (1992) 2207-2232Additional Information about Wavelets• Journal: Applied and Computational Harmonic Analysis: Time-Frequencyand Time-Scaled Analysis, Wavelets, Numerical Algorithms and Applications(ACHA), C.K. Chui, R.R. Coifman and I. Daubechies (eds.), Academic Press,First issue: Nov.-Dec. 199374

• Wavelet Digest, editor: Wim Sweldens, Lucent Technologies, Bell Laboratories,http://wim.sweldens.com. Add yourself via email: add@wavelet.org, URL:http://www.wavelet.org/• Video Tape: Wavelets Making Waves in Mathematics and Engineering, IngridDaubechies, MAA Invited Lectures, Baltimore, Maryland, January 1992,AMS-Selected Lectures in Mathematics• Wavelets on World Wide Web• Special Issue(s) on wavelets of IEEE Transactions on Information Theory(March 1992)• Literature Survey by Stefan Pittner, Institute for Applied and NumericalMathematics, Technical University Vienna, Austria, 700 entries, 220 pages,cost $20, 1993, Contact: pittner@uranus.tuwien.ac.at• Journal: Fourier Analysis and Applications, CRC Press, Boca Raton, Florida(1995-present)75