WAVELET REPRESENTATION OF SENSOR
SIGNALS FOR MONITORING
AND CONTROL
By
VINOD KUMAR RAGHAVAN
Bachelor of Technology
Osmania University
Hyderabad, Andhra Pradesh, India
1992
Submitted to the Faculty of the
Graduate College of the
Oklahoma State University
in partial fulfillment of
the requirements for
the Degree of
MASTER OF SCIENCE
May, 1995
WAVELET REPRESENTATION OF SENSOR
SIGNALS FOR MONITORING
AND CONTROL
Thesis Approved:
Dean of the Graduate College
ii
PREFACE
The chemical Process Industries rely heavily on pattern based information utilization.
Process trends and patterns contain virtually all the information about the process and
provide the operators a basis to determine the process condition. It is based on changes in
these ratterns that operators make the necessary adjustments to the process operating
conditions, depending on their interpretation. In general, process condition can
confidently be judged only by monitoring multiple signals simultaneously and by context
dependent pattern recognition. A simple example of a exothermic CSTR can be used to
illustrate this. When the rector temperature rises and the coolant flow rate remains
constant or decreases, the operator may take it as a premonition of impending doom, but
if the coolant flow rate also goes up to accommodate for this increase in reactor
temperature, the operator might take no prophylactic measures. It can easily be concluded
that good signal processing techniques are necessary to develop the next generation of
automated process monitoring techniques.
Sensor signals samples are periodically collected by data collecting techniques that
usually introduce sensor noise to the already noisy raw signal from the process. This raw
signal has to be converted to a useful form before it can actually be used further. The
challenge is to provide a signal processing technique that can process a signal on-line and
feed it to a process monitoring technique. Howev~r, the catch is that all signal processing
cause signal distortion at least to some extent. This cannot be tolerated, especially
distortions towards the end ofthe signal, samples that correspond to the most recent
juncture in time. This is because it is these samples that trigger off a process monitoring
technique into predicting the process as normal or abnormal. Prediction of a normal state
iii
as abnormal can probably tolerated but predicting an abnormal state as normal can prove
catastrophic.
In this work a novel signal processing method using Wavelets is discussed. Wavelet
Transforms are similar to the age old Fourier Transforms, but outmatches the Fourier
Transforms in many desired attributes. Wavelets not only provide a powerful signal
processing/analysis technique, they are also capable of providing significant data
compression. In this work data compression's ofthe order of 90% were achieved without
significant loss in the information content of the signals.
This work to our knowledge is the first to emphasize the importance of signal
extension techniques for an applications like pattern recognition. A new signal extension
technique (NET) is described and its superiority over other signal extension techniques is
demonstrated.
I wish to thank Dr. James R. Whiteley for providing me the opportunity to work
on this project. As my adviser on this project and a guide in many other issues, he
provided me with guidance and support constantly. I am also grateful to him for lending
an ear whenever I needed advise and support on personal issues too. I also wish to thank
Dr. Arland H. Johannes and Dr. Jan Wagner for serving on my thesis committee.
I would also like to take this opportunity to thank Jack DeVeaux, Bruce Colgate at
PDC Center of the Phillips Petroleum Co., Bartelesville and Tommy Long, Ben Mackay
and Barbara Barton at the Philli!-'.) Refinery in Borger, TX, for their co-operation. I am
also indebted to the School of Chemical Engineering, Oklahoma State University for
providing me financial support.
Thanks are also due to my friends for their constant moral support. I wish to
express my gratitude to my family, especially to the person who made everything
possible, my mother. Special thanks are also due to Mrs. Ann Whiteley, for her
painstaking review of this thesis.
I dedicate this thesis to the fond memory ofmy uncle, Mr. K.G. Murali Babu.
iv
TABLE OF CONTENTS
Chapter Page
I. INTRODUCTION................................................ 1
Thesis Outline . . . . . .. . . . . . . . . .. . . . . . . .. . . . . . . .. .. . .. .. . . . . . . . . . . . . .. 2
II. SIGNAL ANALYSIS 3
Introduction. . . . . . . . . . . . .. . . . . . . . .. . . . . . . . . . .. .. .. .. . . . .. .. .. . .. . .. . . . . 3
Signal Representation.. . .. . .. . .. .. .. . . . .. . .. . .. .. .. .. . .. . .. .. . . .. .. . . . . .. . . . . .. . . 4
The Fourier Transform. . . . .. .. . . . . . . . .. .. . . .. .. . .. . .. . . .. . . .. .. . . .. .. .. . .. .. . . 6
Discrete Wavelet Transforms. . . . . . . .. . . . . . . . .. . . . . . . .. .. . . . . .. . . .. .. .. .. . .. 7
Analogy to Fourier Transforms. . . . . . .. . . . .. .. . .. . .. .. .. .. .. . . . . .. . . . . . . 7
III. WAVELET TRANSFORMS. .. . . . . . . . .. . . .. .. . .. .. . . . . . . . .. .. . . .. .. .. .. . .. . . .. 12
Introduction . .. . .. .. .. .. . . .. . .. . .. . .. .. .. .. . .. . .. .. . . . . .. . . .. .. .. .. .. .. .. .. .. .. .. . .. . . .. 12
Dilation Equations .. .. .. . .. .. . . . . . .. .. . . .. .. . .. . . .. . .. .. .. . . .. .. .. .. . . . . . .. . . .. .. .. 12
Wavelet Construction .. .. .. .. .. . . .. . . .. . . .. .. . .. .. . .. .. . .. . .. . .. .. .. .. .. .. . . . .. . .. . 17
Computing the Scaling Function and Wavelet Coefficients . . .. . .. .. .. .. .... 17
The Daubechies Family. .. .. .. . . . . .. . . .. .. . . . . . . .. .. . .. .. .. . . . .. . .. .. .. .. .. 21
Construction of the Scaling Function and Wavelet .. . .. . . .. .. . .. .. .. . .. .. .. 26
Computing the decomposition coefficients.. .. . . .. .. .. . . .. .. . . .. . . . .. . . .. . .. .. .. 32
Multi-Resolution Analysis.. .. . . .. . . .. . . .. . .. .. . . . . . .. .. . . . . . . . . .. . .. . . 35
Sllffiffiary . .. .. . . .. . .. .. . .. . . .. .. . .. . .. .. . .. . . .. . . .. .. .. . .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 38
IV. SIGNAL EXTENSION. .. . .. . .. .. .. .. .. .. . .. . .. .. .. .. .. .. .. .. .. . .. . .. .. .. .. .. .. . . . . .. .. .. .. .. 47
Need for signal extension for sensor signals from
Chemical Processes . .. .. .. .. .. .. .. .. . . . .. .. .. . .. .. .. .. . .. . .. .. .. .. .. . .. .. .. .. .. .. .. . .. . . 39
Common signal extension methods.. . . . .. . . .. . .. . .. .. .. . .. .. . .. .. . . .. .. .. . . .. .. . .. 40
Circular Extension.. . . .. .. .. .. .. . .. .. . . .. .. .. . .. .. .. .. .. . .. . . . .. .. .. . . ... .. .. . . .. 40
Symmetric Extension.. .. .. . . .. . .. . .. .. . .. .. . .. . .. .. .. .. . .. .. . .. .. .. .. . .. .. .. . . .. . 41
v
..
Chapter Page
Padding with Zeros. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
Padding with a constant value. . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Summary of conventional extension methods . . . . . . . . . . . . . . . . 44
The New Extension Technique. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
NET! 46
NET2............................................. 48
NET3.............................................. 48
NET4.............................................. 49
Demonstration ofNET technique. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Case I . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
Case II . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
Case III . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
v. CONCLUSIONS.............................................. 65
Recommendations and Future work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
vi
LIST OF TABLES
Table Page
I. Scaling function Coefficients for the Daubechies Family
of Wavelets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
vii
Figure.
LIST OF FIGURES
Page
1. A Sampled Process Signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2. Fourier Coefficients ofthe Process Signal in Figure 1 . . . . . . . . . . . . . . . . . . 7
3. Fourth order Daubechies family scaling function and wavelet. . . . . . . . . . . . . 8
4. Wavelet Transform on a time-frequency scale. . . . . . . . . . . . . . . . . . . . . . . . . 9
5. Comparison of 170urier Decomposition and Wavelet Decomposition. . . . . . .. 10
6. Illustration of a Wavelet at different dilations and translations. . . . . . . . . . . .. 14
7. The dyadic sampling ofa Wavelet grid. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
8. Figure showing some Wavelet Families. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
9. The Box Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
10. The Haar Wavelet and its dilates and translates. . . . . . . . . . . . . . . . . . . . . 19
11. The Hat Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
12. Wavelet associated with the Hat function. . . . . . . . . . . . . . . . . . . . . . . . . . . 20
13. Scaling Functions associated with Daubechies Wavelets. . . . . . . . . . . . . . . . 30
14. Daubechies Wavelets. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
15. A basic decomposition and reconstruction representation. . . . . . . . . . . . . . .. 32
16. DepictionofaMRAalgorithm..................................... 35
17. Representation of a Signal Decomposition upto three levels . . . . . . . . . . . . .. 37
viii
18. Periodic Extension Technique. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
19. Example of periodic extension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
20. Symmetric Extension Technique. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
21. Example of symmetric extension. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 42
22. Extension with Zeros. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
23. Example of extension with zeros. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
24. Extension with the boundary value. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
25. Example of extension with a constant value. . . . . . . . . . . . . . . . . . . . . . . . . . 44
26. The New Extension Technique. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
27. Different Extension Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 50
28. Representation of Effect of Signal Extension on Signal
Combination. Case I. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
29. Case II. . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
30. Case III . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
ix
8(t)
f(t)
<pet)
\V(t)
f(~)
1no(~)
H
G
SUMMARY OF NOMENCLATURE
A unit impulse response
Real time representation of a signal
Decomposition basis functions
Decomposition coefficients of a k , ~k respectively
Scaling function
Wavelet
Scaling function coefficients
Wavelet coefficients
Fourier transform off(t)
A periodic function
High pass filter
Low pass filter
}.th blurreddecom·pos·ltlon coefJ~.IC·lent at thel.th
decomposition level
jth detail decomposition coefficient at the lh
decomposition level
Mean of i signal samples
Mean squared deviation of i signal samples
x
CHAPTER I
INTRODUCTION
The process industries rely heavily on pattern-based information utilization. Robust
featur~ extraction and pattern recognition techniques are essential for the successful
automation of a chemical process plant. Pattern based monitoring techniques require
compact representation ofthe sensor data. In this work we describe a wavelet technique
that provides efficient signal representation and trend pattern extraction.
Sensor signals in general are heavily marred by noise. Noise can mask the actual
signal trend and make pattern recognition and feature extraction difficult. To enable
better trend resolution, signals are analyzed using different techniques, e.g. the Fourier
transform. The disadvantage of using conventional signal processing techniques such as
the Fourier transform is that they are ineffective at handling localized signal behavior.
Loss of a vital piece of infonnation can be unacceptable for monitoring and control
purposes. Wavelet transforms provide localized signal processing capability and better
trend pattern representation.
All signal extension methods require the signal to be extended to prevent trend
distortions. This work identifies the drawbacks ofcommon signal extension methods and
recommends a more effective method of signal extension.
Critical factors such as the wavelet family used, the number of levels of signal
decomposition, and possible data compression techniques have also been explored. A
detailed description of these topics is beyond the scope of this work, however.
2
Thesis Outline
The organization of this thesis is as follows, Chapter II describes signal analysis
techniques in general with special emphasis on Fourier and wavelet transfonns. This
chapter also discusses time-frequency relationships and introduces the concept of multiresolution
analysis.
Chapter III discusses general mathematical details of wavelets and wavelet
transforms. Special mention is made of the Daubechies family of wavelets. This wavelet
family is used extensively in this work. This chapter also presents the concept of multiresoiution
analysis from a theoretical perspective.
Chapter IV is the main body of this work. It presents an overview ofcommon signal
extension methods and describes their shortcomings when used with wavelet transforms
for pattern recognition and feature extraction applications. New signal extension methods
are presented which avoid the problems with traditional techniques. Perfonnance of
these new methods is illustrated with case studies.
Chapter V concludes with a summary of this work and sets forth recommendations
for future study.
CHAPTER II
SIGNAL ANALYSIS
Introduction
Signals are commonly analyzed by decomposing them into their frequency
components. Frequency components are portions of the signal with different energies.
Portions with higher energies correspond to the higher frequencies and, likewise, lower
frequencies correspond to lower energy. A sound method for signal analysis requires
detection and explicit representation of the temporal features in a joint time-frequency
space. The ultimate objective of signal processing is to provide a unique signal
representation.
The high frequency part of a signal is generally measurement noise and can be
eliminated to the desired degree by passing it through a low pass filter. Filtering, in
signal processing parlance, is analogous to physical filtering, e.g. separation of suspended
impurities from a liquid. A low pass filter is one that allows the low frequency
components ofthe signal to pass through. Low frequencies form the basic trend patterns
of the signal. High pass filters do the opposite.
Signal processing techniques should avoid distortion of process trend, otherwise the
interpretation of the trend by a process monitoring technique may be erroneous.
Desirable qualities of a signal processing technique for trend extraction include the
following:
3
4
• The technique should have the ability to analyze a signal over numerous resolution
levels and capture the essential features of the process trend.
• The processing technique should ensure that there is neither (a) over-sampling which
results in information redundancy, or (b) under-sampling which could result in
information loss.
• The technique should be able to treat local behavior as a local, not global event. For
example, if a sudden impulse in the process trend shows up sharply on the sensor
reading, this is generally indicative of a valve failure, etc. Analyzed using a global
technique, e.g. the Fourier transforms, the processed signal could show a more
sluggish change like a load change. This representation is misleading.
• The signal analysis technique should provide a unique signal representation immune
to signal translations i.e., the signal translated in time should provide a representation
identical to the original representation, but translated appropriately.
Signal Representation
If "I' is a sampled data representation of a sensor trend pattern containing N samples,
thenfcan be expressed as shown below:
N-l
f(t) = .LPn8(t-n) ,t=O....N-l
n=O
and where 8(n) is defined as a unit impulse response and has a value
(1)
8(X) ={~ ifx=O
otherwise
(2)
5
f(t)
6
Sensor Value
4
2
o 2 3 4 5 6 7
t
Figure 1: A sampled process signal.
In Figure 1, an 8 point signal is depicted. The trend pattern can be expressed using a
time domain representation by a vector of coefficients Pn.
7
l(t)= LPnD(t-n),
n=O
T where p=[l 2 3 3 2 4 5 6] .
(3)
The signal in Figure 1 is localized in time using a time domain representation because it
is expressed as a series of impulses in time. The magnitude (the sensor reading in this
case) of an impulse gives the exact signal value at that instant of time. Frequency
localization can be achieved by employing a Fourier transform to identify the frequency
components that constitute the signal.
There are a number of other ways this signal can be represented. In general, the
signal can be expressed by a pair of basis functions a. and p. The signal can be
expressed as the sum of its decomposition products, or
where k is an index that defines the length ofthe signal.
(4)
6
The basis functions vary depending on the nature of the decomposition method adopted,
and gk and hk are the decomposition coefficients associated with ex. (t) and P(t),
respectively.
The most common signal analysis techniques are discrete Fourier transforms, discrete
cosine transforms and, more recently, discrete wavelet transforms. The remainder of this
chapter describes two methods: Fourier transforms and wavelet transfonns. The Fourier
transform [Bracewell, 1965; Brigham, 1974; Weaver, 1983] is described in detail to
facilitate easier understanding of wavelet transforms.
The Fourier transform
In the Fourier transform, the basis functions in equation (4) are sine and cosine
functions. The Fourier transform decomposes the signal into sines and cosines of
different frequencies and amplitudes, and these functions collectively form the original
signal. For Fourier transforms, h and g in equation (4) are sine and cosine amplitude
coefficients.
The mathematical representation of Fourier decomposition of a signalf(t) is
f(t) =Lgk cos(tk) +Lhk sin(tk)
k k
An alternative representation of the Fourier transforms (FT) is
(5)
(6)
Though powerful and popular, Fourier representations have some serious limitations
[Bracewell, 1965; Brigham, 1974; Weaver, 1983]. They do not provide easy insight into
the time domain behavior ofthe original signal which is essential for trend pattern
representation. Figure 2 shows the time domain decomposition representation of the
7
process signal in Figure 1. This representation is generally inadequate for trend pattern
representation and process monitoring for applications other than vibration monitoring.
This limitation comes about from the fact that the analyzing functions in the Fourier
transform are global in nature.
Cosine Coefficients Sine Coefficients
Figure 2: Fourier coefficients of the process signal in Figure 1
There are at least two ways to overcome this drawback. The first solution is to
introduce time dependency in Fourier transforms so that the signal is analyzed, not on a
global scale, but within windows in time [Akansu and Haddad, 1993; Chui, 1992b].
Another way of handling this problem is to use other basis functions that are more
concentrated in time. This is where wavelet transforms come into the picture. They use
analyzing functions that are localized in time.
Discrete wavelet transforms
Wavelet transforms [Chui, 1992a; Chui, 1992b; Cohen et al., 1992a; Daubechies,
1988; Daubechies, 1990; Daubechies, 1992; Mallat, 1989a; Mallat, 1989b; Strang, 1989]
enable analysis in both time and frequency.
Analogy to Fourier transforms: In the same manner as sines and cosines in the
Fourier transfonns, wavelets and scaling functions form the basis functions in the wavelet
8
transforms. A key difference is that while sines and cosines have simple analytic
expressions, scaling functions·and wavelets are complex functions which are derived
rather than naturally occurring. Wavelet decomposition of a signal is also represented by
equation (4), where
a. k V) is the scalingfunction
Pk V) is the wavelet
A sample scaling function and wavelet is shown in Figure 3.
2
O+,-__--+-_--#-~ _
4 5 6 7
-1
2
o
-1
-1
6 7
(a) (b)
Figure 3: (a) Fourth order Daubechies family scaling function and (b) wavelet
Wavelet transforms provide variable frequency analysis capability instead of constant
frequency analysis as in Fourier transforms [Strang, 1993]. The analysis frequency can
be varied on a logarithmic scale as shown in Figure 4. Since time and frequency are
inversely related, time resolution improves at higher frequencies. Two closely spaced
impulses in a real time representation ofthe signal can be distinguished by a wavelet
decomposition.
9
J~'
I -1 I I
o --
k
fre~uency
n time
Figure 4: Wavelet transform on a time frequency scale
A comparison of a Fourier decomposition of a signal and a wavelet decomposition of
the signal can be seen in Figure 5. The time domain representation ofthe Fourier
transform provides a poor trend pattern representation of the original signaL Its wavelet
counterpart on the other hand, provides an excellent trend pattern representation of the
original signal. This representation is a replica of the original signal, but on a coarser
scale. The second part of the wavelet decomposition contains signal details present in the
original signal but lost in the coarse representation. It can be inferred that wavelet
transforms provide better time-frequency localization than Fourier transforms.
0.80
0.76 ~ _
time
(a)
10
(b)
0.80 0.004
0.002
0.78
0
0.76 -0.002
(c)
Figure 5: Comparison of Fourier decomposition and wavelet decomposition (a) original
signal, (b) Fourier decomposition and (c) wavelet decomposition down one level using
the third order Daubechies family.
This chapter introduced two important signal processing techniques. Fourier
transforms were described, and their shortcomings were discussed. Fourier transforms
work well for stationary signals but not for process signals which are generally not
stationary. Wavelet transforms were introduced and their advantages over Fourier
transforms were outlined. The next chapter deals with wavelet transforms in greater
detail.
11
CHAPTER III
WAVELET TRANSFORMS
Introduction
Though wavelet transforms are relatively simple to implement, the mathematics are
somewhat complex. This chapter discusses the mathematics of wavelet transforms. The
general methodology is discussed, and the Daubchies wavelet family is specifically
illustrated [Cohen et al., 1992a; Daubechies, 1992; Daubechies, 1988]. The finer
mathematical aspects of multi-resolution analysis are discussed as well as various
methods of applying wavelet transforms for multi-resolution analysis. The first subject
presented is the dilation equations that constitute the mother functions of both scaling
functions and wavelets. Computation of the scaling function and wavelet coefficients is
discussed, and the actual synthesis of scaling functions and wavelets is described.
Dilation Equations
Scaling functions and wavelets are represented by special types ofequations called
dilation equations [Daubechies, 1992; Daubechies and Lagarias, 1992a; Daubechies and
Lagarias, 1992b; Strang, 1989]. The general form is
Tl(t) =I1kTl(mt - k)
k
12
(7)
13
Here, I is a vector of dilation function coefficients which represents the impulse
response of the filter associated with the dilation equation. Consequently the Ik's are
referred to as the filter coefficients.
The general equation for the scaling function is
cp(t) = LckCP(2j t - k),
k
where Ck is the scaling function coefficient. Similarly the wavelet equation is given by
\II (t) = Ldk<P(2j t - k) ·
k
where dk is the wavelet coefficient.
(8)
(9)
Note that the scaling functions are expressed as a sum of dilations and translations of the
function. Wavelets are defined as functions of the scaling functions, because the
decomposition basis functions have to be interdependent.
In equation (7), the term m provides the ability to dilate depending on the analysis
level. Frequency localization is thus obtained. Hence, m is called the dilation coefficient.
The function constricts for high frequencies, and dilates to capture lower frequencies.
Figure 6 illustrates the dilations and translations of a wavelet. As the level of analysis
increases, the functions dilate and span a wider frequency range. During analysis,
frequency is gradually sliced in a logarithmic fashion. When m equals 2, the dilation is
binary.
14
" I.
I •
• I
• I.I
.I
I
..I
--original wavelet
- - - dilated and transated
• • • • • dilated and translated
Figure 6: Illustration of a wavelet at different dilations and translations.
The other parameter, k, is the translation coefficient that provides translation in time
and thus time localization. This translating coefficient can assume any range of values,
though a finite range is generally adopted for computational ease.
Together these parameters provide the necessary time-frequency localization, the
most attractive feature of the wavelet method. Generally translations are dyadic. This
means that signal sampling after wavelet transform is dyadic, and every alternate
decomposition coefficient is sufficient to obtain perfect reconstruction. A uniform
sampling grid is maintained in a dyadic wavelet transform [Daubechies, 1990; Oslen and
Seip, 1992; Walter, 1992]. Figure 7 shows a sampling grid for wavelet transforms.
..
xxx XXXXX XXX
X X
x
X x
x
X
Figure 7: The dyadic sampling of a V ""1velet grid.
15
It can be seen that the number of coefficients at any level is half the number of
coefficients at the previous level.
Wavelets provide the ability to analyze a signal at multiple resolutions. To resolve
the finer details, the resolution is increased. Similarly, when resolution is gradually
decreased, the object becomes more and more blurred [Cohen et al., 1992b; Daubechies,
1991; Mallat, 1989a; Mallat, 1989b; Mallat, 1989c]. The coefficients from a wavelet
analysis are representative of the original signal, but across different resolution levels.
Since a variety of scaling functions and wavelets are possible, we retain the liberty to
experiment with different basis functions and to use the decomposition that best suits our
needs. The different types of wavelets come about from the fact that in equation (7), 11
can be different functions and k can take on different values.
A variety of wavelets [Battle, 1987; Chui, 1992; Chui and Wang, 1991; Chui and
Wang, 1992; Cohen et al., 1992a; Daubechies, 1992; Daubechies, 1993; Lemarie, 1990;
Lemarie, 1988; Meyer, 1985] can be constructed depending on conditions imposed
during their construction. Wavelets are broadly classified into families, based on their
nature. These families of wavelets are further sub-divided into orders, depending on the
order of filters, i.e. the filter lengths. Similar to the sines and cosines, wavelets with
infInite support can be defined. Recent advances in wavelet technology have led to the
development of finitely supported wavelets. Finite length wavelets are advantageous in
computational ease over their infinitely supported counterparts, but cause translation
variance. Some common wavelet families (Figure 8) are:
• The Daubechies family [Daubechies, 1988] and coiflets [Daubechies, 1992];
• The Battle-Lemarie family [Battle, 1987; Lemarie, 1988];
• The Meyer family [Meyer, 1985];
• Biorthogonal wavelet [Cohen et al., 1992a];
• Wavelets from Cardinal Splines by Chui [Chui, 1992a; Chui and Wang, 1991;
Chui and Wang, 1992;" Chui and Wang, 1993].
16
scaling function
(a)
(b)
(c)
(d)
wavelet
Figure 8: Figure showing some wavelet families, (a) the Daubechies family, (b)
biorthogonal wavelets, (c) Battle-Lemarie wavelet and (d) coiflets.
17
Wavelet Construction
Before we delve into the actual construction of wavelets, a few terms are defined to
facilitate easier understanding ofthis topic.
Orthogonality: A function ~(t) is said to be orthogonal if the following relationship
is satisfied;
(10)
where "c" is a constant. When c is unity, the function is both orthogonal and normalized,
i.e., it is orthonormal.
In wavelet construction the first step involves computation of the scaling function
coefficients in equation (8), the wavelet coefficients in equations (9), and, finally, the
actual scaling function and wavelet function.
Computing the scaling function and wavelet coefficients: We consider the simplest
form of wavelets, called the Haar wavelets.
Case 1: The Haar case
The Haar wavelet is constructed from the simple box function shown in Figure 9.
The scaling function is a simple box function.
{
I 0 ~ t ~ 1
<pet) = 0
otherwise
(11)
18
o-...._-~o-~.5~-----------
Figure 9: The box function.
This satisfies the dilation equation
<pet) = coq>(2t) +c1<P(2t -1). (12)
Comparing equation (8) with equation (12) gives co=l, cl=1. These are the scaling
function coefficients for the Haar wavelet. In this case, the box function is a sum oftwo
half sized boxes, both of unifonn dilation. The difference between them is that one box is
translated by a unit value. Since the box function is defined by two non-zero coefficients,
it is called a filter of length two.
The wavelet corresponding to the box like scaling function in Figure 9 is depicted
below along with a dilated version and a dilated and translated version.
19
1 - ~
0 0
1 1/2 1/2 1
-1 - -- Figure 10: The Haar wavelet \J1(t) and dilated and translated versions,
w(2t), \J1(2t -1).
The Haar wavelet is given by
where do =1, dl =-1. It can also be defined as
(13)
1,
o/(t) = -1,
0,
05: t < 1/2
1/2 ::; t ::; 1
otherwise
(14)
Case 2: A linear spline case:
This example is a wavelet constructed from the simple hat function shown in Figure
11
t, °::; t ::; 1
<p (t) = 2 - t, 1 ~ t S 2
0, otherwise
(15)
This satisfies
o
1\
I \
I \
I \
I \
I \
, 1 \ /'
,I \ /'
I' /\
1 " /' / \
I , /' \
I ,,/' \
Figure 10: The hat function.
2
20
(16)
Equation (16) is satisfied for co=c2=1/2, cl=l. In this case the scaling function is the
sum of three dilated and translated versions. Thus, a variety of combinations of dilated
and translated versions of these functions can be used to generate wavelets starting from
different basis functions. This is possible, provided the basis functions meet certain
requirements documented in the following sections.
The wavelet for the hat function is shown in Figure 12. This wavelet is expressed in
Figure 12: Wavelet associated with the Hat function
the form of a dilation equation as follows:
21
\11 (t) =do<p(2t) +d1<P(2t -1) +d2<P(2t + 1) (17)
where dO=l,dl=d2= -1/2.
In case 2, the scaling function is composed of combinations ofpolynomials that are
piecewise linear; case 1 is satisfied using simple constants. Piecewise linear polynomials
are not effective at approximating [Rice, 1964] stochastic signals like sensor signals.
Higher order polynomials can also be used [Daubechies, 1988] and are better for
approximating these kinds of behavior. Polynomials that provide both orthogonality and
compact support are preferred over other kinds of approximating functions. This was not
achieved until Daubechies constructed such functions [Daubechies, 1988]. The beauty of
Daubchies wavelets is that they are not only orthogonal but are compactly supported as
well. It was previously believed that orthogonality could only be achieved at the expense
of compact support. The next section extends the procedure for the Daubechies family.
The Daubechies Family: Wavelet construction is actually preferred in the frequency
domain because dilation and translation parameters are more easily handled in the
frequency domain than in the time domain. In the general methodology [Chui, 1992a;
Daubechies, 1992; Daubechies, 1993; Strichartz, 1993] , translates ofthe scaling function
are approximated by a polynomial. The coefficients ofthe polynomial are the scaling
function coefficients ck's .
Depending on the nature of the wavelet desired, constraints are imposed during the
construction ofthe scaling function. For instance, the orthonormality condition imposed
is
f<p(t)dt =1
and for orthogonality,
f\V (t)dt =0 ·
(18)
(19)
22
Orthogonality is necessary fot perfect signal reconstruction.
A limit is imposed on the number of finite non-zero Ck to ensure that the scaling
function and wavelet are compactly supported or span a finite range. Support is defined
as the span of the scaling function or the wavelet; i.e. the spread or expanse over which
the function has a non-zero value.
Once the scaling function is constructed, the wavelet is then constructed from the
dilation equation (9). Constraints imposed on scaling functions and wavelets are actually
constraints on their respective coefficients, as is evident from the equations below.
Integrating equation (8) and comparing the orthonormality condition (equation 18)
I<p(t)dt = IICk<P(2t - k)dt = 1
and
(20)
Similarly, integrating equation (9) with the orthogonal condition described by equation
19
Iw (t)dt =IIdk<P (2t - k )dt = 0
or,
(21)
The condition for a set of wavelet functions to be orthogonal is that the sum of its
scaling function coefficients should be two (from equation (20)). For the wavelet
functions to be orthonormal as well, the sum ofthe wavelet coefficients should be zero.
23
There is a special relationship between the ck and dk when the scaling function and
wavelet function are Quadrature Mirror Filters (QMF's) of each other [Akansu et al.,
1993; Cohen et al., 1992a; Daubechies, 1988]. This relationship is given by
(22)
Simply stated, the wavelet coefficients are obtained from the scaling function coefficients
by reversing the order and changing the signs on every alternate coefficient. This way,
we need not calculate the wavelet coefficients (the dk'S) separately. They can be obtained
directly from the scaling function coefficients (ck'S).
This procedure is illustrated further by an example [Daubechies, 1988]. The
construction of the Daubechies family of compactly supported orthonormal wavelets is
briefly discussed in the following paragraphs. Only an abstract of the construction
method is discussed in this work. The full treatment of this topic can be obtained from
[Akansu and Haddad, 1993; Chui, 1992a; Daubechies, 1992; Daubechies, 1993; Haykin,
1991; Strang, 1989]. The conditions imposed for the construction ofthe Daubechies
family of wavelets are that the scaling function has 2Ncoefficients and lies supported
between 0 and 2N-l, where N is the order of the wavelet within the family. This means
that the coefficients, other than those lying in the region between 0 and 2N-l, are zero in
value. This gives rise to the following condition:
Cn= 0 for n <0 or n>2N-l (23)
Also, they form an orthonormal family of wavelets. Since there are only a finite number
of coefficients, equations (20) and (21) yield
2N-l
~>k =2
k=O
2N-l Ldk =0.
k=O
24
(24)
As mentioned previously, actual construction of scaling functions is performed in the
frequency domain. The Fourier transform of a functionf(x) is defined as
(25)
Note that this is the continuous Fourier transform. The previously described equation (5)
is the discrete version of this transform.
The Fourier transform of the scaling function is
Simplifying,
<p(~) = t LCke-i~/2<p(~).
k
This equation can now be expressed as function of another simple equation. Define
then equation (26) can be expressed as
(26)
(27)
(28)
25
Also, equation (28) can be further extended as
(29)
Equation (29) indicates that the translates of <pet) can be approximated by an infinite
length polynomial. For the Daubechies case, however, approximation is attempted using
a finite length polynomial. Obviously, not all finite length polynomials meet the
orthonormality requirements. Only special types of polynomials can possibly satisfy all
the requirements. This polynomial should also provide the best approximation for the
desired scaling function. From equation (27) it can be seen that the coefficients of the
polynomial directly yield the scaling function coefficients. Since real coefficients are
preferred, the polynomial in equation (27) is a cosine polynomial, the reason being that
the polynomial in equation (27) is expressed as a function of e-i~ whose real part is the
cosine part. The proof that this polynomial is periodic is given in Appendix A. Using
z=e-i~, the polynomial is transformed into a function of z.
The polynomial in equation (27) is subsequently defined as
mo(~)=p(Z)=C;Z)NE(z)
Here E(z) is the cosine polynomial and constitutes the real part of the polynomial in
equation (27). Thus only real coefficients will be generated. The periodicity of P(z)
causes it to satisfy equation (31)
(30)
(31)
26
The importance ofthis equation is our attempt to generate functions that are as symmetric
as possible.
The general solution to equation (31) is given by
2 N-l(N - 1+ k) t: )2k fc 'iN
IE(z)j =I \sint + ~in!} 't o(CO;~).
k=O k
(32)
From equation (32), using a technique called spectral factorization [Chui, 1992a;
Daubechies, 1988]), IE(z)l, the absolute value ofthe polynomial. ·s substituted in
equation (30) which gives the coefficients for the scaling function for the Daubechies
Family. From the QMF relationship described by equation (22), the wavelet coefficients
are computed.
The construction can be summarized as follows :
• The order of the wavelet N is decided and the polynomial in equation (32) is
synthesized.
• The polynomial is spectral factorized and the "square root" ofthe polynomial is
calculated.
• The polynomial from step (2) is multiplied with another binomial polynomial as in
equation (30) to make the resultant polynomial as regular as possible. Binomial
polynomials are always symmetric.
• A polynomial of order 2N-l results from step (2). Normalization of these coefficients
according to equation (27) gives the scaling function coefficients for the Nth order
Daubechies family of wavelets.
• The wavelet coefficients are calculated from the relationship in equation (22).
A more detailed explanation of this procedure is described in Appendix A. Also
illustrated in Appendix A is the procedure for calculating the scaling function and
27
wavelet coefficients for the second order Daubechies family. The next section describes
construction of scaling functions and wavelets from their respective coefficients.
Construction of the scaling function and wavelet: Having generated the scaling
function coefficients and wavelet coefficients, we proceed to calculate the actual function
values using a method adopted from Gilbert Strang's classic article [Strang, 1989].
Scaling function values are calculated at integer points. This is done by constructing a
matrix Mij =C2i- j - l • The left eigenvector for A=I gives the value of the scaling function
at the integer points.
Construction ofthe scaling function and wavelet for the second order Daubechies
family is described below.
By definition, the following equality holds for the Daubechies second order family:
<p (t) =0 E t ~ 0and t ~ 3. (33a)
The value of the scaling function at t=1 and t=2 must be evaluated. For t=1 and t=2 in
equation (8),
<p(1) =co<p(2) + c1<P(I)
<p(2) = c2<P(2) + c3<P(1)
(33b)
(33c)
Solving these two equations simultaneously is identical to solving the matrix
<p(0) Co 0 0 0 <p(O)
<pel) c2 c1 Co 0 <pel)
= (33d)
<p(2) 0 c3 c2 c1 <p(2)
<p(3) 0 0 0 c3 <p(3)
28
equation (8). Inserting zeros between the known values ofthe scaling function, the
resultant vector is [<p(0) 0 <P(1) 0 <p(2) 0 <p(3)]. This vector is convolved with the
scaling function cr '":'fficients. The resultant vector contains the values
[<p(O) <p(O.5) <pel) <p(1.5) <p(2) <p(2.5) <p(3)].
This operation can be repeated to calculate the scaling function values at any number of
intermediate points.
The wavelet values are computed in the same manner. To compute the wavelet
values at n points, the scaling function values at only half the number of points are
needed, i.e., the values of scaling function at only nl2 points are needed. Zeros are
inserted between each value of scaling function and convolved with the wavelet
coefficients. The first n points ofthe convolution product are retained and provide the
values ofthe wavelet at the desired points.
Sample coefficients are shown in Table I. Figures 13 and 14 show some Daubechies
family scaling functions and their corresponding wavelets.
29
Scaling function coefficients for the Daubechies family. N is the order ofthe wavelet
N=2 N=7 N=9
0.48296291314453 0.07785205408501 0.03807794736391
0.83651630373781 0.39653931948193 0.24383467461279
0.22414386804201 0.72913209084625 0.60482312369057
-0.12940952255126 0.46978228740519 0.65728807805169
N=3 -0.14390600392858 0.13319738582482
0.33267055295008 -0.22403618499389 -0.29327378327973
0.80689150931109 0.07130921926683 -0.09684078322325
0.45987750211849 0.08061260915109 0.14854074933814
-0.13501102001025 -0.03802993693502 0.03072568147931
-0.08544127388203 -0.01657454163067 -0.06763282906141
0.03522629188571 0.01255099855610 0.00025094711483
N=4 0.00042957797292 0.02236166212370
0.23037781330890 -0.00180164070405 -0.00472320475776
0.71484657055292 0.00035371379997 -0.00428150368247
0.63088076792986 N=8 0.00184764688306
-0.02798376941686 0.05441584224311 0.00023038576352
-0.18703481171909 0.31287159091434 -0.00025196318894
0.03084138183556 0.67563073629737 0.00003934732032
0.03288301166689 0.58535468365426 N=10
-0.01059740178507 -0.01582910525641 0.02667005790061
N=5 -0.28401554296164 0.18817680007804
0.16010239797420 0.00047248457388 0.52720118893260
0.60382926979720 0.12874742662049 0.68845903945440
0.72430852843778 -0.01736930100181 0.28117234366010
0.13842814590132 -0.04408825393080 -0.24984642432883
-0.24229488706639 0.01398102791740 -0.19594627437823
-0.03224486958464 0.00874609404741 0.12736934033608
0.07757149384005 -0.00487035299345 0.09305736460388
-0.00624149021280 -0.00039174037338 -0.07139414716645
-0.01258075199908 0.00067544940645 -0.02945753682182
0.00333572528547 -0.00011747678412 0.03321267405949
N=6 0.00360655356698
0.11154074335011 -0.01073317548335
0.49462389039846 0.00139535174706
0.75113390802111 0.00199240529519
0.31525035170919 -0.00068585669496
-0.22626469396545 -0.00011646685513
-0.12976686756727 0.00009358867032
0.09750160558732 -0.00001326420289
0.02752286553031
-0.03158203931749
0.00055384220116
0.00477725751095
-0.00107730108531
Table I: Scaling function coefficients for the Daubchies family of wavelets
(Daubechies 1988).
30
2 2
O+------------:-=--==---::::::II'-=~_
3
o........__~_#_........_-__--
5 678 9
-1 -1
(a) (b)
2
4 8 4) 24 2B
5 10 15
o ~-_o\__+_~--------
-1 -1
(c) (d)
Figure 13: Scaling functions associated with the Daubechies wavelets for
(a) order = 2, (b) order = 5, (c) order = 8 and (d) order = 15.
31
2
-2
(a)
-2
(c)
(b)
-1
(d)
31
Figure 14: Daubechies wavelets for (a) order = 2, (b) order = 5, (c) order = 8 and (d)
order = 15.
32
Computing the decomposition coefficients: Signal filtering is mathematically
performed using convolution.· Signals are convolved with the filter coefficients (in this
case, with the scaling function and the wavelet coefficients) and the resultant product
signal dyadically sampled. If f is the process signal, H and G the high pass filters
(scaling functions) and the bandpass filters (wavelets) respectively, then the signal is
decomposed as follows.
Blurred Signal=H*f
Detail Signal=G*f
where "*,, represents the convolution operation followed by downsampling in which
every alternate value is retained. The detail signal contains infonnation in the original
signal which is missing in the blurred signal. There is no loss of information if the
transformation is orthogonal. The original signal can be reconstructed from any level by
simply reversing this operation.
H'I\(Blurred Signal)+G'I\(Detail Signal) = Original Signal.
H' and G' are transpose of H and G. In this step, however, the "1\" represents an
upsampling operation. Upsampling is a doubling of the signal length by insertion of
zeros between each value and convolving with the filter coefficients as described earlier.
Figure 15 best represents this whole procedure.
-®-G-l~ Reconstructed
Signal
~-----I""
Original
Signal
Figure 15: A basic decomposition and reconstruction representation.
33
Consider a signal!at its o!iginal resolution, having n samples and whose time domain
representation is vector aO. At this resolution the elements of the vector aO are the
sampled values ofthe signal itself. The decomposition coefficients generated at the first
level of decomposition are given by
j =1, ... ,!!.-, k =1, ... ,n
2
j =1, ... ,!!.-, k =1, ... ,n,
2
(34a)
(34b)
where the Ck and dk are the scaling function and wavelet coefficients respectively
(computed according to the procedure in the previous section). The 1/2s before the
summations are normalization constants. At any level the signal decomposition
coefficients a and b are computed by recursion from the results at the previous level. The
sequence a~ can be considered as an averaged version of the original signal, but on a
scale twice as large. Equations (34a) and (34b) are equivalent to convolving the signal
with the respective filter coefficients and downsampling them by a factor oftwo. On the
other hand, sequence b~ contains the difference in information between the signal a~ and
the signal a~, i.e., the information present in the original signal butfiltered out in the
averaged version.
A sample illustration of this decomposition procedure is provided using the Haar
wavelet and a short signal. The Haar family is chosen for simplicity. Consider a signal
vector a~ represented by four sampled values [a1 a2 a3 a4}. Let the scaling function
coefficients be [co ell and the wavelet coefficients be [do dll. The scaling function and
wavelet coefficients are [1 i] and [1 -ii, respectively. Recursion using equations (34a
and 34b) results in
1 1 0 0 al = -Cal CI +a2cO]
2
1 1 0 0 a2 = -[a3cI +a4cO]
2
where a l = [a: ~]. Similarly hI = [bi
l b~] is calculated using equation (34b).
From the decomposition coefficients, it is possible to reconstruct the signal. The
recursion is run in reverse as follows:
a~ = La~C2j-k + Lb~d2j-k k = 1, ...,n.
j j
Consider a signal aO =[2 4 6 8]. From equations (34c) and (34d) we get the
decomposition coefficients:
Reconstruction is performed using equation (35),
af = [lx3 + Ox7] +[lx -1 + Ox -1] = 2 ,
ai =[lx3+0x7]+[-lx-1+0x-1]=4,
a~ = [Ox3 + 1x7] + [Ox -1 +1x -1] =6 , and
a~ =[Ox3+1x7]+[Ox-l+-1x-1]=8
and provides the original signal aD = [2 4 6 8].
34
(34c)
(34d)
(35)
All these operations can be generalized to handle filters of any order and signals of
any number of samples.
35
Multi-Resolution Analysis: The previous example illustrated computing
decomposition coefficients fot the first level and then reconstructing the signal from those
coefficients. However, it is possible to decompose the signal further. Coefficients from
the first level of decomposition can be decomposed repeatedly to further smooth the
signal.
This method of viewing a signal at multiple resolutions is called the Multi-Resolution
Analysis (MRA) [Cohen et al., 1992b; Daubechies, 1991; Mallat, 1989a; MalIat, 1989b;
Mallat, 1989c]. This method provides an excellent tool for feature extraction and pattern
recognition. Compact representation of the trends in the original sensor signal can be
obtained from decomposition results many levels down. However, dropping down too
many decomposition levels yields an overly smoothed trend, so a optimum level must be
selected that gives the most compact representation without sacrificing important trend
information. The basic representation ofMRA is given in Figure 16.
Figure. 16: Depiction of the MRA algorithm.
Figure 17 shows the decomposition of a signal down three levels. It is evident that the
signal at any level is the sum of the blurred and detail signals at the previous level.
36
A wavelet toolbox for analyzing sensor signals was developed during the course of
this work here at Oklahoma State University, using MATLAB. Program daub.m
generates the Daubechies wavelet coefficients for the order specified. File scale2.m
computes the scaling function and wavelet and plots them. Programs/wI.m and ifwt.m do
the decomposition and reconstruction sequences. All the figures in this chapter were
generated using this very user-friendly toolbox.
0.80
0.78
0.76
original signal u
37
0.80
0.78
0.76
0'004~ 0.002
o A A A,A A A Af\~/\ AA _
tV '\J ... vV V\I\f <ry~~
-0.002
blurred and detail signals at the first level
U
0.80
0.78
0.76
0'004~ 0.002
o Aft Area c> - /\. M/\ v OS va v~~v~
-0.002
blurred and detail signals at the second level u
0.80
0.78
0.76~__
blurred and detail signals at the third level
Figure 17: Representation of a signal decomposition up to three levels
38
Chapter Summary
This chapter addressed the important mathematical aspects of wavelets and wavelet
transforms in general and the implementation ofwavelet transforms. Emphasis was
placed on Daubechies wavelets because that wavelet family was used for the results
shown in the next chapter. Not only is this family the most well documented compactly
supported wavelet available, but it also exhibits good trend extraction abilities.
The main objective ofthis chapter was to present wavelets and wavelet transforms in
a simplified manner stressing more applied methods and less theoretical details. The next
chapter deals with some critical issues in wavelet transfonns and their application for
trend extraction.
CHAPTER IV
SIGNAL EXTENSION
Need for signal extension for sensor signals from chemical processes
For computational reasons, process monitoring techniques generally require
compact representations of process signals. Raw trend patterns are not preferred
for these applications. Therefore, signal processing methods, like wavelet
transforms, are used for obtaining a representation more suitable for monitoring
purposes. Most signal processing techniques adopt convolution operations for
signal analysis and smoothing. An inherent tendency of convolution is to distort
finite length signals at the boundaries. For pattern recognition applications, this is
unacceptable. It is imperative that signal trends remain unaffected and retain
critical features. Signal extension is employed to overcome distortion at the
boundaries.
This chapter highlights the inadequacies of common signal extension methods
and the development of a new signal extension method.
39
Common signal extension methods
Common signal extension methods include circular padding or periodic
extension, symmetric extension, extension with zeros, and extension with constant
boundary value.
Circular Extension: The signal is assumed to be periodic (Figure 18) i.e., the
signal is assumed to repeat itself with a period equal to its length.
40
(t)
4
-4 -3 -2 -1 0 1 2. 3 4 5 6 7 8 9 10 11 time
J(t)
2
extension
( )1
extension
f )
I
I
(~ (b)
Figure 18: Periodic extension technique, (a) original signal and (b) the extended
signal
This method works fine for cases when signals are steady over a period of time.
The disadvantage ofperiodic extension occurs when the points at the extremities of
the signal differ significantly as shown in Figure 19. The signal in Figure 19a is
decomposed using wavelet transforms and the resultant signal is depicted in Figure
19b. The wavelet decomposition was generated using the sixth order Daubechies
wavelet and shows the signal reconstructed from 9th decomposition level.
0.89 Q~
0.88 Q~
0.87 Q87
0.86 Qas
0.85 QSS
0.84 Q8:t
41
W 00
Figure 19: (a) original signal (b) wavelet decomposition using periodic extension.
The periodic extension introduces sharp differences at the boundaries. For
instance, in Figure 19, the original signal shows a step change that introduces a
significant difference in the sensor value between the two extremities. The
inherent nature of the periodic extension averages out signal trends at the
boundaries. Therefore, this method is inadequate for our purposes because it
seriously distorts trends.
Symmetric Extension: Figure 20 depicts a signal that is symmetrically
extended. The signal is assumed to be symmetric about the boundary sample on
either end.
(t) I(t) extension
6 ( ),
4
extension
I( )
-4 -3 -2 -1 0 1 lirJe 4 5 6 7 8 9 10 11
(~ (b)
Figure 20: Symmetric extension technique, (a) original signal and (b) the
extended signal
Figure 21 depicts a signal with sudden trend changes at the boundaries. The
sensor shows a steady value near the boundaries, but at the boundary itself, there is
a marked change in the signal behavior.
47
0.89!
0.88 ~~
0.87
0.86
0.85
0.84.l.1..-
-----------
0.89
0.88
0.87
0.86
0.85
0.84 .1...- _
(a) (b)
Figure 21: (a) Original signal (b) wavelet decomposition using symmetric
extension. Note distortion of trend at the boundaries.
In signals where sharp trends (upward or downward) begin to at the boundary,
this signal extension method fails. Symmetric extension flattens the trend at the
ends and provides an erroneous representation. This extension procedure is
therefore inadequate for wavelet transforms. An interesting observation is that
both circular and symmetric extension methods work well for Fourier transforms
and other transforms.
Padding with zeros: In this signal extension procedure, the signal is extended
with zeros as illustrated in Fi ure 22.
(t)
2
-4 -3 -2 -1 0 1 ~. 3 4
tIme
(a)
5 6 7 8 9 10 11
j(t)
4
2
extension
< )1
extension
1< )
10 11
(b)
43
Figure 22: Extension with zeros, (a) original signal and (b) the extended signal
This-method fails totally as is illustrated in figure 23.
0.89 1.2
0.88
0.87
0.86
0.8
0.85 0.6
0.84 t 0.4
0.83 I 0.2 1
(a) (b)
Figure 23: (a) original signal (b) wavelet decomposition using zero padding.
The two plots are ofdifferent scale to illustrate the extent ofdistortion of
trend at the boundaries.
It is evident that the zero extension method results in totally misleading trend
patterns. The zero extension method is also inadequate for trend retention.
Padding with a constant value: This method is similar to the zero extension
method, except that the signal is extended to the required length with the boundary
value at either end. This technique is illustrated in Figure 24.
(t) J(t) extension
< )1
44
2
-4 -3 -2 -1 0 1 ~ . 3 4 5 6 7 8 9 10 11 tzme
4
9 10 11
(a) (b)
Figure 24: Extension with the boundary value, (a) original signal and (b) the
extended signal.
Figure 25 shows a signal and the wavelet decomposition when extended using
this method.
0.89 0.89
0.88 0.88
0.87 0.87
0.86 0.86
0.85
1
0.85
0.84 0.84
(~ ~)
Figure 25: (a) original signal (b) wavelet decomposition using a constant
extension method. Note distortion oftrend at the boundaries.
In this case also, signal trends are distorted at the boundaries. The decomposed
signal shows a smooth trend at the boundaries, contrary to the swings in the
original signal.
Summary of conventional signal extension methods
The signal extension methods discussed above are rigid in that they do not take
the signal trend patterns into consideration while extending the signal. Stated
simply, they are not adaptive extension methods and employ the same extension
procedure irrespective of the nature of the signal. A better signal extension
technique would be one which adapts itself to suit the specific signal. A signal
extension technique that best approximates signal behavior towards the boundaries
would provide the best representation.
The New Extension Technique (NET): The basic idea behind the development of
this approach was to provide a technique that would provide an accurate wavelet
decomposition irrespective of the nature of the signaL Unlike the previously
described extension methods, the objective ofthis technique was to provide a
reliable extension for all cases.
This method uses a statistical approach to provide a good approximation of the
signal outside the boundaries of the signal depending on signal trends at the
boundaries. Different statistical approaches were adopted for this purpose and four
new extension methods are described in this work.
The concept behind these methods is the same. Signal samples close to the
boundary are considered and a mean value is determined. The procedure for
detennining this "mean value" differs for each of these four methods and each is
described in the following sections. The signal is then extended by making it
symmetric it with respect to that mean value and then inverting it.
45
NETt: Consider a signal represented by a vector a=[ao aj a2 a3 a4 aj a6 aJ.
A threshold number of samples K is specified by the user. For illustrative purposes
consider K= 1O.
Starting from the first value at the left boundary, ao, this method calculates the
mean
where fo is the mean of the first sample. The swn ofthe mean squared deviation
(Mo) of sample ao from its meanfo is calculated next.
Mo=(ao-f,)2=0.
Then, the first two samples from the boundary are considered and the mean
calculated
The corresponding mean squared deviation of these samples from the mean is
calculated
M
J
= (ao - h/ +(aJ - h)2 .
2
In this way, the procedure is repeated up to K specified samples, the mean,/;, and
the mean square deviation from the mean, M;, is computed at each step using the
following equations :
46
;
La)
f =)=-0 - 1• = 0 K - 1
i i +1 ' ,
i
L(a) - fJ2
M. = )==0 , i =0, K -1 .
I i +1
(36)
(37)
A vector ofmeans/=ljj h 13 14 15 16 17 f8 19 fjo··············fK-l}, and a vector
ofmean square deviations from the mean
minimum. Ai; is considered and the signal is flipped around the corresponding};.
When K=10, and M4 is the minimum mean square deviation, the signal is flipped
around14, the mean of the first 4 samples from the boundary. A similar procedure
is adopted for the right boundary. In the NET! method, M] is always zero, so the
signal is basically flipped around the boundary sample on either side.
The logic behind this approach is that signal trends are approximated well
when the signal extension is close to the actual signal trend. The minimum value
of mean square deviation is chosen because it is at this value that the sensor signal
is more or less representing its basic trend, and deviations are minimum. The
signal is flipped and not symmetrically extended because flipping the signal
provides a smoother transition across the boundaries and trends are preserved.
Figure 26 shows the trend preservation capability of this technique. The left side
ofthe signal follows the downward trend and the right side provides a smooth
upward trend.
47
(t)
-4 -3 -2 ~1 0 1 ~. 3 4 5 6 7 8 9 10 11 tzme
f(t) extension
( )1
o 1
i
~. 3 4 5
tlme
6 7 8 9 10 11
I
(a) (b)
Figure 26: The New Extension Technique, (a) original signal and (b) the extended
signal
NET2: In this technique, the first 40% ofthe threshold (K) samples are neglected
while computing the mean square deviation from the mean. When K= 10 as in the
previous case, 40% of 10 samples or 4 samples are neglected while computing the
minimum M;. In equation (36) and equation (37), the index i would vary from
O.4K to K-l and not 0 to K-l. The vector for computing the minimum Mi. would
now be M=[MO.
4K MO.
4K+1 MO.4K+2 MK-I]. The signal is flipped as in the
earlier case around the mean corresponding to the minimwn M.
The first few samples are neglected while computing the mean to flip the signal
around because in NETl, the signal is always flipped around the boundary value.
This boundary value could contain an unusual amount of noise, and flipping the
signal around this point could provide a poor extension.
NET3: The vector corresponding to the meanf=[fj h 13 14 15 16 f7 18 19
hO !K-l}, and the vector of the sum ofthe mean square deviations from the
mean
M=[M1 M2 M3 M4 Mj M6 M7 M8 M9 M1o. MK-1], are generated as
described in the NETl method. However, in this method, the maximum value of
the mean square deviation (not the minimum value as in NETl) is considered and
the signal flipped with respect to the corresponding mean. For instance, in vector
M, ifM6 corresponds the maximum value, the signal is flipped around the mean16.
This method is tailored for signals containing a significant amount of noise. By
considering the maximum mean square deviation from the mean, the signal is
flipped about a point such that the extended signal fluctuates to its maximum
possible extent about the mean trend value. Thus the averaged smoothed version
would provide a better representation of the actual trend.
NET4: This method is a hybrid of the NET2 and NET3 techniques. The first 40%
of the threshold number of samples specified are neglected while computing the
48
mean around which the signal is f~ :.pped. Analogous to the NET3 method, the
maximum mean square deviation from the mean is used.
Figure 27 illustrates the ways a signal can be extended using the techniques
mentioned above. Figure 27(a) shows the original signal. The windows indicate
the length to which the signal is extended on either side. Figure 27(b) illustrates
the periodic extension method. The signal is extended on the left by using samples
in the right window in the original signal; on the right side the signal is extended
using samples in the left window. The constant extension method in Figure 27(c)
shows extension using the boundary value on either side. Symmetric extension in
Figure 27(d) shows the signal flipped with respect to the boundary value. Figures
27(e)-(t) show the NET extension methods, the difference is explicitly represented.
49
Figure 27(a). The original signal, with the left and right hand side sides
LHS of the Signal
50
RHS of the Signal
: : :'.:" :... .."
~ '. '". '.::: ........... :. "~'..:"..."' :" ..;" ..... :'
Figure 27(b). Periodic extension on the left and right hand sides
LHS of the Signal
RHS of the Signal
Figure 27(c). Constant extension method on the left and right hand sides
LHS of the Signal
51
v···:\\.·:··· ..
.,
:". :.:.•.:....
...:.\.:'..~:.:::.•..../...•.:,:..: ..
"
RHS of the Signal
..::/\:-:..•..:
......... :.........
'. \ ::".;'.
:: .:".....
Figure 27(d). Symmetric extension on the left and right hand sides.
LHS of the Signal
52
6 •• ".. •
f-.~ .---.~.:.~ ..../~ ;'..~ -...-f~~· .~'At ••• ~.~',,:'~::''''.:.:.~~.?~ ~ ••••• -./'
'. ':/,,: ".:.: \......... '. .
RHS of the Signal
Figure 27(e). The left and right hand sides extended using the NETl method
LHS of the Signal
RHS of the Signal
Figure 27(f). The left and right hand sides extended using the NET2 method
LHS of the Signal
RHS of the Signal
........:..... .'. f'.: t.:· .:.
•• ••• •. .•• ;••. :..~...••..\~.-: ,~.~ .•. ~ .• ~.~ ~.:.. ~.•':;.•?'~ !.~.:. ; ;..,;: '..
..\/ :,; .,...••:
Figure 27(g). The left and right hand sides extended using the NET3 method
LHS of the Signal
'.'"'~ .,"::";;:.:..";:':\l.~Ll :,"~"..::"~\."Y::~\l~L~","~ .·.;.\. ..·
.. '
RHS of the Signal
.....~ :....':.. .. ,
.. ..... .·.!·.. :.~"~ !."~ ~". 'V"~"; t..~, ;~"..'"~"..~"". ~;. ~~L ..~ / .
Figure 27(h). The left and right hand sides extended using the NETl method
Figure 27. The different extension methods
53
Demonstration ofNET technique
To compare these techniques, a set of test signals were sliced into two halfsignals,
decomposed, reconstructed separately, and reconnected. If this technique
works perfectly, the reconstructed signals should match perfectly the original
signals. Three process signals were used to test signal extension effectiveness.
Case I: A signal with normal swings is considered (Figure 28). This signal is
sliced at the crest of one of its swings. This signal is ideal for symmetric extension
because the signal is symmetric at the midpoint.
In Figure 28b, the periodic extension method tends to distort the trends of the
left slice by raising the pattern above its actual value on the left side of the slice;
and a reverse effect occurs on the right side of the slice. Similar pattern distortions
are included on the right side of the slice. As a consequence, the boundaries trends
to fall either above or below their actual values. When combined, these signals do
not provide a smooth transition. The periodic extension method performs poorly,
as expected.
The symmetric extension (Figure 28c) performs well in this case due to the
nature of the signal. Although this technique provides good representation for the
left slice and the left side of the right slice, it smoothes off the right side ofthe
right slice. A flat trend is produced instead of the decrease as in the original case.
This distortion is unacceptable for the intended monitoring applications.
The NETI method is implemented by specifying the threshold number of
samples as 1.25% of the total number of samples. Signals in all these test cases
each contained 4096 samples, so the domain of search for the mean to flip the
signal around was approximately 50 samples. The same threshold is specified for
all the NET techniques. Figure 28d shows distortion on left side of the left slice
54
mainly because there is a high level of noise at the point. Instead oftaking the true
trend into consideration only signal behavior at the ends is considered.
Consequently, the wavelet decomposition shows a sudden drop on the left side
when the true signal trend is flat.
NET2 (Figure 28e) however does a good job of retaining all the trends. In the
NET2 method, 40% of the 50 threshold samples or 20 samples are not considered
while computing the mean.
NET3 does not provide accurate signal reproduction on the right side of the left
slice and the left side of the right slice, resulting in a rough transition at the
midpoint in Figure 28f. This technique also shows a slightly rising trend on the
left side of the first slice which is contrary to the flat trend in the original signal.
NET4 (Figure 28g) provides an adequate reconstruction, but on the left side of
the left hand slice it shows a slightly decreasing trend instead of a flat trend.
For this case, the NET2 was most satisfactory. Only this method retained the
essential trend at all the boundaries and provided good reconnection of the two
slices.
55
(a)
(b)
(c)
(d)
Figure 28: Representation of effect of signal extension on signal
combination, (a) original signal (b) using the peridoic extension
(c) using the symmetric extension and (d) the New Extension
Technique, NET1(threshold is 1.25% of the total signal length)
56
(e)
(f)
(g)
Figure 28: (contd) (e). using NET2 (f) using NET3 and
(g) using NET4
57
Case II: A signal with different characteristics is considered in this case (Figure
29). This signal is split into tWo parts. The first part of the signal has ends that
differ significantly. The second part is sliced at points where there are marked
changes in the direction of trends. These are analyzed using the afore mentioned
techniques and recombined.
Figure 29b shows the periodic extension case. The left boundary of the left
slice is significantly above the right boundary, so the trend on the left side ofthe
slice shows a more marked downward change than there actually is. The right side
of this slice originally is at a steady value, but is now distorted to indicate a sharp
upward change. The direction of the trend on the right side ofthe right slice is
reversed. The periodic extension performs poorly, as expected.
The symmetric extension in Figure 29c performs even worse than the periodic
extension method. The left side ofthe left side does not reflect the gradual rise in
the original signal, but shows a sharp rise. The right side of the this slice
approaches a constant value when it actually should show a rise. The right slice
shows a more gradual drop on the left side of the slice than it actually should. The
right side ofthis slice starts to show a slight rise, where there is none in the original
signal. Once again this method is inadequate for our purposes.
The NET! method in Figure 29d provides excellent trend representation on
both sides ofthe right and the left slices.
The right side ofthe left slice in NET2 in Figure 2ge does not rise as sharply as
it should. When the two slices are reconnected, the representation is marginally
poorer than in the NET! case.
NET3 (Figure 29t) also performs poorly on the right side of the left slice. The
trend levels off to a greater degree than it actually should, so this method fails for
this case.
NET4, shown in Figure 29g, provides a representation comparable to NET2.
58
Figure 29: Representation of effect of signal extension on signal
combination, (a) original signal (b) using the periodic
extension (c) using the symmetric extension and
(d) The New Extension Technique
59
Figure 29 (contd) (e) using NET2 (t) using NET3 and
(g) using NET4
60
For Case 2, NET2 and NET1 provide the best representation. The reason
NET! provides a marginally better representation than NET! is that the trend on
the right hand side ofthe left slice is guided more by samples at the boundary.
Since NET2 does not take the immediate vicinity ofthe boundary into
consideration, it fails to replicate the steep trend perfectly but still provides a good
representation.
CASE III: The right boundary ofthe process signal in Figure 30 is actually a
stray sample, away from the actual process trend. The signal is sliced into half at
its midpoint, where the right side ofthe left slice shows an upward trend.
Since the right and left sides ofthe left slice are not at identical levels, the left
side shows a flat trend when the actual trend is rising. The right hand side ofthis
slice gradually levels off, when actually, it should show a constantly rising trend.
The boundaries ofthe right hand slice are at identical levels, so no significant trend
distortion is noticed (Figure 30b) on this side.
The symmetric extension method (Figure 30c) fails in this case. The left side
of the left slice shows a more gradual rise than exists in the original signal. The
right side of the slice flattens when it should actually shown an increasing trend.
The left side ofthe right slice shows a marginal drop when actually there is a steep
one. Its right side also flattens, when it should show a marginal rise.
The NETl technique shown in Figure 30d shows perfect trend reconstruction
on both sides of the left slice and on the left side ofthe right slice. However, on
the right side ofthe right slice the influence of the stray boundary sample reflects a
sharper trend than there actually is.
Figure 30e illustrates the NET2 extension method. On the left and right sides
of the left slice, trends are represented well. However, at the point of reconnection
61
(a)
(b)
(c)
(d)
Figure 30: Representation of effect of signal extension on signal
combination, (a) original signal (4096 samples) (b) using the
periodic extension (c) using the symmetric extension and
(d) The New Extension Technique, NETl
62
(e)
(f)
(g)
Figure 30: (contd) (e) using NET2 (f) using NET3 and
(g) using NET4
63
ofthese slices, it performs marginally poorer. This method provides a more
accurate representation than NET! on the right side of the right hand slice, because
this method ignores samples at the immediate boundaries of the signal.
NET3 (Figure 30t) and NET4 (Figure 30g) provide identical representations
because the mean around which the signal is flipped is identical in both cases.
They provide good trend representation on both sides of both slices, although at the
midpoint they provide a marginally poorer reconnection than NET! or the NET2
methods.
Considering the fact that the signal extension technique must provide good
signal representation and trend retention capability for any signal, it recommended
that the NET2 technique be used for compact representation of sensor data used for
monitoring and control purposes. NET2 has proved to be a good signal extension
method for many cases similar to the test cases presented here.
64
CHAPTER V
CONCLUSIONS
Though wavelets are new, they have contributed much to signal processing. Most
importantly, they provide a much needed alternative to Fourier transforms for certain
applications such as pattern based monitoring and control. I have attempted to present
the applied aspects of wavelets to the process engineer.
The usefulness of wavelet transforms has been compared and contrasted to Fourier
transforms. Effort has been made to provide a technique to extract essential trends from
process signals and provide a compact representation. The effectiveness of a signal
processing technique depends to a large extent on the nature of the signals involved. One
technique that works for specific signal trends might not be effective in dealing with other
signal trends. In the pre-processing stage, signal extension has been identified as the
critical factor influencing signal representation and retention of trends.
Signal extension is especially important when the application is real-time process
monitoring. The signal analysis technique should provide an accurate yet compact
representation of the process trend. The ability of a pattern recognition technique to
recognize abnormal trends depends on how efficiently the signal is represented and to
what extent trend losses are minimized. Trends towards the boundaries are generally
indicative of process condition changes and these patterns flag the monitoring technique
to detect the change. Thus end effects thus have to be minimized for this technique to be
successful.
65
66
Some conclusions can also be made about the signal extension techniques when used
with wavelet transforms. Periodic extension is simple to implement, but causes serious
edge distortions as illustrated in the test cases discussed in the previous chapter. This
signal extension technique is completely inappropriate for our needs.
On the other hand, making the signal symmetric at its ends can help minimize the
problem of end distortions when the boundaries differ significantly. However, this signal
extension technique has a tendency to smooth and flatten trends at the boundaries
especially when a process change is just occurring and the direction of the trend is
changing. In this case, this method results in a trend that erroneously shows a steady
process behavior, masking the trend. This is also unacceptable.
Zero padding and the boundary value extension methods are also inappropriate for
our application. Zero padding always results in sharp boundary distortions. while the
constant extension method results in the flattening of trends at the boundaries leading to
erroneous trend representations.
The NET techniques provide better signal extension than the conventional methods.
Their adaptability to signals make them a better proposition for monitoring applications
than general techniques like symmetric extension methods.
I have also concluded that NET2 is the best signal extension technique to use in our
applications. NET3 and NET4 perform reasonably well in some cases, especially when
the signal shows sharp deviations towards the boundaries. The NETt method performs
well in cases that do not exhibit sharp unusual deviations at the ends of the signal. This
work recommends the use ofNET2 because it provides reasonably accurate
reconstruction under any circumstance.
67
Recommendations and Future work
Future work is required to provide a more theoretical basis to the empirical approach
adopted here. The results presented in this work employed parameters based purely on
experience and knowledge ofthe sensor signal behavior. A more generalized technique
with a mathematical basis needs to be developed.
Some recommendations are
• More wavelet families need to be studied. This work focused mainly on the
Daubechies family of wavelets. Other wavelet families are available and can be used
for this purpose. A generalized technique could be developed that determines the
most appropriate wavelet family depending on the signal or application.
• At present, the order of wavelet is empirically determined. An automated technique
could be developed that takes into account the trend pattern and determines the
appropriate order ofthe wavelet.
• The level of decomposition used for trend pattern extraction is also empirically
determined at present. The optimum decomposition level should to be determined
and an automated procedure developed.
• To provide better decomposition, the wavelet order could be adaptively modified with
the level of decomposition. At the initial stages of decomposition, higher order
wavelets can be used. Further down the decomposition tree, lower order wavelets
could be applied. This would further minimize distortions due to convolution.
• Currently, the NET technique employs mean square deviation of the signal from the
cumulative mean to determine the point with respect to which the signal is extended
symmetrically and inverted. A more rigorous statistical technique could provide
better signal extension by approximating future samples more accurately. Not only
can signal end distortions be avoided this way, but smoother transitions across slices
can also be obtained.
68
• Wavelet packets [Motard and Joseph, 1994] could be investigated to check their
performance with the norrilal decomposition procedures. In the wavelet packet
procedure, the detail signal at each level is decomposed further, just like the blurred
signal, into two components. After the decomposition is carried to the lowest level,
coefficients with the maximum entropy are retained and the remaining deleted. This
way data compression is achieved, and signal reconstruction is also accomplished. It
needs to be verified though if the reconstruction quality is comparable to the scheme
adopted here. The disadvantage is that location ofthe non-zero coefficients need to be
known.
• Instead of using regular decomposition methods, zero crossings of wavelets [Mallat,
1991] can be used to decompose signals. This technique is translation invariant.
BIBLIOGRAPHY
Akansu, A. N. and R. A. Haddad. Multiresolution Signal Decomposition:
Transforms, Subbands and Wavelets. Academic Press, Inc., San Diego, CA (1993).
Akansu, A. N., R. A. Haddad and H. Caglar. The binomial QMF -wavelet transform for
multiresolution signal decomposition. IEEE Transactions on Signal Processing 41, 1319
(1993).
Alpert, B. K. Wavelets and Other Bases for Fast Numerical Linear Algebra.
Wavelets: A Tutorial in Theory and Applications. (C. K. Chui, Eds). Academic
Press, Inc, San Diego (1992).
Battle, G. A block spin construction of ondelettes. part I: lemarie functions.
Communications on Mathematical Physics 110, 601-615 (1987).
Barnard, H. J., J. H. Weber and J. Biemond. Efficient signal extension for
subband/wavelet decomposition of arbitrary length signal. SPIE Conference on Visual
Communications and Image Processing, , (1993).
Bracewell, R. M. The Fourier Transform and Its Applications. McGraw Hill Book
Company, (1965).
69
70
Brigham, O. E. The Fast Fourier Transform. Prenctice-Hall, Inc., Eaglewood Cliffs,
N.J. (1974).
Chui, C. K. and J.-Z. Wang. A cardinal spline approach to wavelets. Proceedings ofthe
American Mathematical Society 113, 785-793 (1991).
Chui, C. K. and J.-Z. Wang. On compactly suppported spline wavelets and a duality
principle. Transactions ofthe American Mathematical Society 330, 903-915 (1992).
Chui, C. K. An Introduction to Wavelets. Academic Press,Inc, College Station, Texas
(1992a).
Chui, C. K. Wavelets:A Tutorial in Theory and Applications. Academic Press,Inc.,
College Station,Texas (1992b).
Chui, C. K. and J.-Z. Wang. An analysis of cardinal spline wavelets. Journal of
Approximation Theory 72, 54-68 (1993).
Cohen, A., I. Daubechies and J. C. Feauveau. Biorthononnal bases of compactly
supported wavelets. Communications on Pure and Applied Mathematics 45, 485-560
(1992a).
Cohen, A., I. Daubechies, B. Jawerth and P. ViaL Multiresolution Analysis, Wavelets
and Fast Algorithms on an Interval. C.R Acad Sci Paris 316,417-421 (1 992b).
71
Cohen, A., I. Daubechies, B. Jawerth and P. Vial. Multiresolution Analysis,
Wavelets and Fast Algorithms on an Interval. C.R Acad Sci Paris 316, 417-421
(1992c).
Combes, J. M., A. Grossmann and P. Tchamitchian. Wavelets:Time-frequency methods
and phase space. Wavelets: Time-frequency methods andphase space
Dec 14-18,1987, Marseille,France, (1987).
Daubechies, I. The wavelet transform, time -frequency localization and signal analysis.
IEEE Transactions on Information Theory 36, 961-1005 (1990).
Daubechies, I. The wavelet transform: a method for time-frequency localization.
Advances in Spectrum Analysis and Array Processing. (S. Haykin, Eds). Prentice Hall,
Eaglewood Cliffs,New Jersey (1991).
Daubechies, I. Ten Lectures on Wavelets. Society for Industrial and Applied
Mathematics, Philadelphia (1992).
Daubechies, I. Orthonormal bases of compactly supported wavelets. II variations on a
theme. SIAMJournal ofMathematical Analysis 24,499-519 (1993).
Daubechies, I. and J. C. Lagarias. Two-scale difference equations
I. existence and global regularity of solutions. SIAMJournal on Mathematical Analysis
22, 1388-1410 (1992a).
72
Daubechies, I. and J. C. Lagarias. Two-scale difference equations
II. local regularity, infinite products of matrices and fractals. SIAMJournal on
Mathematical Analysis 23, 1031-1079 (1992b).
Haykin, S. Advances in Spectrum Analysis and Array Processing. Prentice Hall,
EaglewQod Cliffs, NJ (1991).
Lemarie, P. G. Les Ondelettes en 1989. Springer-Verlag, Berlin (1990).
Lemarie, P. P.-G. Ondelettes a localisation exponentielle. Journal de mathematiques
pures et appliquees 67,227-236 (1988).
Mallat~ ."., G. Multifrequency channel decomposition of images and wavelet models.
IEEE Transactions on Acoustics ,Speech and Signal Processing 37, 2091-2110 (1989a).
Mallat, S. G. Multiresolution approximations and wavelet orthonormal bases ofL2(R).
Transactions ofthe American Mathematical Society 315, 69-87 (1989b).
Mallat, S. G. A theory for multiresolution signal decomposition: the wavelet
representation. IEEE Trans. Pattern Anal. Mach. Intellig. 2, 674-693 (1989c).
MalIat, S. Zero-crossing of a wavelet transform. IEEE Transactions on Information
Theory 37, 1019-1033 (1991).
Meyer, Y. Principe d'incertitude, bases hilbertinnes et algebres d'operateurs. Seminaire
Bourbaki,1985-1986, no. 662. Eds). Societe Mathematique de France, (1985).
73
Motard, R. L. and B. Joseph. Wavelet Applications in Chemical Engineering. Kluwer
Academic Publishers, Norwell, MA (1994).
Oslen, P. A. and K. Seip. A note on irregular discrete wavelet transform. IEEE
Transactions on Information Theory 38, 861-863 (1992).
Papoulis, A. Signal Analysis. McGraw Hill, (1977).
Rice, J. R. The Approximation of Functions. Addison-Wesley Publishing Company,
Warren,Michigan (1964).
Rioul, O. and M. Vetterli. Wavelets and signal processing. IEEE SP Magazine 14-35
(1991 ).
Ruskai, M. B. Wavelets and their applications. Jones and Barlett publishers, Boston
(1992).
Smith, M. J. T. and S. L. Eddins. Analysis/synthesis techniques for subband image
coding. IEEE Transactions on Acoustics ,Speech and Signal Processing 38, 1446-1456
(1990).
Strang, G. Wavelets and dilation equations : A brief introduction. Society for Industrial
and Applied Mathematics 31, 614-627 (1989).
Strang, G. Wavelet transforms versus fourier transforms. Bulletin ofthe American
Mathematical Society 28, 288-305 (1993).
74
Strichartz, R. S. How to make wavelets. The American Mathematical Monthly 100,
539-556 (1993).
Walter, G. G. A sampling theorem for wavelet subspaces. IEEE Transactions on
Information Theory 38, 881-884 (1992).
Weaver, J. H. Applications ofDiscrete and Continuous Fourier Analysis. John Wiley &
Sons, (1983).
APPENDIXES
75
APPENDIX A
Computing the scaling function and wavelet coefficients
for the Daubechies family of wavelets
76
77
The Fourier transform of a functionf(t) is defined as
The Fourier transform of (6) gives,
<p(~) = t L cke-ik~/2<p(t)
k
D fi ( ~)- 1 ~ -ik~ elnemo ,", -IL.Jcke ,
k
(38)
(39)
(40)
(41)
equation (40) indicates that mo(~) is a function, a periodic function with a period 2n.
It also gives another set of boundary conditions, for ~ = 0 and. ~ =1t For ~ =0, equation
(40) gives
mO(O) =t Lcke-i.O.k =t Lck·{cos(O)- isin(O)}
k k
=tLck ·l=1
k
because LCk = 2. Similarly, for ~ =7t , (40) gives
k
moen) =tLcke-i
.1tk =tI Ck {cos(nk) - isin(nk)}
k k
=tIck·(-I)k =tIdk =0
k k
78
So,
(42)
The orthonormality of<p also yields the following relationship, explanation for this
can be found in p132 of [Daubechies, 1992].
~]p(s +27tZt =(27tt
I
substitute (41) into (43), the result is
(43)
(44)
Split (44) into odd and even terms. The following equation results from the LHS of
(44)
The fact that mo is periodic is used now to simplify the expression given above. The
simplification is of the form
(46)
(47)
79
Therefore the result is
Define z = e-il; , also define
rno(~) =t LCkZ
k
= P(z)
k
It is possible to define (48) in terms ofP(z) as
(48)
(49)
(50)
To form an orthonormal basis of wavelets, rnO is defined as in equation (51). The
mathematically sophisticated reader is referred to corollary 5.5.4 on pp. 155-156 in
[Daubechies, 1992].
mo(~)= P(z) = e;z) N E(z) (51)
where E(1) = 1, and E(z) is the Polynomial with real coefficients. The reason why
real coefficients are chosen is because real numbers are easier to deal with. Therefore
IE(e-iE, )12
is a cosine polynomial. Taking
(52)
where R is a polynomial, again with real coefficients. Take t=cos~ = sin2 ~
80
R(t) =R(cos~) =R(l- 2t) (53)
since Ip(z)12 +Ip( _z)1 2 = from (50), on substituting the value of P(z) from (51), the
result is
put t=cos~,
(I-t)N R(t)+tN R(I-t)=1
The general solution to an equation like (55) is
N-t(N-I+kl
R(t) =LI ,Ik + tN1:(t)
k=O f\. k.A
where 1:(1- t) = -'t (t) and 1:0(t) = 1: (1-221 •
(54)
(55)
(56)
For further mathematical details, the reader is directed to pp. 175-176 in [Chui,
1992a]. So,
I
-iE;, 12 - ~(N -1 + k\-;;. ~'ik t:- ~)2N cosE;,
E(e ) - 6k k A~m2) + ~m2 1: 0(-2-) (57)
~ 0 is an odd polynomial. Choosing ~ 0 equal to zero gives scaling functions with
minimum number of coefficients for any given N. Due to this assumption, (57) narrows
down to
I
-i~ 1
2 _ ~(N -l+k)",. ~)2k
E(e ) - L...J \sln"2
k=O k
(58)
81
the problem now is to solve (58) for E(e-iS ) • To do this the Riesz Lemma is used. A
proof of this lemma can be found in pp. 232-233 in [Chui, 1992a]. Once E(e-i~) is
calculated, then P(z) is calculated by equation (51) which then gives the values of the
coefficients of the scaling function.
Then the wavelet filter coefficients are calculated from the QMF relationship. This
procedure is elucidated for the construction of a wavelet of the Daubechies family.
I
-i~ 12 _ ~ ( 2 - 1+ k11> S'ik
E(e ) - L...Jl J~ln2J
k=O f\. k A
= +2sin2 t·
Now sin2
-} is -cos~ = 1-~+( . So,
The roots of this equation are z = 2 ±J3, the root within the unit circle is chosen.
Then IE(e-iS)1 is calculated as shown
Then from equation (51), P(z) is calculated as follows
P(z) =e;zr ±((~ -1)-(~ +1)
1(l+~ 3+~ 3-~ 2 l-~ =- + z+ z + z 3)
2 4 4 4 4
comparing with equation (49) the coefficients are obtained.
l+~ 3+~ 3-J3. _1-J3
Co = 4 ;c1 = 4 ;c2 = 4 ,c3 - -~-4-·
The wavelet coefficients are obtained by the QMF relationship, to give
d =1-J3' d = 3-J3' d =3+J3' d =_I+J3
o 4' 1 4' 2 4' 3 4
Similarly the coefficients can be calculated for higher orders of wavelets.
82
APPENDIXB
Computing signal decomposition coefficients
83
84
Consider an infinitely long signal. The signal is approximated as given in
equation (1).
OJ
fo(t) = ~>~<p~(t)
k=-ex>
OJ 00
=L a~<p~(t)+ Ib~\jI~(t)
k=-OJ k=-ex>
(59)
(60)
Here a~ is the vector that approximates the signal at the original resolution, a~ and b~ are
the blurred and detail coefficients at the first level.
Multiply both sides by <p1(t) and integrate,
co ex>
ai = L aZ f<p ~ (t)<p i (t) dt
k=-ex> -00
Similarly on multiplying by \fI ~ (t) and integrating
00 ex>
hi = IaZ f<p~(t)\jIi(t)dt
k=-«> -«>
ex> ex>
Since Ck = f<p ~ (t)<p i (t) dt and dk = f<p ~ (t)\jI i (t) dt, the resultant equation
(61)
(62)
-00 -ex>
(63)
For finite length sequences, the limits on the summation are also finite.
VITA
Vinod Kumar Raghavan
Candidate for the Degree of
Master of Science
Thesis: WAVELET REPRESENTATION OF SENSOR SIGNALS FOR
MONITORING AND CONTROL
Major Field: Chemical Engineering
Biographical:
Personal Data: Born in Visakhapatnam, Andhra Pradesh, India, February 1, 1971, the
son ofK.G.Srinivasa and Kamala Raghavan.
Education: Graduated from Keshav Memorial Junior College, Hyderabad, AP, India,
in May 1988; received Bachelor of Science Degree in Chemical Engineering from
Osmania University, Hyderabad, India, in May 1992; completed requirements for
the Master of Science degree at Oklahoma State University in May 1995.
Professional Experience: Teaching Assistant, School of Chemical Engineering,
Oklahoma State University, August 1992, to December 1992; Research Assistant,
School of Chemical Engineering, Oklahoma State University, January 1993, to
June 1994.