Lecture B01: Introduction
Although the final desired result may be as simple as
a Yes/No decision, the path to this result may involve looking through
a huge amount of information.
Example:
Qn: Is there Intelligent Life Out There? (See SETI:
Search for Extraterrestrial Intelligence)
Method: monitor frequency content of the fluctuations
in the data measued by several sensors, to see if there are identifiable
patterns which could only have been created by intelligent life. This involves
processing of a huge amount of data, after converting it to digital form
for access by the digital computers.
Qn: Is there another submarine in our vicinity?
To answer the above question, we would usually have to
capture the sound (pressure fluctuations) being felt at several sensors,
sample and digitize it. The digitization would have to be done with a fine
enough time interval (high sampling rate) to capture the high-frequency
signature of machinery, and must be captured for long enough to be able
to tell the direction and velocity of the generating objects, and to separate
out the tiny amount of useful signal from the background clutter. Then,
a tell-tale spectral signature may be obtained, which may be identified
by the computer as being very similar to that of some known device. All
these have to be done with mathematical precision, and without making errors,
despite the tension of knowing that a torpedo may be aiming at oneself,
and must be done faster than the other folks can process their sensor data
to locate you. This is one example application which has driven a great
deal of research into "DSP" algorithms.
Of course there are many other applications of digital
signal processing, which may sound much less dramatic than the above, but
are probably far more generally used.
DSP techniques are amazingly powerful and versatile.
Their ease of use and availability have improved by many orders of magnitude
in the last 15 years. In 1982, we had sign-up sheets to use the department's
Fourier Analyzer, a coke-machine sized minicomputer system, which had to
be started up using a paper tape (which tended to wrap itself around the
user) and rows of lighted push-buttons where one entered ritual number
sequences in octal code. There was one huge disk platter which could hold
as much as 1 Megabyte, but the program space was limited to 64 characters.
The machine included a 4-channel A/D converter and selectable sampling
combinations of sampling parameters, and an oscilloscope. Today, every
PC has a sound card with a DSP chip on it, and A/D cards and software oscilloscopes
are available for use with laptop computers. The best book on the subject
of analyzing random data, which we were able to find in those days, is
still in print:
Reference:
Bendat, Julius S., and Piersol, Allan G., "Random
Data: Analysis and Measurement Procedures" 2nd Edition, Wiley Interscience,
NY 1986.
OVERVIEW
In this brief course, we will see the definitions, and
some examples of usage of several concepts:
Deterministic vs. Random Data
-
Classifications of Deterministic Data
-
Basic descriptive properties
Random Data
-
Time stationary random process
-
Ergodic process
-
Autocorrelation function
-
Cross-correlation function
-
Linear combinations of random processes
Frequency-Domain Information
-
Autospectrum
-
Cross Spectrum
-
Coherence Function
-
Transfer Function
-
Wiener-Khintchine relation
-
Sampling Theorem and Nyquist Frequency
-
Discrete Fourier Transform
-
Fast Fourier Transform
-
Spectral Analysis algorithms
Examples:
1. Transfer function approach to calibrate a microphone
probe
2. Finding the response of a sensor from in-situ data:
thermocouples in flames
3. Extracting the response to one flow variable from
a sensor which responds to more than one variable
4. Digital response compensation
5. Dynamic balancing of rotating machinery
Deterministic vs. Non-Deterministic
Deterministic: We can describe the process in such
a way that knowledge of a sample of signal from the past enables exact
prediction of the value of the signal at a given future instant. Examples
are processes which can be described by rational expressions:
Example: 
. Given A and 
, one can predict the value of x for any desired value of t.
Conversely, given the relation between x and t, plotted
out for several values of t, our task as Diagnosticians would be to find
the above neat expression which would allow us to understand the behavior
of x, and therefore to predict its value at future values of t. Not all
deterministic processes are so obvious. Some take lots of sorting and analysis,
and often some inspired guesswork. Often we have to give up in frustration,
and declare that the process is non-deterministic: knowing what happened
every day in the past to the Dow Jones Index does not allow us to predict
the closing value 3 days from today, let alone next month. We will have
to settle for descriptions in terms of statistics, probabilities, uncertainties
and margins of error. Before we delve into statistical techniques for representing
non-determinate processes, however, it is useful to remember something:
A "random" process becomes a deterministic process if one
finds the entire physics of the process. Of course, this is considered
to be "impossible" for many complicated processes (the stock market, turbulence,
final exam questions...) but we keep trying. There is reason for hope.
Consider
the series of digits:
1, 4, 1, 5, 9, 2, 7,....
Looks pretty random, huh?
Would you believe that (a) this
series goes on for thousands of digits, and (b) it can be described by
a very simple, closed-form analytical expression from which every digit
and its location(s) in the series can be predicted? Its true: try guessing
it.
Hint: Its more difficult to guess
this in Alabama: in fact it may be unlawful to do so.
DIGITAL
SIGNAL PROCESSING: RANDOM SIGNALS
Basic
Concepts in Sampling
1. AFTER you have sampled the data and converted it from
a continuous analog trace to a series of numbers, it is usually not possible
to reverse the process entirely. So decisions about sampling parameters
must be made with some thought and planning.
Now consider the signal above. We can say with some "confidence"
that:
a) the average value of the signal is about 3.5, plus
or minus 1 volt.
b) the extreme excursions of the value are between 5
and 3 volts, but the root-mean-square of the fluctuations is, maybe, about
0.3 volts.
Note: How can we say anything so wild and
imprecise, based on so little observation, and without mathematical proofs,
when there is no exact formula to be found? What about all those little
glitches and wild swings, each different from the next? Are our standards
of scientific enquiry and skepticism so low that we will accept unproven
facts based on anecdotal evidence? Welcome to Flow Diagnostics. We will
make some bold decsions based on extensive experience and engineering judgement
(i.e., guesses), and to some extent we shall be willing to see what we
want to see, in the data. Without such decisions, we cannot make progress.
Rest assured, though, that we won't throw away the data, and we won't be
satisfied with our guesses alone: we will develop rigorous procedures for
getting accurate results, and show that they are indeed accurate.
We can say this even if the analog signal is taken away
and we are left looking at the digital series only. This is because we
are smart:
a) we have sampled the signal for long enough to have seen
"pretty-much" all that the signal is likely to do, and
b) we have sampled it with a sampling rate which is high
enough to reconstruct the most rapid changes in the signal.
Now lets look at another signal:
Here we can't say much about the mean or the fluctuations,
because we have not sampled the signal for long enough. Suppose we can
take and store only 20 values in our computer. We would have to space out
the 20 so that the 20th occurs at a large value of time: the sampling rate
would then be too low to capture the rapid fluctuations in the signal.
These lead to the Nyquist criterion.
Nyquist Criterion
To be able to reconstruct a sine wave with a period Tmin
(frequency Fmax = 1/Tmin), we need at least 2 digital values captured within
Tmin.
Two fundamental rules of digital sampling are:
1. The sample duration must be longer than 1/Fmin,
Fmin being the lowest frequency of fluctuations in the signal.
2. The sampling rate must be greater than 2 Fmax,
Fmax being the highest frequency of fluctuations (Nyquist frequency) present
in the signal.
Note: These are not "suggestions": they are required
conditions to avoid serious error. Even if we are not interested in frequencies
above the Nyquist frequency, or lower than the frequency resolution, our
sampling will suffer serious error if there are fluctuations present beyond
these limits. If there are substantial fluctuations below our frequency
resolution, the effect will be that our sample mean will be in error. If
there are fluctuations faster than the Nyquist frequency present, their
effects will appear as error in the low-frequency data in which we are
interested. So, we must understand the frequency limits of the data before
digitizing, or, if this is impossible, filter the data so that there are
not significant fluctuations outside our digitizing range.
Example:
We expect fluctuations of frequency
as low as 0.1Hz (things may change substantially in 10 seconds), and as
high as 200Hz (changes occurring within 1/200 sec. ). The sampling rate
must be higher than 400Hz, and the sample time must be longer than 10 seconds.
This means that you must grab at least 4000 digital values (10 times 400).
In this example, the Nyquist frequency is 200Hz.
Implications:
If Nyquist frequency Fmax = 200Hz,
Sampling Rate = 400 samples per second,
Sample Time T = 10 seconds.
then
Frequency Resolution 
, and
Time Resolution 
= 2.5 milliseconds.
This can quickly become impractical: our computer memory,
patience and Analog/Digital Converter's maximum sampling rate are all limited.
So, we make sure that there are no fluctuations present in the signal beyond
our range of interest by using a "High-Pass Filter" to cut off fluctuations
slower than Df, and a Low-Pass Filter
to cut off all fluctuations faster than Fmax.
Example: Spectrum of a broad-band
signal with high-pass and low-pass filters applied
Note, above, that filters don't just cut off everything
below or above some magic number: think of them as "attenuators", not as
"switches". Filter effectiveness is described in terms of "dB per Octave".
In other words, if you go to twice the upper frequency limit, how many
deciBels would the signal level have been attenuated? A cut-off of 40dB
per octave is good; 60dB per octave is very expensive. Because of this
gradual cut-off, it is smart to set one's frequency resolution and Nyquist
frequency well beyond the range that one really needs.
Time-Stationary
Random Process
Statistical description does not vary with time.
This assumes that
a) you will sample for long enough to see "pretty much
everything" that the signal will do
b) there must be no trends.
Two examples are shown above, where the statistical description
of the signals keeps changing with time. In the first one, the mean value
keeps decreasing. In the second, the mean value seems to be stable,
but the frequency content is definitely changing: what started as a series
of sharp squiggles (changes occurring with high rate, and therefore containing
fluctuations of high frquency), changes into something which is dominated
by large, but relatively slow (low-frequency) fluctuations.
Two things we can say about a stationary process:
Ensemble
Average at time t:
is constant, regardless of value of t1.
Autocorrelation:
. Depends only on the time delay
, not on the value of t
1.
Ergodic Process
An ergodic process is one for which the following is
true:
Time Average = Ensemble Average.
Time Average is:
Ensemble Average:
.
If one takes large enough T
and N, don't these two ways of computing the mean always give the same
answer? NO!
Counter-Example: Digital data from
a laser Doppler velocimeter, counter processor.
If we manage to seed the flow uniformly,
then the number of seed particles per unit volume should be constant. Then,
the number of seed particles going through the measuring volume per unit
time is proportional to the speed of the flow!
More particles are counted, and hence more velocity values
are obtained, when the speed is high, and fewer when the speed is low.
Thus, if we take a fixed number of particles, or all the particles which
arrive within a fixed amount of time, we will get an average which is biased
towards the higher velocity.
.
This is called Velocity
Bias.
Note: Velocity Bias is easily eliminated.
All we have to do is to use an external clock, and divide the sampling
time into several equal intervals. Now we take an average of all the data
which arrives within each interval. At the end, we take an ensemble average
of all the interval-averages. This means that the high-velocity points
get only as much "voting power" as the low-velocity points.
Autocorrelation
Function for a random process x(t):
. Here E is the "Expected Value", which means "ensemble average".
For a stationary process, the autocorrelation depends
only on the time difference
Note: Expected value of x1, x2:
where
is the
joint probability density function of x1 and x2.
Ergodic:
is equal to the time average of
=
Note: The Autocorrelation is a
function of delay for a stationary process. If the process is not ergodic,
we must distinguish between the ensemble and time autocorrelations.
Physical Significance of the Autocorrelation
The autocorrelation function is the answer to the question:
"What is the relation between the value of this property at time t and
its value Dt before, or Dt
after?"
Examples
1) Autocorrelation of a constant is a constant.
Note: units of Rxx are the same as units of x**2
2) Autocorrelation of a periodic function is periodic.
3) Autocorrelation of a random signal with high-frequency
fluctuations.
Not much correlation between events now and what happened
a short time ago, or what comes a short time hence).
General
Properties of Autocorrelation Functions
1)
is the mean-square value of the process x(t).
2)
is an even function of
. This results from the assumption of stationarity. Thus,
. If it is not stationary, then there is symmetry with respect to the two
arguments, t
1 and t
2.
3) 
for all values of 
. The biggest peak is always at zero.
4) If x(t) contains a periodic component, 
will also contain a periodic component with the same period. However, phase
information is lost: 
is always a cosine function.
5) If x(t) does not contain a periodic component, then 
as 
. That is, 
becomes totally uncorrelated with x(t) for large values of 
if there are no periodicities in the process.
6) 
gives the mean value of the process. In other words, 
implies that the mean is zero.
Autocorrelation
Function examples from Bendat & Piersol, p. 114:
1. Constant
2. Sine Wave
3. White Noise
4. White noise, low-pass
filtered
5. White noise,
band-pass filtered:
6. Exponential
7. Exponential Cosine
8. Exponential cosine
and exponential sine
Cross-Correlation
Function
The autocorrelation considered the relation between the
value of a signal at a given time, with values of the same function at
other times. The cross-correlation considers the relation between the value
of one signal at a given time, with values of another function at other
times.
If the process is stationary, the value of the cross-correlation
should depend only on time delay, not on the precise values of the two
times.
.
Note:
Now if the process is stationary in time, it should not
matter if one moves one's reference for
by a bit. i.e., the process is "invariant under a translation of
. Thus,
and
Examples of
cross-correlations
Case (1):
x(t) and y(t) are exactly the same
random signal.
Rxy = Rxx = Ryy.
Case (2):
One signal (x) is delayed with respect
to the other (y).
. The peak of the cross-correlation occurs at time delay Dt.
Case (3):
y(t) is opposite of x(t), and is
delayed by Dt.
Note: The cross correlation peak
need not be positive!
Example:
. Suppose 
is constant, but U fluctuates randomly. 
, or,
Lets guess what 
looks like:
Correlation
Coefficients
Note:
.
is the mean-square
value of x.
Autocorrelation coefficient is:
Defining the cross-correlation coefficient requires some
thought, because the value of the cross-correlation at zero time delay
has no particular meaning. Thus,
Linear
Combinations of Random Processes
Say,
Thus,
.
Since
, the autocorrelation is
still
an even function (symmetric about
).
Note: If x and y are uncorrelated, then
.
Example: Lets go back to the turbulent flow which we
considered before. In reality, p0 will also fluctuate. Then,
. So, if we get autocorrelations of the stagnation pressure p0, p, and
(and thus of q), we can derive the correlation
between static pressure and velocity fluctuations in turbulent flows.
Lecture
B04
Spectral Density Functions
Even random-looking processes can have distinctive "signatures",
when viewed in the frequency domain. This is done by constructing their
"spectral density" functions. These functions describe how the energy of
fluctuations is distributed as functions of frequency.
Example:
Constructing spectral density function
of a process x is done as follows:
1) Sample the signal x(t)
2) Compute the Fourier Transform
of x(t). This is X(w), where w = 2pf.
3) Find the square of the magnitude
of X(w). This is 
where the * denotes the complex conjugate.
i.e,, if X(273)
= 0.581 + i(0.287), then
X*(273) = 0.581
- i(0.287).
is called the Autospectral density function, or Autospectrum, of x. It
is a positive, real function of frequency.
Examples from Bendat & Piersol, p. 127:
1. Constant signal:
2. Sine Wave
3. White Noise:
Sxx(f) = a, for f>0. Zero otherwise.
4. Low-Pass White Noise
Sxx(f) = a, for 0< f<B. Zero otherwise.
5. Band-Pass White Noise:
Sxx(f) = a, for 
. Zero otherwise.
6. Exponential:
7. Exponential Cosine:
8. Exponential Cosine, exponential Sine
Cross-Spectral Density Functions.
Given 2 signals x(t) and y(t):
Fourier transforms are: X(w) and Y(w). Thus,
is Sxx, the autospectrum of x, a real function.
is Syy, the autospectrum of y, a real function.
What about X*Y and Y*X? These are the cross-spectra Sxy
and Syx. They are complex functions of frequency.
is the magnitude of Sxy, and gives the energy content common to both x
and y.
The phase of Sxy is
. This gives the phase relationship between the signals x and y, as a function
of frequency.
Wiener-Khintchine Relation
Both the correlation and the spectrum are valuable tools
for analyzing signals. They are related through the Wiener-Khintchine relation:
. Conversely, 
,
where F[ ] denotes the forward Fourier Transform, and F-1
the reverse (or inverse) Fourier Transform.
If the process is stationary random, one has to carry the
integral over infinite time to capture all the information at all of the
frequencies present. In practice, one has to select some finite period
of integration, which imposes a lower limit of frequency. So, we should
strictly say:
.
The cross-spectrum is given by: 
. Sxy and Syx are complex functions. Sxx and Syy are real. Also, since
,
Coherence Function
The Coherence function is the equivalent of the correlation
coefficient in the frequency domain.
It is real-valued, and positive.
Its value lies between 0 and 1.
A value near 1.0 indicates that there is a linear relationship
between signals x and y in that frequency interval. This is true even if
there is a substantial phase difference.
Transfer Function
Likewise,
The transfer function Hxy is the ratio y/x in the frequency
domain. So it is a way of expressing the sensitivity (change in output
per unit change in input) as a function of frequency.
It is a complex function of frequency (it has both magnitude
and phase).
Note: The transfer function is only valid at those frequencies
where the coherence is "good" (generally above 0.8). To get accurate phase
information, the coherence should be above 0.9.
Example: Variation of
Spectral Distributions with Velocity
The
hot-film technique can be used to measure the distribution of the energy
contained in the fluctuations as a function of frequency. This is done
by computing the auto-power spectrum of the velocity fluctuation signal
using a standard Fast Fourier Transform algorithm. Basically, this technique
fits a Fourier sine series to the signal, and determines the amplitudes
of the discrete number of sine waves at several discrete frequencies, that
make up the signal. The spectrum is a plot of the square of the amplitude
of the sine wave at each frequency. More precisely, the spectrum is a plot
of the energy of the fluctuation, contained within each discrete interval
of frequency. Such plots quickly reveal the existence of strong, discrete-frequency
fluctuations such as the vortex shedding phenomenon. On the other hand,
if only random turbulence is present, the spectrum consists of a smooth
decrease of the amplitude with increasing frequency.
Such
spectra are computed from the signals measured at the edge of the wake
of the cables. The freestream speed is varied in steps to get spectra at
several velocities. It is seen, as expected, that the center frequency
of the peak of the spectrum (the narrow band of frequencies where the fluctuation
amplitude is highest, corresponding to the shedding of vortices) increases
with freestream velocity. This variation is plotted to obtain the relation
between the vortex shedding frequency and the wind speed.
Relative
Phase Measurement
The
spectral analysis technique is extended to measure the relationships between
fluctuating signals measured by different sensors. Thus, for example, the
signals from two hot-film sensors placed very close together in the flow
may be expected to be virtually identical, except for a factor accounting
for the difference in the calibration constants of the two sensors. This
similarity can be seen by computing the "Coherence Function". This function
of frequency takes on values between 1, which indicates a perfect linear
relationship at that frequency, and 0, which indicates a lack of such a
linear relationship. If one places a two hot-film sensors in tandem in
the wake of the cables, one expects to obtain a coherence close to 1.0
at the frequency where the vortex shedding occurs. This will occur even
though there will be a substantial time lag between the passage of a given
vortex past the two sensors.
At frequencies
where the coherence is "good", i.e., higher than about 0.5, one can determine
a phase relationship between the fluctuations at the two sensors. This
is done by computing the "cross-spectrum", or the "transfer function",
both of which are complex functions of frequency. The magnitude of the
cross-spectrum is a measure of the energy contained in those fluctuations
which are common to both sensors; the phase of the cross-spectrum shows
the relative phase between the occurrences of similar phenomena at the
two sensors.
In this
project, we attempted to measure relative phase between the shedding phenomena
at different points along the cable by holding one sensor in a fixed location
and moving the other in succession to different spanwise separations (parallel
to the cable). As expected, we obtained excellent coherence and very little
phase shift when the separation was small (less than 0.5 inch). The results
obtained with larger spacings was, however, surprising, and varied from
one cable design to another.
