close

Вход

Забыли?

вход по аккаунту

?

Development of the "optimal filters" for mitigation of striping noise in satellite microwave temperature and humidity sounding data

код для вставкиСкачать
FLORIDA STATE UNIVERSITY
COLLEGE OF ARTS AND SCIENCES
DEVELOPMENT OF THE “OPTIMAL FILTERS” FOR MITIGATION OF STRIPING NOISE
IN SATELLITE MICROWAVE TEMPERATURE AND HUMIDITY SOUNDING DATA
By
YUAN MA
A Dissertation submitted to the
Department of Earth, Ocean and Atmospheric Science
in partial fulfillment of the
requirements for the degree of
Doctor of Philosophy
2015
ProQuest Number: 3724310
All rights reserved
INFORMATION TO ALL USERS
The quality of this reproduction is dependent upon the quality of the copy submitted.
In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if material had to be removed,
a note will indicate the deletion.
ProQuest 3724310
Published by ProQuest LLC (2015). Copyright of the Dissertation is held by the Author.
All rights reserved.
This work is protected against unauthorized copying under Title 17, United States Code
Microform Edition © ProQuest LLC.
ProQuest LLC.
789 East Eisenhower Parkway
P.O. Box 1346
Ann Arbor, MI 48106 - 1346
Yuan Ma defended this dissertation on June 4, 2015.
The members of the supervisory committee were:
Ming Cai
Professor Directing Dissertation
Xin Yuan
University Representative
Guosheng Liu
Committee Member
Peter Ray
Committee Member
Jeffery Chagnon
Committee Member
The Graduate School has verified and approved the above-named committee members, and
certifies that the dissertation has been approved in accordance with university requirements.
ii
To Prof. Xiaolei Zou.
iii
ACKNOWLEDGMENTS
First of all I thank Dr. Xiaolei Zou for her continuous guidance, support and suggestions
since 2010. She has provided me everything I could expect from an advisor and I owe her great
deal of gratitude.
After Dr. Zou’s departure for University of Maryland in 2014 Fall, Dr. Zou continued to
direct my Ph. D. thesis and Dr. Ming Cai is very kind to serve as my new major professor. I
really appreciate Dr. Cai’s help and time. I also want to thank Drs. Guosheng Liu, Peter S. Ray,
Jeffery Chagnon and Xin Yuan for serving on my committee.
The work would have been impossible without the generous help from Dr. Fuzhong
Weng and his Calibration/Validation team in NOAA. I have also received lots of help from
scientists and students working with Dr. Zou, especially Dr. Zhengkun Qin.
Last but not least important, I want to thank my family and friends for their unconditional
love.
This research was supported by JPSS proving ground project NA11OAR4320199.
iv
TABLE OF CONTENTS
List of Tables ................................................................................................................................ vii
List of Figures .............................................................................................................................. viii
Abstract ......................................................................................................................................... xii
1.
2.
3.
INTRODUCTION .................................................................................................................. 1
1.1
History of Satellite-based Microwave Instruments....................................................... 1
1.2
Striping Noise Contained in ATMS Observations........................................................ 3
1.3
Existing Methods to Mitigate Striping Noise ............................................................... 4
1.4
Dissertation Organization ............................................................................................. 5
DATA ..................................................................................................................................... 7
2.1
Advanced Technology Microwave Sounder (ATMS) .................................................. 7
2.2
The Community Radiative Model (CRTM) ................................................................. 7
2.3
NCEP GFS data ............................................................................................................ 8
METHODOLOGY ................................................................................................................. 9
3.1
Derivation of Antenna Temperature from Raw Counts ................................................ 9
3.2 Mathematical Expressions of Optimal Filters ............................................................. 10
4.
Optimal Filters for Warm Counts and Cold Counts ....................................... 10
3.2.2
Optimal Filters for Scene Counts .................................................................... 14
3.2.3
Optimal Filters for Brightness Temperatures ................................................. 15
ATMS STRIPING NOISE MITIGATION USING THE OPTIMAL FILTERS ................. 20
4.1
4.2
5.
3.2.1
Striping Noise Mitigation on Sensor Calibration Counts ........................................... 20
4.1.1
Spectrum Analyses of IMFs of Calibration Counts ........................................ 20
4.1.2
Properties of the Optimal Symmetrical Filters ............................................... 21
4.1.3
Comparison between the Optimal Filters and Boxcar Filters ......................... 23
4.1.4
Effects of the Optimal Filters on Antenna Temperatures ............................... 24
Striping Noise Mitigation on Brightness Temperatures ............................................. 24
4.2.1
Weighting Coefficients of the Optimal Filters................................................ 24
4.2.2
Brightness Temperatures before and after Striping Noise Mitigation ............ 26
4.2.3
Liquid Water Path Retrieval Algorithm .......................................................... 28
4.2.4
Impact of Striping Noise on ATMS LWP Retrievals ..................................... 31
STRIPING NOISE MITIGATION FOR FY-3C MWTS DATA ........................................ 68
5.1
Characterization and Mitigation of Striping Noise in FY-3C MWTS Data ............... 68
v
5.2
6.
5.1.1
Fengyun-3C Microwave Temperature Sounder (MWTS) Data ..................... 68
5.1.2
Diagnosis of Striping Noise in FY-3C MWTS Data ...................................... 69
5.1.3
Global Striping Index For FY-3C MWTS Data ............................................. 72
Investigation on the Root-Cause of Striping Noise .................................................... 74
5.2.1
Mathematical Expressions of Radiometric Transfer Function ....................... 74
5.2.2
PSD of the Output Noise of Transfer Function .............................................. 77
SUMMARY AND DISCUSSIONS ................................................................................... 103
6.1
Major Contribution ................................................................................................... 103
6.2
Summary and Conclusions ....................................................................................... 103
6.3
Future Work .............................................................................................................. 106
APPENDIX ................................................................................................................................. 107
A.
LIST OF ACRONYMS ...................................................................................................... 107
B.
COPYRIGHT PERMISSION ............................................................................................. 108
REFERENCES ........................................................................................................................... 110
BIOGRAPHICAL SKETCH ...................................................................................................... 112
vi
LIST OF TABLES
4.1
Numbers of IMFs removed and filter spans of optimal striping filters for ATMS scene
counts at each channel .................................................................................................................. 34
4.2
Filter spans of optimal striping filters for ATMS warm counts and cold counts ............. 35
4.3
Numbers of IMFs removed and the chosen filter spans for ATMS channels ................... 36
5.1
Central frequencies of FY-3C MWTS and corresponding ATMS channels .................... 80
5.2
Scan cycle ( t s ), calibration integration time ( τ c ) and scene integration time ( τ s ) that are
used as input for calculating the transfer functions for AMSU-A, ATMS and FY-3C MWTS
before and after profile changes .................................................................................................... 81
5.3
Numbers of FOV and NEDT at AMSU-A, ATMS, MWTS channels with central
frequencies of 57.290344 GHz ..................................................................................................... 81
vii
LIST OF FIGURES
1.1
(a) Global distribution of O-B fields for ATMS channel 8 on February 24, 2012. (b)
Same as (a) but for a smaller area ................................................................................................... 6
3.1
A schematic illustration of ATMS scan cycle .................................................................. 19
4.1
Fourier spectra with 81-point running mean for the first six IMFs of (a) warm counts and
(b) cold counts at channel 8 .......................................................................................................... 37
4.2
Same as Fig. 4.1 but for warm load temperature. Five IMFs are removed. ..................... 38
4.3
Same as Fig. 4.1 but for the first PC coefficient of scene counts at channel 8. Three IMFs
are removed ................................................................................................................................... 39
4.4
Normalized cost function with respect to filter span N for warm counts (upper), cold
counts (middle) and scene counts (lower). Numbers of scan lines involved in the optimal filter is
indicated in circles ........................................................................................................................ 40
4.5
Weighting coefficients of optimal filters for warm counts (upper), cold counts (middle)
and scene counts (lower). .............................................................................................................. 41
4.6
Response function for warm counts (upper), cold counts (middle) and scene counts
(lower) calculated with weighting coefficients as in Fig. 4.5. Contours over dark blue shading
area all have the value of 0.01 because of oscillations of response functions .............................. 42
4.7
(a) Normalized cost function with respect to filter span N for warm load temperature.
Numbers of scan lines involved in the optimal filter is indicated in circle. (b)-(c) weighting
coefficients and response function for warm load temperature .................................................... 43
4.8
Variations of (a) warm counts and (b) cold counts along 700 scanlines (i.e., a swath
between ±55N ) before (gray) and after (red) applying removing three IMFs at channel 8. ...... 45
4.9
Variations of (a) warm counts and (b) cold counts along 700 scanlines (i.e., a swath
between ±55N ) before (gray) and after (red) applying removing three IMFs at channel 22 ..... 46
4.10 Variations of (a) warm counts and (b) cold counts with optimal filter (red) and 17-point
smoothing (blue) for channel 8 ..................................................................................................... 47
4.11 Variation of response functions with respect to frequency for the 17-point filter (blue)
and the optimal filters (red) applied to warm counts of channel 8 (curve) and channel 22 (circle).
The response functions for cold counts are the same as those for warm counts (figure omitted). 48
4.12 Variation of brightness temperature for channel 8 at nadir with (red) and without (blue)
applying the optimal filter on the scene counts. Warm count, cold counts and warm load
temperatures are all smoothed ...................................................................................................... 49
viii
4.13 Striping noise removed by the optimal symmetric filters for ATMS channels 8, 22 and 1
on ascending node ......................................................................................................................... 50
4.14 (a)-(d) Variation of the normalized cost function with respect to filter span N for ATMS
channels 1-22. The cost function value with the selected symmetrical filter span (2N+1) is
indicated by circle. (e) The cost function for N=2 used for normalization in (a)-(d) for each
channel. ......................................................................................................................................... 51
4.15
Weighting coefficients of symmetric filters for ATMS channels 1-22 ............................ 53
4.16
Response functions of symmetric filters calculated with weighting coefficients. ............ 55
4.17 The power spectral density distributions of the global O-B field of ATMS channels (a) 9
and (b) 10 before (red) and after (blue) removing the striping noise using the optimal filter. ..... 57
4.18 Global distribution of O-B fields (a) before and (b) after applying the optimal filter for
ATMS channel 8 on February 24, 2012. (c) Striping noise removed by optimal filter ................ 58
4.19 Striping noise removed by the symmetric filters for ATMS channel (a) 1, (b) 8 and (c) 22
on February 24, 2012 .................................................................................................................... 59
4.20 Maneuver data (a) before and (b) after optimal smoothing on PCA1 for ATMS channel 1.
(c)-(d) Same as (a)-(b) but for channel 9. ..................................................................................... 60
4.21 (a) The along-track variance, (b) across-track variance and (c) striping index before (red)
and after (blue) smoothing. FOVs 25-72 on 100 scan lines are involved in calculation .............. 61
4.22 Global distributions of striping noise removed by the optimal filter for ATMS channels 1
and 2 on February 24, 2012 .......................................................................................................... 62
4.23 Global distribution of LWP retrieved by ATMS channels 1-2 observation (a) without and
(b) with smoothing and (c) the LWP differences between (a) and (b) during 0000-1200 UTC on
February 24, 2012. ........................................................................................................................ 64
4.24
Same as Fig. 4.23 but for 1200-2400UTC ........................................................................ 65
4.25 Probability density functions of LWP difference within 30S-30N over ocean on February
24, 2012. (a) LWP retrieved by observations, (b) noise removed at ATMS channel 1 and (c)
noise removed at ATMS channel 2 are color coded, respectively. The mean and std of the LWP
differences with and without smoothing (i.e., LWPwithout-LWPwith) within 30S-30N over ocean on
February 24, 2012 are zero and 0.005 kg m-2. .............................................................................. 66
4.26 Scatter plots of the LWP differences with respect to (a)-(b) striping noise of channels 1
and 2 and (c)-(d) LWP retrieved with observations without smoothing. The magnitude of LWP is
indicated in (a) and (b) and the magnitude of striping noise of channels 1 and 2 is indicated in (c)
and (d), respectively. Data within 30S-30N over ocean on February 24, 2012 are used ............. 67
ix
5.1
Fourier spectra for the first four IMFs of the (a) first, (b) second, (c) third and (d) fourth
PC coefficients at channel 8 for May 1, 2014 ............................................................................... 82
5.2
Fourier spectra for the first six IMFs of the (a) first, (b) second, (c) third and (d) fourth
PC coefficients at channel 8 for May 29, 2014. ............................................................................ 83
5.3
Fourier spectra for the first six IMFs of the (a) first, (b) second, (c) third and (d) fourth
PC coefficients at ATMS channel 10 for May 1, 2014................................................................. 84
5.4
Spatial distributions of the first five PC modes (i.e., eigenvectors) for the 47th data sample
(k=47, each sample includes 200 scan lines) on May 1, 2014 ...................................................... 85
5.5
Spatial distributions of the individual PCA component for the first three PCA
components (i.e., the product of eigenvalue and eigenvector) before (left panels) and after (right
panels) striping noise mitigation for the 47th data sample (k=47, each sample includes 200 scan
lines) on May 1, 2014 ................................................................................................................... 86
5.6
Spatial distributions of ATMS channel 8 (a) observed, (b) simulated and (c) smoothed
3PC/4IMF and (d) 1PC/4IMF brightness temperatures for the 47th data sample (k=47, each
sample includes 200 scan lines) on May 1, 2014. O-B differences (e) before and after (f)
3PC/4IMF and (g) 1PC/4IMF noise mitigation ............................................................................ 88
5.7
Noise removed by (a) 3PC/4IMF and (b) 1PC/4IMF as in Fig. 5.6. (c) Differences
between (a) and (b) ....................................................................................................................... 90
5.8
Global distributions of channel 8 (a) O-B and (b) difference between smoothed brightness
temperatures removed on May 1, 2014 on ascending node. The first four IMFs of the first three
PC coefficients are removed for noise mitigation. (c) The removed striping noise ..................... 91
5.9
Global distributions of channel 8 (a) O-B and (b) difference between smoothed brightness
temperatures removed on May 30, 2014 on ascending node. The first three IMFs of the first three
PC coefficients are removed for noise mitigation. (c) The removed striping noise ..................... 92
5.10 Variations of (a) along-track variances as a function of the sample group number and
FOV (i) and (b) cross-track variances as a function of dataset number (k) and scan line (j) before
striping noise mitigation on May 1, 2014. The total number of scan lines in each dataset is 200.
The total number of datasets for both the ascending and descending nodes on May 1 is 162. (c)
Variation of striping index as a function of dataset number (k). .................................................. 93
5.11
Same as Fig. 5.10 except for data with striping noise mitigated (PC3/IMF4).................. 94
5.12
Same as Fig. 5.10 except for data before striping noise mitigation on May 30, 2014...... 95
5.13
Same as Fig. 5.10 except for data with striping noise mitigated (3PC/3IMF).................. 96
x
5.14 (a) Along-track, (b) cross-track variances and (c) striping index at channel 8 during May
1-7, 2014 and May 29-June 4, 2014 before (red) and after (blue) noise. Ascending and
descending nodes are indicated in open and solid markers, respectively ..................................... 97
5.15 Square of transfer function (H(f)) that are calculated using instrument parameters of (a)
AMSU-A and (b) ATMS with two (red) and five (blue) nearest points being averaged in
calibration ..................................................................................................................................... 98
5.16 Same as Fig. 5.15 except for FY-3C MWTS (a) before and (b) after profile change that
occurred during May 12-18, 2014 ................................................................................................ 99
5.17 PSD of (a) pitch-over maneuver data at ATMS channel 10, (b) white noise only and (c)
the sum of white noise and flicker noise with (colored) and without (black) 81-point running
mean ..... ...................................................................................................................................... 100
5.18 The PSD of the noise outputs ( Sn( f ) ) for (a) AMSU-A, (b) ATMS, (c) MWTS before
profile change and (d) MWTS after profile change with (red) and without (green) adding flicker
noise. ... ....................................................................................................................................... 101
5.19 Differences of the PSD of the noise outputs ( Sn( f ) ) between with and without flicker
noise for ATMS (blue), MWTS before profile change (red) and MWTS after profile change
(green). ........................................................................................................................................ 102
xi
ABSTRACT
Advanced Technology Microwave Sounder (ATMS) has been flying on the Suomi
National Polar-orbiting Partnership (NPP) satellite since October 28, 2011. A striping noise
phenomenon was noticed in the global distribution of O (observations) minus B (model
simulations) differences for different ATMS sounding channels. In the first part of this
dissertation, a set of “optimal filters” is developed for smoothing out the striping noise in warm
counts, cold counts, warm load temperatures and scene counts. Using the two-point calibration
equation, antenna temperatures were calculated with and without applying the optimal filters on
warm counts, cold counts, warm load temperatures and scene counts. The patterns and
magnitudes of the striping noise removed are very close to that from an early method that
combines the principal component analysis (PCA) with the Ensemble Empirical Mode
Decomposition (EEMD) method. It was also shown that the optimal filters are superior to the
conventional boxcar filters in terms of being able to effectively remove the striping noise in the
high frequency range but not to alter the lower frequency weather signals. In the second part of
the thesis, a set of 22 “optimal filters” that can be applied directly on brightness temperature was
also developed to remove the striping noise in all ATMS channels. Impacts of striping noise
mitigation on small-scale weather features were investigated by comparing ATMS cloud liquid
water path (LWP) retrieved before and after striping noise mitigation. It was shown that the
optimal filters do not affect small-scale cloud features while smoothing out striping noise in
brightness temperatures. Striping noise is a general problem for microwave sensors, and is also
identified within observations of a recent Fengyun-3C (FY-3C) microwave temperature sounder
(MWTS). In the third part of the thesis, striping noise within MWTS observation was analyzed.
It was found that the magnitude of the striping noise in MWTS is around 1 K, which is much
xii
larger than in ATMS (~0.3 K). Finally, a transfer function was employed to seek the root cause
of the striping noise. This transfer function is controlled by instrument parameters such as scan
cycle, calibration integration time and scene integration time. Instrument noise is simulated by a
white noise series with and without adding so-called flicker noise. A power spectral analysis was
applied to the output noise from transfer function with varying input instrument parameters. It
was shown that flicker noise is the source of striping noise, whose magnitude and peak frequency
can be modified by different setting of input parameters of transfer function.
xiii
CHAPTER ONE
INTRODUCTION
1.1 History of Satellite-based Microwave Instruments
With the successful launch of Television Infrared Observation Satellite -N (TIROS-N) on
October 13, 1978, the first Microwave Sounding Unit (MSU) came to use. MSU is a four-channel
sounding radiometer designed to monitor temperature in troposphere and lower stratosphere.
MSU continued to get on board 8 subsequent National Oceanic and Atmospheric Administration
(NOAA) meteorological satellites until its predecessor Advanced Microwave Temperature
Sounding Units (AMSU) was deployed in 1998.
AMSU consists of Advanced Microwave Sounding Unit-A (AMSU-A) and -B (AMSU-B).
AMSU-A is a 15-channel microwave temperature sounder, and AMSU-B is a five-channel
microwave humidity sounder. AMSU-A and AMSU-B are on aboard NOAA-15, NOAA-16 and 17. AMSU-B was replaced by Microwave Humidity Sounder (MHS) when NOAA-18 was
launched on May 20, 2005. AMSU-A and MHS are on board NOAA-19, European MetOp-A and
MetOp-B satellites, which were launched on February 6, 2009, October 19, 2006 and September
17, 2012, respectively.
The AMSU was replaced by the Advanced Technology Microwave Sounder (ATMS) when
the new US satellite, Suomi National Polar orbiter Partnership (NPP), was launched on October
28, 2011. ATMS combines previous microwave temperature and humidity sounders AMSU-A
and MHS into a single package, and is the most advanced, state-of-the-art satellite-based
microwave instrument that provides temperature and humidity information at the same 96 fieldof-view (FOV) locations, although FOV sizes are different. Compared with its heritages, ATMS
has included an extra temperature channel in the low troposphere and two new humidity channels.
1
ATMS channels 16, 17, 22, 20 and 18 correspond to MHS channels 1-5, respectively. Compared
with its predecessors AMSU-A and MHS, ATMS has several other improvements such as wider
swath width, additional sounding channels and smaller noise equivalent delta temperatures.
Microwave radiance is one of the observations that affect NWP model forecast most. The
microwave temperature sounding data such as AMSU observations have been operationally
assimilated in the major NWP centers. The assimilation is found to have positive impacts on
model forecasts. Microwave humidity sounding data also show the potentials to enhance NWP
forecasts in experiments, such as cloud detection.
Before those microwave observations can be fully put into use, calibrations and validation
are necessary. It is noted those sensors produce some kind of noise, such as white noise and pink
noise, whose power spectral density (PSD) are constant or in a form of 1/f, where f stands for
frequency. The instrument noise is associated with NEDT (Noise Equivalent Delta Temperature).
Noise contained in observations that is larger than NEDT is not negligible. Systematic errors,
even much smaller compared to weather signals, would add to the observation error correlations.
And large error correlations prohibit the data to be properly assimilated. Major sources of
calibration errors include target emissivity, radiometric leakage, measurement uncertainty,
inappropriate use of Rayleigh approximation and antenna side lobe interception.
In order to smooth out the systematic noise, filters are designed and applied to observation
data. There are two types of digital filters: recursive filters and non-recursive filters. Those filters
are sometimes referred to as infinite impulsive filter and finite impulsive response filter,
respectively. The latter type has much lower order than the former one and thus much less
complicated. Fourier filters belong to recursive filters and averaging filters belong to nonrecursive filters.
2
1.2 Striping Noise Contained in ATMS Observations
Satellite microwave temperature and humidity sounders had played an important role in
numerical weather prediction for several decades. For satellite data assimilation, biases and errors
of brightness temperatures observations must be quantified. ATMS data characteristics and biases
were carefully discussed in Weng et al. (2012). A comprehensive discussion on ATMS calibration,
instrument sensitivity, conversion from antenna temperature to brightness temperature, and the
dependence of polarization difference on sea surface wind and sea surface temperature can be
found in Weng et al. (2013). The radiance simulated by an atmospheric radiative transfer model
with global numerical weather prediction (NWP) datasets can serve as the reference to verify the
observed radiance. Borman et al. (2013) and Qin et al. (2013) reported an along-track, striping
type of noise in the global distribution of differences between observations and model simulations
(O-B) at ATMS temperature sounding channels. A striping type of noise was found in the global
distribution of the differences between observations (O) and simulations (B) as in Fig. 1.1 (a),
suggesting a contamination of striping noise in ATMS observations. This striping pattern of this
noise is clearer in the distribution of zoomed-in portion as in (b).
Qin et al. (2013) further confirmed that similar striping noise exists in ATMS humidity
sounding channels and all other heritage humidity sounders (i.e., AMSU-B and MHS), but not
AMSU-A channels. Striping noise may be caused by receiver gain fluctuations (i.e., the 1/f noise)
that are associated mainly with the RF amplifiers (Ulaby et al., 1981; Hersman and Poe, 1981).
AMSU-A does not have any amplifiers in front of the receiver. Analysis of striping noise
structures and magnitudes as well as an effective mitigation of the striping noise from ATMS
brightness temperatures are necessary for maximizing the role of ATMS observations for NWP
3
and climate study. Such studies may also offer some insights on the root cause of the striping
noise.
1.3 Existing Methods to Mitigate Striping Noise
A detailed description of converting raw data counts to antenna brightness temperature
through ATMS calibration process is provided by Weng et al. (2012) and ATMS Advanced
Technical Baseline Documentation (ATBD). The antenna temperature is calculated with a twopoint calibration, in which the earth scene counts are converted into antenna temperature through
a linear relationship defined by warm counts, cold counts, warm load temperature and cold space
temperatures. Those raw counts data are voltage records taken during each scan cycle. A quadratic
term that accounts for the nonlinear relationship between antenna temperature and counts is also
added. In order to reduce the effect of radiometric instrument errors on antenna temperature,
warm counts, cold counts and warm load temperatures are traditionally smoothed in an
operational system using either a triangular or a boxcar filter.
Huang and Wu (2004; 2009) proposed an Ensemble Empirical Mode Decomposition
(EEMD) method for extracting from raw data different frequency components. For a given time
series, the EEMD uses maximum and minimum information of the riding waves in the set of
noise-added ensemble data series itself and extracts successively the oscillatory components,
which are called intrinsic mode functions (IMFs), from the highest to lowest frequencies. Unlike
the pre-existing harmonic base functions of Fourier Frequency Transform (FFT), the IMFs of
EEMD are locally adaptive basis functions that are extracted directly from the data, and thus
become physically meaningful representations. The EEMD method also works for data series
representing nonlinear processes. Wu and Huang had shown that Fourier spectra of IMFs
4
extracted from white noise series have similar and predictable patterns, confirming that the
EEMD method is capable of separating the true signals from the noise.
Qin et al. (2013) pointed out that striping noise are contained in the first Principle
Component of ATMS observations and applied the EEMD method to extract the ATMS striping
noise. This method not only can remove the striping noise in temperature sounding channels, but
also humidity channels for which the striping phenomenon is not visible in O-B distributions
because of larger ranges. Specifically, the first three high frequency IMFs are extracted from the
first PC coefficient of brightness temperatures. The removed striping noise has a frequency range
centered around 10-2 s-1 and has a magnitude of about 0.3 K for ATMS temperature channels and
1 K for ATMS humidity channels.
1.4 Dissertation Organization
This dissertation is organized as follows. Chapter 2 provides a description of all data used in
the work. Chapter 3 presents methodology on transformations from raw counts data to antenna
temperatures, as well as the mathematical expressions of EEMD/PCA and the “optimal” filters.
Numerical results of striping noise analysis and mitigation for both ATMS and MWTS brightness
temperatures are shown in Chapter 4, along with an investigation on the root cause of the striping
noise. Chapter 5 provides a brief summary and future plan.
5
(a)
(b)
Fig. 1.1: (a) Global distribution of O-B fields for ATMS channel 8 on February 24, 2012. (b)
Same as (a) but for a smaller area.
6
CHAPTER TWO
DATA
2.1 The Advanced Technology Microwave Sounder (ATMS)
ATMS is a cross-track, line-scanning sensor. It takes 2.67 seconds to complete a single
scan-line, which contains 96 FOVs for all 22 ATMS microwave temperature and humidity
sounding channels. With a sampling interval of 1.11o, the outmost scan-angle is 52.7o. ATMS
channels 1-3, 5-16 have similar central frequencies as AMSU-A channels 1-15 of the traditional
microwave temperature sounding instrument AMSU-A, and ATMS channels 17-22 contain
channels of the similar central frequencies as the five channels from the traditional humidity
sounding instrument MHS. Compared with its predecessor AMSU-A and MHS, ATMS has an
extra temperature channel 4 with its weighting function located in the lower troposphere and two
new humidity channels 19 and 21. The swath width of ATMS is 2500 km, which is wider than
both AMSU-A and MHS, leaving almost no data gaps over the global. The beam widths of
channels 1-2, 3-16 and 17-22 are 5.2o, 2.2o and 1.1o, respectively. ATMS data can be downloaded
from
NOAA
Comprehensive
Large
Array-data
Stewardship
System
(http://www.class.ncdc.noaa.gov/saa/products/welcome) in HDF format.
2.2 The Community Radiative Transfer Model (CRTM)
The Community Radiative Transfer Model (CRTM) was developed for rapid calculations of
satellite radiances and other derived variables under various atmospheric and surface conditions
by the US Joint Center for Satellite Data Assimilation (JCSDA) (Weng 2007). The model (Han et
al. 2007) is used in the work to produce global simulations of brightness temperatures. The
simulations are served as background field to verify the microwave observations.
7
Variables fed to CRTM are those from satellite observations such as latitude, longitude,
scan angle, zenith angle, and those from GFS data including surface type, surface temperature, 10
m wind speed; vertical profile of pressure, temperature and mixing ratio. The output of CRTM
includes brightness temperature, optical length, etc.
2.3 NCEP GFS Data
National Center for Environmental Prediction (NCEP)’s Global Forecast Systems (GFS) 6hour forecast fields are used to produce model simulations. Horizontal resolution of GFS data is
1ox1o and there are 26 vertical levels at 10.0, 20.0, 30.0, 50.0, 70.0, 100.0, 150.0, 200.0, 250.0,
300.0, 350.0, 400.0, 450.0, 500.0, 550.0, 600.0, 650.0, 700.0, 750.0, 800.0, 850.0, 900.0, 925.0,
950.0, 975.0, 1000.0 hPa, GFS data from March 2004 to present is available at
http://nomads.ncdc.noaa.gov/data/gfsanl/.
8
CHAPTER THREE
METHODOLOGY
3.1 Derivation of Antenna Temperature from Raw counts
During each ATMS scanning cycle of 2.67s, the antenna firstly scans the earth scenes, then
the cold space, and finally the blackbody warm target to record the measured scene counts, warm
counts and cold counts when these three segments are completed. A schematic illustration of
ATMS scan cycle is shown in Fig. 3.1.
The antenna brightness temperature can be derived from the measured raw counts based on
the following two-point calibration equations.
Tb,ch (k,i) ≡ Tchw (k) + (Gch (k))−1 (Cchs (k,i) − Cchw (k)) + Qch (k,i)
(1)
Cchw (k) − Cchc (k)
Gch (k) = w
Tch (k) − Tchc (k)
where the subscript “ch” represents the channel number, Gch (k) is the gain function, Cchs (k,i)
stands for scene count at the ith FOV of the kth scanline, Cchc (k) and Cchw (k) are cold count and
warm count at the kth scanline, respectively; Tchw (k) is the warm load temperature at the kth
scanline, and Tchc (k) is the cold space temperature, which is fixed at each channel. The overbar “
” on Tchw , Cchw and Cchc indicates a boxcar smoothing warm load temperature, warm count and cold
count, respectively. Qch (k,i) is a quadratic correction term which can be written as follows:
Qch (k,i) = b0,ch × (1− 4 × (
where b0,ch is a quadratic coefficient.
9
Tb,ch (k,i) − Tchc (k)
− 0.5)2 )
Tchw (k) − Tchc (k)
(2)
Both the warm load temperature Tchw (k) and the cold space temperature Tchc (k) have been
corrected by ΔTchw and ΔTchc , respectively, to account for channel biases. More details of ATMS
calibration process from raw data counts to antenna brightness temperature can be found in Weng
et al. (2013) and ATMS ATBD (2012).
3.2 Mathematical Expressions of Optimal Filters
3.2.1 Optimal Filters for Warm Counts and Cold Counts
{
} {
}
The method employed to extract the noise in the data series Cchw (k) , Cchc (k) and
{T
w
ch
}
(k) (k=1, 2, …, K) is the EEMD method (Wu and Huang, 2004). Here Cchw (k) , Cchw (k) and
Tchw (k) stand for the warm counts, cold counts and warm load temperatures, respectively, at the kth
scanline of channel “ch” indicated as the subscript.
For any time series of data, any of the above data series, to be denoted as ( C ( k ) , k=1, 2,
…, K) for simplicity, is decomposed into a set of “intrinsic mode functions” (IMFs):
L
C (k ) = ∑ IMFm (k ) + RL (k ) ,
(3)
m =1
where RL is the residual of the data C ( k ) after the first L number of IMFs have been extracted. In
order words, the signals in the data series are decomposed successively from high frequency to
low frequency. The high-frequency noise is described by the first few IMFs.
{
}
Therefore, the warm counts, cold counts and warm load temperature data series Cchw (k) ,
{C
c
ch
}
{
}
(k) and Tchw (k) (k=1, 2, …, K) can be smoothed by removing the first few high-frequency
IMFs to obtain the EEMD-smoothed data series Cchw,eemd (k) , Cchc,eemd (k) and Tchw,eemd (k) are as
follows:
10
Lw
Cchw,eemd (k) = Cchw (k) − ∑ IMFm (k)
(4a)
m=1
Lc
Cchc,eemd (k) = Cchc (k) − ∑ IMFm (k)
(4b)
m=1
Lt
Tchw,eemd (k) = Tchw (k) − ∑ IMFm (k)
(4c)
m=1
The above EEMD-smoothed warm counts, cold counts and warm load temperatures, Cchw,eemd (k) ,
Cchc,eemd (k) and Tchw,eemd (k) , will be used as the reference constraints for developing the optimal
filters for ATMS striping mitigation.
Instead of using the EEMD based method, we may apply an optimal (2N+1) point filter to
{
} {
}
{
}
each of the data series Cchw (k) , Cchc (k) and Tchw (k) (k=1, 2, 3, …, K) to obtain three
{
} {
}
{
}
smoothed data sets Cchw,opt (k) , Cchc,opt (k) and Tchw,opt (k) as follows:
C
w,opt
ch
N
∑α
(k) =
Cchw (k + n)
w
n
(5a)
n=− N
N
Cchc,opt (k) =
∑α C
c
n
c
ch
(k + n)
(5b)
(k + n)
(5c)
n=− N
N
Tchw,opt (k) =
∑αT
t w
n ch
n=− N
where α nw , α nc and α nt (n=0, ± 1, …, ± N) are the optimal weighting coefficients for the optimal
symmetrical filters on warm count, cold count and warm load temperature, respectively, and N is
the total number of scan lines involved in each filter. The method for obtaining the optimal
weighting coefficients is described below.
11
The effect of filter defined in equations (5a)-(5c) on the data can be examined by
comparing the spectrum of the original data series with that of the smoothed series. Taking the
warm counts as an example. Let’s apply Fourier transform to both the original data sequence
{C
w
ch
}
{
}
(k) and the smoothed data sequence Cchw,opt (k) :
K −1
C (k) = ∑ f me− imkβ
w
ch
(6a)
m=0
and
K −1
Cchw,opt (k) = ∑ f me− imkβ
(6b)
m=0
where β =
2π
, m is wave number (m=0, 1, …, K-1), f m and f m are Fourier coefficients which
K
can be written as:
1 K −1 w
Cch (k)e− imkβ
∑
K k=0
(7a)
1 K −1 w,opt
∑ C (k)e− imkβ
K k=0 ch
(7b)
fm =
and
fm =
The ratio of the Fourier coefficient of the filtered data series f m and the Fourier coefficient of the
filtered data series f m
rm =
fm
fm
is called the response function ( rm ) of the filter defined by equation (5a).
12
(8)
Once designed, the filter weighting coefficients are not updated regularly, and the filter are
applied to global data for both ascending and descending nodes. Thus the filter is symmetrical.
For a symmetrical filter, αn = α− n . Equation (5a) can be written as
Cchw,opt (k) =
1 N
α n (Cchw (k − n) + Cchw (k + n))
∑
2 n=0
(9)
By substituting (6a) and (7a) into (9) we may obtain
C
w,opt
ch
1 K −1 1 N
= ∑ ∑ α n (Cchw (k − n) + Cchw (k + n))e− imkβ
K k=0 2 n=0
N
= ∑ α n cos(mnβ ) f m
(10)
n=0
Substituting (10) into (8) we may then obtain an analytic expression for the spectral response
function rm :
N
rm = ∑ α nw cos(mnβ )
(11)
n=0
The weighting coefficients for the symmetric filters for warm counts are obtained by
requiring that the filter-smoothed warm counts approximately equal to the EEMD-smoothed ones,
i.e.,
K
N
⎧
min
J
=
min
(
α nwCchw (k + n) − Cchw,eemd (k))2
∑
∑
⎪
⎪
k=1 n=− N
⎨ N
⎪
∑ α chw = 1
⎪⎩ n=−
N
(12)
The constraint minimization problem in (12) is solved using Lagrange method:
N
⎧
w
L(
α
,
λ
)
=
J
+
λ
(1−
α nw )
∑
⎪
n
⎪
n=− N
⎨
⎪ ∂L = 0, ∂L = 0
w
⎪
∂λ
⎩ ∂α ch
13
(13)
Similar procedures are applied to cold counts and warm load temperatures.
3.2.2 Optimal Filters for Scene Counts
Different from warm counts, cold counts, and warm load temperatures, scene counts (i.e.,
the voltage records associated with observations at 96 FOVs) vary not only in the along-track
direction, but also the across-track direction. The striping noise remains nearly constant in the
along-track direction. A Principle Component Analysis (PCA) method is firstly used to extract the
first PCA component that captures mainly the scan-dependent features of scene counts from the
cross-track radiometer ATMS. The EEMD- and optimal- filtering approaches are applied to the
first PCA component.
The data matrix for the PCA analysis of scene counts can be expressed as:
C Ms ×N
s
s
⎛ Cs
C1,2
C1,N
1,1
⎜
s
s
s
C2,2
! C2,N
⎜ C
= ⎜ 2,1
"
#
"
⎜ "
s
s
s
⎜⎝ C M ,1 C M ,2 ! C M ,N
⎞
⎟
⎟
⎟
⎟
⎟⎠
(14)
where M is the total number of scanlines and N is the total number of FOVs.
The scene counts at the ith FOV of the kth scanline, Cks ,i , can be expressed as follows:
96
C = ∑ e j,iuk , j
s
k ,i
(15)
j=1
!
where e j,i is the ith (scan position) element of the jth PC mode e j , uk , j is the kth (scanline) element
!
of the jth PC coefficient u Tj .
The first L intrinsic mode functions (IMFs) of the first PC coefficient, which describe the
striping noise, are removed and the remaining PCA components are kept the same to obtain the
EEMD-smoothed scene counts
14
L
96
m=1
j=2
s,eemd
Ci,k
= ei,1 (u1,k − ∑ IMFm (k)) + ∑ ei, j u j,k
(16)
A (2N+1)-point filter can be applied to obtain a filter-smoothed scene count data series
{C } as follows:
s,opt
i,k
Cks,opt
=
,i
N
96
n=− N
j=2
∑ α nsei,1u1,k+n + ∑ ei, ju j,k
(17)
where α ns (n=0, ±1, …, ±N) are the optimal weighting coefficients for the scene counts using a
similar procedure described in section 3.2.1.
3.2.3 Optimal Filters for Brightness Temperatures
The data matrix of ATMS observed brightness is expressed as:
AM × N
⎛ y
! y N ,1
1,1
⎜
=⎜ "
#
"
⎜ y
⎝ 1,M ! y N ,M
⎞
⎟
⎟
⎟
⎠
(18)
where yk ,i is the brightness temperature observed at the ith FOV on the kth scan line. M is the total
number of FOVs, for ATMS, M=96; N is the total number of scan lines.
With a PCA method, the above data matrix could be decomposed into several PC modes,
as follows:
96
!!
A = ∑ e j u Tj
(19)
j=1
!
!
where the jth mode is the product of the PC coefficient u j and PC component e j . The PC
coefficient describes the along-track features while the PC component is a function of the FOV
(Qin et al., 2013). The first mode explains the largest portion of the total variance.
Equivalent to equation (19), brightness temperature at each single data point is written as:
15
96
yk ,i = ∑ e j,iuk , j
(20)
j=1
An EEMD method is used to remove striping noise in ATMS observations {yk ,i } to obtain
{
}
smoothed brightness temperatures ykeemd
, as in Qin et al. (2013). The first three intrinsic mode
,i
functions (IMFs) of the first PC coefficient uk ,1 are removed while the other PC components are
not changed, i.e.:
3
96
m=1
j=2
yk,ieemd = e1,i (ui,1 − ∑ IMFm (k)) + ∑ e j,i uk, j
(21)
L
eemd
The EEMD-smoothed first PC coefficient is expressed as: uk,1 = uk,1 − ∑ IMFm (k) , where L
m=1
indicates number of IMFs to remove.
Although the combination of EEMD and PCA methods is effective to detect and mitigate
the visible striping noise, it is not convenient to implement in operational systems. A set of
optimal filters that removes the striping noise extracted by PCA/EEMD is thus developed.
For the same set of ATMS observation data {yk ,i } as in equations (20), applying a (2N+1)
point filter to obtain a smoothed first PC coefficient data set {uk,1 } yields:
uk,1 =
N
∑α u
n k+n,1
(22)
n=− N
where α n (n=0, ± 1, …, ± N) are weighting coefficients and N is the total number of data points
involved in the filter.
16
We aim at obtaining the optimal weighting coefficients
α n (n=0, ± 1, …, ± N) so that the
filter-smoothed first PC coefficient approximately equal to the EEMD-smoothed ones, i.e., to
minimize the following cost function J:
K
N
eemd 2
J = ∑ ( ∑ α nuk+n,1 − uk,1
)
(23)
k=1 n=− N
And there are two constraint conditions:
N
∑α
=1
(24a)
α n = α −n
(24b)
n
n=− N
Equation (24a) indicates that mean value does not change, and equation (24b) requires that the
filters are symmetrical.
The constraint minimization problem stated in equations (23)-(24) is solved using Lagrange
method:
N
⎧
L(
α
,
λ
)
=
J
+
λ
(1−
∑ αn )
n
⎪
⎪
n=− N
⎨
⎪ ∂L = 0, ∂L = 0
⎪⎩ ∂α n
∂λ
(25)
!
where λ is the Lagrange multiplier. And the analytic solutions of weighting coefficients α can be
obtained by combining equations (24) and (25):
!
α = B −1C
(26)
where
⎛ α
0
⎜
!
α
α = ⎜⎜ 1
⎜ "
⎜⎝ α N
17
⎞
⎟
⎟
⎟
⎟
⎟⎠
(27a)
⎛
⎜
⎜
⎜
⎜
B = 2⎜
⎜
⎜
⎜
⎜⎝
K
∑u
K
∑u
2
k,1
k=1
k=1
K
∑u
∑u
u
k+1,1 k,1
K
∑u
!
2
k+1,1
k=1
"
#
"
K
∑u
u
k+N ,1 k,1
k=1
u
k+1,1 k+N ,1
k=1
"
K
u
k,1 k+N ,1
k=1
K
k=1
∑u
K
∑u
!
u
k,1 k+1,1
K
∑u
!
u
k+N ,1 k+1,1
k=1
2
k+N ,1
k=1
K
⎛
eemd
⎜ λ + 2∑ uk,1uk,1
k=1
⎜
K
⎜
eemd
⎜ λ + 2∑ uk+1,1uk,1
C=⎜
k=1
⎜
!
⎜
K
eemd
⎜ λ + 2∑ u
k+N ,1u k,1
⎜⎝
k=1
K
⎛
2
∑ uk,1uk,1eemd
⎜
k=1
⎜
K
⎜
eemd
⎜ 2 uk+1,1uk,1
D=⎜ ∑
k=1
⎜
!
⎜
K
⎜ 2∑ u
u eemd
⎜⎝ k=1 k+N ,1 k,1
⎞
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟⎠
⎞
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟⎠
⎞
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟
⎟⎠
(27b)
(27c)
(27d)
With the Lagrange multiplier solved as:
N +1 N +1
λ=
1− ∑ ∑ (B −1 )ij D j
i=1 j=1
N +1 N +1
∑ ∑ (B
−1
(28)
)ij
i=1 j=1
And the optimally smoothed brightness temperatures could then be expressed as:
y
opt
k,i
= e1,i
N
∑α u
n k+n,1
n=− N
18
96
+ ∑ e j,i uk, j
j=2
(29)
Fig. 3.1: A schematic illustration of ATMS scan cycle.
19
CHAPTER FOUR
ATMS STRIPING NOISE MITIGATION USING THE OPTIMAL FILTERS
4.1 Striping Noise Mitigation Within Sensor Calibration Counts
4.1.1 Spectrum Analyses of IMFs of Calibration Counts
Data series of warm counts, cold counts, warm load temperature and the first PC
coefficient of scene counts for all ATMS channels within 55S-55N on February 24, 2012 are
decomposed into different IMFs. The Fourier spectra of the first six IMFs with respect to
frequency for warm and cold counts at channel 8 are shown in Fig. 4.1. It is seen the higher the
IMF number is, the lower frequency the peak amplitude locates. White noise and flicker noise at
high frequencies are captured by the first few IMFs, while the large-scale variations are in the
remaining IMFs. A main criterion to decide which IMFs are noise, or how many IMFs to remove,
is whether the peak amplitude of a certain IMF is at the striping frequency range (>0.01 Hz).
Also, magnitudes of the noise-related IMFs are not too large, otherwise weather signal might be
removed as well. For both warm counts and cold counts, beginning at the fourth IMF, peak
amplitude is no longer smaller than that of the previous IMFs. It is thus determined that the first
three IMFs are the data noise. Also magnitudes of those three IMFs at low frequencies are quite
small, suggesting that removing the first three IMFs will not change the features of warm counts
and cold counts at the low frequencies. Similar results are obtained for the other 21 ATMS
channels. Thus it is decided the first three IMFs are removed for both warm counts and cold
counts at all ATMS channels.
The Fourier spectra of the first six IMFs for warm load temperature are shown in Fig. 4.2.
Notice that the 6th IMF peaks at close frequency to the 5th IMF, but with obviously smaller
amplitude, which suggests that starting at the 6th IMF there is no more significant signal at even
20
lower frequency. And unlike warm counts or cold counts that change in the along-track direction,
ideal warm load temperature should be constant, implying a zero Fourier spectra for the true
signal. Thus a total of five IMFs are removed from warm load temperatures.
IMFs are removed.
Fourier spectra of the first six IMFs of the first PC coefficient for scene counts at channel
8 are shown in Fig. 4.3. It is seen that the 4th IMF has much larger magnitudes than the third IMF
not only at the peak but also at the low frequencies. Thus to remove the first three IMFs is enough
to eliminate the noise while the large-scale information is kept. Similar analyses are carried out
for 22 channels. The total numbers of IMFs to be removed from scene counts for all ATMS
channels are listed in Table 4.1. For ATMS window channels 1-2 and 16, only two IMFs are
sufficient to capture the data noise and need to be removed. For the remaining channels, three
IMFs are to be removed.
4.1.2 Properties of the Optimal Symmetrical Filters
As described in sections 3 and 4, the EEMD-smoothed warm counts, cold counts, warm
load temperature and scene counts are then taken as standards for constructing optimal filters as
expressed in equations (5) and (17). In order to decide the filter span N, weighting coefficients
and cost functions of (2N+1) point symmetrical optimal filters with different span ranging from 2
to 30 are all calculated. Cost functions describe how close the counts smoothed by EEMD to
those by the optimal filter. Since cost functions at different channels have different magnitudes,
variations of the normalized cost functions of the optimal filters with respect to filter span are
shown in Fig. 4.4 for warm counts, cold counts and scene counts at all ATMS channels. It is seen
cost functions drop rapidly with the filter span N, suggesting that optimal filters tend to capture
the EEMD feature more closely if more scan lines are involved. It is hoped the striping noise be
21
removed with the smallest possible numbers of scan lines needed in the optimal filter. The chosen
filter spans of the optimal filters are indicated in black circles and are shown in Tables 4.1 and
4.2. Surface channels have narrower filters mainly because there are only two IMFs removed so
that the changes to counts data are smaller. Temperature sounding channels have wider filters
than humidity sounding channels for warm counts and cold counts, but narrower filters for scene
counts.
Weighting coefficients of the optimal filters for warm counts, cold counts and scene
counts at all channels are presented in Fig. 4.5. All weighting coefficients are of symmetrical
parabolic shapes, suggesting that data points with identical distance to the filtering point weigh
the same within any filter, and those closer to the filtering point (n=0) are weighed more than data
points further away. This result indicates the filtering points themselves carry most importance
while data points further away have less impact on the smoothed counts, which is reasonable.
Weighting coefficients with the same distance to filtered points are of similar magnitudes for
warm counts and cold counts at different channels. Scene counts at channels 1-2 and 16 have
larger weighting coefficients than those for the other channels because sum of the weighting
coefficients of any filter is equal to unit and a fewer scan lines are involved in these three window
channels.
Figure 4.6 shows the response functions calculated with the optimal weighting coefficients
shown in Fig. 4.5 according to equation (11). For warm counts, cold counts and scene counts at
all channels, response functions remain to have a unit value until they drop sharply to around zero
at the frequencies of 10-2 s-1. This indicates that the magnitudes of low-frequency waves are not
altered while the high-frequency noise is significantly reduced. The optimal filters make very
little changes to the counts data at low frequencies, and the majority of the noise removed
22
centered greater than the frequency of 10-2 s-1, which implies that optimal filters removes the
striping information while retaining large-scale signals. Contours over dark blue shading area all
have the value of 0.01 because of oscillations of response functions. For scene counts, response
functions at the window channels 1-2 and 16 start to decrease from 1 to zero at higher
frequencies, suggesting that more signals are retained, which agree with the fact that only two
IMFs are removed, compared to three IMFs at other channels.
Although corrections might be different among ATMS channels, warm load temperatures
do not change with channel numbers. The cost function, weighting coefficients and response
functions for warm load temperatures are shown in Fig. 4.7.
4.1.3 Comparison between the Optimal Filters and the Boxcar Filters
Figures 4.8 and 4.9 show the warm counts and cold counts before and after applying
optimal filters at channels 8 and 22. It is seen that after the optimal filters are applied, warm
counts and cold counts become much smoother without any visible noise at both channels while
the larger-scale variations are kept. This proves that optimal filters are effective.
Conventionally 17-point boxcar filters are used to smooth warm counts and cold counts.
Variations of warm counts and cold counts after applying both the optimal filter and the boxcar
filter are shown in Fig. 4.10. Compared with the optimal filters, the boxcar filter not only fails to
smooth out all visible noise, but also reduces the minimums and maximum values.
Response functions for the optimal filters and the boxcar filters for channel 8 with the same
filter widths of 8 are also calculated. Figure 4.11 shows that the response function for the optimal
filter fall off from 1 to around 0 when frequency increases, which means after smoothing,
magnitudes of low frequency signals are kept the same while that of high frequencies are
significantly reduced. However, response function for the boxcar filter starts to decrease
23
dramatically at the frequencies lower than 10-2 s-1, implying that the boxcar filter suppresses the
low-frequency signals that should be retained. Another problem is that at the high frequency
range, where the major part of noise exists, the oscillatory behavior of the response function of
boxcar filters hinders this filter from completely removing the actual data noise.
4.1.4 The Effects of Optimal Filters on Antenna Temperatures
Conventionally antenna temperatures are calculated with only warm counts, cold counts
and warm load temperature smoothed while the scene counts are not smoothed. Figure 4.12
shows antenna temperatures for ATMS channel 8 at nadir calculated with and without applying
the optimally filters to the scene counts. For both cases, all of warm counts, cold counts and warm
load temperatures are smoothed using the optimal filters. It is seen that even if noise in warm
counts, cold counts and warm load temperature are all effectively smoothed out, the antenna
temperatures are still contaminated with striping noise if the scene counts are not smoothed. This
confirms the necessity of smoothing the scene counts.
Global distributions of the removed noise in antenna temperatures for temperature
sounding channel 8, humidity sounding channel 22 and window channel 1 are shown in Fig. 4.13.
The striping noise are around 0.3 K for channel 8 and 1K for channels 22 and 1, which agrees
with the EEMD results in Qin et al. (2013).
4.2 Striping Noise Mitigation Within Brightness Temperature
4.2.1 Weighting Coefficient of the Optimal Filters
Data series of brightness temperatures at all 22 ATMS channels within 55S-55N on February
24, 2012 are used as sample data to calculate for weighting coefficients of optimal filters. Similar
to Qin et al. (2013), the first PC coefficients are first decomposed into different IMFs with the
PCA/EEMD methods. It is decided that two IMFs are removed for window channels 1, 2 and 16
24
and three IMFs are removed for all the rest channels. Those EEMD-smoothed first PC coefficients
are then served as the references for establishment of optimal filters.
Cost functions calculated by equation (23) for all ATMS channels with respect to filter
widths N re shown in Fig. 4.14. The 22 channels are divided into four groups, which are window
channels 1, 2 and 16, tropospheric temperature sounding channels 3-9, stratospheric temperature
sounding channels 10-15, and water vapor channels 17-22. It is seen with the increase of filter
spans, cost functions drop sharply at first for all channels, and then decrease slowly until finally
change little. This behavior suggests that the optimally smoothed first PC coefficients are closer to
EEMD-smoothed ones when more scan lines are involved in the filters. However, for sufficiently
wide filters, even if more scan lines are added, the differences between EEMD-smoothed and
optimally smoothed first PC coefficients remain the same. Those sufficiently large filter spans are
then chosen as the filter spans of our optimal filters, which are indicated by circles. The behaviors
of cost functions for different channels are similar, but the ranges of magnitudes are large. Cost
functions values are divided by the cost function with filter span N=2 at each channel to obtain
normalized cost functions as shown. The magnitudes of cost functions with filter spans of two are
shown in Fig 4.14(e). The filter spans of optimal filters for all 22 channels are as listed in Table
4.3.
Weighting coefficients of the optimal filters with respect to distances (n) to the filtering
point, where n=0, are demonstrated in Fig. 4.15. It is seen that weighting coefficients at the points
closer to the filtering points are weighed more than those at the points further away, which implies
that the points with smaller distances to the filtering points carry more importance and have larger
impacts on the smoothed data than those points with larger distances. Two IMFs are removed for
window channels 1, 2 and 16 and three IMFs are removed for other channels. Weighting
25
coefficients at filtering points for these three window channels are larger than other channels,
which indicates that less changes are made to the data, and the filtering points themselves are
weighed more.
Spectral response functions as in equation (11) are shown in Fig. 4.16. For all 22 channels,
response functions are unit values at the low-frequency range, meaning the magnitudes of largescale waves at that frequency are kept un-altered. At frequencies greater than 10-2 s-1, where the
diagnosed striping noise resides (Qin et al., 2013), the response functions plumb to zero, implying
the magnitudes of the high-frequency noise is significantly reduced.
4.2.2 Brightness Temperatures before and after Striping Noise Mitigation
The above 22 optimal filters for the 22 ATMS channels are applied to brightness
temperature according to equation (29). Spectra of differences between observed and simulated
brightness temperatures (i.e., O-B differences) before and after applying optimal filters for ATMS
channels 9 and 10 are shown in Fig. 4.17. Here, simulated brightness temperatures are generated
by using Community Radiative Transfer Model (CRTM) with GFS 6-hr forecast field. It is seen
that magnitudes are not changed at all frequencies except for the deep decreases at the frequencies
greater than 10-2 s-1, which is the lower frequency boundary of the identified striping noise. This
result indicates that the majority of signals removed by optimal filters is at the same frequencies
as EEMD/PCA and confirms that the optimal filters are effective alternatives to the EEMD/PCA
approach.
Global distributions of the O-B differences before and after applying the optimal filters for
ATMS channel 8 are displayed in Fig. 4.18. It is seen that the once obvious striping noise
contained in the O-B are not detectable any more with noise mitigation using the optimal filter.
The distribution of removed noise is displayed in Fig. 4.18(c), which are of a magnitude of 0.3K
26
and in a striping pattern. Global distributions of removed noise at channels 1, 8 and 22 are shown
Fig. 4.19. The magnitudes of striping noise for window channels, temperature channels and
humidity channels are 1 K, 0.3 K and 1K, respectively.
A small reduction in the standard deviation of O-B results from the striping noise filtering.
For example, the standard deviations of the global O-B fields before striping mitigation are 0.484
K and 0.471 K with and without subtracting scan biases, respectively, using data on February 24,
2012. These two numbers are reduced to 0.476 K and 0.463 K, respectively, after filtering striping
noise. If a spatial average is applied to the ATMS temperature sounding channels before their
assimilation into NWP models, the reduction due to the striping noise filtering may be more
significant.
In order to further validate the optimal filters, similar approach as described in Section 2 is
applied on ATMS pitch-over maneuver data. Acquired by only scanning the cold space, the pitchover maneuver data are not affected by weather systems and thus well reflect the noise-related
characteristics of the instrument itself (Weng et al., 2013; Qin et al., 2013). Pitch-over maneuver
data at Channels 1 and 9 before and after applying optimal filters are shown in Fig. 4.20. It is seen
that after smoothing, the striping noise contained in the pitch-over maneuver data are significantly
reduced and contain mainly random noise signals for both channels.
A striping index (SI) for pitch-over maneuver data is defined as follows (Qin et al., 2013):
SI =
σ along−track
σ cross−track
(30)
where σ along−track and σ cross−track stand for along-track variances and cross-track variances,
respectively. Data contaminated with striping noise have a much larger along-track variance than
cross-track variance, and SI exceeds one. By comparison, magnitude of along-track variance for
27
data free from striping noise is of close to that of cross-track variance. Such an ideal SI for
striping-noise-free data tends to be around unit value. Along-track variances, cross-track
variances and SIs before and after applying the optimal filters for pitch-maneuver data at all 22
ATMS channels are shown in Fig. 4.21. It is seen that after applying the optimal filters, alongtrack variances decrease sharply while cross-track variances change little, and SIs decreases to
unit values at each channel, which again confirms the effectiveness of optimal filters.
4.2.3 Liquid Water Path Retrieval Algorithm
The cloud liquid water path (LWP) is the total vertically integrated mass of liquid water per
unit horizontal area in units of kg/m2. A general expression of LWP (L) is written as:
ztop
L=
∫
ρ l dz
(31)
zbottom
where ρ l is the liquid water density.
LWP can be retrieved using measurements at AMSU window channels 1 and 2, as well as
surface temperature, surface emissivity, satellite zenith angle (Weng et al., 1994; Weng et al.,
2003). The algorithm is expressed as follows:
L = a0 µ[ln(Ts − TbCh2 ) − a1 ln(Ts − TbCh1 ) − a2 ]
(32)
where TbCh1 and TbCh2 are brightness temperatures observed at AMSU-A channels 1 and 2,
respectively, Ts is surface temperature, µ is the cosine function of zenith angle. a0 , a1 and a2 are
coefficients as follows:
a0 = −0.5κ vch1 / (κ vch1κ lch2 − κ vch2κ lch1 )
a1 = κ vch2 / κ vch1
(33a)
(33b)
a2 = −2.0(τ och2 − a1τ och1 ) / µ + (1.0 − a1 )ln(Ts ) + ln(1.0 − ε ch2 ) − a1 ln(1.0 − ε ch1 ) (33c)
28
where κ v and κ l are mass absorption coefficients for water vapor and cloud liquid water,
respectively, ε is the surface emissivity, τ o is the oxygen optical thickness and is taken as a
constant for a given channel. Rational for the linear relationship between LWP and logarithm of
window channel observations is presented as follows.
Wavelength of microwave is at least 50 times larger than a typical rain droplet, so even at
the low end with a wavelength of 3mm, the scattering effect could be neglected. With further
assumption that atmosphere is isothermal, at the frequencies below 40 GHz, for both window
channels 1 and 2, there exists the following relationship between observations and surface
temperatures:
Tb = Ts [1− τ 2 (1− ε )]
(34)
where the optimal depth ( τ ) is written as the product of the optimal depths of water vapor ( τ H 2O ),
oxygen ( τ O ) and liquid water droplets ( τ liquid ) as follows:
τ = τ h2oτ oτ liquid
(35)
with
τ H 2O = exp(− ∫
ztop
zbottom
κ v ρ v (z)dz ) = exp(−κ v × PW )
(36)
ρ v representing the density of water vapor, and PW the total precipitable water vapor.
For non-precipitating clouds, liquid water transmittance relies not on droplets sizedistribution but only on LWP, temperature and frequencies (Grody, 1976):
τ liquid = exp(−Kν 2 L)
(37)
where ν is the channel central frequency, K is a coefficient which is a function of only mean
cloud temperature. One way to interpret Equation (37) is to calculate optimal depth for liquid
water droplet similar to Equation (36), which is written as:
29
1
κ l ρ lVl (z)dz ) = exp(−κ l × L)
zbottom σ
τ liquid = exp(− ∫
ztop
(38)
The above equation is only valid when mass absorption coefficient is independent of droplet
radius (r). Given the expression of mass absorption coefficient κ l :
κl =
Qaπ r 2
3Qa
=
3
ρ l (4 / 3)π r
4 ρl r
(39)
where r is the radius of the droplet, Qa is the absorption efficiency, which is expressed as follows
so that κ l for a single droplet relies on the droplet size (Petty, 2006):
⎧ m2 − 1
x 2 m 2 − 1 m 4 + 27m 2 + 38 ⎫ 8 4 ⎧ m 2 − 1 2 ⎫
Qa = 4xℑ ⎨ 2
[1+ ( 2
)
] ⎬ + x ℜ ⎨( 2
) ⎬
15 m + 2
2m 2 + 3
⎩m + 2
⎭ 3
⎩ m +2 ⎭
with the relative refractivity m =
water droplet, respectively; x =
(40)
N2
, where N1 and N 2 the refractivity indices for air and liquid
N1
2π r
is the ratio of droplet perimeter over instrument channel
λ
wavelength λ . As in Equations (39)-(40), the mass absorption coefficient varies with droplet size
so that Equation (38) does not allow accurate estimation of LWP. However, for sufficiently small
droplet ( x ≪ 1 ), Qa is proportional to r:
Qa =
8π r ⎧ m 2 − 1 ⎫
ℑ⎨
⎬
λ ⎩ m2 + 2 ⎭
(41)
Equations (39) and (41) thus yield a mass absorption coefficient κ l independent of the droplet
size, when the droplet radius is sufficiently small. This simplifies the calculation of water droplet
optical depth to Equation (38).
Equations (34)-(41) show both TPW and LWP are in linear relationships with logarithm of
brightness temperatures observed. For the two unknown variables, mathematically only two
equations, or observations at any two channels suffice to obtain solutions. However, more desired
30
observations are from such channels with significantly different focuses on TPW and LWP
separately, otherwise error matrix of the two variables might grow dramatically (Petty, 2006). The
center frequency of AMSU channel 1 is at the water vapor absorption line and is dominant in
retrieving the water vapor content; channel 2 is at a frequency far from absorption line and not
sensitive to water vapor, but is heavily affected by liquid water. Those two channels are thus an
ideal pair to calculate for LWP and TPW with Equations (32)-(33).
With the same center frequencies, ATMS window channels 1 and 2, which are analogous to
AMSU-A channels 1 and 2, respectively, can be employed to retrieve LWP with similar
algorithm. Details of LWP retrieval algorithm with ATMS window channel observations are
found in Weng et al. (2012).
4.2.4 Impact of Striping Noise on ATMS LWP Retrievals
Noise removed by optimal filters at channels 1 and 2 within 30S-30N on February 24, 2014
is shown in Fig. 4.22. Visible striping patterns with magnitudes of no more than 1 K are
demonstrated for both channels. Global distributions of LWP retrieved using ATMS window
channels 1 and 2 with and without striping noise mitigation are displayed in Figs. 4.23-24. Other
variables involved in the retrieving are surface temperature, surface emissivity and instrument
zenith angle. Weather systems captured in both LWP retrievals (Figs. 4.23a-b, 4.24a-b) have
similar features, suggesting that the impact of the striping noise mitigation on LWP retrieval is
rather small. However, differences of LWP before and after striping noise mitigation in channels
1 and 2 (Figs. 4.23c) reveal some striping patterns over places where there exist clouds (Fig.
4.23a,) and striping noise in channels 1 and 2 (Figs. 4.23d-e). Moreover, the LWP differences
could be more than 0.05 kg/m2, which is greater than the estimated error of 0.01 kg/m2 in the
LWP retrieval. These results confirm that the striping noise mitigation does not affect small-scale
31
features in the LWP fields and yet it is important to remove the striping noise in ATMS
brightness temperatures. It is noted that even the LWP differences exhibit striping pattern.
Furthermore, the magnitude of the LWP differences varies with FOVs along the same scan line.
This suggests that the LWP differences are not simply a linear combination of the striping noise
at channels 1 and 2
Figure 4.25 shows the probability density functions (PDFs) of LWP differences, which are
color-coded by magnitudes of LWP retrieval or striping noise at the two ATMS window channels.
All PDFs are in Gaussian distributions, with the majority of LWP differences within the range of
±0.02kg/m2. As in (a), the PDF distribution of LWP difference is symmetrical within each LWP
magnitude interval, which indicates that the positive and negative LWP differences occur equally
frequently. However, as in (b) and (c), the PDFs are not symmetrical within each noise interval at
channels 1 and 2. It is seen for both window channels the positive striping noise is associated with
frequent occurrences of positive LWP differences, and vice versa. The result in Fig. 12 implies that
amplitude of striping-induced LWP differences is related to the striping noise in brightness
temperatures at window channels.
To further investigate the relationships between the removed striping noise within window
channel brightness temperatures and the LWP differences, and to expel the possibility that
striping-induced LWP noise is dependent on LWP retrieval amplitudes, scattering plots of LWP
differences with respect to window channel striping noise or LWP retrieval magnitudes are
provided in Fig. 4.26. Scattering plots are color-coded with LWP retrievals in (a) and (b), and it is
seen no mater viewed as a whole or at each LWP interval, the larger the striping noise, the larger
the LWP differences, and the relationships are linear. With larger LWP retrievals, the linear
relationships have steeper slopes. Scattering plots of LWP differences with respect to magnitudes
32
of LWP retrievals and color-coded with channel 1 and 2 noise are shown in (c) and (d). It is seen
the distribution of the scattering points is random, and the relationships between LWP noise and
LWP retrieval magnitudes vary a lot among different striping noise intervals. This randomness
indicates that the LWP noise is independent of the LWP retrieval magnitudes. However, for a fixed
value at channel 1 or 2, it is seen that if the LWP retrieval is larger, the LWP striping is stronger.
This agrees with the fact that LWP differences are not identical along the same scan line. The
impact of striping noise of brightness temperatures observed at ATMS window channels on LWP
retrievals addresses the necessity of removing the striping noise contaminated in ATMS brightness
temperatures, and this impact also reveals the benefits the striping noise mitigation brings to NWP.
33
Table 4.1: Numbers of IMFs removed and filter spans of optimal striping filters for ATMS scene
counts at each channel.
Central
Channel
Number of
Filter span
Peak WF
Frequency
Number
IMFs removed
(N)
(hPa)
(GHz)
1
23.8
2
14
Surface
2
31.4
2
14
Surface
3
50.3
3
23
Surface
4
51.76
3
23
950
5
52.8
3
18
850
6
53.596±0.115
3
17
700
7
54.4
3
19
400
8
54.94
3
17
250
9
55.5
3
17
200
10
57.2903
3
16
100
11
57.2903±0.115
3
18
50
12
57.2903
3
18
25
13
57.2903±0.322
3
18
10
14
57.2903±0.322±0.010
3
20
5
15
57.2903±0.322±0.004
3
17
2
16
88.20
2
16
Surface
17
165.5
3
23
Surface
18
183.31±7
3
22
800
19
183.31±4.5
3
22
700
20
183.31±3
3
22
500
21
183.31±1.8
3
22
400
22
183.31±1.0
3
23
300
34
Table 4.2: Filter spans of optimal striping filters for ATMS warm counts and cold counts.
Central
Warm Count
Cold Count
Frequency
Filter span
Filter span
(GHz)
(N)
(N)
1
23.8
8
8
Surface
2
31.4
8
8
Surface
3
50.3
10
10
Surface
4
51.76
10
10
950
5
52.8
8
10
850
6
53.596±0.115
8
10
700
7
54.4
8
10
400
8
54.94
8
10
250
9
55.5
10
10
200
10
57.2903
8
10
100
11
57.2903±0.115
10
10
50
12
57.2903
10
10
25
13
57.2903±0.322
10
10
10
14
57.2903±0.322±0.010
10
10
5
15
57.2903±0.322±0.004
10
10
2
16
88.20
8
8
Surface
17
165.5
8
8
Surface
18
183.31±7
8
8
800
19
183.31±4.5
8
8
700
20
183.31±3
8
8
500
21
183.31±1.8
8
8
400
22
183.31±1.0
8
8
300
Channel
Number
35
Peak WF
(hPa)
Table 4.3: Numbers of IMFs removed and the chosen filter spans for ATMS channels.
Channel
Number
Central
Frequency
(GHz)
Number of
Filter span
Peak WF
IMFs removed
(N)
(hPa)
1
23.8
2
14
Surface
2
31.4
2
14
Surface
3
50.3
3
23
Surface
4
51.76
3
22
950
5
52.8
3
18
850
6
53.596±0.115
3
17
700
7
54.4
3
19
400
8
54.94
3
17
250
9
55.5
3
17
200
10
57.2903
3
16
100
11
57.2903±0.115
3
18
50
12
57.2903
3
18
25
13
57.2903±0.322
3
18
10
14
57.2903±0.322±0.010
3
20
5
15
57.2903±0.322±0.004
3
17
2
16
88.20
2
16
Surface
17
165.5
3
22
Surface
18
183.31±7
3
22
800
19
183.31±4.5
3
22
700
20
183.31±3
3
22
500
21
183.31±1.8
3
22
400
22
183.31±1.0
3
23
300
36
Amplitude
(a)
Frequency (s-1)
Amplitude
(b)
Frequency (s-1)
Fig. 4.1: Fourier spectra with 81-point running mean for the first six IMFs of (a) warm counts and
(b) cold counts at channel 8.
37
Amplitude
Frequency (s-1)
Fig. 4.2: Same as Fig. 4.1 but for warm load temperature. Five IMFs are removed.
38
Amplitude
Frequency (s-1)
Fig. 4.3: Same as Fig. 4.1 but for the first PC coefficient of scene counts at channel 8. Three IMFs
are removed.
39
Channel
Warm counts
N
Channel
Cold counts
N
Channel
Scene counts
N
Fig. 4.4: Normalized cost function with respect to filter span N for warm counts (upper), cold
counts (middle) and scene counts (lower). Numbers of scan lines involved in the optimal
filter is indicated in circles.
40
Weighting Coefficient
Warm counts
n
Channel
Weighting coefficient
Cold counts
n
Channel
Weighting coefficient
Scene counts
n
Channel
Fig. 4.5: Weighting coefficients of optimal filters for warm counts (upper), cold counts (middle)
and scene counts (lower).
41
Channel
Warm counts
Frequency (s-1)
Channel
Cold counts
Frequency (s-1)
Channel
Scene counts
Frequency (s-1)
Fig. 4.6: Response function for warm counts (upper), cold counts (middle) and scene counts
(lower) calculated with weighting coefficients as in Fig. 5. Contours over dark blue
shading area all have the value of 0.01 because of oscillations of response functions.
42
Cost function
(a)
N
Weighting coefficient
(b)
n
Fig. 4.7: (a) Normalized cost function with respect to filter span N for warm load temperature.
Numbers of scan lines involved in the optimal filter is indicated in circle. (b)-(c) weighting
coefficients and response function for warm load temperature.
43
Response function
(c)
Fig. 4.7 Continued
Frequency (s-1)
44
Warm Counts
(a)
Scan line
Cold Counts
(b)
Scan line
Fig. 4.8: Variations of (a) warm counts and (b) cold counts along 700 scanlines (i.e., a swath
between ±55N ) before (gray) and after (red) applying removing three IMFs at channel 8.
45
Warm Counts
(a)
Scan line
Cold Counts
(b)
Scan line
Fig. 4.9: Variations of (a) warm counts and (b) cold counts along 700 scanlines (i.e., a swath
between ±55N ) before (gray) and after (red) applying removing three IMFs at channel
22.
46
Warm Counts
(a)
Scan line
Cold Counts
(b)
Scan line
Fig. 4.10: Variations of (a) warm counts and (b) cold counts with optimal filter (red) and 17-point
smoothing (blue) for channel 8.
47
Response function
Frequency (s-1)
Fig. 4.11: Variation of response functions with respect to frequency for the 17-point filter (blue)
and the optimal filters (red) applied to warm counts of channel 8 (curve) and channel 22
(circle). The response functions for cold counts are the same as those for warm counts
(figure omitted).
48
Brightness temperature (K)
Scan line
Fig. 4.12: Variation of brightness temperature for channel 8 at nadir with (red) and without (blue)
applying the optimal filter on the scene counts. Warm count, cold counts and warm load
temperatures are all smoothed.
49
Channel 8
Channel 22
Channel 1
Fig. 4.13: Striping noise removed by the optimal symmetric filters for ATMS channels 8, 22 and
1 on ascending node.
50
Cost Function
(a)
Cost Function
N
(b)
Cost Function
N
(c)
N
Fig. 4.14: (a)-(d) Variation of the normalized cost function with respect to filter span N for ATMS
channels 1-22. The cost function value with the selected symmetrical filter span (2N+1) is
indicated by circle. (e) The cost function for N=2 used for normalization in (a)-(d) for
each channel.
51
Cost Function
(d)
Cost function (K2)
N
(e)
Channel Number
Fig. 4.14 Continued
52
Weighting Coefficients
Weighting Coefficients
n
Weighting Coefficients
n
n
Fig. 4.15: Weighting coefficients of symmetric filters for ATMS channels 1-22.
53
Weighting Coefficients
Fig. 4.15 Continued
Number of Scan line
54
Response coefficients
Response coefficients
Frequency (s-1)
Response coefficients
Frequency (s-1)
Frequency (s-1)
Fig. 4.16: Response functions of symmetric filters calculated with weighting coefficients in Fig.
4.15.
55
Response coefficients
Frequency (s-1)
Fig. 4.16 Continued
56
(b)
Amplitude
Amplitude
(a)
Frequency (s-1)
Frequency (s-1)
Fig. 4.17: The power spectral density distributions of the global O-B field of ATMS channels (a)
9 and (b) 10 before (red) and after (blue) removing the striping noise using the optimal
filter.
57
(a)
(b)
(c)
Fig. 4.18: Global distribution of O-B fields (a) before and (b) after applying the optimal filter for
ATMS channel 8 on February 24, 2012. (c) Global distribution of striping noise, i.e., the
difference between (a) and (b).
58
(a)
(b)
(c)
Fig. 4.19: Striping noise contained in the observaions for ATMS channel (a) 1, (b) 8 and (c) 22 on
February 24, 2012.
59
iance
Scan line
(b)
iance
Scan line
(a)
FOV
FOV
iance
Scan line
(d)
iance
Scan line
(c)
FOV
FOV
Fig. 4.20: Maneuver data (a) before and (b) after optimal smoothing on PCA1 for ATMS channel
1. (c)-(d) Same as (a)-(b) but for channel 9.
60
VDT
(a)
Channel
VCT
(b)
Channel
SI
(c)
Channel
Fig. 4.21: (a) The along-track variance, (b) across-track variance and (c) striping index before
(red) and after (blue) smoothing. FOVs 25-72 on 100 scan lines are involved in
calculation.
61
Ch1, 0000-1200 UTC
Ch2, 0000-1200 UTC
Ch1, 1200-2400 UTC
Fig. 4.22: Global distributions of striping noise removed by the optimal filter for ATMS channels
1 and 2 on February 24, 2012.
62
Ch2, 1200-2400 UTC
Fig. 4.22 Continued
63
(a)
(b)
(c)
Fig. 4.23: Global distribution of LWP retrieved by ATMS channels 1-2 observation (a) without
and (b) with smoothing and (c) the LWP differences between (a) and (b) during 00001200 UTC on February 24, 2012.
64
(a)
(b)
(c)
Fig. 4.24: Same as Fig. 4.23 but for 1200-2400UTC.
65
PDF (%)
(a)
×kg/m2
LWP Difference (kg/m2)
(c)
PDF (%)
PDF (%)
(b)
LWP Difference (kg/m2)
LWP Difference (kg/m2)
Fig. 4.25: Probability density functions of LWP difference within 30S-30N over ocean on
February 24, 2012. (a) LWP retrieved by observations, (b) noise removed at ATMS
channel 1 and (c) noise removed at ATMS channel 2 are color coded, respectively. The
mean and std of the LWP differences with and without smoothing (i.e., LWPwithoutLWPwith) within 30S-30N over ocean on February 24, 2012 are zero and 0.005 kg m-2.
66
(b)
(kg/m2)
Difference (kg/m2)
Difference (kg/m2)
(a)
(kg/m2)
Ch1 striping noise (K)
Ch2 striping noise (K)
(c)
Difference (kg/m2)
Difference (kg/m2)
(d)
(K, ch1)
(K, ch2)
LWP (kg/m2)
LWP (kg/m2)
Fig. 4.26: Scatter plots of the LWP differences with respect to (a)-(b) striping noise of channels 1
and 2 and (c)-(d) LWP retrieved with observations without smoothing. The magnitude of
LWP is indicated in (a) and (b) and the magnitude of striping noise of channels 1 and 2 is
indicated in (c) and (d), respectively. Data within 30S-30N over ocean on February 24,
2012 are used.
67
CHAPTER FIVE
STRIPING NOISE MITIGATION FOR FY-3C MWTS DATA
5.1 Characterzation and Mitigation of Striping Noise in FY-3C MWTS Data
5.1.1 Fengyun-3C Microwave Temperature Sounder (MWTS) Data
Fengyun-3C (FY-3C) was successfully launched on 23 September 2013, which is the third
one in the new series of Chinese FY polar-orbiting satellites. The FY-3 mission aims at enhancing
numerical weather prediction and climate research with observation data, monitoring large-scale
natural disasters, and providing meteorological information for aviation and navigation. The first
and second satellites of this series are FY-3A and FY-3B, which were launched into orbits on
May 27, 2008 and November 5, 2010, respectively. The local equator crossing times of FY-3A, 3B and -3C are 10 am, 2pm and 10pm, respectively.
Microwave Temperature Sounder (MWTS) is one of the 11 sensors on board FY-3C.
MWTS contains 13 channels whose central frequencies range from 50.3 GHz to 57.6 GHz, which
correspond to ATMS channels 3-15, central frequencies and peak weighting function pressures of
MWTS channels can be found in Table 5.1. As a cross-track microwave radiometer, MWTS has
90 FOVs on each scan line, and the swath width of MWTS is 2600 km.
The scan profile of FY-3C MWTS was changed during May 12-18, 2014. Before this scan
profile change, each scan cycle takes 2.67 seconds. The antenna of MWTS scans at cold space,
warm target and earth scene with the same integration time of 0.018 second, and accelerates
during the time in between. After the profile change, MWTS keeps a constant speed during each
entire scan cycle of 5.23 seconds. MWTS data during March 5-11, 2014 and May 29-June 4, 2014
are used in this study. There are 14-15 swaths in a single day. Each swath contains around 2400
and 1200 scan lines before and after scan profile change, respectively.
68
5.1.2 Diagnosis of Striping Noise in FY-3C MWTS Data
The spectra for the first six IMFs of the first four PC coefficients at channel 8 for May 1,
2014 are shown in Fig. 5.1. Peak magnitudes of the first four IMFs are at the frequencies greater
than 10-2 Hz. The amplitude of individual IMF decreases with PC coefficient sequence, i.e., all
IMFs have the largest amplitudes for the first PC coefficient and smallest for the fourth PC
coefficient. This is in accordance with the fact that the first PC component explains the majority
of variances. For the first three PC coefficients, peak magnitudes have an obvious increase
starting from the fifth IMF, which have reached around 10 times of the first four IMFs. For the
fourth PC coefficient, there is no clear difference of magnitudes between IMFs. It is thus decided
that the first four IMFs of the first three PC coefficients are to be removed, which is denoted as
PC3/IMF4 for breivity.
The spectra for the first six IMFs of the first four PC coefficients at FY-3C MWTS channel
8 for May 29, 2014 after the scan profile change are shown in Fig. 5.2. The high-frequency end
for May 29 is different from that on May 1 since the scan cycles before and after the scan profile
change are different. The spectra for data on May 29 are similar to those of May 1, 2014 in terms
of magnitudes of individual IMFs. The fourth IMFs of all the first three PC coefficients at lowfrequency range have much larger amplitudes than the first three IMFs and are thus retained. It is
thus determined that the first three IMFs of the first three PC coefficients are to be removed for
MWTS channel 8 after the scan profile change of MWTS.
By comparison, the spectra for the first six IMFs of the first four PC coefficients at
corresponding ATMS channel 10 during May 1, 2014 are shown in Fig. 5.3. It is shown that
despite comparable magnitudes of the 5th and 6th IMFs to that of MWTS, magnitudes of the first
four IMFs of ATMS are smaller. This fact implies that the striping noise, which is described by
69
the first few IMFs, is much stronger within MWTS signals than ATMS. It is also shown that
unlike MWTS, where the amplitudes of the identified stripinh noise remains above 0.01 for the
first three PC coefficients, amplitudes of the first few IMFs for the second and third PC
coefficients of ATMS are at least five times smaller. This confirms that only the first PC
coefficient is smoothed for ATMS.
The distribution of the first five PC modes (i.e., eigenvectors) for the 47th data sample
(k=47, each sample includes 200 scan lines) on May 1, 2014 is displayed in Fig. 5.4. Note that the
eigenvectors are functions of FOV. All PC modes oscillate around zero, and their values are of
close scales. It is shown that compared with other PC modes, the first PC mode changes little
along with FOV. This is reasonable since the first PC mode represent the major cross-track
features of MWTS while other PC modes capture weather signals. Numbers of extremes for the
second, third, fourth and fifth PC modes are two, three, four and five, respectively.
Figure 5.5 shows spatial distributions of the first, second and third PC components, i.e., the
multiplications of PC coefficients and the corresponding PC modes. The magnitude of the first PC
component is dominating, which is comparable in magnitude to the brightness temperature. The
second and third PC components have nearly same magnitudes. The PC components before and
after smoothing are calculated using the PC coefficients with and without removing the first four
IMFs, respectively. It is seen that before the noise mitigation, visible striping noise exists in all
the three PC components, and after smoothing those striping noise are removed while neither
along-track nor cross-track features of large scale are changed.
Spatial distributions of observed and simulated brightness temperatures are displayed in Fig.
5.6 (a)-(b), and striping noise can be easily identified from the observed brightness temperature
fields. Figures. 5.6 (c) and (d) show the spatial distributions after striping noise mitigation of
70
PC3/IMF4 and PC1/IMF4, respectively. It is seen in Fig. 5.6 (d) that the striping noise is reduced
after removing the first four IMFs in the first PC coefficient but still exist. By comparison, no
such residual striping noise is found in the distribution of PC3/IMF4 smoothed brightness
temperatures. Spatial distributions of O-B before and after noise mitigation of PC3/IMF4, as well
as after noise mitigation of PC1/IMF4 are shown in Fig. 5.6 (e), (f) and (g), respectively. It is seen
that the striping noise pattern is clearly seen in the distributions of O-B. The striping noise is
effectively removed after PC3/IMF4 smoothing while some residual striping noise pattern still
remains after PC1/IMF4 smoothing. This indicates the necessity to remove the first four IMFs of
the second and the third PC coefficients.
Spatial distributions of the striping noise in FY-3C MWTS brightness temperatures
extracted by PC3/IMF4 and PC1/IMF4 are displayed in Fig. 5.7 (a) and (b), respectively. The
striping noise is as large as ±1K. It is shown that the line patterns for PC1/IMF4 smoothing of
detected striping noise cover all 90 FOVs, while the striping noise pattern for PC3/IMF4
smoothing includes not all but some of adjacent FOVs on each scan line. The differences of line
patterns between PC1/IMF4 smoothing and PC3/IMF4 smoothing is consistent with the spatial
patterms of the second and third PC modes, which are not constant with respect to FOV. The
difference between the noise identified by PC3/IMF4 and PC1/IMF4, i.e., (a) minus (b), is shown
in (c). It is seen that magnitude of this difference is of the same order as the magnitude of either
recognized noise. This further implies that it is necessary to remove the first four IMFs from the
first PC coefficient, but also from the second and the third PC coefficient.
Global distributions of differences between observations and simulations before and after
striping noise mitigation of PC3/IMF4 during May 1 and PC3/IMF3 during May 29 are displayed
in Figs. 5.8 and 5.9, respectively. It is seen that after smoothing, the visible striping noise is
71
effectively removed while other weather-related features are not altered. The noise removed is
shown in (c). It is seen that the magnitude of striping noise is on the order of ±1K for FY-3C
MWTS before and after the scan profile change..
5.1.3 Striping Index Calculated for FY-3C MWTS Global Data
In order to quantitatively validate the effectiveness of the striping noise mitigation on FY3C MWTS, a striping index for brightness temperatures is defined as follows. The Striping Index
(SI) for a single day at a given channel is calculated as the following equation:
K
1 N 2
∑ N ∑ σ along (k,i)
i=1
SI = k=1
K
1 M 2
∑ M ∑ σ cross (k, j)
k=1
j=1
(42)
2
2
where σ along
(k,i) and σ cross
(k, j) are along-track and across-track variances, respectively, and are
calculated as:
σ
2
along
1
(k,i) =
M
1
(O − B)k,i =
M
j
2
σ cross
(k, j) =
i
(O − B)k, j
∑ ((O − B)
M
j
k,i, j
− (O − B)k,i
j=1
)
2
M
∑ (O − B)
(43)
k,i, j
j=1
(
i
1 N
(O − B)k,i, j − (O − B)k, j
∑
N i=1
1 N
= ∑ (O − B)k,i, j
N i=1
)
2
(44)
where (O − B)k,i, j stands for O-B difference at the ith FOV on the jth scan line within the kth data set
involving a total of 200 and 100 scan lines before and after scan profile change, respectively; N is
the total number of FOVs, M is the number of scan lines within each of the data set for variance
calculation, and K is the total number of datasets. In the calculation, N=90, M=200 and K is
72
smooth
around 160. For SIs after noise mitigation, smoothed brightness temperatures O
are used
instead of observations O .
2
2
Variations of along-track variances σ along
(k,i) and cross-track variances σ cross
(k, j) are
shown in Fig. 5.10. Both along-track and cross-track variances are calculated using brightness
temperatures before striping noise mitigation on May 1, 2014. The distribution of along-track
variance has line patterns for adjacent FOVs within some datasets, which is related to cross-track
correlations. By comparison, distribution of cross-track variance is random. Shown in obviously
brighter colors, the along-track variances generally have larger values than cross-track variances
within the same dataset. As striping noise adds to along-track variances, the fact that along-track
variances are greater than cross-track variances for all individual dataset confirms the existence of
striping noise. The striping index calculated as Equation (42) is 1.3513. Variations of along-track
2
2
(k,i) and cross-track variances σ cross
(k, j) after noise mitigations are shown in
variances σ along
Fig. 5.11. It is seen that the magnitudes of along-track variances are significantly reduced while
those of cross-track variances changed very little, and the SI dropped to 0.975. Similar
distributions of individual along-track and cross-track variances for May 29 are shown in Figs.
5.12-13. The SI dropped from 1.51 to 1.01.
Variations of SIs of MWTS channel 8 with respect to date during two weeks are shown in
Fig. 5.13. Along-track variances increased after the scan profile change while cross-track
variances remained the same, which indicates that the FY-3C MWTS striping problems are even
severer after scan profile change. It is seen that with striping noise mitigations along-track
variations almost dropped to half. Cross-track variations also decreased, which is related to the
smoothing within the second and third PC coefficients. SI index are all around unit values after
noise mitigation, indicating that the noise mitigation is effective.
73
5.2 Investigation on the Root-Cause of Striping Noise
5.2.1 Mathematical Expressions of Transfer Function
As shown above, striping noise is present in ATMS temperature channels and humidity
channels, previous humidity sounders AMSU-B and MHS, and other temperature sounders such
as FY-3C MWTS. However, striping noise is not found in AMSU-A observations. A major
difference in instrumental structures between AMSU-A and the other microwave sensors is that
AMSU-A does not have a RF amplifier in front of the receiver as the other sensors do. These
amplifiers could induce flicker noise through receiver gain fluctuation, whose power spectral
density (PSD) is in a form of 1/f, where f is frequency. Transformation from this flicker noise to
striping noise can be described as follows.
In the time domain, the output noise gn (t) is the convolution of the “real” sensor noise g(t)
and a so-called “forcing” transfer function h(t) , i.e.,
∞
gn (t) = g(t)* h(t) = ∫ g(t ')h(t − t ')dt '
(45)
−∞
The Fourier transformations of g(t) , h(t) and gn (t) can be represented by G( f ) , H ( f ) and
Gn ( f ) as follows:
∞
G( f ) = ∫ g(t)ei 2π ft dt
(46a)
−∞
∞
H ( f ) = ∫ h(t)ei 2 π ft dt
(46b)
−∞
∞
Gn ( f ) = ∫ gn (t)ei 2 π ft dt
−∞
∞
∞
−∞
−∞
∞
∞
−∞
−∞
= ∫ dt ∫ dt ' gn (t ')h(t − t ')ei 2 π ft
= ∫ dt ∫ dt ' gn (t ')e
= G( f )H ( f )
74
i 2 π ft '
h(t − t ')e
(46c)
i 2 π f (t−t ')
Equation 46c shows that in the frequency domain, the output noise signal is simply a
multiplication of sensor noise and transfer function.
According to Hersman and Poe (year?), the transfer function h(t) can be expressed as:
h(t) = hs (t) − ∑ w(t − kt c )hc (kt c )
(47a)
τs
⎧1
⎪⎪ τ , t ≤ 2
hs (t) = ⎨ s
⎪0, t > τ s
⎪⎩
2
(47b)
k
τc
⎧1
⎪⎪ τ , t ≤ 2
hc (t) = ⎨ c
⎪0, t > τ c
⎪⎩
2
where hs (t) denotes the part related to scene measurements, and
(47c)
∑ w(t − kt )h (kt ) is the noisc
c
c
k
calibrated measurements hc (kt c ) that is convolved by the weighting function w(t − kt c ) .
Substituting (47) into (45) yields:
∞
⎡
⎤
gn (t) = ∫ dt ' g(t ') ⎢ hs (t − t ') − ∑ w(t − kt c )hc (kt c − t ') ⎥
−∞
⎣
⎦
k
(48)
The power spectral density (PSD, Sn ( f ) ) of the output noise gn (t) is the autocorrelation of
the noise itself, which can be written as:
∞
∞
⎡
⎤
Sn ( f ) = ∫ dt '∫ dt "[ g(t ')g(t ")] ⎢ hs (t − t ') − ∑ w(t − kt c )hc (kt c − t ') ⎥
−∞
−∞
⎣
⎦
k
(49)
⎡
⎤
⎢ hs (t − t ") − ∑ w(t − kt c )hc (kt c − t ") ⎥
⎣
⎦
k
where the expectation of the term [g(t ')g(t ")] is the PSD ( S( f ) ) of instrumental noise g(t) , i.e.,
75
E { g(t ')g(t ")} = S( f )
(50a)
The scene integration term (Hs(f)) as well as calibration integration term (Hc(f)) can be simplified
through a series of mathematical manipulation as shown below:
∞
H s ( f ) = ∫ hs (t)ei 2 π ft dt
−∞
τs
= ∫ τs
−
=
1
τs
1 i 2 π ft
e dt
2 τs
2
∫
τs
2
τ
− s2
cos(2π ft) + i sin(2π ft)dt
(50b)
τs
1
sin(2π ft) − i cos(2π ft)] τ2s
[
− 2
τ s (2π f )
sin(π f τ s )
=
τ sπ f
=
hc (kt c − t ') = hc [ −(t − kt c ) + (t − t ')]
(50c)
H c ( f ) = ∫ hc [ −(t − kt c ) + t '] ei 2 π ft 'dt '
∞
−∞
∞
= ∫ hc (t ')ei 2 π f (t '−t+ktc )dt '
−∞
=e
−i 2 π f (t−kt c )
= e−i 2 π f (t−ktc )
∫
τc
2
τ
− c2
(50d)
1 i 2 π ft '
e
dt '
τc
sin(π f τ c )
τ cπ f
Combining 50a-d yields:
⎛
⎞
H ( f ) = ⎜ H s ( f ) − H c ( f )∑ w(t − kt c )⎟
⎝
⎠
k
2
2
sin π f τ s sin π f τ c
=
−
w(t − kt c )e−i 2 π f (t−ktc )
∑
π fτ s
π fτc k
2
(51)
and
Sn( f ) = S( f )H 2 ( f )
(52)
It is seen from Equation (51) that transfer function varies with respect to the instrumental
76
parameters of scan cycle ( t c ), calibration integration time ( τ c ) and scene integration time ( τ s ),
etc. It also depends on calibration weighting functions w(t) . In order to quantify the impact of
transfer function on the output noise magnitude Sn ( f ) , transfer functions for AMSU-A, ATMS
and FY-3C MWTS before and after profile changes are firstly calculated, and then convoluted
with PSD of simulated noise S( f ) .
5.2.2 PSD of the Output Noise of Transfer Function
Parameters of AMSU-A, ATMS and FY-3C MWTS before and after profile changes that
are used as input to transfer function are shown in Table 5.2. It is seen that both the scan cycle
and the integration time of AMSU-A are triple and 7 times longer than those of ATMS.
As previously noted, there is no flicker noise in the simulated signals for AMSU-A. FY-3C
MWTS experienced a scan profile change during May 12-18, 2014. Before this profile change,
antennas of MWTS and other sensors slow down to take scans of cold space, warm load and earth
scenes, respectively, and accelerate between the intervals of taking the scans. After the profile
change, antenna of MWTS starts to rotate at a constant speed. MWTS’s integration time can be
calculated as follows:
2θ m
τ s = 360
n
t 1
r
= c
arcsin(
sin ϕ m )
n 180
r+h
tc
(53)
where n is number of FOVs, which is 90 for MWTS; θ m and ϕ m are outmost scan angle and
zenith angle, respectively; r is the radius of earth, and h is the altitude of FY-3C. It is seen that
even when the scan cycle is doubled, calibration integration time and scene integration time of
MWTS decrease after scan profile change. ATMS has similar values for instrumental parameters
as the MWTS before scan profile change. A uniform weighting of either two points or five points
77
can be employed for calibration weighting functions w(t) , i.e.,
1
w(t) = ,t = ±t c
2
(54a)
1
w(t) = ,t = 0, ±t c , ±2t c
5
(54b)
and
Figures 5.15 and 5.16 show the transfer functions calculated using instrumental parameters
and weighting functions described above. It is seen that all transfer functions increase sharply at
low frequencies, then oscillate with the peak magnitude unchanged, and drop at higher
frequencies. The frequency ranges where transfer functions remain their maximum magnitudes
are around the scan frequency (
1
1
) and integration frequency ( ). Having a longer scan cycle
tc
τc
and a longer integration time than other sensors, AMSU-A has its transfer function peaking at
lower frequencies. Transfer functions with five points uniformly weighted have slightly wider
peaking ranges and weaker oscillations than those with two points averaging.
PSDs of the noise outputs Sn ( f ) for AMSU-A, ATMS and MWTS before and after profile
change are calculated according to Equation (52), which are shown in Fig. 5.18 It is seen that for
all three sensors with white noise only, there exist signals at the striping noise frequencies
although the striping signals are not very pronounced. By comparison, with flicker noise added in
the instrument noise, the striping noise is magnified. It is also shown that even with the same
instrument input signals, the output noise PSD at the striping frequencies are different. Especially,
with a longer scan cycle as MWTS after scan profile change, the transfer function peak shifts
toward the lower frequency range, which favors the presence of the flicker noise. As a result, the
striping noise contained in MWTS observations become more visible after the change of the scan
78
profile. The PSD with and without adding flicker noise is shown in red and green, respectively,
and the differences between the two are shown in Fig. 5.19. It is seen that PSD of the output from
the white noise only maintain the shape of transfer function, while PSD of output for combined
noise is amplified at frequencies that are near and lower than the “knee”.
The contribution of flicker noise to the output PSD, which is seen in Fig. 5.19, resembles
the PSD of striping noise. This PSD component for ATMS and MWTS before scan profile change
is similar, and is much larger for MWTS after scan profile change. Figure 5.19 confirms the
hypothesis that flicker noise is a major cause of the striping noise, and the transfer functions
determined by instrumental parameters can modify the magnitude and frequency of the striping
noise.
79
Table 5.1: Central frequencies of FY-3C MWTS and corresponding ATMS channels.
Channel
Central Frequency
Peak WF
Number
(GHz)
MWTS
ATMS
(hPa)
MWTS
ATMS
1
3
50.3
Surface
2
4
51.76
950
3
5
52.8
850
4
6
53.596±0.115
700
5
7
54.4
400
6
8
54.94
250
7
9
55.5
200
8
10
57.2903
100
9
11
57.2903±0.115
50
10
12
57.2903
25
11
13
57.2903±0.322
10
12
14
57.2903±0.322±0.010
5
13
15
57.2903±0.322±0.004
2
80
MWTS
ATMS
Table 5.2: Scan cycle ( t s ), calibration integration time ( τ c ) and scene integration time ( τ s ) that
are used as input for calculating the transfer functions for AMSU-A, ATMS and FY-3C
MWTS before and after profile changes.
ts
τc
τs
(unit: s)
(unit: s)
(unit: s)
AMSU
8
0.158
0.158
No
ATMS
2.67
0.018
0.018
Yes
MWTS
2.67
0.018
0.018
Yes
MWTS*
5.23
0.016
0.016
Yes
Sensor
Flicker noise
Table 5.3: Numbers of FOV and NEDT at AMSU-A, ATMS, MWTS channels with central
frequencies of 57.290344 GHz.
Sensor
Channel
Scan cycle (s)
Numbers of FOVs
NEDT (K)
AMSU
9
8
30
0.25
ATMS
10
2.67
96
0.75
MWTS
8
2.67
90
0.75
MWTS*
8
5.23
90
0.75
81
Amplitude
(b)
Amplitude
(a)
(d)
Frequency (s-1)
Amplitude
Amplitude
(c)
Frequency (s-1)
Frequency (s-1)
Frequency (s-1)
Fig. 5.1: Fourier spectra for the first four IMFs of the (a) first, (b) second, (c) third and (d) fourth
PC coefficients at channel 8 for May 1, 2014.
82
Amplitude
(b)
Amplitude
(a)
Frequency (s-1)
Frequency (s-1)
(d)
Amplitude
Amplitude
(c)
Frequency (s-1)
Frequency (s-1)
Fig. 5.2: Fourier spectra for the first six IMFs of the (a) first, (b) second, (c) third and (d) fourth
PC coefficients at channel 8 for May 29, 2014.
83
Amplitude
(b)
Amplitude
(a)
Frequency (s-1)
Frequency (s-1)
(d)
Amplitude
Amplitude
(c)
Frequency (s-1)
Frequency (s-1)
Fig. 5.3: Fourier spectra for the first six IMFs of the (a) first, (b) second, (c) third and (d) fourth
PC coefficients at ATMS channel 10 for May 1, 2014.
84
PC component
FOV
Fig. 5.4: Spatial distributions of the first five PC modes (i.e., eigenvectors) for the 47th data
sample (k=47, each sample includes 200 scan lines) on May 1, 2014.
85
k=47, 1st PC
Smoothed 1st PC (4 IMFs)
Smoothed PCA2 (4 IMFs)
k=47, PCA2
Fig. 5.5: Spatial distributions of the individual PCA component for the first three PCA
components (i.e., the product of eigenvalue and eigenvector) before (left panels) and after
(right panels) striping noise mitigation for the 47th data sample (k=47, each sample
includes 200 scan lines) on May 1, 2014.
86
k=47, PCA3
Smoothed PCA3 (4 IMFs)
Fig. 5.5 Continued
87
(b)
(a)
(c)
(d)
Fig. 5.6: Spatial distributions of ATMS channel 8 (a) observed, (b) simulated and (c) smoothed
3PC/4IMF and (d) 1PC/4IMF brightness temperatures for the 47th data sample (k=47, each
sample includes 200 scan lines) on May 1, 2014. O-B differences (e) before and after (f)
3PC/4IMF and (g) 1PC/4IMF noise mitigation.
88
(f)
(e)
(g)
Fig. 5.6 Continued
89
(b)
(a)
(c)
Fig. 5.7: Noise removed by (a) 3PC/4IMF and (b) 1PC/4IMF as in Fig. 5.6. (c) Differences
between (a) and (b).
90
(a)
(b)
(c)
Fig. 5.8: Global distributions of channel 8 (a) O-B and (b) difference between smoothed
brightness temperatures removed on May 1, 2014 on ascending node. The first four IMFs
of the first three PC coefficients are removed for noise mitigation. (c) The striping noise
identified with the EEMD/PCA approach.
91
(a)
(b)
(c)
Fig. 5.9: Global distributions of channel 8 (a) O-B and (b) difference between smoothed
brightness temperatures removed on May 30, 2014 on ascending node. The first three
IMFs of the first three PC coefficients are removed for noise mitigation. (c) The striping
noise identified by the EEMD/PCA approach.
92
FOV (i)
(a)
Dataset Number (k)
Scan line (j)
(b)
Dataset number (k)
Fig. 5.10: Variations of (a) along-track variances as a function of the sample group number and
FOV (i) and (b) cross-track variances as a function of dataset number (k) and scan line (j)
before striping noise mitigation on May 1, 2014. The total number of scan lines in each
dataset is 200. The total number of datasets for both the ascending and descending nodes
on May 1 is 162. (c) Variation of striping index as a function of dataset number (k).
93
FOV (i)
(a)
Dataset number (k)
Scan line (j)
(b)
Dataset number (k)
Fig. 5.11: Same as Fig. 5.10 except for data with striping noise mitigated (PC3/IMF4).
94
FOV (i)
(a)
Dataset Number (k)
Scan line (j)
(b)
Dataset number (k)
Fig. 5.12: Same as Fig. 5.10 except for data before striping noise mitigation on May 30, 2014.
95
FOV (i)
(a)
Dataset number (k)
Scan line (j)
(b)
Dataset number (k)
Fig. 5.13: Same as Fig. 5.10 except for data with striping noise mitigated (3PC/3IMF).
96
Along-track variance
Before Profile Change
After Profile Change
Cross-track variance
Date
Before Profile Change
After Profile Change
Striping index
Date
Before Profile Change
After Profile Change
Date
Fig. 5.14: (a) Along-track, (b) cross-track variances and (c) striping index at channel 8 during
May 1-7, 2014 and May 29-June 4, 2014 before (red) and after (blue) noise. Ascending
and descending nodes are indicated in open and solid markers, respectively.
97
H2(f)
(a)
Frequency (Hz)
H2(f)
(b)
Frequency (Hz)
Fig. 5.15: Square of transfer function (H(f)) that are calculated using instrument parameters (see
Table 5.2) of (a) AMSU-A and (b) ATMS with two (red) and five (blue) nearest points
being averaged in calibration.
98
H2(f)
(a)
Frequency (Hz)
H2(f)
(b)
Frequency (Hz)
Fig. 5.16: Same as Fig.4.41 except for FY-3C MWTS (a) before and (b) after profile change that
occurred during May 12-18, 2014.
99
S(f) (K2/Hz)
(a)
Frequency (Hz)
(c)
S(f) (K2/Hz)
S(f) (K2/Hz)
(b)
Frequency (Hz)
Frequency (Hz)
Fig. 5.17: PSD of (a) pitch-over maneuver data at ATMS channel 10, (b) white noise only and (c)
the sum of white noise and flicker noise with (colored) and without (black) 81-point
running mean.
100
(b)
Sn(f) (K2/Hz)
Sn(f) (K2/Hz)
(a)
Frequency (Hz)
Frequency (Hz)
(d)
Sn(f) (K2/Hz)
Sn(f) (K2/Hz)
(c)
Frequency (Hz)
Frequency (Hz)
Fig. 5.18: The PSD of the noise outputs ( Sn( f ) ) for (a) AMSU-A, (b) ATMS, (c) MWTS before
profile change and (d) MWTS after profile change with (red) and without (green) adding
flicker noise.
101
Sn(f) Difference (K2/Hz)
Frequency (Hz)
Fig. 5.19: Differences of the PSD of the noise outputs ( Sn( f ) ) between with and without flicker
noise for ATMS (blue), MWTS before profile change (red) and MWTS after profile
change (green).
102
CHAPTER SIX
SUMMARY AND DISCUSSIONS
6.1 Major Contribution
A convenient method to remove striping noise, especially for operational implementation is
developed in this dissertation. The new method has been implemented in the following two
contexts: (1) providing research community “stripping-noise-free” brightness temperatures
datasets by eliminating the striping noise in the existing data and (2) removing the striping noise
in raw count data in the real-time operational system. The root causes and factors affecting the
presence and frequency distribution of the striping noise are also investigated. This may help to
design the next generation microwave sensors that would have minimum impact from stripping
noises
6.2 Summary and Conclusions
An along-track striping noise phenomenon was detected in the global O-B distributions of
ATMS temperature sounding channels. A combination of EEMD and PCA methods was
developed by Qin et al. (2013) to identify and eliminate the noise. The extracted noise is at
frequencies greater than 10-2 Hz with magnitudes of 0.3 K for ATMS temperature sounding
channels and 1 K for window channels and ATMS, AMSU-B, MHS humidity channels. Since
antenna temperatures are calculated with warm counts, cold counts, warm load temperatures and
scene counts, the EEMD/PCA method can also be employed for characterizing the noise in these
raw count data. However, the EEMD method is not convenient for operational implementations.
Development of a set of optimal filters that can reduce striping noise in calibration counts as
efficient as the EEMD does is desirable. The conventionally used boxcar filters in satellite
calibration tend to alter low frequency weather signals when suppressing high-frequency noise.
103
Four sets of the optimal symmetrical filters are designed and developed for calibration counts of
all 22 ATMS channels. The optimal filters efficiently remove striping noise within the raw counts
data, so that the striping noise contained in the calculated antenna temperature are smoothed out.
The necessity of smoothing the scene counts is also confirmed. If calibration counts are smoothed
but the scene counts are not, striping noise still exists and is visible in global O-B distributions.
A similar set of 22 optimal filters was designed for removing directly the striping noise in
brightness temperatures for all 22 ATMS channels. These optimal filters were shown to be
effective in smoothing out the striping noise while keeping the large-scale weather signals unaltered. They are also convenient for implementation in the operational systems. For validation
purposes, these optimal filters were applied to ATMS pitch-over maneuver data. The striping
noise in pitch-over maneuver data in all channels was effectively smoothed out. As expected, the
along-track variance is reduced, cross-track variance does not change much, and the SI
approaches a unit value.
The LWP data retrieved from brightness temperatures at ATMS window channels 1 and 2
before and after striping noise mitigation was compared. The LWP retrievals without striping
noise mitigation reveal a striping noise pattern with a magnitude as large as 0.05 kg/m2. The
striping noise in the LWP differences has a linear relationship to the striping noise in brightness
temperatures of ATMS window channels. The impact of ATMS striping noise on LWP retrievals
further confirms the necessity and effectiveness of applying the optimal filters on the ATMS
brightness temperature observations.
Observation from a recent instrument FY-3C MWTS is found to also contain striping noise
of a magnitude much larger than that of ATMS. It is shown the striping noise exists not only
within the first PC coefficient, but also the second and third PC coefficients. The EEMD/PCA
104
method is then used to extract the noise from the first three PC coefficients for MWTS data. The
magnitude of the striping noise in the temperature channels can be as large as 1K, which is much
larger than that in ATMS. As pitch-over maneuver data of MWTS is not available, a global
striping index is defined for brightness temperature observations. It is shown that after noise
mitigation, the global striping index dropped to unit value, which indicates that the noise
mitigation is effective. MWTS has experienced a scan profile change during May 12-18, 2014.
After the scan profile change, MWTS’s antenna starts to rotate at a constant speed. Noise
identification and elimination for two weeks before and after the profile change showed that the
striping noise is even stronger after the scan profile change. This finding confirms the scan speed
is probably not the root-cause of striping noise contaminated in satellite microwave sensor
observations.
In order to gain further insight on the root cause of the striping noise, a transfer function is
calculated with instrument parameters such as scan cycle, calibration integration time and scene
integration time for AMSU-A, ATMS and MWTS before and after profile change. The transfer
function links the instrument noise to the output noise of the amplifier. White noise and flicker
noise are then simulated as the instrument noise and forced by the transfer functions for all the
above-mentioned four microwave sensors. The difference of the output PSD between signals with
and without flicker noise resembles PSD of the striping noise. It is also demonstrated that even
with the same instrument signals, amplitudes of the output noise could vary because of the
differences of transfer functions among different sensors. The FY-3C MWTS after scan profile
change has the largest striping noise. These facts indicate that flicker noise signal induces the
striping noise, and transfer function can modify the striping noise in terms of both its magnitude
and peak frequency.
105
6.3 Future Work
Similar striping noise analysis and mitigation will be conducted for other sensors such as
Tropical
Rainfall
Measuring
Mission
Microwave
Imager
(TMI)
onboard
Global Precipitation Measurement (GPM). The impacts of the striping noise on numerical
weather prediction will be carried out.
106
APPENDIX A
LIST OF ACRONYMS
Acronym
Meaning
AMSU
Advanced Microwave Sounding Unit
ATBD
Advanced Technology Microwave Sounder
ATMS
Advanced Technology Microwave Sounder
EEMD
Ensemble Empirical Mode Decomposition
FOV
Field of View
FY
FengYun
GPM
Global Precipitation Measurement
IMF
Intrinsic Mode Function
NOAA
National Oceanic and Atmospheric Administration
NEDT
Noise Equivalent Delta Temperature
NPP
National Polar-Orbiting Partnership
NWP
Numerical weather Prediction
MSU
Microwave Sounding Unit
MHS
Microwave Humidity Sounder
MWHS
MicroWave Humidity Sounder
MWTS
MicroWave Temperature Sounder
O-B
Difference between observed and simulated brightness temperature
PC
Principle Component
PCA
Principle Component Analysis
RF
Radio Frequency
TIROS-N
Television Infrared Observation Satellite -N
TMI
Tropical Rainfall Measuring Mission Microwave Imager
107
APPENDIX B
COPYRIGHT PERMISSION
JOHN WILEY AND SONS LICENSE
TERMS AND CONDITIONS
Jul 10, 2015
This Agreement between Yuan Ma ("You") and John Wiley and Sons ("John Wiley and
Sons") consists of your license details and the terms and conditions provided by John Wiley and
Sons and Copyright Clearance Center.
License Number 3665390006856
License date Jul 10, 2015
Licensed Content Publisher John Wiley and Sons
Licensed Content Publication Journal of Geophysical Research: Atmospheres
Licensed Content Title Striping Noise Mitigation in ATMS Brightness Temperatures and Its
Impact on Cloud LWP Retrievals
Licensed Content Author Yuan Ma,Xiaolei Zou
Licensed Content Date Jun 18, 2015
Pages 1
Type of use Dissertation/Thesis
Requestor type Author of this Wiley article
Format Electronic
Portion Full article
108
Will you be translating? No
109
REFERENCES
A. Antoniou (1993), Digital Filters: Analysis, Design, and Applications, New York, NY:
McGraw-Hill.
Bormann, N., A. Fouilloux, and W. Bell (2013), Evaluation and assimilation of ATMS data in
the ECMWF system, J. Geophys. Res. Atmos., 118, 12,970–12,980. doi:10.1002/2013JD020325.
Chew, Weng Cho (1995), Waves and fields in inhomogeneous media. Vol. 522. New York: IEEE
press.
Goldberg, M. D. and F. Weng (2006), Advanced Technology Microwave Sounder (book
chapter), in Earth Science Satellite Remote Sensing, vol. 1: Science and Instrument, Springer
Berlin Heidelberg, 243-253. doi: 10.1007/978-3-540-37293-6_13.
Hersman, M. S., and G. A. Poe (1981). Sensitivity of the total power radiometer with periodic
absolute calibration. Microwave Theory and Techniques, IEEE Transactions on, 29(1), 32-40.
JPSS ATMS ATBD (2011), Joint Polar Satellite System (JPSS) Advanced Technology
Microwave Sounder (ATMS) SDR Radiometric Calibration Algorithm Theoretical Basic
Document
(ATBD),
http://www.star.nesdis.noaa.gov/jpss/documents/ATBD/GSFC_47400043_JPSS_ATMS_SDR_
Radiometric_Calibration_ATBD__Alt._doc._no._D43751_.pdf.
Kim, E., C.-H. J. Lyu, K. Anderson, R. V. Leslie, and W. J. Blackwell (2014), S-NPP ATMS
instrument prelaunch and on-orbit performance evaluation, J. Geophys. Res. Atmos., vol. 119,
pp. 5653–5670. doi:10.1002/2013JD020483.
Ma, Y. and X. Zou (2015), Striping noise mitigation in ATMS brightness temperatures and its
impacts on cloud LWP retrievals, J. Geophys. Res. Atmos., vol. 120, doi: 10.1002/2015JD023162.
Petty G. W. (2006), A first course in atmospheric radiation (2nd edition), Sundog Publishing, 444
pp.
Qin, Z., X. Zou, and F. Weng (2013), Analysis of ATMS striping noise from its Earth scene
observations, J. Geophys. Res. Atmos., 118, 13,214–13,229. doi:10.1002/2013JD020399.
Ulaby, F. T., R. K. Moore, and A. K. Fung (1981). Microwave Remote Sensing: Microwave
remote sensing fundamentals and radiometry. Vol. 1. Addison-Wesley Publishing Company,
Advanced Book Program/World Science Division.
Weng, F., and N. C. Grody (1994), Retrieval of cloud liquid water using the special sensor
microwave imager (SSM/I), J. Geophys. Res., 99, D12, 25535–25551, doi:10.1029/94JD02304.
110
Weng, F. X. Zou and Z. Qin (2014), Uncertainty of AMSU-A derived temperature trends in
relationship with clouds and precipitation. Clim. Dyn., 43, 1439-1448.
Weng, F., X. Zou, N. Sun, H. Yang, M. Tian, W. J. Blackwell, X. Wang, L. Lin, and K.
Anderson (2013), Calibration of Suomi national polar-orbiting partnership advanced technology
microwave sounder”, J. Geophys. Res. Atmos., 118, 11,187–11,200. doi:10.1002/jgrd.50840.
Weng, F., L. Zhao, R. R. Ferraro, G. Poe, X. Li, and N. C. Grody (2003), Advanced microwave
sounding unit cloud and precipitation algorithms, Radio Sci., 38, 8068,
doi:10.1029/2002RS002679, 4.
Weng, F., X. Zou, X. Wang, S. Yang, and M. D. Goldberg (2012), Introduction to Suomi
national polar-orbiting partnership advanced technology microwave sounder for numerical
weather prediction and tropical cyclone applications, J. Geophys. Res., 117, D19112
Weng F., H. Yang, and X. Zou, “ On convertibility from antenna to sensor brightness temperature
for ATMS, IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, vol. 10, no. 4, pp. 771775, July 2013.
Wu Z. and N. E. Huang (2004), A study of the characteristics of white noise using the empirical
mode decomposition method, Proc. R. Soc. Lond., vol. 460, pp. 1597–1611.
Wu, Z., and N. E. Huang (2009), Ensemble empirical mode decomposition: A noise-assisted data
analysis method, Adv. Adapt Data Anal, 1(1), 1–41.
Wu, Z., N. E. Huang, and X. Chen (2009), The multi-dimensional ensemble empirical mode
decomposition method, Adv. Adapt Data Anal, 1(3), 339–372.
Zou X., Z. Qin, and F. Weng (2013), Improved quantitative precipitation forecasts by MHS
radiance data assimilation with a newly added cloud detection algorithm, Mon. Wea. Rev., 141,
3203–3221.
111
BIOGRAPHICAL SKETCH
Yuan Ma grew up in Yangzhou, Jiangsu Province, China. She received Bachelor’s degree in
Science from School of Atmospheric Sciences in Nanjing University in June 2010, and a Master’s
degree in Science from the department of Earth, Oceanic and Atmospheric Sciences in Florida
State University in May 2013.
112
Документ
Категория
Без категории
Просмотров
0
Размер файла
47 278 Кб
Теги
sdewsdweddes
1/--страниц
Пожаловаться на содержимое документа