# Development of the "optimal filters" for mitigation of striping noise in satellite microwave temperature and humidity sounding data

код для вставкиСкачатьFLORIDA STATE UNIVERSITY COLLEGE OF ARTS AND SCIENCES DEVELOPMENT OF THE “OPTIMAL FILTERS” FOR MITIGATION OF STRIPING NOISE IN SATELLITE MICROWAVE TEMPERATURE AND HUMIDITY SOUNDING DATA By YUAN MA A Dissertation submitted to the Department of Earth, Ocean and Atmospheric Science in partial fulfillment of the requirements for the degree of Doctor of Philosophy 2015 ProQuest Number: 3724310 All rights reserved INFORMATION TO ALL USERS The quality of this reproduction is dependent upon the quality of the copy submitted. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if material had to be removed, a note will indicate the deletion. ProQuest 3724310 Published by ProQuest LLC (2015). Copyright of the Dissertation is held by the Author. All rights reserved. This work is protected against unauthorized copying under Title 17, United States Code Microform Edition © ProQuest LLC. ProQuest LLC. 789 East Eisenhower Parkway P.O. Box 1346 Ann Arbor, MI 48106 - 1346 Yuan Ma defended this dissertation on June 4, 2015. The members of the supervisory committee were: Ming Cai Professor Directing Dissertation Xin Yuan University Representative Guosheng Liu Committee Member Peter Ray Committee Member Jeffery Chagnon Committee Member The Graduate School has verified and approved the above-named committee members, and certifies that the dissertation has been approved in accordance with university requirements. ii To Prof. Xiaolei Zou. iii ACKNOWLEDGMENTS First of all I thank Dr. Xiaolei Zou for her continuous guidance, support and suggestions since 2010. She has provided me everything I could expect from an advisor and I owe her great deal of gratitude. After Dr. Zou’s departure for University of Maryland in 2014 Fall, Dr. Zou continued to direct my Ph. D. thesis and Dr. Ming Cai is very kind to serve as my new major professor. I really appreciate Dr. Cai’s help and time. I also want to thank Drs. Guosheng Liu, Peter S. Ray, Jeffery Chagnon and Xin Yuan for serving on my committee. The work would have been impossible without the generous help from Dr. Fuzhong Weng and his Calibration/Validation team in NOAA. I have also received lots of help from scientists and students working with Dr. Zou, especially Dr. Zhengkun Qin. Last but not least important, I want to thank my family and friends for their unconditional love. This research was supported by JPSS proving ground project NA11OAR4320199. iv TABLE OF CONTENTS List of Tables ................................................................................................................................ vii List of Figures .............................................................................................................................. viii Abstract ......................................................................................................................................... xii 1. 2. 3. INTRODUCTION .................................................................................................................. 1 1.1 History of Satellite-based Microwave Instruments....................................................... 1 1.2 Striping Noise Contained in ATMS Observations........................................................ 3 1.3 Existing Methods to Mitigate Striping Noise ............................................................... 4 1.4 Dissertation Organization ............................................................................................. 5 DATA ..................................................................................................................................... 7 2.1 Advanced Technology Microwave Sounder (ATMS) .................................................. 7 2.2 The Community Radiative Model (CRTM) ................................................................. 7 2.3 NCEP GFS data ............................................................................................................ 8 METHODOLOGY ................................................................................................................. 9 3.1 Derivation of Antenna Temperature from Raw Counts ................................................ 9 3.2 Mathematical Expressions of Optimal Filters ............................................................. 10 4. Optimal Filters for Warm Counts and Cold Counts ....................................... 10 3.2.2 Optimal Filters for Scene Counts .................................................................... 14 3.2.3 Optimal Filters for Brightness Temperatures ................................................. 15 ATMS STRIPING NOISE MITIGATION USING THE OPTIMAL FILTERS ................. 20 4.1 4.2 5. 3.2.1 Striping Noise Mitigation on Sensor Calibration Counts ........................................... 20 4.1.1 Spectrum Analyses of IMFs of Calibration Counts ........................................ 20 4.1.2 Properties of the Optimal Symmetrical Filters ............................................... 21 4.1.3 Comparison between the Optimal Filters and Boxcar Filters ......................... 23 4.1.4 Effects of the Optimal Filters on Antenna Temperatures ............................... 24 Striping Noise Mitigation on Brightness Temperatures ............................................. 24 4.2.1 Weighting Coefficients of the Optimal Filters................................................ 24 4.2.2 Brightness Temperatures before and after Striping Noise Mitigation ............ 26 4.2.3 Liquid Water Path Retrieval Algorithm .......................................................... 28 4.2.4 Impact of Striping Noise on ATMS LWP Retrievals ..................................... 31 STRIPING NOISE MITIGATION FOR FY-3C MWTS DATA ........................................ 68 5.1 Characterization and Mitigation of Striping Noise in FY-3C MWTS Data ............... 68 v 5.2 6. 5.1.1 Fengyun-3C Microwave Temperature Sounder (MWTS) Data ..................... 68 5.1.2 Diagnosis of Striping Noise in FY-3C MWTS Data ...................................... 69 5.1.3 Global Striping Index For FY-3C MWTS Data ............................................. 72 Investigation on the Root-Cause of Striping Noise .................................................... 74 5.2.1 Mathematical Expressions of Radiometric Transfer Function ....................... 74 5.2.2 PSD of the Output Noise of Transfer Function .............................................. 77 SUMMARY AND DISCUSSIONS ................................................................................... 103 6.1 Major Contribution ................................................................................................... 103 6.2 Summary and Conclusions ....................................................................................... 103 6.3 Future Work .............................................................................................................. 106 APPENDIX ................................................................................................................................. 107 A. LIST OF ACRONYMS ...................................................................................................... 107 B. COPYRIGHT PERMISSION ............................................................................................. 108 REFERENCES ........................................................................................................................... 110 BIOGRAPHICAL SKETCH ...................................................................................................... 112 vi LIST OF TABLES 4.1 Numbers of IMFs removed and filter spans of optimal striping filters for ATMS scene counts at each channel .................................................................................................................. 34 4.2 Filter spans of optimal striping filters for ATMS warm counts and cold counts ............. 35 4.3 Numbers of IMFs removed and the chosen filter spans for ATMS channels ................... 36 5.1 Central frequencies of FY-3C MWTS and corresponding ATMS channels .................... 80 5.2 Scan cycle ( t s ), calibration integration time ( τ c ) and scene integration time ( τ s ) that are used as input for calculating the transfer functions for AMSU-A, ATMS and FY-3C MWTS before and after profile changes .................................................................................................... 81 5.3 Numbers of FOV and NEDT at AMSU-A, ATMS, MWTS channels with central frequencies of 57.290344 GHz ..................................................................................................... 81 vii LIST OF FIGURES 1.1 (a) Global distribution of O-B fields for ATMS channel 8 on February 24, 2012. (b) Same as (a) but for a smaller area ................................................................................................... 6 3.1 A schematic illustration of ATMS scan cycle .................................................................. 19 4.1 Fourier spectra with 81-point running mean for the first six IMFs of (a) warm counts and (b) cold counts at channel 8 .......................................................................................................... 37 4.2 Same as Fig. 4.1 but for warm load temperature. Five IMFs are removed. ..................... 38 4.3 Same as Fig. 4.1 but for the first PC coefficient of scene counts at channel 8. Three IMFs are removed ................................................................................................................................... 39 4.4 Normalized cost function with respect to filter span N for warm counts (upper), cold counts (middle) and scene counts (lower). Numbers of scan lines involved in the optimal filter is indicated in circles ........................................................................................................................ 40 4.5 Weighting coefficients of optimal filters for warm counts (upper), cold counts (middle) and scene counts (lower). .............................................................................................................. 41 4.6 Response function for warm counts (upper), cold counts (middle) and scene counts (lower) calculated with weighting coefficients as in Fig. 4.5. Contours over dark blue shading area all have the value of 0.01 because of oscillations of response functions .............................. 42 4.7 (a) Normalized cost function with respect to filter span N for warm load temperature. Numbers of scan lines involved in the optimal filter is indicated in circle. (b)-(c) weighting coefficients and response function for warm load temperature .................................................... 43 4.8 Variations of (a) warm counts and (b) cold counts along 700 scanlines (i.e., a swath between ±55N ) before (gray) and after (red) applying removing three IMFs at channel 8. ...... 45 4.9 Variations of (a) warm counts and (b) cold counts along 700 scanlines (i.e., a swath between ±55N ) before (gray) and after (red) applying removing three IMFs at channel 22 ..... 46 4.10 Variations of (a) warm counts and (b) cold counts with optimal filter (red) and 17-point smoothing (blue) for channel 8 ..................................................................................................... 47 4.11 Variation of response functions with respect to frequency for the 17-point filter (blue) and the optimal filters (red) applied to warm counts of channel 8 (curve) and channel 22 (circle). The response functions for cold counts are the same as those for warm counts (figure omitted). 48 4.12 Variation of brightness temperature for channel 8 at nadir with (red) and without (blue) applying the optimal filter on the scene counts. Warm count, cold counts and warm load temperatures are all smoothed ...................................................................................................... 49 viii 4.13 Striping noise removed by the optimal symmetric filters for ATMS channels 8, 22 and 1 on ascending node ......................................................................................................................... 50 4.14 (a)-(d) Variation of the normalized cost function with respect to filter span N for ATMS channels 1-22. The cost function value with the selected symmetrical filter span (2N+1) is indicated by circle. (e) The cost function for N=2 used for normalization in (a)-(d) for each channel. ......................................................................................................................................... 51 4.15 Weighting coefficients of symmetric filters for ATMS channels 1-22 ............................ 53 4.16 Response functions of symmetric filters calculated with weighting coefficients. ............ 55 4.17 The power spectral density distributions of the global O-B field of ATMS channels (a) 9 and (b) 10 before (red) and after (blue) removing the striping noise using the optimal filter. ..... 57 4.18 Global distribution of O-B fields (a) before and (b) after applying the optimal filter for ATMS channel 8 on February 24, 2012. (c) Striping noise removed by optimal filter ................ 58 4.19 Striping noise removed by the symmetric filters for ATMS channel (a) 1, (b) 8 and (c) 22 on February 24, 2012 .................................................................................................................... 59 4.20 Maneuver data (a) before and (b) after optimal smoothing on PCA1 for ATMS channel 1. (c)-(d) Same as (a)-(b) but for channel 9. ..................................................................................... 60 4.21 (a) The along-track variance, (b) across-track variance and (c) striping index before (red) and after (blue) smoothing. FOVs 25-72 on 100 scan lines are involved in calculation .............. 61 4.22 Global distributions of striping noise removed by the optimal filter for ATMS channels 1 and 2 on February 24, 2012 .......................................................................................................... 62 4.23 Global distribution of LWP retrieved by ATMS channels 1-2 observation (a) without and (b) with smoothing and (c) the LWP differences between (a) and (b) during 0000-1200 UTC on February 24, 2012. ........................................................................................................................ 64 4.24 Same as Fig. 4.23 but for 1200-2400UTC ........................................................................ 65 4.25 Probability density functions of LWP difference within 30S-30N over ocean on February 24, 2012. (a) LWP retrieved by observations, (b) noise removed at ATMS channel 1 and (c) noise removed at ATMS channel 2 are color coded, respectively. The mean and std of the LWP differences with and without smoothing (i.e., LWPwithout-LWPwith) within 30S-30N over ocean on February 24, 2012 are zero and 0.005 kg m-2. .............................................................................. 66 4.26 Scatter plots of the LWP differences with respect to (a)-(b) striping noise of channels 1 and 2 and (c)-(d) LWP retrieved with observations without smoothing. The magnitude of LWP is indicated in (a) and (b) and the magnitude of striping noise of channels 1 and 2 is indicated in (c) and (d), respectively. Data within 30S-30N over ocean on February 24, 2012 are used ............. 67 ix 5.1 Fourier spectra for the first four IMFs of the (a) first, (b) second, (c) third and (d) fourth PC coefficients at channel 8 for May 1, 2014 ............................................................................... 82 5.2 Fourier spectra for the first six IMFs of the (a) first, (b) second, (c) third and (d) fourth PC coefficients at channel 8 for May 29, 2014. ............................................................................ 83 5.3 Fourier spectra for the first six IMFs of the (a) first, (b) second, (c) third and (d) fourth PC coefficients at ATMS channel 10 for May 1, 2014................................................................. 84 5.4 Spatial distributions of the first five PC modes (i.e., eigenvectors) for the 47th data sample (k=47, each sample includes 200 scan lines) on May 1, 2014 ...................................................... 85 5.5 Spatial distributions of the individual PCA component for the first three PCA components (i.e., the product of eigenvalue and eigenvector) before (left panels) and after (right panels) striping noise mitigation for the 47th data sample (k=47, each sample includes 200 scan lines) on May 1, 2014 ................................................................................................................... 86 5.6 Spatial distributions of ATMS channel 8 (a) observed, (b) simulated and (c) smoothed 3PC/4IMF and (d) 1PC/4IMF brightness temperatures for the 47th data sample (k=47, each sample includes 200 scan lines) on May 1, 2014. O-B differences (e) before and after (f) 3PC/4IMF and (g) 1PC/4IMF noise mitigation ............................................................................ 88 5.7 Noise removed by (a) 3PC/4IMF and (b) 1PC/4IMF as in Fig. 5.6. (c) Differences between (a) and (b) ....................................................................................................................... 90 5.8 Global distributions of channel 8 (a) O-B and (b) difference between smoothed brightness temperatures removed on May 1, 2014 on ascending node. The first four IMFs of the first three PC coefficients are removed for noise mitigation. (c) The removed striping noise ..................... 91 5.9 Global distributions of channel 8 (a) O-B and (b) difference between smoothed brightness temperatures removed on May 30, 2014 on ascending node. The first three IMFs of the first three PC coefficients are removed for noise mitigation. (c) The removed striping noise ..................... 92 5.10 Variations of (a) along-track variances as a function of the sample group number and FOV (i) and (b) cross-track variances as a function of dataset number (k) and scan line (j) before striping noise mitigation on May 1, 2014. The total number of scan lines in each dataset is 200. The total number of datasets for both the ascending and descending nodes on May 1 is 162. (c) Variation of striping index as a function of dataset number (k). .................................................. 93 5.11 Same as Fig. 5.10 except for data with striping noise mitigated (PC3/IMF4).................. 94 5.12 Same as Fig. 5.10 except for data before striping noise mitigation on May 30, 2014...... 95 5.13 Same as Fig. 5.10 except for data with striping noise mitigated (3PC/3IMF).................. 96 x 5.14 (a) Along-track, (b) cross-track variances and (c) striping index at channel 8 during May 1-7, 2014 and May 29-June 4, 2014 before (red) and after (blue) noise. Ascending and descending nodes are indicated in open and solid markers, respectively ..................................... 97 5.15 Square of transfer function (H(f)) that are calculated using instrument parameters of (a) AMSU-A and (b) ATMS with two (red) and five (blue) nearest points being averaged in calibration ..................................................................................................................................... 98 5.16 Same as Fig. 5.15 except for FY-3C MWTS (a) before and (b) after profile change that occurred during May 12-18, 2014 ................................................................................................ 99 5.17 PSD of (a) pitch-over maneuver data at ATMS channel 10, (b) white noise only and (c) the sum of white noise and flicker noise with (colored) and without (black) 81-point running mean ..... ...................................................................................................................................... 100 5.18 The PSD of the noise outputs ( Sn( f ) ) for (a) AMSU-A, (b) ATMS, (c) MWTS before profile change and (d) MWTS after profile change with (red) and without (green) adding flicker noise. ... ....................................................................................................................................... 101 5.19 Differences of the PSD of the noise outputs ( Sn( f ) ) between with and without flicker noise for ATMS (blue), MWTS before profile change (red) and MWTS after profile change (green). ........................................................................................................................................ 102 xi ABSTRACT Advanced Technology Microwave Sounder (ATMS) has been flying on the Suomi National Polar-orbiting Partnership (NPP) satellite since October 28, 2011. A striping noise phenomenon was noticed in the global distribution of O (observations) minus B (model simulations) differences for different ATMS sounding channels. In the first part of this dissertation, a set of “optimal filters” is developed for smoothing out the striping noise in warm counts, cold counts, warm load temperatures and scene counts. Using the two-point calibration equation, antenna temperatures were calculated with and without applying the optimal filters on warm counts, cold counts, warm load temperatures and scene counts. The patterns and magnitudes of the striping noise removed are very close to that from an early method that combines the principal component analysis (PCA) with the Ensemble Empirical Mode Decomposition (EEMD) method. It was also shown that the optimal filters are superior to the conventional boxcar filters in terms of being able to effectively remove the striping noise in the high frequency range but not to alter the lower frequency weather signals. In the second part of the thesis, a set of 22 “optimal filters” that can be applied directly on brightness temperature was also developed to remove the striping noise in all ATMS channels. Impacts of striping noise mitigation on small-scale weather features were investigated by comparing ATMS cloud liquid water path (LWP) retrieved before and after striping noise mitigation. It was shown that the optimal filters do not affect small-scale cloud features while smoothing out striping noise in brightness temperatures. Striping noise is a general problem for microwave sensors, and is also identified within observations of a recent Fengyun-3C (FY-3C) microwave temperature sounder (MWTS). In the third part of the thesis, striping noise within MWTS observation was analyzed. It was found that the magnitude of the striping noise in MWTS is around 1 K, which is much xii larger than in ATMS (~0.3 K). Finally, a transfer function was employed to seek the root cause of the striping noise. This transfer function is controlled by instrument parameters such as scan cycle, calibration integration time and scene integration time. Instrument noise is simulated by a white noise series with and without adding so-called flicker noise. A power spectral analysis was applied to the output noise from transfer function with varying input instrument parameters. It was shown that flicker noise is the source of striping noise, whose magnitude and peak frequency can be modified by different setting of input parameters of transfer function. xiii CHAPTER ONE INTRODUCTION 1.1 History of Satellite-based Microwave Instruments With the successful launch of Television Infrared Observation Satellite -N (TIROS-N) on October 13, 1978, the first Microwave Sounding Unit (MSU) came to use. MSU is a four-channel sounding radiometer designed to monitor temperature in troposphere and lower stratosphere. MSU continued to get on board 8 subsequent National Oceanic and Atmospheric Administration (NOAA) meteorological satellites until its predecessor Advanced Microwave Temperature Sounding Units (AMSU) was deployed in 1998. AMSU consists of Advanced Microwave Sounding Unit-A (AMSU-A) and -B (AMSU-B). AMSU-A is a 15-channel microwave temperature sounder, and AMSU-B is a five-channel microwave humidity sounder. AMSU-A and AMSU-B are on aboard NOAA-15, NOAA-16 and 17. AMSU-B was replaced by Microwave Humidity Sounder (MHS) when NOAA-18 was launched on May 20, 2005. AMSU-A and MHS are on board NOAA-19, European MetOp-A and MetOp-B satellites, which were launched on February 6, 2009, October 19, 2006 and September 17, 2012, respectively. The AMSU was replaced by the Advanced Technology Microwave Sounder (ATMS) when the new US satellite, Suomi National Polar orbiter Partnership (NPP), was launched on October 28, 2011. ATMS combines previous microwave temperature and humidity sounders AMSU-A and MHS into a single package, and is the most advanced, state-of-the-art satellite-based microwave instrument that provides temperature and humidity information at the same 96 fieldof-view (FOV) locations, although FOV sizes are different. Compared with its heritages, ATMS has included an extra temperature channel in the low troposphere and two new humidity channels. 1 ATMS channels 16, 17, 22, 20 and 18 correspond to MHS channels 1-5, respectively. Compared with its predecessors AMSU-A and MHS, ATMS has several other improvements such as wider swath width, additional sounding channels and smaller noise equivalent delta temperatures. Microwave radiance is one of the observations that affect NWP model forecast most. The microwave temperature sounding data such as AMSU observations have been operationally assimilated in the major NWP centers. The assimilation is found to have positive impacts on model forecasts. Microwave humidity sounding data also show the potentials to enhance NWP forecasts in experiments, such as cloud detection. Before those microwave observations can be fully put into use, calibrations and validation are necessary. It is noted those sensors produce some kind of noise, such as white noise and pink noise, whose power spectral density (PSD) are constant or in a form of 1/f, where f stands for frequency. The instrument noise is associated with NEDT (Noise Equivalent Delta Temperature). Noise contained in observations that is larger than NEDT is not negligible. Systematic errors, even much smaller compared to weather signals, would add to the observation error correlations. And large error correlations prohibit the data to be properly assimilated. Major sources of calibration errors include target emissivity, radiometric leakage, measurement uncertainty, inappropriate use of Rayleigh approximation and antenna side lobe interception. In order to smooth out the systematic noise, filters are designed and applied to observation data. There are two types of digital filters: recursive filters and non-recursive filters. Those filters are sometimes referred to as infinite impulsive filter and finite impulsive response filter, respectively. The latter type has much lower order than the former one and thus much less complicated. Fourier filters belong to recursive filters and averaging filters belong to nonrecursive filters. 2 1.2 Striping Noise Contained in ATMS Observations Satellite microwave temperature and humidity sounders had played an important role in numerical weather prediction for several decades. For satellite data assimilation, biases and errors of brightness temperatures observations must be quantified. ATMS data characteristics and biases were carefully discussed in Weng et al. (2012). A comprehensive discussion on ATMS calibration, instrument sensitivity, conversion from antenna temperature to brightness temperature, and the dependence of polarization difference on sea surface wind and sea surface temperature can be found in Weng et al. (2013). The radiance simulated by an atmospheric radiative transfer model with global numerical weather prediction (NWP) datasets can serve as the reference to verify the observed radiance. Borman et al. (2013) and Qin et al. (2013) reported an along-track, striping type of noise in the global distribution of differences between observations and model simulations (O-B) at ATMS temperature sounding channels. A striping type of noise was found in the global distribution of the differences between observations (O) and simulations (B) as in Fig. 1.1 (a), suggesting a contamination of striping noise in ATMS observations. This striping pattern of this noise is clearer in the distribution of zoomed-in portion as in (b). Qin et al. (2013) further confirmed that similar striping noise exists in ATMS humidity sounding channels and all other heritage humidity sounders (i.e., AMSU-B and MHS), but not AMSU-A channels. Striping noise may be caused by receiver gain fluctuations (i.e., the 1/f noise) that are associated mainly with the RF amplifiers (Ulaby et al., 1981; Hersman and Poe, 1981). AMSU-A does not have any amplifiers in front of the receiver. Analysis of striping noise structures and magnitudes as well as an effective mitigation of the striping noise from ATMS brightness temperatures are necessary for maximizing the role of ATMS observations for NWP 3 and climate study. Such studies may also offer some insights on the root cause of the striping noise. 1.3 Existing Methods to Mitigate Striping Noise A detailed description of converting raw data counts to antenna brightness temperature through ATMS calibration process is provided by Weng et al. (2012) and ATMS Advanced Technical Baseline Documentation (ATBD). The antenna temperature is calculated with a twopoint calibration, in which the earth scene counts are converted into antenna temperature through a linear relationship defined by warm counts, cold counts, warm load temperature and cold space temperatures. Those raw counts data are voltage records taken during each scan cycle. A quadratic term that accounts for the nonlinear relationship between antenna temperature and counts is also added. In order to reduce the effect of radiometric instrument errors on antenna temperature, warm counts, cold counts and warm load temperatures are traditionally smoothed in an operational system using either a triangular or a boxcar filter. Huang and Wu (2004; 2009) proposed an Ensemble Empirical Mode Decomposition (EEMD) method for extracting from raw data different frequency components. For a given time series, the EEMD uses maximum and minimum information of the riding waves in the set of noise-added ensemble data series itself and extracts successively the oscillatory components, which are called intrinsic mode functions (IMFs), from the highest to lowest frequencies. Unlike the pre-existing harmonic base functions of Fourier Frequency Transform (FFT), the IMFs of EEMD are locally adaptive basis functions that are extracted directly from the data, and thus become physically meaningful representations. The EEMD method also works for data series representing nonlinear processes. Wu and Huang had shown that Fourier spectra of IMFs 4 extracted from white noise series have similar and predictable patterns, confirming that the EEMD method is capable of separating the true signals from the noise. Qin et al. (2013) pointed out that striping noise are contained in the first Principle Component of ATMS observations and applied the EEMD method to extract the ATMS striping noise. This method not only can remove the striping noise in temperature sounding channels, but also humidity channels for which the striping phenomenon is not visible in O-B distributions because of larger ranges. Specifically, the first three high frequency IMFs are extracted from the first PC coefficient of brightness temperatures. The removed striping noise has a frequency range centered around 10-2 s-1 and has a magnitude of about 0.3 K for ATMS temperature channels and 1 K for ATMS humidity channels. 1.4 Dissertation Organization This dissertation is organized as follows. Chapter 2 provides a description of all data used in the work. Chapter 3 presents methodology on transformations from raw counts data to antenna temperatures, as well as the mathematical expressions of EEMD/PCA and the “optimal” filters. Numerical results of striping noise analysis and mitigation for both ATMS and MWTS brightness temperatures are shown in Chapter 4, along with an investigation on the root cause of the striping noise. Chapter 5 provides a brief summary and future plan. 5 (a) (b) Fig. 1.1: (a) Global distribution of O-B fields for ATMS channel 8 on February 24, 2012. (b) Same as (a) but for a smaller area. 6 CHAPTER TWO DATA 2.1 The Advanced Technology Microwave Sounder (ATMS) ATMS is a cross-track, line-scanning sensor. It takes 2.67 seconds to complete a single scan-line, which contains 96 FOVs for all 22 ATMS microwave temperature and humidity sounding channels. With a sampling interval of 1.11o, the outmost scan-angle is 52.7o. ATMS channels 1-3, 5-16 have similar central frequencies as AMSU-A channels 1-15 of the traditional microwave temperature sounding instrument AMSU-A, and ATMS channels 17-22 contain channels of the similar central frequencies as the five channels from the traditional humidity sounding instrument MHS. Compared with its predecessor AMSU-A and MHS, ATMS has an extra temperature channel 4 with its weighting function located in the lower troposphere and two new humidity channels 19 and 21. The swath width of ATMS is 2500 km, which is wider than both AMSU-A and MHS, leaving almost no data gaps over the global. The beam widths of channels 1-2, 3-16 and 17-22 are 5.2o, 2.2o and 1.1o, respectively. ATMS data can be downloaded from NOAA Comprehensive Large Array-data Stewardship System (http://www.class.ncdc.noaa.gov/saa/products/welcome) in HDF format. 2.2 The Community Radiative Transfer Model (CRTM) The Community Radiative Transfer Model (CRTM) was developed for rapid calculations of satellite radiances and other derived variables under various atmospheric and surface conditions by the US Joint Center for Satellite Data Assimilation (JCSDA) (Weng 2007). The model (Han et al. 2007) is used in the work to produce global simulations of brightness temperatures. The simulations are served as background field to verify the microwave observations. 7 Variables fed to CRTM are those from satellite observations such as latitude, longitude, scan angle, zenith angle, and those from GFS data including surface type, surface temperature, 10 m wind speed; vertical profile of pressure, temperature and mixing ratio. The output of CRTM includes brightness temperature, optical length, etc. 2.3 NCEP GFS Data National Center for Environmental Prediction (NCEP)’s Global Forecast Systems (GFS) 6hour forecast fields are used to produce model simulations. Horizontal resolution of GFS data is 1ox1o and there are 26 vertical levels at 10.0, 20.0, 30.0, 50.0, 70.0, 100.0, 150.0, 200.0, 250.0, 300.0, 350.0, 400.0, 450.0, 500.0, 550.0, 600.0, 650.0, 700.0, 750.0, 800.0, 850.0, 900.0, 925.0, 950.0, 975.0, 1000.0 hPa, GFS data from March 2004 to present is available at http://nomads.ncdc.noaa.gov/data/gfsanl/. 8 CHAPTER THREE METHODOLOGY 3.1 Derivation of Antenna Temperature from Raw counts During each ATMS scanning cycle of 2.67s, the antenna firstly scans the earth scenes, then the cold space, and finally the blackbody warm target to record the measured scene counts, warm counts and cold counts when these three segments are completed. A schematic illustration of ATMS scan cycle is shown in Fig. 3.1. The antenna brightness temperature can be derived from the measured raw counts based on the following two-point calibration equations. Tb,ch (k,i) ≡ Tchw (k) + (Gch (k))−1 (Cchs (k,i) − Cchw (k)) + Qch (k,i) (1) Cchw (k) − Cchc (k) Gch (k) = w Tch (k) − Tchc (k) where the subscript “ch” represents the channel number, Gch (k) is the gain function, Cchs (k,i) stands for scene count at the ith FOV of the kth scanline, Cchc (k) and Cchw (k) are cold count and warm count at the kth scanline, respectively; Tchw (k) is the warm load temperature at the kth scanline, and Tchc (k) is the cold space temperature, which is fixed at each channel. The overbar “ ” on Tchw , Cchw and Cchc indicates a boxcar smoothing warm load temperature, warm count and cold count, respectively. Qch (k,i) is a quadratic correction term which can be written as follows: Qch (k,i) = b0,ch × (1− 4 × ( where b0,ch is a quadratic coefficient. 9 Tb,ch (k,i) − Tchc (k) − 0.5)2 ) Tchw (k) − Tchc (k) (2) Both the warm load temperature Tchw (k) and the cold space temperature Tchc (k) have been corrected by ΔTchw and ΔTchc , respectively, to account for channel biases. More details of ATMS calibration process from raw data counts to antenna brightness temperature can be found in Weng et al. (2013) and ATMS ATBD (2012). 3.2 Mathematical Expressions of Optimal Filters 3.2.1 Optimal Filters for Warm Counts and Cold Counts { } { } The method employed to extract the noise in the data series Cchw (k) , Cchc (k) and {T w ch } (k) (k=1, 2, …, K) is the EEMD method (Wu and Huang, 2004). Here Cchw (k) , Cchw (k) and Tchw (k) stand for the warm counts, cold counts and warm load temperatures, respectively, at the kth scanline of channel “ch” indicated as the subscript. For any time series of data, any of the above data series, to be denoted as ( C ( k ) , k=1, 2, …, K) for simplicity, is decomposed into a set of “intrinsic mode functions” (IMFs): L C (k ) = ∑ IMFm (k ) + RL (k ) , (3) m =1 where RL is the residual of the data C ( k ) after the first L number of IMFs have been extracted. In order words, the signals in the data series are decomposed successively from high frequency to low frequency. The high-frequency noise is described by the first few IMFs. { } Therefore, the warm counts, cold counts and warm load temperature data series Cchw (k) , {C c ch } { } (k) and Tchw (k) (k=1, 2, …, K) can be smoothed by removing the first few high-frequency IMFs to obtain the EEMD-smoothed data series Cchw,eemd (k) , Cchc,eemd (k) and Tchw,eemd (k) are as follows: 10 Lw Cchw,eemd (k) = Cchw (k) − ∑ IMFm (k) (4a) m=1 Lc Cchc,eemd (k) = Cchc (k) − ∑ IMFm (k) (4b) m=1 Lt Tchw,eemd (k) = Tchw (k) − ∑ IMFm (k) (4c) m=1 The above EEMD-smoothed warm counts, cold counts and warm load temperatures, Cchw,eemd (k) , Cchc,eemd (k) and Tchw,eemd (k) , will be used as the reference constraints for developing the optimal filters for ATMS striping mitigation. Instead of using the EEMD based method, we may apply an optimal (2N+1) point filter to { } { } { } each of the data series Cchw (k) , Cchc (k) and Tchw (k) (k=1, 2, 3, …, K) to obtain three { } { } { } smoothed data sets Cchw,opt (k) , Cchc,opt (k) and Tchw,opt (k) as follows: C w,opt ch N ∑α (k) = Cchw (k + n) w n (5a) n=− N N Cchc,opt (k) = ∑α C c n c ch (k + n) (5b) (k + n) (5c) n=− N N Tchw,opt (k) = ∑αT t w n ch n=− N where α nw , α nc and α nt (n=0, ± 1, …, ± N) are the optimal weighting coefficients for the optimal symmetrical filters on warm count, cold count and warm load temperature, respectively, and N is the total number of scan lines involved in each filter. The method for obtaining the optimal weighting coefficients is described below. 11 The effect of filter defined in equations (5a)-(5c) on the data can be examined by comparing the spectrum of the original data series with that of the smoothed series. Taking the warm counts as an example. Let’s apply Fourier transform to both the original data sequence {C w ch } { } (k) and the smoothed data sequence Cchw,opt (k) : K −1 C (k) = ∑ f me− imkβ w ch (6a) m=0 and K −1 Cchw,opt (k) = ∑ f me− imkβ (6b) m=0 where β = 2π , m is wave number (m=0, 1, …, K-1), f m and f m are Fourier coefficients which K can be written as: 1 K −1 w Cch (k)e− imkβ ∑ K k=0 (7a) 1 K −1 w,opt ∑ C (k)e− imkβ K k=0 ch (7b) fm = and fm = The ratio of the Fourier coefficient of the filtered data series f m and the Fourier coefficient of the filtered data series f m rm = fm fm is called the response function ( rm ) of the filter defined by equation (5a). 12 (8) Once designed, the filter weighting coefficients are not updated regularly, and the filter are applied to global data for both ascending and descending nodes. Thus the filter is symmetrical. For a symmetrical filter, αn = α− n . Equation (5a) can be written as Cchw,opt (k) = 1 N α n (Cchw (k − n) + Cchw (k + n)) ∑ 2 n=0 (9) By substituting (6a) and (7a) into (9) we may obtain C w,opt ch 1 K −1 1 N = ∑ ∑ α n (Cchw (k − n) + Cchw (k + n))e− imkβ K k=0 2 n=0 N = ∑ α n cos(mnβ ) f m (10) n=0 Substituting (10) into (8) we may then obtain an analytic expression for the spectral response function rm : N rm = ∑ α nw cos(mnβ ) (11) n=0 The weighting coefficients for the symmetric filters for warm counts are obtained by requiring that the filter-smoothed warm counts approximately equal to the EEMD-smoothed ones, i.e., K N ⎧ min J = min ( α nwCchw (k + n) − Cchw,eemd (k))2 ∑ ∑ ⎪ ⎪ k=1 n=− N ⎨ N ⎪ ∑ α chw = 1 ⎪⎩ n=− N (12) The constraint minimization problem in (12) is solved using Lagrange method: N ⎧ w L( α , λ ) = J + λ (1− α nw ) ∑ ⎪ n ⎪ n=− N ⎨ ⎪ ∂L = 0, ∂L = 0 w ⎪ ∂λ ⎩ ∂α ch 13 (13) Similar procedures are applied to cold counts and warm load temperatures. 3.2.2 Optimal Filters for Scene Counts Different from warm counts, cold counts, and warm load temperatures, scene counts (i.e., the voltage records associated with observations at 96 FOVs) vary not only in the along-track direction, but also the across-track direction. The striping noise remains nearly constant in the along-track direction. A Principle Component Analysis (PCA) method is firstly used to extract the first PCA component that captures mainly the scan-dependent features of scene counts from the cross-track radiometer ATMS. The EEMD- and optimal- filtering approaches are applied to the first PCA component. The data matrix for the PCA analysis of scene counts can be expressed as: C Ms ×N s s ⎛ Cs C1,2 C1,N 1,1 ⎜ s s s C2,2 ! C2,N ⎜ C = ⎜ 2,1 " # " ⎜ " s s s ⎜⎝ C M ,1 C M ,2 ! C M ,N ⎞ ⎟ ⎟ ⎟ ⎟ ⎟⎠ (14) where M is the total number of scanlines and N is the total number of FOVs. The scene counts at the ith FOV of the kth scanline, Cks ,i , can be expressed as follows: 96 C = ∑ e j,iuk , j s k ,i (15) j=1 ! where e j,i is the ith (scan position) element of the jth PC mode e j , uk , j is the kth (scanline) element ! of the jth PC coefficient u Tj . The first L intrinsic mode functions (IMFs) of the first PC coefficient, which describe the striping noise, are removed and the remaining PCA components are kept the same to obtain the EEMD-smoothed scene counts 14 L 96 m=1 j=2 s,eemd Ci,k = ei,1 (u1,k − ∑ IMFm (k)) + ∑ ei, j u j,k (16) A (2N+1)-point filter can be applied to obtain a filter-smoothed scene count data series {C } as follows: s,opt i,k Cks,opt = ,i N 96 n=− N j=2 ∑ α nsei,1u1,k+n + ∑ ei, ju j,k (17) where α ns (n=0, ±1, …, ±N) are the optimal weighting coefficients for the scene counts using a similar procedure described in section 3.2.1. 3.2.3 Optimal Filters for Brightness Temperatures The data matrix of ATMS observed brightness is expressed as: AM × N ⎛ y ! y N ,1 1,1 ⎜ =⎜ " # " ⎜ y ⎝ 1,M ! y N ,M ⎞ ⎟ ⎟ ⎟ ⎠ (18) where yk ,i is the brightness temperature observed at the ith FOV on the kth scan line. M is the total number of FOVs, for ATMS, M=96; N is the total number of scan lines. With a PCA method, the above data matrix could be decomposed into several PC modes, as follows: 96 !! A = ∑ e j u Tj (19) j=1 ! ! where the jth mode is the product of the PC coefficient u j and PC component e j . The PC coefficient describes the along-track features while the PC component is a function of the FOV (Qin et al., 2013). The first mode explains the largest portion of the total variance. Equivalent to equation (19), brightness temperature at each single data point is written as: 15 96 yk ,i = ∑ e j,iuk , j (20) j=1 An EEMD method is used to remove striping noise in ATMS observations {yk ,i } to obtain { } smoothed brightness temperatures ykeemd , as in Qin et al. (2013). The first three intrinsic mode ,i functions (IMFs) of the first PC coefficient uk ,1 are removed while the other PC components are not changed, i.e.: 3 96 m=1 j=2 yk,ieemd = e1,i (ui,1 − ∑ IMFm (k)) + ∑ e j,i uk, j (21) L eemd The EEMD-smoothed first PC coefficient is expressed as: uk,1 = uk,1 − ∑ IMFm (k) , where L m=1 indicates number of IMFs to remove. Although the combination of EEMD and PCA methods is effective to detect and mitigate the visible striping noise, it is not convenient to implement in operational systems. A set of optimal filters that removes the striping noise extracted by PCA/EEMD is thus developed. For the same set of ATMS observation data {yk ,i } as in equations (20), applying a (2N+1) point filter to obtain a smoothed first PC coefficient data set {uk,1 } yields: uk,1 = N ∑α u n k+n,1 (22) n=− N where α n (n=0, ± 1, …, ± N) are weighting coefficients and N is the total number of data points involved in the filter. 16 We aim at obtaining the optimal weighting coefficients α n (n=0, ± 1, …, ± N) so that the filter-smoothed first PC coefficient approximately equal to the EEMD-smoothed ones, i.e., to minimize the following cost function J: K N eemd 2 J = ∑ ( ∑ α nuk+n,1 − uk,1 ) (23) k=1 n=− N And there are two constraint conditions: N ∑α =1 (24a) α n = α −n (24b) n n=− N Equation (24a) indicates that mean value does not change, and equation (24b) requires that the filters are symmetrical. The constraint minimization problem stated in equations (23)-(24) is solved using Lagrange method: N ⎧ L( α , λ ) = J + λ (1− ∑ αn ) n ⎪ ⎪ n=− N ⎨ ⎪ ∂L = 0, ∂L = 0 ⎪⎩ ∂α n ∂λ (25) ! where λ is the Lagrange multiplier. And the analytic solutions of weighting coefficients α can be obtained by combining equations (24) and (25): ! α = B −1C (26) where ⎛ α 0 ⎜ ! α α = ⎜⎜ 1 ⎜ " ⎜⎝ α N 17 ⎞ ⎟ ⎟ ⎟ ⎟ ⎟⎠ (27a) ⎛ ⎜ ⎜ ⎜ ⎜ B = 2⎜ ⎜ ⎜ ⎜ ⎜⎝ K ∑u K ∑u 2 k,1 k=1 k=1 K ∑u ∑u u k+1,1 k,1 K ∑u ! 2 k+1,1 k=1 " # " K ∑u u k+N ,1 k,1 k=1 u k+1,1 k+N ,1 k=1 " K u k,1 k+N ,1 k=1 K k=1 ∑u K ∑u ! u k,1 k+1,1 K ∑u ! u k+N ,1 k+1,1 k=1 2 k+N ,1 k=1 K ⎛ eemd ⎜ λ + 2∑ uk,1uk,1 k=1 ⎜ K ⎜ eemd ⎜ λ + 2∑ uk+1,1uk,1 C=⎜ k=1 ⎜ ! ⎜ K eemd ⎜ λ + 2∑ u k+N ,1u k,1 ⎜⎝ k=1 K ⎛ 2 ∑ uk,1uk,1eemd ⎜ k=1 ⎜ K ⎜ eemd ⎜ 2 uk+1,1uk,1 D=⎜ ∑ k=1 ⎜ ! ⎜ K ⎜ 2∑ u u eemd ⎜⎝ k=1 k+N ,1 k,1 ⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟⎠ ⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟⎠ ⎞ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟ ⎟⎠ (27b) (27c) (27d) With the Lagrange multiplier solved as: N +1 N +1 λ= 1− ∑ ∑ (B −1 )ij D j i=1 j=1 N +1 N +1 ∑ ∑ (B −1 (28) )ij i=1 j=1 And the optimally smoothed brightness temperatures could then be expressed as: y opt k,i = e1,i N ∑α u n k+n,1 n=− N 18 96 + ∑ e j,i uk, j j=2 (29) Fig. 3.1: A schematic illustration of ATMS scan cycle. 19 CHAPTER FOUR ATMS STRIPING NOISE MITIGATION USING THE OPTIMAL FILTERS 4.1 Striping Noise Mitigation Within Sensor Calibration Counts 4.1.1 Spectrum Analyses of IMFs of Calibration Counts Data series of warm counts, cold counts, warm load temperature and the first PC coefficient of scene counts for all ATMS channels within 55S-55N on February 24, 2012 are decomposed into different IMFs. The Fourier spectra of the first six IMFs with respect to frequency for warm and cold counts at channel 8 are shown in Fig. 4.1. It is seen the higher the IMF number is, the lower frequency the peak amplitude locates. White noise and flicker noise at high frequencies are captured by the first few IMFs, while the large-scale variations are in the remaining IMFs. A main criterion to decide which IMFs are noise, or how many IMFs to remove, is whether the peak amplitude of a certain IMF is at the striping frequency range (>0.01 Hz). Also, magnitudes of the noise-related IMFs are not too large, otherwise weather signal might be removed as well. For both warm counts and cold counts, beginning at the fourth IMF, peak amplitude is no longer smaller than that of the previous IMFs. It is thus determined that the first three IMFs are the data noise. Also magnitudes of those three IMFs at low frequencies are quite small, suggesting that removing the first three IMFs will not change the features of warm counts and cold counts at the low frequencies. Similar results are obtained for the other 21 ATMS channels. Thus it is decided the first three IMFs are removed for both warm counts and cold counts at all ATMS channels. The Fourier spectra of the first six IMFs for warm load temperature are shown in Fig. 4.2. Notice that the 6th IMF peaks at close frequency to the 5th IMF, but with obviously smaller amplitude, which suggests that starting at the 6th IMF there is no more significant signal at even 20 lower frequency. And unlike warm counts or cold counts that change in the along-track direction, ideal warm load temperature should be constant, implying a zero Fourier spectra for the true signal. Thus a total of five IMFs are removed from warm load temperatures. IMFs are removed. Fourier spectra of the first six IMFs of the first PC coefficient for scene counts at channel 8 are shown in Fig. 4.3. It is seen that the 4th IMF has much larger magnitudes than the third IMF not only at the peak but also at the low frequencies. Thus to remove the first three IMFs is enough to eliminate the noise while the large-scale information is kept. Similar analyses are carried out for 22 channels. The total numbers of IMFs to be removed from scene counts for all ATMS channels are listed in Table 4.1. For ATMS window channels 1-2 and 16, only two IMFs are sufficient to capture the data noise and need to be removed. For the remaining channels, three IMFs are to be removed. 4.1.2 Properties of the Optimal Symmetrical Filters As described in sections 3 and 4, the EEMD-smoothed warm counts, cold counts, warm load temperature and scene counts are then taken as standards for constructing optimal filters as expressed in equations (5) and (17). In order to decide the filter span N, weighting coefficients and cost functions of (2N+1) point symmetrical optimal filters with different span ranging from 2 to 30 are all calculated. Cost functions describe how close the counts smoothed by EEMD to those by the optimal filter. Since cost functions at different channels have different magnitudes, variations of the normalized cost functions of the optimal filters with respect to filter span are shown in Fig. 4.4 for warm counts, cold counts and scene counts at all ATMS channels. It is seen cost functions drop rapidly with the filter span N, suggesting that optimal filters tend to capture the EEMD feature more closely if more scan lines are involved. It is hoped the striping noise be 21 removed with the smallest possible numbers of scan lines needed in the optimal filter. The chosen filter spans of the optimal filters are indicated in black circles and are shown in Tables 4.1 and 4.2. Surface channels have narrower filters mainly because there are only two IMFs removed so that the changes to counts data are smaller. Temperature sounding channels have wider filters than humidity sounding channels for warm counts and cold counts, but narrower filters for scene counts. Weighting coefficients of the optimal filters for warm counts, cold counts and scene counts at all channels are presented in Fig. 4.5. All weighting coefficients are of symmetrical parabolic shapes, suggesting that data points with identical distance to the filtering point weigh the same within any filter, and those closer to the filtering point (n=0) are weighed more than data points further away. This result indicates the filtering points themselves carry most importance while data points further away have less impact on the smoothed counts, which is reasonable. Weighting coefficients with the same distance to filtered points are of similar magnitudes for warm counts and cold counts at different channels. Scene counts at channels 1-2 and 16 have larger weighting coefficients than those for the other channels because sum of the weighting coefficients of any filter is equal to unit and a fewer scan lines are involved in these three window channels. Figure 4.6 shows the response functions calculated with the optimal weighting coefficients shown in Fig. 4.5 according to equation (11). For warm counts, cold counts and scene counts at all channels, response functions remain to have a unit value until they drop sharply to around zero at the frequencies of 10-2 s-1. This indicates that the magnitudes of low-frequency waves are not altered while the high-frequency noise is significantly reduced. The optimal filters make very little changes to the counts data at low frequencies, and the majority of the noise removed 22 centered greater than the frequency of 10-2 s-1, which implies that optimal filters removes the striping information while retaining large-scale signals. Contours over dark blue shading area all have the value of 0.01 because of oscillations of response functions. For scene counts, response functions at the window channels 1-2 and 16 start to decrease from 1 to zero at higher frequencies, suggesting that more signals are retained, which agree with the fact that only two IMFs are removed, compared to three IMFs at other channels. Although corrections might be different among ATMS channels, warm load temperatures do not change with channel numbers. The cost function, weighting coefficients and response functions for warm load temperatures are shown in Fig. 4.7. 4.1.3 Comparison between the Optimal Filters and the Boxcar Filters Figures 4.8 and 4.9 show the warm counts and cold counts before and after applying optimal filters at channels 8 and 22. It is seen that after the optimal filters are applied, warm counts and cold counts become much smoother without any visible noise at both channels while the larger-scale variations are kept. This proves that optimal filters are effective. Conventionally 17-point boxcar filters are used to smooth warm counts and cold counts. Variations of warm counts and cold counts after applying both the optimal filter and the boxcar filter are shown in Fig. 4.10. Compared with the optimal filters, the boxcar filter not only fails to smooth out all visible noise, but also reduces the minimums and maximum values. Response functions for the optimal filters and the boxcar filters for channel 8 with the same filter widths of 8 are also calculated. Figure 4.11 shows that the response function for the optimal filter fall off from 1 to around 0 when frequency increases, which means after smoothing, magnitudes of low frequency signals are kept the same while that of high frequencies are significantly reduced. However, response function for the boxcar filter starts to decrease 23 dramatically at the frequencies lower than 10-2 s-1, implying that the boxcar filter suppresses the low-frequency signals that should be retained. Another problem is that at the high frequency range, where the major part of noise exists, the oscillatory behavior of the response function of boxcar filters hinders this filter from completely removing the actual data noise. 4.1.4 The Effects of Optimal Filters on Antenna Temperatures Conventionally antenna temperatures are calculated with only warm counts, cold counts and warm load temperature smoothed while the scene counts are not smoothed. Figure 4.12 shows antenna temperatures for ATMS channel 8 at nadir calculated with and without applying the optimally filters to the scene counts. For both cases, all of warm counts, cold counts and warm load temperatures are smoothed using the optimal filters. It is seen that even if noise in warm counts, cold counts and warm load temperature are all effectively smoothed out, the antenna temperatures are still contaminated with striping noise if the scene counts are not smoothed. This confirms the necessity of smoothing the scene counts. Global distributions of the removed noise in antenna temperatures for temperature sounding channel 8, humidity sounding channel 22 and window channel 1 are shown in Fig. 4.13. The striping noise are around 0.3 K for channel 8 and 1K for channels 22 and 1, which agrees with the EEMD results in Qin et al. (2013). 4.2 Striping Noise Mitigation Within Brightness Temperature 4.2.1 Weighting Coefficient of the Optimal Filters Data series of brightness temperatures at all 22 ATMS channels within 55S-55N on February 24, 2012 are used as sample data to calculate for weighting coefficients of optimal filters. Similar to Qin et al. (2013), the first PC coefficients are first decomposed into different IMFs with the PCA/EEMD methods. It is decided that two IMFs are removed for window channels 1, 2 and 16 24 and three IMFs are removed for all the rest channels. Those EEMD-smoothed first PC coefficients are then served as the references for establishment of optimal filters. Cost functions calculated by equation (23) for all ATMS channels with respect to filter widths N re shown in Fig. 4.14. The 22 channels are divided into four groups, which are window channels 1, 2 and 16, tropospheric temperature sounding channels 3-9, stratospheric temperature sounding channels 10-15, and water vapor channels 17-22. It is seen with the increase of filter spans, cost functions drop sharply at first for all channels, and then decrease slowly until finally change little. This behavior suggests that the optimally smoothed first PC coefficients are closer to EEMD-smoothed ones when more scan lines are involved in the filters. However, for sufficiently wide filters, even if more scan lines are added, the differences between EEMD-smoothed and optimally smoothed first PC coefficients remain the same. Those sufficiently large filter spans are then chosen as the filter spans of our optimal filters, which are indicated by circles. The behaviors of cost functions for different channels are similar, but the ranges of magnitudes are large. Cost functions values are divided by the cost function with filter span N=2 at each channel to obtain normalized cost functions as shown. The magnitudes of cost functions with filter spans of two are shown in Fig 4.14(e). The filter spans of optimal filters for all 22 channels are as listed in Table 4.3. Weighting coefficients of the optimal filters with respect to distances (n) to the filtering point, where n=0, are demonstrated in Fig. 4.15. It is seen that weighting coefficients at the points closer to the filtering points are weighed more than those at the points further away, which implies that the points with smaller distances to the filtering points carry more importance and have larger impacts on the smoothed data than those points with larger distances. Two IMFs are removed for window channels 1, 2 and 16 and three IMFs are removed for other channels. Weighting 25 coefficients at filtering points for these three window channels are larger than other channels, which indicates that less changes are made to the data, and the filtering points themselves are weighed more. Spectral response functions as in equation (11) are shown in Fig. 4.16. For all 22 channels, response functions are unit values at the low-frequency range, meaning the magnitudes of largescale waves at that frequency are kept un-altered. At frequencies greater than 10-2 s-1, where the diagnosed striping noise resides (Qin et al., 2013), the response functions plumb to zero, implying the magnitudes of the high-frequency noise is significantly reduced. 4.2.2 Brightness Temperatures before and after Striping Noise Mitigation The above 22 optimal filters for the 22 ATMS channels are applied to brightness temperature according to equation (29). Spectra of differences between observed and simulated brightness temperatures (i.e., O-B differences) before and after applying optimal filters for ATMS channels 9 and 10 are shown in Fig. 4.17. Here, simulated brightness temperatures are generated by using Community Radiative Transfer Model (CRTM) with GFS 6-hr forecast field. It is seen that magnitudes are not changed at all frequencies except for the deep decreases at the frequencies greater than 10-2 s-1, which is the lower frequency boundary of the identified striping noise. This result indicates that the majority of signals removed by optimal filters is at the same frequencies as EEMD/PCA and confirms that the optimal filters are effective alternatives to the EEMD/PCA approach. Global distributions of the O-B differences before and after applying the optimal filters for ATMS channel 8 are displayed in Fig. 4.18. It is seen that the once obvious striping noise contained in the O-B are not detectable any more with noise mitigation using the optimal filter. The distribution of removed noise is displayed in Fig. 4.18(c), which are of a magnitude of 0.3K 26 and in a striping pattern. Global distributions of removed noise at channels 1, 8 and 22 are shown Fig. 4.19. The magnitudes of striping noise for window channels, temperature channels and humidity channels are 1 K, 0.3 K and 1K, respectively. A small reduction in the standard deviation of O-B results from the striping noise filtering. For example, the standard deviations of the global O-B fields before striping mitigation are 0.484 K and 0.471 K with and without subtracting scan biases, respectively, using data on February 24, 2012. These two numbers are reduced to 0.476 K and 0.463 K, respectively, after filtering striping noise. If a spatial average is applied to the ATMS temperature sounding channels before their assimilation into NWP models, the reduction due to the striping noise filtering may be more significant. In order to further validate the optimal filters, similar approach as described in Section 2 is applied on ATMS pitch-over maneuver data. Acquired by only scanning the cold space, the pitchover maneuver data are not affected by weather systems and thus well reflect the noise-related characteristics of the instrument itself (Weng et al., 2013; Qin et al., 2013). Pitch-over maneuver data at Channels 1 and 9 before and after applying optimal filters are shown in Fig. 4.20. It is seen that after smoothing, the striping noise contained in the pitch-over maneuver data are significantly reduced and contain mainly random noise signals for both channels. A striping index (SI) for pitch-over maneuver data is defined as follows (Qin et al., 2013): SI = σ along−track σ cross−track (30) where σ along−track and σ cross−track stand for along-track variances and cross-track variances, respectively. Data contaminated with striping noise have a much larger along-track variance than cross-track variance, and SI exceeds one. By comparison, magnitude of along-track variance for 27 data free from striping noise is of close to that of cross-track variance. Such an ideal SI for striping-noise-free data tends to be around unit value. Along-track variances, cross-track variances and SIs before and after applying the optimal filters for pitch-maneuver data at all 22 ATMS channels are shown in Fig. 4.21. It is seen that after applying the optimal filters, alongtrack variances decrease sharply while cross-track variances change little, and SIs decreases to unit values at each channel, which again confirms the effectiveness of optimal filters. 4.2.3 Liquid Water Path Retrieval Algorithm The cloud liquid water path (LWP) is the total vertically integrated mass of liquid water per unit horizontal area in units of kg/m2. A general expression of LWP (L) is written as: ztop L= ∫ ρ l dz (31) zbottom where ρ l is the liquid water density. LWP can be retrieved using measurements at AMSU window channels 1 and 2, as well as surface temperature, surface emissivity, satellite zenith angle (Weng et al., 1994; Weng et al., 2003). The algorithm is expressed as follows: L = a0 µ[ln(Ts − TbCh2 ) − a1 ln(Ts − TbCh1 ) − a2 ] (32) where TbCh1 and TbCh2 are brightness temperatures observed at AMSU-A channels 1 and 2, respectively, Ts is surface temperature, µ is the cosine function of zenith angle. a0 , a1 and a2 are coefficients as follows: a0 = −0.5κ vch1 / (κ vch1κ lch2 − κ vch2κ lch1 ) a1 = κ vch2 / κ vch1 (33a) (33b) a2 = −2.0(τ och2 − a1τ och1 ) / µ + (1.0 − a1 )ln(Ts ) + ln(1.0 − ε ch2 ) − a1 ln(1.0 − ε ch1 ) (33c) 28 where κ v and κ l are mass absorption coefficients for water vapor and cloud liquid water, respectively, ε is the surface emissivity, τ o is the oxygen optical thickness and is taken as a constant for a given channel. Rational for the linear relationship between LWP and logarithm of window channel observations is presented as follows. Wavelength of microwave is at least 50 times larger than a typical rain droplet, so even at the low end with a wavelength of 3mm, the scattering effect could be neglected. With further assumption that atmosphere is isothermal, at the frequencies below 40 GHz, for both window channels 1 and 2, there exists the following relationship between observations and surface temperatures: Tb = Ts [1− τ 2 (1− ε )] (34) where the optimal depth ( τ ) is written as the product of the optimal depths of water vapor ( τ H 2O ), oxygen ( τ O ) and liquid water droplets ( τ liquid ) as follows: τ = τ h2oτ oτ liquid (35) with τ H 2O = exp(− ∫ ztop zbottom κ v ρ v (z)dz ) = exp(−κ v × PW ) (36) ρ v representing the density of water vapor, and PW the total precipitable water vapor. For non-precipitating clouds, liquid water transmittance relies not on droplets sizedistribution but only on LWP, temperature and frequencies (Grody, 1976): τ liquid = exp(−Kν 2 L) (37) where ν is the channel central frequency, K is a coefficient which is a function of only mean cloud temperature. One way to interpret Equation (37) is to calculate optimal depth for liquid water droplet similar to Equation (36), which is written as: 29 1 κ l ρ lVl (z)dz ) = exp(−κ l × L) zbottom σ τ liquid = exp(− ∫ ztop (38) The above equation is only valid when mass absorption coefficient is independent of droplet radius (r). Given the expression of mass absorption coefficient κ l : κl = Qaπ r 2 3Qa = 3 ρ l (4 / 3)π r 4 ρl r (39) where r is the radius of the droplet, Qa is the absorption efficiency, which is expressed as follows so that κ l for a single droplet relies on the droplet size (Petty, 2006): ⎧ m2 − 1 x 2 m 2 − 1 m 4 + 27m 2 + 38 ⎫ 8 4 ⎧ m 2 − 1 2 ⎫ Qa = 4xℑ ⎨ 2 [1+ ( 2 ) ] ⎬ + x ℜ ⎨( 2 ) ⎬ 15 m + 2 2m 2 + 3 ⎩m + 2 ⎭ 3 ⎩ m +2 ⎭ with the relative refractivity m = water droplet, respectively; x = (40) N2 , where N1 and N 2 the refractivity indices for air and liquid N1 2π r is the ratio of droplet perimeter over instrument channel λ wavelength λ . As in Equations (39)-(40), the mass absorption coefficient varies with droplet size so that Equation (38) does not allow accurate estimation of LWP. However, for sufficiently small droplet ( x ≪ 1 ), Qa is proportional to r: Qa = 8π r ⎧ m 2 − 1 ⎫ ℑ⎨ ⎬ λ ⎩ m2 + 2 ⎭ (41) Equations (39) and (41) thus yield a mass absorption coefficient κ l independent of the droplet size, when the droplet radius is sufficiently small. This simplifies the calculation of water droplet optical depth to Equation (38). Equations (34)-(41) show both TPW and LWP are in linear relationships with logarithm of brightness temperatures observed. For the two unknown variables, mathematically only two equations, or observations at any two channels suffice to obtain solutions. However, more desired 30 observations are from such channels with significantly different focuses on TPW and LWP separately, otherwise error matrix of the two variables might grow dramatically (Petty, 2006). The center frequency of AMSU channel 1 is at the water vapor absorption line and is dominant in retrieving the water vapor content; channel 2 is at a frequency far from absorption line and not sensitive to water vapor, but is heavily affected by liquid water. Those two channels are thus an ideal pair to calculate for LWP and TPW with Equations (32)-(33). With the same center frequencies, ATMS window channels 1 and 2, which are analogous to AMSU-A channels 1 and 2, respectively, can be employed to retrieve LWP with similar algorithm. Details of LWP retrieval algorithm with ATMS window channel observations are found in Weng et al. (2012). 4.2.4 Impact of Striping Noise on ATMS LWP Retrievals Noise removed by optimal filters at channels 1 and 2 within 30S-30N on February 24, 2014 is shown in Fig. 4.22. Visible striping patterns with magnitudes of no more than 1 K are demonstrated for both channels. Global distributions of LWP retrieved using ATMS window channels 1 and 2 with and without striping noise mitigation are displayed in Figs. 4.23-24. Other variables involved in the retrieving are surface temperature, surface emissivity and instrument zenith angle. Weather systems captured in both LWP retrievals (Figs. 4.23a-b, 4.24a-b) have similar features, suggesting that the impact of the striping noise mitigation on LWP retrieval is rather small. However, differences of LWP before and after striping noise mitigation in channels 1 and 2 (Figs. 4.23c) reveal some striping patterns over places where there exist clouds (Fig. 4.23a,) and striping noise in channels 1 and 2 (Figs. 4.23d-e). Moreover, the LWP differences could be more than 0.05 kg/m2, which is greater than the estimated error of 0.01 kg/m2 in the LWP retrieval. These results confirm that the striping noise mitigation does not affect small-scale 31 features in the LWP fields and yet it is important to remove the striping noise in ATMS brightness temperatures. It is noted that even the LWP differences exhibit striping pattern. Furthermore, the magnitude of the LWP differences varies with FOVs along the same scan line. This suggests that the LWP differences are not simply a linear combination of the striping noise at channels 1 and 2 Figure 4.25 shows the probability density functions (PDFs) of LWP differences, which are color-coded by magnitudes of LWP retrieval or striping noise at the two ATMS window channels. All PDFs are in Gaussian distributions, with the majority of LWP differences within the range of ±0.02kg/m2. As in (a), the PDF distribution of LWP difference is symmetrical within each LWP magnitude interval, which indicates that the positive and negative LWP differences occur equally frequently. However, as in (b) and (c), the PDFs are not symmetrical within each noise interval at channels 1 and 2. It is seen for both window channels the positive striping noise is associated with frequent occurrences of positive LWP differences, and vice versa. The result in Fig. 12 implies that amplitude of striping-induced LWP differences is related to the striping noise in brightness temperatures at window channels. To further investigate the relationships between the removed striping noise within window channel brightness temperatures and the LWP differences, and to expel the possibility that striping-induced LWP noise is dependent on LWP retrieval amplitudes, scattering plots of LWP differences with respect to window channel striping noise or LWP retrieval magnitudes are provided in Fig. 4.26. Scattering plots are color-coded with LWP retrievals in (a) and (b), and it is seen no mater viewed as a whole or at each LWP interval, the larger the striping noise, the larger the LWP differences, and the relationships are linear. With larger LWP retrievals, the linear relationships have steeper slopes. Scattering plots of LWP differences with respect to magnitudes 32 of LWP retrievals and color-coded with channel 1 and 2 noise are shown in (c) and (d). It is seen the distribution of the scattering points is random, and the relationships between LWP noise and LWP retrieval magnitudes vary a lot among different striping noise intervals. This randomness indicates that the LWP noise is independent of the LWP retrieval magnitudes. However, for a fixed value at channel 1 or 2, it is seen that if the LWP retrieval is larger, the LWP striping is stronger. This agrees with the fact that LWP differences are not identical along the same scan line. The impact of striping noise of brightness temperatures observed at ATMS window channels on LWP retrievals addresses the necessity of removing the striping noise contaminated in ATMS brightness temperatures, and this impact also reveals the benefits the striping noise mitigation brings to NWP. 33 Table 4.1: Numbers of IMFs removed and filter spans of optimal striping filters for ATMS scene counts at each channel. Central Channel Number of Filter span Peak WF Frequency Number IMFs removed (N) (hPa) (GHz) 1 23.8 2 14 Surface 2 31.4 2 14 Surface 3 50.3 3 23 Surface 4 51.76 3 23 950 5 52.8 3 18 850 6 53.596±0.115 3 17 700 7 54.4 3 19 400 8 54.94 3 17 250 9 55.5 3 17 200 10 57.2903 3 16 100 11 57.2903±0.115 3 18 50 12 57.2903 3 18 25 13 57.2903±0.322 3 18 10 14 57.2903±0.322±0.010 3 20 5 15 57.2903±0.322±0.004 3 17 2 16 88.20 2 16 Surface 17 165.5 3 23 Surface 18 183.31±7 3 22 800 19 183.31±4.5 3 22 700 20 183.31±3 3 22 500 21 183.31±1.8 3 22 400 22 183.31±1.0 3 23 300 34 Table 4.2: Filter spans of optimal striping filters for ATMS warm counts and cold counts. Central Warm Count Cold Count Frequency Filter span Filter span (GHz) (N) (N) 1 23.8 8 8 Surface 2 31.4 8 8 Surface 3 50.3 10 10 Surface 4 51.76 10 10 950 5 52.8 8 10 850 6 53.596±0.115 8 10 700 7 54.4 8 10 400 8 54.94 8 10 250 9 55.5 10 10 200 10 57.2903 8 10 100 11 57.2903±0.115 10 10 50 12 57.2903 10 10 25 13 57.2903±0.322 10 10 10 14 57.2903±0.322±0.010 10 10 5 15 57.2903±0.322±0.004 10 10 2 16 88.20 8 8 Surface 17 165.5 8 8 Surface 18 183.31±7 8 8 800 19 183.31±4.5 8 8 700 20 183.31±3 8 8 500 21 183.31±1.8 8 8 400 22 183.31±1.0 8 8 300 Channel Number 35 Peak WF (hPa) Table 4.3: Numbers of IMFs removed and the chosen filter spans for ATMS channels. Channel Number Central Frequency (GHz) Number of Filter span Peak WF IMFs removed (N) (hPa) 1 23.8 2 14 Surface 2 31.4 2 14 Surface 3 50.3 3 23 Surface 4 51.76 3 22 950 5 52.8 3 18 850 6 53.596±0.115 3 17 700 7 54.4 3 19 400 8 54.94 3 17 250 9 55.5 3 17 200 10 57.2903 3 16 100 11 57.2903±0.115 3 18 50 12 57.2903 3 18 25 13 57.2903±0.322 3 18 10 14 57.2903±0.322±0.010 3 20 5 15 57.2903±0.322±0.004 3 17 2 16 88.20 2 16 Surface 17 165.5 3 22 Surface 18 183.31±7 3 22 800 19 183.31±4.5 3 22 700 20 183.31±3 3 22 500 21 183.31±1.8 3 22 400 22 183.31±1.0 3 23 300 36 Amplitude (a) Frequency (s-1) Amplitude (b) Frequency (s-1) Fig. 4.1: Fourier spectra with 81-point running mean for the first six IMFs of (a) warm counts and (b) cold counts at channel 8. 37 Amplitude Frequency (s-1) Fig. 4.2: Same as Fig. 4.1 but for warm load temperature. Five IMFs are removed. 38 Amplitude Frequency (s-1) Fig. 4.3: Same as Fig. 4.1 but for the first PC coefficient of scene counts at channel 8. Three IMFs are removed. 39 Channel Warm counts N Channel Cold counts N Channel Scene counts N Fig. 4.4: Normalized cost function with respect to filter span N for warm counts (upper), cold counts (middle) and scene counts (lower). Numbers of scan lines involved in the optimal filter is indicated in circles. 40 Weighting Coefficient Warm counts n Channel Weighting coefficient Cold counts n Channel Weighting coefficient Scene counts n Channel Fig. 4.5: Weighting coefficients of optimal filters for warm counts (upper), cold counts (middle) and scene counts (lower). 41 Channel Warm counts Frequency (s-1) Channel Cold counts Frequency (s-1) Channel Scene counts Frequency (s-1) Fig. 4.6: Response function for warm counts (upper), cold counts (middle) and scene counts (lower) calculated with weighting coefficients as in Fig. 5. Contours over dark blue shading area all have the value of 0.01 because of oscillations of response functions. 42 Cost function (a) N Weighting coefficient (b) n Fig. 4.7: (a) Normalized cost function with respect to filter span N for warm load temperature. Numbers of scan lines involved in the optimal filter is indicated in circle. (b)-(c) weighting coefficients and response function for warm load temperature. 43 Response function (c) Fig. 4.7 Continued Frequency (s-1) 44 Warm Counts (a) Scan line Cold Counts (b) Scan line Fig. 4.8: Variations of (a) warm counts and (b) cold counts along 700 scanlines (i.e., a swath between ±55N ) before (gray) and after (red) applying removing three IMFs at channel 8. 45 Warm Counts (a) Scan line Cold Counts (b) Scan line Fig. 4.9: Variations of (a) warm counts and (b) cold counts along 700 scanlines (i.e., a swath between ±55N ) before (gray) and after (red) applying removing three IMFs at channel 22. 46 Warm Counts (a) Scan line Cold Counts (b) Scan line Fig. 4.10: Variations of (a) warm counts and (b) cold counts with optimal filter (red) and 17-point smoothing (blue) for channel 8. 47 Response function Frequency (s-1) Fig. 4.11: Variation of response functions with respect to frequency for the 17-point filter (blue) and the optimal filters (red) applied to warm counts of channel 8 (curve) and channel 22 (circle). The response functions for cold counts are the same as those for warm counts (figure omitted). 48 Brightness temperature (K) Scan line Fig. 4.12: Variation of brightness temperature for channel 8 at nadir with (red) and without (blue) applying the optimal filter on the scene counts. Warm count, cold counts and warm load temperatures are all smoothed. 49 Channel 8 Channel 22 Channel 1 Fig. 4.13: Striping noise removed by the optimal symmetric filters for ATMS channels 8, 22 and 1 on ascending node. 50 Cost Function (a) Cost Function N (b) Cost Function N (c) N Fig. 4.14: (a)-(d) Variation of the normalized cost function with respect to filter span N for ATMS channels 1-22. The cost function value with the selected symmetrical filter span (2N+1) is indicated by circle. (e) The cost function for N=2 used for normalization in (a)-(d) for each channel. 51 Cost Function (d) Cost function (K2) N (e) Channel Number Fig. 4.14 Continued 52 Weighting Coefficients Weighting Coefficients n Weighting Coefficients n n Fig. 4.15: Weighting coefficients of symmetric filters for ATMS channels 1-22. 53 Weighting Coefficients Fig. 4.15 Continued Number of Scan line 54 Response coefficients Response coefficients Frequency (s-1) Response coefficients Frequency (s-1) Frequency (s-1) Fig. 4.16: Response functions of symmetric filters calculated with weighting coefficients in Fig. 4.15. 55 Response coefficients Frequency (s-1) Fig. 4.16 Continued 56 (b) Amplitude Amplitude (a) Frequency (s-1) Frequency (s-1) Fig. 4.17: The power spectral density distributions of the global O-B field of ATMS channels (a) 9 and (b) 10 before (red) and after (blue) removing the striping noise using the optimal filter. 57 (a) (b) (c) Fig. 4.18: Global distribution of O-B fields (a) before and (b) after applying the optimal filter for ATMS channel 8 on February 24, 2012. (c) Global distribution of striping noise, i.e., the difference between (a) and (b). 58 (a) (b) (c) Fig. 4.19: Striping noise contained in the observaions for ATMS channel (a) 1, (b) 8 and (c) 22 on February 24, 2012. 59 iance Scan line (b) iance Scan line (a) FOV FOV iance Scan line (d) iance Scan line (c) FOV FOV Fig. 4.20: Maneuver data (a) before and (b) after optimal smoothing on PCA1 for ATMS channel 1. (c)-(d) Same as (a)-(b) but for channel 9. 60 VDT (a) Channel VCT (b) Channel SI (c) Channel Fig. 4.21: (a) The along-track variance, (b) across-track variance and (c) striping index before (red) and after (blue) smoothing. FOVs 25-72 on 100 scan lines are involved in calculation. 61 Ch1, 0000-1200 UTC Ch2, 0000-1200 UTC Ch1, 1200-2400 UTC Fig. 4.22: Global distributions of striping noise removed by the optimal filter for ATMS channels 1 and 2 on February 24, 2012. 62 Ch2, 1200-2400 UTC Fig. 4.22 Continued 63 (a) (b) (c) Fig. 4.23: Global distribution of LWP retrieved by ATMS channels 1-2 observation (a) without and (b) with smoothing and (c) the LWP differences between (a) and (b) during 00001200 UTC on February 24, 2012. 64 (a) (b) (c) Fig. 4.24: Same as Fig. 4.23 but for 1200-2400UTC. 65 PDF (%) (a) ×kg/m2 LWP Difference (kg/m2) (c) PDF (%) PDF (%) (b) LWP Difference (kg/m2) LWP Difference (kg/m2) Fig. 4.25: Probability density functions of LWP difference within 30S-30N over ocean on February 24, 2012. (a) LWP retrieved by observations, (b) noise removed at ATMS channel 1 and (c) noise removed at ATMS channel 2 are color coded, respectively. The mean and std of the LWP differences with and without smoothing (i.e., LWPwithoutLWPwith) within 30S-30N over ocean on February 24, 2012 are zero and 0.005 kg m-2. 66 (b) (kg/m2) Difference (kg/m2) Difference (kg/m2) (a) (kg/m2) Ch1 striping noise (K) Ch2 striping noise (K) (c) Difference (kg/m2) Difference (kg/m2) (d) (K, ch1) (K, ch2) LWP (kg/m2) LWP (kg/m2) Fig. 4.26: Scatter plots of the LWP differences with respect to (a)-(b) striping noise of channels 1 and 2 and (c)-(d) LWP retrieved with observations without smoothing. The magnitude of LWP is indicated in (a) and (b) and the magnitude of striping noise of channels 1 and 2 is indicated in (c) and (d), respectively. Data within 30S-30N over ocean on February 24, 2012 are used. 67 CHAPTER FIVE STRIPING NOISE MITIGATION FOR FY-3C MWTS DATA 5.1 Characterzation and Mitigation of Striping Noise in FY-3C MWTS Data 5.1.1 Fengyun-3C Microwave Temperature Sounder (MWTS) Data Fengyun-3C (FY-3C) was successfully launched on 23 September 2013, which is the third one in the new series of Chinese FY polar-orbiting satellites. The FY-3 mission aims at enhancing numerical weather prediction and climate research with observation data, monitoring large-scale natural disasters, and providing meteorological information for aviation and navigation. The first and second satellites of this series are FY-3A and FY-3B, which were launched into orbits on May 27, 2008 and November 5, 2010, respectively. The local equator crossing times of FY-3A, 3B and -3C are 10 am, 2pm and 10pm, respectively. Microwave Temperature Sounder (MWTS) is one of the 11 sensors on board FY-3C. MWTS contains 13 channels whose central frequencies range from 50.3 GHz to 57.6 GHz, which correspond to ATMS channels 3-15, central frequencies and peak weighting function pressures of MWTS channels can be found in Table 5.1. As a cross-track microwave radiometer, MWTS has 90 FOVs on each scan line, and the swath width of MWTS is 2600 km. The scan profile of FY-3C MWTS was changed during May 12-18, 2014. Before this scan profile change, each scan cycle takes 2.67 seconds. The antenna of MWTS scans at cold space, warm target and earth scene with the same integration time of 0.018 second, and accelerates during the time in between. After the profile change, MWTS keeps a constant speed during each entire scan cycle of 5.23 seconds. MWTS data during March 5-11, 2014 and May 29-June 4, 2014 are used in this study. There are 14-15 swaths in a single day. Each swath contains around 2400 and 1200 scan lines before and after scan profile change, respectively. 68 5.1.2 Diagnosis of Striping Noise in FY-3C MWTS Data The spectra for the first six IMFs of the first four PC coefficients at channel 8 for May 1, 2014 are shown in Fig. 5.1. Peak magnitudes of the first four IMFs are at the frequencies greater than 10-2 Hz. The amplitude of individual IMF decreases with PC coefficient sequence, i.e., all IMFs have the largest amplitudes for the first PC coefficient and smallest for the fourth PC coefficient. This is in accordance with the fact that the first PC component explains the majority of variances. For the first three PC coefficients, peak magnitudes have an obvious increase starting from the fifth IMF, which have reached around 10 times of the first four IMFs. For the fourth PC coefficient, there is no clear difference of magnitudes between IMFs. It is thus decided that the first four IMFs of the first three PC coefficients are to be removed, which is denoted as PC3/IMF4 for breivity. The spectra for the first six IMFs of the first four PC coefficients at FY-3C MWTS channel 8 for May 29, 2014 after the scan profile change are shown in Fig. 5.2. The high-frequency end for May 29 is different from that on May 1 since the scan cycles before and after the scan profile change are different. The spectra for data on May 29 are similar to those of May 1, 2014 in terms of magnitudes of individual IMFs. The fourth IMFs of all the first three PC coefficients at lowfrequency range have much larger amplitudes than the first three IMFs and are thus retained. It is thus determined that the first three IMFs of the first three PC coefficients are to be removed for MWTS channel 8 after the scan profile change of MWTS. By comparison, the spectra for the first six IMFs of the first four PC coefficients at corresponding ATMS channel 10 during May 1, 2014 are shown in Fig. 5.3. It is shown that despite comparable magnitudes of the 5th and 6th IMFs to that of MWTS, magnitudes of the first four IMFs of ATMS are smaller. This fact implies that the striping noise, which is described by 69 the first few IMFs, is much stronger within MWTS signals than ATMS. It is also shown that unlike MWTS, where the amplitudes of the identified stripinh noise remains above 0.01 for the first three PC coefficients, amplitudes of the first few IMFs for the second and third PC coefficients of ATMS are at least five times smaller. This confirms that only the first PC coefficient is smoothed for ATMS. The distribution of the first five PC modes (i.e., eigenvectors) for the 47th data sample (k=47, each sample includes 200 scan lines) on May 1, 2014 is displayed in Fig. 5.4. Note that the eigenvectors are functions of FOV. All PC modes oscillate around zero, and their values are of close scales. It is shown that compared with other PC modes, the first PC mode changes little along with FOV. This is reasonable since the first PC mode represent the major cross-track features of MWTS while other PC modes capture weather signals. Numbers of extremes for the second, third, fourth and fifth PC modes are two, three, four and five, respectively. Figure 5.5 shows spatial distributions of the first, second and third PC components, i.e., the multiplications of PC coefficients and the corresponding PC modes. The magnitude of the first PC component is dominating, which is comparable in magnitude to the brightness temperature. The second and third PC components have nearly same magnitudes. The PC components before and after smoothing are calculated using the PC coefficients with and without removing the first four IMFs, respectively. It is seen that before the noise mitigation, visible striping noise exists in all the three PC components, and after smoothing those striping noise are removed while neither along-track nor cross-track features of large scale are changed. Spatial distributions of observed and simulated brightness temperatures are displayed in Fig. 5.6 (a)-(b), and striping noise can be easily identified from the observed brightness temperature fields. Figures. 5.6 (c) and (d) show the spatial distributions after striping noise mitigation of 70 PC3/IMF4 and PC1/IMF4, respectively. It is seen in Fig. 5.6 (d) that the striping noise is reduced after removing the first four IMFs in the first PC coefficient but still exist. By comparison, no such residual striping noise is found in the distribution of PC3/IMF4 smoothed brightness temperatures. Spatial distributions of O-B before and after noise mitigation of PC3/IMF4, as well as after noise mitigation of PC1/IMF4 are shown in Fig. 5.6 (e), (f) and (g), respectively. It is seen that the striping noise pattern is clearly seen in the distributions of O-B. The striping noise is effectively removed after PC3/IMF4 smoothing while some residual striping noise pattern still remains after PC1/IMF4 smoothing. This indicates the necessity to remove the first four IMFs of the second and the third PC coefficients. Spatial distributions of the striping noise in FY-3C MWTS brightness temperatures extracted by PC3/IMF4 and PC1/IMF4 are displayed in Fig. 5.7 (a) and (b), respectively. The striping noise is as large as ±1K. It is shown that the line patterns for PC1/IMF4 smoothing of detected striping noise cover all 90 FOVs, while the striping noise pattern for PC3/IMF4 smoothing includes not all but some of adjacent FOVs on each scan line. The differences of line patterns between PC1/IMF4 smoothing and PC3/IMF4 smoothing is consistent with the spatial patterms of the second and third PC modes, which are not constant with respect to FOV. The difference between the noise identified by PC3/IMF4 and PC1/IMF4, i.e., (a) minus (b), is shown in (c). It is seen that magnitude of this difference is of the same order as the magnitude of either recognized noise. This further implies that it is necessary to remove the first four IMFs from the first PC coefficient, but also from the second and the third PC coefficient. Global distributions of differences between observations and simulations before and after striping noise mitigation of PC3/IMF4 during May 1 and PC3/IMF3 during May 29 are displayed in Figs. 5.8 and 5.9, respectively. It is seen that after smoothing, the visible striping noise is 71 effectively removed while other weather-related features are not altered. The noise removed is shown in (c). It is seen that the magnitude of striping noise is on the order of ±1K for FY-3C MWTS before and after the scan profile change.. 5.1.3 Striping Index Calculated for FY-3C MWTS Global Data In order to quantitatively validate the effectiveness of the striping noise mitigation on FY3C MWTS, a striping index for brightness temperatures is defined as follows. The Striping Index (SI) for a single day at a given channel is calculated as the following equation: K 1 N 2 ∑ N ∑ σ along (k,i) i=1 SI = k=1 K 1 M 2 ∑ M ∑ σ cross (k, j) k=1 j=1 (42) 2 2 where σ along (k,i) and σ cross (k, j) are along-track and across-track variances, respectively, and are calculated as: σ 2 along 1 (k,i) = M 1 (O − B)k,i = M j 2 σ cross (k, j) = i (O − B)k, j ∑ ((O − B) M j k,i, j − (O − B)k,i j=1 ) 2 M ∑ (O − B) (43) k,i, j j=1 ( i 1 N (O − B)k,i, j − (O − B)k, j ∑ N i=1 1 N = ∑ (O − B)k,i, j N i=1 ) 2 (44) where (O − B)k,i, j stands for O-B difference at the ith FOV on the jth scan line within the kth data set involving a total of 200 and 100 scan lines before and after scan profile change, respectively; N is the total number of FOVs, M is the number of scan lines within each of the data set for variance calculation, and K is the total number of datasets. In the calculation, N=90, M=200 and K is 72 smooth around 160. For SIs after noise mitigation, smoothed brightness temperatures O are used instead of observations O . 2 2 Variations of along-track variances σ along (k,i) and cross-track variances σ cross (k, j) are shown in Fig. 5.10. Both along-track and cross-track variances are calculated using brightness temperatures before striping noise mitigation on May 1, 2014. The distribution of along-track variance has line patterns for adjacent FOVs within some datasets, which is related to cross-track correlations. By comparison, distribution of cross-track variance is random. Shown in obviously brighter colors, the along-track variances generally have larger values than cross-track variances within the same dataset. As striping noise adds to along-track variances, the fact that along-track variances are greater than cross-track variances for all individual dataset confirms the existence of striping noise. The striping index calculated as Equation (42) is 1.3513. Variations of along-track 2 2 (k,i) and cross-track variances σ cross (k, j) after noise mitigations are shown in variances σ along Fig. 5.11. It is seen that the magnitudes of along-track variances are significantly reduced while those of cross-track variances changed very little, and the SI dropped to 0.975. Similar distributions of individual along-track and cross-track variances for May 29 are shown in Figs. 5.12-13. The SI dropped from 1.51 to 1.01. Variations of SIs of MWTS channel 8 with respect to date during two weeks are shown in Fig. 5.13. Along-track variances increased after the scan profile change while cross-track variances remained the same, which indicates that the FY-3C MWTS striping problems are even severer after scan profile change. It is seen that with striping noise mitigations along-track variations almost dropped to half. Cross-track variations also decreased, which is related to the smoothing within the second and third PC coefficients. SI index are all around unit values after noise mitigation, indicating that the noise mitigation is effective. 73 5.2 Investigation on the Root-Cause of Striping Noise 5.2.1 Mathematical Expressions of Transfer Function As shown above, striping noise is present in ATMS temperature channels and humidity channels, previous humidity sounders AMSU-B and MHS, and other temperature sounders such as FY-3C MWTS. However, striping noise is not found in AMSU-A observations. A major difference in instrumental structures between AMSU-A and the other microwave sensors is that AMSU-A does not have a RF amplifier in front of the receiver as the other sensors do. These amplifiers could induce flicker noise through receiver gain fluctuation, whose power spectral density (PSD) is in a form of 1/f, where f is frequency. Transformation from this flicker noise to striping noise can be described as follows. In the time domain, the output noise gn (t) is the convolution of the “real” sensor noise g(t) and a so-called “forcing” transfer function h(t) , i.e., ∞ gn (t) = g(t)* h(t) = ∫ g(t ')h(t − t ')dt ' (45) −∞ The Fourier transformations of g(t) , h(t) and gn (t) can be represented by G( f ) , H ( f ) and Gn ( f ) as follows: ∞ G( f ) = ∫ g(t)ei 2π ft dt (46a) −∞ ∞ H ( f ) = ∫ h(t)ei 2 π ft dt (46b) −∞ ∞ Gn ( f ) = ∫ gn (t)ei 2 π ft dt −∞ ∞ ∞ −∞ −∞ ∞ ∞ −∞ −∞ = ∫ dt ∫ dt ' gn (t ')h(t − t ')ei 2 π ft = ∫ dt ∫ dt ' gn (t ')e = G( f )H ( f ) 74 i 2 π ft ' h(t − t ')e (46c) i 2 π f (t−t ') Equation 46c shows that in the frequency domain, the output noise signal is simply a multiplication of sensor noise and transfer function. According to Hersman and Poe (year?), the transfer function h(t) can be expressed as: h(t) = hs (t) − ∑ w(t − kt c )hc (kt c ) (47a) τs ⎧1 ⎪⎪ τ , t ≤ 2 hs (t) = ⎨ s ⎪0, t > τ s ⎪⎩ 2 (47b) k τc ⎧1 ⎪⎪ τ , t ≤ 2 hc (t) = ⎨ c ⎪0, t > τ c ⎪⎩ 2 where hs (t) denotes the part related to scene measurements, and (47c) ∑ w(t − kt )h (kt ) is the noisc c c k calibrated measurements hc (kt c ) that is convolved by the weighting function w(t − kt c ) . Substituting (47) into (45) yields: ∞ ⎡ ⎤ gn (t) = ∫ dt ' g(t ') ⎢ hs (t − t ') − ∑ w(t − kt c )hc (kt c − t ') ⎥ −∞ ⎣ ⎦ k (48) The power spectral density (PSD, Sn ( f ) ) of the output noise gn (t) is the autocorrelation of the noise itself, which can be written as: ∞ ∞ ⎡ ⎤ Sn ( f ) = ∫ dt '∫ dt "[ g(t ')g(t ")] ⎢ hs (t − t ') − ∑ w(t − kt c )hc (kt c − t ') ⎥ −∞ −∞ ⎣ ⎦ k (49) ⎡ ⎤ ⎢ hs (t − t ") − ∑ w(t − kt c )hc (kt c − t ") ⎥ ⎣ ⎦ k where the expectation of the term [g(t ')g(t ")] is the PSD ( S( f ) ) of instrumental noise g(t) , i.e., 75 E { g(t ')g(t ")} = S( f ) (50a) The scene integration term (Hs(f)) as well as calibration integration term (Hc(f)) can be simplified through a series of mathematical manipulation as shown below: ∞ H s ( f ) = ∫ hs (t)ei 2 π ft dt −∞ τs = ∫ τs − = 1 τs 1 i 2 π ft e dt 2 τs 2 ∫ τs 2 τ − s2 cos(2π ft) + i sin(2π ft)dt (50b) τs 1 sin(2π ft) − i cos(2π ft)] τ2s [ − 2 τ s (2π f ) sin(π f τ s ) = τ sπ f = hc (kt c − t ') = hc [ −(t − kt c ) + (t − t ')] (50c) H c ( f ) = ∫ hc [ −(t − kt c ) + t '] ei 2 π ft 'dt ' ∞ −∞ ∞ = ∫ hc (t ')ei 2 π f (t '−t+ktc )dt ' −∞ =e −i 2 π f (t−kt c ) = e−i 2 π f (t−ktc ) ∫ τc 2 τ − c2 (50d) 1 i 2 π ft ' e dt ' τc sin(π f τ c ) τ cπ f Combining 50a-d yields: ⎛ ⎞ H ( f ) = ⎜ H s ( f ) − H c ( f )∑ w(t − kt c )⎟ ⎝ ⎠ k 2 2 sin π f τ s sin π f τ c = − w(t − kt c )e−i 2 π f (t−ktc ) ∑ π fτ s π fτc k 2 (51) and Sn( f ) = S( f )H 2 ( f ) (52) It is seen from Equation (51) that transfer function varies with respect to the instrumental 76 parameters of scan cycle ( t c ), calibration integration time ( τ c ) and scene integration time ( τ s ), etc. It also depends on calibration weighting functions w(t) . In order to quantify the impact of transfer function on the output noise magnitude Sn ( f ) , transfer functions for AMSU-A, ATMS and FY-3C MWTS before and after profile changes are firstly calculated, and then convoluted with PSD of simulated noise S( f ) . 5.2.2 PSD of the Output Noise of Transfer Function Parameters of AMSU-A, ATMS and FY-3C MWTS before and after profile changes that are used as input to transfer function are shown in Table 5.2. It is seen that both the scan cycle and the integration time of AMSU-A are triple and 7 times longer than those of ATMS. As previously noted, there is no flicker noise in the simulated signals for AMSU-A. FY-3C MWTS experienced a scan profile change during May 12-18, 2014. Before this profile change, antennas of MWTS and other sensors slow down to take scans of cold space, warm load and earth scenes, respectively, and accelerate between the intervals of taking the scans. After the profile change, antenna of MWTS starts to rotate at a constant speed. MWTS’s integration time can be calculated as follows: 2θ m τ s = 360 n t 1 r = c arcsin( sin ϕ m ) n 180 r+h tc (53) where n is number of FOVs, which is 90 for MWTS; θ m and ϕ m are outmost scan angle and zenith angle, respectively; r is the radius of earth, and h is the altitude of FY-3C. It is seen that even when the scan cycle is doubled, calibration integration time and scene integration time of MWTS decrease after scan profile change. ATMS has similar values for instrumental parameters as the MWTS before scan profile change. A uniform weighting of either two points or five points 77 can be employed for calibration weighting functions w(t) , i.e., 1 w(t) = ,t = ±t c 2 (54a) 1 w(t) = ,t = 0, ±t c , ±2t c 5 (54b) and Figures 5.15 and 5.16 show the transfer functions calculated using instrumental parameters and weighting functions described above. It is seen that all transfer functions increase sharply at low frequencies, then oscillate with the peak magnitude unchanged, and drop at higher frequencies. The frequency ranges where transfer functions remain their maximum magnitudes are around the scan frequency ( 1 1 ) and integration frequency ( ). Having a longer scan cycle tc τc and a longer integration time than other sensors, AMSU-A has its transfer function peaking at lower frequencies. Transfer functions with five points uniformly weighted have slightly wider peaking ranges and weaker oscillations than those with two points averaging. PSDs of the noise outputs Sn ( f ) for AMSU-A, ATMS and MWTS before and after profile change are calculated according to Equation (52), which are shown in Fig. 5.18 It is seen that for all three sensors with white noise only, there exist signals at the striping noise frequencies although the striping signals are not very pronounced. By comparison, with flicker noise added in the instrument noise, the striping noise is magnified. It is also shown that even with the same instrument input signals, the output noise PSD at the striping frequencies are different. Especially, with a longer scan cycle as MWTS after scan profile change, the transfer function peak shifts toward the lower frequency range, which favors the presence of the flicker noise. As a result, the striping noise contained in MWTS observations become more visible after the change of the scan 78 profile. The PSD with and without adding flicker noise is shown in red and green, respectively, and the differences between the two are shown in Fig. 5.19. It is seen that PSD of the output from the white noise only maintain the shape of transfer function, while PSD of output for combined noise is amplified at frequencies that are near and lower than the “knee”. The contribution of flicker noise to the output PSD, which is seen in Fig. 5.19, resembles the PSD of striping noise. This PSD component for ATMS and MWTS before scan profile change is similar, and is much larger for MWTS after scan profile change. Figure 5.19 confirms the hypothesis that flicker noise is a major cause of the striping noise, and the transfer functions determined by instrumental parameters can modify the magnitude and frequency of the striping noise. 79 Table 5.1: Central frequencies of FY-3C MWTS and corresponding ATMS channels. Channel Central Frequency Peak WF Number (GHz) MWTS ATMS (hPa) MWTS ATMS 1 3 50.3 Surface 2 4 51.76 950 3 5 52.8 850 4 6 53.596±0.115 700 5 7 54.4 400 6 8 54.94 250 7 9 55.5 200 8 10 57.2903 100 9 11 57.2903±0.115 50 10 12 57.2903 25 11 13 57.2903±0.322 10 12 14 57.2903±0.322±0.010 5 13 15 57.2903±0.322±0.004 2 80 MWTS ATMS Table 5.2: Scan cycle ( t s ), calibration integration time ( τ c ) and scene integration time ( τ s ) that are used as input for calculating the transfer functions for AMSU-A, ATMS and FY-3C MWTS before and after profile changes. ts τc τs (unit: s) (unit: s) (unit: s) AMSU 8 0.158 0.158 No ATMS 2.67 0.018 0.018 Yes MWTS 2.67 0.018 0.018 Yes MWTS* 5.23 0.016 0.016 Yes Sensor Flicker noise Table 5.3: Numbers of FOV and NEDT at AMSU-A, ATMS, MWTS channels with central frequencies of 57.290344 GHz. Sensor Channel Scan cycle (s) Numbers of FOVs NEDT (K) AMSU 9 8 30 0.25 ATMS 10 2.67 96 0.75 MWTS 8 2.67 90 0.75 MWTS* 8 5.23 90 0.75 81 Amplitude (b) Amplitude (a) (d) Frequency (s-1) Amplitude Amplitude (c) Frequency (s-1) Frequency (s-1) Frequency (s-1) Fig. 5.1: Fourier spectra for the first four IMFs of the (a) first, (b) second, (c) third and (d) fourth PC coefficients at channel 8 for May 1, 2014. 82 Amplitude (b) Amplitude (a) Frequency (s-1) Frequency (s-1) (d) Amplitude Amplitude (c) Frequency (s-1) Frequency (s-1) Fig. 5.2: Fourier spectra for the first six IMFs of the (a) first, (b) second, (c) third and (d) fourth PC coefficients at channel 8 for May 29, 2014. 83 Amplitude (b) Amplitude (a) Frequency (s-1) Frequency (s-1) (d) Amplitude Amplitude (c) Frequency (s-1) Frequency (s-1) Fig. 5.3: Fourier spectra for the first six IMFs of the (a) first, (b) second, (c) third and (d) fourth PC coefficients at ATMS channel 10 for May 1, 2014. 84 PC component FOV Fig. 5.4: Spatial distributions of the first five PC modes (i.e., eigenvectors) for the 47th data sample (k=47, each sample includes 200 scan lines) on May 1, 2014. 85 k=47, 1st PC Smoothed 1st PC (4 IMFs) Smoothed PCA2 (4 IMFs) k=47, PCA2 Fig. 5.5: Spatial distributions of the individual PCA component for the first three PCA components (i.e., the product of eigenvalue and eigenvector) before (left panels) and after (right panels) striping noise mitigation for the 47th data sample (k=47, each sample includes 200 scan lines) on May 1, 2014. 86 k=47, PCA3 Smoothed PCA3 (4 IMFs) Fig. 5.5 Continued 87 (b) (a) (c) (d) Fig. 5.6: Spatial distributions of ATMS channel 8 (a) observed, (b) simulated and (c) smoothed 3PC/4IMF and (d) 1PC/4IMF brightness temperatures for the 47th data sample (k=47, each sample includes 200 scan lines) on May 1, 2014. O-B differences (e) before and after (f) 3PC/4IMF and (g) 1PC/4IMF noise mitigation. 88 (f) (e) (g) Fig. 5.6 Continued 89 (b) (a) (c) Fig. 5.7: Noise removed by (a) 3PC/4IMF and (b) 1PC/4IMF as in Fig. 5.6. (c) Differences between (a) and (b). 90 (a) (b) (c) Fig. 5.8: Global distributions of channel 8 (a) O-B and (b) difference between smoothed brightness temperatures removed on May 1, 2014 on ascending node. The first four IMFs of the first three PC coefficients are removed for noise mitigation. (c) The striping noise identified with the EEMD/PCA approach. 91 (a) (b) (c) Fig. 5.9: Global distributions of channel 8 (a) O-B and (b) difference between smoothed brightness temperatures removed on May 30, 2014 on ascending node. The first three IMFs of the first three PC coefficients are removed for noise mitigation. (c) The striping noise identified by the EEMD/PCA approach. 92 FOV (i) (a) Dataset Number (k) Scan line (j) (b) Dataset number (k) Fig. 5.10: Variations of (a) along-track variances as a function of the sample group number and FOV (i) and (b) cross-track variances as a function of dataset number (k) and scan line (j) before striping noise mitigation on May 1, 2014. The total number of scan lines in each dataset is 200. The total number of datasets for both the ascending and descending nodes on May 1 is 162. (c) Variation of striping index as a function of dataset number (k). 93 FOV (i) (a) Dataset number (k) Scan line (j) (b) Dataset number (k) Fig. 5.11: Same as Fig. 5.10 except for data with striping noise mitigated (PC3/IMF4). 94 FOV (i) (a) Dataset Number (k) Scan line (j) (b) Dataset number (k) Fig. 5.12: Same as Fig. 5.10 except for data before striping noise mitigation on May 30, 2014. 95 FOV (i) (a) Dataset number (k) Scan line (j) (b) Dataset number (k) Fig. 5.13: Same as Fig. 5.10 except for data with striping noise mitigated (3PC/3IMF). 96 Along-track variance Before Profile Change After Profile Change Cross-track variance Date Before Profile Change After Profile Change Striping index Date Before Profile Change After Profile Change Date Fig. 5.14: (a) Along-track, (b) cross-track variances and (c) striping index at channel 8 during May 1-7, 2014 and May 29-June 4, 2014 before (red) and after (blue) noise. Ascending and descending nodes are indicated in open and solid markers, respectively. 97 H2(f) (a) Frequency (Hz) H2(f) (b) Frequency (Hz) Fig. 5.15: Square of transfer function (H(f)) that are calculated using instrument parameters (see Table 5.2) of (a) AMSU-A and (b) ATMS with two (red) and five (blue) nearest points being averaged in calibration. 98 H2(f) (a) Frequency (Hz) H2(f) (b) Frequency (Hz) Fig. 5.16: Same as Fig.4.41 except for FY-3C MWTS (a) before and (b) after profile change that occurred during May 12-18, 2014. 99 S(f) (K2/Hz) (a) Frequency (Hz) (c) S(f) (K2/Hz) S(f) (K2/Hz) (b) Frequency (Hz) Frequency (Hz) Fig. 5.17: PSD of (a) pitch-over maneuver data at ATMS channel 10, (b) white noise only and (c) the sum of white noise and flicker noise with (colored) and without (black) 81-point running mean. 100 (b) Sn(f) (K2/Hz) Sn(f) (K2/Hz) (a) Frequency (Hz) Frequency (Hz) (d) Sn(f) (K2/Hz) Sn(f) (K2/Hz) (c) Frequency (Hz) Frequency (Hz) Fig. 5.18: The PSD of the noise outputs ( Sn( f ) ) for (a) AMSU-A, (b) ATMS, (c) MWTS before profile change and (d) MWTS after profile change with (red) and without (green) adding flicker noise. 101 Sn(f) Difference (K2/Hz) Frequency (Hz) Fig. 5.19: Differences of the PSD of the noise outputs ( Sn( f ) ) between with and without flicker noise for ATMS (blue), MWTS before profile change (red) and MWTS after profile change (green). 102 CHAPTER SIX SUMMARY AND DISCUSSIONS 6.1 Major Contribution A convenient method to remove striping noise, especially for operational implementation is developed in this dissertation. The new method has been implemented in the following two contexts: (1) providing research community “stripping-noise-free” brightness temperatures datasets by eliminating the striping noise in the existing data and (2) removing the striping noise in raw count data in the real-time operational system. The root causes and factors affecting the presence and frequency distribution of the striping noise are also investigated. This may help to design the next generation microwave sensors that would have minimum impact from stripping noises 6.2 Summary and Conclusions An along-track striping noise phenomenon was detected in the global O-B distributions of ATMS temperature sounding channels. A combination of EEMD and PCA methods was developed by Qin et al. (2013) to identify and eliminate the noise. The extracted noise is at frequencies greater than 10-2 Hz with magnitudes of 0.3 K for ATMS temperature sounding channels and 1 K for window channels and ATMS, AMSU-B, MHS humidity channels. Since antenna temperatures are calculated with warm counts, cold counts, warm load temperatures and scene counts, the EEMD/PCA method can also be employed for characterizing the noise in these raw count data. However, the EEMD method is not convenient for operational implementations. Development of a set of optimal filters that can reduce striping noise in calibration counts as efficient as the EEMD does is desirable. The conventionally used boxcar filters in satellite calibration tend to alter low frequency weather signals when suppressing high-frequency noise. 103 Four sets of the optimal symmetrical filters are designed and developed for calibration counts of all 22 ATMS channels. The optimal filters efficiently remove striping noise within the raw counts data, so that the striping noise contained in the calculated antenna temperature are smoothed out. The necessity of smoothing the scene counts is also confirmed. If calibration counts are smoothed but the scene counts are not, striping noise still exists and is visible in global O-B distributions. A similar set of 22 optimal filters was designed for removing directly the striping noise in brightness temperatures for all 22 ATMS channels. These optimal filters were shown to be effective in smoothing out the striping noise while keeping the large-scale weather signals unaltered. They are also convenient for implementation in the operational systems. For validation purposes, these optimal filters were applied to ATMS pitch-over maneuver data. The striping noise in pitch-over maneuver data in all channels was effectively smoothed out. As expected, the along-track variance is reduced, cross-track variance does not change much, and the SI approaches a unit value. The LWP data retrieved from brightness temperatures at ATMS window channels 1 and 2 before and after striping noise mitigation was compared. The LWP retrievals without striping noise mitigation reveal a striping noise pattern with a magnitude as large as 0.05 kg/m2. The striping noise in the LWP differences has a linear relationship to the striping noise in brightness temperatures of ATMS window channels. The impact of ATMS striping noise on LWP retrievals further confirms the necessity and effectiveness of applying the optimal filters on the ATMS brightness temperature observations. Observation from a recent instrument FY-3C MWTS is found to also contain striping noise of a magnitude much larger than that of ATMS. It is shown the striping noise exists not only within the first PC coefficient, but also the second and third PC coefficients. The EEMD/PCA 104 method is then used to extract the noise from the first three PC coefficients for MWTS data. The magnitude of the striping noise in the temperature channels can be as large as 1K, which is much larger than that in ATMS. As pitch-over maneuver data of MWTS is not available, a global striping index is defined for brightness temperature observations. It is shown that after noise mitigation, the global striping index dropped to unit value, which indicates that the noise mitigation is effective. MWTS has experienced a scan profile change during May 12-18, 2014. After the scan profile change, MWTS’s antenna starts to rotate at a constant speed. Noise identification and elimination for two weeks before and after the profile change showed that the striping noise is even stronger after the scan profile change. This finding confirms the scan speed is probably not the root-cause of striping noise contaminated in satellite microwave sensor observations. In order to gain further insight on the root cause of the striping noise, a transfer function is calculated with instrument parameters such as scan cycle, calibration integration time and scene integration time for AMSU-A, ATMS and MWTS before and after profile change. The transfer function links the instrument noise to the output noise of the amplifier. White noise and flicker noise are then simulated as the instrument noise and forced by the transfer functions for all the above-mentioned four microwave sensors. The difference of the output PSD between signals with and without flicker noise resembles PSD of the striping noise. It is also demonstrated that even with the same instrument signals, amplitudes of the output noise could vary because of the differences of transfer functions among different sensors. The FY-3C MWTS after scan profile change has the largest striping noise. These facts indicate that flicker noise signal induces the striping noise, and transfer function can modify the striping noise in terms of both its magnitude and peak frequency. 105 6.3 Future Work Similar striping noise analysis and mitigation will be conducted for other sensors such as Tropical Rainfall Measuring Mission Microwave Imager (TMI) onboard Global Precipitation Measurement (GPM). The impacts of the striping noise on numerical weather prediction will be carried out. 106 APPENDIX A LIST OF ACRONYMS Acronym Meaning AMSU Advanced Microwave Sounding Unit ATBD Advanced Technology Microwave Sounder ATMS Advanced Technology Microwave Sounder EEMD Ensemble Empirical Mode Decomposition FOV Field of View FY FengYun GPM Global Precipitation Measurement IMF Intrinsic Mode Function NOAA National Oceanic and Atmospheric Administration NEDT Noise Equivalent Delta Temperature NPP National Polar-Orbiting Partnership NWP Numerical weather Prediction MSU Microwave Sounding Unit MHS Microwave Humidity Sounder MWHS MicroWave Humidity Sounder MWTS MicroWave Temperature Sounder O-B Difference between observed and simulated brightness temperature PC Principle Component PCA Principle Component Analysis RF Radio Frequency TIROS-N Television Infrared Observation Satellite -N TMI Tropical Rainfall Measuring Mission Microwave Imager 107 APPENDIX B COPYRIGHT PERMISSION JOHN WILEY AND SONS LICENSE TERMS AND CONDITIONS Jul 10, 2015 This Agreement between Yuan Ma ("You") and John Wiley and Sons ("John Wiley and Sons") consists of your license details and the terms and conditions provided by John Wiley and Sons and Copyright Clearance Center. License Number 3665390006856 License date Jul 10, 2015 Licensed Content Publisher John Wiley and Sons Licensed Content Publication Journal of Geophysical Research: Atmospheres Licensed Content Title Striping Noise Mitigation in ATMS Brightness Temperatures and Its Impact on Cloud LWP Retrievals Licensed Content Author Yuan Ma,Xiaolei Zou Licensed Content Date Jun 18, 2015 Pages 1 Type of use Dissertation/Thesis Requestor type Author of this Wiley article Format Electronic Portion Full article 108 Will you be translating? No 109 REFERENCES A. Antoniou (1993), Digital Filters: Analysis, Design, and Applications, New York, NY: McGraw-Hill. Bormann, N., A. Fouilloux, and W. Bell (2013), Evaluation and assimilation of ATMS data in the ECMWF system, J. Geophys. Res. Atmos., 118, 12,970–12,980. doi:10.1002/2013JD020325. Chew, Weng Cho (1995), Waves and fields in inhomogeneous media. Vol. 522. New York: IEEE press. Goldberg, M. D. and F. Weng (2006), Advanced Technology Microwave Sounder (book chapter), in Earth Science Satellite Remote Sensing, vol. 1: Science and Instrument, Springer Berlin Heidelberg, 243-253. doi: 10.1007/978-3-540-37293-6_13. Hersman, M. S., and G. A. Poe (1981). Sensitivity of the total power radiometer with periodic absolute calibration. Microwave Theory and Techniques, IEEE Transactions on, 29(1), 32-40. JPSS ATMS ATBD (2011), Joint Polar Satellite System (JPSS) Advanced Technology Microwave Sounder (ATMS) SDR Radiometric Calibration Algorithm Theoretical Basic Document (ATBD), http://www.star.nesdis.noaa.gov/jpss/documents/ATBD/GSFC_47400043_JPSS_ATMS_SDR_ Radiometric_Calibration_ATBD__Alt._doc._no._D43751_.pdf. Kim, E., C.-H. J. Lyu, K. Anderson, R. V. Leslie, and W. J. Blackwell (2014), S-NPP ATMS instrument prelaunch and on-orbit performance evaluation, J. Geophys. Res. Atmos., vol. 119, pp. 5653–5670. doi:10.1002/2013JD020483. Ma, Y. and X. Zou (2015), Striping noise mitigation in ATMS brightness temperatures and its impacts on cloud LWP retrievals, J. Geophys. Res. Atmos., vol. 120, doi: 10.1002/2015JD023162. Petty G. W. (2006), A first course in atmospheric radiation (2nd edition), Sundog Publishing, 444 pp. Qin, Z., X. Zou, and F. Weng (2013), Analysis of ATMS striping noise from its Earth scene observations, J. Geophys. Res. Atmos., 118, 13,214–13,229. doi:10.1002/2013JD020399. Ulaby, F. T., R. K. Moore, and A. K. Fung (1981). Microwave Remote Sensing: Microwave remote sensing fundamentals and radiometry. Vol. 1. Addison-Wesley Publishing Company, Advanced Book Program/World Science Division. Weng, F., and N. C. Grody (1994), Retrieval of cloud liquid water using the special sensor microwave imager (SSM/I), J. Geophys. Res., 99, D12, 25535–25551, doi:10.1029/94JD02304. 110 Weng, F. X. Zou and Z. Qin (2014), Uncertainty of AMSU-A derived temperature trends in relationship with clouds and precipitation. Clim. Dyn., 43, 1439-1448. Weng, F., X. Zou, N. Sun, H. Yang, M. Tian, W. J. Blackwell, X. Wang, L. Lin, and K. Anderson (2013), Calibration of Suomi national polar-orbiting partnership advanced technology microwave sounder”, J. Geophys. Res. Atmos., 118, 11,187–11,200. doi:10.1002/jgrd.50840. Weng, F., L. Zhao, R. R. Ferraro, G. Poe, X. Li, and N. C. Grody (2003), Advanced microwave sounding unit cloud and precipitation algorithms, Radio Sci., 38, 8068, doi:10.1029/2002RS002679, 4. Weng, F., X. Zou, X. Wang, S. Yang, and M. D. Goldberg (2012), Introduction to Suomi national polar-orbiting partnership advanced technology microwave sounder for numerical weather prediction and tropical cyclone applications, J. Geophys. Res., 117, D19112 Weng F., H. Yang, and X. Zou, “ On convertibility from antenna to sensor brightness temperature for ATMS, IEEE GEOSCIENCE AND REMOTE SENSING LETTERS, vol. 10, no. 4, pp. 771775, July 2013. Wu Z. and N. E. Huang (2004), A study of the characteristics of white noise using the empirical mode decomposition method, Proc. R. Soc. Lond., vol. 460, pp. 1597–1611. Wu, Z., and N. E. Huang (2009), Ensemble empirical mode decomposition: A noise-assisted data analysis method, Adv. Adapt Data Anal, 1(1), 1–41. Wu, Z., N. E. Huang, and X. Chen (2009), The multi-dimensional ensemble empirical mode decomposition method, Adv. Adapt Data Anal, 1(3), 339–372. Zou X., Z. Qin, and F. Weng (2013), Improved quantitative precipitation forecasts by MHS radiance data assimilation with a newly added cloud detection algorithm, Mon. Wea. Rev., 141, 3203–3221. 111 BIOGRAPHICAL SKETCH Yuan Ma grew up in Yangzhou, Jiangsu Province, China. She received Bachelor’s degree in Science from School of Atmospheric Sciences in Nanjing University in June 2010, and a Master’s degree in Science from the department of Earth, Oceanic and Atmospheric Sciences in Florida State University in May 2013. 112

1/--страниц