Fundamentals of Statistical Signal Processing--Estimation Theory

HA LL SIGNAL PROCESSING SERIES SERIES  Alan V. Oppenheim, Series Editor  H UNT   Digital Image Restomtion Th e Fast Fast Fourie Fourierr T mnsfo rm Th e Fast Fast Fourier  and Its Applications Analysis, BURDIC Underwater Acoustic System Analysis, C ASTLEMAN   Digital Image Processing AND G RANT   Adaptive Filters AND RABINER   Multimte Digital Signal Processing   Multidimensional Digital Signal Processing DUDGEON AND HAMMING   Digital Filters,  Advances in Spectrum Analysis and Array Processing, Vols. ED. Processing ED.  Array Signal Processing AND NOLL   Digital Coding of  J OHNSON A N D D UDGEON  Array Signal Processing: Concepts and Techniques of  Statistical Signal Processing: Estimation Theory KAY Spectral Estimation KAY   Acoustic Waves: Devices, Imaging, and Analog Signal Processing LEA , ED. in Speech Recognition LIM Two -  Dimensional Signal and Image Processing L IM , ED. Speech Enhancement    Advanced Topicsin Signal Processing I ,EDS. LIM AND OP PE  N  N HE  M  M ARPLE  Digital Spectral Analysis with Applications   Number Theory  Digital Processing AND  Lessons in   Digital Estimation Theory  Digital Processing OPPENHEIM, ED.  Applications of  Digital OPPENHEIM AND NAWAB, EDS. Symbolic and Knowledge-  Based Signal Processing Signals and Systems OPPENHEIM, WITH YOUNG   Digital Signal Processing OPPENHEIM AND  Discrete - Tim e Signal Processing Processing OPPENHEIM AND AL . Objective Measures of Speech Speech Q uality QUACKENBUSH Theory and Applications of Digital Signal Processing AND G OLD   Digital Processing of Speech AND ROBINSON AN D TREITEL Geophysical Signal Analysis STEARNS AND DAVID Processing Algorithms STEARNS AND HUSH   Digital Signal Analysis, Seismic Applications of  Homomorphic  Homomorphic Signal Processing Processing VAIDYANATHAN   Multimte Systems and Filter Banks Processing AND STEARNS  Adaptive AND

Fundamentals of  Statistical Statis tical Signal Signal Processing: Processing: Est

ion Theory

Steven M. Kay University of Rhode Island

For book  For book  and   and bookstore 

I

http://wmn.prenhrll.com gopher to gopher.prenhall.com

I

Upper Saddle River, NJ 07458

Contents

Preface Introduction 1.1 Estimation in Signal Processing . . . . . . . . . . . . . . . . . . . . . . . 1.2 The Mathematical Estimation Problem . . . . . . . . . . . . . . . . 1.3 Assessing Estimator Performance . . . . . . . . . . . . . . . . . . . . . . Notes to the Reader Reader . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Some Notes

xi 1 1

..

Minimum Variance Unbiased Estimation Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Unbiased Estimat ors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Unbiased 2.4 Minimum Variance Criterion . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Existence of the Minimum Variance Unbiased Estimator . . . . . . . . . Variance Unbiased Estimator . . . . . . . . . . . 2.6 Finding th e Minimum Variance Extension t o a Vector Parameter . . . . . . . . . . . . . . . . . . . . . . 2.7 Extension 2.1

3

7 

9 12

15 15 15

16 19

20 21 22

Cramer-Rao Lower Lower Boun d 27 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 Accuracy Considerations . . . . . . . . . . . . . . . . . . . . . 3.3 Estimator Accuracy 28 3.4 Cramer -Rao Lower Bound . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.5 General CRLB for Signals in White Gaussian Noise . . . . . . . . . . . . 35 3.6 Transformation of Parameters . . . . . . . . . . . . . . . . . . . . . . . . 37 Extension t o a Vector Vector P arameter . . . . . . . . . . . . . . . . . . . . . . 3.7 Extension 39 3.8 Vector Parameter CRLB for Transformations . . . . . . . . . . . . . . . 45 3.9 CRLB for the General Gaussian Case . . . . . . . . . . . . . . . . . . . 47 3.10 Asymptotic CRLB for WSS Gaussian Random Processes . . . . . . . . . 50 3.1 1 Signal Processing Examples . . . . . . . . . . . . . . . . . . . . . . . . . 53 3A Derivation of Scalar Parameter CRLB . . . . . . . . . . . . . . . . . . . 67 3B Derivation of Vector Parameter CRLB . . . . . . . . . . . . . . . . . . . 70 Derivation of General Gaussian CRLB . . . . . . . . . . . . . . . . . . . 73 3D Derivation of Asymptotic CRLB . . . . . . . . . . . . . . . . . . . . . . 77 3.1

Contents

Preface Introduction 1.1 Estimation in Signal Processing . . . . . . . . . . . . . . . . . . . . . . . 1.2 The Mathematical Estimation Problem . . . . . . . . . . . . . . . . 1.3 Assessing Estimator Performance . . . . . . . . . . . . . . . . . . . . . . Notes to the Reader Reader . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Some Notes

xi 1 1

..

Minimum Variance Unbiased Estimation Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Unbiased Estimat ors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Unbiased 2.4 Minimum Variance Criterion . . . . . . . . . . . . . . . . . . . . . . . . 2.5 Existence of the Minimum Variance Unbiased Estimator . . . . . . . . . Variance Unbiased Estimator . . . . . . . . . . . 2.6 Finding th e Minimum Variance Extension t o a Vector Parameter . . . . . . . . . . . . . . . . . . . . . . 2.7 Extension 2.1

3

7 

9 12

15 15 15

16 19

20 21 22

Cramer-Rao Lower Lower Boun d 27 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 3.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 Accuracy Considerations . . . . . . . . . . . . . . . . . . . . . 3.3 Estimator Accuracy 28 3.4 Cramer -Rao Lower Bound . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.5 General CRLB for Signals in White Gaussian Noise . . . . . . . . . . . . 35 3.6 Transformation of Parameters . . . . . . . . . . . . . . . . . . . . . . . . 37 Extension t o a Vector Vector P arameter . . . . . . . . . . . . . . . . . . . . . . 3.7 Extension 39 3.8 Vector Parameter CRLB for Transformations . . . . . . . . . . . . . . . 45 3.9 CRLB for the General Gaussian Case . . . . . . . . . . . . . . . . . . . 47 3.10 Asymptotic CRLB for WSS Gaussian Random Processes . . . . . . . . . 50 3.1 1 Signal Processing Examples . . . . . . . . . . . . . . . . . . . . . . . . . 53 3A Derivation of Scalar Parameter CRLB . . . . . . . . . . . . . . . . . . . 67 3B Derivation of Vector Parameter CRLB . . . . . . . . . . . . . . . . . . . 70 Derivation of General Gaussian CRLB . . . . . . . . . . . . . . . . . . . 73 3D Derivation of Asymptotic CRLB . . . . . . . . . . . . . . . . . . . . . . 77 3.1