Fundamentals of Statistical Signal Processing, Volume I Estimation Theory by Steven M.kay

PRENTICE H A L L SIGNAL PROCESSING SERIES

Alan V. Oppenheim, Series Editor

ANDREWSAND H UNT Digital Image Restomtion BRIGHAM T h e Fast Fourier Tmnsform BRIGHAM T h e Fast Fourier Transform and Its Applications BURDIC Underwater Acoustic System Analysis, 2/E CASTLEMAN Digital Image Processing COWAN AND G RANT Adaptive Filters CROCHIERE AND R ABINER Multimte Digital Signal Processing D UDGEON AND MERSEREAU Multidimensional Digital Signal Processing H AMMING Digital Filters, 3 / E HAYKIN,ED. Advances in Spectrum Analysis and Array Processing, Vols. I € 5 II HAYKIN,ED. Array Signal Processing JAYANT AND N OLL Digital Coding of waveforms J OHNSON A N D D UDGEON Array Signal Processing: Concepts and Techniques K AY Fundamentals of Statistical Signal Processing: Estimation Theory KAY Modern Spectral Estimation KINO Acoustic Waves: Devices, Imaging, and Analog Signal Processing L EA , ED. Trends in Speech Recognition LIM Two-Dimensional Signal and Image Processing L IM , ED. Speech Enhancement L IM AND OPPENHEIM,EDS. Advanced Topics in Signal Processing M ARPLE Digital Spectral Analysis with Applications MCCLELLAN AND RADER Number Theory an Digital Signal Processing MENDEL Lessons in Digital Estimation Theory OPPENHEIM, ED. Applications of Digital Signal Processing OPPENHEIM AN D NAWAB, EDS. Symbolic and Knowledge-Based Signal Processing OPPENHEIM, WILLSKY, WITH Y OUNG Signals and Systems OPPENHEIM AND SCHAFER Digital Signal Processing OPPENHEIM AND SCHAFERDiscrete- Time Signal Processing Q UACKENBUSH ET AL. Objective Measures of Speech Quality RABINERAND G OLD Theory and Applications of Digital Signal Processing RABINERAND SCHAFERDigital Processing of Speech Signals ROBINSON AND TREITEL Geophysical Signal Analysis STEARNS AND DAVID Signal Processing Algorithms STEARNS AND HUSH Digital Signal Analysis, 2/E TRIBOLETSeismic Applications of Homomorphic Signal Processing VAIDYANATHAN Multimte Systems and Filter Banks WIDROW AND STEARNS Adaptive Signal Processing

Fundamentals of Statistical Signal Processing: Est imat ion Theory

Steven M. Kay University of Rhode Island

For book and bookstore information

I

http://wmn.prenhrll.com gopher to gopher.prenhall.com

I

Upper Saddle River, NJ 07458

Contents Preface

xi

1 Introduction 1.1 Estimation in Signal Processing . . . . . . . . . . . . . . . . . . . . . . . 1.2 The Mathematical Estimation Problem . . . . . . . . . . . . . . . . . . 1.3 Assessing Estimator Performance . . . . . . . . . . . . . . . . . . . . . . 1.4 Some Notes to the Reader . . . . . . . . . . . . . . . . . . . . . . . . . .

1 1 7 9 12

2 Minimum Variance Unbiased Estimation 15 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 15 2.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.3 Unbiased Estimators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Minimum Variance Criterion . . . . . . . . . . . . . . . . . . . . . . . . 19 2.5 Existence of the Minimum Variance Unbiased Estimator . . . . . . . . . 20 2.6 Finding the Minimum Variance Unbiased Estimator . . . . . . . . . . . 21 2.7 Extension to a Vector Parameter . . . . . . . . . . . . . . . . . . . . . . 22 3 Cramer-Rao Lower Bound 3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Estimator Accuracy Considerations . . . . . . . . . . . . . . . . . . . . . 3.4 Cramer-Rao Lower Bound . . . . . . . . . . . . . . . . . . . . . . . . . . 3.5 General CRLB for Signals in White Gaussian Noise . . . . . . . . . . . . 3.6 Transformation of Parameters . . . . . . . . . . . . . . . . . . . . . . . . 3.7 Extension to a Vector Parameter . . . . . . . . . . . . . . . . . . . . . . 3.8 Vector Parameter CRLB for Transformations . . . . . . . . . . . . . . . 3.9 CRLB for the General Gaussian Case . . . . . . . . . . . . . . . . . . . 3.10 Asymptotic CRLB for WSS Gaussian Random Processes . . . . . . . . . 3.1 1 Signal Processing Examples . . . . . . . . . . . . . . . . . . . . . . . . .

3A 3B 3C 3D

Derivation Derivation Derivation Derivation

of Scalar Parameter CRLB of Vector Parameter CRLB of General Gaussian CRLB of Asymptotic CRLB . . . vii

27 27 27 28 30 35 37 39 45 47 50 53 . . . . . . . . . . . . . . . . . . . 67 . . . . . . . . . . . . . . . . . . . 70 . . . . . . . . . . . . . . . . . . . 73 ................... 77