Maria S. Greco

(Department of Information Engineering, University of Pisa, Italy)

https://s3-seminar.github.io/seminars/maria-s-greco

Title — Modeling and mismodeling in radar applications: parameter estimation and bounds

Abstract — The problem of estimating a deterministic parameter vector of acquired data is ubiquitous in signal processing applications. A fundamental assumption underlying most estimation problems is that the true data model and the model assumed to derive an estimation algorithm are the same, that is, the model is correctly specified. This lecture will focus on the general case in which, for some non-perfect knowledge of the true data model or for operative constraints on the estimation algorithm there is a mismatch between assumed and true data model. After a short first part dedicated to explain the radar framework of the estimation problem, the lecture will be dedicated to the evaluation of lower bounds on the Mean Square Error of the estimate of a deterministic parameter vector under misspecified model with particular attention to Mismatched Maximum Likelihood estimator and Huber bounds.

Biography — Maria S. Greco graduated in Electronic Engineering in 1993 and received the Ph.D. degree in Telecommunication Engineering in 1998, from University of Pisa, Italy. From December 1997 to May 1998 she joined the Georgia Tech Research Institute, Atlanta, USA as a visiting research scholar where she carried on research activity in the field of radar detection in non-Gaussian background. In 1993 she joined the Department of Information Engineering of the University of Pisa, where she is Associate Professor since December 2011. She’s IEEE fellow since January 2011 and she was co-recipient of the 2001 IEEE Aerospace and Electronic Systems Society’s Barry Carlton Award for Best Paper and recipient of the 2008 Fred Nathanson Young Engineer of the Year award for contributions to signal processing, estimation, and detection theory. She has been co-general-chair of the 2007 International Waveform Diversity and Design Conference (WDD07), Pisa, Italy, in the Technical Committee of the 2006 EURASIP Signal and Image Processing Conference (EUSIPCO), Florence, Italy, in the Technical Committee of the 2008 IEEE Radar Conference, Rome, Italy, in the Organizing Committee of CAMSAP09, Technical co-chair of CIP2010 (Elba Island, Italy), General co-Chair of CAMSAP2011 (San Juan, Puerto Rico), Publication Chair of ICASSP2014, Florence, Italy, Technical Co-Chair of the CoSeRa2015, Pisa, Italy and Special Session Chair of CAMSAP2015, Cancun, Mexico. She is lead guest editor of the special issue on "Advanced Signal Processing for Radar Applications" to appear on the IEEE Journal on Special Topics of Signal Processing, December 2015, she was guest co-editor of the special issue of the Journal of the IEEE Signal Processing Society on Special Topics in Signal Processing on "Adaptive Waveform Design for Agile Sensing and Communication," published in June 2007 and lead guest editor of the special issue of International Journal of Navigation and Observation on” Modelling and Processing of Radar Signals for Earth Observation published in August 2008. She’s Associate Editor of IET Proceedings – Sonar, Radar and Navigation, Associate Editor-in-Chief of the IEEE Aerospace and Electronic Systems Magazine, member of the Editorial Board of the Springer Journal of Advances in Signal Processing (JASP), Senior Editorial board member of IEEE Journal on Selected Topics of Signal Processing (J-STSP), member of the IEEE Signal Array Processing (SAM) Technical Committees. She's also member of the IEEE AES and IEEE SP Board of Governors and Chair of the IEEE AESS Radar Panel. She's as well SP Distinguished Lecturer for the years 2014-2015, AESS Distinguished Lecturer for the years 2015-2016 and member of the IEEE Fellow Committee. Maria is a coauthor of the tutorials entitled “Radar Clutter Modeling”, presented at the International Radar Conference (May 2005, Arlington, USA), “Sea and Ground Radar Clutter Modeling” presented at 2008 IEEE Radar Conference (May 2008, Rome, Italy) and at 2012 IEEE Radar Conference (May 2012, Atlanta, USA), coauthor of the tutorial "RF and digital components for highly-integrated low-power radar" presented at the same conference, of the tutorial "Recent Advances in Adaptive Radar Detection" presented at the 2014 International Radar Conference (October 2014, Lille, France) and co-author of the tutorial "High Resolution Sea and Land Clutter Modeling and analysis", presented at the 2015 IEEE International Radar Conference (May 2015, Washington DC, USA). Her general interests are in the areas of statistical signal processing, estimation and detection theory. In particular, her research interests include clutter models, spectral analysis, coherent and incoherent detection in non-Gaussian clutter, CFAR techniques, radar waveform diversity and bistatic/mustistatic active and passive radars. She co-authored many book chapters and more than 150 journal and conference papers.

Transcript

1. Centrale Supélec – June 2015 MODELING AND MISMODELING IN RADAR APPLICATIONS: PARAMETER ESTIMATION AND BOUNDS Maria S. Greco Dipartimento di Ingegneria dell’Informazione University of Pisa (Italy)
2. Outline of the talk 2 The radar scenario: covariance matrix estimation in non-Gaussian clutter The Misspecified Cramér-Rao Bound (MCRB) and the Huber Bound Matrix estimation for CES distributed disturbance Examples and conclusions
3. 3 • Radar systems detect targets by examining reflected energy, or returns, from objects • Along with target echoes, returns come from the sea surface, land masses, buildings, rainstorms, and other sources. This disturbance is called clutter Radar scenario •Much of this clutter is far stronger than signals received from targets of interest • The main challenge to radar systems is discriminating these weaker target echoes from clutter • Coherent signal processing techniques are used to this end
4. Radar scenario: the clutter The function of the clutter model is to define a consistent theory whereby a physical model results in an analytical model which can be used for radar design and performance analysis.  Radar clutter is defined as unwanted echoes, typically from the ground, sea, rain or other atmospheric phenomena.  These unwanted returns may affect the radar performance and can even obscure the target of interest.  Hence clutter returns must be taken into account in designing a radar system. 4
5. •In early studies, the resolution capabilities of radar systems were relatively low, and the scattered return from clutter was thought to comprise a large number of scatterers •From the Central Limit Theorem (CLT), researchers in the field were led to conclude that the appropriate statistical model for clutter was the Gaussian model (for the I & Q components), i.e., the amplitude R is Rayleigh distributed) •In the quest for better performance, the resolution capabilities of radar systems have been improved •For detection performance, the belief originally was that a higher resolution radar system would intercept less clutter than a lower resolution system, thereby increasing detection performance • However, as resolution has increased, the clutter statistics have no longer been observed to be Gaussian, and the detection performance has not improved directly. The radar system is now plagued by target-like “spikes” that give rise to non-Gaussian observations 5 Radar scenario: the clutter
6. • These spikes are passed by the detector as targets at a much higher false alarm rate (FAR) than the system is designed to tolerate • The reason for the poor performance can be traced to the fact that the traditional radar detector is designed to operate against Gaussian noise • New clutter models and new detection strategies are required to reduce the effects of the spikes and to improve detection performance Spikes in the horizontally polarized sea clutter data Spikes in the vertically polarized sea clutter data 6 Radar scenario: the clutter
7. Empirical studies have produced several candidate models for spiky non- Gaussian clutter, the most popular being the Weibull distribution, the K distribution, the log-normal, the generalized K, the Student-t (or Pareto), etc. Measured sea clutter data (IPIX database) the Weibull , K, log-normal etc. have heavier tails than the Rayleigh 10-3 10-2 10-1 100 101 102 0 0.2 0.4 0.6 0.8 1 Weibull Lognormal K GK LNt IG Histogram pdf Amplitude (V) 7 Radar scenario: the clutter
8. 8 In general, taking into account the variability of the local power , that can be modeled itself as a random variable, we obtain the so-called compound-Gaussian model, very popular in clutter modeling, where 2 2 ( | ) exp ( ) r r p r u r      =      0 ( ) ( | ) ( ) ; 0 p r p r p d r     =     According to the CG model: ( ) ( ) ( ) z n n x n  = ( ) ( ) ( ) I Q x n x n jx n =  Speckle: complex Gaussian process, takes into account the local backscattering Texture: non negative random process, takes into account the local mean power 8 Radar scenario: compound-Gaussian model
9. Particular cases of CG model (amplitude PDF):     1 1 4 4 4 ( ) 2 R p r r K r u r                 =          K (Gamma texture)   2 2 0 2 ( ) exp b b b R br r p r d                    =                   GK (Generalized Gamma texture) t-Student or Pareto   1 ( ) exp ( ) c c R c r p r r b u r b b      =        W, Weibull 9 ( 1) 2 ( ) 2 1 ( ) R p r r r u r l h h l     =      Compound-Gaussian model and CES The CG model belongs to the family of Complex Elliptically Symmetric (CES) distributions
10. 10 Any radar detector should be designed to operate in an unknown interference environment, i.e. it should adaptively estimate the disturbance covariance matrix R=2S (i.e. power 2 and scatter matrix S). Detectors that estimate adaptively (on-line) the disturbance covariance matrix are named “adaptive detectors”. When the interference environment is a-priori unknown, several approaches are possible. The most commonly used are: 1) Assume that the disturbance is white, i.e. R=2I, and implement the non-adaptive detector which is optimal for that scenario (clearly, it will perform suboptimally in correlated disturbance). 2) Model the clutter as an autoregressive (AR) process and estimate the clutter covariance matrix by estimating the AR parameters [see e.g. S. Haykin and A. Steinhardt. Adaptive Radar Detection and Estimate. John Wiley and Sons, 1992]. Radar scenario: adaptive detection
11. 3) Assume that a set of secondary data xi =di , i=1,…,M is available, free of target signal components, sharing the same correlation characteristics as the disturbance in the Cell Under Test (CUT), then estimate 2 and S from M>N secondary data vectors. This scenario is usually referred to as homogeneous environment and this is the case we treat here. Secondary data are usually obtained by processing range gates in spatial proximity with the CUT. The data from the CUT are termed primary data. Problem: we want to estimate the scatter matrix (shape matrix) S of Complex Elliptically Symmetric (CES) distributed clutter for adaptive radar detection 11 Radar scenario: adaptive detection
12. 12 Radar scenario: disturbance matrix estimation A fundamental assumption underlying most estimation problems is that the true data model and the model assumed to derive an estimation algorithm are the same, that is, the model is correctly specified. If the unknown parameters are deterministic, a typical estimator is the Maximum Likelihood one and its performance asymptotically tends to the Cramér-Rao Lower Bound (CRLB). But for some non-perfect knowledge of the true data model or for operative constraints on the estimation algorithm (i.e. easy of implementation, computational cost, very short operational time) there can be a mismatch between assumed and true data model. In this case, which is the bound on the performance of ML and other estimators? Problem: we want to estimate the scatter matrix S of Complex Elliptically Symmetric (CES) distributed clutter in presence of model mismatch
13. Parameter estimation and bounds Most treatments of parameter bounds assume perfect knowledge of the data distribution. When the assumed distribution for the measured data differs from the true distribution, the model is said to be misspecified. While much attention has been devoted to understanding the performance of Maximum Likelihood (ML) and Least Squares (LS) estimators under model misspecification (see e.g. [Hub67], [Whi81], [Fom03]), little consideration appears to have been given to the concomitant bounds on performance. 3
14. 14 Parameter bounds under misspecified models Problem: Given a parameter vector , we are interested in finding a lower bound on the mean square error (MSE) of any estimator, based on the assumed probability density function (pdf) fX (x; ) in the presence of misspecified data model  misspecified bound (MCRB) Choice of score function  The tightest bounds are obtained via a choice of score function with two properties [McW93]: (1) zero mean; (2) dependence on the sufficient statistic T(x) for estimating . ( ) true pdf of , ( ; ) assumed pdf, ( ) ( ; ) X X X X p f p f = =  x x x θ x x θ
15. 15 First, we define a score function by subtracting to the score function used to derive the classical CRLB its possibly non-zero mean value :   ( ) ln ( ; ) ln ( ; ) X p X f E f    θ θ θ s x x θ x θ    ( ) ln ln ( ) ( ; ) X p X X p p D p f E p d f f         =              x x x x θ   where we expressed the 2nd term as a function of the Kullback- Leibler (KL) divergence between the assumed density and the true density :   0 ( ; ) ( ) X X D p f f p  =  = x θ x    ln ( ; ) X f D p f =    θ θ θ x θ Parameter bounds under misspecified models
16. 16 The MSE on the estimation of the elements of the true parameter vector is given by the diagonal entries of the following matrix :                ˆ ˆ ˆ ( ), ( ) ( ) true parameter vector ˆ ˆ ( ) ( ) bias vector ˆ ( ) ( ) ˆ covariance matrix of ( ) ˆ ˆ ( ) ( ) ( ) ( ) T T p p p T T T p p E E E E E   =  =   =   = =  θ θ θ θ θ θ θ θ θ θ θ M θ x θ θ x θ θ x θ C b b θ μ θ x b θ θ x θ μ e x θ x μ C θ x C e x e x θ x θ x μ μ      Parameter bounds under misspecified models
17. 17 The covariance inequality we are looking for can be obtained as follows: Ω is a Grammian matrix  it is positive semi-definite. If Ω is symmetric and K is invertible, the following properties hold [Boy04]:   1 ( ) ( ) ( ) 0 ( ) (Shur complement of in ) T T p T T E        =               θ θ θ θ θ θ θ θ θ θ θ e x C T Ω e x s x s x T K S C T K T C Ω   1 0 0 T       θ θ θ θ Ω S C T K T Parameter bounds under misspecified models
18. 18     1 ˆ( ), , : T T MCRB p f   =  θ θ θ θ θ M θ x θ θ T K T b b     ln ( ; ) p X E f D p f   =  θ θ θ θ d x θ  The right–hand side term is the lower bound on the MSE of any estimator in the presence of misspecified data model: the misspecified bound (MCRB). T is the so-called expansion coefficient (EC) matrix [McW93]:             ˆ ( ) ( ) ( ) ln ( ; ) ˆ( ) ln ( ; ) T T p p X T T p X E E f D p f E f =     =   θ θ θ θ θ θ θ θ θ T e x s x θ x μ x θ θ x x θ μ d  Parameter bounds under misspecified models
19. 19              ( ) ( ) ln ( ; ) ln ( ; ) ln ( ; ) T T p p X T T p X X E E f D p f E f f =    =    θ θ θ θ θ θ θ θ θ θ K s x s x x θ x θ x θ d d   K is the information matrix corresponding to the given score function:     ln ( ; ) p X E f D p f   =  θ θ θ θ d x θ  This is the version for vector parameter of the result that was derived in a different way for the scalar case by Richmond [Ric13].     1 ˆ( ), , : T T MCRB p f   =  θ θ θ θ θ M θ x θ θ T K T b b Parameter bounds under misspecified models
20. 20 Lower Bound for the MML Estimator Consider the Mismatched Maximum Likelihood (MML) estimator: ˆ ( ) argmax ( ; ), ( ; ) ( ) true pdf MML X X X f f p =  θ θ x x θ x θ x In [Hub67], Huber proved that under some assumptions, the MML converges almost surely (a.s.) to the value 0 that minimizes the KL divergence:   1 1 ( ; ) ( ; ) M M k X X k k k IID f f = =  =  x x θ x θ         . . 0 ˆ ( ) argmin argmin ln ( ; ) a s MML p X M D p f E f     = =  θ θ θ θ z θ x θ
21. 21 Lower Bound for the MML Estimator Additionally, Huber [Hub67] and White [Whi82] proved the consistency and asymptotic normality of the MML under some mild regularity conditions:     0 0 0 1 1 0 ˆ ( ) , MML M M     θ θ θ θ z θ 0 A B A     2 , ln ( ; ) X k p i j i j f E        =         θ x θ A   , ln ( ; ) ln ( ; ) X k X k p i j i j f f E         =          θ x θ x θ B 0 ( ) 1 HB for M = θ where:
22. 22 Lower Bound for the MML Estimator If the model is correctly specified:   ; ( ) true pdf X X f p = x θ x 2 ln ( ) ln ( ) ln ( ) , 0 X k X k X k p p i j i j p p p E E i j                 =                    =    = θ θ θ θ x x x A B A B       1 MCRB HB CRB  = = = θ θ θ B θ   ; ( ), 1,2 , X k X k f p k M  = = x θ x 
23. 23 Lower Bound for the MML Estimator Let’s express it in terms of 0 : The general MCRB we derived:               0 0 0 0 0 0 ˆ ˆ ˆ ( ), ( ) ( ) ˆ ˆ ( ) ( ) ˆ( ), 2 T p T p T T E E =   =       =   θ M θ x θ θ x θ θ x θ θ x θ θ θ θ x θ θ θ M θ x θ b r rr     1 ˆ( ), , : T T MCRB p f   =  θ θ θ θ θ M θ x θ θ T K T b b   0 0 0 ˆ where ( ) , p E   θ b θ θ x r θ θ     0 ( 0) D p f =  = θ θ θ θ   0 0 0 0 0 0 1 ˆ( ), 2 T T T T      θ θ θ θ θ θ M θ x θ T K T b b b r rr Then:
24. Lower Bound for the MML Estimator We have to see how the MCRB specializes when: ˆ ˆ ( ) ( ) MML = θ x θ x   ˆ ( ), ? MML  M θ x θ Assume first that the MML estimator is unbiased w.r.t. 0 :   0 ˆ ( ) p MML E = = θ μ θ x θ This is always true asymptotically [Hub67] and approximately for large M [Ric13]. Then, we define as consistent a mismatched estimator if, as the number of observations M goes to infinity, it converges in probability to the true parameter vector: 24   0 0 0 ˆ ( ) 0 p MML E   =  = θ θ b θ θ x θ μ  . 0 0 ˆ ( ) 0 prob MML M    =  =  = θ x θ θ θ r θ θ
25. 25 Lower Bound for the MML Estimator Hence, our covariance inequality, evaluated in θ0 , becomes: In conclusion, the MSE of an unbiased (w.r.t. θ0 ) MML estimator for a parameter vector θ is lower bounded by:   0 0 0 0 0 0 1 1 1 0 0 1 ˆ ( ), ( ) T MML HB M     = = θ θ θ θ θ θ M θ x θ T K T A B A θ   0 0 0 0 1 1 ( ) 1 ˆ ( ), T MML HB M     θ θ θ θ M θ x θ A B A rr     0  r θ θ 
26. 26 Lower Bound for the MML Estimator We refer to the right-hand side of the inequality as the Huber bound, that we denote by HB( ), that is a function of the true parameter vector, and that simplifies to the classical form HB(θ0 ) when the MML is consistent, i.e. r=0:     0 0 0 1 1 1 ˆ ( ), T MML HB M     θ θ θ θ M θ x θ A B A rr     This MCRB is valid only for unbiased (w.r.t. θ0 ) MML estimators. We now illustrate the meaning of the MCRB and its relationship with the CRB through some numerical examples. θ
27. 27 Toy example: Estimation of the variance of Gaussian data Problem: we want to estimate the variance of a Gaussian pdf in the presence of misspecified mean value, e.g. we erroneously assume that the mean value is zero. 2 True data pdf: ( ) ( , ), 1,2, , ; X i X X p x i M IID    =   Assumed data pdf: ( ; ) ( , ), 1,2, , ; X i f x i M IID     =   2 1 2 1 1 ˆ ( ) argmax ( ; ) ( ) 1 ˆ ˆ ( ) ( ) ( ) M MML X i k M MML ML i X k f x M x M        = = = =   =    x x x x 2 , , (misspecified model) X X     = =  θ θ
28. 28 In this case, the KL divergence between p and fθ is given by: 2 2 2 ( ) 1 ( ) 1 ln 2 2 X X X D p f             =            By taking the derivative with respect to θ and by setting the resulting expression equal to 0: 2 2 0 ( ) 0 ( ) X X D p f       =  =    2 2 0 0 ( ) X X       =  =  =   r θ θ 0 0 the MML is not consistent       r Toy example: Estimation of the variance of Gaussian data
29. 29 According to our definition, the MML is unbiased w.r.t. 0 , in fact: Since the MML estimator is unbiased  MCRB=HB   2 2 2 0 1 1 ˆ ( ) ( ) ( ) M p MML p i X X k E E x M         =   = =  =   =      x We derived the Huber covariance. Matrices Aθ and Bθ (calculated in θ0 ) are scalars:   0 2 2 2 0 ln 1 2 i p f x A E       =    = =           0 0 4 2 2 4 2 0 4 0 ln ln 3 6 ( ) ( ) 4 i i p X X X X f x f x B E                =     =             = Toy example: Estimation of the variance of Gaussian data
30. 30         0 0 0 4 2 2 1 1 4 4 2 2 2 1 2 4 ( ) ( ) 2 T X X X X X X X X HB M M M MCRB HB CRB M              =  =     =  = θ θ θ θ A B A rr Finally, we get the MCRB as: The Huber bound is always greater or equal to the CRB! If we correctly specify the mean value (no mismodeling), then the HB and the CRB coincide:     4 2 2 2 0 2 X X X X X HB CRB M        =  =  = = Toy example: Estimation of the variance of Gaussian data
31. 31 0 5 10 15 20 20 40 60 80 100 RMSE HB CRLB RMSE, Root of the HB and of the CRLB Number of available data: K 0 50 100 150 200 -10 -5 0 5 10 RMSE HB CRLB RMSE, Root of the HB and of the CRLB Assumed mean value: m 2 3, 4, 10 X X M   = = =  M The Huber bound is always greater or equal to the CRB! The MSE of the MML estimator (red curve) coincides with the HB, even for a small data size (small M). Hence, the MCRB is a tight bound for the MML estimator, whereas the CRB is not. 2 3, 4, 0 X X    = = = 3 X   = = Toy example: Estimation of the variance of Gaussian data
32. 32 Problem: we want to estimate the scatter matrix (shape matrix) S of a Complex Elliptically Symmetric (CES) distributed random vector in the presence of misspecified model.         1 1 , True data pdf: ( ) , , , 1, , ; ( ) , X i N H X i N g i i p CE g i M IID p c g with    = =   = x 0 Σ x Σ x γ Σ x γ γ 0    Assumed data pdf: ( ; ) ( , , ), , X i N f CE h IID vecs  x θ 0 Σ θ Σ  true density generator, assumed density generator g h = =   ( 1) 2 1 . : , , M N N N N N i i Assumption   =     x Σ Σ θ θ    MCRB for the Estimation of the Scatter Matrix for CES Data 32 and     rank rank ( ) N full rank = = Σ Σ
33. 33 Example 1: 2 Assumed data pdf: Multivariate Complex Gaussian (CG) ( ; ) ( , , ), , ( ) exp X i N q f CE h IID h q     =      x θ 0 Σ     True data pdf: Multivariate Complex -student ( ) , , , ( ) X i N N t p CE g IID g q q l l h      =      x 0 Σ MCRB for the Estimation of the Scatter Matrix for CES Data
34. 34   1 2 2 1 ( ; ) exp , 1, , H i i X i N f i M      =  =     x Σ x x θ Σ  Assumed data pdf: Multivariate Complex Gaussian (CG) The MML estimator is given by the well−known Sample Covariance Matrix (SCM): 2 2 1 1 1 1 ˆ ˆ = M H MML MML i i i M   =  =   Σ M x x Here, we assume that the clutter power is known, otherwise it can be estimated as   2 ˆ ˆ MML MML tr N  = M MCRB for the Estimation of the Scatter Matrix for CES Data
35. 35 To derive the MCRB the first step is to find 0 , i.e. the matrix that minimizes the KL divergence. We found that, whatever is g(q), the density generator of the true pdf, the gradient of the KL divergence is given by:   2 1 1 1 2 0 X D p f       =  = θ Σ Σ ΣΣ     2 2 0 0 0 2 2 X X vecs vecs     =  = Σ Σ θ Σ Σ  Hence, the MML estimator converges a.s. to a scaled version of the true scatter matrix [ This is not a problem if the estimate has to be plugged in some adaptive radar detectors, such as e.g. the ANMF ]. MCRB for the Estimation of the Scatter Matrix for CES Data
36. 36 The MML estimator is unbiased w.r.t. 0 : 2 2 X   =   2 0 2 2 1 1 1 ˆ M H X p MML p i i i E E M    =   = =  = =      θ μ Σ x x Σ Σ Let us derive numerically the MCRB under the assumption that , so that the MML estimator is also consistent. 2 2 X   =       0 ˆ p MML E MCRB HB = =  = θ μ Σ Σ Σ Σ The MML estimator is consistent only if MCRB for the Estimation of the Scatter Matrix for CES Data
37. 37 True data pdf: multivariate complex t-Student       1 1 ( ) N H X i i i N N p l l l l l l h h           =            x x Σ x Σ l is the shape parameter and h is the scale parameter characterizing the model. The clutter power is given by: The complex Gaussian pdf is a particular case of the complex t- Student that is obtained when l  . The lower is l and the spikier the clutter (heavier pdf tails). 2 ( 1) X l  h l =  MCRB for the Estimation of the Scatter Matrix for CES Data
38. 38 The Huber bound in compact form:                   † † 0 1 † 2 1 1 1 HB HB vec vec . 2 2 ( T T N N T N N N N N N is the Moore Penrose pseudoinverse of uplication matrix of order N The duplication matrix is implicitly defined as the u M is the d N N n ue N iq l l l     = =           θ θ D Σ Σ Σ Σ D D D D D D D  : 1) 2 vecs( ) vec( ) N that satisfies the following equality for any symmetric matri mat i x r x  = A D A A MCRB for the Estimation of the Scatter Matrix for CES Data
39. 39 In [Gre13] we derived the Fisher Information Matrix (FIM) and the CRB for CES distributed random vectors. The CRB for multivariate complex t-distributed data can be expressed as: We also proved that: We now show some numerical results. For the sake of comparison, in the following figures we report, along with the RMSE of the MML, the HB and the CRB, and the RMSE of the robust Tyler’s estimator [Tyl87] (also called Approximate ML [Gin02] or Fixed Point (FP) estimator [Pas08]).         † † 1 1 1 CRB vec vec ( ) T T N N N N M N N l l l l l       =         θ D Σ Σ Σ Σ D MCRB for the Estimation of the Scatter Matrix for CES Data     HB CRB  θ θ
40. 40 Tyler’s estimator belongs to the class of M-estimators [Mar76] and has been derived in the context of the CES distribution as a robust estimator [Tyl87]. It can be obtained as the recursive solution of the following fixed point matrix equation:   1 1 (0) ( 1) 1 ( ) 1 ˆ ˆ ˆ ˆ , 0,1, , ˆ H M i i H i i i MML SCM H M k i i it H k i i i N M N k N M  =   = =  = =   = =     x x Σ x Σ x Σ Σ Σ x x Σ x Σ x  MCRB for the Estimation of the Scatter Matrix for CES Data
41. 41            2 , 1,2 4 HB CRB , , : 1 ( 1) 10 , 3, 0.9, 8, 3 , 3 ˆ ˆ ˆ ˆ , vecs( ), tr HB CRB , i j X i j it T p T F F F F F F F MC runs N M N N E l r r h  h l l r     = = = =          = = = = = =   = = Σ Σ θ θ θ θ θ Σ A A A Σ θ θ Σ Σ    MCRB for the Estimation of the Scatter Matrix for CES Data
42. 0,2 0,22 0,24 0,26 0,28 0,3 10 20 30 40 50 MML (SCM) HB CRLB Tyler RMSE, root of the CRLB and of the HB Shape parameter of the t-distribution:  42 1,2     Σ For large values of the shape parameter l, HB and CRB tend to be equal, because when l  the t-student pdf tends to the complex Gaussian pdf, then the Gaussian assumption becomes correct (no mismodeling). Tyler’s estimator is robust but not efficient, it is not the ML estimator for any CES, but it performs better than the MML for l <10. 1 1 0,15 0,2 0,25 0,3 10 20 30 40 50 MML (SCM) HB CRLB Tyler Normalized Frobenius norm Shape parameter of the t-distribution:  MCRB for the Estimation of the Scatter Matrix for CES Data
43. 0,05 0,06 0,07 0,08 0,09 0,1 0,2 40 80 120 160 200 240 280 MML (SCM) HB CRLB Tyler RMSE, root of the CRLB and of the HB Number of available data: M 43 1,2     Σ As expected, the MML estimator (that in this case is the SCM), does not achieve the CRB but rather the HB. In heavy-tailed clutter (l =3), Tyler’s estimator has performance sub-optimal but better then the MML (SCM). 0,01 0,1 1 40 80 120 160 200 240 280 MML (SCM) HB CRLB Tyler Normalized Frobenius norm Number of available data: M MCRB for the Estimation of the Scatter Matrix for CES Data
44. 0,1 0,15 0,2 0,25 0,3 0,35 0,4 0,2 0,4 0,6 0,8 1 MML (SCM) HB CRLB Tyler RMSE, root of the CRLB and of the HB  44 1,2     Σ In heavy-tailed clutter (l =3), due to its robustness, Tyler’s estimator achieves better estimation performance than the MML for all values of r. When r increases, the performance of the MML gets closer to the HB, whereas the performance of the Tyler-estimator gets closer to the CRB (but does not achieve it). 0,1 0,2 0,3 0,4 0,5 0,6 0,2 0,4 0,6 0,8 1 MML (SCM) HB CRLB Tyler Normalized Frobenius norm  MCRB for the Estimation of the Scatter Matrix for CES Data
45. 45 Example 2:   Assumed data pdf: Multivariate Complex Generalized Gaussian (MGG) ( ; ) , , , , ( ) exp X i N q f CE h IID h q b b    =      x θ 0 Σ     True data pdf: Multivariate Complex -student ( ) , , , , ( ) N X i N t p CE g IID g q q l l h      =      x 0 Σ MCRB for the Estimation of the Scatter Matrix for CES Data
46. Assumed data pdf: Multivariate Generalized Gaussian (MGG)       1 1 ( ; ) exp H N i i X i N N b f N b b b b  b        =        x Σ x x θ Σ b is the shape parameter and b is the scale parameter characterizing the model. The complex Gaussian pdf is a particular case of the MGG that is obtained when b =1. The distribution has heavy tails when 0< b <1. The MML estimator (i.e. the ML estimator for MGG data) is the solution of the following fixed−point (FP) matrix equation [Oli12], [Gre13]: MCRB for the Estimation of the Scatter Matrix for CES Data       1 1 1 (0) 1 ( 1) ( ) 1 1 , ( ) ˆ ˆ 1 ˆ ˆ , 0,1, , M H H i i i i i SCM M k H k H i i i i it i t t M b k N M b b      =   = = =  =   = =     Σ x Σ x x x Σ Σ Σ x Σ x x x 
47. 47 It can be proved that the iteration converges to the MML estimate if and only if 0<b<1: When M goes to infinity, the MML estimator converges a.s. to the matrix that minimizes the KL divergence: Then, we found that the MML estimator (for large M) is unbiased: Hence, MCRB=HB. ( ) ˆ ˆ i 0 1 l m k MML k iff b   =  Σ Σ   0 0 D p f  =  =  = Σ Σ Σ 0 Σ Σ   0 ˆ MML E  = = = μ Σ Σ Σ MCRB for the Estimation of the Scatter Matrix for CES Data
48. 48            4 , HB CRB : 50 : 3 ( ) 10 , , 0.9, 8, 3 24 ˆ ˆ ˆ ˆ , vecs( ), tr HB CRB , i j i j T p T F F F F F F F Quasi Gaussian scenario Super Gaussian scenario heavy tailed clutter MC runs N M N E l l r r      =  =  = = = = = =       = = Σ θ θ θ θ θ Σ A A A Σ θ θ Σ Σ    MCRB for the Estimation of the Scatter Matrix for CES Data
49. 49 : 50 Quasi Gaussian scenario l  = When l=50, the true t-distribution is pretty close to the Gaussian distribution, so the performance of the MML estimator improves as β gets closer to 1, the reverse when l=3. The CRB and the MSE of Tyler’s estimator do not depend on β. : 3 Super Gaussian scenario l  = 0,1 1 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 MML HB CRLB Tyler Normalized Frobenius norm Shape parameter of the GG distribution:  0,1 1 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 MML HB CRLB Tyler Normalized Frobenius norm Shape parameter of the GG distribution:  MCRB for the Estimation of the Scatter Matrix for CES Data
50. 0,01 0,1 1 40 80 120 160 200 240 280 MML HB CRLB Tyler Normalized Frobenius norm Number of available data: M 0,01 0,1 1 40 80 120 160 200 240 280 MML HB CRLB Tyler Normalized Frobenius norm Number of available data: M 50 : 50 Quasi Gaussian scenario l  = 0.1 b = 0.8 b = When β=0.1 (i.e. we assume heavy-tailed GG clutter), the performance of MML and Tyler’s estimators are pretty close and the HB is slightly tighter than the CRB. When β=0.8 (i.e. we assume quasi-Gaussian clutter), HBCRB and the MML performs slightly better than the Tyler estimator. MCRB for the Estimation of the Scatter Matrix for CES Data
51. 0,01 0,1 1 0,2 0,4 0,6 0,8 1 MML HB CRLB Tyler RMSE, root of the CRLB and of the HB  0,01 0,1 1 0,2 0,4 0,6 0,8 1 MML HB CRLB Tyler RMSE, root of the CRLB and of the HB  51 : 50 Quasi Gaussian scenario l  = 0.1 b = 0.8 b = In both cases, the MSE, the HB, and the CRB get worse when the clutter one-lag correlation coefficient increases. When β=0.8, the MML has the same performance as Tyler’s estimator (up to very large r) and HBCRB. When β=0.1, the HB is slightly tighter than the CRB. MCRB for the Estimation of the Scatter Matrix for CES Data
52. 0,01 0,1 1 40 80 120 160 200 240 280 MML HB CRLB Tyler Normalized Frobenius norm Number of available data: M 0,01 0,1 1 40 80 120 160 200 240 280 MML HB CRLB Tyler Normalized Frobenius norm Number of available data: M 52 : 3 Super Gaussian scenario l  = 0.1 b = 0.8 b = When β=0.1 (i.e. we assume heavy-tailed GG clutter), HBCRB and the MML performs better than Tyler’s estimator. When β=0.8 (i.e. we assume quasi-Gaussian clutter), the HB is a tighter bound than the CRB for the performance of the MML. MCRB for the Estimation of the Scatter Matrix for CES Data
53. 0,01 0,1 1 0,2 0,4 0,6 0,8 1 MML HB CRLB Tyler RMSE, root of the CRLB and of the HB  0,01 0,1 1 0,2 0,4 0,6 0,8 1 MML HB CRLB Tyler RMSE, root of the CRLB and of the HB  53 : 3 Super Gaussian scenario l  = 0.1 b = 0.8 b = In both cases, the MSE, the HB, and the CRB get worse when the clutter one-lag correlation coefficient increases. Clearly, in the super-Gaussian scenario (l=3) the effect of mismatching is more evident when we assume b=0.8 (almost-Gaussian GG clutter) than when we assume b=0.1 (spiky GG clutter). MCRB for the Estimation of the Scatter Matrix for CES Data
54. 54 of 28 Conclusions Vector version of Misspecified Cramér-Rao bound (MCRB) MCRB for Mismatched Maximum Likelihood (MML) estimators. When the MML is unbiased it coincides with the Huber bound Numerical examples, related to the problem of estimating the scatter matrix of a CES data vector under mismodeling. We are now investigating the effects of mismodeling in terms of detector CFAR property and detection performance.       0 0 0 1 1 1 0 1 1 ˆ , HB CRB T MML M M     =   =  θ θ θ θ M θ θ θ A B A rr θ F r θ θ 
55. 55 References [Boy04] S. Boyd, L. Vandenberghe. Convex Optimization. Cambridge University Press, first edition, 2004. [Fom03] T. B. Fomby and R. C. Hill, Eds., Maximum-Likelihood Estimation of Misspecified Models: Twenty Years Later. Kidington, Oxford, UK: Elsevier Ltd, 2003. [Gin02] F. Gini, M. S. Greco, “Covariance matrix estimation for CFAR detection in correlated heavy tailed clutter,” Signal Processing, vol. 82, no. 12, pp. 1847-1859, December 2002. [Gre13] M. S. Greco, F. Gini, “Cramér-Rao Lower Bounds on Covariance Matrix Estimation for Complex Elliptically Symmetric Distributions,” IEEE Trans. on Signal Proc., vol. 61, no. 24, pp. 6401-6409, Dec. 2013. [Hub67] P. J. Huber, “The behavior of maximum likelihood estimates under nonstandard conditions,” Proceedings of the Fifth Berkley Symposium on Mathematical Statistics and Probability, 1967, pp. 221–233. [Mar76] R. A. Maronna, “Robust M-estimators of multivariate location and scatter,” The Annals of Statistics, vol. 4, no. 1, pp. 51-67, January 1976. [McW93] L. T. McWhorter, L. L. Scharf, “Properties of quadratic covariance bounds,” The Twenty-Seventh Asilomar Conference on Signals, Systems and Computers, 1993. , pp.1176 – 1180, vol. 2, 1-3 Nov 1993[Pas08] F. Pascal, Y. Chitour, J. Ovarlez, P. Forster, P. Larzabal, “Covariance Structure MLE in Compound Gaussian Noise: Existence and Algorithm Analysis,” IEEE Trans. on Signal Processing, vol.56, no.1, pp. 34,48, Jan. 2008. [Oli12] E. Ollila, D.E. Tyler, V. Koivunen, V.H. Poor, “Complex Elliptically Symmetric Distributions: Survey, New Results and Applications,” IEEE Trans. on Signal Proc., Vol. 60, No. 11, pp.5597-5625, 2012. [Ric13] C. D. Richmond, L. L. Horowitz, “Parameter bounds under misspecified models,” Conference on Signals, Systems and Computers, 2013 Asilomar, pp.176-180, 3-6 Nov. 2013. [San12] K. J. Sangston, F. Gini, M. Greco, “Coherent radar detection in heavy-tailed compound-Gaussian clutter”, IEEE Trans. on Aerospace and Electronic Systems, Vol. 42, No.1, pp. 64-77, 2012. [Tyl87] D. Tyler, “A distribution-free M-estimator of multivariate scatter,” The Annals of Statistics, vol. 15, no. 1, pp. 234-251, January 1987. [Whi81] H. White, “Consequences and detection of misspecified nonlinear regression models,” Journal of the American Statistical Association, vol. 76, pp. 419-433, 1981.
56. Acknowledgements Fulvio Gini, University of Pisa, Italy Stefano Fortunati, University of Pisa, Italy 56