Maria S. Greco

Centrale Supélec – June 2015
MODELING AND MISMODELING IN
RADAR APPLICATIONS:
PARAMETER ESTIMATION AND BOUNDS
Maria S. Greco
Dipartimento di Ingegneria dell’Informazione
University of Pisa (Italy)

View Slide

Outline of the talk
2
The radar scenario: covariance matrix
estimation in non-Gaussian clutter
The Misspecified Cramér-Rao Bound
(MCRB) and the Huber Bound
Matrix estimation for CES distributed
disturbance
Examples and conclusions

View Slide

3
• Radar systems detect
targets by examining
reflected energy, or returns,
from objects
• Along with target echoes,
returns come from the sea
surface, land masses,
buildings, rainstorms, and
other sources. This
disturbance is called clutter
Radar scenario
•Much of this clutter is far stronger than signals received from targets of
interest
• The main challenge to radar systems is discriminating these weaker target
echoes from clutter
• Coherent signal processing techniques are used to this end

View Slide

Radar scenario: the clutter
The function of the clutter model is to define a consistent theory
whereby a physical model results in an analytical model which can be
used for radar design and performance analysis.
 Radar clutter is defined as unwanted echoes, typically from the
ground, sea, rain or other atmospheric phenomena.
 These unwanted returns may affect the radar performance and can
even obscure the target of interest.
 Hence clutter returns must be taken into account in designing a
radar system.
4

View Slide

•In early studies, the resolution capabilities of radar systems were relatively
low, and the scattered return from clutter was thought to comprise a large
number of scatterers
•From the Central Limit Theorem (CLT), researchers in the field were led to
conclude that the appropriate statistical model for clutter was the Gaussian
model (for the I & Q components), i.e., the amplitude R is Rayleigh
distributed)
•In the quest for better performance, the resolution capabilities of radar
systems have been improved
•For detection performance, the belief originally was that a higher resolution
radar system would intercept less clutter than a lower resolution system,
thereby increasing detection performance
• However, as resolution has increased, the clutter statistics have no longer
been observed to be Gaussian, and the detection performance has not
improved directly. The radar system is now plagued by target-like “spikes”
that give rise to non-Gaussian observations
5

View Slide

• These spikes are passed by the detector as targets at a much higher false
alarm rate (FAR) than the system is designed to tolerate
• The reason for the poor performance can be traced to the fact that the
traditional radar detector is designed to operate against Gaussian noise
• New clutter models and new detection strategies are required to reduce the
effects of the spikes and to improve detection performance
Spikes in the horizontally polarized
sea clutter data Spikes in the vertically polarized sea
clutter data
6

View Slide

Empirical studies have produced several candidate models for spiky non-
Gaussian clutter, the most popular being the Weibull distribution, the K
distribution, the log-normal, the generalized K, the Student-t (or Pareto), etc.
Measured sea clutter data
(IPIX database)
the Weibull , K,
log-normal etc. have
heavier tails than the
Rayleigh
10-3
10-2
10-1
100
101
102
0 0.2 0.4 0.6 0.8 1
Weibull
Lognormal
K
GK
LNt
IG
Histogram
pdf
Amplitude (V)
7

View Slide

8
In general, taking into account the variability of the local power , that
can be modeled itself as a random variable, we obtain the so-called
compound-Gaussian model, very popular in clutter modeling, where
2
2
( | ) exp ( )
r r
p r u r

 
 
= 
 
 
0
( ) ( | ) ( ) ; 0
p r p r p d r
  

=   

According to the CG model:
( ) ( ) ( )
z n n x n

=
( ) ( ) ( )
I Q
x n x n jx n
= 
Speckle: complex Gaussian
process, takes into account the
local backscattering
Texture: non negative random
process, takes into account the
local mean power
8
Radar scenario: compound-Gaussian model

View Slide

Particular cases of CG model (amplitude PDF):
 
 
1
1
4 4 4
( )
2
R
p r r K r u r



   
  


   
=    
    
K (Gamma texture)
 
2
2
0
2
( ) exp
b b
b
R
br r
p r d


 
  
   


 
   
=  
 
   
    
 
 

GK (Generalized
Gamma texture)
t-Student
or Pareto
 
1
( ) exp ( )
c
c
R
c r
p r r b u r
b b

 
 
= 
   
 
W, Weibull
9
( 1)
2
( ) 2 1 ( )
R
p r r r u r
l
h
h
l
 
 
= 
 
 
Compound-Gaussian model and CES
The CG model belongs to the family of Complex Elliptically
Symmetric (CES) distributions

View Slide

10
Any radar detector should be designed to operate in an unknown
interference environment, i.e. it should adaptively estimate the
disturbance covariance matrix R=2S (i.e. power 2 and scatter matrix S).
Detectors that estimate adaptively (on-line) the disturbance covariance
matrix are named “adaptive detectors”.
When the interference environment is a-priori unknown, several
approaches are possible. The most commonly used are:
1) Assume that the disturbance is white, i.e. R=2I, and implement the
non-adaptive detector which is optimal for that scenario (clearly, it will
perform suboptimally in correlated disturbance).
2) Model the clutter as an autoregressive (AR) process and estimate the
clutter covariance matrix by estimating the AR parameters [see e.g. S.
Haykin and A. Steinhardt. Adaptive Radar Detection and Estimate. John
Wiley and Sons, 1992].
Radar scenario: adaptive detection

View Slide

3) Assume that a set of secondary data xi
=di
, i=1,…,M is available, free of
target signal components, sharing the same correlation characteristics
as the disturbance in the Cell Under Test (CUT), then estimate 2 and S
from M>N secondary data vectors.
This scenario is usually referred to as homogeneous environment
and this is the case we treat here.
Secondary data are usually obtained by processing range gates in
spatial proximity with the CUT. The data from the CUT are termed
primary data.
Problem: we want to estimate the scatter matrix (shape matrix)
S of Complex Elliptically Symmetric (CES) distributed clutter for
adaptive radar detection
11
Radar scenario: adaptive detection

View Slide

12
Radar scenario: disturbance matrix estimation
A fundamental assumption underlying most estimation problems is
that the true data model and the model assumed to derive an
estimation algorithm are the same, that is, the model is correctly
specified. If the unknown parameters are deterministic, a typical
estimator is the Maximum Likelihood one and its performance
asymptotically tends to the Cramér-Rao Lower Bound (CRLB).
But for some non-perfect knowledge of the true data model or for
operative constraints on the estimation algorithm (i.e. easy of
implementation, computational cost, very short operational time)
there can be a mismatch between assumed and true data model. In
this case, which is the bound on the performance of ML and other
estimators?
Problem: we want to estimate the scatter matrix S of Complex
Elliptically Symmetric (CES) distributed clutter in presence of
model mismatch

View Slide

Parameter estimation and bounds
Most treatments of parameter bounds assume perfect
knowledge of the data distribution.
When the assumed distribution for the measured data differs from
the true distribution, the model is said to be misspecified.
While much attention has been devoted to understanding the
performance of Maximum Likelihood (ML) and Least Squares (LS)
estimators under model misspecification (see e.g. [Hub67], [Whi81],
[Fom03]), little consideration appears to have been given to the
concomitant bounds on performance.
3

View Slide

14
Parameter bounds under misspecified models
Problem: Given a parameter vector , we are interested in finding a
lower bound on the mean square error (MSE) of any estimator, based
on the assumed probability density function (pdf) fX
(x; ) in the
presence of misspecified data model  misspecified bound (MCRB)
Choice of score function  The tightest bounds are obtained via
a choice of score function with two properties [McW93]:
(1) zero mean;
(2) dependence on the sufficient statistic T(x) for estimating .
( ) true pdf of , ( ; ) assumed pdf, ( ) ( ; )
X X X X
p f p f
= = 
x x x θ x x θ

View Slide

15
First, we define a score function by subtracting to the score function
used to derive the classical CRLB its possibly non-zero mean value :
 
( ) ln ( ; ) ln ( ; )
X p X
f E f
  
θ θ θ
s x x θ x θ

  ( )
ln ln ( )
( ; )
X
p X
X
p p
D p f E p d
f f


 
   
=
 
   
 
 
 
 x
x x
x θ
 
where we expressed the 2nd term as a function of the Kullback-
Leibler (KL) divergence between the assumed density and the true
density :
  0 ( ; ) ( )
X X
D p f f p

=  =
x θ x

 
ln ( ; )
X
f D p f
=   
θ θ θ
x θ

View Slide

16
The MSE on the estimation of the elements of the true parameter
vector is given by the diagonal entries of the following matrix :
    
 
   
   
ˆ ˆ ˆ
( ), ( ) ( )
true parameter vector
ˆ ˆ
( ) ( ) bias vector
ˆ
( ) ( )
ˆ
covariance matrix of ( )
ˆ ˆ
( ) ( ) ( ) ( )
T
T
p
p p
T T T
p p
E
E E
E E
  = 
=
  = 

=
= 
θ θ θ
θ θ θ
θ
θ
θ θ θ
M θ x θ θ x θ θ x θ C b b
θ
μ θ x b θ θ x θ μ
e x θ x μ
C θ x
C e x e x θ x θ x μ μ

 



View Slide

17
The covariance inequality we are looking for can be obtained as
follows:
Ω is a Grammian matrix  it is positive semi-definite.
If Ω is symmetric and K
is invertible, the following properties hold
[Boy04]:
 
1
( )
( ) ( ) 0
( )
(Shur complement of in )
T T
p T
T
E

 
   
= 
 
   
   
 

θ θ
θ
θ θ θ
θ θ θ θ θ
e x C T
Ω e x s x
s x T K
S C T K T C Ω


1
0 0 T

    
θ θ θ θ
Ω S C T K T

View Slide

18
    1
ˆ( ), , : T T
MCRB p f 
 = 
θ θ θ θ θ
M θ x θ θ T K T b b
   
ln ( ; )
p X
E f D p f
  = 
θ θ θ θ
d x θ

The right–hand side term is the lower bound on the MSE of any
estimator in the presence of misspecified data model: the
misspecified bound (MCRB).
T
is the so-called expansion coefficient (EC) matrix [McW93]:
     
 
 
 
ˆ
( ) ( ) ( ) ln ( ; )
ˆ( ) ln ( ; )
T
T
p p X
T T
p X
E E f D p f
E f
=    
=  
θ θ θ θ θ θ
θ θ θ
T e x s x θ x μ x θ
θ x x θ μ d


View Slide

19
   
  
 
 
 
( ) ( ) ln ( ; )
ln ( ; ) ln ( ; )
T
T
p p X
T T
p X X
E E f D p f
E f f
=   
=   
θ θ θ θ θ θ
θ θ θ θ
K s x s x x θ
x θ x θ d d
 
K
is the information matrix corresponding to the given score
function:
   
ln ( ; )
p X
E f D p f
  = 
θ θ θ θ
d x θ

This is the version for vector parameter of the result that was
derived in a different way for the scalar case by Richmond [Ric13].
    1
ˆ( ), , : T T
MCRB p f 
 = 
θ θ θ θ θ

View Slide

20
Lower Bound for the MML Estimator
Consider the Mismatched Maximum Likelihood (MML) estimator:
ˆ ( ) argmax ( ; ), ( ; ) ( ) true pdf
MML X X X
f f p
= 
θ
θ x x θ x θ x
In [Hub67], Huber proved that under some assumptions, the MML
converges almost surely (a.s.) to the value 0
that minimizes the KL
divergence:
 
1
1
( ; ) ( ; )
M
M
k X X k
k
k
IID f f
=
=
 = 
x x θ x θ
 
   
 
. .
0
ˆ ( ) argmin argmin ln ( ; )
a s
MML p X
M
D p f E f
  
 = = 
θ
θ θ
θ z θ x θ

View Slide

21
Additionally, Huber [Hub67] and White [Whi82] proved the
consistency and asymptotic normality of the MML under some
mild regularity conditions:
   
0 0 0
1 1
0
ˆ ( ) ,
MML
M
M  

 θ θ θ
θ z θ 0 A B A
 
 
2
,
ln ( ; )
X k
p
i j
i j
f
E
 
 

 
=  
 
 
 
θ
x θ
A
 
,
ln ( ; ) ln ( ; )
X k X k
p
i j
i j
f f
E
 
 
 
 
= 
 
 
 
 
θ
x θ x θ
B
0
( ) 1
HB for M =
θ
where:

View Slide

22
If the model is correctly specified:  
; ( ) true pdf
X X
f p
=
x θ x
2
ln ( ) ln ( ) ln ( )
,
0
X k X k X k
p p
i j i j
p p p
E E i j
   
   
  
   
 =  
   
   
   
   
 =    =
θ θ θ θ
x x x
A B A B
     
1
MCRB HB CRB

= = =
θ
θ θ B θ
 
; ( ), 1,2 ,
X k X k
f p k M
 = =
x θ x 

View Slide

23
Let’s express it in terms of 0
:
The general MCRB we derived:
    
 
  
 
 
0
0 0 0 0
0
ˆ ˆ ˆ
( ), ( ) ( )
ˆ ˆ
( ) ( )
ˆ( ), 2
T
p
T
p
T T
E
E
=  
=      
=  
θ
M θ x θ θ x θ θ x θ
θ x θ θ θ θ x θ θ θ
M θ x θ b r rr
    1
ˆ( ), , : T T
MCRB p f 
 = 
θ θ θ θ θ
 
0
0 0
ˆ
where ( ) ,
p
E
 
θ
b θ θ x r θ θ
 
 
0
( 0)
D p f
=
 =
θ θ θ θ
 
0 0 0 0 0 0
1
ˆ( ), 2
T T T T

   
θ θ θ θ θ θ
M θ x θ T K T b b b r rr
Then:

View Slide

We have to see how the MCRB specializes when: ˆ ˆ
( ) ( )
MML
=
θ x θ x
 
ˆ ( ), ?
MML

M θ x θ
Assume first that the MML estimator is unbiased w.r.t. 0
:
  0
ˆ ( )
p MML
E
= =
θ
μ θ x θ
This is always true asymptotically [Hub67] and approximately for
large M [Ric13]. Then, we define as consistent a mismatched
estimator if, as the number of observations M goes to infinity, it
converges in probability to the true parameter vector:
24
 
0
0 0
ˆ ( ) 0
p MML
E
  =  =
θ θ
b θ θ x θ μ

.
0 0
ˆ ( ) 0
prob
MML
M 
  =  =  =
θ x θ θ θ r θ θ

View Slide

25
Hence, our covariance inequality, evaluated in θ0
, becomes:
In conclusion, the MSE of an unbiased (w.r.t. θ0
) MML estimator for
a parameter vector θ is lower bounded by:
 
0 0 0 0 0 0
1 1 1
0 0
1
ˆ ( ), ( )
T
MML
HB
M
  
 = =
θ θ θ θ θ θ
M θ x θ T K T A B A θ
 
0 0 0
0
1 1
( )
1
ˆ ( ), T
MML
HB
M
 
 
θ θ θ
θ
M θ x θ A B A rr



 0

r θ θ


View Slide

26
We refer to the right-hand side of the inequality as the Huber
bound, that we denote by HB( ), that is a function of the true
parameter vector, and that simplifies to the classical form HB(θ0
)
when the MML is consistent, i.e. r=0:
 
 
0 0 0
1 1
1
ˆ ( ), T
MML
HB
M
 
 
θ θ θ
θ
M θ x θ A B A rr




This MCRB is valid only for unbiased (w.r.t. θ0
) MML estimators.
We now illustrate the meaning of the MCRB and its relationship
with the CRB through some numerical examples.
θ

View Slide

27
Toy example: Estimation of the variance of Gaussian data
Problem: we want to estimate the variance of a Gaussian pdf in the
presence of misspecified mean value, e.g. we erroneously assume that
the mean value is zero.
2
True data pdf: ( ) ( , ), 1,2, , ;
X i X X
p x i M IID
 
 = 

Assumed data pdf: ( ; ) ( , ), 1,2, , ;
X i
f x i M IID
  
 = 

2
1
2
1
1
ˆ ( ) argmax ( ; ) ( )
1
ˆ ˆ
( ) ( ) ( )
M
MML X i
k
M
MML ML i X
k
f x
M
x
M

  
  
=
=
= = 
 = 


x x
x x
2 , , (misspecified model)
X X
   
= = 
θ θ

View Slide

28
In this case, the KL divergence between p and fθ
is given by:
2 2 2
( ) 1
( ) 1 ln
2 2
X X X
D p f
   
  
 
 

=   
 
 
 
 
By taking the derivative with respect to θ and by setting the
resulting expression equal to 0:
2 2
0
( )
0 ( )
X X
D p f    


=  =  

2 2
0 0
( )
X X
    
 =  =  =  
r θ θ
0
0 the MML is not consistent
 
   
r

View Slide

29
According to our definition, the MML is unbiased w.r.t. 0
, in fact:
Since the MML estimator is unbiased  MCRB=HB
  2 2 2
0
1
1
ˆ ( ) ( ) ( )
M
p MML p i X X
k
E E x
M

      
=
 
= =  =   =
 
 

x
We derived the Huber covariance. Matrices Aθ
and Bθ
(calculated
in θ0
) are scalars:
 
0
2
2 2
0
ln 1
2
i
p
f x
A E 

 
 
=
 

= = 
 

 
   
0
0
4 2 2 4 2
0
4
0
ln ln
3 6 ( ) ( )
4
i i
p
X X X X
f x f x
B E  

 
 
      

=
 
 
= 
 
 
 
    
=

View Slide

30
 
     
0 0 0
4 2 2
1 1 4
4
2 2 2
1 2 4 ( )
( )
2
T X X X
X
X
X X X
HB
M M M
MCRB HB CRB
M
   
 

  
 

=  =   
 =  =
θ θ θ
θ A B A rr
Finally, we get the MCRB as:
The Huber bound is always greater or equal to the CRB!
If we correctly specify the mean value (no mismodeling), then
the HB and the CRB coincide:
    4
2 2 2
0
2 X
X X X X
HB CRB
M

     
=  =  = =

View Slide

31
0
5
10
15
20
20 40 60 80 100
RMSE
HB
CRLB
RMSE, Root of the HB and of the CRLB
Number of available data: K
0
50
100
150
200
-10 -5 0 5 10
RMSE
HB
CRLB
RMSE, Root of the HB and of the CRLB
Assumed mean value: m
2
3, 4, 10
X X
M
 
= = =

M
The Huber bound is always greater or equal to the CRB!
The MSE of the MML estimator (red curve) coincides with the
HB, even for a small data size (small M). Hence, the MCRB is a tight
bound for the MML estimator, whereas the CRB is not.
2
3, 4, 0
X X
  
= = =
3
X
 
= =

View Slide

32
Problem: we want to estimate the scatter matrix (shape matrix)
S of a Complex Elliptically Symmetric (CES) distributed random
vector in the presence of misspecified model.
 
   
 
1 1
,
True data pdf: ( ) , , , 1, , ;
( ) ,
X i N
H
X i N g i i
p CE g i M IID
p c g with
 
 =
=   =
x 0 Σ
x Σ x γ Σ x γ γ 0

 
Assumed data pdf: ( ; ) ( , , ), ,
X i N
f CE h IID vecs

x θ 0 Σ θ Σ

true density generator, assumed density generator
g h
= =
  ( 1) 2
1
. : , ,
M N N N N N
i i
Assumption  
=
   
x Σ Σ θ θ
  
MCRB for the Estimation of the Scatter Matrix for CES Data
32
and    
rank rank ( )
N full rank
= =
Σ Σ

View Slide

33
Example 1:
2
Assumed data pdf: Multivariate Complex Gaussian (CG)
( ; ) ( , , ), , ( ) exp
X i N
q
f CE h IID h q

 
 = 
 
 
x θ 0 Σ
 
 
True data pdf: Multivariate Complex -student
( ) , , ,
( )
X i N
N
t
p CE g IID
g q q
l
l
h
 

 
= 
 
 
x 0 Σ

View Slide

34
 
1
2
2
1
( ; ) exp , 1, ,
H
i i
X i N
f i M



 
=  =
 
 
x Σ x
x θ
Σ

Assumed data pdf: Multivariate Complex Gaussian (CG)
The MML estimator is given by the well−known Sample Covariance
Matrix (SCM):
2 2
1
1 1 1
ˆ ˆ
=
M
H
MML MML i i
i
M
  =
 =  
Σ M x x
Here, we assume that the clutter power is known, otherwise it can
be estimated as
 
2
ˆ
ˆ MML
MML
tr
N
 =
M

View Slide

35
To derive the MCRB the first step is to find 0
, i.e. the matrix that
minimizes the KL divergence.
We found that, whatever is g(q), the density generator of the true pdf,
the gradient of the KL divergence is given by:
 
2
1 1 1
2
0
X
D p f


  
 =  =
θ
Σ Σ ΣΣ
   
2 2
0 0 0
2 2
X X
vecs vecs
 
 
=  =
Σ Σ θ Σ Σ

Hence, the MML estimator converges a.s. to a scaled version of the
true scatter matrix [ This is not a problem if the estimate has to be
plugged in some adaptive radar detectors, such as e.g. the ANMF ].

View Slide

36
The MML estimator is unbiased w.r.t. 0
:
2 2
X
 
=
  2
0
2 2
1
1 1
ˆ
M
H X
p MML p i i
i
E E
M

 
=
 
= =  = =
 
 

θ
μ Σ x x Σ Σ
Let us derive numerically the MCRB under the assumption that
, so that the MML estimator is also consistent.
2 2
X
 
=
     
0
ˆ
p MML
E MCRB HB
= =  =
θ
μ Σ Σ Σ Σ
The MML estimator is consistent only if

View Slide

37
True data pdf: multivariate complex t-Student
 
 
 
1
1
( )
N
H
X i i i
N
N
p
l l
l l l
l h h

 

     
=  
   
    
x x Σ x
Σ
l is the shape parameter and h is the scale parameter
characterizing the model. The clutter power is given by:
The complex Gaussian pdf is a particular case of the complex t-
Student that is obtained when l  .
The lower is l and the spikier the clutter (heavier pdf tails).
2
( 1)
X
l

h l
=


View Slide

38
The Huber bound in compact form:
   
 
     
 
 
 
† †
0
1
†
2
1
1 1
HB HB vec vec
.
2 2
(
T
T
N N
T
N N N N N
N
is the Moore Penrose pseudoinverse of
uplication matrix of order N The duplication matrix is
implicitly defined as the u
M
is the d
N N
n ue N
iq
l
l l

 

= =  





  
θ θ D Σ Σ Σ Σ D
D D D D D
D

:
1) 2
vecs( ) vec( )
N
that satisfies
the following equality for any symmetric matri
mat i
x
r x

=
A
D A A

View Slide

39
In [Gre13] we derived the Fisher Information Matrix (FIM) and the
CRB for CES distributed random vectors. The CRB for multivariate
complex t-distributed data can be expressed as:
We also proved that:
We now show some numerical results.
For the sake of comparison, in the following figures we report, along
with the RMSE of the MML, the HB and the CRB, and the RMSE of the
robust Tyler’s estimator [Tyl87] (also called Approximate ML
[Gin02] or Fixed Point (FP) estimator [Pas08]).
       
† †
1 1 1
CRB vec vec
( )
T
T
N N
N N
M N N
l l
l l l
 
   
=  
 
 
 
θ D Σ Σ Σ Σ D
   
HB CRB

θ θ

View Slide

40
Tyler’s estimator belongs to the class of M-estimators [Mar76] and
has been derived in the context of the CES distribution as a robust
estimator [Tyl87].
It can be obtained as the recursive solution of the following fixed
point matrix equation:
 
1
1
(0)
( 1)
1
( )
1
ˆ ˆ ˆ
ˆ , 0,1, ,
ˆ
H
M
i i
H
i i i
MML SCM
H
M
k i i
it
H k
i
i i
N
M
N
k N
M

=


=
=
 = =

 = =




x x
Σ
x Σ x
Σ Σ Σ
x x
Σ
x Σ x


View Slide

41
  
 
 
   
2
, 1,2
4
HB CRB
, , : 1
( 1)
10 , 3, 0.9, 8, 3 , 3
ˆ ˆ
ˆ ˆ
, vecs( ), tr
HB CRB
,
i j
X
i j
it
T
p
T
F
F
F
F F
F F
MC runs N M N N
E
l
r r h 
h l
l r

 

= = = =
   
    
= = = = = =
 
= =
Σ Σ
θ θ θ θ
θ Σ A A A
Σ
θ θ
Σ Σ

 

View Slide

0,2
0,22
0,24
0,26
0,28
0,3
10 20 30 40 50
MML (SCM)
HB
CRLB
Tyler
RMSE, root of the CRLB and of the HB
Shape parameter of the t-distribution: 
42
1,2
 
 
Σ
For large values of the shape parameter l, HB and CRB tend to be
equal, because when l  the t-student pdf tends to the complex
Gaussian pdf, then the Gaussian assumption becomes correct (no
mismodeling).
Tyler’s estimator is robust but not efficient, it is not the ML estimator
for any CES, but it performs better than the MML for l <10.
1
1
0,15
0,2
0,25
0,3
10 20 30 40 50
MML (SCM)
HB
CRLB
Tyler
Normalized Frobenius norm
Shape parameter of the t-distribution: 

View Slide

0,05
0,06
0,07
0,08
0,09
0,1
0,2
40 80 120 160 200 240 280
MML (SCM)
HB
CRLB
Tyler
Number of available data: M
43
1,2
 
 
Σ
As expected, the MML estimator (that in this case is the SCM), does
not achieve the CRB but rather the HB.
In heavy-tailed clutter (l =3), Tyler’s estimator has performance
sub-optimal but better then the MML (SCM).
0,01
0,1
1
40 80 120 160 200 240 280
MML (SCM)
HB
CRLB
Tyler

View Slide

0,1
0,15
0,2
0,25
0,3
0,35
0,4
0,2 0,4 0,6 0,8 1
MML (SCM)
HB
CRLB
Tyler

44
1,2
 
 
Σ
In heavy-tailed clutter (l =3), due to its robustness, Tyler’s estimator
achieves better estimation performance than the MML for all values of
r.
When r increases, the performance of the MML gets closer to the HB,
whereas the performance of the Tyler-estimator gets closer to the CRB
(but does not achieve it).
0,1
0,2
0,3
0,4
0,5
0,6
0,2 0,4 0,6 0,8 1
MML (SCM)
HB
CRLB
Tyler


View Slide

45
Example 2:
 
Assumed data pdf: Multivariate Complex Generalized Gaussian (MGG)
( ; ) , , , , ( ) exp
X i N
q
f CE h IID h q
b
b
 
 = 
 
 
x θ 0 Σ
 
 
True data pdf: Multivariate Complex -student
( ) , , , , ( )
N
X i N
t
p CE g IID g q q
l
l
h
 
 
 = 
 
 
x 0 Σ

View Slide

Assumed data pdf: Multivariate Generalized Gaussian (MGG)
 
 
 
1
1
( ; ) exp
H
N
i i
X i N
N b
f
N b
b
b
b
 b

  

 
=  
  
 
x Σ x
x θ
Σ
b is the shape parameter and b is the scale parameter characterizing
the model. The complex Gaussian pdf is a particular case of the MGG that
is obtained when b =1. The distribution has heavy tails when 0< b <1.
The MML estimator (i.e. the ML estimator for MGG data) is the solution
of the following fixed−point (FP) matrix equation [Oli12], [Gre13]:
 
 
 
1 1
1
(0)
1
( 1) ( )
1
1
, ( )
ˆ ˆ
1
ˆ ˆ , 0,1, ,
M
H H
i i i i
i
SCM
M
k H k H
i i i i it
i
t t
M b
k N
M
b
b
 

 
=


=
= =
 =


= =




Σ x Σ x x x
Σ Σ
Σ x Σ x x x 

View Slide

47
It can be proved that the iteration converges to the MML estimate if
and only if 0When M goes to infinity, the MML estimator converges a.s. to the
matrix that minimizes the KL divergence:
Then, we found that the MML estimator (for large M) is unbiased:
Hence, MCRB=HB.
( )
ˆ ˆ
i 0 1
l m k
MML
k
iff b


= 
Σ Σ
 
0
0
D p f 
=
 =  =
Σ Σ Σ
0 Σ Σ
  0
ˆ
MML
E 
= = =
μ Σ Σ Σ

View Slide

48
  
 
 
   
4
,
HB CRB
: 50
: 3 ( )
10 , , 0.9, 8, 3 24
ˆ ˆ
ˆ ˆ
, vecs( ), tr
HB CRB
,
i j
i j
T
p
T
F
F
F
F F
F F
Quasi Gaussian scenario
Super Gaussian scenario heavy tailed clutter
MC runs N M N
E
l
l
r r

 

 =
 = 
= = = = = =
 
 
 
= =
Σ
θ θ θ θ
θ Σ A A A
Σ
θ θ
Σ Σ

 

View Slide

49
: 50
Quasi Gaussian scenario l
 =
When l=50, the true t-distribution is pretty close to the Gaussian
distribution, so the performance of the MML estimator improves as β gets
closer to 1, the reverse when l=3.
The CRB and the MSE of Tyler’s estimator do not depend on β.
: 3
Super Gaussian scenario l
 =
0,1
1
0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9
MML
HB
CRLB
Tyler
Shape parameter of the GG distribution: 
0,1
1
0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9
MML
HB
CRLB
Tyler
Shape parameter of the GG distribution: 

View Slide

0,01
0,1
1
40 80 120 160 200 240 280
MML
HB
CRLB
Tyler
0,01
0,1
1
40 80 120 160 200 240 280
MML
HB
CRLB
Tyler
50
: 50
 =
0.1
b = 0.8
b =
When β=0.1 (i.e. we assume heavy-tailed GG clutter), the performance
of MML and Tyler’s estimators are pretty close and the HB is slightly
tighter than the CRB.
When β=0.8 (i.e. we assume quasi-Gaussian clutter), HBCRB and the
MML performs slightly better than the Tyler estimator.

View Slide

0,01
0,1
1
0,2 0,4 0,6 0,8 1
MML
HB
CRLB
Tyler

0,01
0,1
1
0,2 0,4 0,6 0,8 1
MML
HB
CRLB
Tyler

51
: 50
 =
0.1
b = 0.8
b =
In both cases, the MSE, the HB, and the CRB get worse when the
clutter one-lag correlation coefficient increases.
When β=0.8, the MML has the same performance as Tyler’s
estimator (up to very large r) and HBCRB.
When β=0.1, the HB is slightly tighter than the CRB.

View Slide

0,01
0,1
1
40 80 120 160 200 240 280
MML
HB
CRLB
Tyler
0,01
0,1
1
40 80 120 160 200 240 280
MML
HB
CRLB
Tyler
52
: 3
 =
0.1
b = 0.8
b =
When β=0.1 (i.e. we assume heavy-tailed GG clutter), HBCRB and
the MML performs better than Tyler’s estimator.
When β=0.8 (i.e. we assume quasi-Gaussian clutter), the HB is a
tighter bound than the CRB for the performance of the MML.

View Slide

0,01
0,1
1
0,2 0,4 0,6 0,8 1
MML
HB
CRLB
Tyler

0,01
0,1
1
0,2 0,4 0,6 0,8 1
MML
HB
CRLB
Tyler

53
: 3
 =
0.1
b = 0.8
b =
In both cases, the MSE, the HB, and the CRB get worse when the clutter
one-lag correlation coefficient increases.
Clearly, in the super-Gaussian scenario (l=3) the effect of mismatching
is more evident when we assume b=0.8 (almost-Gaussian GG clutter)
than when we assume b=0.1 (spiky GG clutter).

View Slide

54 of
28
Conclusions
Vector version of Misspecified Cramér-Rao bound (MCRB)
MCRB for Mismatched Maximum Likelihood (MML) estimators.
When the MML is unbiased it coincides with the Huber bound
Numerical examples, related to the problem of estimating the
scatter matrix of a CES data vector under mismodeling.
We are now investigating the effects of mismodeling in terms of
detector CFAR property and detection performance.
     
0 0 0
1 1 1
0
1 1
ˆ , HB CRB
T
MML
M M
  
 =   =

θ θ θ θ
M θ θ θ A B A rr θ F
r θ θ


View Slide

55
References
[Boy04] S. Boyd, L. Vandenberghe. Convex Optimization. Cambridge University Press, first edition, 2004.
[Fom03] T. B. Fomby and R. C. Hill, Eds., Maximum-Likelihood Estimation of Misspecified Models: Twenty
Years Later. Kidington, Oxford, UK: Elsevier Ltd, 2003.
[Gin02] F. Gini, M. S. Greco, “Covariance matrix estimation for CFAR detection in correlated heavy tailed
clutter,” Signal Processing, vol. 82, no. 12, pp. 1847-1859, December 2002.
[Gre13] M. S. Greco, F. Gini, “Cramér-Rao Lower Bounds on Covariance Matrix Estimation for Complex
Elliptically Symmetric Distributions,” IEEE Trans. on Signal Proc., vol. 61, no. 24, pp. 6401-6409, Dec. 2013.
[Hub67] P. J. Huber, “The behavior of maximum likelihood estimates under nonstandard conditions,”
Proceedings of the Fifth Berkley Symposium on Mathematical Statistics and Probability, 1967, pp. 221–233.
[Mar76] R. A. Maronna, “Robust M-estimators of multivariate location and scatter,” The Annals of Statistics,
vol. 4, no. 1, pp. 51-67, January 1976.
[McW93] L. T. McWhorter, L. L. Scharf, “Properties of quadratic covariance bounds,” The Twenty-Seventh
Asilomar Conference on Signals, Systems and Computers, 1993. , pp.1176 – 1180, vol. 2, 1-3 Nov
1993[Pas08] F. Pascal, Y. Chitour, J. Ovarlez, P. Forster, P. Larzabal, “Covariance Structure MLE in Compound
Gaussian Noise: Existence and Algorithm Analysis,” IEEE Trans. on Signal Processing, vol.56, no.1, pp. 34,48,
Jan. 2008.
[Oli12] E. Ollila, D.E. Tyler, V. Koivunen, V.H. Poor, “Complex Elliptically Symmetric Distributions: Survey,
New Results and Applications,” IEEE Trans. on Signal Proc., Vol. 60, No. 11, pp.5597-5625, 2012.
[Ric13] C. D. Richmond, L. L. Horowitz, “Parameter bounds under misspecified models,” Conference on
Signals, Systems and Computers, 2013 Asilomar, pp.176-180, 3-6 Nov. 2013.
[San12] K. J. Sangston, F. Gini, M. Greco, “Coherent radar detection in heavy-tailed compound-Gaussian
clutter”, IEEE Trans. on Aerospace and Electronic Systems, Vol. 42, No.1, pp. 64-77, 2012.
[Tyl87] D. Tyler, “A distribution-free M-estimator of multivariate scatter,” The Annals of Statistics, vol. 15,
no. 1, pp. 234-251, January 1987.
[Whi81] H. White, “Consequences and detection of misspecified nonlinear regression models,” Journal of
the American Statistical Association, vol. 76, pp. 419-433, 1981.

View Slide

Acknowledgements
Fulvio Gini, University of Pisa, Italy
Stefano Fortunati, University of Pisa, Italy
56

View Slide

Maria S. Greco

Maria S. Greco

S³ Seminar

More Decks by S³ Seminar

Other Decks in Research

Featured

Transcript