The Fisher information matrix (FIM) has long been of interest in statistics and computational mathematics for its various uses. Its scalar case, the Fisher information number (FIN), is also widely used. However, in many cases it is challenging to obtain the true value for the Fisher information so we need to use the unbiased estimation as a proxy.
In this paper, we compare the accuracy of two unbiased estimations for FIN, i.e. the sample mean of squared gradient and the sample mean of second-order derivative of log-likelihood. A set of sufficient conditions are deducted, based on which we can judge the accuracy of each estimation. The conclusion will be case sensitive, meaning that either method can be more accurate than the other depending on the density function.
Three examples are provided in numerical study as illustrations to our analysis. We look at independent and identically distributed samples from Laplace distribution and normal distribution, respectively. We also look at a signal-plus problem that features an independent but non-identically distributed samples.
|