Abstract:
|
There are many models whose normalizing constants are intractable, which are often called unnormalized models. When estimating parameters for such models, there are some popular methods such as score matching, contrastive divergence method and noise contrastive estimation. In our research, we focused on the noise contrastive estimation, which is a method using one auxiliary distribution, just as with importance sampling. Built upon the original paper in M. Gutmann and A. Hyv¨arinen [2010], we consider a general case where there are several target distributions and auxiliary distributions. By involving a nonparametric maximum likelihood approach in Monte Carlo integration, we derive the efficient estimator among a broader class, which includes the original noise contrastive estimation as a special case. In addition, we propose several techniques to reduce the asymptotic variance of the estimator, and we also provide a Bayesian perspective.
|