|
Abstract:
|
Bayesian nonparametrics based on completely random measures (CRMs) offers a flexible modeling approach when the number of latent components in a dataset is unknown. However, the infinite dimensionality of CRMs typically prevents the use of general-purpose inference methods and leads to long compute times. We propose a general but explicit recipe to construct a finite-dimensional approximation that can replace the infinite-dimensional CRMs. Our independent finite approximation (IFA) is a generalization of important cases that are used in practice. The independence of IFA atom weights (i) makes the construction well-suited for distributed computation and (ii) facilitates convenient inference schemes. We quantify the approximation error between IFAs and the target prior. We compare IFAs with an alternative approximation — truncated finite approximations (TFAs), where the atom weights are constructed sequentially. We prove that for worst-case observation likelihoods, TFAs are a more efficient approximation than IFAs. However, in real-data experiments with image denoising and topic modeling, we find that IFAs perform very similarly to TFAs in terms of task-specific accuracy metrics.
|
