Supernova type Ia (SNIa) are an essential tool to constrain cosmological parameters, including dark energy, thought to make up 70% of the energy density of the universe. Given a small, non-representative set of confirmed SNIa from high-quality (and observationally expensive) spectra, upcoming large surveys will rely on supervised classification of low-resolution photometric data in order to identify transients as SNIa. To exploit the large photometric data, we present a fully hierarchical and pragmatic Bayesian framework which accounts for uncertainties arising from the probabilistic classification of SNIa, improving accuracy and precision of cosmological parameter estimates. In a fully Bayesian framework, correct specification of a contamination model (scientifically not well-known) is essential, and parts of the data are used twice, which is avoided using a pragmatic Bayesian approach. Our first stage classification of SNIa includes a new general-purpose method to improve supervised learning from non-representative training sets, exploiting methodology from causal inference, by fitting classifiers to strata based on estimated propensity scores.