The two-stage process of propensity score analysis (PSA) includes a design stage where propensity scores are estimated and implemented to create an unconfounded ``pseudo-population" by way of matching/sub-classification/pruning/weighting procedures. Then in the analysis stage the treatment effect is estimated conditional upon the output of the design stage. Design uncertainty may be defined as uncertainty around the estimation of this pseudo-population, tied not only to the estimated propensity score but also further stochasticity contributed by the implementation stage. This paper presents a Bayesian framework for PSA, which allows the statistician to sample from a posterior distribution of treatment effects after ''marginalizing" over design stage estimation and implementation uncertainty. By creating a parameter which links the estimated propensity score to the result of the implementation step, a posterior distribution of design stage outputs may be defined even with regards to non-parametric implementation steps. The general framework of this paper is supported by and gives conceptual structure to Bayesian computational methods for PSA (BPSA) which were recently developed.