Keywords: multiple testing, hidden Markov model, Bayesian hierarchical model, false discovery rate, model selection
I am current working on proposing a Bayesian testing procedure for the multiple testing problem with temporally correlated data. I aim to examine two challenges of the multiple testing, which are the dependence correlation between null and non-null state and the unknown number of components in mixture distributed non-null state. Ignore correlation between hypotheses will lead to large variance of estimated proportion of false rejections. In practice, it is hard to have prior information to decide the number of components in the hypotheses test. Hidden Markov Model are introduced to capture the dependence structure between null and non-null hypotheses. Bayesian framework estimation allows for greater flexibility in the specification of distribution and in parameters, while considering of FDR defined in terms of the posterior probability. So, my interest is to see how the Bayesian test procedure performance under count data and bivariate normal data. Determine the number of components is a typical question in HMM model and several methods are discussed for HMM model selection. Bayesian Information Criterion (BIC), Likelihood ratio test and the value of marginal likelihood are commonly used methods. As the difficult of direct calculation of marginal likelihood for the HMM, under Bayesian framework, I examine the sampling-Based estimators based on Importance Sampling, Reciprocal Importance Sampling, Harmonic Mean Estimator and Bridge Sampling Technique in the model selection section. Compare the model selection decision between BIC and sampling-Based estimators. In order to compare the performance of proposed Bayesian testing procedure with other commonly used multiple testing procedure, False discovery rate (FDR) and False non-discovery rate (FNR) are calculated to find the optimal testing procedure. In my research study, both simple distributed and mixture distributed non-null state is discussed in simulation and case study.