Abstract:

This poster is concerned with the optimal computational effort (i.e., allocation of Monte Carlo simulations) in multiple testing. In practice, pvalues of all hypotheses are usually unknown and thus have to be approximated with Monte Carlo simulations. Consider the allocation of a prespecified total integer number of simulations 'K' to a given finite number 'm' of hypotheses in order to approximate their pvalues in an optimal way, in the sense that the allocation minimises the expected number of misclassified hypotheses (with respect to the correct decisions obtained if pvalues were known). Under the assumption that the pvalues are known and K is realvalued, and using a normal approximation of the Binomial distribution, the optimal realvalued allocation of K simulations to m hypotheses is derived when correcting for multiplicity with the Bonferroni correction, both when computing pvalue estimates with or without a pseudocount. Empirical evidence is given that the optimal integer and realvalued allocations are likely of the same form, that both seem to coincide asymptotically, and that a published multiple testing algorithm allocates simulations in a closetooptimal way.
