All Times ET
Virtual
A Framework for Improving the Efficiency of Operational Testing Through Bayesian Adaptive Design (304187)
Fletcher G W Christensen, University of New Mexico*Victoria Rose Carrillo Sieck, University of New Mexico / Air Force Institute of Technology
Keywords: Bayesian adaptive design, operational testing, defense acquisition efficiency, planned interim analysis, mission set analysis
When developing a product, it is important to consider product performance from a user perspective. This can be done through operational testing---assessing the ability of representative users to satisfactorily accomplish tasks with the product in operationally-representative environments. This process can be expensive and time-consuming, but is critical for evaluating a product. We show how an existing design of experiments (DOE) process for operational testing can be leveraged to construct a Bayesian adaptive design. This method, nested within the larger design created by the DOE process, allows interim analyses using predictive probabilities to stop testing early for success or futility. Furthermore, operational environments with varying probabilities of encountering are directly used in product evaluation. Representative simulations demonstrate how these interim analyses can be used in an operational test setting, and reductions in necessary test events are shown. This method will allow future testing to be conducted in less time and at less expense, on average, without compromising the ability of the existing process to verify the product meets the user's needs.