|
Abstract:
|
The Department of Defense Test and Evaluation Community uses power as a key metric in the design planning phase to scope test size. Power depends on many elements of the design, including the selection response variables and factors , the model formulation, and sample size. The experimental objectives are expressed as hypothesis tests, and power reflects the risk associated with correctly assessing those objectives. If a test design is adequately scoped to address an irrelevant objective a Type III error occurs. In this presentation, we focus on a specific Type III error that test planners unknowingly commit for the purpose of reducing test size and resources. Specifically, we show how reparameterizing a factor space from few factors with many levels per factor to a space that has many factors with few levels per factor fundamentally changes the hypothesis tests and hence the objectives. Despite the increase in power and decrease in test resources that comes from this reparameterization, we conclude it is not a prudent way to gain test efficiency. Through a case study example, we highlight the information that is lost in this decision and its implications on test objectives.
|
