Sample Size Calculations Using Techniques from Power Analysis (303273)

Keywords: Computer Engineering, Computer Science, Cyber, Security, Defense, Power Analysis, Hypothesis Testing, Automization

When the population of elements is extremely large and individual samples taken only on the order of hundreds of times per second, the sample size required for statistical significance becomes extremely important. In this study, after selecting which hypothesis tests are going to be used, a power analysis is performed on each test to determine the optimal number of samples required to achieve the maximum power for each test. To this end, both the effect size, sample size, and significance level are varied and 3-dimensional plots are produced. These plots represent how effective it is to increase the sample size when using each of these tests. We are able to calculate a useful number that provides a comfortable certainty that the sample size is large enough to place confidence in the testing framework. Specifically this analysis is applied to the Student's T two-sample hypothesis test and the chi-square goodness-of-fit test. The technique discussed in this poster can be generalized, and automated to allow for automatic sampling subroutines to be formulated. The result of this analysis is a software system that automatically determines appropriate sample size based on power analysis.