Online Program

Experimental Design for Testing with Multiple Segments (303029)

Keywords: Experimental Design, Test and Control, Multiple testing, Segmentation

In the business world, experiments with test and control are used widely to assess a new product or marketing offer effect. Although experiment design has been commonly used, we still need to fully use its related statistical theory to address the following three areas of concerns. The first concern is sample size calculation. It needs to be calculated by considering variation in the population and the purpose of the experiment. The second concern is result interpretation. Directional comparison does not tell the whole story. A larger mean measure does not necessarily indicate a better choice. It could be purely due to chance. And the last concern is testing with multiple segments. The measure of interest can vary between segments. Therefore, the sample size needs to be calculated accordingly for each segment. In addition, we need to consider family-wise error rate in testing with segments. In this presentation, an experimental design and according analysis of a marketing campaign with multiple segments are demonstrated, where all three concerns above are addressed.