All Times ET
Virtual
An Application of Multiverse Analysis to Address the Inconsistency of Results Caused by Data Processing Choices (305259)
*Aleksandra Kazakova, CUNY, The Graduate Center
Keywords: Multiverse Analysis, Results Inconsistency, Multiverse Model
One of the challenges social scientists face is choosing the right methodological approach for their analysis. Even after the study design is determined and all information is collected, there are still various ways to process the data based on the combination of computational tools, model selection, variables measurements, and other methodological decisions. Such variability may create the issue of inconsistent results that are nearly impossible to replicate. To illustrate this, we reinvestigated the relationship between mother-father-child interactions and a child's receptive vocabulary at age five. This study aimed to show that research based on a single methodology may lead to an inconstant and contradicting conclusion. As a solution to this problem, we performed a Multiverse Analysis by testing different combinations of some common methodological choices producing thirty-six model variations. The significance of predictors across all models was measured and juxtaposed with each other. Such a comparison highlighted the advantages of the Multiverse approach over a single analysis by revealing that even a small difference in data processing may lead to inconsistency in results.