Online Program

Meta-analysis with Proportion Data (302469)

 View Presentation

Keywords: meta-analysis, synthesis, methodology, effect size, proportion, percentage

Meta-analysis has continuously garnered attention over the past thirty years due to its power for examining accumulated evidence. While methods for synthesizing studies involving mean differences, correlations, and odds ratios are well developed and documented (e.g., Cooper, 1998; Cooper, Hedges, & Valentine, 2009; Hunter & Schmidt, 2004; Lipsey & Wilson, 2001; Sutton, Abrams, Jones, Sheldon, & Song, 2000), methods for synthesizing studies reporting proportions (i.e., percentages) have not been discussed and recorded thoroughly. As a proportion is one of the commonly seen outcomes in the studies in several fields (e.g. drop-out rate in education; sensitivity/specificity of a test in medicine; unemployed rate in business), it is important to fill in the gap in the methodological literature and provide meta-analysts the information regarding how to conduct a meta-analysis when proportion data are involved.

This presentation will start with the discussion of using proportion as the effect size in meta-analysis. Effect sizes extracted from the selected studies constitute the data for a meta-analysis. Most of the effect sizes used in meta-analyses represent effects between treatment and control groups, such as group mean differences and odds ratios. Because proportions usually are used for describing scenarios for one group rather than comparing differences between groups, it has not gained much attention before. Cooper, Hedges, & Valentine (2009) discussed the differences between proportions of two groups, but they did not discuss the adoption of the proportion itself as the effect size. Intuitively, a proportion, ranging from .00 (or 0.00%) to 1.00 (or 100.00%), can be seen as measured on a continuous scale and the conventional meta-analytical approaches assuming normality may be applied. However, the sampling distribution of proportions may not be normal, if the proportion is not around .5 and the sample size is not large. The statistical characteristics of proportions and the possible methods of transforming proportions to smooth the distribution will be discussed in the beginning of this presentation, which will help the users of meta-analysis understand this specific type of effect size and so they can prepare their data for meta-analyses appropriately.

The second component in this presentation will include the discussion of summarizing and modeling proportions. Conventional methods in meta-analysis based on fixed- and random- effects models along with the test of homogeneity of effect sizes will be discussed, with some necessary twists for synthesizing proportions. The methods for appropriately calculating between- and within- groups variance based on proportions as well as the methods for conducting moderator analyses will be demonstrated using a real dataset (more below). Demonstrating the process with a real data example allows the participants of this conference to learn the process practically and visually.

The dataset that will be used to demonstrate the methods of synthesizing proportions is based on a project on meta-analyzing response rates in online survey research (Wu, Young, Zhao, & Clark, 2012). In this project, the trend and the average of online response rate during the past five years in education field were reported based on both fixed- and random- effects models. The factors impacted the response rates of internet-base survey were examined through a set of moderator analyses. The analyses will be demonstrated using SPSS (and also could be done in SAS if preferred by the audience). The syntax will be provided to the audiences so they can modify for their projects as necessary. The figures and plots that could help to efficiently covey the synthesis results will be presented so the audience can learn an informative way to communicate with their customers or clients. Some errors in previous meta-analyses on the same topic in different fields, likely due to the lack of guideline for conducting meta-analyses with proportion data, will also be discussed so the audience can be aware of those errors and avoid them in the future. The examples will cover the field of organizational psychology, education, and medicine.