Self-Initiated vs. Randomized Selection Effects in Web Site Evaluation

• Activity Number: 292
• Type: Contributed
• Date/Time: Tuesday, August 5, 2003 : 2:00 PM to 3:50 PM
• Sponsor: Section on Survey Research Methods
• Author(s): Elizabeth C. Smith*+
• Companies: Northwestern Mutual Life Insurance Company
• Keywords: website evaluation ; survey method ; sample selection

Abstract

It is well known that self-initiated surveys produce different results from surveys in which a probability sample is drawn from a sampling frame. A company web site was evaluated using the same survey mode and measurements with three different sample selection strategies: recruitment of a customer random sample supplemented by a vendor listing in 2001, a self-initiated online survey with uncontrolled sampling in 2002, and a pop-up survey in which 5% of site visitors received a survey participation request in 2002. In this study, we look at differences in the purpose of web site visits, item rating distributions for characteristics of a company web site, and demographic characteristics of responders by sampling process. Job seekers were more likely to be represented in the pop-up survey than in the other formats. Scores were very concentrated at the extremes of a 10-point rating scale for the self-initiated survey compared to the other formats. Respondents to the pop-up survey were younger than for the self-initiated survey. The number of responses was greater over a similar length of time with the random pop-up recruitment box compared to the self-initiated form.