The Role of Previous Experience and Attitudes Toward Statistics in Statistics Assessment Outcomes among Undergraduate Psychology Students

Martin Dempster
Noleen K. McCorry
Queen's University Belfast

Journal of Statistics Education Volume 17, Number 2 (2009), jse.amstat.org/v17n2/dempster.html

Copyright © 2009 by Martin Dempster and Noleen K. McCorry, all rights reserved. This text may be freely shared among individuals, but it may not be republished in any medium without express written consent from the authors and advance notification of the editor.

Key Words: Cognitive competence; Value of statistics; Difficulty of statistics; Affect about statistics.

Abstract

Previous research has demonstrated that students’ cognitions about statistics are related to their performance in statistics assessments. The purpose of this research is to examine the nature of the relationships between undergraduate psychology students’ previous experiences of maths, statistics and computing; their attitudes toward statistics; and assessment on a statistics course. Of the variables examined, the strongest predictor of assessment outcome was students’ attitude about their intellectual knowledge and skills in relation to statistics at the end of the statistics curriculum. This attitude was related to students’ perceptions of their maths ability at the beginning of the statistics curriculum. Interventions could be designed to change such attitudes with the aim of improving students’ learning of statistics.

1. Introduction

Training in statistics is an important (and compulsory) part of the psychology undergraduate curriculum. It has been argued that the statistics element of the psychology undergraduate programme is one of the subjects that enhances critical reasoning among students (Lehman & Nisbett, 1990). Lawson (1999) states that critical thinking is an ability that differentiates psychology graduates from other types of students who have not been required to undergo training in statistics or research methodology. This is supported by VanDerStoep and Shaughnessy (1997) who found that psychology students who studied research methods and statistics had a greater ability to apply critical reasoning to real-life events than psychology students who had not yet undertaken these courses. Therefore, the inclusion of statistics in the undergraduate psychology programme is not only compulsory but has a beneficial effect on the psychologist’s cognitive abilities.

However, a statistics course is the part of the psychology curriculum most dreaded by students (Conners McCown and Roskos-Ewoldsen 1998; Schutz Drogosz White and Distefano 1998). Indeed, these feelings are not restricted to psychology students. Field (2000, p.xiv) states that "since time immemorial, social science students have despised statistics." The dread of statistics courses has also been reported for social work students (Forte 1995) and for education and sociology students (Murtonen and Lehtinen 2003). Students entering these disciplines often do not have a strong mathematics background and often dislike anything "mathematical". The threat of working with numbers can be enough to cause some students to transfer courses.

Unsurprisingly, statistics exam performance among psychology students is positively related to previous experience of statistics (Schutz et al 1998) and of maths (Lalonde and Gardner 1993; Schutz et al 1998).

Performance on statistics assessments is also clearly shown to be related to students’ attitudes toward statistics. There have been several instruments developed to assess attitudes towards statistics: the Attitudes Towards Statistics scale (ATS: Wise 1985), the Statistics Attitude Survey (SAS: Roberts and Saxe 1982) and the Survey of Attitudes Toward Statistics (SATS: Schau Stevens Dauphinee and Del Vecchio 1995). The differences between these instruments are not the focus of the present study but it is interesting to note that a relationship between statistics assessment outcomes and attitudes has been found in previous research regardless of the instrument employed (Cashin and Elmore 2005; Rhoads and Hubele, 2000; Roberts and Bilderback 1980; Roberts and Reese 1987; Shultz and Koshino, 1998; Tremblay Gardner and Heipel 2000; Vanhoof et al 2006; Waters Martelli Zakrajsek and Popovich 1988; Waters Martelli Zakrajsek and Popovich 1989; Wise, 1985).

Indeed, it may be the case that previous experiences of maths and statistics might lead to the formation of attitudes toward statistics (Carmona Martinez and Sanchez 2005), which in turn will affect assessment outcome. Using a longitudinal design, this hypothesis is examined in the present study.

2. Method

Students were administered the Survey of Attitudes Toward Statistics (Schau Stevens Dauphinee and Del Vecchio 1995; see http://www.flaguide.org/tools/attitude/attitudes_toward_statistics.php) at their first statistics class in the first year of their undergraduate psychology programme and at the end of the taught statistics curriculum within the psychology programme (at the end of the students’ second year). The taught statistics curriculum for psychology undergraduates consists of 4 statistics courses – 2 in year 1 (1 per semester) and 2 in year 2 (1 per semester). Each course is 12 weeks in length. The courses which run in year 1 consist of a 1 hour lecture per week and a 2 hour practical class. The practical classes are held in a computer lab, where students are provided with data sets and asked to address questions of the data set which are designed to lead them to run specific statistical tests on a statistical software package. The statistical tests focused on in the practical classes are the ones which have most recently been addressed in the lectures. In year 1, semester 1, after an introduction to research design and research ethics, students in year 1 cover the following topics: levels of measurement, survey design, experimental design, presenting data and describing shape, summary statistics (eg. mean and standard deviation), crosstabulations, scatterplots, and correlation coefficients (from a descriptive rather than inferential perspective). The course which runs in year 1, semester 2 includes: principles of statistical inference, inferences about correlation coefficients, t-tests, Wilcoxon and Mann-Whitney tests, and chi-square.

In year 2, the statistics courses include a 2 hour lecture per week and 5 computer-based practical classes of 2 hours duration spread throughout each semester. In year 2, semester 1 the course focuses on ANOVA models, including one-way, two-way, repeated measures and mixed ANOVAs. In year 2, semester 2, the course examines the topics of reliability and validity, factor analysis and linear regression.

The Survey of Attitudes Toward Statistics (SATS) contains 28 items which are divided into four dimensions: affect – positive and negative feelings concerning statistics (6 items), cognitive competence – attitudes about intellectual knowledge and skills when applied to statistics (6 items), value – attitudes about the usefulness, relevance and worth of statistics in personal and professional life (9 items), and difficulty – attitudes about the difficulty of statistics as a subject (7 items). All items are scored on a 7 point scale and items within each dimension are summed to provide a score on that dimension. Higher total scale scores correspond to more positive attitudes. The structure of the SATS has been confirmed with data obtained from undergraduates (Dauphinee Schau and Stevens 1997). In the present study, Cronbach’s alpha for the affect, cognitive competence, value and difficulty scales was 0.81, 0.85, 0.81 and 0.75 respectively.

In addition, single items (using a 7 point response scale) were included asking the respondent about their previous experience with maths, statistics and computing. These items are based on the additional items included in the SATS to assess previous experience (see Appendix). A higher score on these items represented a perception of stronger maths ability, and more statistics and computing experience.

Students were asked to complete these single item measures and the SATS at the beginning of their first statistics class in their first year. They were asked to complete the SATS a second time at the end of their statistics curriculum (at the end of their second year). At both testing time points, students were asked to complete the questionnaire at the end of a lecture session in a large lecture theatre. Completed questionnaires were left in a box at the back of the room by the students as they exited. The students were not observed while completing the questionnaires and were informed that they were not obliged to complete the questionnaires. Students were asked to include their unique student number on the questionnaire for the purposes of matching questionnaires across the 2 points in time.

A total of 154/162 (95%) students returned completed questionnaires at time 1 and 103/125 (82%) students returned completed questionnaires at time 2. However, questionnaires at the 2 points in time could be matched for 82 students (mean (SD) age = 19.34 (2.64); 78% female) only.

There were no statistically significant differences on the SATS scales (affect: t = 0.243, p = .809; cognitive competence: t = 0.945, p = .346; value: t = 1.398, p = .164; difficulty: t = 0.127, p = .899) between those who were matched on the questionnaires at both points in time (n = 82) and those who completed the questionnaires at time 1 but were not included in the matched group (n = 72).

3. Results

Participants’ average rating of their maths ability was 4.62 (SD=1.08). They reported previous experience with statistics at slightly lower than the midpoint of the scale (Mean (SD) = 3.59 (1.67)) and more previous experience with computers (Mean (SD) = 4.68 (1.34)). Table 1 displays the summary statistics for the SATS scales at each time point. Participants attained an average assessment score of 65.88% (SD=12.04) at the end of the statistics curriculum (end of their second year).

Table 1. Change in SATS Scores

Correlation coefficients between assessment scores and previous experiences of maths, statistics and computing and the four SATS scales are detailed in Table 2. The results suggest that attitudes about cognitive competence, affect and value at time 2 are most strongly related (out of those included in the model) to performance on the statistics assessment (see Table 2). From Table 2 it can also be seen that affect at time 2 is strongly related to cognitive competence at time 2. Therefore, when assessment outcome was regressed on these variables only (to provide a more parsimonious model), affect at time 2 was omitted from the analysis to avoid violating the assumption of no collinearity. The regression analysis indicated that cognitive competence at time 2 was the only variable (out of those included in the model) which made a statistically significant contribution to the explanation of the variance in assessment outcome (cognitive competence: beta = .252, p = .038; value: beta = .117, p = .331).

A hierarchical regression model (Table 3) examined the correlates of reported cognitive competence at time 2. The independent variables in step 1 are perception of maths ability and previous experience of statistics (previous experience of computing was omitted on the basis of its low correlation with cognitive competence at time 2) and in step 2 are the 4 SATS scales at time 1. The regression analysis suggests that the variable in the model which contributes most to the explanation of variance in cognitive competence at time 2 is cognitive competence at time 1. There is also an indication that cognitive competence at time 1 mediates the relationship between perception of maths ability and cognitive competence at time 2. To clarify this mediation effect, the hierarchical regression presented in Table 3 was repeated with perception of maths ability and cognitive competence at time 1 only (see Table 4). Furthermore, cognitive competence at time 1 was regressed on perception of maths ability and a significant model was found (b = 3.040, SE = 0.576, p < .001). A Sobel test based on these analyses suggests that the indirect effect of the perception of maths ability on cognitive competence at time 2 via cognitive competence at time 1 is statistically significant (Sobel test = 3.116, p = 0.002).

Table 3. Predicting Cognitive Competence at Year 2

Table 4. Predicting Cognitive Competence at Year 2 – testing for mediation

4. Discussion

Previous research indicates that prior experiences of maths and statistics and students’ attitudes to statistics are important predictors of performance on statistics assessments at undergraduate level. The present research has clarified further the nature of these relationships. Using a longitudinal design, the present research has shown that statistics assessment outcome is correlated more highly with specific attitudes held at the time of the assessment rather than attitudes about statistics which are held by students at the beginning of their statistics course. Furthermore, attitudes about cognitive competence is more strongly related to assessment outcomes than previous experience with maths, statistics or computing.

The series of analyses suggests that attitudes about cognitive competence held at the beginning of the statistics curriculum intervene in the relationship between maths experience and attitudes about cognitive competence at the end of the statistics curriculum. This suggests that perception of maths ability may contribute to the formation of students’ beliefs about their intellectual ability to understand statistics before they encounter the statistics curriculum within their undergraduate psychology program. Once formed, this attitude changes little before the end of the statistics curriculum and this helps to explain the performance in assessment.

Given that attitudes have a stronger relationship with assessment outcomes than previous experiences, there is potential for developing and implementing interventions which will modify these attitudes. Such interventions should focus primarily on the attitude of cognitive competence – we need to help students to believe that they have the intellectual capacity to cope with the demands of the statistics curriculum and we need to maintain this attitude throughout the curriculum. One important way that this might be achieved is by enhancing students’ perceptions of their maths ability. Perhaps some remedial teaching of basic maths is required to improve students’ maths ability and improve their confidence in approaching the subject of statistics before the statistics curriculum begins. Maybe explicit information about the difference between statistics and maths would also help in the formation of more positive attitudes toward statistics at the beginning of the curriculum.

The literature contains some potentially useful examples of such interventions (eg. Gourgey, 2000; Dyck and Gee 1998; Johnson 1989; Zerbolio 1989). The results from the present research suggest that an intervention such as that proposed by Bartsch (2006) is likely to be most effective as it aims to enhance students’ self-efficacy about statistics by drawing on students’ intuition. In this intervention, students are asked questions about numbers and are expected to estimate answers to these questions rather than using any formal calculations. For example, students will be told the value of a set of numbers (with a small n) and then asked to estimate the average. Introducing the concept of the mean (and other statistics) in this way (rather than using a mathematical formula) improved students’ attitudes toward statistics.

Whether such an intervention is effective because it encourages students to feel comfortable with numbers and avoids the students remembering previous poor experiences with math, or whether it is effective because students receive positive feedback at an early point in the course (because there are no "wrong" answers) is an appropriate future research question. The search for the effective ingredient of interventions is an important one but, in the meantime, we know that some things work to improve important attitudes and we should strive to employ them in our teaching.

Appendix

Single items used to assess previous experience with maths, statistics and computing:

How good at mathematics are you?

            Very poor                                                                    Very good

                        1          2          3          4          5          6          7

How much computer experience have you had?

            None                                                                           Great deal

                        1          2          3          4          5          6          7

How much experience with statistics have you had (e.g., courses, research studies)?

            None                                                                           Great deal

                        1          2          3          4          5          6          7

References

Bartsch, R.A. (2006), "Improving attitudes toward statistics in the first class," Teaching of Psychology, 33, 197-198.

Carmona, J., Martinez, R.J., and Sanchez, M. (2005), "Mathematical background and attitudes toward statistics in a sample of Spanish college students," Psychological Reports, 97, 53-62.

Cashin, S.E., and Elmore, P.B. (2005), "The Survey of Attitudes Toward Statistics Scale: A construct validity study," Educational and Psychological Measurement, 65, 509-524.

Conners, F.A., McCown, S.M., and Roskos-Ewoldsen, B. (1998), "Unique challenges in teaching undergraduate statistics," Teaching of Psychology, 25, 40-42.

Dauphinee, T. L., Schau, C., and Stevens, J. J. (1997), "Survey of attitudes toward statistics: Factor structure and factorial invariance for women and men," Structural Equation Modeling, 4, 129-141.

Dyck, J. L., and Gee, N. A. (1998), "A Sweet Way to Teach Students About the Sampling Distribution of the Mean," Teaching of Psychology, 25(3), 192-195.

Field, A. (2000), "Discovering Statistics Using SPSS for Windows," London: Sage.

Forte, J.A. (1995), "Teaching statistics without sadistic," Journal of Social Work Education, 31, 204-218.

Gourgey, A.F. (2000), "A classroom simulation based on political polling to help students understand sampling distributions," Journal of Statistics Education, 8(3). (http://jse.amstat.org/secure/v8n3/gourgey.cfm).

Johnson, D.E. (1989), "An intuitive approach to teaching analysis of variance," Teaching of Psychology, 16, 67-68.

Lalonde, R. N., and Gardner, R. C. (1993), "Statistics as a 2nd language - a model for predicting performance in psychology students," Canadian Journal of Behavioural Science, 25, 108-125.

Lawson, T.J. (1999), "Assessing psychological critical thinking as a learning outcome for psychology majors," Teaching of Psychology, 26, 207-209.

Lehman, D.R. & Nisbett, R.E. (1990). A longitudinal study of the effects of undergraduate training on reasoning. Developmental Psychology, 26, 952-960.

Murtonen, M., and Lehtinen, E. (2003), "Difficulties experienced by education and sociology students in quantitative methods courses," Studies in Higher Education, 28, 171-185.

Rhoads, T.R., and Hubele, N.F. (2000), "Student attitudes toward statistics before and after a computer-integrated introductory statistics course," IEEE Transactions on Education, 43, 182-187.

Roberts, D.M., and Bilderback, E.W. (1980), "Reliability and validity of a statistics attitude survey," Educational & Psychological Measurement, 40, 235-238.

Roberts, D.M., and Reese, C.M. (1987), "A comparison of two scales measuring attitudes towards statistics," Educational & Psychological Measurement, 47(3), 759-764.

Roberts, D.M., and Saxe, J.E. (1982), "Validity of a statistics attitude survey – a follow-up study," Educational and Psychological Measurement, 42, 907-912.

Schau, C., Stevens, J., Dauphinee, T. L., and Del Vecchio, A. (1995), "The development and validation of the Survey of Attitudes Toward Statistics," Educational & Psychological Measurement, 55(5), 868-875.

Schutz, P.A., Drogosz, L.M., White, V.E., and Distefano, C. (1998), "Prior knowledge, attitude, and strategy use in an introduction to statistics course," Learning and Individual Differences, 10 (4), 291-308.

Shultz, K. S., and Koshino, H. (1998), "Evidence of reliability and validity for Wise's attitude toward statistics scale," Psychological Reports, 82(1), 27-31.

Tremblay, P. F., Gardner, R. C., and Heipel, G. (2000), "A model of the relationships among measures of affect, aptitude, and performance in introductory statistics," Canadian Journal of Behavioural Science, 32(1), 40-48.

VanDerStoep, S. W., and Shaughnessy, J. J. (1997), "Taking a course in research methods improves reasoning about real-life events," Teaching of Psychology, 24(2), 122-124.

Vanhoof, S., Sotos, A.E.C., Onghena, P., Verschaffel, L., Van Dooren, W., and Van den Noortgate, W. (2006), "Attitudes toward statistics and their relationship with short and long term exam results," Journal of Statistics Education, 14(3). (http://jse.amstat.org/v14n3/vanhoof.html).

Waters, L.K., Martelli, T.A., Zakrajsek, T., and Popovich, P.M. (1988), "Attitudes toward statistics: An evaluation of multiple measures," Educational and Psychological Measurement, 48, 513-516.

Waters, L.K., Martelli, T.A., Zakrajsek, T., and Popovich, P.M. (1989), "Measuring attitudes toward statistics in an introductory course on statistics," Psychological Reports, 64(1), 113-114.

Wise, S.L. (1985), "The development and validation of a scale measuring attitudes toward statistics," Educational and Psychological Measurement, 45, 401-405.

Zerbolio, D.J. (1989), "A ‘bag of tricks’ for teaching about sampling distributions," Teaching of Psychology, 16, 207-209.

Martin Dempster
School of Psychology
Queen’s University Belfast
University Road, Belfast, N.
Ireland, UK
Tel: +44 28 90975547
Fax: +44 28 90664144
Email: m.dempster@qub.ac.uk

Noleen McCorry
School of Psychology
Queen’s University Belfast
University Road, Belfast, N.
Ireland, UK
Tel: +44 28 90975650
Fax: +44 28 90664144
Email: n.mccorry@qub.ac.uk