Joan Garfield

University of Minnesota

Bob Hogg

University of Iowa

Candace Schau

University of New Mexico

Dex Whittinghill

Rowan University

Journal of Statistics Education Volume 10, Number 2 (2002), ww2.amstat.org/publications/jse/v10n2/garfield.html

Copyright © 2002 by Joan Garfield, Bob Hogg, Candace Schau, and Dex Whittinghill, all rights reserved. This text may be freely shared among individuals, but it may not be republished in any medium without express written consent from the authors and advance notification of the editor.

**Key Words:** Introductory statistics; Undergraduate Statistics Education Initiative (USEI).

Over the past twenty years much has been written about the introductory or service course in statistics. Historically, this course has been viewed as difficult and unpleasant by many students and frustrating and unrewarding to teach by many instructors. Dissatisfactions with the introductory course have led people to suggest new models for the course, to lead workshops to reexamine this course (Hogg 1992), and to offer recommendations for how the course should be changed (Cobb 1992). This paper presents the results of a survey of teachers of the first statistics course, to determine the impact of reform efforts on the teaching of statistics. Suggestions and guidelines for teaching these courses are offered, based on the results of the survey.

Introductory statistics courses continue to be the focus of recommendations for changes in statistics education, and the number of students taking introductory statistics is steadily increasing. Loftsgaarden and Watkins (1998) estimated that 236,000 students were enrolled in post-secondary elementary-level statistics courses offered by mathematics or statistics departments in the United States in the fall semester of 1995. This number underestimates the actual current enrollment for at least two reasons. The first is that the estimate is five years old and enrollments are increasing. The second is that the estimate does not include introductory statistics courses taught by faculty in many other departments (psychology, sociology, business, and economics, for example). Introductory statistics is often the one and only statistics course taken by students who are not majoring in this discipline.

In recent years many statisticians have become involved in the reform movement in statistical education aimed at the teaching of introductory statistics, and the National Science Foundation (NSF) has funded numerous projects designed to implement aspects of this reform (Cobb 1993). Moore (1997a) describes the reform in terms of changes in content (more data analysis, less probability), pedagogy (fewer lectures, more active learning), and technology (for data analysis and simulations). Hoaglin and Moore (1992) offered a set of readings to inform statistics instructors of new content and techniques, Garfield (1995) offered a research perspective on why and how teaching methods should be changed, and many statisticians have suggested ways to incorporate technology into the introductory course (see, for example, Velleman and Moore 1996; Lock 2000).

A principle aspect of the reform movement is the focus on concepts, reasoning, and thinking. Butler (1998), in a recent article titled “On the Failure of the Widespread Use of Statistics,” suggested that, in spite of the increasing numbers of adults who complete introductory statistics courses, these adults do not often use statistical methods in their jobs and, when they do try, “the results are a shambles” (p. 84). This may be due to the way statistics courses have traditionally been taught: with a focus on computation, skills, and compartmentalized knowledge. We believe that while an introductory course cannot make novice students into expert statisticians, it can help students develop statistical thinking, which they should be able to apply to real world situations.

Snee (1990) defined statistical thinking as thought processes which recognize that variation is all around us and present in everything we do. According to this definition, all work is viewed as a series of interconnected processes. Identifying, quantifying, controlling, and reducing variation provide opportunities for improvement. While Snee discussed statistical thinking as a way of improving products and services in business and industry, his ideas extend to students in a first course who should be able to recognize that variation occurs in almost everything and that their ability to respond to various situations should be at least partially determined by that recognition.

Wild and Pfannkuch (1999) caution that “statistical thinking” is a term that is not clearly defined or understood and one that evokes a vague, intuitive understanding of how statisticians think and solve problems. They believe that no one really knows how to help students develop statistical thinking, and that instructors’ best guess is to assign projects and hope that this type of thinking will develop. Wild and Pfannkuch’s qualitative study of statisticians and statistics students resulted in a detailed model of statistical thinking that includes four dimensions that operate at the same time. In his response to the Wild and Pfannkuch paper, Snee (1999) urged changes in the content and delivery of statistics courses, and more research on the development of statistical thinking.

Hogg (1992) outlined the goals of a course designed to develop statistical thinking, where the focus is on the process of learning how to ask appropriate questions, how to collect data effectively, how to summarize and interpret that information, and how to understand the limitations of statistical inferences. The details of this course include:

- Emphasize the elements of statistical thinking:
- the need for data,
- the importance of data production,
- the omnipresence of variability,
- the measuring and modeling of variability.

- Incorporate more data and concepts, fewer recipes and derivations and wherever possible, automate computations and graphics. An introductory course should
- rely heavily on real (not merely realistic) data,
- emphasize statistical concepts such as causation vs. association, experimental vs. observational studies, and longitudinal vs. cross-sectional studies,
- rely on computers rather than computational recipes,
- treat formal derivations as secondary in importance.

- Foster active learning, through the following alternatives to lecturing:
- group problem solving and discussion,
- laboratory exercises,
- demonstrations based on class-generated data,
- written and oral presentations,
- projects, either group or individual.

A more detailed examination of the desired outcomes of an introductory course includes the following three categories: learning (students’ understanding, reasoning, thinking), persistence (leading students to use their statistical knowledge and skills after they leave the course), and attitudes and beliefs (about the value and importance of statistics and about themselves as learners and users of statistics). While the traditional and most commonly discussed course outcome is that of statistical learning, the other outcomes listed above are also important to consider as they will greatly affect whether or not our students are able to appropriately use statistical skills, ideas, and techniques. Therefore, our courses should attempt to build strong positive attitudes towards statistics and reinforce students’ use of statistics in the real world to increase their chances of using statistics after they leave our courses.

Enabling students to achieve the desired course outcomes is a complex endeavor, and one that is affected by a myriad of factors. Schau (personal communication, 2000) proposes a preliminary model (see Figure 1) that displays the various factors that affect student outcomes.

**Figure 1.** Model of Factors Influencing Student Outcomes.

This model shows that the course instructor is but one component in the overall instructional experience that students have, and one that interacts with (and is often constrained by) other factors such as institutional characteristics, learner characteristics, and other course components.

Despite the involvement of statisticians in the educational reform movement, Moore (1997b) points out that many people teaching introductory statistics are not statisticians. In fact, far more sections of introductory statistics are taught in mathematics departments or in other disciplines than by statisticians in statistics departments. While some statistics instructors who teach in other disciplines such as psychology, business, and sociology, have participated in reform efforts, little is known about the teaching of introductory statistics courses across those disciplines.

Garfield (2000) surveyed a large number of statistics instructors from mathematics and statistics departments, and a smaller sample of statistics instructors from departments of psychology, sociology, business, and economics, to determine how the introductory course is currently being taught and to begin to explore the impact of the educational reform movement.

The first part of her study gathered data on the infrastructure of introductory statistics courses. Brief surveys were sent to chairs of all 106 statistics departments and to a stratified random sample of 400 mathematics departments (from colleges offering two-year, four -year, and graduate degrees). Department chairs were asked about the type of introductory course taught, and were also asked to indicate names and email addresses of faculty to whom a more detailed survey should be sent. Responses were received from 227 mathematics departments (57%) and 81 statistics departments (76%).

The data gathered on the types of introductory statistics course indicated that the most typical structure for two and four-year college mathematics departments (66% and 61%) , as well as for mathematics departments offering the MS as their highest degree (59%), was one, common, introductory course. For statistics departments, the most typical structure was multiple introductory courses (46%). Thirty-five percent of all of the departments surveyed indicated that other departments at their institution also teach statistics, and 14% responded that they do not offer a non-calculus based introductory statistics course at all.

Based on the data gathered in the preliminary survey, a more detailed survey on the teaching of the introductory course was sent to the generated list of statistics instructors in mathematics and statistics departments, as well as other departments that teach an introductory course (psychology, sociology, business, and economics). This “Snapshot of the Introductory Statistics Course” asked a variety of questions grouped into categories, including particular uses of technology, teaching methods, and student assessment; changes made or to be made to the introductory course, the impact of changes made on faculty and students, and faculty views regarding reform efforts.

A total of 243 people responded to this survey. Results were analyzed according to the following categories: mathematics departments in two-year colleges (*n* = 56), mathematics departments that offer undergraduate or graduate degrees (*n* = 91), statistics departments (*n* = 65) and other departments (*n* = 31). It is important to note that these numbers may not necessarily represent all the statistics courses taught in these departments and that these results may be biased as teachers supportive of the reform recommendations could have been more likely to respond.

Despite these cautions, it is still possible to describe the teaching of statistics in the courses of those instructors who completed the survey. In the summary below, 2YEAR represents statistics courses in two-year colleges, 4YEAR represents courses in math departments that offer four year or graduate degrees, STAT represents courses in statistics departments, and OTHER represents courses in psychology, business, sociology, and economics. For these courses, the following results were observed:

Students taking STAT course are most likely to have textbooks written by Moore (1995), Moore and McCabe (1999), McClave, Dietrich, and Sincich (1997) or Freedman, Pisani, and Purves (1998). Students in 4YEAR courses are likely to have textbooks by Moore (1995) or Moore and McCabe (1999), McClave, et al. (1997) as well as texts by Bluman (1997) and others. Students in 2YEAR courses are most likely to have Triola (1998) as their text, followed by Moore (1995), and Weiss (1999). Data were not tabulated for students in OTHER courses as there were no consistent trends across the courses.

Most students in the courses surveyed are required to use some type of technology, although students in 2YEAR courses are more likely to use graphing calculators (for computations using small data sets) and about one-fourth of the instructors across courses require students to learn a spreadsheet such as EXCEL®. About one-half of the faculty surveyed involve students in using a statistical software program, typically Minitab®, although SPSS® is often used in the OTHER courses. Web resources such as data sets, applets, and discussion groups, are used more often by STAT instructors.

The most frequent teaching method used is the lecture, although most instructors incorporate some type of demonstration or experiment, discussions of statistics in the media, or case studies. The main users of videos such as the

*Against All Odds*(COMAP 1989) series are STAT instructors. Small group activities and student presentations are used more often in 2YEAR courses, and writing to learn activities are used more in statistics courses taught in mathematics departments.Exams, homework, and quizzes are the most frequently used assessment tools, although some teachers use team projects, lab activities, and critiques of articles in this role. Projects and take-home exams are used more often in courses outside of statistics departments. 2YEAR courses use the widest variety of assessment techniques. OTHER courses use more out of class group assignments, fewer multiple-choice exams, and fewer quizzes, compared to STAT, 2YEAR, and MATH instructors.

Courses are often being revised. More than two-thirds of the faculty surveyed reported making moderate to major revisions in their course over the past few years. The most common changes include the increased use of technology (70 to 80%, across the four groups); followed by teaching methods (56 to 66%), course content (43 to 60%), and assessment (25 to 34%). For STAT instructors, these changes are most often due to students’ dissatisfaction with the course, followed by their own dissatisfaction with the course, and because of suggestions made by influential colleagues in their institution or elsewhere. More math instructors reported that they were influenced by recommendations from statistics education articles or presentations.

Instructors’ reactions to changes made in their courses appear to be mostly positive, despite the increased demands on their time that these changes require. Most report that their students appear to be enjoying the course more (63 to 76%), work harder or the same as before (but not less), learn more content, and learn somewhat different content. Most faculty enjoy teaching more, share ideas more, and need more time for preparing for their classes.

The colleagues of the instructors surveyed may or may not be aware of reform efforts, and may or may not be supportive of reform recommendations. However, many of the instructors report increased involvement in statistical education activities. STAT faculty reported more seminars and guest speakers on teaching statistics, and more sharing of materials on educational reform. 2YEAR instructors are more likely to enroll in mini courses and participate in other faculty development efforts to improve their teaching.

A large percentage of the faculty surveyed (82 to 90%) anticipate more changes to be made in the use of technology, and a majority also anticipate changes in teaching methods (60 to 65%). Fewer respondents project changes in course content (44 to 59%) or assessment (23 to 47%).

The results of this survey suggest that major changes are being made in the introductory course, that the primary area of change is in the use of technology, and that the results of course revisions generally appear to be positive, although they require more time of the course instructor. Results were surprisingly similar across departments, with the main differences found in the increased use of graphing calculators, active learning and alternative assessment methods in courses taught in math departments in two year colleges, the increased use of Web resources by instructors in statistics departments, and the reasons cited for why changes were made (more math instructors were influenced by recommendations from statistics education). The results are also consistent in reporting that more changes will be made, particularly as more technological resources become available.

While it is difficult to compare the content covered in the courses taught by the instructors surveyed, the textbooks used in these courses give an indication of the extent to which course content is more traditional or more in alignment with reform recommendations. The textbooks by Moore fall into the “reform” category and are the most frequently used books in introductory courses offered in statistics departments and in mathematics departments offering four year or advanced degrees. However, the favored textbook in mathematics departments in two-year colleges is Triola’s, which is considered to be a more traditional text.

To better understand the process of changing one’s course, and to provide a more detailed picture of what some “reformed” courses look like, the last phase of Garfield’s project was a case study of a select group of statistics instructors, representing the different types of departments and courses. A small group of teachers (n=14) were interviewed who appear to be teaching innovative courses incorporating reform recommendations. Interviews were conducted either by e-mail or by telephone. Instructors were asked to describe the key features of their introductory course, how it differs from a “traditional” course, the process that led them to develop their course, what types of support they received, and how the course will continue to be revised in the future.

The results reveal surprising differences from course to course and illustrate the complexities detailed in Schau’s model. Although all instructors were implementing some reform recommendations, the nature and extent of the implementation varied quite a bit, sometimes due to available resources, sometimes due to the characteristics of students at a particular institution, and often due to the instructor’s experience and beliefs about teaching.

When asked how their course differs from a traditional course, the responses included:

- I teach statistics as a language course, and try to help the students develop literacy about statistics.
- I have students keep journals of both statistical problems and reactions to the course.
- There is no memorization required of students. On exams, I give credit for effort and explanation.
- I use a mastery exam (scored but not graded), which students must pass, like a drivers’ test, before they are allowed to carry out a real statistical investigation.
- I co-teach the course with someone from a different discipline, and we often have arguments during class.
- I use lots of pairs and group work.
- I emphasize data production and simulation.
- Students have many opportunities for self-assessment.
- I create an interactive learning environment.
- I use two types of technology tools in my class; Minitab® for Homework and projects, Fathom for illustrating and developing concepts.
- I use the PACE model to create a highly interactive learning-centered classroom. PACE stands for Projects, Activities, Cooperative learning in a Computer-based classroom environments, and reinforcement through Exercises.

Despite the differences listed above, there was also a common theme among many instructors who stated that they focus more on concepts and big ideas and on data analysis and interpretation and less on computation, formulas, and theory.

The process that led these instructors to their current course often included conversations with other statistics educators, reading articles in the *Journal of Statistics Education* or listening to presentations at professional meetings, and trial and error testing of new techniques. Challenges faced along the way included lower teaching evaluations due to problems that arise when trying new techniques for the first time, the lack of rewards for effort applied to teaching (as opposed to research), student resistance to changing from passive to active learners (where more is demanded from them), and colleagues who want to see introductory statistics courses with more math, rigor, and probability.

The interviews revealed that these instructors have spent a great deal of effort thinking about their courses, and have dedicated huge amounts of time to improving and revising their courses. Although generally pleased with the results, most shared ideas they have for how they will continue to make changes and indicated that their courses are still being developed. Most feel that they are “moving in the right direction” but still “have a ways to go.” Some report that each time they teach it’s a different course. Others commented that the first time they taught a “reform course” was difficult but that they felt things would go better the next time.

One instructor commented on the changing student population who are working more hours, don’t read newspapers, and have less interest and motivation. In order to find topics that interest her students she’s “pretty much down to weather and cell phones and fast food.” Another remarked that “students struggled in the course but many learned a lot and were able to retain a fair bit. Often they didn’t appreciate that they were learning more until they saw how much other students struggled in later courses. Students often began to appreciate the prevalence of statistics in everyday life, and how much more cautious we should be using statistical statements and interpretations”. Some instructors have been pleased to see much better quality in student projects that are well written and use appropriate graphs and analyses. Others note increases in student satisfaction and attitudes about statistics. One instructor commented that “a large majority of my students see the course as a positive experience.”

A number of instructors indicated that they have been able to devote their time and effort to teaching because of having tenure and academic freedom. Some have enjoyed freedom to experiment with their course because no one in their department knows or cares about how they are teaching the courses (one instructor referred to this a ‘benign neglect’). A few instructors have appreciated the support of a department chair or colleagues or have received internal or external funding to support their efforts. A consistent result is that most of the faculty studied cite colleagues from outside of their institution as their main source of teaching support, particularly those they see at their professional meetings. One instructor commented: “I’ve received absolutely no support on campus. Although we teach many sections of introductory statistics at my university, the instructors never get together to share ideas or discuss problems. Consequently we don’t know what others do in the course.”

Despite the positive findings of reform recommendations being implemented, and instructors’ perceptions of positive outcomes, Garfield (2000) states the need for some high quality assessments to use to determine how well the “new” courses are preparing students to think, reason, and communicate, using statistics. An examination of assessment results might indicate that more changes are needed beyond the increased use of technology to achieve desired course outcomes. These assessments would also allow for comparison of the effectiveness of different activities and materials in helping students develop statistical thinking. So far, there is no published research that provides this data although there are some comparisons of different teaching experiments that use one or more of the suggested teaching methods (cooperative learning, for example, as discussed in Keeler and Steinhorst 1995 and Giraud 1997). A new grant from the National Science Foundation (Garfield, delMas, and Chance 2002) will develop an assessment instrument that may be used to assess statistical reasoning and thinking across many types of first statistics courses.

Garfield also suggests that more opportunities be offered to provide support for statistics instructors. These might include workshops like the Mathematical Association of America (MAA) "Statistical Thinking with Active Teaching Strategies" workshops of the last several years. Especially important are support groups like the “Isolated Statisticians” and their regional meetings, and the new MAA Special Interest Group for Statistics Education (formerly the Isolated Teachers of Statistics). Collegial support is often lacking at their own institutions, and faculty who are successfully teaching “reform” courses need more outlets through which to share information on what they are doing and how well it is working with students, and to provide detailed examples of student outcomes. Finally, Garfield recommends that more programs are needed to help prepare future teachers of statistics, particularly while they are in graduate school.

It is well known that mathematics departments teach about four times as many students in introductory statistics courses as do statistics departments (Loftsgaarden and Watkins 1998). Moreover, on average the mathematicians have kept their class sizes much smaller than those taught by statisticians, while many statisticians have somehow agreed to teach “megaclasses” of up to 500 students. Furthermore, it can be inferred from these facts and Garfield’s findings that there is a need for statistically trained statistics teachers. Some of the above findings raise the following concerns:

While many statistics instructors have readily adopted some of these reform suggestions, there are others teaching statistics, particularly in the biological and social sciences, business, and engineering as well as mathematics, who are harder to reach. We must find ways to inform and support these other teachers of statistics, so that they may learn about and implement ways to improve their courses. For example, many instructors outside of statistics departments are unaware of the many excellent Web resources, such as applets, that may be used to enhance student learning. We must also find a way to reach future teachers of statistics regardless of the department in which they receive their graduate training.

With very large classes, it is much more difficult to introduce a number of the changes suggested in this reform movement. Efforts like those of Zahn and Davis (1996) and Aliaga (1998) to develop activities and demonstrations for large classes need to be encouraged.

- Considering the needs of business, industry, and government, as well as shortages of statistics teachers, it is clear that we must encourage more students to become statistical scientists. We have devoted much effort to improving that first non-calculus course; yet we fail in convincing many of these students to enroll in a second statistics course or to become statistics majors. If a well taught course will help entice a student into a career, we need to make a strong effort to develop an attractive “second” course in statistics that will target both the students who have taken and enjoyed a “first” course and want to learn more statistics, and the students who have successfully completed an AP course in high school. Because math majors form a pool from which we get many of our majors and future statisticians, we also need to focus attention on the first statistics course for students with better mathematical backgrounds. We should pay close attention to the efforts of Allan Rossman, Beth Chance, and Karla Ballman, who have an NSF project to develop an activities-based course for these students. Another model to consider is a Chance course (see Rockmore and Snell, 1999) that builds on students’ mathematical background and emphasizes connections to statistics in everyday life (such as DNA fingerprinting and estimating risks).

The problems of teaching large classes, reaching out to other statistics teachers, and attracting students to becoming statistical scientists are extremely important concerns. In dealing with these problems, we need to remember that statistical thinking and understanding and appreciating the role of statistics in the scientific process and in life should be outcome goals in all of our introductory courses.

It is one thing to state that statistical thinking and reasoning should be the focus of a course, or should be the desired course outcomes. It is another matter entirely to achieve this, and to determine how well students are able to think and reason using statistical information upon completion of their first course, or later in their lives. We believe that appropriate content, a focus on data analysis and real problems, and careful use of high quality technological tools will help students better achieve the suggested course goals and outcomes (see appendix). However no one has yet demonstrated that a particular set of teaching techniques or materials will lead to the desired outcomes. Indeed, there is no consensus among the exemplary teachers interviewed in Garfield’s study about how the introductory course should be taught. Furthermore, it remains unclear what the essential components for teaching a successful course should be and how they should be implemented.

We think that research should be encouraged to investigate how to help students develop statistical thinking, and how to assess whether or not students have this ability. One possibility is to encourage statistics instructors to conduct high quality classroom research using their own students and classes. This type of research is best conducted in a collaborative manner, with other colleagues who teach introductory statistics (see delMas, Garfield, and Chance 1999). We should also look to the technique of “lesson study,” identified in the Third International Mathematics and Science Study, as an important contributor to Japanese students’ success in mathematics (Stigler and Hiebert, 1999) .

Lesson study is a collaborative process that involves a group of teachers who focus on a particular learning goal. The teachers meet regularly to discuss how to achieve this goal in their classrooms. They try out different techniques and evaluate how effective they are, observe each other teaching, and discuss these topics together on an ongoing basis. Applied to statistics education, a small group of statistics instructors might decide to focus on an aspect of statistical thinking, such as reasoning about variability. They would discuss different strategies to use in helping students develop this understanding, such as activities, simulations, or other techniques. They would also develop ways to assess whether or not students are able to reason about variability. Each instructor would try out the methods and assessments in their class, and perhaps have videos made. They would take detailed notes about how well the lesson went, where students seemed to be learning well or having difficulties, and where they might make changes. The group of instructors would then meet together to view the videos, review the assessment results, discuss what they learned, and make plans for how they would make changes in the future.

In closing, we suggest some specific ways that we can continue to study, and try to improve, the introductory course.

Encourage interaction and discussion of teaching the introductory course among our colleagues and engaging colleagues in collaborative classroom research. This might be done informally through brown-bag lunches and department seminars, or by gathering colleagues outside of the department (at the same institution) or from nearby institutions.

Continue organizing and teaching short courses and workshops focused on statistics education, and continue to share ideas, materials, and data sets via articles, presentations, and discussion lists. Try to reach out to colleagues in other departments who may not be aware of all these resources.

Begin an annual or biannual conference on “Making the Introductory Statistics Course More Effective,” like business statisticians do in their annual “Making Statistics More Effective in Schools of Business” meeting. There have been some small, regional conferences on this theme, but a larger, national conference should be explored. Encourage faculty to participate in this conference who can demonstrate that their teaching methods help students achieve desired course outcomes. Showing videos of their course in operation, bringing samples of student work, and engaging participants in an open discussion may help turn these sessions into more of a “lesson study” than a formal presentation.

Develop and disseminate high quality assessment instruments that better evaluate desired student outcomes such as statistical thinking and reasoning and attitudes and beliefs, and use these to determine how well our courses achieve these goals. Schau (personal communication) suggests a Web Assessment Warehouse as a mechanism for sharing instruments and collecting large amounts of student data from a variety of educational institutions. These data then can be used to study the impact on students of “reformed” courses.

Create seminars or training programs for graduate students in statistics, mathematics, and other disciplines, to prepare them to teach the introductory statistics course. These programs would introduce these future or potential college statistics teachers to reform recommendations for teaching the introductory course, thus providing opportunities for them to learn about appropriate course content, effective teaching and assessment techniques, and preferred types and uses of technology.

While we do not plan to prepare a syllabus for an introductory course, for there are many of them, we will list some topics that might be included. Here “1” represents highest priority, “2” second, “3” third, and “4” fourth in our opinion.

- 1. Recognizing that statistics surround us in everyday living. Reported statistics are sometimes incorrect or misused; thus it is important for each of us to be a critical consumer of statistics given by the media. We must ask questions about the quality of the data and the reliability of the analysis.
- 1. Understanding variability: bias, sampling error, systematic error, measurement error, and the regression effect. In particular, understanding: Actual observation = Fitted + Residual, and in statistics we try to detect the pattern (fitted) and describe the variation (residual) from that pattern.
- 1. Collection and summarization of data, including basic exploratory data analysis. (Some felt we should do more with writing up and explaining results.)
- 1. Graphs, including plotting data taken sequentially (that is, basic time-series concepts).
- 2. Sampling and surveys, including the importance of getting quality data.
- 2. Elementary designs of experiments, with some discussion about the ethics of experimentation and the distinction between observational and experimental investigating.
- 2. Formulation of problems and understanding the importance of operational definitions and the process of inquiry. That is, understanding the iterative nature of the scientific method, including the Plan-Do-Check [Study]-Act cycle. We want the capability to make and understand predictions. That is, statistics represents a process concerned with gaining knowledge and solving problems and is not a collection of isolated tools.
- 2. Basic distributions (such as the normal and binomial) as approximations to variability in data sets; that is, study modeling.
- 2. Correlation and regression and other measures of association. For example, there should be some illustrations of Simpson’s paradox.
- 3. Elementary probability, including trees and conditional probability (some wanted to include Bayes’ Theorem).
- 3. Central limit theorem and law of averages.
- 3. Elementary inference from samples, recognizing there are not unique answers in statistics.
- 3. Ability to use at least one statistical software package.
- 4. Outliers and how statistical measures change with various changes in the data (that is, aspects of robustness).
- 4. Statistical significance vs. practical significance.
- 4. Categorical data and contingency tables.
- 4. Simulation.

Aliaga, M. (1998), "Re-thinking Stat 101," Paper presented at the Fifth International Conference on Teaching Statistics, Singapore.

Bluman, A. G. (1997), *Elementary Statistics: A Step by Step Approach* (3rd ed), Boston: WCB/McGraw-Hill.

Butler, R. S. (1998), "On the failure of the widespread use of statistics," *Amstat News*, No. 251, 84.

Cobb, G. (1992), "Teaching Statistics," in *Heeding the Call for Change*, ed. L. Steen, MAA Notes No. 22, Washington: Mathematical Association of American, 3-34.

----- (1993), "Reconsidering Statistics Education: A National Science Foundation Conference," *Journal of Statistics Education* [Online], 1(1). (ww2.amstat.org/publications/jse/v1n1/cobb.html)

The Consortium for Mathematics and Its Applications (1989), *Against All Odds* (Video), South Burlington, VT: The Annenberg/CPB Collection.

delMas, R., Garfield, J., and Chance, B. (1999), "A model of classroom research in action: Developing simulation activities to improve students’ statistical reasoning," *Journal of Statistics Education* [Online], 7(3).
(ww2.amstat.org/publications/jse/secure/v7n3/delmas.cfm)

Freedman, D., Pisani, R., and Purves, R. (1998), *Statistics* (3rd ed), New York: W. W. Norton and Co.

Garfield, J. (1995), "How students learn statistics," *International Statistical Review*, 63, 25-34.

----- (2000), "An Evaluation of the Impact of Statistics Reform," Final Report for National Science Foundation project REC-9732404.

Garfield, J., delMas, R., and Chance B. (2002), "The Web-based ARTIST: Assessment Resource Tools for Improving Statistical Thinking" Project, pending National Science Foundation Grant.

Giraud, G. (1997), "Cooperative Learning and Statistics Instruction,"
*Journal of Statistics Education* [Online], 5(3). (ww2.amstat.org/publications/jse/v5n3/giraud.html)

Hoaglin, D. C., and Moore, D. S., (eds.) (1992), *Perspectives on Contemporary Statistics*, MAA Notes No. 21, Washington, DC: Mathematical Association of America.

Hogg, R. (1992), "Report of Workshop on Statistics Education," in *Heeding the Call for Change*, ed. L. Steen, MAA Notes
No. 22, Washington: Mathematical Association of American, 34-43.

Keeler, C. M., and Steinhorst, R. K. (1995), "Using Small Groups to Promote Active Learning in the Introductory Statistics Course: A Report from the Field," *Journal of Statistics Education* [Online], 3(2).
(ww2.amstat.org/publications/jse/v3n2/keeler.html)

Lock, R. (2000), "A Sampler of WWW Resources for Teaching Statistics," in *Teaching Statistics: Resources for Undergraduate Instructors*, ed. T. Moore, MAA Notes
No. 52, Washington: Mathematical Association of American, 191-197.

Loftsgaarden, D. O., and Watkins, A. E. (1998), "Statistics teaching in colleges and universities: Courses, instructors, and degrees in fall 1995," *The American Statistician*, 52, 308-314.

McClave, J. T., Dietrich, F. H., and Sincich, T. (1997), *Statistics* (7th ed), Upper Saddle River, NJ: Prentice Hall.

Moore, D. S. (1995), *The Basic Practice of Statistics*, New York: W. H. Freeman and Co.

----- (1997a), "New pedagogy and new content: the case of statistics," *International Statistical Review*, 65, 123-137.

----- (1997b), "Response," *International Statistical Review*, 65, 162-165.

Moore, D. S., and McCabe, G. P. (1999), *The Introduction to the Practice of Statistics* (3rd ed.), New York: W. H. Freeman and Company.

Rockmore, D., and Snell, J. L. (1999), "To Teach, per Chance To Dream,"
*Notices of the American Mathematical Society*, 46, 874-877.

Snee, R. (1990), "Statistical thinking and its contribution to quality," *The American Statistician*, 44, 116-121.

----- (1999), "Discussion: development and use of statistical thinking: a new era," *International Statistical Review*, 67, 255-258.

Stigler, J., and Hiebert, J. (1999), *The Teaching Gap*, New York: Free Press.

Triola, M. F. (1998), * Elementary Statistics* (7th ed), Boston: Addison-Wesley.

Velleman, P. F., and Moore, D. S. (1996), "Multimedia for teaching statistics: Promises and pitfalls," *The American Statistician*, 50, 217-225.

Weiss, N. A. (1999), *Introductory Statistics* (5th ed), Boston: Addison-Wesley.

Wild, C. J., and Pfannkuch, M. (1999), "Statistical thinking in empirical enquiry," *International Statistical Review*, 67, 221-248.

Zahn, D. A., and Davis, N. A. (1996), "Toward creating a learning community in large-section, introductory statistics courses," Poster presented at the Joint Statistical Meetings, Chicago.

Joan Garfield

Department of Educational Psychology

University of Minnesota

Minneapolis, MN 55455

USA

jbg@umn.edu

Bob Hogg

Department of Statistics and Actuarial Science

University of Iowa

Iowa City, IA 52242

USA

Candace Schau

CS Consultants

12812 Hugh Graham Road NE

Albuquerque, NM 87111

University of New Mexico

USA

cschau@unm.edu

Dex Whittinghill

Department of Mathematics

Rowan University

Glassboro, NJ 08028

USA

whittinghill@rowan.edu

Volume 10, Number 3, of the *Journal of Statistics Education* contains a
Letter to the Editor concerning this article.

THS, March 20, 2003

Volume 10 (2002) | Archive | Index | Data Archive | Information Service | Editorial Board | Guidelines for Authors | Guidelines for Data Contributors | Home Page | Contact JSE | ASA Publications