# Teaching an Advanced Methods Course to a Mixed Audience

Paul J. Roback
St. Olaf College

Journal of Statistics Education Volume 11, Number 2 (2003), ww2.amstat.org/publications/jse/v11n2/roback.html

Copyright © 2003 by Paul J. Roback, all rights reserved. This text may be freely shared among individuals, but it may not be republished in any medium without express written consent from the author and advance notification of the editor.

Key Words: Applied regression; Case studies; Conceptual understanding; Mathematics majors; Prerequisites.

## Abstract

I recently introduced an advanced statistical methods course into our curriculum with a two-tiered prerequisite system - students were required to have taken either an introductory statistics course or Calculus II. As a result, this course served as a first course in statistics for some quantitatively strong students and a follow-up course for others. I used a case study approach to introduce and motivate ideas to students new to statistics while engaging and challenging students for whom some ideas were review. Given constraints on resources which exist at smaller schools, a data-centered course such as this offered a good first experience in statistics for math students, one which piqued their interest and set a solid foundation for further study. In addition, the mixed audience led to an intellectually exciting class atmosphere for all students in the class. A quantitative assessment of students’ understanding of important statistical concepts is described to provide insight into whether or not students with no statistical experience can comprehend and apply basic ideas as well as if they had taken an introductory statistics class.

## 1. Introduction

"If you build it, will they come?" I found myself asking this question as I designed a brand new advanced statistical methods course to follow up introductory statistics at Connecticut College. Even if the course looked great on paper, would it translate into a classroom full of salivating students? Interest in a course like this was reportedly bubbling on a couple of fronts. A number of majors in fields such as psychology, economics, and the biological sciences, having enjoyed their first excursion into statistics or straining under the prospect of heavy quantitative demands for their senior theses, expressed interest in a second course in applied statistical methods. The advisors of these students also welcomed a new statistics course, since the burden of teaching statistical methodologies not covered in introductory statistics inevitably fell to them. In addition, mathematics majors were poised to sample a more applied offering. Math majors at Connecticut College often graduated without glimpsing the world of statistics, since they received no credit for taking introductory statistics and the only other option, mathematical statistics, was rarely taught before my arrival.

A natural prerequisite for Advanced Statistical Methods (MAT 207) would be a course in introductory statistics, either from me in the math department or from an instructor in another department. However, I chose to set a two-tiered prerequisite system, so that students were required to have taken either an introductory statistics course or Calculus II. In this way, I anticipated attracting students from both sources of bubbling interest, even if it meant accepting students with a certain degree of mathematical aptitude, but with absolutely no background in statistics. An immediate practical implication of this decision would be increased enrollment in the initial offering of MAT 207. At a small school such as Connecticut College (1800 students) and in a small mathematics department (6-9 majors per year), the risk of small enrollments in upper level courses (especially brand new ones in areas that have not been traditionally popular) is nontrivial. Thus, a two-tiered system of prerequisites would have the practical effect of filling more seats, creating more momentum for both the course and a recently approved minor in applied statistics. However, the choice of prerequisites had ramifications for both the Calculus II students and for the Advanced Statistical Methods class as a whole, so it was not a decision I made impulsively or based on practical concerns alone. For the Calculus II students, it meant quickly developing an understanding of key statistical concepts normally emphasized over an entire semester in an introductory statistics course. For the class as a whole, it meant a slightly slower pace at the beginning and a potentially awkward mix of statistical and mathematical sophistications.

Thus, it was only after carefully considering pedagogical issues in addition to practical ones that I concluded that a two-tiered prerequisite system was reasonable and even advisable. In this paper, I will describe my experiences with an Advanced Statistical Methods course - motivations and rationale for using the two-tiered prerequisite system, ideas from the literature about first statistics courses, the place of this course in our college’s curriculum, course objectives I set, quantitative comparisons between the student populations, and, finally, impressions about whether or not the mixed audience was advisable.

## 2. Suggestions from the Literature

### 2.1 First Statistics Courses for Quantitatively Strong Students

Even before designing MAT 207, I found myself asking: What should be a mathematics student’s first course in statistics? Related to this, we could ask: Should mathematics students take and be given mathematics credit for a typical course in introductory statistics? Are mathematics students innately able to understand basic but essential statistical concepts more quickly than others? Without introductory statistics, is a student’s knowledge gained from an advanced methods course somehow incomplete?

Although not a great deal has been written about mathematics students and introductory statistics, Cobb and Moore (1997) do address this question directly. Although a theoretical offering in mathematical statistics often serves as a math major’s first statistics course, Cobb and Moore (p. 821) argue that a data-oriented course which introduces statistical ideas and applications is preferable, provided that it "move quickly enough to present genuinely useful statistics and serious applications." As Moore wrote earlier (1992, p. 14), "Statistics has its own substance, its own distinctive concepts and modes of reasoning. These should be at the heart of the teaching of statistics to beginners at any level of mathematical sophistication." Thus, the question is not if a mathematics student interested in statistics should be exposed to essential basic concepts, but rather how and when. Cobb and Moore envision a first statistics course following probability - some traditional mathematical statistics integrated with data analysis and data production - but admit there is no consensus on how to best introduce quantitatively strong students to statistics. In fact, their final line (p. 822) instructs readers, "This is your take-home exam: design a better one-semester statistics course for mathematics majors."

Similarly, Iversen (1992) describes three levels of introductory courses depending on the mathematics background of the students; all introductory courses would "expose the students to the central aspects of statistics" (p. 39), but other objectives and activities would depend on the mathematical rigor possible. Whereas I am uncomfortable with Iversen’s notion of starting qualified students in a traditional mathematical statistics course, I think that his notion of a faster-paced, more in-depth introductory statistics class for undergraduates who are well-prepared mathematically merits consideration. Teaching an "advanced" version of an introductory course is not a novel idea, since it is used in many institutions for courses such as Calculus and physics. However, this splitting of the introductory statistics audience is not often possible in many smaller schools.

Some commentary can also be found in position papers prepared recently for the Undergraduate Statistics Education Initiative (USEI). In their position paper on first courses in statistics, Garfield, Hogg, Schau, and Whittinghill (2002, paragraph 40) mention that "we also need to focus attention on the first statistics course for students with better mathematical backgrounds." In addition, they argue that one goal of our courses should be "to build strong positive attitudes towards statistics" (paragraph 7). While not making specific recommendations, they point to an National Science Foundation project of Rossman, Chance, and Ballman, and to the Chance project (Rockmore and Snell 1999) as possible models for such a first course in statistics. In another USEI position paper on curriculum guidelines, Tarpey, Acuna, Cobb, and De Veaux (2002, paragraph 14) state that "there are successful majors that do not require the usual sort of introductory course. An applied regression course is recommended as an excellent vehicle for teaching core material." Thus, both USEI working groups seem to suggest that the current common model for introductory statistics courses, while it has come far over the past decade and has proven relatively successful, may not be the only model or even the best model for students with better mathematical backgrounds.

Perhaps the best guidance for a first course in statistics for quantitatively strong students comes from the "Report of the CRAFTY Workshop on Statistics" (see Moore 2001). Responding to an invitation from the Calculus Reform And the First Two Years (CRAFTY) subcommittee of the Mathematical Association of America (MAA), a variety of statisticians commented on the role of statistics in the mathematics curriculum and the role of mathematics in preparing students to study statistics. Workshop participants endorsed the 1991 Committee on the Undergraduate Program in Mathematics recommendation that every mathematical sciences major should study statistics, although they advocated relaxing the Calculus prerequisite stated in the original recommendation. They argued (p. 6) that a required course for all mathematical sciences majors should adhere to the fundamental principles set out by Cobb (1992) on behalf of the American Statistical Association/Mathematical Association of America (ASA/MAA) joint committee:

1. More data and concepts, fewer derivations and recipes; automate calculations using a modern statistical package.

2. Emphasize statistical thinking: the omnipresence of variability and the importance of data production.

3. Foster active learning: student projects, group work, activities, writing, and oral presentations.

The highlight of this report is the collection of seven diverse course descriptions presented in the appendix - actual or planned courses which embrace and illustrate the report’s recommendations regarding a first statistics course for mathematics majors. These courses - ranging from experimental design to time series to Bayesian statistics to archaeometrics - vividly and concretely portray the variety of creative and interesting ways that mathematics students can be successfully introduced to statistics. In general, this report provides much food for thought and should be disseminated to all undergraduate statistics educators.

### 2.2 Course Objectives in Introductory Statistics

Another way to tackle the prerequisite question for MAT 207 is to consider the objectives of an introductory course in statistics. Once these goals are identified, we could ask: Is it reasonable to expect that a student could achieve the goals commonly set for an introductory statistics course by taking MAT 207, without the benefit of a previous course in statistics?

Over a decade ago, participants in a workshop on statistical education came up with the following (Hogg 1992, p. 8) "Our aim in a first course is to develop critical reasoning skills necessary to understand our quantitative world. The focus of the course is the process of learning how to ask appropriate questions, how to collect data effectively, how to summarize and interpret that information, and how to understand the limitations of statistical inferences. Statistical thinking is central to education." Based on this, workshop participants listed important topics to be covered in an introductory course (Hogg 1992, p. 10), the highest priorities falling on: recognition that statistics surround us every day, understanding of variability and error, exploratory data analysis, and graphs.

Other authors (Gal and Ginsburg 1994; Chance 1997; Gal and Garfield 1997; Hoerl, Hahn, and Doganaksoy 1997; Moore 1997) mention enthusiasm, interest, and communication skills right along with development of statistical literacy and thinking as desired outcomes from an introductory statistics course. In fact, Gal and Ginsburg (1994) show that attitudes, expectations, and motivations play a role in a student’s success in a statistics course and his or her impression of the discipline of statistics. With this in mind, I believe there are two dangerous scenarios which could emerge from blindly herding all mathematics students who express an interest in statistics into existing introductory statistics courses. In one scenario, the student processes the ideas being presented at a faster rate than most of the class. In that scenario, the student might obtain excellent mastery of the basic ideas in statistics and perform well in the course, but because the pacing of the course was geared to other students, he or she might emerge with an impression of statistics as "soft" or "easy" and not stimulating enough for future study. In a second scenario, the student mistakenly views statistics in terms of the mathematical content only and fails to appreciate and master the non-mathematical aspects of the discipline of statistics. In this case, the student’s performance falls short of expectations, and he or she can develop an antagonistic attitude toward statistics. Assuming a goal of any introductory course is to help students appreciate a subject and stimulate students toward further exploration in that field, neither scenario I outlined is desirable. Yet, I have had experiences with students fitting both scenarios.

Thus, the primary objectives of an introductory course in statistics run from the concrete (summarizing data graphically and numerically; designing studies; fitting models) to the more abstract (understanding ideas such as variability, random sampling, and sampling distributions; thinking about causation and generalizability; promoting enthusiasm about and appreciation for statistics). I hoped that my course would lead students new to statistics toward achievement of these objectives, while moving the entire class beyond the usual stopping point for content in introductory statistics. In Section 4, I offer some assessment of students’ achievement of these objectives.

## 3. The Course

I formed the syllabus for "MAT 207: Advanced Statistical Methods" after some consultation with faculty members from several departments who were teaching statistics courses or doing quantitative research. As stated in my syllabus, course objectives were:

1. To develop "statistical literacy," a working understanding of basic elements of statistics that can help in critically evaluating data-driven results in your field of interest and in your everyday lives.

2. To obtain a rich set of statistical tools for data analysis, with an understanding of the underlying mathematical models.

3. To enable you to confidently and carefully interpret the results of data analyses and clearly communicate those results.

4. To receive practical experience in probing real sets of data addressing meaningful research questions.

5. To explore how one might design studies to collect appropriate data for addressing research questions.

The syllabus for MAT 207, along with handouts, assignments, exams, and other course materials are available at ww2.amstat.org/publications/jse/v11n2/roback/mat207.html.

Students who had taken any of the three flavors of introductory statistics on campus were able to enroll in MAT 207. I’ll provide a brief description of these three flavors. First, Introduction to Statistics (MAT 107) is taught in the mathematics department (by me) to an audience consisting primarily of majors in the biological sciences and human development, and students seeking to fulfill a general education requirement. The textbook is The Basic Practice of Statistics by Moore (2000); SPSS is used extensively in weekly computer labs, homework assignments, a group project, and even on exams; and topics include graphical and numerical summaries of data through analysis of variance and inference for regression. Second, Psychological Statistics (PSY 201) is required of all psychology majors. This course typically contains over 100 students (by comparison, enrollment is capped at 30 in MAT 107). Although the syllabus of PSY 201 is similar to that of MAT 107, constraints imposed by the size of PSY 201 lead to less hands-on analysis of data, and exams which feature more memorization and rote application of ideas. Third, Econometrics I (ECO 230) is required of all economics majors. Its syllabus is also similar to MAT 107 until the end of the course, where two or three weeks of multiple regression and time series methods are substituted for analysis of variance and inference for categorical data. In addition to the three courses above, students who scored a 4 or 5 on the AP Statistics exam during high school also satisfied the prerequisite for MAT 207. Finally, under the two-tiered prerequisite system, students with no previous experience in statistics but with experience in math through Calculus II were also able to enroll in Advanced Statistical Methods.

As a result of the two-tiered prerequisite system, Advanced Statistical Methods was designed to be (a) a second course in applied statistical methods for non-math majors, and (b) a first course in statistics for math majors and minors. For non-math majors, MAT 207 was an elective chosen by students who had enjoyed introductory statistics and wished to build on that knowledge, often with an eye toward independent research in their field or graduate work. For math majors and minors, MAT 207 served as an elective which counted toward the major or minor. In addition, for majors in any department, MAT 207 could lead to a recently approved minor in Applied Statistics. To earn this minor, students must take five qualifying courses, at least one of which must be outside the mathematics department. Each prospective minor must complete at least one of the two two-course "core sequences": Advanced Regression Techniques (MAT 207 - more on the name change later) and Design and Analysis of Experiments (MAT 208), or Probability (MAT 316) and Mathematical Statistics (MAT 317). Other courses which can count toward the minor include introductory statistics, Econometrics II, Research Methods in Psychology, Advanced Psychological Statistics, three different offerings in the biological sciences (Genetics, General Limnology, and Ecology of Terrestrial and Wetland Plant Communities), Calculus I (except for math majors and minors), and approved independent studies.

I chose to teach MAT 207 from The Statistical Sleuth by Ramsey and Schafer (2002), a book organized around case studies with a relatively low mathematics background assumed. Other books could have been chosen, but I believe that the case study approach was central to guiding a mixed audience of students through topics in applied regression. Advantages of the case study approach and experiences with it have been detailed elsewhere (see, for example, Love 1998; Peck, Haugh, and Goodman 1998; and Radke-Sharpe 2000). In general, I favor introducing topics through thoughtful and interesting examples; case studies go beyond topic introduction and motivation, acting as fertile topsoil from which seeds of inquiry bloom into full-formed knowledge. In addition, following a case study from background to research hypothesis to statement of conclusions provides a perfect opportunity for students to generate statistical reports. These reports help students’ communication skills grow, especially as they learn to navigate between verbal and mathematical expressions of research questions.

Naively following recommendations in the preface to The Statistical Sleuth, I planned to cover two-sample problems, several-sample problems, simple linear regression, multiple regression, two-way analysis of variance, and logistic regression, hence my generic title for the course. As it turned out, the list of topics proved to be a bit ambitious, and the course became more of a regression methods course, with logistic regression for binomial counts and nonparametrics (rank-based and permutation methods for two independent samples) replacing several-sample problems and two-way analysis of variance in the list of topics. In fact, the name of this course has been officially changed to Advanced Regression Techniques.

Students were assessed via weekly homework assignments, two take-home examinations, and a group project. The homework assignments related to SPSS analyses of real sets of data, which we began in weekly computer lab sessions; students answered specific quantitative and interpretive questions and also wrote statistical reports to convey conclusions about research questions. Through the weekly homework assignments, computer labs, statistical reports, and group projects, course objectives 2 through 4 were addressed to a good extent. Course objectives 1 and 5, however, dealing with statistical literacy and study design, were stressed through other means. For instance, I used the case studies as springboards for probing key statistical ideas: To what population can these results be extrapolated? Could this observational study have been conducted as an experiment? What does a p-value of 0.075 mean precisely, and how does it relate to a sampling distribution? How can we express our research hypothesis as a mathematical model? In addition, we paused during the middle of each class period for an installment of "Halftime Statistics," in which we discussed statistical issues recently appearing in the mass media or in subject-specific journals. Not only did this halftime break give students a chance to recharge for the end of the 75-minute class period, but it motivated discussions of design issues, adequate controls, scope of inference, practical significance, and more.

I designated the first five weeks of MAT 207 for studying independent samples and simple linear regression. By covering that ground, we could then proceed into multiple regression, a topic that was new to most of the students in the class, by the sixth week. The first five weeks were therefore critically important, and this is where the case study approach allowed me to lead diverse groups of students to the same place. For students who had taken introductory statistics, these first five weeks provided a much-needed review of statistical ideas. In addition, these early weeks allowed students to gain familiarity with several important aspects of the course: the textbook and its case study approach; statistical reports as a way of summarizing data-driven conclusions about research questions; and tangential ideas motivated by the rich and unfiltered examples found in The Statistical Sleuth (such as interpreting results after log transforming data and thinking about the scope of inference). For students with no statistics background, the case studies comparing two independent samples provided motivation for discussing essential elements of statistics, such as graphs, descriptive statistics, variability, distributions, randomness, hypothesis tests, and confidence intervals. Thus, the same examples which introduced and motivated ideas to students who were new to statistics captivated and engaged students for whom the ideas were a review. Admittedly, students uninitiated in statistics needed to work hard in the first five weeks to digest the new material and unfamiliar concepts placed before them, but the students were forewarned and, I believe, were up to the challenge. Nevertheless, an important question remained unresolved: Was MAT 207 an adequate or even an optimal place for mathematically strong students to receive their introduction to statistics?

## 4. Course Assessment

Assessing the success of MAT 207, given its diverse audience with heterogeneous needs, is a multi-layered problem. This section summarizes data collected from three sources - traditional end-of-the-semester student evaluations, an end-of-the-semester questionnaire on statistical concepts, and surveys administered one year after completion of the course.

### 4.1 End-of-the-Semester Student Evaluations

Student evaluations were administered by the Math Department’s Student Advisory Board during the last week of the semester. There seemed to be evidence that, across-the-board, MAT 207 students truly enjoyed their experience in Advanced Statistical Methods. On a four-point scale, the average grade given to the course by the students was 3.7, and the average instructor grade 3.9. In terms of the pace of the class and the difficulty of the material, 9 of the 14 thought it was just right, while 3 students thought the class was a bit too hard and 2 thought it was a bit slow. The mix of theory, application, computer work, and discussion seemed appropriate, since 11 of the 14 students rated the organization of class time as excellent, while the other 3 rated it as very good.

Written comments praised the "real world applications," the "usefulness of the material in the future," and the "integration of mathematical thought and practical significance." The words "fun" and "interesting" appeared in many evaluations; several students stated their desire for "another advanced course." The case studies were positively received; for instance, one student wrote that the case studies "clearly illustrated the topics and were interesting in and of themselves." The few negative comments were focused primarily on the textbook, which was considered "confusing," "hard to follow," "too mathematical," and "dull and boring." In addition, two students mentioned that they would have liked more mathematical detail.

### 4.2 Questionnaire on Conceptual Understanding

In order to create an assessment framework, Garfield (1994) creates the following five categories of goals for an introductory statistics course: concepts, skills, applications, attitudes, and beliefs. I felt confident that MAT 207 students taking statistics for the first time would perform at least as well in most areas as my introductory statistics students. Even more than my introductory statistics classes, MAT 207 stressed case studies with messy, real data; data analysis in an open, explorative atmosphere; and interpretation and communication of statistical ideas through reports. In addition, as described in the previous section, students enjoyed their experience in MAT 207 and appreciated their improved abilities in practical problem solving. Thus, it was Garfield’s first category - concepts - which gave me most pause. Were students who jumped right into MAT 207 able to understand and apply important ideas in statistics as well as if they had taken an introductory statistics class? By experiencing a condensed development of ideas such as random sampling and sampling variability, does a student obtain a fuzzier mastery of these concepts?

To partially examine the relationship between previous courses taken and conceptual understanding, I administered a questionnaire on statistical concepts to my MAT 207 students during the last week of classes. A closed-book, unannounced quiz (which I assured the class would not count in their class grade, but which I implored them to answer thoughtfully), it asked students a series of 12 short-answer questions (see Appendix). Each question was scored on a 5-point scale in blinded fashion (that is, quizzes were only identified by student number until after scoring was completed). In addition to looking at total scores, I a priori divided the questions into four categories:

1. general knowledge (center, spread, elements of statistics, types of studies),
2. distributions (sampling, t-distributions, histograms),
3. general inference (p-values, confidence intervals), and
4. method-specific ideas (independent samples, simple regression).

Categories 1 and 2 contained concepts which I spent a good deal of time discussing, especially early in the course, but which I did not repeatedly stress on exams or homework assignments. I separated the two categories because Category 2, the notion of distributions, historically produces confusion in introductory statistics students. Intuitive inferential notions in Category 3 were hammered home throughout the semester across all statistical methods, while the ideas in Category 4 were central to specific methods covered in MAT 207.

For evaluating performance on the Statistical Concepts Quiz, I divided my class (14 students) into three groups - those meeting the Calculus II prerequisite but with no introductory statistics, those meeting the introductory statistics prerequisite but with no math beyond Calculus I (if that), and those meeting both prerequisites. A sketch of each student is provided in Table 1. One student was left out of the analysis, since this student missed a stretch of time due to an accident and never really caught up to the class.

Table 1. Profiles of MAT 207 students.

 Prerequisites Met ID Year Intro Stats Major (Minor) Last Math Course Calculus II only 578a1012 JrJrJrSoSo nonenonenonenonenone MathPre-Med (Math)MathGovernment (Math)Economics, Math beyond Calculus IIbbeyond Calculus IIbeyond Calculus IIbeyond Calculus IIbeyond Calculus II Intro Stats only 234 Fr.So.Jr. AP StatsPsychologyPsychology English, BotanyPsychologyPsychology none at Connnone at Connnone at Conn Both 169111314 Jr.Sr.Sr.Sr.Sr.Jr. Math, EconEconomicsPsychologyEconomicsEconomicsMath, Econ Pre-Med, EconEconomics (Math)PsychologyEconomics, MathEconomics (Math)Economics (Math) Calculus IIbeyond Calculus IICalculus IIbeyond Calculus IIcbeyond Calculus IIbeyond Calculus II

a Dropped from analysis.
b Took Probability.
c Took Probability and Mathematical Statistics.

From the small class size to the lack of baseline data and controls to the testing instrument itself, my evaluation process is an admittedly imperfect one (which I will discuss more fully later), but it still provides some quantitative data on the ability of students to understand important statistical concepts and whether or not this understanding is related to prerequisites met. A summary of the Statistical Concepts Quiz results is presented in Table 2; to calculate "median average score," I found the average score over all questions in a single category for each group member, and then I recorded the middle score of these averages to minimize the impact of outliers.

Table 2. Summary of Statistical Concepts Quiz results.

 Median Average Score (Scale: 0-5) Calculus II only (n = 4) Intro Stats only (n = 3) Both Prerequisites (n = 6) General KnowledgeDistributionsGeneral InferenceMethod-Specific 2.82.35.04.2 3.53.34.04.7 3.83.04.34.5 Total 3.3 3.6 3.9

Among students in my Advanced Statistical Methods class, those without previous statistical experience typically showed slightly lower levels of overall conceptual understanding (median average score of 3.3) compared to those with previous statistical experience (3.6 if no Calculus II, 3.9 if Calculus II). Since outliers are not problematic in total scores, one could also examine mean average scores, which show even smaller differences among the three groups (3.4 for Calculus II only, 3.5 for introductory statistics only, and 3.7 for both prerequisites met). The biggest gaps in conceptual understanding between those with and without statistical experience occurred among General Knowledge and Distribution questions - the two areas covered early in MAT 207 but not pointedly stressed throughout the course. These gaps could be explained by the difference in seeing a concept for the second time rather than the first time, but they may also point to things I may want to emphasize to a greater extent in future installments of MAT 207. In teaching MAT 207, I focused more constant attention on ideas from the other two categories - General Inference and Method-Specific questions - throughout the semester. Possibly as a result, those without previous statistical experience performed as well in those two categories as students with a previous statistics course. Obviously, with such limited data, no grand general conclusions can be drawn. I find evidence in these data that students still attain a reasonable understanding of essential statistical ideas even when skipping introductory statistics and jumping into Advanced Statistical Methods.

### 4.3 Surveys One Year Following Completion of the Course

I tried to reach all 14 members of my MAT 207 class through an email survey administered one year following the completion of the course. The short survey was designed to gauge feelings about the course upon reflection, and to probe issues of repetition of material from previous courses, disadvantages from seeing statistical concepts for the first time, and the challenges and usefulness of the course. I received replies from 8 students, including 3 of 4 from the "Calculus only” group and all 3 from the "Intro Stats only” group. Even in retrospect, students rated MAT 207 as an enjoyable experience - on a 10-point scale, the median enjoyment rating was 8 overall (median score of 9 for the "Calculus only” group versus 8 for the "Intro Stats group”). In addition, the median rating of the difficulty of the material (with 10 most difficult) was 6.5 overall (7 for "Calculus only” versus 7 for "Intro Stats only”), and the median rating of the workload (with 10 the heaviest) was 7 overall (7 for "Calculus only” versus 7 for "Intro Stats only”). Thus, student preparation appeared to have no numerical effect on both the enjoyment and the perceived difficulty of the class.

I also asked those without previous statistical experience specifically about their crash course into statistical thinking over the first few weeks. All respondents agreed that "the early introduction to basic statistical concepts was sufficient” and the overall pace was good, but two students reported feeling a slight disadvantage early on. One respondent said, "it took me some time to understand the relevance of some of the tests … instead of just knowing how they were created” and suggested "more background for non-stats students.” On one hand, this comment makes me feel good that students in MAT 207 realize that statistics is more than rote plug-and-chug, but it also reminds me that this higher-level understanding that I demand of my students takes time to sink in, especially for newcomers to the discipline. Among students with previous statistical experience, all reported being "glad to have taken a stats course first," yet no one found too much repetition from previous courses. Finally, the two students who had fulfilled both prerequisites offered interesting commentary. These two respondents found MAT 207, with its case studies and its "learn-by-doing as opposed to learn-by-learning” to be much different than other math courses they had taken - "more interesting," "more real world connections," and "one of the few math classes that actually had personality.”

## 5. Discussion

Will I keep the same prerequisites and teach to a mixed audience when I teach Advanced Statistical Methods (now Advanced Regression Techniques) again? My answer is "Yes!" The case study approach of The Statistical Sleuth, along with its well-chosen examples and thoughtful presentation, provides a nice way to proceed through the course topics, allowing material that is brand new for some students to still be fresh and exciting to others. All students become more fluent at moving between statistical and verbal expressions of research questions, sharpen their interpretive skills, and expand their methodological toolbox. Plus, I must consider the practical situation I find myself in at Connecticut College, where enrollment and teaching resources would not support either a separate section of introductory statistics for math majors or a second-level statistics course just for those who have completed introductory statistics.

I hadn’t anticipated (but was excited to see) the improved quality of interaction in class sessions resulting from a mixed audience. I found that class questions operated on many different levels, all of which enhanced the learning atmosphere in the class. Those with no previous statistics experience often wanted clarification on mathematical details and statistical concepts, their perceptive questions helping to solidify those ideas for the rest of the class. Those with previous statistics experience tried to connect class material with research they were conducting, other problems they had worked on, or other courses they had taken, and they enjoyed proposing alternative ways to attack research questions presented in class. Finally, those with both previous statistics experience and a good mathematics background inquired about deeper mathematical ideas underlying methods we were discussing, allowing me to preview ideas developed in other courses (such as Design and Analysis of Experiments, Probability, or Mathematical Statistics) or which are the subjects of current research. For example, we touched on topics such as conditional probabilities, Bayesian statistics, generalized linear models, response surfaces, cross-validation, likelihood ratio tests, and the quasi-likelihood approach for extra-binomial variation.

So did I really complete the take-home exam of Cobb and Moore and "design a better one-semester statistics course for mathematics majors?" Well, I fudged a bit, adjusting a second course in statistical methods to accommodate quantitatively strong students with no statistical background. But given my constraints on faculty resources and student enrollment, I believe that I have created a good first experience in statistics for math students, one that has piqued their interest and set a solid foundation for further study. Of course, like most good take-home exam questions, Cobb and Moore’s question encourages active and insightful thought, and it undoubtedly has more than one "right" answer. I believe that the time is right in statistical education to place greater focus on quantitatively strong students; we need consensus on what should characterize their first experience in statistics. Even more, we need to develop, implement, and evaluate working models. Hopefully this is but one voice in a soon-to-be vibrant conversation.

## Appendix: Statistical Concepts Quiz [Category in Brackets]

1. Why is it important to measure variability? How do we measure variability? [General Knowledge]

2. How does one decide between the mean and the median as a measure of central tendency? [General Knowledge]

3. When do we use a histogram? What can we learn from it? [Distributions]

4. Consider the most recent problem from homework - comparing time until a baby first walks for an active exercise group and a control group. Identify the population, parameter, sample, and statistic in this problem (there may be more than one). [General Knowledge]

5. State advantages and disadvantages of experiments versus observational studies. [General Knowledge]

6. Describe sampling variability and a sampling distribution in terms of an example. (Or, if you’re a bit fuzzy on that, just describe what the distribution of a random variable is.) [Distributions]

7. What does a p-value attempt to measure? [General Inference]

8. If we form a 95% confidence interval, what do we have 95% confidence about? [General Inference]

9. We see the t-distribution all over the place in CI’s and significance tests - what’s up with that? What is the t-distribution, what is it used for, and why is it used so often? [Distributions]

10. When we’re performing an independent-samples t-test, how might we know if assumptions have been violated? [Method Specific]

11. What is the method of least squares? [Method Specific]

12. What is a residual, and why do we make residual plots in regression analyses? [Method Specific]

## Acknowledgements

The author gratefully acknowledges the effort and enthusiasm of the MAT 207 class in providing the inspiration for this paper. In addition, the author thanks Karen, a great teacher who offered careful and thoughtful readings of early versions of this paper despite her baby-induced sleep deprivation, and the associate editor and two referees for their helpful comments and suggestions.

Parts of the paper were presented at the Joint Statistical Meetings in New York City on August 15, 2002, in a session entitled "Exemplary Data Sets and Case Studies for Teaching."

## References

Chance, B. L. (1997), "Experiences with Authentic Assessment Techniques in an Introductory Statistics Course," Journal of Statistics Education [Online], 5(3). (ww2.amstat.org/publications/jse/v5n3/chance.html)

Cobb, G. (1992), "Teaching Statistics," in Heeding the Call for Change: Suggestions for Curricular Action, ed. L. Steen, MAA Notes No. 22, Washington, DC: Mathematical Association of America, pp. 3-43.

Cobb, G., and Moore, D. S. (1997), "Mathematics, Statistics, and Teaching," The American Mathematical Monthly, 104, pp. 801-823.

Gal, I., and Garfield, J. B. (eds.) (1997), The Assessment Challenge in Statistics Education, Amsterdam, The Netherlands: IOS Press.

Gal, I., and Ginsburg, L. (1994), "The Role of Beliefs and Attitudes in Learning Statistics: Towards an Assessment Framework," Journal of Statistics Education [Online], 2(2). (ww2.amstat.org/publications/jse/v2n2/gal.html)

Garfield, J. B. (1994), "Beyond Testing and Grading: Using Assessment to Improve Student Learning," Journal of Statistics Education [Online], 2(1). (ww2.amstat.org/publications/jse/v2n1/garfield.html)

Garfield, J., Hogg, B., Schau, C., and Whittinghill, D. (2002), "First Courses in Statistical Science: The Status of Educational Reform Efforts," Journal of Statistics Education [Online], 10(2). (ww2.amstat.org/publications/jse/v10n2/garfield.html)

Hoerl, R., Hahn, G., and Doganaksoy, N. (1997), "Discussion: Let’s Stop Squandering Our Most Strategic Weapon," International Statistical Review, 65(2), pp. 147-153.

Hogg, R. V. (1992), "Towards Lean and Lively Courses in Statistics," in Statistics for the Twenty-First Century, eds. F. Gordon and S. Gordon, MAA Notes No. 26, Washington, DC: Mathematical Association of America, pp. 3-13.

Iversen, G. R. (1992), "Mathematics and Statistics: An Uneasy Marriage," in Statistics for the Twenty-First Century, eds. F. Gordon and S. Gordon, MAA Notes No. 26, Washington, DC: Mathematical Association of America, pp. 37-44.

Love, T. E. (1998), "A Project-Driven Second Course," Journal of Statistics Education [Online], 6(1). (ww2.amstat.org/publications/jse/v6n1/love.html)

Moore, D. S. (1992), "Teaching Statistics as a Respectable Subject," in Statistics for the Twenty-First Century, eds. F. Gordon and S. Gordon, MAA Notes No. 26, Washington, DC: Mathematical Association of America, pp. 14-25.

----- (1997), "New Pedagogy and New Content: The Case of Statistics," International Statistical Review, 65(2), pp. 123-137.

----- (2000), The Basic Practice of Statistics (2nd ed.), New York: W. H. Freeman and Company.

Moore, T. (2001), "The Curriculum Foundations Workshop in Statistics," MAA Online [Online], The Mathematical Association of America.
(www.maa.org/features/cf_statistics.html)

Peck, R., Haugh, L. D., and Goodman, A. (1998), Statistical Case Studies: A Collaboration Between Academe and Industry, Philadelphia, PA: Society for Industrial and Applied Mathematics, and Alexandria, VA: American Statistical Association.

Radke-Sharpe, N. (2000), "Curriculum in Context: Teaching with Case Studies in Statistics Courses," in Teaching Statistics: Resources for Undergraduate Instructors, ed. T. L. Moore, MAA Notes No. 52, Washington, DC: Mathematical Association of America, pp. 35-40.

Ramsey, F. L., and Schafer, D. W. (2002), The Statistical Sleuth: A Course in Methods of Data Analysis (2nd ed.), Pacific Grove, CA: Duxbury.

Rockmore, D., and Snell, J. L. (1999), "To Teach, per Chance To Dream," Notices of the American Mathematical Society, 46(8), pp. 874-877.

Tarpey, T., Acuna, C., Cobb, G., and DeVeaux, R. (2002), "Curriculum Guidelines for Bachelor of Arts Degrees in Statistical Science," Journal of Statistics Education [Online], 10(2). (ww2.amstat.org/publications/jse/v10n2/tarpey.html)

Paul J. Roback
Department of Mathematics
St. Olaf College
1520 St. Olaf Ave.
Northfield, Minnesota 55057
USA
roback@stolaf.edu