Leon Pritzker (LEPRIT@JUNO.COM) wrote as follows:

Dear Dr. Gardenier,

I offer the following comments on “Ethical Guidelines for Statistical Practice” published in the May, 1998 issue of Amstat News.

These comments deal with the limits of competence (Expertise would perhaps be a more appropriate term) of the statistician, qua statistician. If a statistician is competent in the subject matter field with which his or her study deals, the only ethical concerns for me deal with the thorny issues of what constitutes competence and how it is to be determined.

If, however, a statistician is not competent in the subject matter field with which the study deals and is not collaborating with a specialist in the subject matter, then I have a serious ethical concern. Subject matter incompetence is, I suspect, most serious in fields closely connected to statistics, notably Demography, Economics, Political Science and Marketing/Advertising. At any rate these are fields in which I have observed statisticians working without collaboration beyond their own competence.

The underlying ethical principle is relatively easy to state, but, I suspect, exceedingly difficult to implement:

Statisticians should not go beyond their competence in any aspect of a
statistical study, including problem definition, design, implementation,
analysis, and interpretation.

One of the most formidable problems faced by a statistician is the understanding of the process of measurement in any field. Every process has its own set of assumptions, techniques of calibration and skill levels. In addition there are the two problems of relevance and validity. The statistician, qua statistician, has no competence to deal with these problems without collaboration, and therefore should not.

The fact that, in may cases, the measurement processes in the “softer” sciences are vague and ill-defined does place a responsibility on the part of the statistician to call such problems to the attention of the subject matter specialists. However, the statistician is on dangerous ground when he or she takes over the measurement process.

I recall that in my discussions on ethical issues with W. E. Deming many years ago, he took an even tougher line than I am now proposing. As several of my former Indian statistical colleagues have told me, one measure of the success of statisticians is how narrowly they open their mouths: the narrower the better.

Sincerely, Leon Pritzker, Fellow, ASA

********************************************************************
Dr. Gardenier’s response to Leon Pritzker:

Thank you for your comments. Yes, Dr. Deming’s writings, as well as his conversations, reflect a near obsession with the statistician, qua statistician, claiming expertise only in statistical theory and its application. That is quite different from undertaking solo studies in disciplines in which one is not professionally trained and experienced. This principle may be somewhat mitigated in practice when the statistician has performed multiple studies with subject matter collaborators over a few years, thereby acquiring a limited, but perhaps quite serviceable, familiarity with the field. There are those who claim that Dr. Deming himself went beyond his written strictures when he felt his subject matter expertise was adequate.

The ethical guidelines, you may note, are not addressed to “statisticians” alone, but rather to all “statistical practitioners” whatever their discipline. In many fields, including those you mention, statistics is a “tool” subject; professional education requires some study of statistics within the context of the subject matter discipline. There are several questions related to that: At what point does a particular study demand expertise in statistics beyond that which the practitioner possesses? At what point does a particular study demand expertise in the subject matter beyond what the individual possesses? What is the “half-life” of adequate statistical expertise - in general or in a specialized field? What constitutes adequate continuing professional education? Similarly with subject matter expertise and so on.

Another very crucial issue is what is the likelihood that five different subject matter experts would view the same set of measurements as appropriate and adequate for a given study? What is the likelihood that five different individuals, all of whom are acknowledged to have adequate statistical experience for a given project, would use the same methodological approach to analysis? The seldom examined, but commonly known, fact is that there is a lot “artistic” difference of opinion/approach in scientific studies.

Our draft ethical guidelines have always emphasized that honest differences of opinion are not matters of ethics. The version you read in the May Amstat News tried to get at this problem simply by saying in II.A.1. “Strive for practical significance, not just statistical significance.” Others have found that inadequate and/or confusing. The latest draft adds, “Typically, combine normative understanding of the subject matter issues, statistical protocols that are clearly defined before looking at the data, and power analyses or similar justification of both the practical significance of the study and the sample sizes needed for valid results.”

We could add within Section II.A. Professionalism, “Assure that adequate statistical expertise and adequate subject matter expertise both are applied to any given study. If this criterion is not met initially, it is important to add the missing expertise before completing the study design.” I will offer this option to the Committee. Leon and others are also urged to comment on this issue.

Meanwhile, please note that the current draft of the guidelines is dated August 27, 1998. A number of further changes are under consideration.