Guidelines Should Be Tougher

Fri, 29 May 1998 16:43:16 +0000
Harry Khamis

After reading the draft, I feel that it is unnecessarily weak in three
areas.

1. In Part I, C. 9, "Exposure of dishonest or incompetent uses of
statistics" is listed as an activity that practitioners "...may choose
to work toward." I believe that identifying instances in which improper
statistical technique have clearly been used, and in which unreliable or
potentially incorrect conclusions are made, should be more than good
professional citizenship, it IS an ethical obligation. To do otherwise
is to implicitly promote the misuse of statistics. My suggestion is to
be proactive in these instances -- this takes a substantive step beyond
what is stated in Part II, G.1.

2. In relation to Part II, A. 4., I feel that the statistician should
strive to become involved in a research project as early as possible,
preferably in the planning stages. Perhaps a sentence can be added to
this section, such as "This understanding can best be accomplished by
becoming involved in the research process as early as possible,
preferably in the planning stages." [Note: See Pat Nahas' comments for
a similar suggestion.]

3. My third criticism relates to Part II, A.3: "Strive for valid
practical significance, not just statistical significance." I have long
felt that too few researchers pay attention to sample size
issues; i.e., the type I error rate is carefully guaranteed by the
statistical method while the type II error rate is given little or no
attention. I believe that a sample size (power) analysis should be done
if it is at all feasible in the planning phase of research studies. In
terms of reporting responsibilities, the power of the study (or lack
thereof) should be reported (see Part II, B., 6, 8, and 10).

Thank you for the opportunity to respond to the draft.

Harry J. Khamis
Wright State University
-----------------------------------------------------------------------
Date: Sat, 23 May 1998 04:00:01 -0400
From: Ted Hermary

Ted initially noted that the American Sociological Association removes
the membership privilege from violators of its ethics, as do medical and
legal societies. Dr. Gardenier replied:

In addition to the medical and legal professions, engineering societies
tend to have "regulatory" codes. In order to do that, the societies have
to have formal rules of procedure, trained staff, and legal support. In
that such practices threaten the livelihoods of those accused, lawsuits
are not uncommon and are planned for. (ASA does not have resources for
that.)

Ted responded:

Perhaps I am just naive, but...

I believe that the American Sociological Association is justified in
saying that membership in its association is a "privilege". In my view,
it is certainly not a "right", and that the association is warranted in
making membership contingent upon adherence to its ethical standards --
no less than paying one's financial dues. In terms of issuing official
statements, that appears to be what it has done. I can see where the
privilege/livelihood line could be crossed, especially in a courtroom
contest. And perhaps there is a significant difference in this respect
between the American Statistical Association and the American
Sociological Association. Membership in the former may be more necessary
to statisticians' livelihood than membership in the latter is to
sociologists'.

I would think that conditional membership would apply even more to
honourary scientific societies. Years ago, executives of Sigma Xi The
Honor Society considered making a promise that membership could be
revoked if ones of its members was found to have committed fraud or
misconduct (see: Jackson, C. I., & Prados, J. W. (1983). Honor in
Science. American Scientist, 71(5), 462-464.). To the best of my
knowledge, this never materialized in anything even as concrete as a
official society statement.

[John also writes:

As in many fields, there are many open questions and many areas of
honest disagreement. Thus, there is a reluctance to make accusations of
misconduct even where the disagreements are passionate.
Similarly, there are many cases where statisticians participate in multi
disciplinary studies, but have virtually no power over the conduct of
the work or the contents of the publication.]

The above applies even more so to sociology. Take the meaning of
informed consent. As I recall sociologists were among the most vehement
opponents to attempts to devise a single code for informed consent,
partly because of covert ethnographic studies of "cults", "underground
organizations", and the like. (Psychologists, as I recall were more
concerned about the experimental uses of deception.) The American
Sociological Association Code does include a statement on informed
consent, for which one qualified exemption are observations which occur
"in public". Sociologists themselves routinely debate the meaning of
this word even at an intellectual level, never mind when ethics are
concerned. (For ethical didactic purposes, we focus mainly on
uncontested extremes - e.g., a busy public street and a stranger's
private home.)

So, I can certainly appreciate John's point about the practical
difficulties of concluding whether there is wrongdoing. However, this
might only mean that where there is an intellectual (or ethical) "draw",
no action would be taken. This may be treated as a separate issue from
having a written provision that allowed for action on the part of the
professional organization where there was not a "draw'.

[John: Finally, early commenters on the drafts wanted to make a
distinction between "good citizenship" practices, which are admirable
plus but not ethically compelling for everyone, versus dishonest,
disrespectful, and hurtful practices which damage the profession if they
are seen as widely condoned. In short, guidelines such as the draft from
ASA only serve two functions:
(1) they encourage practitioners to adhere to high standards and (2)
they give documentary support to practitioners who are trying to
maintain high standards against pressures to behave otherwise.]

Ted: But they also serve a third, perhaps unintended, function, that of
so-called public relations. If one wanted to be cynical about things,
one could read this as a recommendation for at least having a
"regulatory feel" to ethics codes, even if no "regulation" ever actually
occurred as a result. That may even be the case with the hefty American
Sociological Association's Code.

My general comments may go beyond the appropriate role of "ethical
guidelines", but: What, if anything, will the ASA do in response to
serious departures from its guidelines? Drawing on items mentioned in
the ASA draft, how would the Association respond to one of it members
who:

used, dishonest statistical methods or techniques?
OR
grossly failed to expose such practices?
OR
retaliated against those who undertake to expose such practices?
OR
made seriously un-professional or unfair criticism of the statistical
practices of others?
OR
falsely accused another statistician of misconduct when it was merely a
matter of different scholarly opinions or interpretations (especially if
doing so damaged the reputation of another statistician)?
OR
violated the confidentiality of an ongoing misconduct investigation?
OR, generally, committed scientific misconduct?

Such questions have been posed to representatives of other professional
scientific societies -- for example, during the meetings of the DHHS
Commission on Research Integrity. My general impression from reading
transcripts of the CRI discussions is that few or none of the
representatives wanted any part of such nasty business. To some, this
might sound like scientists are not so interested in "self-regulation"
after all.

For comparison, as part of their claim to professional status, Legal and
Medical Associations promise that they will take action in response to
breached of their professional ethics -- up to and including debarring
or de-licensing members. One can certainly debate whether, or how much,
they deliver these promises, but they do make them. Would the ASA
consider striking membership if faced with very serious breaches of its
ethics? (Nothing like an old-fashioned excommunication for bolstering up
ethical boundaries.:-))

More reasonably, professional societies need to state a range of
sanctions which they would levy against their wayward members, up to and
including revoking membership for very serious violations? Some ethical
standards could be argued to have a high consensus among society members
and, hence, that serious breaches of these would be open to serious
professional sanctions, while others could be phrased in more
exhortatory in character (see some ASA examples quoted below).

Also, how would ASA respond to findings of "misconduct" issued by
federal funding oversight bodies? They need not follow the "government
lead" on such matters. For example, it's not inconceivable that a
funding agency could decide that, for *its* purposes, misconduct had
occurred, while a professional society, with *its* purposes, reviewed
the evidence and decided otherwise - or vice versa. (Either situation
would be fascinating, methinks.)

As a final comment, along different lines, the draft ASA preamble reads,
in part:

While the following are not ethical obligations so much as good
professional citizenship, practitioners who are willing and able to do
so may choose to work toward:
..................... [snip]

8. Support for sound statistical practice when it is unfairly
criticized.

9. Exposure of dishonest or incompetent uses of statistics.
.................... [snip]

These exhortatory points seem doubly qualified, and hence
extra-tentative. They are already qualified by the reference to ability
or willingness. Do they need to be further qualified by "may choose to
work toward..."? How about "should work toward..."? Or, if one really
wanted to get heavy-handed, "must work toward..."?

Martin (Ted) Hermary (ABD)
McGill University
_________________________________________________

Additional comments on issues regarding the toughness (tightness) of the
ASA draft Ethical Guidelines could be very helpful to the Committee.
There are no ASA plans to formulate procedures for investigation of
alleged ethical violations or an array of "punishments." It is possible
that some universities or other organizations may have misconduct
investigation procedures which include a comparison of the behavior of
the accused individual with the ethical codes or guidelines of the
relevant professional societies.