WILLIAM DAUBERT and JOYCE DAUBERT, Individually and As Guardians Ad Litem for JASON DAUBERT,
and ANITA DE YOUNG, Individually And As Guardian Ad Litem For ERIC SCHULLER,
Petitioners,

v.

MERRELL DOW PHARMACEURICALS, INC., Respondent,

ON WRIT OF CERTIORARI TO THE UNITED STATES COURT OF APPEALS FOR THE NINTH CIRCUIT

BRIEF AMICI CURIAE OF
DARYL CHUBIN, ET AL.

SUMMARY OF ARGUMENT

The peer review system has many virtues. Serving as a litmus test for truth, as a guarantor of scientific certainty, and as the gatekeeper for expert witnesses for the litigation process are not among them. The peer review system was not designed to serve those functions, makes no claims to do so, and, at least as presently instituted, is incapable of performing those tasks. The purposes of the journal peer review system in science are to screen for obvious errors in methodology and reasoning, and to ensure that the research is novel and 'important.' The peer review system serves those purposes well, but, like other tools, its utility is marginalized when it is employed for purposes for which it was not designed. For example, the peer review system is not a truth certification process, nor does it guarantee that the particular data, research and analytical methodologies, or conclusions of the accepted papers represent the consensus opinion of the relevant community.

Each of the amici has carefully read the decisions filed by the United States Court of Appeals for the Ninth Circuit of the United States District Court for the Southern District of California in this case as well as the petition for certiorari, the respondent's opposition to the petition, and the petitioners' reply brief. We are troubled by the portions of those decisions that purport to describe and use the peer review system as tools in the litigation process. We are troubled by the lower courts decision to rely - exclusively - on peer review journals to determine whether an expert witness can be allowed to testify before a jury and what he or she may say if allowed to testify. We believe that the lower courts' misunderstanding of the peer review system led them to devise and ordain two mischievous presumptions: an apparently rebuttable presumption that anything that is published in a peer review journal is ‘good science’ and therefore admissible; and, conversely, an evidently irrebuttable presumption that unless a theory, fact, and. analysis has been previously published in a peer review journal such ideas and information cannot be 'good science' and cannot form the basis for admissible expert testimony.

We are also disturbed by the fact that the lower courts rendered those decisions without an accurate understanding of the peer review system and, apparently, without any factual background concerning that system. We cannot understand why the lower courts made such a choice and we have read nothing in the lower courts' decisions that explains their reasoning.

Before we endeavor to suggest where the lower courts erred in the understanding of the peer review system and before we describe what we believe to be the real strengths and not inconsiderable weaknesses of that system, we must confess that we are unqualified to offer this Court any guidance concerning the legal issues presented by the petition for certiorari filed in this case: whether the common law test for admissibility of expert opinion testimony that originated with Frye v. United States (1923), survived Congress' enactment of a rule of evidence that deals with the same subject, whether federal judges possess some sort of inherent power that they can employ to supplement whatever rules are enacted by Congress, and whether the U.S. Constitution authorizes federal judges to devolve to the editors of peer review journals (and to the anonymous referees who are handpicked by those editors) decisions about what is (and what is not) so-called 'good science' and regarding which expert witnesses can be allowed to testify in criminal and civil cases.

Moreover, although we have made some efforts to familiarize ourselves with the underlying factual controversy in this case (having gone so far as to read, among other things, the affidavits and curriculum vitae submitted to the District Court by the plaintiffs' experts and the defendant's expert), we also acknowledge the we can offer no help in helping to decide whether Bendectin can cause birth defects, in general, and whether Bendectin caused the birth defects that the plaintiffs-petitioners allegedly suffered, in particular. Thus, if we were today to be seated as jurors in this case we could not be say who should prevail - not unless we had heard all the experts testify and not unless we had seen these same experts cross-examined under oath. I

Why, then, have we been vexed to involve ourselves in this matter and why do we think that the Justices of this Court should pay any mind to what we have to say? The reason is simple. We are knowledgeable about the peer review process and its useful contributions to the dissemination of scientific learning. But, because we appreciate the system's limitations as well as its advantages, we believe that the burden placed by the lower courts on the peer review system is a weight that system cannot bear. If we were sitting on a jury in a case such as this, we assuredly would think it important to learn what articles had been written, pro and con, about the issues in dispute. Nevertheless, in general, we do not think that the fact that one expert could point to a published article in support of his views and the fact that another expert could not cite a published article in support of her views would be the acid test in our decision-making. We do not think that such facts should be the acid test for the decision-making.

Such facts might influence, to a greater or lesser degree, our preliminary decision about which expert(s) to believe, but we would not allow our decisions on this sort of threshold issue to turn on a single fact such as publication or non-publication. In our view, the lower courts did not truly comprehend how the peer review system does (and does not) work, how well (and bow poorly) the system operates, and what are the system's true strengths (and its not inconsiderable weaknesses). If the lower courts had understood the system as we do they would not have relied on it as a shortcut for and they certainly would not have proposed to rely on it as he exclusive indicator of reliable and valid expert testimony.

Although it is appropriate for courts to use the peer review system to help guide the decisions of judges and jurors, courts should not rely on solitary facts - such as publication, multiple publication, publication in prestigious journals, or non-publication - as the exclusive determinant of whether in expert -has anything useful to say and whether that expert should be allowed to testify. The temptation to rely upon the peer review system to provide a quick fix for the sometimes irritating and frequently time-consuming problems presented by cases that involve complicated questions of science may be an understandable one. But this impulse to rely exclusively on peer review literature (and its interpreters), like many ideas born of frustration, should be resisted. This temptation should be resisted because the allure of relying on the peer review system is based on ignorance of and myths about that system, not knowledge and understanding of it. The lure of the peer review system stems from six largely erroneous beliefs, myths that gain no validity from the fact that they may be widely held. As discussed at length below, none of these myths can withstand serious scrutiny.

First, the peer review system is not intended to yield "the truth, the whole truth, and nothing but the truth." Second, peer review journals do not replicate and verify the experiments, research and analytical techniques, or data reported in the papers submitted for publication. Third, peer review journals do not warrant that the ideas and information contained in the articles they publish are accurate, valid, certain, reliable, or true, or otherwise amount to 'good science.' Fourth, the mere fact of publication does not mean that the ideas and information reported in an article are 'generally accepted' by, or represent the 'consensus' views of, the relevant academic community. Fifth, conversely, the fact that ideas and information have not been published in a peer review journal does not mean that they are not 'generally accepted', or that they are 'generally rejected', or that they cannot represent ’good science.' Finally, the peer review system should not be regarded as more rigorous and reliable than the jury system's use of cross-examination. For these reasons, the courts should not rely exclusively on peer review literature, but should consider peer review literature in conjunction with other materials and with expert testimony.

In sum, the courts should not put all of their eggs in the peer review basket.

I. CONTRARY TO POPULAR MYTH, THE PURPOSES AND DESIGN OF THE PEER REVIEW SYSTEM ARE NOT TO PRODUCE "THE TRUTH, THE WHOLE TRUTH, AND NOTHING BUT THE TRUTH."

Although witnesses are obliged to swear to 'tell the truth, the whole truth, and nothing but the truth,' the litigation process overall does not seek absolute truth but, instead, a much more modest, yet still quite challenging goal - providing justice to the parties to a case.' Similarly, although the overarching goal of the peer review system may be to add to the total sum of scientific knowledge and to the grand scheme of scientific understanding of the universe, the practical and immediate purposes, and the concomitant operation, of the peer review system are, much more modest. 'Briefly defined, peer review is an organized method for evaluating scientific work which is used by scientists to certify the correctness of procedures, establish the plausibility of results, and allocate scarce resources (such as journal space, research funds, recognition, and special honor)." The primary goal of the peer review system is to provide a relatively permanent and readily accessible documentary record of ongoing scientific research and analysis, that is, printed reports rather than anecdotal or apocryphal accounts, and thus to facilitate the development and exchange of ideas.' The peer review system is designed to compel authors of such reports to comply with certain minimal formal and stylistic standards in order to weed out essays that appear facially implausible, indefensible, erroneous, fallacious, or fraudulent. The articles that survive the screening process are presented in a standardized format in journals that are made readily available to all scholars and practitioners in a particular discipline, who may then use them in their own work or may challenge them in their own writing, e.g., by attempting to replicate the experiments described or to reanalyze the results reported. Thus, the peer review system is designed to provide a common and convenient starting point for scientific debate, not the final summation of existing scientific knowledge.

In a nutshell, the peer review system is not designed, and does not function, to add absolutely true, complete, and definitive chapters to the mythical encyclopedia of knowledge. Instead, the peer review system provides a more practical and still extremely valuable service - to furnish, in a timely fashion, interim progress reports of ongoing research experiments, promising investigatory techniques and methodological protocols, useful analytical tools, interesting findings, or provocative hypotheses. 7le provisional nature of science 'truths' is reflected in the fact that peer review journals welcome, and frequently publish, articles, con-Lments, and letters to the editor that are critical of the initial article. 7bus, each article stands not as the final word- on a given subject but as an invitation to or part of a continuing debate, a never-ending dialogue. As Niels Bohr, one of the great theoretical physicists of the 20th century endlessly reminded his colleagues: Every sentence I utter must be understood not as an affirmation, but as a question.

II. CONTRARY TO GENERALLY ACCEPTED BELIEFS, THE EDITORS OF A PEER REVIEW JOURNAL, AND THE REFEREES PICKED BY SUCH EDITORS TO REVIEW A SUBMITTED MANUSCRIPT DO NOT REPLICATE AND VERIFY THE EXPERIMENTS, TECHNIQUES, ANALYSES, OR DATA REPORTED IN SUCH MANUSCRIPTS.

The myth prevails, at least among non-scientists, that the peer review system subjects articles submitted for publication to scrutiny at least as exacting as that of a recipe submitted to Gourmet magazine, a new model car reviewed by the professional drivers and automotive engineers employed by CAR AND DRIVER, or a toaster tested by the editors of CONSUMERS PEPORTS. Would that it were so. Although the abilities of the editors and referees of peer review journals are presumptively higher than the talents of those persons who critique recipes, recreation vehicles, and refrigerators (and although the issues are doubtlessly more transcendentally important), the resources - lab space and testing equipment, money, and time - available to part-time peer review editors and unpaid volunteer referees for the purpose of scrutinizing the work of other scientists are far less than enjoyed the persons employed by cooking, car, and consumer magazines. Given these stubborn handicaps, peer review referees and editors limit their assessment of submitted articles to such matters as style, plausibility, and defensibility; they do not duplicate experiments from scratch or plow through reams of computer-generated data in order to guarantee accuracy or veracity or certainty. Once again, whatever their desire to do so, they are hampered by financial constraints, and, as noted below, occasionally by ethical restrictions, too.

III. CONTRARY TO THE "GENERAL ACCEPTANCE" MYTH, PUBLICATION OF AN ARTICLE IN A PEER REVIEW JOURNAL IS NO ASSURANCE THAT THE RESEARCH, DATA, METHODOLOGIES, ANALYSES, OR CONCLUSIONS DESCRIBED THEREIN ARE TRUE, ACCURATE, VALID, RELIABLE, OR CERTAIN OR THAT THEY REPRESENT "GOOD SCIENCE."

The failure of peer review journals to adequately screen out false, substandard, or fraudulent reports might be excusable if the journals did a creditable job of subsequently informing their readers about the existence of the published mistakes (and the dangers of relying on reports containing such flaws) once the errors were noticed. Unfortunately, the journals do not do much better after the fact than they do before.

Both types of problems - the initial failure to screen out faulty reports and the subsequent failure to adequately, if ever, inform readers that inaccurate or fallacious articles had been published - are compounded by the fact that there is no licensing board that sets uniform criteria for what a journal should and should not publish or what a magazine needs to do to qualify as a peer review journal. Along the same lines, although professional basketball referees must go to school to learn what is or is not a foul, peer review journal referees receive no comparable training. Even more amazingly, the peer review industry is a wholly unregulated collection of completely independent and unsupervised periodicals, as there is no licensing board that certifies that a magazine qualifies as peer review journal. Like boilerroom 'investment counselors' before the establishment of the Securities and Exchange Commission, calling oneself the editor of a peer review journal simply makes one so. At least, there is no entity to challenge such puffery.

IV. CONTRARY TO POPULAR OPINION THE FACT THAT AN ARTICLE HAS BEEN PUBLISHED IN A PEER REVIEW JOURNAL DOES NOT ESTABLISH THAT THE FACTS, RESEARCH OR ANALYNCAL METHODOLOGIES, OR CONCLUSIONS REPORTED THEREIN ARE "GENERALLY ACCEPTED" BY OR REPRESENT THE "CONSENSUS" VIEWS OF THE RELEVANT SCIENTIFIC COMMUNITY.

Understandably, most authors would prefer to see their articles published in JAMA than in the less renowned publication. Consequently, an author may first take a crack at getting published in JAMA and then, only if failing there, try to have his paper published in a slightly less celebrated journal, and on and on, until the quality of the paper has met its match in the quality of a journal. If the first journal rejects the submitted manuscript, it is sent on to next journal and so on, until either the article is published or the author relents. The number of bites at the apple is limited by only two factors: the number of journals in a relevant field and the size of the author's budget for postage. Thus, a manuscript may have been rejected by ten or twenty journals before it is finally accepted for publication by one.

By this light, to suggest, as the Ninth Circuit did below, that the appearance of a paper in a peer review journal betokens "general acceptance" by a "consensus" of the scientific community is senseless. If anything, the numerous rebuffs that generally precede eventual acceptance connotes 'general rejection' much more than a single endorsement manifests 'general acceptance.' Thus, by the time a paper is finally accepted by a single journal it may have been rejected by countless others.

VI. THE PEER REVEEW SYSTEM IS NOT MORE RIGOROUS AND MORE RELIABLE THAN THE LEGAL SYSTEM'S USE OF CROSS-EXAMINATION

Cross-examination often subjects a witness to scrutiny that is longer-lasting, more intensive in depth, and more extensive in the breadth of subjects examined. First, there are rarely time limits placed on cross-examination itself (and as most expert witnesses will attest, they must spend at least as much time preparing their testimony and explaining what it is all about to the lawyers who have hired them as they do on the witness stand.) Second, once on the witness stand, a witness may be obliged to endure a line-by-line exegesis of his article and asked to explain and defend every point. Third, the questions asked on cross-examination are not mercifully limited to the specific parameters of the paper or report that is the foundation of the expert's conclusion; instead, almost anything the expert has done, said, or written is fair game. For example, on cross-examination an expert witness can be subject to close questioning not only on the basis of the specific sources of his or her testimony, but can be impeached by anything he or she had said or written that is potentially inconsistent. His or her motives and credentials can be scrutinized in ways unimaginable to peer review editors and referees. The peer review system has nothing comparable. All in all, whatever the pressures are generated by the peer review system, cross-examination is, quite literally, a far more trying experience.