Matrixx v. Siracusano: what do courts mean by statistical significance?

Transcription

1 Law, Probability and Risk (2012) 11, Advance Access publication on December 12, 2011 doi: /lpr/mgr022 Matrixx v. Siracusano: what do courts mean by statistical significance? JOSEPH B. KADANE Department of Statistics, Carnegie Mellon University, Pittsburgh, PA 15217, USA [Received on 26 September 2011; revised on 11 October 2011; accepted on 14 October 2011] The Supreme Court ruled in Matrixx that statistical significance is not necessary to show that a drug caused an adverse reaction. Five circuit court decisions holding otherwise preceded this decision. This paper examines the extent to which the Supreme Court s reasoning differed from those of the circuit courts. Keywords: statistical significance; adverse drug reactions; securities fraud; summary judgement; materiality; scienter. 1. Introduction In Matrixx, 1 a unanimous Supreme Court held that the premise that statistical significance is the only reliable indication of causation...is flawed. 2 In so holding, it was apparently overruling five previous decisions by the First, Second and Third Circuit Courts of Appeals requiring statistical significance and upholding the Ninth Circuit s decision to the contrary. A hint that this is not the whole story can be found in the fact that two of the five overruled circuit court opinions were those of Supreme Court Justices Alito and Sotomayer when they were serving, respectively, on the Third and Second Circuits. The issue is what is meant by the phrase statistical significance. All five of the preceding cases involve motions for summary judgement by defendant drug companies accused of securities fraud for failure to disclose after-market adverse reactions in a timely way. To understand them, it is necessary to review their procedural posture and the facts upon which each decision is based. 2. After-market adverse reactions and their legal implications A new medical product (drug or device) is approved for distribution in the USA after clinical studies have persuaded the Food and Drug Administration (FDA) of its safety and efficacy. The process is not foolproof, of course. After a product is on the market, the manufacturer and the FDA receive reports of patients who have unfortunate medical circumstances that may or may not be the result of the drug or device. Depending on the seriousness and unusual nature of the circumstance reported, the frequency with which it is reported, and the medicine/biology/physics of possible causation, at some point it is reasonable to suspect that there may be a causal link between the medical product and the alleged adverse reaction. This can occasion new targeted clinical trials, FDA warnings and/or removal of the product from the market. kadane@stat.cmu.edu S.Ct. 1309; 79 U.S.L.W (2011). 2 ibid. c The Author Published by Oxford University Press. All rights reserved.

2 42 J. B. KADANE There are at least two kinds of legal consequences that can result from the marketing of a product that turns out to have adverse effects. One, of course, is that the company may be liable to patients harmed by the product. The second is that the company may be liable to investors who bought the stock of the company before it made public the circumstances concerning the adverse effect of its product. It is the latter kind of case, allegations of securities fraud, that is the focus here. 3. Securities fraud and summary judgement The law of securities fraud applies to all publicly traded companies not just those marketing drugs and medical devices. Succinctly, to state a valid securities fraud claim under Rule 10b-5, a plaintiff must first establish that defendant, in connection with the purchase or sale of a security, made a materially false or misleading statement or omitted to state a material fact necessary to make a statement not misleading... (see In re: Burlington Coat Factory Securities Litigation, 114 F. 3d 1410, 1417 (3d Cir. 1997). The plaintiff must additionally establish that the defendant acted with scienter and that plaintiff s reasonable reliance on defendant s misstatement proximately caused him injury. See In re: Phillips Petroleum Securities Litigation, 811 F. 2d 1236, 1244 (3d Cir. 1989) (Oran et al. v. American Home Products Corp., et al. (226 F.3d 275, 3d. Cir., 2000). The key concepts here are materiality and scienter. Materiality is satisfied when there is a substantial likelihood that the disclosure of the omitted fact would have been viewed by the reasonable investor as having significantly altered the total mix of information made available (Basic, Inc. v. Levinson, 485 U.S. 224, ). Scienter is a mental state embracing intent to deceive, manipulate or defraud (Tellabs, Inc. v. Makor Issues and Rights, Ltd., 551 U.S. 308, 319 (2007)). In many civil cases, there is often a request for summary judgement. The idea behind summary judgement is that, even if every fact alleged by the opposing party were found to be true, the case would still fail for legal reasons. Thus, courts evaluate motions for summary judgement assuming all facts and reasonable inferences therefrom as alleged by the non-moving party. A defendant s successful motion for summary judgement ends the case or defeats the particular claims that are the subject of the motion. 4. Circuit Courts of Appeals decisions on motions for summary judgement on cases of securities fraud against drug manufacturers A. Carter-Wallace: The first case to discuss is In re: Carter-Wallace, Inc. Securities Litigation (150 F.3d 153 2nd circuit, 1998). Carter-Wallace marketed Felbatol as a drug to treat epilepsy. It advertised Felbatol in medical journals from January to July 1994 as having an unprecedented safety profile and that no life-threatening liver toxicities or blood dyscrasias have been attributed to Felbatol monotherapy. From October 1993 until July 1994, Carter-Wallace received and was aware of at least 56 adverse medical reports relating to Felbatol, including at least six deaths and six cases of aplastic anemia, a frequently fatal disease in which bone marrow stops producing cells. In July 1994, four more cases of aplastic anemia were reported, and on 1 August 1994 Carter-Wallace, in association with FDA, recommended that patients be immediately withdrawn from Felbatol. Carter-Wallace was sued for securities fraud and moved for summary judgement. The Appeals Court, upholding the District Court s granting of summary judgement in this respect, held

3 MATRIXX V. SIRACUSANO: WHAT DO COURTS MEAN BY STATISTICAL SIGNIFICANCE? 43 The statements...did not become materially misleading until Carter-Wallace had information that Felbatol had caused a statistically significant number of aplasticanemia deaths and therefore had reason to believe that the commercial viability of Felbatol was threatened (references omitted). Drug companies need not disclose isolated reports of illnesses suffered by users of their drugs until those reports provide statistically significant evidence that the ill effects may be caused by rather than randomly associated with use of the drugs and are sufficiently serious and frequent to affect future earnings. In the present case, four of the ten reported deaths occurred in July the disclosure was on August 1 and earlier reports are not by themselves sufficient to support inferences of either actual knowledge or recklessness (citations omitted). The court upheld the granting of summary judgement because the alleged facts would not support scienter. B. American Home Products and Fen-Phen: American Home Products Corporation (AHP) marketed two weight-loss drugs, Pondimin (fenfluramine) and Redux (dexfenfluramine). Pondimin was marketed with another drug (phentermine), hence the popular name fen-phen. Pondimin and Redux were approved by the FDA in 1973 and 1996, respectively. In February 1994, AHP learned of one European doctor who found seven patients who had been taking Pondimin or Redux and who had leaky heart valves. By November 1995, AHP knew of at least 31 heart valve abnormalities among European diet pill users but informed the FDA about only eight. In March 1997, AHP met with researchers from the Mayo Clinic and MeritCare Health Systems who informed AHP of a total of 17 more cases. By 8 July 1997, the Mayo data, now with 24 cases, were publicly announced. Finally, on 12 September 1997, FDA informed AHP of a survey showing 92 of 291 fen-phen users had heart-valve abnormalities. Both drugs were withdrawn from the market on 15 September AHP was sued for securities fraud. The Third Circuit (226 F.3d 275, 2000) upheld the motion for summary judgement, writing Plaintiffs do not allege that the European data and adverse reaction reports, taken by themselves, established any statistically significant relationship between AHP s products and valvular heart disease. Nor does the Amended Complaint assert that the withheld data, even when viewed in conjunction with the Mayo report, could have demonstrated any medically conclusive link in light of the millions of prescriptions written for Pondimin and Redux. In fact, plaintiffs never clearly explain how the accumulation of additional anecdotal data, short of the point of statistical significance, would have added anything to the disclosures already made on July 8, Because the link between the two drugs and heart-valve disorders was never definitely established during the relevant period even after the withheld data is taken into account, AHP s failure to disclose this data cannot render its statements about the inconclusiveness of the relationship materially misleading. Later in the same opinion, the Court writes it is clear that until the FDA notified AHP on September 12 of its own data showing a link between the two drugs and heart-valve

4 44 J. B. KADANE disorders, there was no statistically significant evidence establishing a serious health risk. Then Circuit Judge Alito wrote this opinion, with the concurrence of two colleagues. C. Biogen: On 23 November 2004, Biogen announced that the FDA had granted accelerated approval for TYSABRI, a drug that treats multiple sclerosis, Crohn s disease and other autoimmune diseases. At the time of approval, it was recognized that there was a danger that TYSABRI might allow opportunistic infections, which might be fatal. On 18 February 2005, Biogen learned that two patients taking TYSABRI in clinical trials had contracted progressive multifocal leukoencephalipathy (PML) and that one patient had died. On 28 February 2005, the company, in consultation with the FDA, withdrew the drug from the market. On 30 March 2005, Biogen announced that the death of a patient in one of the clinical trials had been misclassified but was from PML. On 7 8 March 2006, a Biogen employee presented data showing three patients with opportunistic infections (two with PML) in a multiple sclerosis trial, and five with opportunistic infections (one with PML) from a Crohn s disease trial, out of 3900 patients total. Reviewing these data, the Court of Appeals wrote: There is no basis to conclude that these results, excluding the PML infections, were statistically significant. There is no plausible inference from the reports of just five patients with non-pml opportunistic infections that defendants knew of any causal relationship between the use of TYSABRI and the separate opportunistic infections diagnosed for the five patients, and then intentionally withheld data. Because the court finds that Biogen had no reason to believe that TYSABRI led to unacceptably large numbers of opportunistic infections, it concluded that scienter did not exist and sustained the District Court s granting of summary judgement. D. Smith-Kline-Beecham and Paxil/Augmentin: Masters complains that GSK [Glaxo Smith Kline] [did not] timely disclose alleged adverse results of certain research trials, and by sponsoring researchers to publish false and misleading materials, concerning the safety and efficacy of Paxil for treatment of children and adolescents. We have held that reports of harmful drug effects are immaterial and thus need not be disclosed unless those reports (1) show statistically significant evidence of adverse effects (2) establish that the adverse effect threatens the commercial viability of the drug and (3) show that the effect poses a significant risk to the company s future earnings. The complaint does not explain how the results of the research trials at issue could be deemed statistically significant in light of the test results from another trial that GSK did disclose. 3 Judge Sotomayor, now on the Supreme Court, participated in the three-judge panel making this decision. E. Avandia: GlaxoSmithKline was sued for securities fraud for failing to disclose cardiovascular risks of Avandia, a drug for the treatment of type 2 diabetes. In particular, a published metaanalysis 4 suggested such a risk. A safety alert published by the FDA on the day that the Fed. Appx. 46 (2d circuit, 2008). 4 Nissen, S.E. and K. Wolski (2007) Effect of Rosiglitazone on the Risk of Myocardial Infarction and Death from Cardiovascular Causes, New England Journal of Medicine, 356(24) 2457.

5 MATRIXX V. SIRACUSANO: WHAT DO COURTS MEAN BY STATISTICAL SIGNIFICANCE? 45 meta analysis was published drew attention to the risk but did not mandate or request action. The FDA Commissioner testified to Congress that the meta analysis presented inconsistent data with regard to the potential cardiovascular risk...the data are inconsistent and conclusions are not clear. The second Circuit court held that While the complaint conclusively alleges that the results of the meta analyses showed an estimate of an increased risk for heart attack, it pleads no facts indicating that the test results were even statistically significant, and hence supported the granting of summary judgement for failure to allege materiality or scienter. 5. Statistical Significance of Adverse Reactions Statisticians have given the phrase statistical significance a precise technical meaning. It involves specification of a data base, specification of a specific test statistic, specification of a null hypothesis, and of a probability distribution of the test statistic under that null hypothesis, and of a level of the test. Then statistical significance is said to occur when the probability of the test statistic as or more extreme than that which occurred is less than the specified level of the test. How would statistical significance apply to allegations of adverse reactions to a drug or device? In the first instance, there would be reports from physicians that n patients had the adverse reaction. In and of itself, there is no obvious way to apply statistical significance to such reports. Suppose the reports were richer and said that n patients had the adverse reaction out of a total of m given the drug or device. So what? Unless we know what the rate of the adverse reaction is among those not given the drug or device, this information is not probative. Even if we knew that, we need to be careful. People given a drug or device are given it for a reason because they suffer from some specific disease or disability. To assess whether the drug or device has anything to do with the adverse reaction, the comparison population must be comparable to the population that received the drug or device. Using data from a health maintenance organization that would have detailed records of patients medical histories, observational studies of adverse reactions can be done and could be summarized by a significance test. The tricky part is in ensuring that the untreated population is in all other respects comparable to the treated population. This, of course, is the rationale for controlled clinical trials, which is another possibility. 5 There is another less formal sense in which the term statistical significance might be used. This usage is close to Berkson s Interocular traumatic test (does it hit you between the eyes? 6 ) This usage would permit putting together the reports of adverse reactions with other information, such as the general prevalence of the adverse reaction. The question to be addressed here is which sense of statistical significance is being used by the courts in deciding motions for summary judgement in securities fraud cases against drug and device manufacturers. 5 See Kaye, D.H. (2011) Trapped in the Matrixx: The U.S. Supreme Court and the Need for Statistical Significance, 39 Product Safety and Liability Reporter, 1007 (09/12/2011) and Gastwirth, J.L., Statistical Considerations Support the Supreme Court s Decision in Matrixx Initiatives vs. Siracusano (12 August 2011) (unpublished manuscript). 6 Edwards, W. H. Lindman and L.J. Savage (1963) Bayesian Statistical Inference for Psychological Research, Psychological Review, 70(3), at p. 217.

6 46 J. B. KADANE 6. Statistical significance in Circuit Court cases Table 1 lists the important facts about the five circuit court cases discussed above. In all five cases, the circuit court sustained the trial court s granting of summary judgement against the plaintiffs. All five circuit court decisions mention statistical significance as a requirement to distinguish random case reports from a causative relation between the drug in question and a possible side effect. But what, exactly, do these courts mean by statistical significance? The Paxil case was dismissed because it concerned an off-label use of the drug involving a very small proportion of the sales and hence failed materiality. Felbatol and TYSABRI were withdrawn after a very few cases of serious side effects and without any obvious statistical studies. Hence, the two cases that might shed some light on the question are fen-phen and Avandia. What can be learned from them? With respect to fen-phen, there is an FDA report to consult. 7 While there are some confidence intervals mentioned in the report, the heart of the matter is that 92 of 291 fen-phen users had developed heart-valve abnormalities (court decision). This is compared in the FDA report to an estimate of 5% among similar patients not taking fen-phen reported in personal communication by a specific doctor at New York Hospital-Cornell Medical Center. Thus, the analysis here is Bayesian in spirit. The discrepancy between a personal opinion of 5% and the data of 92 out of 291, roughly 30%, was sufficient to cause the FDA and the manufacturer to act. With respect to Avandia, the Nissen and Wolski metaanalysis found two p values (0.03 for myocardial infarction and 0.06 for death). However, this information was not regarded as definitive by the FDA, as the commissioner testified. Nor was it regarded as definitive by the court, which deferred to the Commissioner. Thus, during the period of relevance to the case, even significance (or close to it, depending on your taste with respect to the appropriate level of the test) was not enough to trigger action nor enough for the court to find statistical significance in its sense. In summary, the circuit courts found statistical significance in the strict sense neither necessary (fen-phen, Felbatol, TYSABRI) nor sufficient (Avandia) to require action to remove a drug from the market. 7. Zicam Cold Remedy Matrixx Initiatives was sued for securities fraud concerning its product Zicam Cold Remedy, which had been shown to cause anosmia (permanent loss of the ability to smell). The class on behalf of whom the suit was filed were those who bought stock between 22 October 2003 and 6 February Before the period began, Zicam knew of a paper presented to the American Rhinologic Society about a patient who lost his sense of smell after using Zicam. By 30 January 2004, there were three product liability cases against Matrixx concerning anosmia. Nonetheless, Matrixx released a press report on 2 February 2004 saying in part Matrixx believes statements alleging that intranasal Zicam products caused anosmia (loss of smell) are completely unfounded and misleading. Also, Zicam Cold Remedy was alleged to amount to 70% of Matrixx s sales. 7 FDA (1997) Cardiac Valvulopathy Associated with Exposure to Fenfluramine or Dexfenfluramine: U.S. Department of Health and Human Services Interim Public Health Recommendations November 1997, Morbidity and Mortality Weekly Report, 46(45),

7 MATRIXX V. SIRACUSANO: WHAT DO COURTS MEAN BY STATISTICAL SIGNIFICANCE? 47 The District Court granted Matrixx s motion to dismiss the case, which the Ninth Circuit Court of Appeals overturned. 8 The reasoning of the Ninth Circuit rejected the notion that statistical significance was required, instead holding that the complaint made sufficient allegations that materiality and scienter should be determined by the trier of fact. Furthermore, the Circuit Court noted allegations that in the period in question Matrixx had admitted that it did not know whether Zicam cold remedy caused loss of smell and had done no studies on the subject. This Court did not comment on what it meant by statistical significance. The case was appealed to the Supreme Court. Zicam s defense was that statistical significance had not been shown and relied upon the reasoning of the First, Second and Third Circuits that statistical significance was essential to prove materiality. The Supreme Court rejected this argument. They cite the Federal Judicial Center s Reference Manual on Scientific Evidence as follows: To test for significance, a researcher develops a null hypothesis e.g. the assertion that there is no relationship between Zicam use and anosmia. The researcher then calculates the probability of obtaining the observed data (or more extreme value) if the null hypothesis is true (called the p value). Finally, the researcher compares the p value to a preselected value called the significance level. If the p value is below the preselected value, the difference is deemed significant. Thus, the Supreme Court makes it very clear that statistical significance means to them the more formal definition. Given that the Supreme Court rejects the argument that statistical significance is required, what does the Court suggest instead? It notes that medical researchers and the FDA consider the totality of the evidence, including use such indicia as strength of association temporal relationship of product use and the event consistency of findings across available data sources evidence of a dose response for the effect biologic plausibility seriousness of the event relative to the disease being treated potential to mitigate the risk in the population feasibility of further study using observational or controlled clinical study designs degree of benefit the product provides, including availability of other therapies It further notes that the FDA does not require proof of causation but rather reasonable evidence of an association before a warning is issued. It is easy to understand the line of reasoning in this case using ideas of decision theory. From the perspective of a patient, a cold is a nuisance but will be gone in a few days. The prospect of a substantial probability of permanent loss of the sense of smell would deter almost everyone from using the product. From the perspective of an investor, the nearly total loss of sales from the company s 8 Technically, the motion was under rule 12b6, which provides for dismissal for failure to state a claim upon which relief can be granted. Since the issues addressed are materiality and scienter for the purposes of this paper the difference between rule 12b6 and summary judgement does not matter.

8 48 J. B. KADANE TABLE 1 Summary of circuit court cases upholding district court s grant of summary judgement for drug companies accused of securities fraud Drug Maker Disease Adverse effect Legal reference Precipitating event 1. Felbatol Carter Wallace Epilepsy Aplastic anemia 150 F.3d 153 (2nd circuit, 1998) Six old cases, 220 F.3d 36 (2nd circuit, 2000) Four new in July 1994 American Home Leaking heart 2. Fen-phen Weight 226 F.3d. 275 (3rd circuit, 2000) FDA study Products valve Multiple Opportunistic Two patients 3. TYSABRI Biogen Idec. 537 F.3d 35 (1st circuit, 2008) sclerosis infections PML Paxil/ Smith Kline Suicidal thoughts 271 Fed. Appx. 46 Off label use 4. Depression augmentation Beechem suicide (2nd circuit, 2008) On children Glaxo Smith Type Fed. Appx Avandia Heart attack Metaanalysis Kline diabetes (2nd circuit, 2009)

9 MATRIXX V. SIRACUSANO: WHAT DO COURTS MEAN BY STATISTICAL SIGNIFICANCE? 49 leading product indeed significantly alters the total mix of information available. The power of this line of reasoning is revealed by imagining two possible changes in the facts. Suppose, leaving all other facts as they are, that Zicam cured cancer. Compared to the prospect of surgery, chemotherapy, radiation and possible recurrence of cancer and death, many patients would choose to use Zicam and endure possible loss of smell. Now suppose we revert to Zicam Cold Remedy but suppose this product contributed only 1% of Matrixx s sales. A reasonable investor might conclude that an adverse reaction to such a minor product was not important and hence materiality would fail. It is peculiar that acts committed by a small company might make it liable for securities fraud, while the same acts committed by large company would escape such liability because of materiality. The fact-based consideration that the Supreme Court endorses is very much in line with the Bayesian decision-theoretic approach that models how to make rational decisions under uncertainty. The presence or absence of statistical significance (in the formal, narrow sense) plays little role in such an analysis. 8. Conclusion Did or did not the Supreme Court overrule the preceding decisions of the First, Second and Third Circuits? In a formal sense, it did. Those courts held that statistical significance was required for a plaintiff s case to survive a motion for summary judgement in a security fraud case against a drug company. The Supreme Court ruled to the contrary. But looking below those words, there is a strong sense of continuity among all six cases. The first five cases were (to my eyes) factually weak, and summary judgement was properly granted. In the Matrixx case, the facts are strong enough to survive summary judgement. That means that Matrixx must defend itself in a trial, not that Matrixx is necessarily liable. As the Supreme Court remarks, whether respondents can ultimately prove their allegations and establish scienter is an altogether different question. In rejecting statistical significance in the strict, formal sense, the Supreme Court was endorsing the looser sense of statistical significance that the circuit courts had been using all along.