NBER WORKING PAPER SERIES USES AND ABUSES OF EMPIRICAL EVIDENCE IN THE DEATH PENALTY DEBATE John J. Donohue III Justin Wolfers Working Paper 11982 http://www.nber.org/papers/w11982 NATIONAL BUREAU OF ECONOMIC RESEARCH 1050 Massachusetts Avenue Cambridge, MA 02138 January 2006 This paper was prepared for Stanford Law Review. The authors wish to thank Sasch Becker, Chris Griffin, and Joe Masters for extremely valuable research assistance, and Dale Cloninger, Larry Katz, Naci Mocan, Joanna Shephard, and Paul Zimmerman for generously sharing their data and code with us. We are grateful to Richard Berk, Gerald Faulhaber, David Freedman, Andrew Leigh, David Rosen, Peter Siegelman, Carol Steiker, Betsey Stevenson, Joel Waldfogel, and Matthew White for useful discussions and comments. The views expressed herein are those of the author(s) and do not necessarily reflect the views of the National Bureau of Economic Research. 2006 by John J. Donohue III and Justin Wolfers. All rights reserved. Short sections of text, not to exceed two paragraphs, may be quoted without explicit permission provided that full credit, including notice, is given to the source.
Uses and Abuses of Empirical Evidence in the Death Penalty Debate John J. Donohue III and Justin Wolfers NBER Working Paper No. 11982 January 2006 JEL No. K14, K42 ABSTRACT Does the death penalty save lives? A surge of recent interest in this question has yielded a series of papers purporting to show robust and precise estimates of a substantial deterrent effect of capital punishment. We assess the various approaches that have been used in this literature, testing the robustness of these inferences. Specifically, we start by assessing the time series evidence, comparing the history of executions and homicides in the United States and Canada, and within the United States, between executing and non-executing states. We analyze the effects of the judicial experiments provided by the Furman and Gregg decisions and assess the relationship between execution and homicide rates in state panel data since 1934. We then revisit the existing instrumental variables approaches and assess two recent state-specific execution morartoria. In each case we find that previous inferences of large deterrent effects based upon specific examples, functional forms, control variables, comparison groups, or IV strategies are extremely fragile and even small changes in the specifications yield dramatically different results. The fundamental difficulty is that the death penalty at least as it has been implemented in the United States is applied so rarely that the number of homicides that it can plausibly have caused or deterred cannot be reliably disentangled from the large year-to-year changes in the homicide rate caused by other factors. As such, short samples and particular specifications may yield large but spurious correlations. We conclude that existing estimates appear to reflect a small and unrepresentative sample of the estimates that arise from alternative approaches. Sampling from the broader universe of plausible approaches suggests not just reasonable doubt about whether there is any deterrent effect of the death penalty, but profound uncertainty even about its sign. John J. Donohue III Yale Law School PO Box 208215 New Haven, CT 06520-8215 and NBER j.donohue@yale.edu Justin Wolfers Business and Public Policy Department Wharton School, University of Pennsylvania 3620 Locust Walk Room 1456 Steinberg-Dietrich Hall Philadelphia, PA 19104-6372 and NBER jwolfers@wharton.upenn.edu
I. Introduction Over much of the last half-century, the legal and political history of the death penalty in the United States has closely paralleled the debate within social science about its efficacy as a deterrent. Sociologist Thorsten Sellin s careful comparisons of the evolution of homicide rates in contiguous states from 1920 to 1963 led to doubts about the existence of a deterrent effect caused by the imposition of the death penalty (Sellin 1967a, 1967b). This work likely contributed to the waning reliance on capital punishment, and executions virtually ceased in the late 1960s. In the 1972 Furman decision, the Supreme Court ruled that existing death penalty statutes were unconstitutional. In 1975, Isaac Ehrlich s analysis of national time-series data led him to claim that each execution saved eight lives (Ehrlich 1975). Solicitor General Robert Bork cited Ehrlich s work to the Supreme Court a year later, and the Court, while claiming not to have relied on the empirical evidence, ended the death penalty moratorium when it upheld various capital punishment statutes in Gregg v. Georgia and related cases. 1 The injection of Ehrlich s conclusions into the legal and public policy arenas, coupled with the academic debate over Ehrlich s methods, led the National Academy of Sciences to issue a 1978 report which argued that the existing evidence in support of a deterrent effect of capital punishment was unpersuasive (Blumstein et al., 1978). Over the next two decades, as a series of academic papers continued to debate the deterrence question, the number of executions gradually increased, albeit to levels much lower than those seen in the first half of the twentieth century. The current state of the debate over capital punishment is one of disagreement, controversy, and division. Governor George Ryan of Illinois suspended executions in that state in 2000 and commuted the death sentences of all Illinois death row inmates in 2003. As a number of other jurisdictions were considering similar moratoria, New York s highest court ruled in 2004 that the state s death penalty statute was unconstitutional. Executions in California are virtually nonexistent, although the state continues to add prisoners to death row at a rapid pace. 2 Meanwhile, executions continue apace in Texas, which accounts for over one-third of all post-gregg executions. A host of more recent academic studies has examined the death penalty over the last decade, with mixed results. While Katz, Levitt and Shustorovich (2003) found no robust evidence of deterrence, several researchers claim to have uncovered compelling 1. In Gregg, Justice Stewart stated: Although some of the studies suggest that the death penalty may not function as a significantly greater deterrent than lesser penalties, there is no convincing empirical evidence either supporting or refuting this view. Yet, he then asserted: We may nevertheless assume safely that there are murderers, such as those who act in passion, for whom the threat of death has little or no deterrent effect. But for many others, the death penalty undoubtedly is a significant deterrent. Justice Stewart did not clarify whether he believed that murders would increase if convicted murderers who might otherwise be executed instead received sentences of life without parole, and, if so, on what basis this might be safely assumed. 2. By the end of 2004, California s death row population was the highest in the country (637 inmates). See Bonczar and Snell (2005). 1
evidence to the contrary. 3 This latter research appears to have found favor in Sunstein and Vermeule (2005), which describes the studies as powerful and impressive and refers to many decades worth of data about [capital punishment s] deterrent effects. In light of this conflicting evidence, our aim in this paper is to provide a thorough assessment of the statistical evidence on this important issue. We test the sensitivity of existing studies in a number of intuitively plausible ways testing their robustness to alternative sample periods, comparison groups, control variables, functional forms, and estimators. We find that the existing evidence for deterrence is surprisingly fragile, and even small changes in specifications yield dramatically different results. Our key insight is that the death penalty at least as it has been implemented in the United States since Gregg ended the moratorium on executions is applied so rarely that the number of homicides it can plausibly have caused or deterred cannot be reliably disentangled from the large year-to-year changes in the homicide rate caused by other factors. Our estimates suggest not just reasonable doubt about whether there is any deterrent effect of the death penalty, but profound uncertainty. We are confident that the effects are not large, but we remain unsure even of whether they are positive or negative. The difficulty is not just one of statistical significance: whether one measures positive or negative effects of the death penalty is extremely sensitive to very small changes in econometric specifications. Moreover, we are pessimistic that existing data can resolve this uncertainty. We start in Section II by sketching the relevant economic theories of crime and the difficulties in identifying their effects. We then begin our tour of the statistical evidence. Section III analyzes aggregate time-series evidence, while Section IV analyzes first differences the change in homicide rates that occurs following death penalty reforms. In Section V, we turn to panel data analysis, and Section VI analyzes the key instrumental variables estimates. Section VII contains our attempt at reconciling the conflicting evidence, assessing the limited precision with which we might be able to pin down the deterrent effect of the death penalty with existing data. Our organizing theme involves an attempt to examine the evidence compiled by previous scholars with the aim of highlighting the ways in which this evidence can both provide insight but also potentially mislead policy analysts. Section VIII concludes. II. Theory: What Are the Implications of the Death Penalty for Homicide Rates? The theoretical premise underlying the deterrence argument is simple: raise the price of murder for criminals, and you will get less of it. In general, the death penalty raises the price of homicide as long as execution is worse than life imprisonment for most potential murderers. 4 3. These studies include Dezhbakhsh, Rubin and Shepherd (2003), Mocan and Gittings (2003) and Zimmerman (2004b). Joanna Shepherd, an author of several studies finding a deterrent effect, has recently argued before Congress that recent research has created a strong consensus among economists that capital punishment deters crime, going so far as to claim that [t]he studies are unanimous. See Shepherd (2004c). Upon further probing from the committee chairman about the findings of anti-death penalty advocates that are 180 degrees from your conclusions,, Shepherd responded: There may be people on the other side that rely on older papers and studies that use outdated statistical techniques or older data, but all of the modern economic studies in the past decade have found a deterrent effect. So I am not sure what the other people are relying on. 4. The general rule is subject to a caveat. A necessary condition for the death penalty to act as a deterrent is that 2
While this argument is qualitatively reasonable, its quantitative significance may be minor. In 2003, there were 16,503 homicides (including nonnegligent manslaughter), but only 144 inmates were sentenced to death (Federal Bureau of Investigation, 2003; Bonczar and Snell, 2004). Moreover, of the 3,374 inmates on death row at the beginning of the year, only 65 were executed. Thus, not only did very few homicides lead to a death sentence, but the prospect of execution did not greatly affect the life expectancy of death row inmates. Indeed, Katz, Levitt, and Shustorovich (2003) have made this point quite directly, arguing that the execution rate on death row is only twice the death rate from accidents and violence among all American men and that the death rate on death row is plausibly lower than the death rate of violent criminals not on death row. As such they conclude that it is hard to believe that in modern America the fear of execution would be a driving force in a rational criminal s calculus. 5 Moreover, even if there were a deterrent effect, capital punishment is sufficiently expensivethat it may potentially divert resources away from more effective crime prevention strategies (Fagan, 2005). A more sociological approach notes that there may be social spillovers as statesanctioned executions cheapen the value of life, potentially demonstrating that deadly retribution is socially acceptable. Thus, executions may actually stimulate more homicide through the so-called brutalization effect. 6 With theory inconclusive, we now turn to examining the data. III. A Century of Murders and Executions Several of the early studies of the death penalty were based on analysis of the aggregate U.S. time-series data. Figure 1 depicts the homicide and execution rates for the United States over the last century. Because data issues can be a concern with crime data, we present two series for homicides one from the Uniform Crime Reports and the other compiled from Vital Statistics sources, based on death certificates. 7 (most) potential killers view a death sentence to be worse than life imprisonment. For these execution-averse potential murderers, a perverse incentive will exist once the criminal believes his conduct has reached the threshold needed to secure a death sentence. At that point, with marginal deterrence lost, the cost of killing to avoid capture goes to zero, and the death penalty may increase incentives to kill to avoid execution. In other words, if the death penalty is considered much worse than incarceration by criminals, it will have a direct deterrent effect and an indirect antideterrent effect (witness elimination murders and murders that facilitate escape) by those already subject to execution for their pre-existing crimes. 5. Katz, Levitt, and Shustorovich (2003). On the other hand, even if criminals are not effective calculators, the vivid character of the death penalty might give criminals pause to a greater degree than its likely risk of implementation alone would warrant. The recent literature suggests two possibilities: (1) many individuals treat events with small likelihoods of occurrence as having zero probability, which would mean that the highly unlikely event of execution would essentially have a zero possibility of deterring instead of just a very small likelihood of deterring; and (2) certain catastrophic events that occur with low frequency are given greater prominence in decision-making than their likelihood warrants if individuals are given frequent vivid reminders of these events, which could conceivably make the death penalty more of a deterrent than a rational calculation of the risk such as that offered by Katz, Levitt, and Shustorovich (2003) would suggest. See Cooter and Ulen (2004). Again, only empirical investigation can answer the question of which effect would be more dominant on potential murderers. 6. Bowers and Pierce (1980). See also Steiker (2005), which discusses the brutalization effect as initially brought up in Sunstein and Vermeule (2005). 7. Given the incomplete nature of Vital Statistics reporting in the first half of the century, we rely on Eckberg s (1995) estimates of the homicide rate. 3
Figure 1 12.5 Homicide and Execution in the United States Execution rate (right axis) Homicide rate: Vital Statistics, corrected Homicide rate: FBI.25 10.2 Homicides per 100,000 residents 7.5 5 2.5 0 Ehrlich's sample Passell & Taylor's sample.15.1.05 0 Executions per Homicide Dezhbakhsh & Shepherd's sample 1900 1920 1940 1960 1980 2000 No clear correlation between homicides and executions emerges from this long time series. In the first decade of the twentieth century, execution and homicide rates seemed roughly uncorrelated, followed by a decade of divergence as executions fell sharply and homicides trended up. Then for the next forty years, execution and homicide rates again tended to move together first rising together during the 1920s and 1930s, and then falling together in the 1940s and 1950s. As the death penalty fell into disuse in the 1960s, the homicide rate rose sharply. The death penalty moratorium that began with Furman in 1972 and ended with Gregg in 1976 appears to have been a period in which the homicide rate rose. The homicide rate then remained high and variable through the 1980s while the rate of executions rose. Finally, homicides dropped dramatically during the 1990s. By any measure, the resumption of the death penalty in recent decades has been fairly minor, and both the level of the execution rate and its year-to-year changes are tiny: since 1960 the proportion of homicides resulting in execution ranged from 0% to 3%. By contrast, there was much greater variation in execution rates over the previous sixty years, when the execution rate ranged from 2.5% to 18%. This immediately hints that even with modern econometric methods it is unlikely that the last few decades generated enough variation in execution rates to overturn earlier conclusions about the deterrent effect of capital punishment. This simple chart reconciles many of the conflicting results from the death penalty literature. Ehrlich (1975) provocatively argued that he could isolate the movements in the homicide rate caused by changing execution policies, concluding that each execution 4
deterred an average of seven to eight homicides. Passell and Taylor (1977) showed that Ehrlich s result relied heavily on movements from 1963 to 1969. When they limited the Ehrlich model to the period from 1935 to 1962, they found no deterrent effect. Indeed, this led the subsequent National Academy of Sciences report to argue that the real contribution to the strength of Ehrlich s statistical findings lies in the simple graph of the upsurge of the homicide rate after 1962, coupled with the fall in the execution rate in the same period. While Ehrlich s used a sophisticated theoretical and econometric model, the National Academy report went on to note that his whole statistical story lies in this simple pairing of these observations and not in the theoretical utility model, the econometric type specification, or the use of best econometric method. Everything else is relatively superficial and dominated by this simple statistical observation. (Klein et al., 1978.) Most recently, Dezhbakhsh and Shepherd (2004, henceforth DS ) have analyzed national time-series data from 1960 to 2000. In light of Figure 1, it is not surprising that they find a strong negative relationship between executions and the homicide rate. While they do not report their results in terms of lives saved per execution, their estimates suggest that each execution reduces the homicide rate by about 0.05 homicides per 100,000 people, which translates to around 150 (!) fewer homicides per execution. Why does the correlation between executions and homicides vary so much over time? One possibility is simply that the deterrent effect has truly changed over time and that capital punishment has suddenly become very effective starting in the 1990s. If so, more recent estimates are obviously to be preferred. If anything, however, administration of the death penalty has become both slower and execution methods less vivid, which would lead one to expect that any deterrent effect would be weakened in this period. Alternatively, it may be that despite efforts in all of these studies to control for a range of social and economic trends, other omitted factors are preventing the relationship between executions and homicides from being correctly captured. To illustrate that these factors are indeed omitted from national time-series analyses, we introduce comparison groups into the analysis. IV. The Importance of Comparison Groups As economists have come to understand how difficult it is to control convincingly for all relevant factors, many have lost faith in the ability of pure time-series analysis to isolate causal relationships. An alternative approach borrows a page from medical studies, emphasizing the importance of comparing results among those groups or regions receiving the treatment of the death penalty with a comparison group that is untreated, but otherwise susceptible to similar influences (a placebo or control group ). If the execution rate is driving the homicide rate, then one should not expect to see a similar pattern in the homicide time series for these comparison groups. A. Canada versus the United States Given its proximity and different pattern of reliance on capital punishment, Canada presents an interesting comparison group for the United States, and Figure 2 compares the evolution of their homicide rates through time. The Canadian homicide rate 5
(right axis) is roughly one-third as high and one-third as variable as the rate in the United States (left axis). Figure 2 12 Homicide Rates and the Death Penalty in the U.S. and Canada 4 Homicides per 100,000 residents 9 6 3 Bill C-168 Furman Decision Gregg Decision U.S. Vital Stats Left axis Canada Vital Stats Right axis 3 2 1 Death penalty in U.S. Abolished Death penalty in U.S. re-established 0 Death penalty in Canada Canadian death penalty abolished (some exception apply until 1998) 0 1950 1953 1956 1959 1962 1965 1968 1971 1974 1977 1980 1983 1986 1989 1992 1995 1998 2001 The most striking finding is that the homicide rate in Canada has moved in virtual lockstep with the rate in the United States, while approaches to the death penalty have diverged sharply. Both countries employed the death penalty in the 1950s, and the homicide trends were largely similar. However, in 1961, Canada severely restricted its application of the death penalty (to those who committed premeditated murder and murder of a police officer only); in 1967, capital punishment was further restricted to apply only to the murder of on-duty law enforcement personnel. As a result of these restrictions, no executions have occurred in Canada since 1962. Nonetheless, homicide rates in both the United States and Canada continued to move in lockstep. The Furman case in 1972 led to a death penalty moratorium in the United States. While many death penalty advocates attribute the subsequent sharp rise in homicides to this moratorium, a similar rise is equally evident in Canada, which was obviously unaffected by this U.S. Supreme Court decision. In 1976, the capital punishment policies of the two countries diverged even more sharply: the Gregg decision led to the reinstatement of the death penalty in the United States, while the death penalty was dropped from the Canadian criminal code. Over the subsequent two decades, homicide rates remained high in the United States while they fell in Canada. It is only over the last decade that homicide rates have started to decline in the United States, a fact that is difficult to attribute to reforms occurring decades earlier. 6
The Canadian move towards abolition is also interesting because it represented a major policy shock: prior to abolition, the proportion of murderers executed in Canada was considerably higher than that in the United States. 8 Of course, one might still be concerned that Canada is not quite an appropriate comparison group perhaps Canadaspecific factors were driving its homicide rate down following the abolition of its death penalty, back up during the U.S. moratorium, and back down over the ensuing period effectively hiding the effects of execution-related changes. As such, it might be worth considering an alternative comparison group that is more clearly subject to the same set of economic and social trends. B. Non-Death Penalty States versus Other States in the United States Naturally, those states that have never had the death penalty should be unaffected by changes in death penalty policy throughout the rest of the country. Figure 3 facilitates the comparison of homicide rates across states that should be influenced by changes in death penalty law and practice from those that should not. 8. A comparison of the Canadian abolition experiment with the post-furman Texas experiment is instructive. Over the two decades prior to abolition, the annual number of homicides in Canada fluctuated from around 150 to 250. From the 1970s to the 1990s, the number of murders in Texas was about ten times larger, fluctuating from 1200 to 2500 per year, despite having only half the Canadian population. However, the number of executions was fairly similar: roughly seven per year in both Canada and Texas during the respective periods. Specifically, Canada had 148 executions for the years 1943 to 1962 (two decades before policy change), or an average of 7.4 executions per year. From 1977 to 1996 (two decades after the moratorium), Texas averaged seven executions per year. As a result, the change in the likelihood that a homicide would result in execution caused by the Canadian death penalty abolition is an order of magnitude larger than that caused by Texas s reinstatement. 7
Figure 3 Homicide Rates in the United States Controls: Non-death penalty states Treatment states (all others) Annual homicides per 100,000 residents 12 9 6 3 0 Last execution until 1977 Furman decision: Death penalty abolished Gregg decision: Death penalty re-instated 1960 1970 1980 1990 2000 Year Non-death penalty states are those without a death penalty throughout 1960-2000: AK HI ME MI MN WI We begin by considering the cleanest comparison group: there are six states that have not had the death penalty on the books at any point in our 1960 to 2000 sample. Deterrence in these states was unaffected by either the Gregg or Furman decisions, and hence homicide rates in these states are a useful baseline for comparing the evolution of the homicide rates in other states. The remaining states are considered treatment states because either Gregg abolished their existing death penalties or Furman enabled their subsequent reinstatement (or, more commonly, both). Again, the most striking finding is the close co-movement of homicide rates in these two groups of states. Both sets of states experienced higher homicide rates during the death penalty moratorium than over the subsequent decade; the gap widened for the subsequent decade and narrowed only in the late 1990s. It is very difficult to find evidence in these Supreme Court-mandated natural experiments that the death penalty has any causal effects at all on the homicide rate. Clearly, most of the action in homicide rates in the United States is unrelated to capital punishment. The lesson from examining these time-series data is that it is crucial to take account of the fact that most of the variation in homicide rates is driven by factors that are common to both death penalty and non-death penalty states, and to both the United States and Canada. The empirical difficulty is that these factors may be spuriously correlated with executions, and hence the plausibility of any attempt to isolate the causal effect of executions rests heavily on either finding useful comparison groups or convincingly controlling for these other factors. 8
This issue is particularly relevant to Dezhbakhsh and Shepherd s analysis of changes in capital punishment laws. These authors present a series of before-and-after comparisons, focusing only on states that abolished the death penalty or only on states adopting the death penalty. Unfortunately, by focusing only on the states experiencing these reforms, the authors risk confounding the effects of changes in capital punishment laws with broader forces that are equally evident in homicide data in states not experiencing these reforms. The DS analysis is reproduced in Panel A of Table 1. The authors analyze each change in state laws during the sample. For each instance in which the death penalty was abolished, they compare the homicide rate one year prior to and one year after the abolition and report the average and median percentage change across all such abolitions. They also repeat this analysis for two- and three-year windows and for those times in which the death penalty was reinstated. Panel A exactly reproduces the numbers from their study, while Panel B shows our attempt at replicating their analysis. 9 In each case, they find that the abolition of the death penalty was associated with rising homicide rates, and the reinstatement of the death penalty was associated with falling homicide rates. Our replication largely succeeds in generating similar estimates: abolition of the death penalty is associated with a 10% to 20% increase in homicide, while reinstatement is associated with a 5% to 10% decrease. 9. They drop outliers from their calculation of the means, and we follow them in doing so; the medians are obviously more robust to such outliers. We were best able to match their numbers by assuming that North Dakota had capital punishment until Furman, although this seems a questionable judgment. Unfortunately, we cannot be confident of their coding because the authors were unwilling to share their data with us. 9
TABLE 1: ESTIMATING HOW CHANGES IN DEATH PENALTY LAWS EFFECT MURDER SELECTED BEFORE AND AFTER COMPARISONS: 1960-2000 Dependent Variable: % Change in State Murder Rates Around Regime Changes 1-Year Window (1) Death Penalty Abolition 2-Year Window (2) 3-Year Window (3) Death Penalty Reinstatement 2-Year 3-Year Window Window (5) (6) 1-Year Window (4) Panel A: Reproducing Dezhbakhsh and Shepherd Tables 5, 6 Mean Change 10.1% *** (2.8) 16.3% *** (2.2) 21.9% *** (2.5) -6.3% ** (3.4) -6.4% ** (2.9) -4.1% (2.9) Median Change 8.3% 14.9% 18.4% -9.3% -6.8% -7.5% Number of States Where Homicide Increased Mean Change 33/45 39/45 41/45 12/41 16/39 13/39 Panel B: Our Replication: Changes Around Death Penalty Shifts (Treatment) 10.1% *** (2.9) 16.0% *** (2.3) 21.5% *** (2.6) -6.3% * (3.4) -7.0% ** (2.9) -3.8% (2.9) Median Change 8.5% 13.8% 18.5% -9.3% -8.5% -7.4% Number of States Where Homicide Increased 35/46 39/46 41/46 12/41 15/39 14/39 Panel C: Our Innovation: Changes in Comparison States (Control) Mean Change 8.7% *** (0.5) 16.0% *** (0.8) 20.6% *** (1.1) -7.5% *** (1.5) -6.6% *** (1.5) -3.7% *** (1.3) Median Change 8.5% 16.1% 20.9% -11.5% -9.8% -5.2% Number of States Where Homicide Increased Mean Change Median Change 44/46 44/46 44/46 7/41 8/39 8/39 1.4% (2.9) <0.001% (2.7) Panel D: Difference-in-Difference Estimates (Treatment-Control) -0.1% (2.4) -2.3% (2.5) 0.9% (2.8) -2.4% (3.6) 1.2% (3.7) 2.2% (3.5) -0.5% (3.2) 1.3% (4.5) -0.1% (3.2) -2.2% (2.0) Notes: Sources, data, and specification are as described in DS. Standard errors are in parentheses. Standard errors on median change are estimated by bootstrap. ***, **, and * denote statistically significant at 1%, 5%, and 10%, respectively. Each cell reports the mean or median percentage change in homicide rates in states that either abolished or reinstated the death penalty. The one-year window reports how murder rates changed from one year before abolition or reinstatement to one year after; the two-year window is the change in the homicide rate over the two years subsequent to reform compared to the two years before, with similar calculations for the three-year window. Panel A and our replication in Panel B might seem to suggest that crime rises when the death penalty is abolished and falls when it is reinstated, but Panel C shows that the same changes in murder rates also occur in the states that do not alter their death penalty laws (the control group). Panel D shows no differential change in murder rates between the treatment (change in death penalty law) and control groups (no change in death penalty law). 10
However, these calculations may be confounding the effects of abolition or reinstatement of the death penalty with other broader trends. To test for this, we provide a comparison group for the abolition states in Panels A and B: we collect data on the change in homicide rates in all states that did not abolish the death penalty in that year. 10 These states did not experience any reform and so constitute a natural control group. Comparing Panel B with Panel C shows that the measured effects in states that changed their death penalty laws are similar to those in states that did not. Indeed, some of the effects in the comparison states are larger than those in the treatment states. Panel D in Table 1 shows this formally, computing the difference between means (or medians) in treatment and control states effectively a difference-in-differences approach. In no case do the figures in Panel D provide statistically or economically significant evidence for or against the deterrent effect. Half of the six estimates of the effects of abolition are positive and half are negative; the same is true for the effects of reinstating the death penalty. None of the estimates in Panel D are statistically significant. In sum, this analysis provides no evidence that the death penalty affects homicide rates and does not even paint a consistent picture of whether it is more likely to raise or lower rates. The estimates in Table 1 involve direct comparison of treatment and control states, but they do not account for other factors that may have affected the homicide rate differently in each state. This suggests that a panel data analysis may provide more reliable estimates. With this motivation, we now turn to expanding the above analysis into a formal panel structure. V. Panel Data Methods The simplest panel data extension to the previous analysis above involves running the regression: Murders s,t = β Death Penalty Law + State Effects + Time Effects + λcontrols + ε ( Population / 100,000 ) s,t (1) 1 s,t s t s,t s,t s t where the dependent variable is the homicide rate in a given state and year, and the variable of interest is an indicator set equal to one when a state has an active death penalty law. As such, β 1 measures the effects on the homicide rate of a state having a death penalty law in place. The inclusion of state fixed effects controls for persistent differences across states, the time fixed effects control for national time trends that are common across states, and control variables include indicators of state economic conditions, demographics, and law enforcement variables. Following Dezhbakhsh and Shepherd, we restrict our sample to the period from 1960 to 2000 and run a weighted least squares regression, clustering standard errors at the state level. In Column 1 of Table 2, we report the results from Dezhbakhsh and Shepherd s estimation, in which they estimate the above equation without year fixed effects, but controlling for decade fixed effects. 11 Column 2 shows our replication attempt based on 10. Similarly, we collect the appropriate comparison groups for the states that reinstated the death penalty. 11. It is easy to lose this point: Dezhbakhsh and Shepherd refer only to controlling for time-specific binary variables, and it was only through corresponding with the authors that we understood this to mean decade rather than 11
independently collected data (but using the same sources). 12 While our coefficient estimates do not precisely match theirs, the difference is tolerable. The real difference comes in the estimate of the standard error (which speaks to the persuasiveness of the data): we report a standard error nearly three times larger than theirs, and hence our coefficient is statistically insignificant. We do not know for certain the source of this divergence, and the authors provided no useful guidance. Thus, despite their claims that their estimates of standard errors are further corrected for possible clustering effects dependence within clusters (groups), our best guess is that they report simple ordinary least squares (OLS) standard errors (Dezhbakhsh and Shepherd, 2004). As Bertrand, Duflo and Mullainathan (2004) note, using OLS standard errors in panel estimation involving autocorrelated data may severely understate the standard deviation of the estimators (and hence exaggerate claims of statistical significance). TABLE 2: PANEL DATA ESTIMATES OF THE EFFECTS OF DEATH PENALTY LAWS ON MURDER RATES: 1960-2000 Dependent Variable: Annual Homicides Per 100,000 Residents s,t Dezhbakhsh and Shepherd (1) Death Penalty Law -0.87 *** (.21) Active Death Penalty Law ( 1 Execution in Previous Decade) Our Replication (2) -0.95 (.57) Controlling for Year Fixed Effects (3) -0.47 (.74) De Facto Versus De Jure Laws (4) -0.57 (.63) Inactive Death Penalty Law (No Executions in Previous Decade) -0.45 (.77) State Fixed Effects Yes Yes Yes Yes Decade Fixed Effects Yes Yes Yes Yes Year Fixed Effects No No Yes Yes Adjusted R 2.804.791.834.834 Sample Size (Excludes DC, HI) (unknown) 2009 2009 2009 Notes: Sources and data are as described in DS. Observations are for a single state in a single year; population-weighted least squares regression was utilized; ***, **, and * denote statistically significant at 1%, 5%, and 10%, respectively. DS find that a death penalty law is associated with less crime, but our replication in Column 2, as well as other plausible estimates in Columns 3 and 4, show no significant effect. Further controls include per capita real income, the unemployment rate, police employment, proportions of the population nonwhite, aged 15-19, and aged 20-24. year fixed effects. Indeed, they never use the term decade in connection with their econometric specification. 12. While Dezhbakhsh and Shepherd were unwilling to share their data for this Article, we have reconstructed it as closely as possible using the sources noted in their data appendix. 12
Given the importance of not confounding overall crime trends in the 1970s with changes in death penalty laws (a lesson illustrated sharply in Table 1), we add controls for year fixed effects in Column 3. Indeed, in failing to control for year fixed effects, Dezhbakhsh and Shepherd s study is a clear outlier in the literature. 13 This is important: as Figure 2 shows, homicide rates were higher during the death penalty moratorium than during the early or late 1970s, and so simply controlling for the average crime rate in the 1970s would lead the regression to find a deterrent effect, even though the same pattern was observed in states that experienced no change to their death penalty laws. It turns out that controlling for these confounding trends cuts the coefficient on the death penalty in half and makes the coefficient clearly statistically insignificant. One possible objection to this analysis is that there are many states that are de jure death penalty states but de facto nonexecuting, and hence, the binary legal classification is inadequate. Thus, in Column 4 we make a distinction between those states that actively apply their death penalty statutes and those that do not. We define a death penalty statute as inactive if that state had no executions over the preceding ten years, an admittedly crude approach. In each case, we find no statistically significant effects of the death penalty. Moreover, the data suggest that active death penalty statutes are neither more nor less (in)effective than inactive death penalty statutes. The most important finding in Table 2 is simply how difficult it is to isolate any causal effects with confidence. The standard errors in our preferred estimates suggest that even if death penalty laws deterred 15% of all homicides (or caused 15% more homicides), the data speak so unclearly that they could not rule out the possibility of no effect. These data also allow us to extend the analysis of the distribution of estimates across death penalty experiments. Specifically, we extend our panel data approach, but rather than analyzing a single variable describing whether a state has a death penalty law, we estimate separate effects for each experiment. 14 That is, for each of the fortyfive death penalty abolitions in the sample, we analyze its effects by including a separate dummy variable set equal to one for that state subsequent to the law change. We also include forty-one further dummy variables for each death penalty adoption in the sample. In all other respects, the specification remains the same as in Dezhbakhsh and Shepherd, although we continue to control for year fixed effects. Table 3 reports these results. 13. Papers using year fixed effects include Dezhbakhsh, Rubin and Shepherd (2003), Shepherd (2005) and Shepherd (2004b). Mocan and Gittings (2003) both include year fixed effects and control for state-specific time trends. Katz, Levitt and Shustorovich (2003) control for year fixed effects and, in various specifications, also control for statespecific trends, state-decade interactions, and separate time fixed effects by region. 14. As such, this approach is also a natural extension of the analysis in Table 1, with the advantage that the panel analysis allows for regression-adjusted comparisons and takes account of the full time series, rather than an arbitrary comparison window. Note that while Table 1 included Washington, D.C., missing police data force us to drop it from this analysis. 13
TABLE 3: ESTIMATING THE INDIVIDUAL EFFECTS OF DEATH PENALTY REFORM ON THE HOMICIDE RATE FOR 41 REINSTATEMENTS AND 45 ABOLITIONS: 1960-2000 Dependent Variable: Annual Homicides per 100,000 Residents s,t State Death Penalty Reinstatement Death Penalty Abolition Year Estimated Effect 95% Confidence Interval Year Estimated Effect 95% Confidence Interval Alabama 1976-3.2 (-4.1, -2.4) 1972-1.2 (-2.8, 0.5) Arizona 1976 1.1 (0.2, 1.9) 1972-1.5 (-3.2, 0.2) Arkansas 1976-0.5 (-1.4, 0.3) 1972-2.4 (-4.1, -0.8) California 1977 2.3 (1.3, 3.2) 1972 1.1 (-0.8, 2.9) Colorado 1976-0.8 (-1.9, 0.3) 1972-1.7 (-3.7, 0.2) Connecticut 1976 0.6 (-0.8, 2.0) 1972-2.5 (-4.4, -0.6) Delaware 1976-2.2 (-3.1, -1.4) 1972-2.7 (-4.6, -0.7) 1961-1.6 (-2.2, -1.0) Florida 1976-3.4 (-4.2, -2.6) 1972-0.2 (-2.0, 1.5) Georgia 1976-5.1 (-6.0, -4.3) 1972 1.0 (-0.6, 2.7) Idaho 1976 0.2 (-0.6, 1.0) 1972-2.8 (-4.6, -1.0) Illinois 1976 0.3 (-0.7, 1.2) 1972-0.3 (-2.2, 1.6) Indiana 1976 0.2 (-0.5, 1.0) 1972-0.4 (-2.2, 1.4) Iowa 1965-3.2 (-4.7, -1.6) Kansas 1994 3.1 (1.8, 4.4) 1972-2.2 (-4.1, -0.3) Kentucky 1976-1.6 (-2.5, -0.8) 1972-1.6 (-3.3, 0.0) Louisiana 1976 1.4 (0.7, 2.1) 1972 1.5 (-0.2, 3.2) Maryland 1976-0.6 (-1.6, 0.4) 1972-0.1 (-2.1, 1.9) Massachusetts 1982-0.3 (-1.2, 0.7) 1972-2.8 (-4.6, -0.9) 1984-0.3 (-1.0, 0.5) Mississippi 1976-1.9 (-2.9, -0.9) 1972 0.6 (-1.1, 2.3) Missouri 1976 0.3 (-0.5, 1.0) 1972-1.4 (-3.1, 0.4) Montana 1976 0.6 (-0.5, 1.8) 1972-2.6 (-4.5, -0.7) Nebraska 1976 0.3 (-0.5, 1.1) 1972-2.9 (-4.8, -0.9) Nevada 1976-0.8 (-1.8, 0.3) 1972 1.2 (-0.5, 2.9) New Hampshire 1991 0.1 (-0.7, 1.0) 1972-3.5 (-5.4, -1.6) New Jersey 1982-1.3 (-2.3, -0.2) 1972-1.3 (-3.3, 0.7) New Mexico 1979 0.3 (-0.5, 1.1) 1969 0.5 (-0.9, 1.8) New York 1995-2.9 (-4.4, -1.5) 1965 2.9 (1.0, 4.7) North Carolina 1977-2.4 (-3.4, -1.5) 1972-1.3 (-3.0, 0.3) North Dakota 1972-3.8 (-5.6, -2.0) Ohio 1976-1.2 (-1.9, -0.5) 1972-0.4 (-2.2, 1.3) Oklahoma 1976 1.1 (0.3, 1.8) 1972-1.8 (-3.5, -0.1) Oregon 1978-0.6 (-1.6, 0.4) 1964-1.8 (-2.8, -0.7) Pennsylvania 1976-0.1 (-0.9, 0.7) 1972-0.9 (-2.6, 0.8) Rhode Island 1977-1.1 (-2.4, 0.2) 1984 0.6 (0.1, 1.0) South Carolina 1976-4.8 (-5.6, -3.8) 1972-0.5 (-2.2, 1.2) South Dakota 1979 0.5 (-0.1, 1.1) 1972-4.4 (-6.3, -2.6) Tennessee 1976-2.1 (-2.9, -1.3) 1972-0.1 (-1.8, 1.7) Texas 1976-0.1 (-1.1, 0.9) 1972-0.1 (-1.7, 1.6) Utah 1976 0.8 (-0.1, 1.6) 1972-3.1 (-4.8, -1.4) Vermont 1965-2.9 (-4.4, -1.4) Virginia 1976-2.7 (-3.6, -1.7) 1972-2.0 (-3.8, -0.3) Washington 1976 0.7 (-0.5, 1.9) 1972-1.8 (-3.6, -0.0) West Virginia 1965-2.8 (-4.5, -1.0) Wyoming 1977-0.9 (-1.5, -0.2) 1972-3.4 (-5.3, -1.4) Simple Average -0.70-1.32 Precision-Weighted Average -0.67-0.86 14
Population-weighted Average -0.72-0.39 Notes: This table shows the effect on murder rates of 41 reinstatements of death penalty laws and 45 abolitions of such laws. It is derived from the same data and models that were used to estimate aggregated effects of such legal changes averaged over all switching states. Alaska, Hawaii, Maine Michigan, Minnesota and Wisconsin never had the death penalty throughout the sample period. The District of Columbia and Hawaii had missing police data. Sources, data, and specification follow DS as described in Table 2, except that we add year fixed effects and include 41 death penalty reinstatements and 45 death penalty abolition dummy variables (set equal to zero before the change and one subsequently), rather than a single binary variable covering all 86 experiments. Controls include per capita real income; the unemployment rate; police employment; proportions of the population nonwhite, aged 15-19 and aged 20-24; and state and year fixed effects. Standard errors are in parentheses, clustered at the state level. The precision-weighted average is generated by weighting by the inverse of the squared standard error. For neither death penalty abolitions nor reinstatements do we see a particularly coherent picture. Estimates of the effect of death penalty abolition on the homicide rate (conditional on the control variables) are positive in eight cases and negative in thirtyseven cases. Likewise, reinstatement of the death penalty was subsequently associated with a higher homicide rate in seventeen states and a lower rate in twenty-four states. On average, the homicide rate appears to be lower than otherwise suggested by developments in the control variables following either abolition or reinstatement of the death penalty. That said, these differences are not statistically significant, and these comparisons merely point to the difficulty in discerning any causal effect of death penalty laws. Figure 4 shows the distribution of before-and-after comparisons across states, using the data in Table 3. These distributions highlight the problem of getting the data to speak clearly: the variance of individual state homicide rates is so great that it is difficult to discern the average effects of these changes with any precision, even with 86 experiments to analyze. Shepherd has reanalyzed three related papers that examine the effects of executions (rather than the presence of a death penalty law), and she also finds that there are about as many states whose experiences are consistent with the deterrence hypothesis as with anti-deterrence. 15 15. See Shepherd (2005), reanalyzing data from Dezhbakhsh and Shepherd (2004), Dezhbakhsh, Rubin and Shepherd (2003) and Shepherd (2004b). Shepherd argues that anti-deterrence is evident in some states because they do not execute sufficient convicts to reach a threshold effect required for deterrence. 15
Figure 4 Distribution of Regression-Estimated Effects Across States Death Penalty Reinstatement Death Penalty Abolition Density 0.05.1.15.2.25 Density 0.05.1.15.2.25-6 -4-2 0 2 4 6 Estimated Effect on Homicide Rate Annual murders per 100,000 people -6-4 -2 0 2 4 6 Estimate Effect on Homicide Rate Annual murders per 100,000 people Kernel density estimates using Epanechnikov kernel It is worth noting that Mocan and Gittings (2003) also include an analysis of the efficacy of death penalty laws over a sample running from 1977 to 1997, although their regressions only include data from 1980 to 1997. Despite their professed confidence in their results, Mocan and Gittings analysis includes only six policy change experiments. We have reanalyzed their data following a similar design to that above: we follow their data and programs (which they graciously shared) but analyze the death penalty effects separately for each state, making sure to control for the same variables as in their main specification. For the four states adopting the death penalty, their specification suggests that homicide rates were subsequently higher in Kansas and New Hampshire and lower in New Jersey and New York. In their sample, only Massachusetts and Rhode Island abolished the death penalty, and in both cases homicide rates fell following the law change (relative to the baseline established by their regression). These facts make it difficult to conclude with any confidence that the death penalty raises or lowers homicide rates. 16 Given the demonstrated difficulties in linking the presence of death penalty laws with homicide rates, several authors also have tried to exploit variation in the intensity with which death penalty laws have been applied. Consequently, the variable of interest in these studies does not describe the presence of a death penalty law but rather a variable 16. That Mocan and Gittings obtain statistically significant estimates reflects the fact that New York and New Jersey were the two states consistent with deterrence, and their influence in a population-weighted regression dwarfs that of the four states inconsistent with deterrence. 16
measuring the propensity to invoke the death penalty. The intensity with which a state pursues death penalty prosecutions may be highly politicized, raising the possibility that such estimates may reflect omitted factors related to the political economy of punishment. On the demand side, variation in crime rates may change the political pressure for executions. Equally on the supply side, it seems plausible that more vigorous deployment of the death penalty might occur at the same time that the government elects to get tough on crime along a range of other dimensions, including sentencing, prison conditions, arrests, police harassment, and so on. As these studies move beyond the sharp judicial or legislative experiments analyzed above, the issues involved in distinguishing correlation from causation may become even more salient. However as Katz, Levitt, and Shustorovich (2003) emphasize, beyond the usual difficulties in establishing a causal relationship, there is a much simpler statistical dilemma: the annual number of executions fluctuates very little while the number of homicides varies dramatically. Under these conditions, it is a difficult challenge to extract the execution-related signal from the noise in homicide rates. (Katz, Levitt, and Shustorvich, 2003.) Indeed, following their own empirical investigation for the years 1950 to 1990, Katz, Levitt, and Shustorovich (2003) conclude that [e]ven if a substantial deterrent effect does exist, the amount of crime rate variation induced by executions may simply be too small to be detected and that [t]here simply does not appear to be enough information in the data on capital punishment to reliably estimate a deterrent effect. Countering these words of caution, several recent studies claim to have compiled robust evidence of the deterrent effect of capital punishment. We begin by updating Katz, Levitt, and Shustorovich (2003) to incorporate data revisions and add data from 1991 to 2000, before turning to these alternative studies. A. Katz, Levitt, and Shustorovich (2003) Katz, Levitt, and Shustorvich generously provided us with their 1950 to 1990 dataset, so we were easily able to replicate their results. These authors regressed state homicide rates on the number of executions per 1000 prisoners (with a rich set of controls), concluding that the execution rate coefficient is extremely sensitive to the choice of specification Panel A of Table 4 shows our replication of their original estimates over the 1950 to 1990 sample using revised data; these estimates are very close to those reported in their paper. 17 Panel B reports results over our updated 1950 to 2000 sample, while Panel C analyzes the largest possible sample, extending back as far as 1934 and forward through to 2000. 17. Note that we report standard errors clustered at the state level, although this makes little practical difference because Katz, Levitt and Shustorovich (2003) reported standard errors clustered at the state-decade level. 17