Scientific Credibility, Disagreement, and Error Costs in 17 Biotechnology Policy Subsystemspsj_

Transcription

1 The Policy Studies Journal, Vol. 39, No. 3, 2011 Scientific Credibility, Disagreement, and Error Costs in 17 Biotechnology Policy Subsystemspsj_ Éric Montpetit One of the original objectives of the advocacy coalition framework (ACF) was to shed light on the role of science in policymaking. The ACF depicts subsystem scientists as political actors just like any other. Unfortunately, science has never become a major theme of research within the framework and, as a consequence, its role in policymaking remains under-theorized, leaving ample room for interpretation. This article seeks to explore the validity of three propositions about the role of science in policy. The first two are derived from the ACF: (i) the capacity of scientists to provide credible advice is affected by the harshness of the political debates dividing the policy subsystem; and (ii) agreement among scientists is just as common as among other groupings of policy actors. The third is derived from an error costs argument: (iii) Disagreements among scientists are even more pronounced than disagreements among other policy actors. Using the results of a survey of policy actors in 17 biotechnology subsystems, this article finds support for the first and third propositions. Indeed, scientists participation in political divisions might even be underestimated by the ACF. The article concludes with attempts to clarify the role of scientists within the ACF, including discussions of ambiguity regarding the role of professional forums and of scientists in between-coalition learning within policy subsystems. KEY WORDS: scientific disagreement, scientific consensus, cost of errors in scientific knowledge, political adverseness, use expert knowledge in policymaking Introduction The argument that politics clouds policymakers decisions is commonly made in technical policy domains. Indeed, politics in these domains is frequently pointed out as responsible for inadequate account of sound scientific evidence. Actors putting forward such grievance believe that science enables making better policy decisions than politics and, therefore, they will often support proposals to delegate policy authority to scientists. They assume that science can be made straightforwardly into an instrument of policy decisions, protected against the ways of politics. The argument has held sway in several Organization for Economic Cooperation and Development (OECD) countries where scientists were granted key roles in policy decisions in a large variety of so-considered technical domains, including finance X 2011 Policy Studies Organization Published by Wiley Periodicals, Inc., 350 Main Street, Malden, MA 02148, USA, and 9600 Garsington Road, Oxford, OX4 2DQ.

2 514 Policy Studies Journal, 39:3 (Coleman, 2003), waste management (Bourdeaux, 2008), drug approval (Jasanoff, 1990), and biotechnology (Levidow & Carr, 2007). The argument supporting science as an instrument of policy decisions (the instrumental use model hereafter) is premised on the possibility of science and politics forming two independent worlds. The argument supposes a world of science obeying a logic of accumulation of knowledge that is foreign to politics. In fact, the logic whereby science encourages agreement and consensus is opposed to adverseness and division as found in the world of politics. As Collingridge and Reeve (1986, p. 9) put it, policymakers and scientists frequently consider consensus as the normal state of science, debate and disagreement marking, at best, an inadvertent failure to apply scientific method properly, or, at worst, outright bias and distortion by one of the parties. This article shows empirically that the assumption of science forming a unified world outside politics, an assumption associated with the instrumental use model, is an inaccurate portrayal, at least in the case of biotechnology policy. Thus, this article endeavors to shed a more realistic light on the political role of scientists. It does so by examining the relationship between scientific credibility and the intensity of political conflicts, as well as by assessing the cohesion of scientists beliefs in comparison with that of other policy actors. Clarifying the role of science in policymaking was among the key original objectives of the advocacy coalition framework (ACF). Notably inspired by the work of Carol Weiss (1979) on knowledge utilization, ACF scholars have made an important contribution in associating scientists with the politics of policy subsystems. Nevertheless, the role of science in policymaking in the ACF remains undertheorized, leaving room for conflicting interpretations and confusion. This article contributes to the ACF literature on the role of science in policy by examining the level of adverseness in relation to scientific credibility and by investigating the level of agreement among scientists compared with other subsystem participants. Additionally, the argument is made that the ACF may underestimate the degree of involvement of scientists in political divisions and for the need to incorporate Collingridge and Reeve s (1986) notion of the error cost into the ACF to help clarify the role of science and scientists in policy conflicts. The article s empirical focus is on scientific expertise about the risks and benefits of biotechnology applications in human genetics and in the agri-food domain in 17 subsystems located in Europe and North America. The analysis shows that the credibility of the scientific knowledge informing policy decisions is affected by political divisions. As divisions among actors over biotechnology increase within policy subsystems, the perception of scientific credibility weakens. In other words, the analysis shows that science and politics are interconnected worlds. The analysis further shows that scientists, even those with a natural scientific background, are even less likely than the other actors to agree among themselves on the risks and benefits of biotechnology applications. Concerns for the cost of error in policyrelevant scientific knowledge explain disagreement among scientists. The argument of the article should be of interest beyond the circle of ACF scholars. Not only does it draw from literatures on knowledge utilization, social psychology, and science and technology studies, it has a general relevance for the

3 Montpetit: Biotechnology Policy Subsystems 515 understanding of democracy. While proponents of the instrumental use model view the entanglement of scientific knowledge with politics as an interference of the latter in the formulation of good public policy, the argument that politics stems from concerns for error costs suggest otherwise. Far from interfering, scientists who voice concerns about possible errors, thus fueling debates about the validity of the knowledge informing policy, also contribute to avoiding policy failures. They surely encourage policymakers to think about a large set of potential consequences of policy alternatives before making a choice. Moreover, the argument that science does not provide uncontestable answers to policy puzzles places final responsibility for policy choices where it should be: with politicians who, unlike scientists, can be held accountable for their decisions in democracies. In short, the political dynamics created by concerns for error costs has far more positive normative implications for democracy than popular prejudice about politics would admit. The article is divided into five sections. First, some background is provided on the ACF and the biotechnology policy context. Second, the role of science in policymaking as depicted by the ACF is presented and specific implications of political divisions for the credibility of the science informing policy and for disagreement among scientists are spelled out. Third, the political dynamics triggered by concerns for the error cost and its implication are introduced. Fourth, the method, empirical material, and statistical analyses are presented. A discussion of the inclusion of the error cost within the ACF concludes the article. The Advocacy Coalition Framework, Science, and Politics The Biotechnology Policy Subsystems This article s outlook on policymaking rests on two key insights provided by the ACF (Sabatier & Jenkins-Smith, 1993). First, the framework assumes that most policy-relevant activities occur in policy subsystems, which are geographically and domain bounded. Moreover, subsystems include a variety of actors, from interest group representatives to government officials and nongovernmental scientists. Second, the framework posits that actors within policy subsystems are more or less divided into coalitions seeking the realization of competing beliefs. Concretely, this study covers biotechnology subsystems, in two distinctive domains, surveyed once in 2006 and once in The domains are agri-food biotechnologies (genetically modified organisms [GMOs]) and human genetics (notably stem cells and new reproductive technologies); the geographical areas are Canada, the United States, U.K., and France. Biotechnology actors from Brussels (the European Union [EU]) were also surveyed in 2008, although very few belong to human genetics as the EU is nearly absent from this domain. Therefore, the analysis includes two domains in four countries, whose actors were surveyed twice during an interval of 2 years, plus agri-food biotechnology in Brussels in The analysis thus covers 17 subsystems. Two unusual features of the subsystem comparison deserve explanations. First, the same subsystem surveyed on two different years is counted twice. For example,

4 516 Policy Studies Journal, 39:3 France in the GMO sector in 2006 counts as one subsystem and France in the GMO sector in 2008 counts also as one. The decision to count them twice stems strictly from empirical observation. Although the year difference is relatively small, the subsystems in 2006 are different from the subsystems in They were not aggregated to prevent information loss. Second, the GMO subsystem in Brussels in 2008 belongs to the supranational level, that of the EU, while the other 16 subsystems belong to the national level. Despite potential relative dependence of France and U.K. on the EU, the Brussels subsystem is compared with the other subsystems as if they all enjoyed the same level of independence. The decision to include the Brussels subsystem in the comparison also stems from observation. All European respondents were asked to indicate the proportion of their professional time dealing with EU policy. Eighty-eight percent of U.K. respondents indicated that they spend less than 10 percent of the time working on EU policy. The proportion of respondents giving the same answer in France is 72 percent. Conversely, over 50 percent of Brussels respondents indicated spending a majority of their professional time working on EU policy. Moreover, empirical research shows that member states biotechnology subsystems are largely independent from the EU (Montpetit, 2009). Qualitative research also shows that the 17 subsystems are more or less divided in a simple manner between actors who believe that biotechnology (including human genetics) contributes positively toward humanity s progress and those who believe it deteriorates human condition (Hula, 2005; Jasanoff, 2005; Mulkay, 1997; Paarlberg & Pray, 2007; Toke & Marsh, 2003; Vogel, 2002). The latter group worries about the risks of biotechnology while the former group emphasizes benefits. In other words, the main division within the 17 subsystems is between those who believe that benefits outweigh risk and those who believe that risks outweigh benefits. The former advocate policies that promote biotechnologies and the latter policies that induce precaution in the manner that they are authorized by governments. Beliefs in benefits and beliefs in risks do not have to be mutually exclusive. Actors can recognize the potential benefits of biotechnology for human progress while acknowledging the presence of risks. Likewise, actors can associate some biotechnological applications with benefits and others with risks. Such actors hold intermediate beliefs between enthusiasm for benefits and anxiety about risks. A subsystem comprising a large number of actors holding intermediate beliefs is characterized by low adverseness. A subsystem divided between a coalition made of actors who strongly believe that biotechnology carries large benefits and little risks and a coalition made of actors who believe with equal strength that biotechnology carries large risks and little benefits is characterized by high adverseness. In such a subsystem, political debates should be harsh as trust across coalitions should run low (Leach & Sabatier, 2005). The degree of adverseness within subsystems is, thus, a matter of empirical investigation, although the ACF and socio-psychological studies suggest that, all else being equal, high adverseness is a frequent scenario to encounter (Alhakami & Slovic, 1994; Leach & Sabatier, 2005; Siegrist, Cvetkovich, & Roth, 2000). A method to measure adverseness in biotechnology subsystems is provided below, after the following discussion of the role of science in policymaking.

5 Montpetit: Biotechnology Policy Subsystems 517 The Political Role of Science in the Advocacy Coalition Framework Some of the notions of the ACF could be interpreted as yielding support to the idea that science can serve as an instrument of policymaking. One such notion invests scientists with a special role in between-coalition learning. Coalitions provide context for most policy actors within the ACF, but in subsystems involving natural scientific objects, however, including biotechnology, scientists can also be exposed to a context arising from reputed professional forums. As Sabatier and Zafonte (2001, p ) argue, when the reputation of these forums warrants it, they assure a general consensus on appropriate norms of evidence even among scientists clearly associated with opposing coalitions. Additionally, between-coalition learning has good chances of occurring when these forums comprise relatively neutral scientists, whose role will be to convince coalition experts that professional norms require the alteration of one or more beliefs important to a coalition. Reputed professional forums exist in the domains of biotechnology. They are relatively coordinated and they are reputed enough to transcend several of the subsystems. They have produced scientific norms for biotechnology since the 1980s, notably within the OECD, and these norms were further developed in the 1990s and 2000s within the Codex Alimentarius (a joint initiative of the World Health Organization and of the Food and Agriculture Organization) and in the context of the United Nations Cartagena Protocol on Biosafety (Abels, 2007; Tiberghien, 2009). In the domain of human genetics, the Human Genome Project was among the most reputed professional forums. These forums provide rigorous and scientifically attractive norms of evidence to assess the environmental and health risks associated with biotechnology applications. 1 The emergence of these scientific norms in the domain of biotechnology would have nothing surprising to students of the ACF, who argue that it is easier for scientists to agree about a natural rather than a social object. The study of natural objects would be amenable to quantitative methods and analytic compatibility, which would have the propensity to channel discussion and avoid dialogues of the deaf (Sabatier & Jenkins-Smith, 1999, p. 145; Weible, 2008, p. 626). In contrast, the plurality of methods relied upon in the study of society tends to disperse discussion in several directions and within a diversity of uncoordinated professional forums. Adjustments of beliefs stemming from empirical evidence about society are less common than adjustments of beliefs stemming from restricted sets of evidence about nature. Convergence of beliefs between coalitions, the ACF suggests, may be more difficult to achieve among social science experts than among experts of nature. The role of scientists in between-coalition learning and the distinction between natural and social sciences can be interpreted as evidence that the ACF attributes a role to scientists that is not quite as political as the role attributed to the other actors. The role of scientists would be shaped by the value they accord to rigorous reasoning and empirical evidence over political beliefs and convictions. Parallels might even be drawn between this special role of scientists within the ACF and the assumed capacity of scientists to stay out of politics within the instrumental use model. Nevertheless, already in 1990, Jenkins-Smith (1990, p. 119) concludes his argument

6 518 Policy Studies Journal, 39:3 about the role of researchers in policymaking by emphasizing politics: the primary dynamics of analysis is driven by the level of conflict within the policy subsystem. In other words, the potential for confusion and contradicting understandings of the role of scientists within the ACF is high, possibly owing to the scarcity of research on this theme and the resulting under-theorization. Since its inception, the ACF literature has rejected what Sabatier and Zafonte (2001, p ) refer to as the civics text-book view of the role of science, which involves an instrumental use of scientific knowledge (Sabatier & Zafonte, 2001, p ). That is, the ACF clearly states that scientists tend to be members of advocacy coalitions (Jenkins-Smith & Sabatier, 1993; Sabatier, 1988, p. 139; Sabatier & Weible, 2007, p. 196). Most scientists are primarily motivated by their convictions and beliefs, just as any other actor. They might even be particularly useful to a coalition, as they can affirm that fellow members policy beliefs are supported by research. In fact, members of coalitions will often seek scientific reassurance, but most of the time they inform their policy preferences (the secondary aspects of their belief system) with evidence that avoids upsetting their political convictions or core beliefs. Scientists within their coalition are best positioned to provide evidence in such a manner. This line of reasoning is largely inspired by work in social psychology showing that the credibility of scientists depends less on their capacity to produce advice from rigorous analysis and evidence than on their capacity to formulate advice that resonate positively on actual political convictions. Actors are likely to consider as credible only those scientists who share their beliefs (Kahan, Jenkins-Smith & Braman, 2010; Siegrist et al., 2000). As Sabatier and Zafonte (2001, p ) put it: The ACF explicitly rejects the assumption that most bureaucrats and researchers involved in a policy area will be neutral. Some may well have no strong policy preference, at least initially. But the framework contends that, as conflict between coalitions increases and as the interrelationships among sets of beliefs become clearer over time, initially loose groups with amorphous beliefs will coalesce into increasingly distinct coalitions with coherent belief systems; in the process, most neutral actors, particularly university scientists, will drop out. The ACF thus contends in well-developed subsystems, most agency officials and researchers who are active will be members of specific coalitions in terms of sharing a set of policy core beliefs and acting in concert to some degree. Weible (2008) has most forcefully tried to dispel confusion about the policy role of scientists, arguing that political context matters. He, thus, distinguishes between three types of subsystem contexts: unitary, collaborative, and adversarial (see also Weible, Sabatier, & Pattison, 2010). He argues that the instrumental use of scientific knowledge is most likely in highly centralized unitary subsystems. Those subsystems are dominated by a single coalition, which uses scientists within as a source of advice. This manner of using knowledge, however, is hardly the politics-free instrumental form idealized by proponents of the instrumental use model. In fact, unitary subsystems involve the exclusion of discordant beliefs from subsystems, whether they are held by scientists or not. Collaborative subsystems are less authori-

7 Montpetit: Biotechnology Policy Subsystems 519 tarian, the relative cohesion of beliefs occurring less through exclusion than from learning between coalitions (Weible & Sabatier, 2009). In collaborative subsystems, scientists who display rigorous reasoning, who give priority to evidence over beliefs, are also likely to enjoy sufficient credibility to convince experts from competing coalitions to alter some of their beliefs. Conversely, in adversarial subsystems, coalitions are distant from each other and beliefs are held with more conviction in an exclusionary fashion. Scientists trying to maintain relative neutrality will likely be ignored by coalitions wanting information to support their positions and, thus, are unlikely to be influential in any of the coalitions. These scientists will likely drop out of the policy subsystem. Scientists who choose not to drop out of the policy subsystem will, therefore, prefer associating themselves clearly with a given coalition, although by doing so, they undermine their credibility across coalitions (Leach & Sabatier, 2005). In short, the ACF insists less on the propensity of scientists to isolate themselves from politics and more on the extent to which their policy role is conditioned by subsystems politics. This article is interested in two logical implications of this characterization of the role of science in policymaking, both of them running against the assumptions of the instrumental use model. First, the capacity of scientists to provide credible advice declines as the harshness of the political debates dividing the policy subsystem increases (proposition 1). The instrumental model would instead assume that such a capacity would be high in technical domains that involve a large number of natural scientists who would look after their credibility by putting forward rigorous analysis of hard evidence rather than engaging directly in debates over the beliefs and the political convictions that divide policy subsystems. Second, agreement among scientists is just as common as among other groupings of policy actors (proposition 2). Analytical compatibility and consensual norms of evidence arising from professional forums would not be powerful enough to discourage disagreements over beliefs among scientists in adversarial subsystems. Scientists surely agree more among themselves in collaborative subsystems, but the same is true of all the other policy actors. In short, scientists, just likely any other policy actor, are motivated by beliefs and political convictions, to which they are strongly committed. Introducing the Error Cost One additional possibility is that scientists have a special role in policymaking, but one that further rejects the role advocated by proponents of the instrumental use model. Acknowledging that scientists are embedded within political contexts is also acknowledging that disagreements among them are significant. However, the logical implication does not have to be that disagreements among scientists are just as prevalent as among the other policy actors. In fact, scientists may take the lead in political divisions and conflicts, providing the knowledge that ignite disagreements and keep them vigorous over time. Scientific curiosity, the culture of investigation, and especially aversion to errors are likely to produce dissention among scientists over time (Montpetit, 2008). In comparison, stimuli for internal dissention are not

8 520 Policy Studies Journal, 39:3 nearly as powerful within other groupings of policy actors (industry representatives for instance). Disagreements among scientists, therefore, are possibly even more pronounced and enduring than among the other policy actors. Science and technology studies have made important contributions to understanding scientific disagreements (Collingridge & Reeve, 1986; Guston, 2006; Jasanoff, 2005; Roy, 2001). A key explanation put forward by this literature rests on nuances put forward about the positivist conception of science, including precisions on the limits of the accumulation of knowledge and on the parallels between processes of scientific knowledge production and politics. As Guston (2006, p. 381) argues: Scientific views are thus compelled by many of the same elements as are political opinions. Guston (2006) goes on to explain that scientific consensus frequently results from exclusionary processes. Two processes can be identified from his analysis. First, some scientists can use their authoritative position to claim that their views are widely shared within the broader scientific community. The extent to which these views are shared, however, is most of the time unknown as the scientific community is rarely surveyed. Again, more scientific disagreement than is claimed by scientists in position of authority can be expected as scientific views come about just like political opinion. Second, the paradigmatic manner whereby knowledge is produced encourages scientists to ignore discordant scientific views. Kuhn (1970) has in fact revealed that any given scientific community depends on a contestable paradigm or worldview that most members avoid contesting in normal times. As Guston (2006, p. 383) puts it: By allowing scientists who have settled on a worldview to focus their researches on jointly uncovering the unknown, consensus prevents them from unproductively disputing the known. Incentives to ignore discordant scientific views are even greater among policy actors promoting instrumental use of scientific knowledge. In fact, consensus-building paradigms are convenient for agreeable policymakers who believe that they can rely on science to avoid making politically divisive decisions. Nevertheless, Collingridge and Reeve (1986, pp ) argue that disagreements among scientists are the norm rather than the exception, even if these disagreements are not always aired in public. Error costs associated with the science informing policy encourage this dynamic of disagreement. When policymakers turn to a scientific community for policy advice, the cost of their potential errors suddenly increases. In fact, when science informs policy, scientific errors produce policy failures, which might have dramatic consequences. Therefore, questioning the science behind the advice and expanding the debate to unattended questions become the reasonable thing to do. Scientists outside the advice-providing community are thus solicited with insistence by political opponents to investigate the science behind the advice and to raise new questions. In fact, scientists may even be compelled to investigate further out of their own concern for error costs. In this sense, scientists are not political actors just like any other, they are ahead of political conflicts; their knowledge fuels political divisions. In short, if the exclusionary processes associated with the production of knowledge sometimes provide the illusion of a scientific consensus upon which policymakers can rely, reliance on this knowledge by policymakers increases error cost to

9 Montpetit: Biotechnology Policy Subsystems 521 a level that encourages scientific contestation. Scientific disagreement and eventually political divisions are the most likely outcomes of such a dynamic. The exclusionary processes of scientific consensus and their eventual contestation are particularly apparent in agri-food biotechnology policy subsystems. Early in the 1980s, biotechnology risks in most OECD countries were assessed by a scientific community dominated by molecular biologists adhering to a paradigm more likely to yield observations of benefits rather than risks (Roy, 2001). Initially, this scientific community s assessments of biotechnology were, by and large, uncontested from the inside, but criticisms from the outside were not made public either (Jasanoff, 2005; Montpetit & Rouillard, 2008). It was easy then to mistake the effect of paradigmatic exclusion with the effect of a focus on evidence and analytic rigor that produces truth suited to inform policy. As these assessments began informing the regulation of newly developed biotechnology applications, however, scientists, experts, and other actors came to understand the cost of error. Questions and new scientific investigations followed, working toward the erosion of the apparent scientific consensus (Domingo & Bordonaba, 2011; Montpetit, 2008). Two forms of contestation occurred, one leading to a disagreement inside the original scientific community and the other expanding the debate to questions outside the area of expertise of this community. First, new findings by mainstream scientists (including molecular biologists) came to undermine the expertise supporting the view that biotechnology applications carried small risks and large benefits. A 1999 article in Nature on the risks of Bt corn for the monarch butterfly and the debates triggered by the article came to symbolize disagreement within the mainstream scientific community (Jasanoff, 2005, p. 108). Second, claims on biotechnology expertise exploded. The perspective of molecular biologists was increasingly criticized as too narrow to suffice as the foundation for policy decisions. Scientists in ecological and other biological sciences increasingly argued that linear associations between genes and their respective functions, the paradigmatic view of molecular biology, was insufficient to properly assess the risks arising from modern biotechnology (Roy, 2001). Molecular biologists, they insisted, were not sufficiently accounting for complex interactions between genes and their environment in sensitive ecological and biological systems. Scientific disagreements eventually broadened beyond natural science to include ethicists, sociologists, and rural economists who showed that risks do not only bear on biology, but also on social relations (Levidow & Carr, 2007; Schrefler, 2010). Despite continued claims of a scientific consensus over biotechnology, scientific disagreement had become apparent in the 2000s (Domingo & Bordonaba, 2011), although it was never measured with a survey instrument. Similar scientific disagreements were also observed by advocacy coalition scholars some time ago. These scholars notably observed division between wildlife biologists and engineers in water protection, and between life scientists and physicists in nuclear waste management (Barke & Jenkins-Smith, 1993; Sabatier & Zafonte, 1995). However, these scholars never went as far as suggesting that disagreements among scientists drove subsystems adverseness between different coalitions. If concerns for error costs place scientists ahead of political conflicts, a third proposition is

10 522 Policy Studies Journal, 39:3 warranted: Disagreements among scientists are even more pronounced than disagreements among other policy actors (proposition 3). The proposition may be particularly relevant in natural scientific domains, in which error costs are frequently high, encouraging scientists to always deepen their investigations and to ask new questions. The Biotechnology Actor Survey The Biotechnology Actor Survey is a comparative survey of actors involved in human genetics and agri-food biotechnology policy in the countries mentioned above. Surveyed actors include government officials, representatives of industry and advocacy groups, scientists affiliated with universities and research centers, journalists, and some politicians. Two waves of the survey were administered, one in the summer of 2006 and one in the summer of Surveying the actors of policy subsystems located on two continents represents a serious challenge, as their identity and contact information do not appear on a single listing. To assemble a list of names of potential respondents, newspaper articles, parliamentary hearings, and scholarly work on biotechnology were examined. This examination was followed by a web search for the contact information of the identified individuals and their organizations. Applying the snowball method, the individuals were then asked via to provide additional names and so on until saturation. In 2006, 1,273 potential respondents with valid contact information were identified through this method. The list was updated in 2008 with the same method and comprised 1,927 names. The figure for 2006 is lower partly because respondents from Brussels were solicited in 2008 only. The survey was hosted on a web site accessible only through invitations. Out of the 1,273 potential respondents contacted in 2006, 270 returned usable questionnaires. The figure was 396, out of 1,927, in Therefore, together, the two waves yielded 666 completed questionnaires and response rates around 21 percent. More importantly, in 2006 and 2008, no significant discrepancy appeared between the proportion of respondents belonging to different gender, subsystem, actor category, and country. The lists of potential respondents between the two waves also displayed similar proportions. The only comparative survey of policy subsystems in several countries of which I am aware, a mail survey, has a higher response rate, but fewer respondents (Aerni & Bernauer, 2005). Beliefs in benefits and in risks were measured using agreement with a set of statements. The benefits included in the statements were improvements in crops and in various aspects of public health while the risks were plant resistance, contamination of organic crops, and various moral hazards. The precise statements are in the Appendix. The level of agreement for each statement was reported on a 5-point Likert scale. Beliefs in benefits were measured using the sum of the statements on benefits and beliefs in risks using the sum of the statements on risks. Also relevant for proposition 1 is a survey question asking respondents whether or not they agree that biotechnology policy decisions in their country (or EU for Brussels respondents) are based on credible scientific knowledge. Again, agreement was reported on a 5-point Likert scale. Lastly, propositions 2 and 3 require information on the profile of respondents. This latter information was obtained with a

11 Montpetit: Biotechnology Policy Subsystems 523 question on their involvement in biotechnology policy as an industry representative, as the representative of an advocacy group, as a government official, or as a scientist. The information provided by respondents was verified with a web search. The independence of the research centers of members claiming to be scientists was ascertained, notably by looking at sources of funding and reputation. Some scientists were thus reclassified as industry representatives or representatives of advocacy groups. The web search was also the occasion to obtain information on the disciplinary background of all respondents. Disciplines were regrouped in two categories: natural sciences and social sciences, including law and bioethics. These two categories distinguish between disciplines dealing with natural objects and those dealing with social objects. Adding distinctions within the natural science category between disciplines that adhere to the molecular biology paradigm and those that do not would have been of interest. The variety of the disciplinary backgrounds of respondents, however, is large and the distinction not always straightforward. For instance, some agricultural schools train their students within a genetic engineering perspective, while other schools emphasize biology. In any case, the ACF simply distinguishes between the natural and social objects of scientists. And any further narrowing reduces the number of scientists within each of the categories, perhaps to a point where belief cohesion becomes self-evident in some categories. I therefore keep with the distinction between natural and social sciences for the purpose of this article. The descriptive statistics are presented in Table 1. Scientists believe that biotechnology carries more benefits than risks, but not as much as government officials and industry representatives. Only advocacy group representatives believe that risks are higher than benefits. Respondents with natural and social science backgrounds have similar views on the benefits of biotechnology, although the latter believe in risks slightly more than the former. Unsurprisingly, government officials are most likely to agree that policy decisions are based on credible science and advocacy group representatives least likely. Respondents with a social science background are slightly less likely than those with a natural science background to agree that decisions are based on credible science. Science in North American and European Biotechnology Subsystems Proposition 1 advances that scientists lose credibility as their subsystem s adverseness increases. A measure of adverseness is therefore needed. Adverseness in this article is partially measured by the average distance separating the beliefs of the actors about biotechnology benefit on the one hand and risk in each subsystem on the other. Calculating the average distance per subsystem requires organizing the data set in dyads, that is, in lines of pairs of every possible combination of respondents. Noteworthy, dyads pair respondents who participated in the same survey wave only and a pair of respondents appears only once. 2 The 666 survey respondents thus yielded 114,525 dyads. This organization of the data enabled the calculation of the distance in the beliefs about risk and about benefit of biotechnology between the two respondents of every single dyad. In each dyad, the standard deviation of the

12 524 Policy Studies Journal, 39:3 Table 1. Descriptive Statistics for Beliefs and Scientific Credibility by Actor Category and Disciplinary Background Categories/Variables Beliefs in Benefits Beliefs in Risks Scientific Credibility of Decisions Scientists Obs Mean Std. dev Max Min Government officials Obs Mean Std. dev Max Min Industry representatives Obs Mean Std. dev Max Min Advocacy group representatives Obs Mean Std. dev Max Min Natural science background Obs Mean Std. dev Max Min Social science background Obs Mean Std. dev Max Min Total Obs Mean Std. dev Max Min beliefs measures the distance between the two respondents. 3 The distances for risk and for benefit were then aggregated. To obtain a measure at the subsystem level, a mean distance in beliefs was calculated per subsystem, conditioned on the affiliation of the two respondents of each dyad with the same subsystem. Belief distance alone, however, does not fully capture the level of adverseness. In fact, average distance in a subsystem might be high, because of disagreements on

13 Montpetit: Biotechnology Policy Subsystems 525 Credibility of the science behind policy U.S. GMO 06 France GEN 06 U.K. GEN 08 Canada GMO 06 U.K. GEN 06 Canada GEN 08 Canada GMO 08 France GEN 08 U.K. GMO 06 Canada GEN 06 Brussels GMO 08 France GMO 06 U.S. GEN 08 U.S. GMO 08 France GMO 08 U.K. GMO 08 U.S. GEN Adverseness Coef = ; SE = ; t = 1.2 Figure 1. Regression between Subsystem Adverseness and the Credibility of the Scientific Knowledge Informing Policy. GEN, human genetics; GMO, genetically modified organism; SE, standard error. risks, despite agreement on benefits. Distance encourages adverseness, but adverseness is even deeper when beliefs are held in a mutually exclusive fashion. Compare two actors who are very distant in terms of their perception of the risk involved in biotechnology applications, but who agree on benefit, with two actors who disagree slightly less than the first two on risk, but also disagree on benefit. Alone, a measure of distance could indicate more adverseness within the first pair of actors, while adverseness is higher in the second pair. In fact, actors in the second pair disagree about everything, while actors in the first pair at least share similar perceptions of benefit. To account for the mutual exclusivity of the beliefs of actors within a subsystem, the correlation coefficient between beliefs in risk and beliefs in benefit was calculated. 4 The higher the correlation coefficients, the more the beliefs are held in a mutually exclusive fashion, further indicating adverseness. The correlation coefficient was then added to the average distance in beliefs to obtain an overall measure of adverseness per subsystem. The subsystem with most adverseness, U.S. GEN 06, thus scores 4.84 and the subsystem with least adverseness, Canada GEN 08, scores The mean of the 17 subsystems is 3.7 and the standard deviation is Seventeen subsystems provide too small a number of cases to obtain statistically significant regression results. Even with a low number, however, a regression can still reveal interesting patterns, as Figure 1 indicates. Therefore, an ordinary least squares (OLS) regression was run on the mean perception of the credibility of the science upon which decisions rest for each subsystem. The key independent variable

14 526 Policy Studies Journal, 39:3 in the regression was subsystem adverseness and the regression also included two control variables: power concentration within the subsystem and the presence of an independent regulatory agency. 5 Figure 1 shows the regression diagnostic plot for adverseness and clearly indicates a negative relationship. The coefficients of the control variables were lower and much less significant. 6 In short, an increase of 1 point in adverseness reduces the extent to which the science behind biotechnology policy decisions is perceived as credible by 0.17 point. It is noteworthy that the difference between the subsystem where scientific credibility is highest (U.S. GMO 08) and the subsystem where it is lowest (U.S. GEN 08) is only In other words, a relatively small increase in adverseness reduces significantly the credibility of science prevailing in a subsystem. It is safe to conclude that these observations match proposition 1. Scientific credibility is affected by political divisions. The slope in Figure 1 indicates that the science behind policy decisions is perceived as less credible in the most divided subsystems and more credible in the least divided subsystems. What part do scientists have in adverseness? In other words, do scientists have a role to play in the erosion of their own credibility? While proposition 2 suggests that scientists are just like any other actors in terms of their capacity to agree among themselves, proposition 3 goes further, suggesting that, as key actors in political divisions, scientists disagree more than the other actors. In other words, scientists would bear a responsibility for the erosion of their credibility, owing to their role in fueling adverseness. To test these propositions, I performed a dyad analysis, commonly used in studies of networks (see Mutz, 2006). The organization of the data set in dyads is explained earlier. The data in dyads are used here in regressions, with the main goal of verifying whether the difference in beliefs within dyads of two scientists is lower than the difference in beliefs within dyads of actors with different professional profiles. According to proposition 2, average belief differences should not be any larger within dyads comprising scientists only than in any of the other types of dyads. Conversely, proposition 3 suggests larger average difference within dyads of scientists. Belief difference was calculated in a simple fashion: The scores for beliefs in benefit of the two respondents within dyads were subtracted. The same was done for beliefs in risks. The absolute values of these two differences were then added into an overall measure of belief difference. 7 The mean belief difference is 4.57 and the standard deviation The minimum is 0 and the maximum 16. All the independent variables in the tested models are dummy variables distinguishing dyads of actors with a common trait, homogenous dyads, from those without the common trait, heterogeneous dyads. For example, a dyad comprising two scientists is homogenous and a dyad comprising a scientist and a government official is heterogeneous. In addition to scientists (the key independent variable), the models include dummies identifying homogenous dyads of industry representatives, of advocacy group representatives, of government officials, of respondents with a natural science background, of respondents with a social science background, of respondents who belong to the same subsystem, of male respondents, and of female respondents. Fixed effects for every single respondent when he or she

15 Montpetit: Biotechnology Policy Subsystems 527 Table 2. Differences in Beliefs between Homogeneous and Heterogeneous Dyads of Respondents Variables Model 1: Without Interactions Model 2: Includes Scientists with Natural Sc. Back. Model 3: Includes Scientists with Social Sc. Back. Scientist dyads 0.29*** (0.03) 0.31*** (0.04) 0.28*** (0.04) Industry representative dyads -0.55*** (0.06) -0.55*** (0.06) -0.55*** (0.06) Advocacy representative dyads -2.61*** (0.05) -2.62*** (0.09) -2.61*** (0.09) Government official dyads -0.29*** (0.05) -0.29*** (0.05) -0.29*** (0.05) Social science background dyads (0.05) (0.05) -0.11* (0.06) Natural science background dyads -0.22*** (0.04) -0.21*** (0.04) -0.22*** (0.04) Scientists with natural sc. back. N/A (0.05) N/A Scientists with social sc. back. N/A N/A 0.17* (0.08) Subsystem dyads -0.11*** (0.02) -0.11*** (0.02) -0.12** (0.02) Female dyads (0.21) (0.21) 0.91*** (0.22) Male dyads 0.13 (0.21) 0.17 (0.21) -0.94*** (0.21) Constant 2.86*** (0.21) 2.89*** (0.22) 2.63*** (0.21) N 62,892 62,892 62,892 Adjusted R *p < 0.1; **p < 0.05; ***p < Standard errors in parentheses. These results were obtained with fixed effects controlling for respondents appearing in several dyads. appears in a dyad were also included in the model to ascertain the independence of the observations. 8 The results of the OLS regression are presented in Table 2. Model 1 excludes interactions and thereby allows adequate interpretations of constitutive terms (Brambor, Clark, & Golder, 2006). According to Model 1, dyads of scientists have a 0.29 higher difference in beliefs about risks and benefits than heterogeneous dyads. In fact, scientists belong to the only category of actors characterized by a higher average difference in beliefs than the average heterogeneous dyad. In comparison with heterogeneous dyads, the difference in beliefs among representatives of advocacy groups is 2.61 lower. It is lower by 0.55 among industry representatives and by 0.29 among government officials. Therefore, the results are consistent with proposition 3 and do not match proposition 2. Clearly, scientists disagree more among themselves than any other groupings of actors. Following the ACF, the disagreement among scientists may be related to their training in social and natural science disciplines. In fact, the variable scientists mix scientists with backgrounds in natural and in social sciences. Moreover, model 1 suggests that when respondents, overall, share a background in natural science, belief difference is lower than among heterogeneous dyads. Interestingly, a similar background in social science does not produce the same effect. This result is consistent with the ACF s suggestion that natural and social sciences are associated with distinct capacities to agree, although the framework insists on background differences among scientists only. Therefore, additional regressions were run with interactive variables allowing estimations of whether scientists with a background in natural science (model 2) or in social science (model 3) were more or less likely to have lower belief differences than scientists taken together. Surprisingly, the interactive term in model 2 is not statistically significant, suggesting that even scientists with a common background in natural science disagree. The significance of the

16 528 Policy Studies Journal, 39:3 interactive term in model 3 is low, but the coefficient suggests that disagreement is even higher when social science experts are considered separately. Clearly, the dyad analysis produced results matching proposition 3. 9 One control variable deserves a few comments: the subsystem dummy. The three models indicate that belief difference among dyads of respondents who belong to the same subsystem is smaller than belief difference among dyads of respondents who belong to different subsystems. This result underscores the relevance of the concept of the subsystem. Recall that international professional forums, deemed influential across subsystems, provide scientific standards for the assessment of biotechnology applications. These forums, one might assume, erode the autonomy of subsystems. In fact, the results show that subsystems have dynamics of their own, more independent from each other than it has been suggested recently (Jochim & May, 2010). The result also indicates that subsystems are relatively cohesive in terms of beliefs. All else equal, two respondents who belong to the same subsystem are more likely to be closer in terms of beliefs than two respondents from two distinct subsystems. Therefore, dynamics of polarization and dialogues of the deaf, which have featured prominently in the ACF, do not fully summarize subsystem activities. Dialogues across coalitions also occur, despite the incapacity of scientists to agree among themselves. Given the results presented in this article, scientists might not promote between-coalition learning even in the most collaborative of the subsystems, but between-coalition learning occurs nonetheless. Conclusion The goal of this article was to confront three propositions arising from different conceptions of the role of science in policymaking with the reality of biotechnology policymaking in 17 North American and European subsystems. Two are propositions of the ACF, which closely associate science and politics, and the third proposition arises from concerns for the error cost. The first proposition suggests that scientific credibility is dependent on subsystem adverseness. The second related proposition is that scientists have a similar capacity to agree among themselves than other policy actors. The third proposition is the mirror image of the instrumental use model, not only suggesting that science and politics are interconnected, but that subsystem adverseness is fuelled by particularly important disagreements among scientists. The analysis of the results of a survey of biotechnology policy actors presented in this article yielded results consistent with propositions 1 and 3. The credibility of scientists is seriously undermined as political divisions within subsystems increase. In fact, scientists, including those with a natural science background, are more divided than any other groupings of policy actors. Despite the several revisions of the ACF (Sabatier & Jenkins-Smith, 1999; Sabatier & Weible, 2007), some ambiguity remains about the role of science in policymaking. The ambiguity arises from the notion of the professional forum and from the distinction between the social and the natural sciences. The capacity of neutral scientists within professional forums dealing with natural scientific issues to encourage between-coalition learning might be interpreted as suggesting a discon-