Positions and salience in European Union politics: Estimation and validation of a new dataset

Transcription

1 Article Positions and salience in European Union politics: Estimation and validation of a new dataset European Union Politics 12(2) ! The Author(s) 2011 Reprints and permissions: sagepub.co.uk/journalspermissions.nav DOI: / eup.sagepub.com Tim Veen University of Mannheim, Germany Abstract This article introduces the Positions and Salience in European Union Politics dataset. The dataset comprises positional and salience estimates of more than 250 parties and governments in the European Union (EU). These estimates, which all come with measures of uncertainty, pertain to 10 important EU policy domains as well as a European integration and a left right scale. The dataset exploits statistics from hand-coded European party manifestos provided by the Euromanifestos project and uses simulation to correct stochastic error. The manifestos are scaled using a technique for count data that employs principles from psychophysics. For most European domestic parties and major European Parliament groups, the estimates range from 1979 to 2004, while for member state governments time-series between 1998 and 2007 are available. The dataset may be of use to scholars interested in European integration, Europeanization, compliance research or EU legislative decision-making. Keywords Compliance research, Europeanization, European integration, Longitudinal analysis, Political salience Introduction In European Union (EU) studies, the array of measurement techniques employed to collect information on actors is impressive. Quantitative data have been derived from expert interviews (e.g. Bueno de Mesquita and Stokman, 1994; Thomson et al., 2006), expert surveys (e.g. Farrell et al., 2006; Steenbergen and Marks, 2007; Hooghe et al., 2010), the analysis of voting behaviour (e.g. Hagemann, 2007; Corresponding author: Tim Veen, University of Mannheim, Parkring 47, D Mannheim, Germany tim.veen@uni-mannheim.de

2 268 European Union Politics 12(2) Hix et al., 2007), public opinion surveys (e.g. Thomassen, 2005) or political text (e.g. Klu ver, 2009; Proksch and Slapin, 2010). Indeed, each technique has its distinct benefits and disadvantages (see Marks et al., 2007), and certainly none has the monopoly on the truth. Rather, each technique fits a particular context, and thus the choice for the right measurement instrument fundamentally depends on a combination of a study s research questions and logistical considerations. This important work notwithstanding, there are still considerable shortcomings with regard to the information pertaining to positions and saliences of preferences of political actors in EU studies. As exemplified above, some changes have clearly been made since Gabel et al. (2002, p. 482) argued that the development of European political science has been retarded unnecessarily by the dearth of public and systematically collected data on EU politics. Nevertheless, to advance the field s scientific profile to a level approximating American politics, much remains to be done regarding our efforts in collecting and disseminating data that enables the comprehensive study how actors behave. By introducing a novel dataset that employs state-of-the-art techniques to collect information on numerous EU political actors over a considerable period of time, this article addresses key propositions of the cumulative agenda for EU research set out by Gabel et al. (2002): collecting data, reporting the process and making available the dataset to the scientific community. The present dataset capitalizes on the statistics from the Euromanifestos project (EMP) (Braun et al., 2004), which coded the content of party platforms for European Parliament (EP) elections. Perhaps most salient to many EU scholars, this dataset, called Positions and Salience in European Union Politics, contains a European integration and a left right scale for 251 domestic parties and four EP party groups between 1979 and For EU member state governments, these scales are available for the period between 1998 and 2007, a period chosen to optionally supplement the estimates with information from Council of Ministers voting records but which can be extended easily to cover the whole period of investigation. In addition, the dataset includes positional and salience estimates for all actors above on 10 important policy domains, such as Common Agricultural Policy (CAP), Enlargement or Common Market. These positional estimates have been derived using a novel scaling technique for count data that employs principles from psychophysics (see Lowe et al., 2011). The EMP evolved from the Comparative Manifesto Project (CMP) (Budge et al., 2001; Klingemann et al., 2006). It employs an extended version of the CMP coding scheme, using additional coding categories directly relating to the political system of the EU. Consequently, the critique relating to reliability of the CMP statistics, which has recently gained some additional momentum (see Benoit et al., 2009; Mikhaylov et al., 2010), has equal validity for the EMP data. By correcting for stochastic error in the Euromanifestos using simulation (see Benoit et al., 2009), however, the Positions and Salience in European Union Politics dataset allows to answer many pertinent research questions that require

3 Veen 269 time-series data on political actors in the EU, without researchers having to be afraid that the analysis is fully driven by measurement error. To introduce and vindicate data collection and measurement, particularly for government estimates, the remainder is structured as follows. The next section discusses reliability issues with EMP statistics before turning to the construction of the scales. Then I introduce and test scaling techniques for count data. Turning to the delicate question of how to derive government positions from party manifestos, I finally discuss the estimation of salience from electoral platforms. The section on validating positions and salience provides validity tests for both positional and salience estimates, with subsections devoted to the analysis of party and government estimates. The conclusion formulates a research agenda inspired by the possible prospects of the dataset. Methodology Euromanifestos can be a salient source for data collection on EU politics since the party platforms for EP elections deal predominantly with EU-related issues. This allows for relatively straightforward inferences of actors positions and valence on various domains. Critics may argue, however, that Euromanifestos consist of cheap talk only since little is at stake in EP elections. But given that election manifestos are primarily designed to define party lines within and amongst parties, this criticism has to be qualified. Manifesto research in general yields valid estimates, ideal for longitudinal analysis (Marks et al., 2007). Moreover, party platforms contain not only positional, but also salience information. However, due to the fact that every Euromanifesto has been coded only once, the reliability is certainly not sublime (see e.g. Benoit and Laver, 2006; Benoit et al., 2009). Mikhaylov et al. (2010) find that reliability issues arise predominantly from misclassification due to overlapping or vague boundaries between coding categories. The problem of misclassification into wrong categories, however, can to some extent be eliminated by joining categories into scales representing policy dimensions. This is what I have done below (see Table 1). 1 Despite these issues, many scholars find that manifesto data reflect real party competition quite well (e.g. Klingemann et al., 2006). When assessing their validity through comparison with positions derived from other sources, results are generally promising (Marks et al., 2007). A recent account is provided by Schmitt and van der Eijk (2009), who reduce the manifesto data to six core-meanings by means of a two-step factor analysis and subsequently regress left right party estimates collected by mass surveys from the European Voter database (Thomassen, 2005). When controlling for variations of the meaning of left and right over time and across countries, their model explains 83 percent of the variance. In other words, although there can be error due to variation in coder classification, the aggregate validity of the data of the characterization of a party s manifesto has always been underscored (Schmitt and van der Eijk, 2009, p. 10).

4 270 European Union Politics 12(2) Table 1. Additive policy scales for the actors in EU politics Policy Dimension Left Position Right Position CAP 7032 (Agriculture and Farmers: Negative) 7031 (Agriculture and Farmers: Positive) 407 (Protectionism: Negative) 406 (Protectionism: Positive) Common Market 4085 (Single Market: Negative) 4084 (Single Market: Positive) 4083 (Labour Migration: Negative) 4082 (Labour Migration: Positive) 413* (Nationalization) 401 (Free Enterprise) Centralization 204 (Constitutionalism: Negative) 203 (Constitutionalism: Positive) 301 (Decentralization: Positive) 302 (Decentralization: Negative) 3012 (Transfer of Power to the EC/EU: Positive) 3021 (Transfer of Power to the EC/EU: Negative) 309 (Competences of the European Commission: 308 (Competences of the European Commission: Negative) Positive) 313 (Competences of the ECJ: Negative) 312 (Competences of the ECJ: Positive) EC/EU Enlargement 317 (EC/EU Enlargement: Negative) 316 (EC/EU Enlargement: Positive) Environmental Protection 410 (Productivity) 501 (Environmental Protection) 416 (Anti-Growth Economy) Justice and Home Affairs 2011 (Freedom) 605 (Law and Order) 7053 (Immigrants and Foreigners in the Manifesto Country) Monetary Policy 4087 (European Monetary Union/European Currency: Negative) 3151 (Negative Mentions of the European Central Bank) 414 (Economic Orthodoxy) 6011 (Immigration: Negative) 2012 (Human Rights) 4086 (European Monetary Union/European Currency: Positive) 3141 (Positive Mentions of the European Central Bank)

5 Veen 271 More Competences to the EP 307 (Competences of the European Parliament: Negative) 306 (Competences of the European Parliament: Positive) Research & Development 507 (Education Limitation) 506 (Education Expansion) 411 (Technology and Infrastructure) Welfare and Social Security 503* (Social Justice) 505* (Welfare State Limitation) 504 (Welfare State Expansion) 7061 (NEDG: Women) 7062 (NEDG: Old People) 7063 (NEDG: Young People) European Integration 110 (Europe, European Community/Union: 108 (Europe, European Community/Union: Negative) Positive) 3011 (Transfer of Power to the EC/EU: Negative) 3021 (Transfer of Power to the EC/EU: Positive) 307 (Competences of the EP: Negative) 306 (Competences of the EP: Positive) 309 (Competences of the European Commission: 308 (Competences of the European Commission: Negative) Positive) 311 (Competences of the European Council/ Council of Ministers: Negative) 313 (Competences of the European Court of Justice: Negative) 315* (Competences of Other EC/EU Institutions: Negative) 310 (Competences of the European Council/ Council of Ministers: Positive) 312 (Competences of the European Court of Justice: Positive) 314* (Competences of Other EC/EU Institutions: Positive) 318 (Complexity of the EC/EU Political System ) 2021 (Lack of Democracy in Europe, the EC/EU ) Left Right 103 (Anti-Imperialism) 104 (Military: Positive) 105 (Military: Negative) 2011 (Freedom) [amended] 106 (Peace) 2012 (Human Rights) 107 (Internationalism: Positive) 203 (Constitutionalism: Positive) (continued)

6 272 European Union Politics 12(2) Table 1. Continued Policy Dimension Left Position Right Position 202 (Democracy) 305 (Political Authority) 403 (Market Regulation) 401 (Free Enterprise) 404 (Economic Planning) 402 (Incentives) 406 (Protectionism: Positive) 407 (Protectionism: Negative) 412 (Controlled Economy) 414 (Economic Orthodoxy) 413 (Nationalization) 505 (Welfare State Limitation) 504 (Welfare State Expansion) 601 (National Way of Life: Positive) 506 (Education Expansion) 6021 (Retaining the National Way of Life in Europe, the EC/EU) 701 (Labour Groups: Positive) 603 (Traditional Morality: Positive) 605 (Law and Order) 606 (Social Harmony) Note: * ¼ includes corresponding EMP subcategories not reported here. The Welfare and Social Security scale was taken from Budge et al (2001), albeit in slightly amended format. The Left Right Scale was taken from Laver and Budge (1992). The Environmental Protection was taken from Lowe et al (2011). All other scales were constructed by the author.

7 Veen 273 How is the EMP being exploited to generate point estimates for the Positions and Salience in European Union Politics dataset? First, I construct additive policy scales from the EMP coding categories. In doing so, I identify combinations of categories that I believe to constitute opposing positions on an underlying policy domain, or scale. For instance, the domain Common Market comprises the categories Single Market: Negative and Protectionism: Negative as the left and Single Market: Positive and Protectionism: Positive as the right end of a continuum. An overview of the dataset s policy scales and the corresponding coding categories can be found in Table 1. The second step for deriving governments policy positions from EMP data involves identifying an appropriate scaling technique for turning the categories comprising the policy scales into positions of actors. The original CMP salience or proportional scaling method semi-scales all coding categories into proportions and subtracts the relative emphasis of one side of the scale from the other side. The problem of such relative instead of absolute scaling techniques is their sensitivity to the number of quasi-sentences that are neutral sentences. The larger these are, the more the positions of any party will approximate zero. Thus there is serious risk of arriving at neutral positions even where parties take extreme stances. The second CMP scaling approach is the so-called ratio scaling (Kim and Fording, 2002). It sums and subtracts the number of quasi-sentences of an additive scale s side from the other, respectively. These are then divided by another, yielding a positional score. This approach therefore does not suffer from the proportionality problem. However, there is a different problem with this scaling technique. It cannot yield valid position estimates when the overall number of quasi-sentences available for a scale is low, because it compares counts of quasi-sentences only to the number of quasi-sentences in the opposing category rather than to counts of all quasisentences. As a remedy, Lowe et al. (2011) propose to take the log ratio of the one side of the scale divided by the other. Unlike the previous approaches, where the indices endpoints range from 1 to 1, this scaling approach does not make any assumption about indices endpoints. It can therefore generate positions at any level of extremity. Applied to CMP data and plotted against estimated party positions collected by Benoit and Laver (2006), the logit scale appears to be the only scaling method that has a relatively linear relationship with expert estimates. Also, it does not suffer from skewed extremes or dispersed points in the plot s middle. As the logit scale is basically nothing more than a method to parameterize count data, it can also be applied in the present article. Figure 1 shows a comparison of the three scaling procedures based on this study s European integration scale. The ratio scale clearly suffers from skewed extremes and overestimates high positive positions. While the proportional and the logit scale show a (relatively) normal distribution, the proportional scale is heavily clustered around zero, with a density that exceeds the logit scale by factor 20. The salience scale therefore masks the real distribution of positions. Overall, the logit scale has the best dispersion. When plotting the distributions of

8 274 European Union Politics 12(2) Density Logit Scale Density Proportional Scale Density Ratio Scale Figure 1. Kernel density of positional estimates for the European integration scale. positions derived from each scale against Chapel Hill (CH) expert estimates (Hooghe et al., 2010) for the EP scale as shown in Figure 2, the pattern of dispersion is consistent with the earlier observations. The logit scale seems to be the only measure with an approximately linear relationship with the expert survey. The variation of the positions compared to expert estimates is satisfactory. The scale has no suppressed variation at the midpoint, and there is no censored variation at the scale s endpoints. Moreover, residual analysis suggests that the relationship between CH estimates and the logit scale s scores is linear and homoskedastic. This is not the case for the other scales. Based on these tests, I therefore use logit scaling in estimating actors positions. Finally, a problem inherently related to positions derived from hand-coded manifestos is the absence of uncertainty estimates. Without these, however, it is impossible to distinguish between signal and noise, between measurement error and real differences in positions. To address this problem, Benoit et al. (2009) proceed to reconstruct the stochastic processes that generate manifestos. They do this by simulation, re-creating the stochastic processes that constructed each text, based on assumption that there are many texts to express an intended message. Following these authors, I bootstrapped the analysis of each coded manifesto, based on re-sampling 1500 times from the set of quasi-sentences in each manifesto reported by the EMP. This procedure not only generates standard errors for coding categories, but also for each policy scale.

9 Veen R²= Chapel Hill Logit Scale R²= R²= Chapel Hill Chapel Hill Proportional Scale Ratio Scale Figure 2. Comparison of scaling methods with expert estimates for the EP scale. Computing government estimates Perhaps one of the most delicate tasks in EU research is the collection of government positions and corresponding salience. Both expert interviews and the analysis of voting behaviour have yielded impressive insights from their estimates already, but to test and develop their findings more data are certainly needed. To yield government estimates from party manifestos, the CMP suggests that government position X g is the sum of each coalition party s i out of n policy position p weighted by each coalition party s relative number of seats in Parliament (Kim and Fording, 2001, p. 161) Government Position X g ¼ Xn i¼1 Seats in Parliament i p Coalition s Seats in Parliament. ð1þ For these calculations, I collected information on all elections in the EU between 1998 and 2007 from European Journal of Political Research yearbooks and election notes in Electoral Studies. This included the dates of election, government

10 276 European Union Politics 12(2) formation and the number of seats each party received. Irregular changes in government composition were also considered (for details, also about missing parties, please resort to the article s Web appendix). In the few cases where parties merged while this was not captured in the EMP data, I composed a new manifesto from the founding parties manifestos by aggregating them into a single platform with each manifesto counting equally. In cases where a party s manifesto was not included, for instance because it was formed after the 2004 elections (e.g. UDEAUR and MfI in Italy), I dropped the party from government. Accordingly, government positions consist only of parties with coded manifestos. In cases where a manifesto was missing but the party was coded at a close election, the available estimates served as a proxy for the party s missing programme. Covering government positions between 1998 and 2007, the question arises into what intervals the period of investigation should be organized. As the EP is elected in five-year intervals, there will be only very limited longitudinal variation in estimates when using monthly or semi-annual intervals. Intervals of multiple years, however, can reduce the potential for the analysis of events between 1998 and 2007, of which Eastern enlargement is the most notable. Yearly periods in turn allow for the analysis of Council politics over time with sufficient longitudinal variation in governments positions and salience. When using yearly periods, the next question concerns where to let these start and end. These points in time should ideally be chosen so that changes in government composition in member states have a minimal distorting effect on the analysis. To minimize these effects, an assessment is required during which months electoral activity in the EU member states is minimal. Table 2 provides the relevant Table 2. Parliamentary elections in the EU ( ) Month Frequency Percentage January February March April May June July 0 0 August 0 0 September October November December 0 0 Total Note: Collected from EJPR Political Yearbooks and the Electoral Studies supplement.

11 Veen 277 information. It shows that between 1998 and 2008 no national parliamentary elections were held during either July, August and December. This includes regularly scheduled as well as early elections, such as the Dutch elections in 2002 after the abrupt end of the cabinet Balkenende-1. Most elections appear to be clustered just before or just after July and August. This implies that the overall effect of attributing incorrect positions to a government will be considerably smaller when annual periods are defined as starting in July and August rather than in December. 2 A second consideration for determining the start of the annual periods is that it should fall in a month of low legislative activity in the EU Council. This may also decrease the problem of assigning incorrect positions. To assess during which month legislative activity is the lowest, Table 3 shows the Council s legislative activity as measured by three recent studies, complemented with estimates specifically collected for this study. Table 3 shows that August is the month with the significantly lowest legislative activity by far. Taking into account that during August electoral activity was also zero, this article therefore defines yearly periods as starting in this month. Indeed, this design handles some strong assumptions that might not perfectly reflect the empirical constitution of government behaviour. For instance, the relationship between policy preferences of parties for EP elections and the position of these parties as members of predominantly coalition governments is not necessarily strictly linear. Moreover, since there are only EP elections once in every five years, the assumed stability of position over this period of time might perhaps omit some Table 3. Legislative activity in the EU Council of Ministers Month Hayes-Renshaw Settembri Mattila Own January February March April May June July August September October November December Total Note: Hayes-Renshaw et al. (2006) n ¼ 513; Settembri (2007) n ¼ 376; Mattila (2009), ¼ 1358; Own n ¼ Activity is measured as monthly proportions of legislative acts adopted.

12 278 European Union Politics 12(2) dynamics in national and European politics. A final point relates to the impact of large debates such as the Treaty of Lisbon at the time when the manifestos were drafted. As these debates may have dominated party discourse in this period, manifestos could be biased by the constitutional rather than the legislative agenda of the EU. Deducing the nature of everyday conflict may therefore be partly misleading. However, since we know relatively little about the actual formation and dynamics of governments attitudes towards European policy issues we cannot discard the design a priori. Despite these problematic issues, prior manifesto research has shown that it can yield valid government estimates across time and countries (Kim and Fording, 2002). The validity in EU studies has also been demonstrated recently (e.g. Franchino, 2007). Research based on the Positions and Salience in European Union Politics dataset also produced findings relating to government coalition behaviour and the Council s political space that closely resemble findings from studies basing inferences on expert interviews or roll-call votes (e.g. Veen, 2011a, 2011b). Perhaps it is the binding contract nature of manifestos that accounts for this surprising effect. As party wings have given their consent, parties are committed to comply with its content. Manifestos thus define party lines within and amongst parties, perhaps even amongst government parties (cf. Gabel and Huber, 2000). Indeed, this does not mean the proposed data set is the solution to all of our problems. Other sources are certainly at least of equal importance. Yet the present data can yield substantive additional insights into EU politics. Research on compliance, Europeanization or legislative decision-making may capitalize on this. Estimating salience from electoral platforms The previous sections elaborated how positions in EU politics can be extracted from European election manifestos. In what follows, I introduce a technique for deriving the salience that governments attach to their positions. In party politics, salience can be used to explain how parties compete. Applying this logic to governments in the EU, salience may be used to explain votes-trading between these actors in different institutional contexts (e.g. Council, European Council or co-decision with Parliament). As for positions, I argue that salience can be estimated from election platforms. Ultimately a party has only a finite number of words available to express its political stance. In proportional terms, the total budget available therefore is one. Moreover, it might only take a few words to make a clear positional statement about a policy domain. But to show the voter that the party does not only have a position on that issue, but that it particularly cares about it, requires to devote a larger proportion of the budget to it than to other policy domains. Thus, the logic underlying the measurement of salience presented here is that the bigger the share of the manifesto s total budget being allocated to a particular policy domain, the more salience an actor attaches to it and vice versa. This rationale is similar to the original saliency approach to party competition (see Robertson, 1976). However,

13 Veen 279 when computing the salience S of any policy domain d, the approaches fundamentally differ. Saliency S d ¼ Positive Mentions d þ Negative Mentions d ð2þ N Instead of subtracting positive and negative mentions of d, I sum these and then divide them by the total number of sentences N to yield a proportion. This alternative method is in congruence with the budgetary constraint assumption of salience as elaborated above. Subtracting the dimensions such as proposed by advocates of the saliency approach would yield sub-optimal positional estimates, not salience. As for positions, I employ the simulation of stochastic processes to yield uncertainty estimates for each point estimate. Validating positions and salience According to Krippendorf (2003, p. 313) a measurement instrument is considered valid if it measures what it claims to measure. To validate the scaled and bootstrapped positional estimates, and to allow for better comparison, all scores were standardized on a 100 to +100 scale denotes the most extreme opposition while +100 equals unequivocal support. Accordingly, zero is similar to neutrality or even indifference. Validating party positions The correlative validity of a new measurement technique requires not only a good correlation of the results with established estimates of the trait it aims to measure, but also low or zero correlation with measures of traits the new measure wants to distinguish itself from. These two forms of correlative validity are called convergent validity and discriminant validity, respectively. This section focusses on the convergent validity for the EMP s positional estimates. However, while establishing the validity of saliency estimates, the section on validating salience: parties and governments also shows that the positional estimates have a high discriminant validity. To assess convergent validity, I correlated the party positions with expert judgements compiled by the CH expert survey (Steenbergen and Marks, 2007; Hooghe et al., 2010). The CH project collected most EU member states domestic parties policy positions towards several EU policy fields. From the CH data, I identified five fields that pertain to the same policy dimensions as the policy scales of the Positions and Salience in European Union Politics dataset: CAP, Common Market, EU Integration, Enlargement and increased competences to the EP. Because increasing the n limits the effects outliers have on the correlation coefficients, and thus to enhance the explanatory power of the validity test, I included estimates for both the EU-15 and EU-25. Table 4 illustrates the correlation of these scales with expert evaluations. All but the EP scale show a good convergent validity. Perhaps most encouragingly, the

14 280 European Union Politics 12(2) European Integration scale correlates very highly with the expert judgements. The European Integration scale might therefore confidently be used to assess the integrationist preferences of domestic parties for periods where the CH data lack coverage information. Note moreover that correlating expert judgements against manifesto estimates is unlikely to produce very high correlation coefficients. First, a manifesto mirrors parties ideal positions while an expert assesses parties observed behaviour. Second, while expert surveys are a cheap and straightforward way to generate positional data on party policy positions, the positions are computed as the mean of the experts judgements. Thus data quality fundamentally depends on the knowledge of experts and their level of agreement. To make more sense of correlations, therefore, they should be corrected for attenuation. Correlation coefficients are weakened by measurement error. Correcting for this yields the correlation if the variables would have been measured with perfect reliability. There are reliability estimates for all three CH surveys, with an average standard deviation of roughly 0.14 over time and across issues (see Steenbergen and Marks, 2007; Hooghe et al., 2010). The standard deviations of the dataset s scales range from 0.08 (Parliament) to 0.11 (CAP). When correcting for attenuation, the correlation coefficients therefore substantively increase by about Moreover, while the test variables in Table 4 have not been measured for exactly the same years and considering that parties do change their positions over time (McDonald and Mendes, 2001), the real convergent validity should exceed even more the correlation coefficients reported above. Validating government positions However, although the party positions validity has been established, there is no guarantee that these necessarily translate into valid government estimates. Table 4. Convergent Validity: Validating EMP Positions against the CH Expert Survey CH Expert Survey Policy Field n CAP Common Market Enlargement EU Integration EP CAP ** Common Market ** EU Enlargement ** European Integration ** European Parliament ** Note: ** ¼ Correlation is significant at the 0.01 level (two-tailed). For EP and EU Integration, the estimates of the 1999 and 2004 European Election manifestos were correlated with the 1999 (Steenbergen and Marks, 2007) and 2006 (Hooghe et al., 2010) Chapel Hill surveys. For CAP, Enlargement and Common Market, the estimates of the 1999 European Election Manifestos were correlated with the 2002 (Hooghe et al., 2010) (CAP, Enlargement) and 1999 survey (Common Market), respectively.

15 Veen 281 To verify the aggregate validity of the government estimates, the positions face validity need to be checked as well. In doing so, I conducted two separate facevalidity tests. The first assesses countries internal face validity. This concerns issues such as the extent to which positional changes over time are coherent with changes in government and the impact of external events (e.g. Eastern enlargement). The second series of face-validity tests looks at the validity of inter-government positions. In other words, in any given year and policy domain, does the position of government x with regard to governments y and z accurately reflect literature s expectations? These tests are offered below. The results of the internal validity tests are promising. For instance, in cases where a conservative government succeeded a social-democratic one, positional shifts in the data are in reasonable accordance with the ideological transformation of government. In this scenario, the data consistently report an increasing support for the CAP. Overall, positional shifts due to changes in government ideology are observable across all countries and time. Position Position Common Market EU Enlargement Position Position EU Environmental Protection Policies Limitation of EU Welfare and Social Security Policies Figure 3. Positional changes of German governments with 95% confidence intervals computed ( ).

16 282 European Union Politics 12(2) Moreover, the tests have shown that the data are even capable of reflecting positional changes within governments. An example is for instance the socialdemocratic government of Chancellor Schro der that ruled Germany from 1998 to mid This government gradually shifted from social-democratic to neoliberal policy-making, manifested for instance in its Agenda This shift obviously impacted also upon the EU policy level. As illustrated in Figure 3, the German government s inclination to limit EU welfare and social security policies increases from 1998 to 1999 and then remains fairly constant for three years. It eventually experiences a significant increase in The confidence intervals show that the second increase cannot be attributed to noise in the data. Overall, the four time-series for Germany in Figure 3 demonstrate clearly the internal face validity. In all graphs, the change from social-democratic to a coalition government of Christian Democrats and Social Democrats in 2005 is in accordance with theoretical and empirical expectations. The validity tests of government positions on individual policy domains per year also yield remarkable results. Overall, the proximity of actor positions is in congruence with the EU literatures expectations and evidence. An example is given by the dendrogram in Figure 4. It represents a hierarchical cluster analysis of 15 governments positions towards the CAP in If the cut-off point is set at around 50 on the dissimilarity axis, two distinct clusters of countries become visible. The left cluster is composed of states that are net-receivers of the CAP policy scheme in The cluster at the right comprises the net-contributors to the CAP. Moreover, as both clusters have an approximately unbiased (au) p-value of 0.95, the hypothesis that the cluster does not exist is rejected with a significance level of In other words, both highlighted clusters are not caused by sampling error, but will remain stable if the number of observations is increased. Figure 4. Cluster dendrogram for government positions on the CAP in 1998.

17 Veen 283 Finally, to establish the validity of the ideological scale, I present the left right scale s (RILE) accuracy in computing governments ideological positions in Figure 5 (a graph depicting the governments positions on RILE in the EU-25 can be found in the article s Web appendix). Figure 5 shows the spatial distribution of the EU-15 governments between 1998 and 2003 on the left right policy scale. To illustrate a government s partisan stance, the head of government s EP party affiliation is provided with the country label (cf. Hagemann and Høyland, 2008). It presents a clear division between conservative and social democratic governments. With UMP France, Forza Italia and Haider s Austria populating the right and the Nordic countries placed on the left of the ideological spectrum, the RILE estimates reflect the empirical ideological orientations of these governments during that period. EPP France EPP Italy EPP Austria EPP Luxembourg EPP Netherlands PES Greece EPP Spain PES Italy ELDR Denmark EPP Belgium ELDR Belgium PES Germany PES Austria PES Portugal PES Netherlands PES United Kingdom AEN Ireland IND France PES Finland PES Denmark PES Sweden Left Right Scale Figure 5. Left right position of governments in the EU-15. Note: The error bars indicates 95 percent confidence intervals. Period ranges from 1998 to 2003.

18 284 European Union Politics 12(2) Table 5. Discriminant Validity: Exploratory Factor Analysis One Factor Two Factors (Imposed) Measure Dimension 1 Dimension 1 Dimension 2 Logit Scale Ratio Scale Proportional Scale Salience Eigenvalues Explained Variance (cum.) Note: Extraction Method: PCA; Rotation: Varimax Rotation with Kaiser Normalization. When using Horn s Test of Principal Components with 5,000 iterations and the 95th percentile value for estimating bias, the suggested component is 1 with an adjusted eigenvalue of 2.74 and an estimated bias of Validating salience: Parties and governments Estimating salience according to the rationale outlined above derives a good portion of its validity from being grounded in budgetary theory of salience. But to establish that the resulting salience scores are not just conflated with positional estimates, an exploratory factor analysis is employed here to assess the measures discriminant validity. Table 5 shows the results of a Principal Component Analysis (PCA) with a default eigenvalue cut-off of 1.0. The three scaling methods to extract positions introduced earlier all load extremely well on one dimension, while the salience measure does not at all. If the PCA s original assumptions are adjusted and two dimensions are imposed on the analysis, the salience scores load nearly perfectly on the second dimension, while the positional estimates show low negative loadings. However, while this analysis does not prove that the salience estimates are correct, it demonstrates that the salience scores measure a different dimension than the positional estimates. The question whether the salience scores behave correctly therefore still remains. Salience and positions are theoretically and empirically distinct from each other in this study. A government can have an extreme position but may attach only moderate salience to it. Although this assumption might seem a paradox, Thomson and Stokman (2006, p. 43) show that in Council politics extreme positions are only weak/moderately positively correlated with salience. In the third of all cases, there is even a negative correlation between the extremity of governments positions and the levels of salience they attach to it. As Table 6 shows, this is also found in the relation between governments positions on the 10 policy domains and the respective saliences. A rank-order correlation between the distance of a government s position to the mean and median position on a domain and the level salience demonstrates this. Whilst half of the domains show a modest positive relationship between extremity and salience, there are even negative correlations in the other half. The latter indicates that more extreme actors attach lower salience to these policy domains. In summary, this suggests that the salience measure reflects what other Council studies have measured as salience.

19 Veen 285 Table 6. Rank Order Correlation: Extreme Policy Position and Salience across Governments and Countries Policy Dimension N Mean Median CAP * ** Centralization ** Common Market Enlargement Environment European Integration European Parliament ** ** Monetary Policy Research & Development Welfare and Security ** Note: * ¼ Significant at the 0.05 level (2-tailed) ** ¼ Significant at the 0.01 level (2-tailed). Extremity of position operationalized as the distance to the mean and median position, respectively. Conclusion This article introduced the Positions and Salience in European Union Politics dataset. The dataset constitutes an alternative and fresh source for researchers interested in the political behaviour of actors in the EU. In providing information about many key actors across many policy domains and time, the data will hopefully be a useful source for many scholars interested in European affairs. Moreover, with a dataset available featuring longitudinal information about the party political make-up of EU institutions (Warntjen et al., 2008), this information can, for instance, be used to compute positions of the European Commission from the present data, hence expanding the dataset s applicability even further. Perhaps the most problematic measurement exercise has been the estimation of government positions and salience. Based on aggregated party estimates, neither the assumed linear relationship between party position and government action nor the relatively stability of these positions has been without problems. Nevertheless, and in congruence with empirical research that employs these data, the results are surprisingly encouraging (see e.g. Veen, 2011b). While the Positions and Salience in European Union Politics dataset definitely has not the final say in collecting data on EU governments, I hope that scholars might find it useful and perhaps even build upon the research design laid out in these pages. Acknowledgments I am greatly indebted to Cees van der Eijk, Will Lowe, the editors and three anonymous referees for detailed comments and critique. The dataset and replication material are freely

20 286 European Union Politics 12(2) available at This research was conducted whilst I the author was a PhD candidate at the University of Nottingham, UK. Funding This research has been partially funded by an ESRC and 3 open competition awards (ES/ G015112/1). Notes 1. There are no assessments of the EMP data s reliability. However, a subtle indication for the overall advantage in reliability compared to the CMP may be found in the number of quasi-sentences that could not be allocated to coding categories. Whereas coders were unable to identify proper categories for an average of 3.5 percent of all quasi-sentences coded with the CMP scheme, this number is only 0.4 percent for the EMP. However, further research is needed to quantify the extent to which reliability in the EMP data is affected in detail. 2. Obviously, an election does not equal government formation. It takes time to form a government, time during which the previous government formally remains in office. However, coalition formation does not on average take long in the EU member states. According to the Parliamentary Democracy Data Archive (Mu ller and Strøm, 2000), the average duration of government formation in the EU between 1944 and 1998 was 22 days. It is thus reasonable to conceive of the month of election as on average the month of government formation. 3. To construct a standardized policy scale, I computed the maximum positive and negative position the most extremist actor in the dataset can theoretically take. This actor is Belgium s AGALEV, with the 1999 manifesto composed of 2832 quasi-sentences. One can locate the party s extreme positions at 8.64 and +8.64, respectively. These then constitute the reference estimates to standardize all other actor positions to 100 (dissent) and +100 (consent) policy scales. 4. The correlation between two variables X and Y with correlation r xy, and a known reliability for each variable, r xx and r yy, the correlation between X and Y corrected for attenuation is r x 0 y 0 ¼ p ffiffiffiffiffiffiffiffi rxy r xxr yy. The correlation coefficients after correcting for attenuation for all policy dimensions from Table 4 are the following: CAP: 0.74; Common Market: 0.74; Enlargement: 0.71; European Integration: 0.85; EP: References Benoit K and Laver M (2006) Party Policy in Modern Democracies. London: Routledge. Benoit K, Mikhaylov S and Laver M (2009) Treating words as data with error: Uncertainty in text statements of policy positions. American Journal of Political Science 53: Braun D, Salzwedel M, Stumpf C and Wu st AM (2004) Euromanifesto Documentation. Arbeitspapiere-Mannheimer Zentrum fu r Europa ische Sozialforschung Mannheim. Budge I, Klingemann H-D, Volkens A, Bara J and Tanenbaum E (2001) Mapping Policy Preferences: Estimates for Parties, Electors and Governments Oxford: Oxford University Press. Bueno de Mesquita B and Stokman F (eds) (1994) European Community Decision Making. New Haven, CT: Yale University Press.

21 Veen 287 Farrell D, Hix S, Johnson M and Scully R (2006) EPRG 2000 and 2006 MEP Surveys Dataset Franchino F (2007) The Powers of the Union. Delegation in the EU. Cambridge: Cambridge University Press. Gabel M, Hix S and Schneider G (2002) Who is afraid of cumulative research? Improving data on EU politics. European Union Politics 3: Gabel MJ and Huber JD (2000) Putting parties in their place: Inferring party left-right ideological positions from party manifestos data. American Journal of Political Science 44: Hagemann S (2007) Applying ideal point estimation methods to the Council of Ministers. European Union Politics 8: Hagemann S and Høyland B (2008) Parties in the council? Journal of European Public Policy 15: Hayes-Renshaw F, van Aken W and Wallace H (2006) When and why the EU council votes explicitly. Journal of Common Market Studies 44: Hix S, Noury A and Roland G (2007) Democratic Politics in the European Parliament. Cambridge: Cambridge University Press. Hooghe L, Bakker R, Brigevich A, de Vries C, Edwards E, Marks G, et al (2010) Measurement validity and party positioning: Chapel Hill expert surveys of 2002 and European Journal of Political Research 49: Kim H-M and Fording RC (2001) Extending party estimators to governments and electors. In: Budge I, Klingemann H-D, Volkens A, Bara J and Tanenbaum E (eds) Mapping Policy Preferences: Estimates for Parties, Electors and Governments Oxford: Oxford University Press, Kim H-M and Fording RC (2002) Government partisanship in western democracies, European Journal of Political Research 41: Klingemann HD, Volkens A, Bara J, Budge I and McDonald M (2006) Mapping Policy Preferences II: Estimates for Parties, Electors, and Governments in Central and Eastern Europe, European Union and OECD Oxford: Oxford University Press. Klu ver H (2009) Measuring interest group influence using quantitative text analysis. European Union Politics 10: Krippendorf K (2003) Content Analysis: An Introduction to Its Methodology. Thousand Oaks: Sage Publications. Lowe et al. (2011) Scaling policy positions from coded units of political texts. Legislative Studies Quarterly 36(1): McDonald M and Mendes S (2001) The policy space of party manifestos. Estimating the Policy Positions of Political Actor. London: Routledge, Marks G, Hooghe L, Steenbergen M and Bakker R (2007) Cross-validating data on party positioning on European integration. Electoral Studies 26: Mattila M (2009) Roll call analysis of voting in the EU Council of Ministers after the 2004 enlargement. European Journal of Political Research 48: Mikhaylov S, Laver M and Benoit K (2010) Coder Reliability and Misclassification in the Human Coding of Party Manifestos. Mu ller WC and Strøm K (eds) (2000) Coalition Governments in Western Europe. Oxford: Oxford University Press. Proksch S-O and Slapin JB (2010) Position taking in European Parliament speeches. British Journal of Political Science 40: Robertson D (1976) A Theory of Party Competition. London: Wiley.

22 288 European Union Politics 12(2) Schmitt H and van der Eijk C (2009) On the changing and variable meaning of left and right. Paper presented at the XXI Congress of IPSA. Santiago de Chile, July Settembri P (2007) The surgery succeeded. Has the patient died? The impact of enlargement on the European Union. Jean Monnet Working Paper 04/07: NYU School of Law. Steenbergen M and Marks G (2007) Evaluating expert surveys. European Journal of Political Research 46: Thomassen J (ed.) (2005) The European Voter, Vol. 4. Oxford: Oxford University Press. Thomson R and Stokman FN (2006) Research design: Measuring actors positions, salience and capabilities. In: Thomson R, Stokman FN, Achen CH and Ko nig T (eds) The European Union Decides. Cambridge: Cambridge University Press, Thomson R, Stokman FN, Achen CH and Ko nig T (eds) (2006) The European Union Decides. Cambridge: Cambridge University Press. Veen T (2011a) The Political Economy of Collective Decision-Making - Conflicts and Coalitions in the Council of the European Union. New York: Springer. Veen T (2011b) The dimensionality and nature of conflict in European Union politics: On the characteristics of intergovernmental decision-making. European Union Politics 12(1): Warntjen A, Crombez C and Hix S (2008) The party political make-up of EU institutions. Journal of European Public Policy 15: