European Journal of Political Research 37: 103 113, 2000. 2000 Kluwer Academic Publishers. Printed in the Netherlands. 103 Research Note Expert judgements of party policy positions: Uses and limitations in political research IAN BUDGE University of Essex, UK Introduction Estimates of party policy positions obtained from specialists through postal surveys (Castles & Mair 1984; Huber & Inglehart 1995; Laver & Hunt 1992; Ray 1997) have been increasingly used in the last 15 years to provide spatial pictures of party policy and ideology (Kitschelt 1994: 256; Ware 1996: 27 28). Their popularity rests partly on sheer accessibility the mean judgements of specialists about Left Right locations or particular policy positions can be used as reported without tedious data-processing. Expert judgements are also perceived as authoritative if we cannot trust the consensus of party specialists about where their national parties stand, what can we trust? Estimating the policy preferences of collective actors is crucially important to all branches of political science, particularly to studies of democracy and the mandate (Klingemann et al.1994) and to rational choice models which show how the distribution of initial preferences interact with rules and structural constraints to produce given outcomes (Baron 1991; Budge 1994). With expect surveys, have we found an easy, economical and authoritative way to identify party preferences as they change over time (Knutsen 1998), which is also capable of operationalising theory and checking it out (Laver & Shepsle 1996)? Maybe. But some problems remain. These have rarely been systematically discussed because the spatial picture experts give of party systems is both beguiling and in accord with previous institutional classifications (e.g., of party family, cf. Smith 1984: 328 331). Thus the judgements have been taken at face value rather than having their bases critically examined. 1 Major problems relate to: 1. The party whose position is being judged is it the leaders, activists or voters or all three combined?
104 IAN BUDGE 2. The criteria experts base their judgements on - particularly when making a general Left Right classification (Huber & Inglehart 1995: 78). 3. Whether judgements refer to intentions and preferences or overt behaviour an important distinction when most theories use declared or implicit party preferences to explain overt behaviour. 4. What time period are judgements of policy position based on? the whole post-war era up to the survey? The instant at which the survey is administered? The election or inter-election period in which the survey is conducted? Answers to these points very much affect the uses to which expert surveys can be put. After a brief review of the way they have developed to date, this note discusses these problems concluding that they confine expert judgements to a fundamentally descriptive and static role rather than an analytic or dynamic one. From Left Right judgements to policy domains: the development of expert surveys Expert surveys developed out of the judgements made by individual scholars about the policy positions of West European parties, usually on a Left Right continuum These judgements were driven by the desire to explain government coalition formation through the policy closeness of parties which joined the coalition (Axelrod 1970; de Swann 1973; Taylor & Laver 1973). By administering a postal questionnaire asking experts in 16 countries to locate their national parties on a 10-point scale between Left and Right, Castles & Mair (1984) sought to balance out the biases inherent in such individual judgements by using the mean judgement of specialists to locate parties a brilliant innovation which produced results at once authoritative and easily accessible. As a consequence they have been widely used and quoted ever since (Mair & Castles 1997). What was in many ways a follow-up study based on a similar Left Right scale and methodology was carried through by Huber & Inglehart (1995) in the mid-nineties, for 40 countries including Central and Eastern Europe. Their innovation was to ask experts the criteria they based their Left Right judgements on (p. 78). Those nominated by specialists fell into ten categories ranging from economic or class conflict, the dominating one used by over 50 percent, to xenophobia, centralisation and new politics. These criteria are summarised in Table 1. The range of criteria used by experts raises questions about the exact nature not only of the Left Right locations obtained by these authors but also of those produced earlier. If specialists in one country base their judgements
EXPERT JUDGEMENTS OF PARTY POLICY POSITIONS 105 Table 1. The 10 broad types of criteria used by experts to place parties to Left and Right, 1993 1. Economic or class conflict 6. Xenophobia 2. Centralisation of power 7. Conservatism versus Change 3. Authoritarianism versus Democracy 8. Property rights 4. Isolation versus Internationalism 9. Constitutional reform 5. Traditional versus new culture 10. National defence Source: Adapted from Huber & Inglehardt (1995), p. 78. exclusively on party attitudes to democracy while those in another use only attitudes to nationalisation and privatisation of industry, will the resulting Left Right classification in the different countries be comparable? Worse, if half of the specialists in one country use democracy as a criterion for placements and the other half use different criteria, does the resulting mean party location rest on averaging bananas and oranges in policy terms? Huber and Inglehart are very honest in raising this question but do not really resolve it. The third major expert survey (Laver & Hunt 1992: 12) anticipated this difficulty by asking experts to locate parties on each of eight specific policy scales rather than on one overarching Left Right dimension. These are shown in Table 2. Each gives an explicit domain in which to make a judgement of location. The authors reason for getting judgements in each domain separately was to tap the multi-dimensionality of policy space (pp. 5, 45 46), so the survey also gave experts judgements on the salience of each dimension. In at least some national follow-up surveys done in 1997 the overall policy space reduces to a basic Left Right dimension, however (Laver 1998). The latest expert survey also taps into a specific policy domain party attitudes to European integration (Ray 1997). Having sketched the ways expert surveys have developed, we now consider, in turn, the methodological problems raised previously. These derive from ambiguities, possibly inherent in expert judgements, about exactly what is being characterised in policy terms, how and when. What party is being located? Laver & Hunt (1992: 124) asked their experts to place party leaders and voters separately on their policy scale a wise precaution, since what leaders advocate is not necessarily what voters support. Parties are complex organisations with many levels, from Parliaments through apparatchiks to activists. Other
106 IAN BUDGE Table 2. Policy scales used in the Laver Hunt expert survey 1. Taxes versus public services Promote raising taxes to increase public services. (1) Promote cutting public services to cut taxes. (20) 2. Foreign policy Promote development of friendly relations with Soviet Union. (1) Oppose development of friendly relations with Soviet Union. (20) 3. Public ownership Promote maximum public ownership of business and industry. (1) Oppose all public ownership of business and industry. (20) 4. Social policy Promote permissive policies on matters such as abortion and homosexual law. (1) Oppose permissive policies on matters such as abortion and homosexual law. (20) 5. The religious dimension Strongly anticlerical. (1) Strongly proclerical. (20) 6. Urban versus rural interests Promote interests of urban and industrial voters above others. (1) Promote interests of rural and agricultural voters above others. (20) 7. Centralisation of decision making Promote decentralisation of all decision making. (1) Oppose any decentralisation of decision making (20) 8. Environmental policy Support protection of environment, even at the cost of economic growth. (1) Support economic growth, even at the cost of damage to environment. (20) For each policy-domain, respondents were asked to locate the positions of party leaders and party voters, and to rank each in terms of the importance attached by the leaders of each party to it. Source: Laver & Hunt (1995), pp. 124 125. expert surveys have not specified what they meant by the party to be placed, 2 thus leaving in doubt whether it is leaders or other groups, or all of them, which are being characterised. The disadvantages in such ambiguity are illustrated in Ray, Scott & Steenbergen s (1998) attempt to relate party positions on European integration to their supporters opinions on it, and to check which influences the other. Their research design conceives supporters and party positions as separate and independent variables. This is certainly correct at a conceptual level. However, the measure leaves unclear the extent to which judgements of party positions were based
EXPERT JUDGEMENTS OF PARTY POLICY POSITIONS 107 on voters as against leaders, or activists, opinions. Ambiguity about the measure of party location introduces a potential tautology into the empirical research, depending on the (unknown) extent to which party locations are assigned on voters as opposed to leaders preferences. To the extent that party voters preferences are being related to party voters preferences (measured somewhat differently) the correlations are not especially illuminating. How are parties located by experts? Criteria for Left Right placements The terms Left and Right obviously mean something in political terms. But it is not clear if they mean the same thing to different experts. Huber & Inglehart s (1995) finding is that they could mean ten different things! It may be that the same criteria underlie the locations of parties when experts are asked to place them from Left to Right. But they may not. We do not know. This uncertainty should caution against making easy comparisons between different national rankings. It is easy also to conceive of the referents for judgement changing over time even within the same country. Laver, for example, alters his foreign policy scale between 1989 and 1997. From covering attitudes towards the Soviet Union it switches to attitudes towards Nato and the EU! (Laver & Hunt 1992; Laver 1998). Presumably experts making general Left Right judgements also changed their foreign affairs focus similarly over this time period. To the extent that foreign affairs enters into Left Right judgements such change of content would them non-comparable. Knutsen s (1998) comparison of party Left Right positions in 1982 and 1993 assumes quite uncritically that the experts judgements are exactly comparable. Against such a criticism it could be argued that the Left Right dimension is general, ideological and contentless an overarching spatial representation capable of incorporating a variety of salient political issues and conflicts but not specifically bound to any of them. This is possibly an implication which emerges from Mair & Mudde s (1998: 218 219) distinction between party ideology as opposed to specific policy positions. If this were so the problem of different criteria underlying Left Right judgements would go away. One would expect their substantive meaning to change over time and possibly between countries. The research problem this would raise, however, is what referents Left and Right have if they do not relate consistently to certain policies? What point is there in comparing parties in terms of a common scale if they all stand differently on it? Closeness of location would not tell us anything about policy agreements if the
108 IAN BUDGE positions concerned were based on different substantive criteria for different parties, countries and time periods. As against this, most scholars who have applied Left Right distinctions in actual research have imputed a substantive policy content to them. We have cited earlier coalition literature. In terms of economic voting Hibbs (1977) and Anderson (1995) see Left parties as permanently concerned with employment and Right parties with inflation. Laver & Hunt (1992) associate Left and Right with support and opposition to public ownership. Klingemann, Hofferbert & Budge et al. (1994), from extensive analyses of post war party election programmes, find the Left consistently associated with state intervention, welfare and peace and the Right with freedom, tradition and strong defence. If Left Right positions are not policy-empty and do reflect real party preferences, then the varying criteria used by experts to characterise these positions is a serious problem. The ambiguity of resulting Left Right locations would render expert judgements unreliable as a measure of policy intentions. Locating parties in specific policy domains One way of reducing such ambiguity is to ask experts to locate parties in specific policy domains support or oppose public ownership, raise taxes and extend services, or lower taxes and cut services, anti clerical versus clerical, for or against European integration (Laver & Hunt 1992; Ray 1997). Focussing on a specific policy area should aid experts in applying the same criteria to each set of locations. There are, however, two problems with such specific policy judgements: (a) as noted above the dimensions considered relevant will change, rendering over time comparison of overall party positions speculative rather than authoritative (Laver 1998); (b) posing a closed question rather than an open question to experts makes the measure highly dependant on the original investigators judgement about what party differences really are. It is difficult to get clear judgements from experts on separate policy domains without posing hypothetical contrasts which parties actually spend a lot of time avoiding in their own declarations of intent. A prime example is Laver & Hunts (1992) contrast Taxes versus Public Services. Laver, himself, has noted that parties hardly ever advocate tax increases (Laver & Garry 1998; 11) an observation supported by abundant evidence from analyses of party manifestos (Budge, Robertson & Hearl 1987). The same applies to being against religion.
EXPERT JUDGEMENTS OF PARTY POLICY POSITIONS 109 What are we to make of such hypothetical policy contrasts? Of course, experts faced with the question will try to respond somehow (I have myself). Possibly they will look for guidance at whether parties have in fact increased taxes or cut services (or vice versa) in the past, administratively or legislatively. Possibly they will rely on the general reputation Socialist parties have of being profligate. The answer will have little to do with declared intentions, however, since parties notoriously avoid making such declarations. If intentions are what interests investigators, they would probably be better content-analysing documents which give the parties own statements of intent. Do expert judgements reflect intention or behaviour? If unrealistic contrasts between scalar opposites (Table 2) force experts into locating parties on the basis of what they have done, rather than what they have said, this imposes serious limitations on the extent to which expert locations of parties can be used to explain their behaviour non-tautologically. To put the case at its most extreme, characterisations of party behaviour as Left-wing cannot be used to explain their Left-wing behaviour. Stated so sharply this is an obvious tautology which investigators would avoid. Yet, as noted above, earlier analysts of party coalition behaviour probably fell into it through basing their own characterisations of policy locations on party behaviour (Axelrod 1970; De Swaan 1973; Taylor & Laver 1973). Expert judgements have also been applied to theories of coalition behaviour (Laver & Hunt 1992: 90 120). To the (unknown) extent that party closeness has been judged by the experts on the basis of who forms coalitions with who, a similar tautology is involved here. Subtler forms of tautology also crop up deriving from ambiguous judgements made in more extended investigations of government coalition formation. A party may be characterised as centrist because it takes the important ministries in coalition with all sorts of other parties. Its central bargaining position may then be used to explain why it always gets these ministries (Laver & Shepsle 1996). Again, to the extent the party s coalition behaviour enters into the expert judgements there is a possible tautology, though we do not know its extent. This is the trouble with experts we do not generally know what the basis of the judgement is, though we may reasonably fear contamination between intention and behaviour. There is, it is true, a legitimate sense in which judgements of party behaviour for a past period could be used as indicators of initial intentions for the next time-period. The problem is that time periods are usually demarcated by elections at their beginning and end. Past behaviour is thus regularly super-
110 IAN BUDGE seded as a direct indicator of current policy preferences by new declarations of intention, the party manifesto or election programme. As these documents are regularly analysed and quantified they provide indicators of policy tied to a particular date whose referents are clear (Budge et al., forthcoming). What time-period? Nevertheless, expert judgements could be legitimately used in dynamic theories if one were sure that they were based on behaviour before the beginning of the interactions studied. Thus Castles and Mair s Left Right placements from 1982 83 could well be used to explain coalition formation in the late 1980s, if one were prepared to assume that party policy positions did not change much. However, Laver and Hunt s placements from 1989 could not be so used because they are probably based on the coalition governments and polices of that era and thus are not independent of them. This consideration renders the exact time period to which expert judgements refer quite crucial. It is unclear whether most expert surveys have specified the exact time point for which they want locations to be made. But Huber & Inglehart (1995) do explicitly ask for judgements of party position TODAY (block capitals in original). 3 How far, however, can experts be tied down to focusing on a specified time point by the mere question-wording? It all depends whether they are Downsians or ideologists in their approach to party policy making. Downsian experts may well characterise party location on the basis of the last manoeuvre by the party leadership. The party position is coterminous with its last policy package, which could be quite at odds with previous ones (Downs 1957). Ideologist experts on the other hand see the party as what it is rather than what it does (Mair & Mudde 1998: 224). Thus its present policy position is tied to its past policy position, e.g., on unemployment or inflation (Hibbs 1977; Anderson 1995). In assessing party policy positions today its post-war (if not pre-war) history has to be brought to bear. It is thus not clear what time span underlies the judgements of party position made by different specialists. What is clear is that it cannot easily be bounded by a simple change of question wording. The time period relevant to today s parties goes deep into the varying concepts specialists have of parties in the first place. A further complication is that surveys covering 20 40 countries may elicit some judgements during an election campaign, which are thus keyed primarily to party declarations, and in others during an inter-election period based on actual party alliances and the behaviour of Government and Opposition. Laver and Hunt, for example, obtained their first set of British placements
EXPERT JUDGEMENTS OF PARTY POLICY POSITIONS 111 in 1989, mid-way between the elections of 1987 and 1992. A second set were obtained by Laver just after the General Election of 1997 (Laver 1998). How comparable are these judgements? We cannot say and neither can the investigators. The advantage of using election programmes as indicators of intention, on the other hand, is that they are comparable across time and space. Conclusions Expert surveys produce immediately usable locations for parties at a particular point in time which have made them popular for descriptive studies. They produce an instant snapshot of where parties are at one point in time. Unfortunately this snapshot lacks clear definition. The real virtues of expert judgements have encouraged an uncritical acceptance which overlooks their central weakness: We do not really know what they are judgements of. Are they about party leaders or party voters? the present moment, or past history? Above all, do they reflect party intentions or behaviour? If they are based even in part on behaviour they cannot be used to explain behaviour such as the type of party government which forms. Yet this is one of the main reasons why estimating party policy positions has become so important in contemporary political science. If expert judgements do not constitute pure measures of intentions and preferences they lose most of their analytic use. When judgements about policy location were extended from single individuals to the whole community of specialists, the implicit assumption was that these all had the same idea of parties, so that variations in individual judgements occurred within the same set of theoretical parameters. Both theoretically and empirically it seems that this assumption is incorrect. Experts use different criteria, which go back to their conception of what parties are in the first place. Differences on this point can hardly be controlled by elaborating or redesigning questionnaires. Ambiguities about what exactly is conveyed by expert judgements of party location are therefore likely to remain, and this severely limits their research uses. It is reassuring that the originators of the expert survey themselves have been entirely proper in their use of the judgements, limiting themselves in their recent retrospective (Mair & Castles 1997) to cautious comments about party polarisation. Huber & Inglehart (1995) are scrupulous in trying to check the basis of the judgements out. Unfortunately, the operational short-cut offered by the expert technique has misled others into an incautious extension of their use, which runs squarely into the methodological problems discussed here.
112 IAN BUDGE Notes 1. A parallel can be found with the general acceptance of multidimensional spatial representations of parties in the 1970s, based on electors perceptions of party closeness On critical examination these fell out of favour as being both non authoritative and unstable over time (Budge & Farlie 1978). 2. The questionnaire used by Castles & Mair is missing (1997, fn. 3). It is reasonable to assume however that it was not more complex than successor surveys. 3. Ray (1997) also deals explicitly with the time points judgements are solicited for, getting experts to locate parties between 1984 and 1996. This might raise questions about the reliability and independence of the retrospective judgements. But the time points for which judgements are invited are clearly specified. References Anderson, C. (1995). Dynamics of public support for coalition governments, Comparative Political Studies 28: 350 383. Axelrod, R. (1970). Conflict of interest. Chicago: Markham. Budge, I. (1994). A new spatial theory of party competition, British Journal of Political 24: 443 467. Budge, I. (forthcoming), Estimating party policy positions: From ad hoc judgements to theoretically validated standards, Essex Research Papers. Colchester, UK: Department of Government, University of Essex. Budge, I., Robertson, D. & Hearl, D.J., eds. (1947). Ideology strategy and party change. Cambridge: Cambridge University Press. Budge, I. & Farlie, D. (1978). The potentiality of dimensional analyses for explaining voting and party competition, European Journal of Political Research 6(2): 203 231. Budge, I., Klingemann, H-D., Volkens, A. & Bara, J. (forthcoming). Estimating collective policy preferences: Parties, governments and electors, 1945 1996. Castles, F. & Mair, P. (1984). Left Right political scales: Some expert judgements, European Journal of Political Research 12: 73 88. De Swaan, A. (1973). Coalition theories and cabinet formations. Amsterdam: Elsevier. Downs, A. (1957). An economic theory of democracy. NewYork:Harper. Hibbs, D. (1977). Political parties and macroeconomic policy, American Political Science Review 71: 1467 1487. Huber, J. & Inglehart, R. (1995). Expert interpretations of party space and party locations in 42 societies, Party Politics 1: 73 111. Kitschelt, H. (1994). The transformation of European social democracy. Cambridge: Cambridge University Press. Klingemann, H-D., Hofferbert, R.I., Budge, I. et al. (1994). Parties, policies and democracy. Boulder, CO: Westview Press. Knutsen, O. (1998). Change in party policy positions: Comparing expert surveys, West European Politics 21: 63 94. Laver, M. (1998). Party policy in Britain 1997: Results from an expert survey, Political Studies 46: 336 347. Laver, M. & Hunt, W.B. (1992). Policy and party competition. London: Routledge. Laver, M. & Shepsle, K.A. (1996). Making and breaking governments. Cambridge: Cambridge University Press.
EXPERT JUDGEMENTS OF PARTY POLICY POSITIONS 113 Laver, M. & Garry, J. (1998). Estimating policy positions from party manifestos. Paper presented at ECPR Joint Sessions, Warwick. Mair, P. & Castles, F. (1997). Revisiting expert judgements, European Journal of Political Research 31: 150 157. Mair, P. & Mudde, C. (1998). The party family and its study, Annual Review of Political Science 1: 211 229. Ray, L. (1997). Measuring party orientations towards European integration. Chapelhill, NC: Department of Political Science, University of North Carolina. Ray, L., Scott, D.J. & Steenbergen, M.R. (1998). Parties, constituents and the EU: The dynamics of representation and cue-taking. Paper presented to American Political Science Association Meeting, Boston. Smith, G. (1984). Politics in Western Europe. Aldershot. Taylor, M. & Laver, M. (1973). Government coalitions in Western Europe, European Journal of Political Research 1: 205 248. Ware, A. (1996). Political parties and party systems. Oxford, Oxford University Press. Address for correspondence: Professor Ian Budge, Department of Government, University of Essex, Wivenhoe Park, Colchester, CO4 3SQ, United Kingdom Phone: +44 1206 873.333; Fax: +44 1206 873.598; E-mail: budgi@essex.ac.uk