Measuring Human Rights and the Impact of Human Rights Policy

Measuring Human Rights and the Impact of Human Rights Policy Dr. Todd Landman Deputy Director, Human Rights Centre Department of Government University of Essex Wivenhoe Park Colchester, Essex CO4 3SQ United Kingdom Tel/Fax: +44-(0)1206-872129 Email: todd@essex.ac.uk http://www.essex.ac.uk Conference on Human Rights Impact Assessment 19-20 November 2001 Brussels

1 Introduction Human rights scholars, practitioners, and activists use a variety of measures and indicators to describe the advances and setbacks in the promotion and protection of human rights, explain their overall global variation, and find solutions to guarantee their improved protection in the future. The international community has established an ideal standard of human rights protection laid out in a series of international legal instruments to which countries can become signatories, such as the International Covenant on Civil and Political Rights; the International Covenant on Economic, Social, and Cultural Rights; the Convention on the Prevention and Punishment of the Crime of Genocide; etc. By 2000, there were between 95 and 191 countries that are signatories to these various instruments. i Yet, global evidence suggests that 'there are more countries in the world today where fundamental rights and civil liberties are regularly violated than countries where they are effectively protected'. ii This disparity between official proclamation and actual implementation of human rights protection has been a fruitful area for systematic comparative research in political science. And the persistent difference between so-called 'rights in principle' and 'rights in practice' iii has motivated academics, policy makers, NGOs, and other activists to promote public and international policies that bring actual human rights practices closer in line with the expectations laid out in the international human rights regime. Achieving useful, meaningful, internally and externally valid, and time-series measures of human rights protection is thus paramount for the work of the human rights community, and has long been a quest of academic political science. Indeed, in 1945, Fitzgibbon and Johnson devised a standardised measure of democracy, based on the perceptions and rankings of country experts, which was updated every five years and included civil and political rights as part of the ranking criteria. Since then, the academic community has produced a phenomenal number of scales and measures of democracy, democratic institutions, and rights. iv This paper outlines the methodological challenges and problems of measuring rights in an effort to see how such measures may help the human rights community describe the global variation in human rights protection and understand the degree to which human rights policies have an impact on such protection. Objective of Measuring Rights and Impact The underlying assumptions of this conference is that human rights can be measured in a comparable fashion, and that policy intervention from outside powers should make a positive difference in the promotion and protection of human rights within the targeted third country. Figure 1 illustrates in the abstract these assumptions of human rights impact research. The vertical axis shows a measure of human rights protection that ranges from low to high. The horizontal axis depicts time, and the upward rising line shows an increasing protection of human rights over time. The vertical 'interruptions' are possible policy interventions from third parties in the affairs of the state in question. Early intervention at time t 1 may lead to a gradual improvement in rights protection. Intervention at time t 2 just precedes a dramatic improvement in rights protection, while intervention at time t 3 directly follows such a dramatic improvement. In

2 addition, there is fourth possibility: human rights protection would have improved without such interventions in the first place. This counterfactual situation is an important and often neglected idea in human rights impact research. This logic of impact is illustrated with two examples: (1) the Pinochet years in Chile 1973-1990 and (2) European Union human rights policy for applicant states. Human rights policy impact Human rights protection Policy interventions Rights Improvement t 1 Time t 2 t 3 Figure 1: Assumptions of human rights impact research Chile and US Foreign Policy In the years that immediately followed the military coup d'état of 11 September 1973, the military regime engaged in a counter-offensive against civil society and committed gross violations of human rights. By 1976, Chile had passed a series of Constitutional Acts that institutionalised the regime and made General Pinochet the President of the Republic. Alongside these developments, the regime created the National Intelligence Directorate (DINA), which was responsible for internal security and arguably, committed most of the human rights abuses for the remainder of the Pinochet years. Jimmy Carter was elected President of the United States in 1976, and shortly thereafter declared that US foreign policy would be guided by a concern for human rights. Diplomatic pressure and public pronouncements by the administration may have led to the promulgation of the 1980 Chilean Constitution, which gave a further institutional legitimacy to the regime and was accompanied by an easing in repressive tactics. The arrival of the Reagan Administration in the US led to a policy shift away from a concern for human rights, and Chile plunged back into a highly repressive period through the mid-1980s, and then experienced a failed plebiscite for Pinochet in 1988, followed by a rapid transition to democracy. In the event, the Reagan administration claimed that its policy of support for the Pinochet regime led to its eventual return to democracy.

3 In this brief account it is possible to see how both the Carter and the Reagan policy could be viewed as having a positive impact on human rights protection, even though ideologically and in practical terms, both policies were diametrically opposed to one another. An easing of repression followed Carter's human rights policy pronouncements, while Reagan's support for Pinochet was ultimately followed by a democratic transition. Judging the impact of human rights policy in either case, is difficult from this kind of anecdotal evidence, where time -series measures of rights protection and estimates of possible impact of policy can be better approximated with quantitative analysis (see below). EU Enlargement The European Union has an expansionist logic to incorporate old and 'new' states of Europe, a process made more acute with the collapse of the Soviet Union and the consequent proliferation of independent nation states in the region. The EU also has economic and political criteria for membership that includes the promotion and protection of human rights. Any nation state wishing to join the EU is subject both 'push' and 'pull' factors. For example, increased economic benefit from a common market, with its pool of labour and capital offer attractive incentives for nations to join the EU. On the other hand, the EU can insist on certain conditions for membership that make nation states reform their domestic institutional, political, and economic arrangements to promote sustainable economic progress, democracy, and human rights. There are two important considerations that make this relationship between policy intervention and nation state practice problematic. First, it is not clear whether states improve their promotion of human rights before or after entering into negotiations with the EU (t 2 vs. t 3 in Figure 1). There is thus a complex temporal relationship between policy intervention and nation state behaviour. Second, it is not clear whether the proliferation of new countries seeking entry has led the EU to formulate new policies on the promotion and protection of human rights that would not have been formulated in the absence of such new states. After all, the origin of the European Community was to protect vital resources and promote healthy economies within a liberal democratic framework, where the original members were all broadly speaking, liberal democratic capitalist states. Thus, the development of the European Union's political criteria for entry may be seen as a reaction to developments in the wider Europe following the collapse of the Soviet Union. In both of these cases, the promotion of human rights is the dependent variable, or that which is to be explained by the presence or absence of policy intervention from outside. For systematic comparative and time-series analysis on the relationship between policy intervention and human rights protection, both cases demonstrate why measures of rights protection are crucial. It is possible to code rights and use time series interruption 'dummy variables' for the years of policy intervention, where regression techniques will yield co-efficients and significance levels for each of the yearly dummies. v It is also possible to lag the dependent and independent variables to test the temporal ordering of policy and rights protection. But it is to the methodological, ethical, and political problems associated with measuring human rights that the discussion now turns.

4 Methodological, Ethical, and Political Problems with Measurement While the reporting of human rights violations in various parts of the world suggests those areas that may have the most problems, establishing equivalent measures is often problematic for ethical, methodological, and political reasons. Ethically, it can be dehumanising to use statistics to analyse violations of human rights vi and it is difficult to judge the relative weight of one type of violation over another, thereby committing some form of moral relativism. For example, is torture worse than rape or detention without charge worse or better than disappearance? Methodologically, raw numbers of violations are continuous without an upper limit, which can make them intractable for comparative purposes, vii while the level of available information on violations ranges from an ideal of full information to only those violations that are reported by the international press. viii Bollen has argued that there are six levels of information on human rights violations, which are depicted in Figure 2. ix The most ideal level is that of all characteristics, either reported or unreported. The most biased would include only those violations reported in US sources, such as the New York Times Index. Current work in this area seeks to obtain lower levels of information in much greater detail. For example, the Torture Reporting Handbook, published by the Human Rights Centre at the University of Essex is a manual that defines torture, outlines the legal protections against torture, and provides the ways in which testimony and evidence from victims of torture can be collected. x While such increased information at all levels is helpful for systematic human rights research, however, there is still a tradeoff or tension between low levels of information gathering and the ability to make systematic comparative inferences about human rights. xi In order for equivalent measures to 'travel' for comparative research, there will necessarily be some loss of information. IDEAL All characteristics (recorded and unrecorded) Recorded Accessible Locally reported Internationally reported US Reported BIASED Figure 2: Bollen s Levels of human rights information and reporting Finally, politically, international government and non-governmental organisations refuse to rank the countries for fear of recrimination and loss of credibility. Indeed,

5 the UNDP came under strong political criticism for its 1991 Human Development Report, which used a measure of human rights that ranked all UN member states. xii For these reasons, NGOs such as Amnesty International refuse to rank the countries in their Annual Reports. While cognisant of these concerns, political scientists seeking to construct comparable measures of human rights start from the assumption that human rights can be 'more or less' protected in nation states, and that this 'more or less' can be measured in some fashion. Accepting the tentative nature of these measurements, comparative human scholars using statistical methods agree with Strouse and Claude's argument that 'to forswear the use of available, although imperfect, data does not advance scholarship'. xiii To date, such global comparisons tend to concentrate on a narrow conception of human rights that includes more salient violations such as torture, extra-judicial killings, political imprisonment, and disappearances. xiv These categories, considered to comprise life integrity violations, are coded on a standard scale. xv One popular measure is known as the 'political terror scale', which scores a country according to the frequency of these violations, and ranks countries from low protection of rights (i.e. frequent violations) to high protection of rights (no violations). xvi Most studies treat the components of the political terror scale as having equal value, while some argue that violations of these rights are sequentially ordered from least to most egregious. xvii Whatever the case, the components are aggregated into a single score, which the serves as a dependent variable for which a variety of independent variables are specified and tested using advanced statistical techniques. Such a measure of violation to personal integrity rights also could serve as a useful measure of human rights for testing the hypotheses outlined above in the cases of Chile and the EU. Their one drawback, however, is that they are not sensitive enough to capture the variation in rights protection in regions of the world with democratic political institutions. They thus would not be able to distinguish between the human rights practices of Germany vs. the US, but would be able to make a distinction between Germany and Brazil, for example. They thus capture larger variation in human rights protection, but are not able to represent the nuances in similar sets of democratic countries, such as North America and Europe. Finally, there has been a great dispute normatively, empirically, and methodologically about the primacy of certain human rights over others. Whether couched in terms of 'core rights vs. others', or 'generations of rights', or 'blue, red, and green rights', normative debates have suggested that civil and political rights have greater primacy over economic, social, and cultural rights, which in turn are more important than solidarity rights. Empirically, it is has been argued that civil and political rights are easier to protect and require less fiscal capacity of the state than economic and social rights. Moreover, economic and social rights are argued to be programmatic and aspirational rather then pragmatic and enforceable. Methodologically, political science has tended to focus on measuring civil and political rights, which partly reflects a greater ease of standardisation of such rights measures as well as a cultural predisposition towards civil and political rights in the

6 American political science profession. Over the years, two different types of consensus have emerged about the indivisibility of human rights. On the one hand, social and political theories of rights tend to ground them in long historical processes of struggle and therefore see an important distinction between different categories of rights. On the other hand, heads of state, NGOs, and IGOs, make simple declarations at important global conferences (e.g. the Vienna Declaration and Program of Action 1993) that proclaim all rights to be equally important. xviii The normative and empirical problems are outside the scope of this present paper, but the methodological concerns are crucial. Political scientists have long argued that the ideal standards for civil and political rights are laid in such a way as to be more easily quantifiable, since they are 'negative' rights. Negative rights are those that should not be violated. Thus the practice of torture is a violation of a negative right, i.e. individuals have a right not to be tortured. Similar arguments could be made for the right to a fair trial, the right to association and assembly, and the right not to be arbitrarily detained. Economic and social rights, on the other hand, are positive rights, or those rights that ought to be achieved. It is thus a serious challenge for empirical social science research on such rights to come up with standardised measures. For example, individuals have the right to health, but how can the right to health by operationalised? Is it free medical care for he whole population? Private healthcare for those who can afford it and public healthcare for those who cannot? Moreover, the achievement of economic and social rights is a function of comparative welfarism and public policy legislation, making it difficult to set universal standards of achievement. Human rights lawyers have long struggled with the judiciability of economic and social rights, while the Committee on Economic, Social and Cultural Rights can only ever make recommendations for targets in social spending and welfare, which states parties to the International Covenant on Economic, Social, and Cultural Rights are obliged to pursue. Despite these doubts about intractability of measuring economic and social rights, standard measures of development processes can provide proxy measures for such rights. For example, the Human Development Index combines measures for per capita GDP, literacy, infant mortality, and longevity, indicating the general level of human well being within a society. It is still unclear what bottom threshold there is for such indicators, below which a country is seen to have violated certain economic and social rights. Determining such benchmarks is a key area of research and practical work that needs to be carried by academia, with help of NGOs, IGOs, and other institutions. Conclusions This paper has argued that measures of human rights promotion and protection are paramount for public policy making. They offer a comparable set of indicators that serve as important dependent variables, useful for both descriptive accounts and more advanced statistical analysis. Having such measures allows for systematic synchronic and diachronic comparative analysis on human rights policy impact, if policy intervention to improve human rights practices in third countries is conceived in similar fashion to a clinical or surgical intervention. The direct causal relationship

7 between policy intervention and improvement in human rights promotion and protection can only ever be estimated as a 'realised causal effect'. xix Despite the importance of measuring human rights, there are numbers of controversies that remain. First, there is a necessary trade-off between the level of analysis and the depth of information. Second, there is the intractability of measuring certain categories of rights. Third, there is a choice to be made between using continuous measures and abstract scales of human rights protection. Fourth, there remain deep ethical questions surrounding human rights and statistics. Finally, there are broader political considerations concerning the rank ordering of countries based on human rights protection. Monday, 12 November 2001

8 i See T. Landman, 'Measuring the International Human Rights Regime, paper prepared for the 97th Annual Meeting of the American Political Science Association, 30 August - 2 September, 2001, San Francisco, Table 1. For an electronic version of the paper, see www.apsanet.org. ii A. H. Robertson and J. G. Merrills, Human Rights in the World: An Introduction to the Study of the International Protection of Human Rights, 4th edn., (Manchester, University of Manchester Press, 1996), p. 2. iii J. Foweraker and T. Landman, Citizenship Rights and Social Movements: A Comparative and Statistical Analysis, (Oxford: Oxford University Press). iv See R. L. Barsh, 'Measuring Human Rights: Problems of Methodology and Purpose', Human Rights Quarterly, 15 (1): 87-121. v Such 'multiple interrupted time series analysis' (MITS) is summarised in M. Lewis-Beck, 'Interrupted Time-Series', in W. D. Berry and M. Lewis-Beck (eds.) New Tools for Social Scientists, (Beverley Hills CA: Sage Publications, 1986), 209-240. The technique has been applied in such public policy research areas as social security, coal mine safety, and civil rights; the effects of specific laws, such as traffic and gun control measures; the influence of major political shifts like reform or revolution; and the effects of elections on macroeconomic performance. See Foweraker and Landman, op cit. Chapter 8. vi T. B. Claude and R. P. Jabine, Human Rights and Statistics: Getting the Record Straight, (Philadelphia: University of Pennsylvania Press). vii H. Spirer, 'Violations of Human Rights-How Many? American Journal of Economics and Sociology, 49 (1990), 199-204. viii K. A. Bollen, 'Political Rights and Political Liberties in Nations: An Evaluation of Rights Measures, 1950 to 1984,' in Jabine and Claude, op cit., p. 198. ix Bollen Political Rights and Political Liberties in T. Jabine and R. Claude (eds.) op cit., 198. x See C. Giffard, Torture Reporting Handbook: How to document and respond to allegations of torture within the international system for the protection of human rights, (Colchester: Human Rights Centre, 2000); http://www.essex.ac.uk/torturehandbook/. xi For a treatment of this trade-off between levels of abstraction and the scope of countries under comparison, see T. Landman, Issues and Methods in Comparative Politics, (London: Routledge, 2000), 22-24. xii R. L. Barsh, R. L. (1993) 'Measuring Human Rights: Problems of Methodology and Purpose,' Human Rights Quarterly, 15 (1993), 87-121. xiii J. C. Strouse and R. P. Claude, R. P. 'Empirical Comparative Rights Research: Some Preliminary Tests of Development Hypotheses', in Claude R. P. (ed.) (1976) Comparative Human Rights, Baltimore and London: Johns Hopkins University Press, 1976, p. 52. This sentiment has been reiterated recently and more generally by King, Keohane, and Verba, who provide strategies for reducing the presence of systematic error in any research project, while reporting uncertainty in the findings. See G. King, R. O. Keohane, and S. Verba, Designing Social Inquiry: Scientific Inference in Qualitative Research, (Princeton: Princeton University Press, 1994). xiv Strouse and Claude's (1976) pioneering work in this area is a notable exception, which uses the political and civil liberties measure devised by Raymond D. Gastil, later taken over by Freedom House. xv For example, see N. J. Mitchell and J. M. McCormick, 'Economic and Political Explanations of Human Rights Violations, World Politics, 40 (1988), 476-498; S. C. Poe and C. N. Tate, Repression of Human Rights to Personal Integrity in the 1980s: A Global Analysis, American Political Science Review, 88 (1994), 853-72; S. C. Poe, C. N. Tate, and L. C. Keith, 'Repression of the Human Right to Personal Integrity Revisited: A Global Cross-National Study Covering the Years 1976-1993', International Studies Quarterly, 43 (1999), 291-313; S. C. Zanger, 'A Global Analysis of the Effect of Regime Changes on Life Integrity Violations, 1977-1993', Journal of Peace Research, 33 (2000). xvi One version of the political terror scale ranges from 1 to 5, while a more recent coding scheme uses a scale of 0 to 2 for each separate violation. The former scale uses both the US State Department Country Reports and Amnesty International's Annual Reports, while the latter relies exclusively on the Amnesty reports. In either case, a country is awarded a higher score for a lower protection of human rights. See R. D. Gastil, Freedom in the World: Political Rights and Civil Liberties (Westport, CT, Greenwood Press, 1980); Poe and Tate, op cit.; Poe et al., op cit.; M. Gibney and M. Dalton 'The Political Terror Scale', in D.L. Cingranelli (ed) Human Rights and Developing Countries (Greenwich, CT, JAI Press, 1996), pp. 73-84; D. L. Cingranelli and D. L. Richards, 'Measuring the Level, Pattern, and Sequence of Government Respect for Physical Integrity Rights', International Studies Quarterly, 43 (1999), 409-10.

xvii J. M McCormick and N. J. Mitchell, 'Human Rights Violations, Umbrella Concepts, and Empirical Analysis', World Politics, 49 (1997), 510-525; Cingranelli and Richards op cit. xviii For a heated discussion of this point, see M. Freeman, 'Is a Political Science of Human Rights Possible?' Netherlands Quarterly of Human Rights 19 (2): 121-137. xix For a full discussion of realised causal effect, see King, et al., op cit., 75-114. 9