Comparing NOMINATE and IDEAL: Points of Difference and Monte Carlo Tests

Transcription

1 Comparing and 555 Royce Carro Rice University Jeffrey B. Lewis James Lo University of Caifornia, Los Angees Keith T. Pooe University of Caifornia, San Diego howard Rosentha New York University Comparing and : Points of Difference and Monte Caro Tests Empirica modes of spatia voting aow us to infer egisators ocations in an abstract poicy or ideoogica space using their ro-ca votes. Over the past 25 years, these modes have provided new insights about the U.S. Congress, and egisative behavior more generay. There are now a number of aternative modes, estimators, and software packages that researchers can use to recover atent issue or ideoogica spaces from voting data. These different toos usuay produce substantivey simiar estimates, but important differences aso arise. We investigated the sources of observed differences between two eading methods, and. Using data from the 1994 to 1997 Supreme Court and the 109th Senate, we determined that whie some observed differences in the estimates produced by each mode stem from fundamenta differences in the modes underying behaviora assumptions, others arise from arbitrary differences in impementation. Our Monte Caro experiments reveaed that neither mode has a cear advantage over the other in the recovery of egisator ocations or ro-ca midpoints in either arge or sma egisatures. Over the past 25 years, the study of Congress has increasingy invoved the anaysis of ro-ca-voting data. Empirica modes of spatia voting, often referred to as idea-point estimators, aow us to infer egisators ocations in an abstract poicy or ideoogica space using the egisators ro-ca votes. These modes have provided new insights about the U.S. Congress in particuar and egisative behavior more generay (see, for exampe, Pooe and Rosentha 1997). Recenty, LEGISLATIVE STUDIES QUARTERLY, XXXIV, 4, November

2 556 Royce Carro et a. idea-point modes have aso been appied to voting in nonegisative venues, such as the United Nations (Voeten 2000), eections (for exampe, Herron and Lewis 2007), and courts (for instance, Martin and Quinn 2002). Researchers can now use a number of aternative modes, estimators, and software packages to recover a atent issue or ideoogica space from voting data. These approaches are often taiored to particuar probems, such as studying voting in sma chambers (Londregan 2000), measuring dynamics (Martin and Quinn 2002), or appying estimators to very arge datasets (Lewis 2001). The proiferation of estimators raises some genera questions. Which (if any) approach is most appropriate in any given research situation? And what eads to observed differences in resuts when the methods are appied to the same data? Despite some casua assertions in the iterature and a good dea of fok wisdom among practitioners, there is itte systematic research regarding the conditions under which the various statistica estimators and the programs that impement them are more or ess appropriate in either reative or absoute terms. In this artice, we attempt to shed some ight on this question. We wi focus on two eading modes: Pooe and Rosentha s (1985) and Cinton, Jackman, and Rivers s (2004). has been the standard in the fied since its deveopment in the eary 1980s. Legisators scores have been used in hundreds of pubished papers in the poitica science and economics iteratures. Deveoped in the ate 1990s, idea is a eading impementation of the Markov chain Monte Caro-based methods of idea-point estimation that have recenty been introduced in the iterature (Bafumi et a. 2005; Martin and Quinn 2002; Quinn 2004). Other we-known idea-point estimators that we do not consider here incude Pooe s optima cassification (2000) and Heckman-Snyder scores (1997). We chose and idea because continues to be the most widey used estimator and idea has many of the features of the more-recent entrants most notaby, the assumption of quadratic spatia utiity that is common to neary every approach other than and MCMC estimation. 1 Athough both idea and can be and often are used to estimate mutidimensiona issue spaces, we focus here on one-dimensiona issue spaces. Mutipe dimensions introduce greater compexity and difficuty in comparing estimates across modes. We eave the comparison of higher-dimensiona spaces for future work. Our decision to refer to these approaches by the names of the software routines that impement them is more than a choice of convenience. and idea are both based on behaviora modes used

3 Comparing and 557 to derive statistica estimators, and there are fundamenta differences in their forma properties. There are aso important differences in how the modes are impemented, and these differences in impementation are, in practice, an important source of differences between the resuts yieded by each approach. We seek to understand not ony the differences that arise from the forma mathematica features of each mode, but aso those that arise from ess-fundamenta sources. In Section 1, we provide a brief history of idea and. In Section 2, we present the important potentia sources of difference between the modes, in terms of both forma features and impementation. In Section 3, we use data from the U.S. Supreme Court and U.S. Senate to compare estimates generated by idea,, and a version of based on Markov chain Monte Caro (MCMC) estimation. Comparing idea and estimates to estimates generated by the MCMC-based version of NOMI- NATE aows us to isoate differences between idea and that arise directy from their different spatia utiity functions rather than from their different estimation techniques. In Section 4, we present the resuts of Monte Caro experiments assessing the practica differences between the two modes under a variety of conditions. We concude with some genera observations in Section A Brief History of and Idea The origina one-dimensiona was deveoped at Carnegie-Meon University from 1982 to 1984, and the mutidimensiona was deveoped at the Purdue Supercomputer Center between 1986 and The initia mutidimensiona program was written in CDC Vector FORTRAN (D- evoved from this program). W- was initiay written by Noan M. McCarty and Keith T. Pooe in 1991 and has essentiay been unchanged, except for very minor bug fixes, since This version has been impemented in R (Pooe et a. 2009; R Deveopment Core Team 2007). Pooe and Rosentha first reported the resuts of Monte Caro anayses in These resuts were for the origina one-dimensiona. Pooe and Rosentha reported the resuts of Monte Caro studies of the two-dimensiona D- program in 1991 and of Monte Caro studies of W- in A of these studies showed that the various versions of accuratey recovered egisator configurations and ro-ca midpoints. In addition to these direct studies of, other studies have found that the egisator coordinates estimated by W- are highy simiar

4 558 Royce Carro et a. to those obtained through the Heckman-Snyder (1997), KYST, 3 optima cassification (Pooe 2000), and quadratic-norma (Pooe 2001) methods. Deveoped by Joshua Cinton, Simon Jackman, and Dougas Rivers at Stanford University in the ate 1990s, idea is a Bayesian quadratic-norma procedure (Cinton, Jackman, and Rivers 2004; Jackman 2001). Idea uses an MCMC agorithm to assign egisator and bi parameters from ro-ca-voting data. The basic framework has been extended to the dynamic idea-point mode of Martin and Quinn (Martin and Quinn 2002). Martin and Quinn have aso provided an aternative computer impementation of the basic idea mode in their MCMCpack software (Martin and Quinn 2009). Idea and extensions buit upon it have been widey used in the discipine. Idea was originay deveoped in the C computer anguage as a stand-aone program but has since been repackaged to be caed directy from the R statistica environment (Jackman 2007; R Deveopment Core Team 2007). Lewis and Pooe (2004), Hagemann (2007), and Cinton, Jackman, and Rivers (2004) have compared idea and other estimators, and uncovered few differences between the estimators when the size of the egisature under study is arge but, more-substantia differences for smaer voting bodies, such as the U.S. Supreme Court (Cinton, Jackman, and Rivers 2004). and idea each have practica advantages over the other. can be run reasonaby quicky on very arge datasets. For exampe, the DW- scores provided by Pooe and Rosentha for the U.S. Congress are derived from over 92 thousand ro cas and 12 thousand egisators (McCarty, Pooe, and Rosentha 1997). The computationa intensity of idea s MCMC agorithm renders that approach impractica for such a arge probem. But idea has a of the advantages of an MCMC estimator, such as the easy cacuation of auxiiary quantities of interest, and it provides measures of estimation uncertainty for a estimated quantities. As computers become faster, however, these sources of practica advantage are waning for both methods. The MCMC agorithm can now be appied to arger and arger datasets. Faster computers aso aow the appication of the bootstrap to, narrowing the gap between idea and with respect to measuring the estimation uncertainty associated with mode parameters and auxiiary quantities of interest (Lewis and Pooe 2004). As practica considerations become ess binding, the choice of procedure becomes more difficut. Often the modes provide very simiar estimates, but which mode shoud we prefer when resuts differ? To answer this question, we must first understand the possibe sources of those differences.

5 Comparing and Sources of Difference between and Idea and idea usuay produce simiar idea-point and bi-parameter estimates, but the estimates do differ at times and, in some cases, the differences can be substantivey important. For exampe, the two methods might identify a different median member of the body, or they might disagree about how far the eftmost member is from next-most extreme member. These discrepancies arise for a number of reasons, from fundamenta differences in the behaviora modes (utiity functions), to differences in identifying restrictions, to differences in estimation technique (MCMC versus maximum ikeihood), and finay to differences that arise from how the modes are impemented in computer code. Understanding where and why and idea differ is centra to making an informed decision about which, if either, estimator is preferabe in a given situation. In this section, we detai these potentia sources of difference. 2.1 Both Idea and Are Random Utiity Modes We begin with one point on which there is no difference between idea and : both are random utiity modes (McFadden 1974) of Eucidean spatia voting (Eneow and Hinich 1984; Hinich and Munger 1994, 1997). In both modes, one assumes that the voter assigns a utiity to each of the two possibiities associated with each ro ca. We wi refer to these aternatives as the bi and the status quo. 4 The utiity associated with each aternative is determined party by the distance between the aternative and the egisator s most preferred position and party by an additive random shock. For each ro ca (bi status quo pair), the egisator chooses the aternative that provides the greater utiity. The systematic spatia utiity difference between the bi and the status quo arising from the ocation of the egisator and the ocations of the two aternatives is, of course, the main object of substantive interest. The parameters reated to this aspect of egisators utiity functions are what and idea are designed to hep us infer. The random shocks ink the systematic spatia utiity differences to probabiities of voting for each aternative. The random utiity shocks in and idea can be, and have been assumed to be, (type I) extreme-vaue distributed (eading to a ogit ink function) or normay distributed (eading to a probit ink function). Arbitrary differences in scae notwithstanding, the choice between these two error processes has itte effect on the estimates. 5 For the purposes of this discussion, we consider idea and to have i.i.d. norma

6 560 Royce Carro et a. utiity shocks and i.i.d. extreme vaue shocks, respectivey. The norma shocks are assumed to have means of 0 and variances that depend, in part, on how the issue space is parameterized, as we wi ater discuss in greater detai. 2.2 The Choice of Utiity Function The most obvious difference in the underying behaviora modes used in idea and invoves the choice of utiity function. Whie both utiity functions are Eucidean, in the sense that a egisator is aways more ikey to choose the aternative coser to her or his idea point than an aternative that is farther away, the shapes of utiity functions differ, eading to different choice probabiities for given bi and status-quo ocations. Over a wide range of bis and status quos, however, the two modes cosey match each other. Indeed, as previousy shown, idea s quadratic utiity function is a first-order power series approximation of s Gaussian utiity function. In idea, egisators preferences are quadratic. That is, if we assume one dimension and et X be the egisator s idea point and B be the ocation of a proposed bi, then U (X, B) = (X B) 2 + e B, where e B is the random utiity shock associated with the bi. As is we known and easy to work out, quadratic utiity impies inear utiity differences. If we et S be the ocation of the status quo, then idea s utiity difference between the bi and status quo can be written as = ( X, S, B) ( S B ) 2( S B) X eb es. If the predicted choice probabiities depend on utiity differences, then the inear form of these differences is particuary convenient. Because we assume the utiity shocks to be norma and we et α 0 = (S 2 B 2 ) and α 1 = (S B), we can write Pr (Vote = B) = F(α 0 + α 1 X), (1) where F is the cumuative norma distribution function with a mean of 0 and a variance of σ 2 > 0. This function is identica to the two-parameter item-response-theory mode (IRT) used in educationa testing, and, indeed, a good dea of idea s estimation and impementation foows from the MCMC IRTs of the eary 1990s (Abert and Chib 1993). This form is amenabe to MCMC estimation, because a mutivariate norma prior over the parameters eads to conjugate posteriors and a simpe Gibbs samping scheme. 6

7 Comparing and 561 In the mode, egisator utiity functions have a Gaussian, or be, shape. Formay, the utiity function is 1 2 U, Nom ( X, B) = β exp( - w( X - B) ) + eb 2 where β and w are positive constants. The utiity difference function admits no usefu simpification and is Pr ( X, S, B) = β exp( - w( X - B) ) - β exp( w( X - S) ) + eb -es. 2 2 Assuming the ogistic ink function, we determine the probabiity of voting for the bi over the status quo is Pr ( Vote = B) = L β exp( w( X B) ) β exp( w( X S) ), (2) 2 2 where L is the ogistic cumuative distribution function. Perhaps not surprisingy, this characterization of the choice probabiities in terms of the ocations of the aternatives and the egisator s idea point offers itte in the way of computationa convenience. Rather, the choice of the Gaussian form is made on theoretica grounds. Figure 1 pots the corresponding and idea spatia utiity functions. 7 The curves are corresponding; the egisator has the same idea point in each case, and we normaized the utiity scaes to yied eves that are maximay simiar for bi ocations in the neighborhood of the egisator s idea point. The key differences manifest in the tais of the potted curves. In the tais, the margina oss in utiity is decreasing in the formuation. In the tais of the idea utiity function, the margina oss in utiity is increasing at an increasing rate. Thus, when using idea, if one hods fixed the distance between the bi and the status quo, then a egisator is increasingy more disposed to support the coser aternative the farther away both the bi and status quo are from the egisator s idea point. On the other hand, in, the utiity function is not gobay concave, and as the bi and status quo move sufficienty far from the egisator s idea point, the utiity difference between the bi and status quo decreases. The vertica hashes at the bottom of each pane in Figure 1 represent 200 randomy seected bi or status-quo ocations from the 109th Senate that we estimated using. 8 Approximatey 90% of these ocations fa in the 1.5 to 1.5 interva. Over that interva, there is itte difference between the and idea utiity functions for a egisator whose idea point is at or cose to 0.

8 562 Royce Carro et a. FIGURE 1 and Utiity Functions Utiity Bi Location Utiity Bi Location Note: Vertica hashes at the bottom of each pane represent 200 randomy seected bi or status quo ocations from the 109th Senate, as estimated by. The dotted ine in the efthand pane shows the quadratic approximation to the utiity curve. Choice-probabiity functions for the two modes are shown in figure 2. These functions are typica, refecting parameter vaues associated with estimates obtained from the contemporary United States Congress. For each pane, the axes are the ocation of the bi and the ocation of the status quo, and the contour ines represent sets of bi status quo pairs that produce the indicated probabiities of supporting the bi. The idea point of the egisator is represented in each pane by a soid back circe. Notice that, in both idea and, the contours of indifference (points where the bi and status quo are supported with equa probabiity) are 45 degree and 45 degree ines that intersect at the egisator s idea point. Aong these ines, the aternative and the status quo are equidistant from the egisator. Consequenty, the egisator experiences zero systematic utiity difference for those bi status quo pairs. Notice that the two choice functions are very simiar for bis and status-quo pairs around the egisator s idea point. The differences arise in the determination of choices that invove bi status quo pairs that are both reativey far from the egisator s idea point. In idea (quadratic preferences), the choice function becomes knifeedged with distance from the egisator s idea point aong the contours of indifference. In, the equiprobabiity contour ines do not converge as the bi status quo pair moves away from the idea point. Indeed, for bi status quo pairs very far from the egisator s most preferred position, the equiprobabiity contours bend back away from one another.

9 Comparing and 563 FIGURE 2 Choice Probabiity Functions for and IdEA Status Quo ocation Status Quo ocation Bi ocation Bi ocation Status Quo ocation Status Quo ocation Bi ocation Bi ocation Note: Each contour ine shows the set of bi status quo pairs that resut in the indicated probabiity of supporting the bi. The eft two panes show choice probabiities associated with IdEA. The right two panes show probabiities associated with. The top two panes set the egisator s idea point to 0. The bottom two panes set the egisator s idea point to 0.9. These contours are based on parameter vaues typicay derived from U.S. congressiona ro-ca data. The ight-coored dots are -estimated bi and status quo (more precisey, yea and nay) ocations from the 109th Senate. The ight-coored dots in each pane of Figure 2 represent estimated bi status quo (yea-nay) pairs from the 109th Senate, as estimated by. Reativey few of these pairs fa in regions where idea and assign substantiay different choice probabiities. This ack of difference hods both for egisators ocated at 0 and for those at 0.9 (-estimated idea points range from 1 to +1). Nevertheess, there is a significant minority of ro cas for which egisators ocated at 0 and 0.9 are predicted to be substantiay ess ikey to support the coser aternative under than

10 564 Royce Carro et a. under idea. We consider the empirica significance of this difference between modes in Section 3.2. The cose correspondence of and idea resuts for aternatives in the neighborhood of the egisator s idea point becomes a the more apparent when we examine a power-series expansion of the utiity function: β w β w U Nom ( X ; B) = β ( X B) + ( 1) ( X B) 2 k = 2 2 k! β w 2» β ( X B ) 2 β 2 k 2k for aternatives (B) in the neighborhood of X. Apart from the arbitrary scaing constants β and w, this approximation is exacty idea s utiity function. Athough and idea use different spatia utiity functions, those functions differ in their predictions about voting behavior mainy for votes invoving bis and status quos far from the egisator s idea point. Thus, we shoud expect differences between idea and due to their differences in spatia utiity functions to manifest in the estimated ocations of members with idea points far from the bi and status quo (usuay these wi be members with extreme idea points) and aso in the estimated ocations of bis and status quos that are farthest from the center of the idea-point distribution. 2.3 Parameterization of the Ro Cas and idea both assign choice probabiities to each vote choice as a function of egisators idea points and the ocations of the aternatives associated with each ro-ca vote. Neither method directy estimates the bi and status-quo ocations. Rather, for each B + S B S ro ca, estimates m = and s =, and idea 2 2 estimates α 0 and α 1 as previousy defined. These are choices of convenience. Parameterizing idea in terms of α 0 and α 1 yieds the probit regression form shown in equation (1). Mutivariate norma priors over α 0 and α 1 create conjugate posteriors convenient for MCMC estimation. s m and s parameters are referred to as the midpoint and spread, respectivey.

11 Comparing and 565 These differences in parameterization are of itte substantive importance, and the transation between the parameterizations is α straightforward: s = 1 α and m = Identifying Restrictions 2 A of the parameters of interest in and idea are atent and, consequenty, have no inherent objective scae. Both modes are ony identified up to a choice of scae. Differences in the estimated parameters that are returned by each mode are driven, in part, by differences in how each mode is identified (how one fixes the scae of the poicy space). For the most part, these differences are arbitrary and can be removed by simpe inear transformations in the same way that one temperature scae is converted to another with no oss of information. Yet these arbitrary choices of scae have subte effects on the uncertainty associated with each mode s estimates. For exampe, as typicay identified, idea s estimated egisator idea points are usuay ess precise for members ocated at the ends of the continuum than for members ocated near the midde (Cinton, Jackman, and Rivers 2004; Jackman 2001). In contrast, often reports greater certainty about the ocations of extremists than centrists (Lewis and Pooe 2004). Athough this variation might appear to indicate a fundamenta difference between the two modes and it might be tempting to concude that is reativey better at ocating extremists and idea is reativey better at ocating moderates, such concusions are incorrect. We wi demonstrate that it is argey differences in the arbitrary choice of identifying restrictions that drive differences in the uncertainty associated with the estimated idea points (and ro-ca parameters) reported by each mode. Because the choice of scae is arbitrary, so too are the consequences that arise from the choice of scae, incuding those consequences reated to how uncertainty is apportioned across parameters. In, one determines the scae by fixing the endpoints of the idea-point continuum. 10 The eftmost egisator is fixed at 1 and the rightmost egisator is fixed at 1. Of course, the poarity of the scae is aso arbitrary. The poarity is fixed by constraining (in the United States context) the ocation of a known ibera (or conservative) to the negative (or positive) vaues. These restrictions aone are sufficient to identify the space uniquey, but paces additiona constraints on the parameters reated to the bi and α 1

12 566 Royce Carro et a. status-quo ocations. The midpoints (m) are constrained to fa in the [ 1,1] interva. Second, the spread parameters (s) are constrained such that at east one of the two ro-ca aternatives fas in the [ 1,1] interva. This constraint impies that min(m+s,m s) 1. The effects of these two additiona constraints can be seen in figure 2. The two sets of ight-coored dots faing aong the 45 degree ines in the top right and bottom eft of each pane refect estimated bi and status-quo ocations for which the midpoint constraint was binding. The set of ight-coored dots forming notches in the upper-eft and ower-right corners of the panes in Figure 2 represent ro cas on which the spread constraint was binding. The constraints typicay bind in cases of perfect or near-unanimous voting. Ro cas with perfect and near-unanimous voting provide itte information about their bi and status-quo ocations. Whie not stricty required for identification, these constraints hep to pin down the ro-ca parameters when the data provide reativey itte information. For this reason, the appication of these constraints has itte effect on the estimates of the egisators idea points. Identification of atent quantities in Bayesian modes such as idea can be achieved in two ways: through the priors, and through constraints simiar to those used in. In eary versions of idea, the anayst estabished identification by assuming that the prior distribution of the idea points was standard norma. Because the data contained no information about the mean and variance of the idea points (that is, the scae), the assumed mean of 0 and unit variance of the prior determined the scae. As Lewis and Pooe (2004) have noted, this identification strategy eads to some overstatement of the estimation uncertainty (posterior variances), because the identification via the priors does not competey pin down the choice of scae. Some uncertainty in the (arbitrary) choice of scae therefore manifests in the posterior distributions of the parameters, causing some overstatement of the actua uncertainty. The prior impies that the set of egisators is drawn from a popuation with a mean of 0 and unit variance; it does not constrain the observed sampe of egisators to have mean zero and unit variance (which woud be sufficient to pin down the scae). As the number of egisators increases, this source of excess uncertainty diminishes. More recenty, Cinton, Jackman, and Rivers (2004) have recommended identification of idea through parameter constraints. In idea, the ocations of any two members can be fixed, even though the identities of the members to be fixed are determined a priori by the anayst (as a resut of the constrained egisators extremity) rather than during the estimation, as in. Identification can aso

13 Comparing and 567 be estabished in idea by constraining the idea-point distribution to have a mean of exacty 0 with unit variance (Z-scoring). Differences in how we estabish the atent issue space ead to differences in how we appy estimation uncertainty to each parameter estimate. Fundamentay, these modes ony identify distances between pairs of egisators up to a choice of unit. The fundamenta uncertainty is associated with those distances. For exampe, when a egisator s ocation is fixed, there is no uncertainty reated to that egisator s ocation. A of the uncertainty associated with the distances between the fixed egisator and a unfixed egisators is refected in the ocations of those other egisators. By fixing the ocation of the most extreme egisators, associates estimation uncertainty with the ocations of ess-extreme members. By restricting the distribution of idea points to have a mean of 0 and a standard deviation of 1, idea spreads the uncertainty more eveny across the members (because no singe member s position is fixed). When one considers, however, that ony distances between members are truy identified, these apparent differences are seen to be ess important. 11 A fina parameter in idea is the variance of the random utiity shocks, σ 2 /2. Because the variance of random shocks cannot be separatey identified from the α 0 and α 1 parameters, we simpy set this variance to 1/2. Setting the variance of each shock to 1/2 means the difference in shocks has a variance of 1, and the cumuative norma distribution function (CDF), F in equation (1), is the standard norma. Whie this normaization is innocuous from the perspective of estimating egisators idea points, it is not innocuous for estimations of bi and status-quo ocations. In particuar, note that by normaizing the variance of the difference in the shocks to be 1, as in standard probit regression, idea estimates not α 0 and α 1 for each ro ca, but α 0 /σ and α 1 /σ, where σ is the true (but unobserved) standard deviation of the difference in the bi and status-quo shocks. The ro-ca midpoint can be identified because it is expressed as a ratio of α 0 /σ and α 1 /σ, canceing out the unidentified σ. The distance between the bi and status quo cannot be uniquey determined in idea, however, and is ony identified up to the arbitrary choice of σ. In, the scaing parameter β caibrates the reative importance of the random shock versus the spatia utiity in the determination of vote choices. The ocations of the bi and status quo can be identified from the data (if one is given a choice of scae), but this identification is weak because it foows entirey from the noninearity of the choice function, equation (2). Thus, in, the midpoints are strongy identified but the spreads are not.

14 568 Royce Carro et a. In both idea and, ocating the bi and the statusquo aternatives requires that we empoy very strong assumptions. In particuar, if the error component is in fact nonhomogenous (if the error variances differ across ro cas), then those differences wi be refected in the bi and aternative ocations. On the other hand, the ro-ca midpoints are more strongy identified by the data, as aready described. The reader shoud regard with suspicion any use of either or idea that turns on the estimated bi or status-quo ocations; auxiiary evidence must be brought to bear by the researcher to justify the very strong assumptions required in this case. Two additiona concerns reated to identification invove unbounded parameter estimates and empirica underidentification. In the absence of prior beiefs, the ocations of the bi and status quo cannot be uniquey identified when a vote is unanimous; the ro-ca parameters are therefore underidentified in that case. Near-unanimous votes are dropped by but can be incuded when one uses idea. 12 Of course, such votes are not informative about idea points in the absence of strong prior beiefs about the ro-ca parameters, and the ro-ca parameters associated with (near-) unanimous ro cas are themseves ony identified by the prior beiefs. In, nearunanimous ro cas often have midpoints ocated at 1 or +1 because of parameter constraints we wi ater describe. Simiar empirica underidentification occurs whenever a vote is orthogona to the spatia dimension. In such a case, concudes that the bi and status quo are identica, whie idea concudes that the bi and status quo are neary identica in expectation. But neither mode can identify where the bi and status quo are coocated aong the poicy dimension. In idea, this ocation wi depend upon the priors. Unbounded parameter estimates typicay arise in the context of what Pooe (2005) and others have referred to as perfect spatia voting ro cas on which every egisator votes for the aternative coser to her or his idea point. 13 When perfect spatia voting occurs in idea, the associated ro-ca parameter α 1 tends to be ±. As Figure 2 shows, the choice function becomes increasingy infected as the ocations of the bi and status quo are moved away from the voter s idea point. By moving the bi and status quo unboundedy far from the egisators whie hoding fixed the set of egisators who are coser to each aternative, we can estimate a egisators to cast their observed votes with a probabiity of 1. In this case, the mode of the posterior distribution of α 1 is ony bounded away from infinity by the prior distribution paced upon it. In Section 3.1, we discuss in more detai the effect of the prior distributions that idea paces on the discrimination parameters.

15 Comparing and 569 Because of the backward-bending choice-probabiity contour ines of the choice-probabiity function shown in Figure 2, perfect ro cas do not ead to unbounded estimates of the bi and status-quo ocations under. Athough not unbounded, the spread parameters (s) coud become quite arge in the presence of perfect voting. It is for these situations that constrains min(m+s,m s) 1. Perfect voting aso creates minor compications for the estimation of the ro-ca midpoints. Under idea, when voting is perfect, there is not a unique most ikey midpoint. Without oss of generaity, suppose a perfect vote such that a egisators to the eft of a given point vote nay and a members to the right of that point vote yea. Under idea, any point between the rightmost nay-voting member and eftmost yea-voting member has the same ikeihood of being the bi s midpoint. Therefore, the ocation of the midpoint is not uniquey determined by the data, and the priors on the bi parameters determine the unique posterior mode (or expectation). Under, the midpoints associated with perfect votes are uniquey identified by the data, because the estimated choice probabiities are not being driven to 1, as they are under idea. Thus, the ikeihood is responsive to the ocation of the midpoint even in the interva between the eftmost yea-voting member and the rightmost nay-voting member. Despite these compications, the midpoints associated with perfect votes in a but the smaest voting bodies are typicay quite precisey estimated by either mode. The procedure incudes an additiona constraint designed to address the possibiity of egisators having perfect ibera or conservative voting records. As we previousy noted, idea attempts to push a perfect-voting egisator far away from a nonperfect-voting egisators. In that case, we determine exacty where the perfect-voting egisator is ocated by referring to the prior beiefs. In, perfect-voting egisators woud not be paced arbitrariy far from their coeagues in the absence of further constraints, but there is often a arge discontinuous change in a egisator s estimated ocation that accompanies perfect voting if no further constraint is imposed. This change has no effect on the rank ordering of egisators, but it can distort estimates of their reative ocations. The probem is especiay serious for sma datasets, because perfect voting is more ikey to occur by chance in smaer egisatures. To avoid pacing perfect-voting egisators far from their nonperfect-voting neighbors, constrains the distance between those egisators ocated at 1 and 1 (the eftmost and rightmost positions) and their nearest neighbors not ocated at 1 or 1 to

16 570 Royce Carro et a. be no more than 0.1 units (or 5% of the 1 to 1 scae). 14 In some cases, particuary when the number of egisators is sma, this constraint can bind in the absence of perfect voting. In very sma egisatures (fewer than 20 members), the constraint is not appied at a (see Section 3.1). 2.5 Impementation Detaied descriptions of how idea and are impemented have been provided by Jackman (2001; 2007) and Pooe (2005), so we wi not rehash those discussions here. Rather, we wi focus on a few saient points of difference between the modes and on what effects, if any, those differences can be expected to produce. As we have aready discussed, idea is impemented using MCMC, which has many advantages for assessing mode uncertainty and for cacuating auxiiary quantities of interest aong with their associated uncertainties (standard errors). In genera, determining the convergence criteria for MCMC estimators is something of an art, but the idea estimator seems to mix rapidy. The stochastic nature of the optimization entais that the exact estimates wi differ sighty from run to run, but itte of the difference between idea and can be attributed to idea s MCMC estimation. As a Bayesian estimator, idea requires that priors be paced on each mode parameter. By defaut, these priors are quite diffuse and have itte effect on the estimated quantities beyond those aready described. Stronger priors coud, however, induce important differences between the two modes. For the foowing simuations, we empoyed the defaut diffuse (uninformative) priors. uses an iterative constrained maximum-ikeihood agorithm. Starting vaues for the egisator ocations are produced via anaysis of a egisator-by-egisator matrix of disagreement scores (the fraction of times a given pair of members disagreed across ro cas upon which both members of the pair voted). Conditiona on these starting vaues for the egisator ocations and provisiona vaues for the scaing constants w and β, a ikeihood formed from the choiceprobabiity function shown in equation (2) is maximized over the roca parameters. That same ikeihood is then maximized over w and β, with the egisator and ro-ca parameters hed fixed. Finay, new egisator parameter estimates are obtained, conditiona on the current vaues of the ro-ca parameters and β and w. This cyce repeats unti a convergence criterion is met. This convergence criterion is a hodover from a time of ess-powerfu computing and is not particuary strict, but there is itte evidence that faiing to iterate unti fu convergence entais any important effects on the estimates produced by.

17 Comparing and Two Exampes: Voting in the Supreme Court and the 109th Senate We compared estimates obtained from idea and for data from the 1994 to 1997 U.S. Supreme Court (Jackman 2007) and from the 109th Senate (Lewis and Pooe 2008). With ony nine members and 213 nonunanimous decisions, the Supreme Court provides a dataset about as sma as any dataset to which these estimators are ikey to be appied. The U.S. Senate dataset has 102 members (incuding the president) and 520 nonunanimous votes; it is more typica of the data to which idea and are usuay appied. In addition to appying the and idea estimators, we appied a version of that is estimated via MCMC and uses the same identifying restrictions as idea (Carro et a. 2009) and Pooe s Quadratic Norma () mode (Pooe 2001). 15 As the name suggests, the mode assumes quadratic utiity. It appies parameter constraints simiar to those imposed by W- and is a (constrained) maximum-ikeihood estimator. These additiona estimators enabe us to better isoate differences between and idea estimates that arise from their distinct spatia utiity functions and those that arise from how the modes identify restrictions or are impemented. 3.1 United States Supreme Court, In recent years, idea-point modes have been increasingy appied to nonegisative voting bodies and, in particuar, to the United States Supreme Court (for exampe, Martin and Quinn 2002). Smaer voting bodies, such as egisative committees, have aso been considered in the recent iterature (for exampe, Baiey 2001, Londregan 2000, and Peress 2009). We compared and idea estimates of the decision midpoints (ms) and justices idea points (Xs) obtained when each method is appied to the 213 nonunanimous Supreme Court decisions made between 1994 and Figure 3 aows comparison of the estimated idea points. The panes above the main diagona pot pairs of idea-point estimates against each other. For the purposes of comparison, we normaized the dimensions recovered by each of the three methods, so the idea points range from 1 to +1. Note that a four methods produce quite simiar estimates. The idea and estimates correate at 0.99, idea and the MCMC version of correate at 0.99, and the two versions of correate at over A three methods identify Justice John Stevens as the eftmost justice and Justice

18 572 Royce Carro et a. FIGURE 3 Estimated Supreme Court Justice Locations, (MCMC) (MCMC) (MCMC) (MCMC) (MCMC) (MCMC) (MCMC) Note: Each pane pots justice ocations or associated standard errors as estimated by two of four estimators. Panes above the main diagona show point estimates and 95% intervas. The hashes in the margins show the ocations of estimated cut-points for each of the given estimation methods. The panes beow the main diagona pot the standard errors associated with each method against those from each other method. Carence Thomas as the rightmost justice, and a methods agree on the rank order of the remaining seven justices. Differences in the estimated standard errors across methods argey arise from differences in identifying restrictions. This reationship becomes cearer when we consider the 109th Senate. For now, ooking at the Supreme Court data, we see that estimates impy near certainty that Stevens and Thomas anchor the space (are the most

19 Comparing and 573 extreme egisators); thus associates no uncertainty with these justices ocations (because the extremes are fixed by construction). For idea and (MCMC), there is substantia uncertainty associated with the ocations of the extreme members. Interestingy, the (MCMC) estimated ocations of the extreme members are consideraby ess certain than the idea-estimated ocations. Indeed, the idea estimates generate smaer standard errors than (MCMC) overa. We need to investigate this discrepancy further, but we specuate that idea s ower standard errors may arise from the nonconcave tais of the utiity function. Figure 4 aows us to compare the estimated decision midpoints of the three methods. The decisions are spit into two types. The first type, represented in the upper row of panes, comprises those 142 decisions with an underying spatia dimension sufficienty predictive that some meaningfu information about the ocation of the midpoint coud be recovered. The ower panes show the estimated decision midpoints for those 71 decisions without a sufficienty predictive underying dimension to recover meaningfu information about the midpoint s ocation. For exampe, the underying dimension is not predictive when the five most centrist justices vote in opposition to the remaining four justices. In such cases, we woud infer the bi and status-quo ocations to be quite cose together. Whenever the bi and status-quo ocations are very cose together and the distance between them is hed fixed, the choice probabiities are ony weaky reated to where the bi and status quo are jointy ocated. Thus, the ocation of the midpoint is empiricay underidentified (or weaky identified). 16 For these 213 decisions, the estimated midpoints correate at 0.71 for idea and, at 0.83 for and (MCMC), and at 0.79 for (MCMC) and idea. If we consider ony the 142 decisions for which the midpoints can be reasonaby we pinned down, then those correations increase to 0.96, 0.99, and 0.97, respectivey. Thus, there is substantia agreement between the various methods for those votes on which we have any sizeabe amount of information as to the ocation of the ro-ca midpoint. Of course, with, at most, nine members voting on each decision, the uncertainty associated with the estimated midpoints is substantia. As one woud expect and as our study of voting in the 109th Senate wi demonstrate, we can estimate the midpoints with substantiay more precision when the number of votes on each ro ca is increased tenfod.

20 574 Royce Carro et a. 3.2 The 109th Senate The data from the 109th Senate is more typica of the ro-ca voting matrices to which and idea are usuay appied. Incuding the announced positions taken by President George W. Bush as votes, there were 102 voters in the 109th Senate and 520 votes on which at east 3 members voted on the osing side (nonunanimous or non near-unanimous Note: Each pane pots estimated decision midpoints and 95% confidence intervas as estimated by two of four estimators. Panes above the main diagona show estimates for the 142 we- identified decision midpoints. The panes beow the main diagona pot estimates for the 72 weaky identified decision midpoints. Confidence intervas for -estimated midpoints are not avaiabe. FIGURE 4 Estimated Supreme Court Decision Midpoints, (MCMC) (MCMC) (MCMC) (MCMC) (MCMC) (MCMC) (MCMC)

21 575 Comparing and votes). Figure 5 faciitates comparison of the estimated senator ocations associated with idea,,, and (MCMC). Because of the arger number of voters and ro cas in the 109th Senate, these comparisons are ceaner than those shown for the U.S. Supreme Court in the previous section. As with the Court resuts, a methods produced simiar idea-point estimates. Correations among estimates generated by the various estimators range from to Note: Each pane pots senator ocations or associated standard errors as estimated by two of four estimators. Panes above the main diagona show point estimates and 95% confidence intervas. The hashes in the margins show the ocations of estimated cut-points for each of the given estimation methods. The panes beow the main diagona pot the standard errors associated with each method against those from each other method. FIGURE 5 Estimated Senator Locations, 109th Congress (MCMC) (MCMC) (MCMC) (MCMC) (MCMC) (MCMC) (MCMC)

22 576 Royce Carro et a. Surprisingy, the argest correation among estimates is between the and estimators. The smaest correation is between idea and. and idea use the same choice function and differ ony by idea s use of prior distribution and s ro-ca parameter constraints. We therefore expected and idea estimates to be very neary perfecty correated. The number of voters is reativey arger in the Senate than in the Supreme Court data, so the effect of the ro-ca parameter priors shoud be minima in this context. One possibe expanation for the high correation is that and use the same procedure for generating idea-point starting vaues, and the defaut settings prevent either estimator from iterating to fu convergence. The reativey high correation between and and the reativey ow correation between and idea may refect insufficient optimization of the and estimates. By defaut, and each take a sma and fixed number of optimization steps rather than iterating unti meeting a convergence criterion. To check the possibiity that ack of convergence expains the pattern of correations, we refit the and modes, iterating them to conventiona fu convergence. 17 We found some evidence that ack of convergence may indeed infuence the reationships. The fuy converged estimates correate with idea estimates at The sight change in the estimates when more fuy converged is aso refected in the smaer correation of between the defaut estimates and the fuy converged estimates. On the other hand, the defaut estimates are correated with the fuy converged estimates at Thus, whie it may be advisabe to iterate the mode beyond the defaut setting, additiona iterations of the mode (at east, in this case) had ess of an effect on the recovered senator ocations. The variation in the reported uncertainty of the estimates across methods is more obvious in the case of the Senate than in the case of the Supreme Court. The horizonta and vertica ines in the top row of panes in Figure 5 revea that the greatest uncertainty in the idea and (MCMC) estimates is associated with senators whose idea points fa in the tais of the distribution. oads the estimation uncertainty argey onto the ocations of ess-extreme members (particuary, in this case, on those positioned eft of center). The ower row of panes in Figure 5 reveas that estimation uncertainty, whie consistenty greater for NOMI- NATE (MCMC) than for idea, is apportioned reativey simiary across members by the two methods., on the other hand, apportions the uncertainty in the estimated idea points in a markedy different way. These differences and simiarities are not fundamenta, however, and foow directy from the choice of identifying restrictions.

23 Comparing and 577 Note, in the first row of panes in Figure 5, the point ocated near the midde of the coud representing Repubican senators on the top and right of each pane. This point, associated with an unusuay arge confidence interva, represents President George W. Bush, who ony took a position on 81 of the 520 ro-ca votes considered. The effects of other constraints imposed by can be seen in Figure 5, as we. The hash marks in the margins of each pane in the upper row show the estimated ocations of the 520 ro-ca midpoints. Notice that the midpoints a fa within the 1 to 1 interva (or, more precisey, the range of the egisator idea points). Figure 6 presents the estimated midpoints across methods. As we did in our Supreme Court anaysis, we separated out the ro cas for which the midpoint coud be ocated with any certainty from ro cas for which the midpoint coud not be pinned down. For those ro cas with reasonaby estimabe midpoints, the reative ocations of the midpoints are very highy correated across methods. The one notabe outier in the comparison of egisator idea points across methods is Senator Russ Feingod (D-WI). Feingod s isoation can be seen most ceary via comparison of idea and (MCMC); we find a one point far above the 45-degree ine, near 0.5 on the x-axis. This point represents Feingod, who is the most ibera member if one uses the MCMC version of but ony the fifth-most ibera member as estimated by and ony the twenty-second-most ibera member as estimated by idea. Tabe 1 shows the rank positions of the four members who are deemed the two eftmost members by at east one of the four methods considered. Whie there are sma differences in how Senators Boxer, Corzine, and Kennedy are ranked, Feingod is ranked quite differenty by the methods that assume quadratic utiity and those that assume Gaussian utiity. This difference is attributabe to the difference in idea (quadratic) and (Gaussian) choice probabiities described in Section 2. When one uses quadratic utiity, choice probabiities move quicky to 0 or 1 as the ocation of the bi and the aternative are moved farther from each other and farther from the ocation of the egisator s idea point. If one uses Gaussian utiity, then the choice probabiities do not fa as rapidy when the aternatives are moved farther apart or farther from the egisator s idea point. Feingod is an occasiona ideoogica maverick; on a number of ro cas, he voted with the Repubicans and against amost everyone ese in his party. These maverick votes by an extreme egisator are particuary unikey to occur under idea, and they account for the more-moderate position given to Feingod by that method.