Subset Selection Via Implicit Utilitarian Voting

Transcription

1 Subset Selection Via Iplicit Utilitarian Voting Ioannis Caragiannis University of Patras, Greece Swaprava Nath Carnegie Mellon University, USA Ariel D. Procaccia Carnegie Mellon University, USA Abstract How should one aggregate ordinal preferences expressed by voters into a easurably superior social choice? A well-established approach which we refer to as iplicit utilitarian voting assues that voters have latent utility functions that induce the reported ranings, and sees voting rules that approxiately axiize utilitarian social welfare. We extend this approach to the design of rules that select a subset of alternatives. We derive analytical bounds on the perforance of optial (deterinistic as well as randoized rules in ters of two easures, distortion and regret. Epirical results show that regret-based rules are ore copelling than distortion-based rules, leading us to focus on developing a scalable ipleentation for the optial (deterinistic regret-based rule. Our ethods underlie the design and ipleentation of an upcoing social choice website. Introduction We are interested in the classic social choice proble of aggregating the preferences of a set of voters represented as ranings over a set of alternatives into a collective decision. Traditional social choice theory typically taes a norative approach, by specifying desirable axios that the aggregation ethod (also nown as a voting rule should satisfy [Arrow, 95]. In contrast, researchers in coputational social choice [Brandt et al., 26] often advocate quantitative approaches to the sae proble. The high-level idea is to identify a copelling objective function, and design voting rules that optiize this function. Here we focus on a specific objective function: utilitarian social welfare. Specifically, we assue that each voter assigns a utility to each possible outcoe, and the socially optial outcoe axiizes the su of utilities. This sounds siple enough at first glance, but there is a ajor obstacle we ust overcoe: voters preferences are expressed as ordinal preferences (ranings, rather than cardinal preferences (utilities. While this reduces the cognitive load on voters, and aes preference elicitation uch easier, it does see to be at odds with the utilitarian viewpoint. Nisarg Shah Carnegie Mellon University, USA nshah@cs.cu.edu Procaccia and Rosenschein [26] reconcile these differences via an approach that we refer to as iplicit utilitarian voting. They propose that voters have latent utility functions, and report ranings that are consistent with these utilities, that is, the voters ran the alternatives by their utility. The perforance of a voting rule which can only access the subitted ranings, not the iplicit utility functions can then be quantified via a easure called distortion: the worst-case (over utility functions consistent with the reported profile of ranings ratio between the social welfare of the optial (welfare-axiizing alternative, and the social welfare of the alternative selected by the voting rule. While Procaccia and Rosenschein focus on analyzing the distortion of existing voting rules, Boutilier et al. [25] design voting rules that iniize distortion. In particular, they bound the worst-case distortion, and show that the distortion-iniizing (randoized voting rule can be ipleented in polynoial tie. The wor of Boutilier et al. [25] provides a good understanding of optiized aggregation of ranings fro the utilitarian viewpoint but only when a single alternative is selected by the voting rule. Indeed, this understanding does not extend to coon applications that require selection of a subset of alternatives, such as choosing a coittee, or selecting restaurants for the next four group lunches. Our goal is therefore to... build on the utilitarian approach to design optial voting rules for selecting a subset of alternatives, and understand the guarantees they provide, as well as their perforance in practice. We ae four ain contributions. First, on a conceptual level, we introduce the additive notion of regret into the iplicit utilitarian voting setting, as an alternative to the ultiplicative notion of distortion. Second, in Section 3, we derive worst-case bounds on the distortion and regret of optial deterinistic and randoized voting rules. Third, in Section 4, we copare the worst-case-optial deterinistic voting rules with respect to distortion and regret denoted f dist and f reg, respectively with a slew of well-nown voting rules, in ters of average-case distortion and regret, using experients on synthetic and real data. We find that f reg outperfors all other rules on average, even when easuring Cf. utilitarian voting, which has sporadically been used to refer to both approval voting and range voting.

2 distortion! Fourth, in Section 5, we develop a scalable ipleentation for f reg (which, we show, is N P-hard to copute.. Direct Real-World Iplications Research in coputational social choice has frequently been justified by potential applications in ultiagent systes. But recently researchers have begun to realize that, arguably, the ost exciting products of this research are coputer progras that help huans ae decisions via AI-driven algoriths. One exaple is Spliddit ( a fair division website [Goldan and Procaccia, 24]. In the voting space, existing exaples include Whale (whale3.noiraudes.net/whale3/ and Pnyx (pnyx.dss.in.tu.de but these websites generally adopt the axioatic viewpoint. Since May 25, soe of us have been woring on the design and ipleentation of a new not-for-profit social choice website, RoboVote ( which is scheduled to launch in May 26. The novelty of RoboVote is that it relies on optiization-based approaches. For the case of objective votes when a ground truth raning of the alternatives exists (e.g., the order of different stocs by the relative change in their prices toorrow RoboVote ipleents voting rules that pinpoint the ost liely best alternative [Young, 988], or the set ost liely to contain it [Procaccia et al., 22]. For the case of subjective votes the classic setting which is the focus of this paper, with applications to everyday scenarios such as a group of friends selecting a ovie to watch or a restaurant to go to we use the results of Boutilier et al. [25] to select a single alternative. But, previously, the extension to subset selection was unavailable this is precisely the otivation for the wor described herein. Based on the results of Sections 4 and 5, we have ipleented the deterinistic regret iniization rule on RoboVote..2 Related Wor In addition to the aforeentioned papers [Procaccia and Rosenschein, 26; Boutilier et al., 25], several other papers eploy the notion of distortion to quantify how close one can get to axiizing utilitarian social welfare when only ordinal preferences are available [Caragiannis and Procaccia, 2; Anshelevich et al., 25; Anshelevich and Sear, 26]. In particular, Anshelevich et al. [25] study the sae setting as Boutilier et al. [25], but in addition assue the preferences of voters are consistent with distances in a etric space. We refer the reader to the paper by Boutilier et al. [25, Section.2] for a thorough discussion of wor (in philosophy, econoics, and social choice theory related to iplicit utilitarian voting ore broadly. There is quite a bit of wor in coputational social choice on voting rules that select subsets of alternatives. Typically it is assued that ordinal preferences are translated into a position-based score for each alternative (in contrast to our wor. Just to give a few exaples, under the Chaberlin- Courant ethod, each voter assigns a score to a set equal to the highest score of any alternative in the set, and the (coputationally hard objective is to choose a subset of size that axiizes the su of scores [Chaberlin and Courant, 983; Procaccia et al., 28]. Sowron et al. [25] generalize the way in which the score of a voter for a subset of alternatives is coputed. The budgeted social choice fraewor of Lu and Boutilier [2a] is ore general in that the nuber of alternatives to be selected is not fixed; rather, each alternative has a cost that ust be paid to add it to the selection. 2 The Model Let [t] = {,..., t}. Let A be the set of alternatives, and denote = A. Let N = [n] be the set of voters. Let L = L(A denote the set of ranings over the alternatives. Each voter i [n] subits a raning σ i L over the alternatives, and which can alternatively be seen as a perutation of A. Therefore, σ i (a is the position in which voter i rans alternative a ( is best, is worst. Moreover, a σi b denotes that voter i prefers alternative a to alternative b. The collection of voters (subitted ranings is called the preference profile, and denoted by σ L n. We assue the ranings are induced by coparisons between the voters underlying utilities. For i N and a A, let u i (a [, ] be the utility of voter i for alternative a. As in previous papers [Boutilier et al., 25; Caragiannis and Procaccia, 2], we assue that the utilities are noralized such that a A u i(a = for all i N. The collection of voter utilities, denoted u, is called the utility profile. We say that utility profile u is consistent with preference profile σ denoted u σ if for all a, b A and i N, a σi b iplies u i (a u i (b. Next we need to define the utility of a voter for a set of alternatives. For S A, we define u i (S = ax a S u i (a, that is, each voter derives utility for his favorite alternative in the set; this is in the sae spirit as previous papers on set selection [Chaberlin and Courant, 983; Monroe, 995; Procaccia et al., 28; Lu and Boutilier, 2a]. Then, the (utilitarian social welfare of S given the utility profile u is sw(s, u = n i= u i(s. We are interested in voting rules that, given a preference profile, select a subset of given cardinality. 2 Therefore, it will be useful to denote A = {S A : S = }. In order to unify notation, we directly define a randoized voting rule as a function f : L n (A, that is, the rule is allowed to select alternatives randoly, and forally f( σ is a probability distribution over A. A deterinistic voting rule siply gives probability to a specific subset. A voting rule can only access the preference profile σ, yet the goal is to axiize social welfare with respect to the latent utility function u σ. We study two notions that quantify how well a rule achieves this goal: distortion and regret. The distortion [Procaccia and Rosenschein, 26] of a (randoized voting rule f on a preference profile σ is dist(f, σ = sup u σ ax S A sw(s, u. E[sw(f( σ, u] In words, it is the worst-case over utility profiles consistent with the given preference profile ratio between the social 2 Forally, this is a special case of social choice correspondences with fixed output cardinality [Capbell and Kelly, 996].

3 welfare of the best subset, and the expected social welfare of the subset selected by the voting rule. We define the distortion of a voting rule f by taing the worst case over preference profiles: dist(f = ax σ L n dist(f, σ. The second easure is regret. While it has not been studied as part of the agenda of iplicit utilitarian voting, it has been explored in other social choice settings, especially partial preferences [Lu and Boutilier, 2b]; siilar easures have been extensively studied in decision theory and achine learning [Blu and Mansour, 27; Bubec and Cesa- Bianchi, 22]. The regret of a (randoized voting rule f on a preference profile σ is given by reg(f, σ = ( n sup ax sw(s, u E[sw(f( σ, u]. u σ S A As before, define the regret of a rule f to be reg(f = ax σ L n reg(f, σ. We divide by n because the total (worst-case regret of any voting rule f is provably linear in n (so this is per vote regret. Note that distortion is a ultiplicative easure of loss, whereas regret is its additive version. 3 Worst-Case Bounds In this section we provide bounds on worst-case distortion and regret, for both deterinistic and randoized voting rules. Boutilier et al. [25] show that for selecting a single winner ( =, we can achieve O( log distortion using a randoized rule, where log is the iterated logarith of (the nuber of alternatives. This bound is asyptotically alost tight: they also show that the worstcase distortion is always Ω(. For a large, though, one can hope for a better bound. Clearly, when = there is only one voting rule (which selects every alternative, and its distortion is. More generally, it is easy to show that the voting rule f that selects a subset fro A uniforly at rando has dist(f /. However, since we can already achieve O( log distortion for =, a bound of / provides an iproveent only for = Ω( / log. Can we achieve better distortion for saller values of as well? It is not even clear whether the optial worst-case distortion should onotonically decrease in, because as our flexibility grows with, so does the flexibility of the welfare-axiizing solution. In fact, a part of our ain result shows that the worst-case distortion reains Ω( for all values of up to Θ(. Theore. Let = A, and let be the nuber of alternatives to be selected.., deterinistic rules: There exists a deterinistic voting rule f with dist(f + ( /. Moreover, for every deterinistic voting rule f, + ( 3 6 if 9, dist(f + if 9 < 2, + ( otherwise. These bounds are tight up to a constant factor of 8. 2., randoized rules: There exists a randoized voting rule f such that 2 H if 2 H +H, dist(f 4 if 2 H +H < ( 3 4, otherwise, where H = Θ(log is the th haronic nuber. Moreover, for every randoized voting rule f, { 2 if (, dist(f otherwise. +/ These bounds are tight up to a factor of 6.35 /6. 3., deterinistic rules: There exists a deterinistic voting rule f such that { reg(f 2 if 2, otherwise, and this upper bound is copletely tight. 4., randoized rules: There exists a randoized voting rule f such that reg(f /2 ( 2 / 2. Moreover, for every randoized voting rule f, { reg(f 2 4 if /2 ( otherwise. These bounds are tight up to a constant factor of 2. All the upper bounds above can be achieved via polynoialtie algoriths. Bound Det-UB Det-LB Rand-UB Rand-LB (a Bound (b Figure : The upper and lower bounds on worst-case distortion and regret for =. The bounds presented above are siplified fors of the exact bounds that we derive. Figure shows our exact bounds for =. 3 The intricate proof of Theore appears in the full version of the paper. 4 Below, we just setch one part of the proof that we find especially interesting. 3 The second upper bound in part 2 of Theore (which increases with does not play a role unless is very large. 4 Available at: arielpro/papers.htl

4 Proof setch of the upper bound in part 2 of Theore. Our construction builds on the one used by Boutilier et al. [25] for =, but uses additional tools and introduces novel techniques. As entioned at the beginning of this section, choosing a set uniforly at rando fro A (under which the arginal probability of every alternative being chosen is / has distortion at ost /. However, this approach does not wor well if soe alternatives are significantly better than others. In that case, one ay wish to choose the alternatives with probabilities proportional to their quality. For a A, let us define its quality by its haronic score har(a, σ = i [n] /σ i(a. Then, we wish to choose alternative a with arginal probability har(a, σ/ b A har(b, σ. Note that this quantity ay be greater than. Moreover, this approach fails when all sets are alost equally good. Hence, we eploy a cobination of the two approaches. Fix α, and for an alternative a A define p a = α har(a, σ + ( α ( b A har(b, σ. Using the bihierarchy extension [Budish et al., 23] of the Birhoff-von Neuann theore [Birhoff, 946; von Neuann, 953], we can show that there exists a distribution over A under which the arginal probabilities of selected alternatives are consistent with Equation ( if and only if a A, p a and p a =. a A Suppose such a distribution D exists. Consider a preference profile σ and a utility profile u σ. Let S arg ax S A sw(s, u. Define H X = α α, where H = t= /t is the th haronic nuber. Note that a A har(a, σ = n H. Now, consider two cases. Case : Suppose sw(s, u n X. Then, E S D [sw(s, u] = ( n Pr D [S] ax u i(a a S S A i= ( n a S Pr D [S] u i(a i= S A = n u i (a Pr S D [a S] Hence, the distortion is i= a A n i= a A sw(s, u E S D [sw(s, u] n X α n/ = X = α u i (a α = α n. H α ( α. Case 2: Suppose sw(s, u > n X. Then, for each alternative a S, let N a denote the subset of voters who ran a above any other alternative of S, i.e., N a = {i [n] : b S \ {a}, a σi b, }. Let sw Na (S, u denote the welfare of the voters in N a for the set of alternatives S under the utility profile u. Let T a denote the total utility that agents in N a have for alternative a, i.e., T a = i N a u i (a. It can be shown (although it is nontrivial that har(a, σ T a for all a A. Because {N a } a S is a partition of the set of voters, we have [ ] E S D [sw(s, u] = E S D sw Na (S, u a S T a Pr S D [a S] a S har(a, σ T a ( α a S b A har(b, σ ( α ( (T a 2 α n H n H a S = α n H (sw(s, u 2. a S T a Here, the fourth transition uses har(a, σ T a, the fifth transition uses the power-ean inequality, and the final transition uses sw(s, u = a S T a. Now, the distortion is sw(s, u E S D [sw(s, u] n H ( α sw(s, u < 2 H α ( α, where the final transition uses our assuption sw(s, u > n X along with the definition of X. Cobined analysis: In both cases, the distortion is at ost H /(α( α. The final step involves choosing the optial value of α by iniizing this quantity subject to our constraints: p a for all a A. Siplifying the bound obtained along with our universal distortion bound of / yields the required upper bound. 4 Epirical Coparisons In Section 3 we provided analytical results for both deterinistic and randoized rules. In our view, randoized rules are especially practicable when the output distribution is sapled ultiple ties, or when the voters are well-infored, or when the voters are indifferent about the outcoe (e.g., they are software agents. Moreover, we believe that the results for randoized rules are of substantial theoretical interest. But our wor is partly driven by its direct applications in RoboVote (see Section., which does not satisfy the above conditions. This leads us to use deterinistic voting rules, which is what we focus on hereinafter. Let f dist and f reg be the deterinistic rules that iniize the worst-case distortion and regret, respectively, on every given preference profile. The deterinistic results of Section 3 establish upper and lower bounds on their worst-case

5 f reg f dist Plurality Borda STV Other (a Avg (b Avg (a Avg (b Avg. Figure 2: Uniforly rando utility profiles. Figure 3: Utility profiles fro the Jester dataset (a Sushi: Avg (b Sushi: Avg (c T Shirt: Avg (d T Shirt: Avg. Figure 4: Preference profiles fro the Sushi and the T-Shirt datasets, uniforly rando consistent utility profiles. distortion/regret. In this section, we evaluate their averagecase perforance on siulated as well as real data, and copare the against nine well-nown voting rules: plurality, approval voting, Borda count, STV, Keeny s rule, the axiin rule, Copeland s rule, Buclin s rule, and Tidean s rule. 5 We perfor three experients: (i choosing a utility profile uniforly at rando fro the siplex of all utility profiles, (ii drawing a real-world utility profile fro the Jester datasets [Goldberg et al., 2], and (iii drawing a realworld preference profile fro the PrefLib datasets [Mattei and Walsh, 23], and choosing a consistent utility profile uniforly at rando. For each experient, we have 8 voters and alternatives, and test for [4]. 6 For each setting, we perfor rando siulations, and easure both distortion and regret for the actual utility profile, as opposed to the worst-case utility profile. The figures show the average perforance with 95% confidence intervals. In all of our siulations, we observed that three of the classical voting rules stand out: Borda count perfors well for choosing a single alternative (but not for choosing larger subsets whereas plurality and STV perfor well for choosing larger subsets (but not for choosing a single alternative. 5 For the score-based rules, the -subset is selected by picing the top alternatives based on their scores. 6 In RoboVote, we expect typical instances to have few voters and alternatives. Hence, all of our graphs specifically distinguish these three rules in addition to fdist and f reg. Figure 2 shows the results for the first experient where we choose the utility profile uniforly at rando. Figure 3 shows the results for the second experient where real-world utility profiles are drawn fro one of the Jester datasets, in which ore than 5 voters rated 5 joes on a realvalued scale; the results fro the other Jester dataset are alost identical. Finally, Figure 4 shows the results for the third experient where real-world preference profiles are drawn fro the Sushi dataset (5 voters raning different inds of sushi and the T-Shirt dataset (3 voters raning T-shirt designs fro PrefLib. Experients on other datasets fro PrefLib (AGH Course Selection, Netflix, Sate, and Web Search yielded siilar results. Right off the bat, one can observe that the average-case distortion and regret values are uch lower than their worst-case counterparts. For exaple, average regret is generally lower than. copare with the tight worst-case deterinistic bound of /2 for /2. Much to our surprise, in all of our experients, freg outperfors fdist in ters of both average-case distortion (ultiplicative loss and regret (additive loss. While both easures of loss have been studied extensively in the literature, we are not aware of any previous wor that copares the two approaches. At least in our social choice doain, the regret-

6 based approach is clearly better on average. Moreover, in all cases but one ( = in the Jester experient, f reg also outperfors all the classical voting rules under consideration. We therefore conclude that, on rando as well as on real-world instances, f reg provides superior perforance in ters of social welfare axiization. 5 Coputation and Ipleentation In this section, we analyze and copare the two deterinistic optial rules f dist and f reg fro a coputational viewpoint. Selecting optial subsets turns out to be challenging, as both rules are N P-hard to copute; the proof of this nontrivial result appears in the full version. Theore 2. Given a preference profile σ and an integer, coputing a -subset of alternatives that has the iniu distortion or the iniu regret on σ is N P-hard. Given that freg outperfors fdist in the experients of Section 4, and that both rules are coputationally hard, freg stands out as the clear choice for ipleentation in our website RoboVote. We therefore devoted our efforts to developing a scalable ipleentation for freg. The first step is to siplify the description of freg. Given a raning σ and an alternative a A, recall that σ(a denotes the position of a in σ. For a set S A, let σ(s = in a S σ(a. For sets S, T A, we say T σ S if σ(t < σ(s, i.e., if there exists an alternative in T that is preferred in σ to every alternative in S. Using these notations, it is relatively straightforward to prove that f reg,( σ = arg in T A ax S A i N:S σi T σ i (S. (2 To better understand this equation, we consider the special case of =. In this case, f reg( σ arg in a A ax b A i [n]:b σi a σ i (b. Note that this voting rule is very siilar to the classical axiin rule: replacing /σ i (b with would yield the axiin rule. Thus, in soe sense, this is a sooth version of the axiin rule, where the victory of b over a in voter i s vote is weighted by the strength of b in this vote (easured by /σ i (b. In our view, this intuitive structure aes f reg even ore copelling. We now briefly describe six approaches we have developed for coputing f reg:. Naïve: This uses Equation (2, and requires Ω(n ( 2 operations, which is prohibitive even for sall. 2. Subodular: The regret for set S in choosing set T, i.e., i [n]:s σi T /σ i(s, is subodular in S. Hence, for each T A we can optiize over S A using any algorith for the subodular axiization subject to cardinality constraint (SMCC proble. We use the SFO toolbox for Matlab [Krause, 2]. 3. Subodular+Greedy: This iproves the previous approach by first coputing a /e greedy approxiation to the SMCC instance for set T, and pruning T if this is already greater than the best regret found so far. 4. MultiILP: Instead of using SMCC, for each T A we optiize over S A by solving an integer linear progra (ILP with roughly n variables and n 2 constraints. Note that ( such ILPs need to be solved. 5. MultiILP+Greedy: This iproves the MultiILP approach by using a greedy pruning procedure as before. 6. SingleILP: This approach solves a single but huge ILP with ( additional constraints. Figure 5 shows the average running ties of these approaches (and 95% confidence intervals over instances with n = 5, = 3, and varying fro to 5. 7 The experients were perfored on a single achine with quad-core 2.9 GHz CPU and 32 GB RAM. A tie liit of 2 inutes was set because a running tie greater than this would not be helpful for our website, where the results need to be delivered quicly to the users. While the greedy pruning procedure does help reduce the running tie of both the Subodular and MultiILP approaches, SingleILP still coputes f reg uch faster than any other approach, solving instances with 5 alternatives in less than seconds. We have therefore ipleented SingleILP on RoboVote. Tie (s Naive Subodular Subodular+Greedy MultiILP MultiILP+Greedy SingleILP Tie Liit Figure 5: Running ties of six approaches to coputing f reg. 6 Discussion We find it exciting that new theoretical questions in coputational social choice are driven by concrete real-world applications. And while research in the field is often otivated by potential applications to ultiagent systes, we focus on helping people not software agents ae joint decisions. We also rear that we consider the epirical doinance of f reg, in ters of both regret and (surprisingly distortion, to be especially significant. It would be interesting to understand, on a theoretical level, why this happens. A proising starting point is to derive analytical bounds on the averagecase distortion of f dist and f reg under uniforly rando utility profiles. 7 The running tie scales linearly in n, and increases with (.

7 Acnowledgents This wor was supported in part by NSF grants CCF , CCF , and IIS-35598; and by a Sloan Research Fellowship. References [Anshelevich and Sear, 26] E. Anshelevich and S. Sear. Blind, greedy, and rando: Algoriths for atching and clustering using only ordinal inforation. In Proceedings of the 3th AAAI Conference on Artificial Intelligence (AAAI, 26. Forthcoing. [Anshelevich et al., 25] E. Anshelevich, O. Bhardwaj, and J. Postl. Approxiating optial social choice under etric preferences. In Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI, pages , 25. [Arrow, 95] K. Arrow. Social Choice and Individual Values. Wiley, 95. [Birhoff, 946] G. Birhoff. Three observations on linear algebra. Universidad Nacional de Tucuán, Revista A, 5:47 5, 946. [Blu and Mansour, 27] A. Blu and Y. Mansour. Learning, regret iniization, and equilibria. In N. Nisan, T. Roughgarden, É. Tardos, and V. Vazirani, editors, Algorithic Gae Theory, chapter 4. Cabridge University Press, 27. [Boutilier et al., 25] C. Boutilier, I. Caragiannis, S. Haber, T. Lu, A. D. Procaccia, and O. Sheffet. Optial social choice functions: A utilitarian view. Artificial Intelligence, 227:9 23, 25. [Brandt et al., 26] F. Brandt, V. Conitzer, U. Endress, J. Lang, and A. D. Procaccia, editors. Hanboo of Coputational Social Choice. Cabridge University Press, 26. [Bubec and Cesa-Bianchi, 22] S. Bubec and N. Cesa- Bianchi. analysis of stochastic and nonstochastic ulti-ared bandit probles. Foundations and Trends in Machine Learning, 5(: 22, 22. [Budish et al., 23] E. Budish, Y.-K. Che, F. Kojia, and P. Milgro. Designing rando allocation echaniss: Theory and applications. Aerican Econoic Review, 3(2: , 23. [Capbell and Kelly, 996] D. E. Capbell and J. S. Kelly. Arrovian social choice correspondences. International Econoic Review, 37(4:83 823, 996. [Caragiannis and Procaccia, 2] I. Caragiannis and A. D. Procaccia. Voting alost axiizes social welfare despite liited counication. Artificial Intelligence, 75(9 :655 67, 2. [Chaberlin and Courant, 983] J. R. Chaberlin and P. N. Courant. Representative deliberations and representative decisions: Proportional representation and the Borda rule. Aerican Political Science Review, 77(3:78 733, 983. [Goldberg et al., 2] K. Y. Goldberg, T. Roeder, D. Gupta, and C. Perins. Eigentaste: A constant tie collaborative filtering algorith. Inforation Retrieval, 4(2:33 5, 2. [Goldan and Procaccia, 24] J. Goldan and A. D. Procaccia. Spliddit: Unleashing fair division algoriths. SIGeco Exchanges, 3(2:4 46, 24. [Krause, 2] A. Krause. SFO: A toolbox for subodular function optiization. Journal of Machine Learning Research, :4 44, 2. [Lu and Boutilier, 2a] T. Lu and C. Boutilier. Budgeted social choice: Fro consensus to personalized decision aing. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI, pages , 2. [Lu and Boutilier, 2b] T. Lu and C. Boutilier. Robust approxiation and increental elicitation in voting protocols. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI, pages , 2. [Mattei and Walsh, 23] N. Mattei and T. Walsh. Preflib: A library of preference data. In Proceedings of the 3rd International Conference on Algorithic Decision Theory (ADT, pages , 23. [Monroe, 995] B. L. Monroe. Fully proportional representation. Aerican Political Science Review, 89(4:925 94, 995. [Procaccia and Rosenschein, 26] A. D. Procaccia and J. S. Rosenschein. The distortion of cardinal preferences in voting. In Proceedings of the th International Worshop on Cooperative Inforation Agents (CIA, pages 37 33, 26. [Procaccia et al., 28] A. D. Procaccia, J. S. Rosenschein, and A. Zohar. On the coplexity of achieveing proportional representation. Social Choice and Welfare, 3(3: , 28. [Procaccia et al., 22] A. D. Procaccia, S. J. Reddi, and N. Shah. A axiu lielihood approach for selecting sets of alternatives. In Proceedings of the 28th Annual Conference on Uncertainty in Artificial Intelligence (UAI, pages , 22. [Sowron et al., 25] P. Sowron, P. Faliszewsi, and J. Lang. Finding a collective set of ites: Fro proportional ultirepresentation to group recoendation. In Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI, pages , 25. [von Neuann, 953] J. von Neuann. A certain zero-su two-person gae equivalent to the optial assignent proble. In W. Kuhn and A. W. Tucer, editors, Contributions to the Theory of Gaes, volue 2, pages [Young, 988] H. P. Young. Condorcet s theory of voting. The Aerican Political Science Review, 82(4:23 244, 988.