Coalition formation among autonomous agents: Strategies and complexity. Abstract. Autonomous agents are designed to reach goals that were

Transcription

1 Coalition formation among autonomous agents: Strategies and complexity (preliminary report)? Onn Shehory Sarit Kraus Department of Mathematics and Computer Science Bar Ilan University Ramat Gan, Israel fshechory, Tel: Fax: Abstract. Autonomous agents are designed to reach goals that were pre-dened by their operators. An important way to execute tasks and to maximize payo is to share resources and to cooperate on task execution by creating coalitions of agents. Such coalitions will take place if, and only if, each member of a coalition gains more if he joins the coalition than he could gain before. There are several ways to create such coalitions and to divide the joint payo among the members. Variance in these methods is due to dierent environments, dierent settings in a specic environment, and dierent approaches to a specic environment with specic settings. In this paper we focus on the cooperative (super-additive) environment, and suggest two dierent algorithms for coalition formation and payo distribution in this environment. We also deal with the complexity of both computation and communication of each algorithm, and we try to give designers some basic tools for developing agents for this environment.? This material is based upon work supported in part by the NSF under Grant No. IRI

2 1 Introduction Cooperation among autonomous agents may be mutually benecial even if the agents are selsh and try to maximize their own expected payos [9, 20, 24]. Mutual benet may arise from resource sharing and task redistribution. Coalition formation is an important method for cooperation in multi-agent environment. Agent membership in a coalition may increase the agent's ability to satisfy its goals and maximize its own personal payo. There are two major questions concerning coalition formation: 1. which procedure should a group of autonomous agents use to coordinate their actions and cooperate; namely, how should they form a coalition? 2. among all possible coalitions, what coalition will form, and what reasons and processes will lead the agents to form that particular coalition? Work in game theory such as [14, 17, 10, 12] describes which coalitions will form in N-person games under dierent settings and how the players will distribute the benets of the cooperation among themselves. That is, they concentrate mainly on the second problem above. These results do not take into consideration the constraints of a multi-agent environment, such as communication costs and limited computation time. In this paper we adjust the game theory concepts to autonomous agents, and present dierent coalition-formation procedures. The resulting coalitions of each of these procedures may be dierent even in the same setting. We developed general criteria for choosing between the coalition-formation procedures. Giving a specic setting and using these criteria will enable the agents to decide which coalition-formation procedure to prefer. We will concentrate on widely cooperative environments [4, 7] such as the postmen problem of [23]. The coalitionformation procedures that will be presented are either computation-oriented or negotiation-oriented. The appropriate procedure can be chosen according to the constraints of the environment and will lead to benecial coalition formation. We begin by describing the environment with which we will deal and give the basic denitions of coalitions in section 2.1. In section 3 we concentrate on super-additive environments. The negotiation protocol is described in section 3.1, together with its complexity analysis. The Shapley value protocol and its complexity are described in section 3.2. Finally, we give the criteria for choosing between the two protocols. 2 Environment description This paper deals with autonomous agents, each of which has tasks it must fulll and access to resources that it can use to fulll these tasks. Resources may include: materials, energy, information, communications and others. Autonomous agents can act and reach goals by themselves, but may also join together to reach some or all of their goals. In such a case we say that the agents form a coalition. In order to deal with dierent types of agents' environments, we give some general notations and denitions for concepts (i.e., resources, tasks, methods of reaching goals, success in reaching them, etc). Resources will be treated by 1

3 numbers which will denote the quantities of each. Success in fullling tasks will be dened as the production or the payo of an agent. The concept of payo will be preferred specically when reaching tasks is exchangeable with some kind of payment (e.g., money). The ways of reaching goals can be formalized by functions from the resources to the production or to the payo. Each agent will have such a function of its own that tells what its way of using the resources to reach goals is. An example of such an environment is the postmen domain [23]. While Zlotkin and Rosenschein consider only bi-agent environments (e.g., two postmen), we provide cooperation procedures for multi-agent environments (e.g., several agents, possibly more than two). In addition, Zlotkin and Rosenschein do not consider both the resource allocation problem and task distribution in the same setting e.g., we provide the postmen with procedures for division of their overall transportation capabilities and the benets from fullling the letter distribution. In order to make the creation of mutually benecial coalitions possible, we make the following two assumptions: Assumption 1 Communication We assume that various communication methods exist, so that the agents can negotiate and make agreements [22]. We also assume that communications require time and eort on the part of the agent. Assumption 2 Goods transferability & side-payments We assume that resources and products can be transferred between agents in the environment, and that there is a monetary system that can be used for sidepayments. Multi-agents may reach agreements and form coalitions even if the second assumption is not valid (e.g., [10, 23, 1, 2]). However, the ability of goods transferability or side-payments may help the agents form more mutually-benecial coalitions. Our postmen example satises the assumptions above; they can communicate, and letters, transportation and money (their resources and production) can be transferred. 2.1 Denitions Let N be a group of n autonomous agents, N = fa1;a2;:::;a n g. Let L be a set of m resources, L = fl1;l2;:::;l m g. The resources in the environment are limited. We consider only the resources that are distributed among the agents. This is formalized by the following denition: Denition1. Resource domain We say that Q is the possible resource domain if Q is a set of the possible vectors of quantities of resources, Q = fhq 1 ;q 2 ;:::;q m ig, where q j is a quantity of the resource l j. 2

4 Each agent A i has its own resources, therefore it has a vector q i 2 Q, q i = hq 1 i ;q2 i ;:::;qm i i, which denotes the quantities of resources that A i has. Due to the limitation of the quantities of the resources, we dene q D 2 Q to be the vector of the quantities of the resources P of the domain. It is obvious that every element k of q D,must satisfy qd k = n j=1 qk j, where qk j is agent A j's quantity of resource k. The vector q D denotes the total amounts of resources available to all agents in the environment, together. We are interested in q D particularly in cases of cooperation between agents, where q D may sometimes be re-distributed among them. The agents use the resources they have to execute their missions. Depending on the quantities of resources an agent has and on its way of using them to fulll tasks, each agent can reach all, part or none of its goals. For example, the postmen resources are the letters they have to distribute and their transportation abilities. Production is a quantitative way of measuring the agent's success in fullling tasks. As such, we shall call production any kind of success in fullling tasks or reaching goals. Therefore, we give the following denition: Denition 2. Production set We say that P is a production set if P is a set of values which are the possible production values of a group of agents. For example, if the agents are postmen, then the production set may be the set of amounts of letters that they can distribute, or their potential income. Each agent in the environment has its own method of using the resources to reach goals. Therefore, we say that each agent A i has a function fp i that formulates its way of using resources to reach goals; this function is dened as: Denition 3. Production Function We say that fp i : Q! P is a production function, if f P i is a function which gives avalue within the production set that measures the production of agent A i with an arbitrary resources vector. It may be inconvenient to use the production function for agents' decision making. Each designer of automated agents has to provide its agent with a decision mechanism based on some given set of preferences [21]. Therefore, we suggest that each autonomous agent will be provided with a numerical payo function that gives a transformation from the production set values and the resources to the reals. Each agent A i has such a function, U i, that exchanges its resources and production into monetary units. This monetary system can be used for evaluation of production and for side-payments. Denition 4. Payo Function We say that U i :(P; Q)!Ris a payo function if U i gives a measure, in monetary units, of the outcome that an agent has for some arbitrary resources and production. A postman's payo function is the dierence between his transportation costs and the payment that he receives for distributing letters. The payments that postmen receive are transferable among them. 3

5 Individual self-motivated agents may be cooperative; they may cooperate by sharing resources, redistributing tasks and passing side-payments. Self-motivated agents will cooperate only if, as a result of this cooperation, they increase their payo. A coalition can be dened as a group of agents that decided to cooperate and also decided how the total benet should be disbursed among its members. Formally,we dene: Denition5. Coalition Given a group of agents N, a resource domain Q, and a production set P,we dene a coalition as a quadrate C = hn C ;Q C ; q; U C i. In this quadrate, N C N; Q C = hq 1 ;q 2 ;:::;q m ), Q C 2 Q, is a coalitional resource vector, where q j = PA i2n C q j i is the quantity of resource l j that the coalition has. q is the set of vectors of resource quantities after the redistribution of Q C among the members of N C (q satises q j = P A i2n C q j i ). U C = hu 1 ;u 2 ;:::;u jcj i, u i 2R,isthe coalitional payo vector, where u i is the payo of agent A i after the redistribution of the payos. We say that C is a set of possible coalitions if C is the group of all possible coalitions over N. In order to provide the agents with a method for coalition evaluation, we give each coalition a value (as can be found in game theory [12]), and a function for calculating this value. Denition6. Coalition Value & Coalitional Function Let C = hn C ;Q C ; q; U C i.wesay that V is the value of P C if the members of N C can together reach a joint payo V. That is, V = A i2n C U i (q i ), where U i is the payo function of agent A i and q i is its vector of resources after their redistribution in the coalition. Let q i be agent A i 's original vector of resource quantities; we assume that 8A i, q i 2 q, u i U i (q i ). That is, an agent will join a coalition only if the payo it will receive in the coalition is greater than, or at least equal to, what it can P obtain by staying outside the coalition. Hence, it is easy to conclude that V A U i i2c (q i ). Moreover, we assume that the resources are redistributed within q inaway that maximizes the value of the coalition. Therefore, the coalition value V of a specic group of agents N C is unique. The complexity of computing the redistribution of the resources and calculating the coalitional value depends on the structure of the coalition's members' payo functions. For example, for linear payo functions the simplex method can be used. Now, let us take the production function in denition 3 and expand its scope: We say that fp C :C! P is a coalitional production function if f P C attaches a value p 2 P to all coalitions in C. Wesay that U C :(P; Q)! R is a coalitional payo function if U C transforms the coalitional production into coalitional payo. We now formally state more assumptions that consider a multi-agent environment in which the agents are self-motivated and try to maximize their own payo function. 4

6 Assumption 3 Coalition joining (personal rationality) We assume that each agent in the environment has personal rationality, i.e., it joins a coalition only if it can benet at least as much within the coalition as it could benet by itself. An agent benets if it fullls some or all of its tasks, or gets a payo that compensates it for the loss of resources or non-fulllment of some of its tasks 2. Assumption 4 Personal payo maximization We assume that each agent in the environment tries to maximize its personal payo; among all the possibilities that it has, an agent will choose the one that will give it the maximum expected payo 3. Our postmen example satises the two assumptions above: they are personally rational, and they try to maximize their personal payo. Now we shall dene a new concept { the coalitional rationality. The environment is coalitionally rational if each coalition in it will add a new member only when the value of the coalition that will be formed by this addition is greater than (or at least equal to) the value of the original coalition. Not all environments are coalitionally rational. Situations come to mind whereby an agent is added to an existing coalition only because of the additional benet for powerful agents and for the newly-added agent. Thus, the assumption of personal rationality is fullled, while the value of the new coalition is lower than the value of the original coalition. One can say that such a situation is not likely to happen, and can justify this statement by claiming that if some agents benet more and the value of the new coalition is less than the value of the original one, then there must be at least one agent that will benet less in the new coalition. That agent should never let such a coalition form, as the personal rationality assumption dictates. We reject this claim and say that such situations may happen if, after the change in the coalition, the agents that benet less still prefer to stay in the new coalition, because they believe that this is the best option, given the new situation. 3 Super-additive environment The concept of coalitional rationality can be broadened if we project it from the relation between a coalition and a single agent to the relation between two coalitions. By this expansion we dene the super-additive environment: Denition 7. Super-additive environment A super-additive environment is a set of possible coalitions C that satises the following rule: for each pair of coalitions C1,C2 in the set C, C1 \ C2 =, if C1,C2 join together to form a new coalition, then the new coalition will have a new value V new C V 1 C + V 2 C, where V 1 C,V 2 C,V new C are the values of the coalitions. 2 This assumption is usually called \personal rationality" in the game theory literature [8, 17, 12]. 3 Note that assumption 3 can be derived from assumption 4. We present here both assumptions only for clarication purposes. 5

7 Not all environments are super-additive, but if an environment is superadditive, then, under assumptions A1 - A4, after a sucient time period a grand coalition will form. A grand coalition is a coalition that includes all of the agents. We come to the conclusion that a grand coalition will form because in any other situation there will be at least two coalitions that are not a grand coalition. If the environment is super-additive then the coalitional rationality and the personal payo maximization will make them join together and form a joint coalition, that, according to the super-additivity concept, will haveavalue V new V 1 +V 2 [18]. We can let the agents negotiate coalition formation, but all agents know that in a super-additive environment a grand coalition will form. Hence, the only problem that still remains is how the payo should be distributed among its members. It seems that the best way tosave time and eort for all agents will be to agree, without any argument, to form a grand coalition, and to solve the problem of payo distribution either by negotiation or by calculation. Denition8. Common extra payo In a super-additive environment, if two coalitions C i ;C j with corresponding coalitional values V i ;V j can form a joint coalition and obtain together V ij = V i + V j + D ij, then D ij is the common extra payo (the indices i; j may be dropped). We present two algorithms for solving the problem stated above: one that requires negotiations and another that requires only computations. We discuss the advantages of each later. 3.1 Negotiation in a super-additive environment The initial coalitional state consists of n, single agent, coalitions. The coalitions then begin negotiating [11] and, step by step, form coalitions. Coalitions will continue negotiations via agents who are representatives of the coalitions of which they are members. These will form bigger coalitions, until a grand coalition forms. At the beginning of each step of this coalition formation process, each coalition will nd what the common extra payo from forming a new coalition with each of the other current coalitions is. Next, each coalition will sort its list of extra payos. The rst coalition on each sorted list (i.e., the one that can bring maximum extra payo) is \wanted" by the coalition that made this list. Two coalitions that want one another can start a bargaining process. In a superadditive environment, at least one such pair must exist. If there is more than one pair, then all will start a bargaining process. A bargaining process entails coalitions oering one another partition of their common D. A coalition that received such an oer may accept it, and then a new joint coalition can form. A coalition may also reject the oer, and either ask for a better one, or make its own oer. To establish order in the bargaining process, we dene a strength relation between coalitions, and use this relation to determine which coalitions will be rst to make oers. 6

8 Strong coalition We shall dene a strong coalition by saying that coalition C i is stronger than coalition C j,ifc i has an extra payo D ik with some other coalition C k, k 6= j, which is bigger than D ij. It can be shown that two members of a pair of coalitions that are \wanted" by one another (such as that mentioned above), are both stronger than all other coalitions 4 ; therefore we shall call them \powerful" [3]. We denote these two coalitions as C 1 p ;C2 p. The bargaining process is begun by the powerful coalitions. Within the pair of powerful coalitions, the coalition with the higher computational capabilities will be the rst to calculate an oer and send it to its partner. We shall designate this rst coalition as C1 p and its associate as Cp 2. The rst oers made to one another will be exactly half of D, their joint extra payo 5. Then, each powerful coalition will contact all coalitions that are weaker than it, inform them of the amount of the rst oer it received and nd among them the one coalition that is willing to give it the largest share of their common D. Any strong coalition, having received an oer (or oers) with higher than that presented by powerful coalition, will use the oer to challenge this powerful coalition. That is, the strong coalition will ask the powerful coalition for exactly as much of the common D as it could get from other coalitions. At this point, the powerful coalition C 1 p must contact its parallel coalition Cp, 2 inform it about: 1. the amount of the highest oer it received, and 2. the coalition Cw i that oered it. If C 2 p was also challenged, and its highest oer was given by a coalition Cw, j i 6= j, then C 1 p and C 2 p will reject one anothers' oers and each will join with its challenger. If both C 1 p and C2 p are challenged by Ci w, then C 1 p has priority to join Cw i and so C 2 p should check if it has another challenging coalition. If so, then again, as above, both will reject one another. If C 2 p was not challenged by the oers made, it should nd the highest oer among these. If the dierence between D12 p and this highest oer is greater than C1 p's highest oer, C 2 p will accept the new oer and join Cp. 1 Otherwise it will reject C 1 p and C 1 p will join Cw. i Weak coalition Each weak coalition, C w, that was approached by another coalition, C s, will calculate what it can oer by considering two issues: a. whether or not it is able to oer at least as much asc s could get without cooperation with C w ; b. if the answer to (a) is positive, how much more it can oer. Consequently, C w will follow this procedure: among all coalitions, excluding C s, the coalition C w will nd those that are weaker or equal to it (the stronger ones will approach C w ). If there are none, then C w will oer C s as much asc s claims it can get, plus " ( " is a positive innitesimally small number), up to a maximum of all of their common D ws. This proposition is valid, unless D ws is smaller than the oer that C s already has { then C w will have to oer D ws, which will 4 Note that it can be that Ci is stronger than Cj and Cj is stronger than Ci. 5 This partition is suggested only as a fair starting point of the negotiation. During the negotiation process the partition of D will change according to the relative strength of the coalitions. 7

9 probably be rejected. If weaker coalitions exist, C w will contact them, and each of these coalitions will have to go through the same procedure, recursively, to calculate their answer to the calling coalition. After C w was answered by all of the contacted coalitions, it nds the one that oers it the maximum amount of payo. This amount is the minimal part of D that C w should ask for, from its common D with C s, and the maximum is the amount that C s claimed it could get, plus ". To make it more clear, the following should be the typical answer of a requested coalition to its caller: \You said that you can get x, and therefore I am willing to give you x + ", but if you get a better oer, I can give you a maximum of x +, and I will still be satised". 6 Payo disbursement When two coalitions C1 and C2 join together to form a new coalition, they get extra payo D12, which is divided between them into D1;D2, such that D1 + D2 = D12. Now, we must present a method for distributing D1 and D2 among the members of C1;C2, respectively. Under the assumptions of coalitional rationality and super-additivity, we suggest that once a coalition forms, it will not split up. Therefore, in order to make these assumptions acceptable to the members of a coalition, we say that within an existing coalition, each member keeps its previous strength { the strength that was expressed by the ratio between the total D and the member's part of it when it joined the coalition. Therefore, any new added payo should be divided among the coalition members according to their strengths during the process of coalition formation. Of course, this does not prevent them from keeping for themselves any amount of payo which they had before-hand. Complexity of the negotiation algorithm The eciency of this algorithm should be judged from two main perspectives: computations and communications. The rst step of the negotiation process will be the transfer of all of the relevant information, i.e., payo functions and resources vectors, between the agents (unless it is known in advance). This requires (n, 1) 2 communication operations. The negotiation process will proceed from the initial state of n single agents to the nal state of a grand coalition, and will take upton, 1 steps, since at each step the number of coalitions will decrease by at least 1. At each step, each determines its extra common payo with all other coalitions; this requires, n, 1 maximization operations, which are rather complicated. Upon deriving this information, each coalition must sort it (o(n lg 2 n)), and then contact the rst coalition on the sorted list to nd out if they are a powerful pair. Next, all strong coalitions contact all weaker coalitions, and all weaker coalitions, after having nished their calculations, re-establish contact with the corresponding strong coalitions in order to answer them. Altogether, there are 2(n, 1) 2 communication operations, and each such operation is connected with o(1) calculations. Afterwards, the powerful coalitions contact one another once again, 6 This assumes honesty or complete information on both the coalitional values and the messages that are transmitted. Otherwise, it will require long and exhausting negotiations to reach an agreement. 8

10 and search for the highest oer they received. At last, coalitions join together to form new ones. Since, in the end, all coalitions join together, there can be at least dlg 2 (n, 1)e and at most n, 1 such events. Any two coalitions that join together and form a new coalition, must perform the following actions: a. choose a representative from among its members. This action will take o(n) communication operations and the same order of computations. b. distribute the extra payo among all members. With a given simple algorithm to do so, the complexity of this action is o(n). Now that we have determined the complexity of all the parts of the algorithm, we can construct the complexity of the general algorithm. There are at most n,1 negotiation steps; at each step there are o(n 2 ) communication operations, and o(n 2 lg 2 n) computations. Therefore, the complexity of the general algorithm is, in the worst case, o(n 3 ) communication operations, and o(n 3 lg 2 n) computations. Example We shall use here an example to illustrate the negotiation algorithm. There are three agents in a super-additive environment. Each agent receives no payo by itself, so that the coalitional values of a single-member coalition is zero. The values of the other coalitions are V12 = 10, V13 =7,V23 =2,V123 = 15. Agents A1 and A2 are the pair of powerful coalitions, since their common extra utility D12 is greater than their extra utility with A3 (This is easy to conclude since in our specic example 8i; j, D ij = V ij ). Suppose that agent A1 starts the negotiation by oering A2 5, half of their common D. A2 will oer A1 the same amount. At this point A1 will approach A3, inform it about the current oer and ask for a better oer. Agent A3 will oer A1 5+", and will announce that it can oer up to 7, if necessary. Meanwhile, Agent A2 will also approach A3. The oer of A3 to A2 will be 2. With the oer A1 has received from A3 it will approach A2 and ask for more than 5, since it can receive more than 5 from A3. Agent A2 will compete with A3's oer by oering A1 7+". At this point, none of the agents can make an oer that will change the payos and will be accepted. Therefore A1 and A2 will form a coalition; A1 will receive a payo of 7+" and A2 will receive apayo of 3, ". Thus, one step of the negotiation is accomplished. If agent A3 would not have indicated that he can oer up to 7, a possible scenario is that A2 will oer A1 5+ for some > ", and A3 will respond by increasing its oer. This incremental process can proceed until eventually A2 oers A1 7 +". As we explained above, the fact that A3 indicated that it can oer up to 7 shortened the bargaining process. The next step of the algorithm will cause the formation of the grand coalition. Since there are only two coalitions at this stage, and their common extra payo is 5, the only possible oer is to divide D by 2, so that each coalition will receive 2.5. This oer will be accepted, and the coalition of A1 and A2 will divide its new additional payo according to the previous relative strengths as was shown in the last step. The resulting payo vector will be: U1 = 7 + 0:7 2:5 +" 8:75, U2 =3+0:3 2:5, " 3:75, U3 =2:5. Note that the nal payos of the agents reect their relative strength. 9

11 Discussion of the negotiation algorithm The outcome that a coalition gains as a result of the negotiation algorithm reects its relative contribution to possible coalitions and its computational capabilities. Coalitions which contribute more relative to others will be preferred by the other coalitions and will receive a larger share of the common extra payo. As written above, when a pair of \powerful" coalitions is created, the coalition with the higher computational capabilities will be the rst to calculate an oer and send it to its partner. The situation of being the rst to make an oer in the negotiation process may give coalitions an advantage when joining with computationally weaker coalitions. This means that the algorithm may be advantageous to agents that have better computational capabilities. There may be occasions when a coalition A obtains its highest common extra payo with more than one other coalition. If A's highest common extra payo with two dierent coalitions B1 and B2 is identical, then A should choose one of them to be its partner in a "powerful" pair. Being chosen by A may give the chosen coalition an advantage over the other. The ability to choose from among B1 and B2 the one that will be included in a pair provides A with some advantage in the negotiation. For example, A can demand to be designated as C 1 p, i.e., to be the rst to make an oer when negotiating the formation of a joint coalition. One possible criteria that A may use for choosing among B1;B2 is their computational power. The algorithm requires that after a coalition has chosen its partner in a pair, it cannot depart. This requirement leads to stability of the negotiation process. Being chosen as a "powerful" coalition according to the algorithm does not necessarily give an advantage to the chosen coalition. This is because a powerful coalition C p has to approachaweaker coalition C w. The proposal that C w returns to C p may be the minimum necessary for cooperation with C p, although C p can decide to avoid cooperation with C w and cooperate with another weak coalition or with its associate, the other powerful coalition. The main factor that plays a role in the coalition's eventual outcome is its relative contribution to possible coalitions. Whether being "powerful" is advantageous for the powerful coalition depends on the details of the specic situation, and cannot be predicted without complex computation. Although powerful coalitions do not necessarily have anadvantage over other coalitions, in the negotiation algorithm we do prefer that these coalitions, which have a larger common extra payo, will begin the negotiation process. This is because coalitions with larger common extra payos will have more exibility when negotiating coalition formation. Therefore, it is more likely that such coalitions will form joint coalitions faster and with less computations than other coalitions. Thus, the overall costs will decrease, an advantage that the designers of agents should seek. 10

12 3.2 Shapley value Awell-known solution to the problem stated above was suggested by Shapley 7 [18]. Given a group of agents in a super-additive environment, and assuming that a grand coalition had formed, Shapley suggests a function ' that attaches to each agent A i in the grand coalition a unique value v i, which is its share of the joint payo. ' is a general payo function, which is dened for all of the agents. Shapley suggested that ' should satisfy the following axioms: Axiom 3.1 Symmetry For every pair of agents A i ;A j 2 N, for all the coalitions such that A i ;A j 62 N C, if V (N C [fa i g)=v (N C [fa j g) then v i = v j, and we say that A i ;A j are exchangeable. Axiom 3.2 Null agent For every agent A i 2 N, for all the coalitions such that A i 62 N C,ifV (N C [ fa i g)=v (N C ) then v i =0, and we say that A i is null. Axiom 3.3 Eciency For all the agents A i 2 N, V (N) = P i v i. Axiom 3.4 Additivity For every pair of coalitions C i ;C j 2C, such that N i C = N j C = N C,ifQ i C ;Qj C and q i ; q j give these coalitions the values V i ;V j, then the value of the coalition C ij = hn C ;Q i + Q j ; q ij ;U ij C i is V ij = V i + V j. Shapley proves [18] that the four axioms above are enough for determining ' uniquely, and also gives an explicit formula for calculating '. We shall denote by R a permutation of the members of N. Itisobvious that there are n! dierent permutations. We shall denote by Ci R the group of agents that were included in R before agent A i.thus, the expression V (Ci R [fa i g), V (Ci R) is the marginal payo that agent A i brings to the coalition Ci R.Ifwe sum agent A i 's marginal payo over all possible R and calculate the mean by division by n!, then we obtain Shapley's formula: '(A i )= n!x 1 V (C R i [fa i g), V (Ci R ) R We suggest a method for using Shapley's formula: Assuming honesty, we can randomly choose an agent A r that will be responsible for calculating the Shapley values (A r may be paid for its eorts). A r may also be elected by some voting procedure [15, 16]. To be able to calculate the Shapley values, A r must rst contact all other agents and ask for all of the relevant information. Therefore it contacts n, 1 agents, which respond by transmitting their payo functions and their resource vectors. After A r has received all of the 7 For more discussion on Shapley see [13, 6, 5]. 11

13 information, it computes the Shapley values; a process which requires the computation of all 2 n possible coalitions of the agents. For each possible coalition, A r must calculate the corresponding payo. This calculation is a maximization of a function of several variables, which is rather complicated. When the computation is complete, A r once again contacts all of the agents and informs them of the results. That is, A r directs all agents how to divide the resources, tasks and extra payo among them. Complexity of the Shapley algorithm The Shapley algorithm requires o(n) communication operations to be performed in the rst stage. The number of computations for the calculation of Shapley values is dictated by 2 n, and hence the computational complexity iso(2 n ). After calculating these values, A r must contact all of the agents; this requires o(n) communication operations and the same order of computations. Altogether, the complexity of the Shapley algorithm is o(n) communication operations, and o(2 n ) computations. A new formula for the Shapley value that oers a reduction in computation time was presented by [5], but it remains of order o(2 n ). Example Recall of the last example of the three agents in a super-additive environment. Using Shapley's formula to calculate the payos of the agents in the grand coalition yields the following results: '(A1) = 7:167, '(A2) = 4:667, '(A3) = 3:167. These results are dierent from the results of the negotiation algorithm example, but the dierence is small with respect to the payos. We can indicate that in this specic case, the negotiation algorithm strengthens the stronger and weakens the weaker, while the Shapley formula tends to impose \fairness". However, in other examples the negotiation algorithm may strengthen the weaker and weaken the stronger. 3.3 Discussion Previously we presented two algorithms for payo distribution among the members of a grand coalition in a super-additive environment. If communication is expensive compared to computation and all designers agree that a \fair" payo distribution function should satisfy axioms 1 { 4, then it is worthwhile for all designers to agree, in advance, to use the Shapley algorithm. We derive the rst condition (communication is expensive compared to computation) by comparing the complexities of the two algorithms. It is obvious that the Shapley algorithm is better only when computation is cheap in comparison with communication. From the above, it follows that designers have some information about when to choose the Shapley algorithm. We must now compare it to the negotiation algorithm. Both the Shapley algorithm and the negotiation algorithm start with transmission of information; at this stage, the negotiation algorithm requires o(n 2 ) communication operations, whereas the Shapley algorithm requires only o(n) communication operations. Next, both algorithms require calculations of common payos of coalitions; each of these calculations is a maximization over 12

14 many variables, which is quite a complex calculation. In the Shapley algorithm, the common payo calculation must be done for all 2 n possible coalitions, and all of these calculations are performed by one single agent in sequence. In the negotiation algorithm, common payo calculations should be done only for coalitions that form during the negotiation process; that is, only o(n 3 ) such calculations. Moreover, these calculations are distributed among the agents. This distribution is \natural" in the sense that it is an outcome of the algorithm characteristics, i.e., each agent performs only those calculations that are required for its own actions during the process. In contrast, the Shapley algorithm does not enable distribution of the calculations, mainly because of the ultimate need for agreement upon the redistribution of resources and payo; if calculations are distributed, agreement requires negotiation. An important advantage of the negotiation process is the ability to suspend it before it ends (i.e., anytime algorithm). If this algorithm stops before completion, then there is already a coalitional conguration which is for all agents at least equal to, and probably better than, their initial condition. It is obvious that such termination of the Shapley algorithm will bring the agents to the starting point and no coalition will form. The negotiation algorithm can be performed without real negotiations, if we let one agent simulate the negotiation process and make all of the calculations. In this case the number of calculations will not change, but they will no longer be distributed. The number of communication operations will decrease to o(n), exactly as in the Shapley algorithm. The Shapley solution takes all possible negotiation scenarios into consideration while the negotiation algorithm follows only one scenario. Hence, we expect that the resulting payo distributions of the two algorithms will rarely be equal (we have explicitly shown it for n = 3). It is hard to predict which algorithm will grant more to a given agent. Therefore, we cannot give an agent a method that will enable it, for any given setting, to determine which of the two methods it should prefer. Hence, we suggest to the designers of agents to take into consideration the costs of both communication and computation, and the signicance of computation distribution. The procedures presented above were developed for super-additive environments where forming bigger coalitions is always benecial. However, in environments where there are unresolvable conicts between the agents, forming bigger coalitions may not be benecial or may even be harmful to some agents. Such environments are non-super-additive environments. The Shapley value algorithm is designed particularly for super-additive environments and cannot be adjusted to non-super-additive environments. However, the negotiation algorithm can be adjusted, by making small changes, to environments which are not super-additive environments. Note that in several such situations the grand coalition will not form. This is beyond the scope of this paper. Non-super-additive environments are discussed in [19]. 13

15 4 Conclusion This paper discusses multi-agent environments where agents are designed to reach goals that were pre-dened by their operators. An important way to execute tasks and to maximize payo is to share resources and cooperate on task execution by creating coalitions of agents. The paper discusses the advantages of two methods of coalition-formation and payo distribution in super-additive environments and suggests occasions when each is most suitable. Both algorithms ultimately lead to grand coalition formation and payo disbursements among coalition members. The algorithms are: 1. the Shapley value algorithm, which employs one representative, is computation-oriented and is best used in instances where communication is expensive. If calculations are halted in the middle, the process will not lead to any coalition formation, and; 2. the negotiation algorithm, which leads to distribution of both calculations and communications, and is primarily communication-oriented. If halted in the middle this algorithm still provides the agents with a set of formed coalitions. References 1. R. J. Aumann. The core of a cooperative game without side-payments. Transactions of the American Mathematical Society, 98:539{552, R. J. Aumann and B. Peleg. Von Neumann-Morgenstern solutions to cooperative games without side-payments. Bulletin of the American Mathematical Society, 66:173{179, C. Castelfranchi. Social power. In Y. Demazeau and J. P. Muller, editors, Decentralized A.I., pages 49{62. Elsevier Science Publishers, R. Conte, M. Miceli, and C. Castelfranchi. Limits and levels of cooperation: Disentangling various types of prosocial interaction. In Y. Demazeau and J. P. Muller, editors, Decentralized A.I. - 2, pages 147{157. Elsevier Science Publishers, G. Gambarelli. A new approach for evaluating the Shapley value. Optimization, 21(3):445{452, S. Guiasu and M. Malitza. Coalition and Connection in Games. Pergamon Press, J. C. Harsanyi. A simplied bargaining model for n-person cooperative game. International Economic Review, 4:194{220, J. C. Harsanyi. Rational Behavior and Bargaining Equilibrium in Games and Social Situations. Cambridge University Press, S. Kraus. Agents contracting tasks in non-collaborative environments. In Proc. of AAAI93, pages 243{248, France, S. Kraus and J. Wilkenfeld. Negotiations over time in a multi agent environment: Preliminary report. In Proc. of IJCAI-91, pages 56{61, Australia, T. Kreifelts and F. Von Martial. A negotiation framework for autonomous agents. In Proc. of the Second European Workshop on Modeling Autonomous Agents in a Multi Agent World, pages 169{182, France, R. D. Luce and H. Raia. Games and Decisions. John Wiley and Sons, Inc, M. Maschler and G. Owen. The consistent Shapley value for games without side payments. In R. Selten, editor, Rational Interaction, pages 5{12. Springer-Verlag,

16 14. J. Von Neumann and O. Morgenstern. Theory of Games and Economic Behavior. Princeton University Press, Princeton, N.J. 15. B. Peleg. Consistent voting systems. Econometrica, 46:153{161, B. Peleg. Game Theoretic Analysis of Voting in Committees. Cambridge University Press, Cambridge, A. Rapoport. N-Person Game Theory. University of Michigan, L. S. Shapley. A value for n-person game. In H. W. Kuhn and A. W. Tucker, editors, Contributions to the Theory of Games. Princeton University Press, O. Shehory and S. Kraus. Feasible formation of stable coalitions in general environments. Technical Report, Institute for Advanced Computer Studies, University of Maryland, K. P. Sycara. Persuasive argumentation in negotiation. Theory and Decision, 28:203{242, M. Wellman and J. Doyle. Modular utility representation for decision-theoretic planning. In Proc. of AI planning Systems, pages , Maryland, Eric Werner. Toward a theory of communication and cooperation for multiagent planning. In Proceedings of the Second Conference on Theoretical Aspects of Reasoning about Knowledge, pages 129{143, Pacic Grove, California, March G. Zlotkin and J. Rosenschein. Negotiation and task sharing among autonomous agents in cooperative domain. In Proc. of the nth International Joint Conference on Articial Intelligence, pages 912{917, Detroit, MI, G. Zlotkin and J. Rosenschein. Cooperation and conict resolution via negotiation among autonomous agents in noncooperative domains. IEEE Transactions on Systems, Man, and Cybernetics, Special Issue on Distributed Articial Intelligence, 21(6):1317{1324, December This article was processed using the LaT E X macro package with LLNCS style 15