Deviating from Traditional Game Theory: Complex Variations in Behavior and Setting
Jun Park
A literature review submitted for Advanced Topics in Cognitive Science taught by Professor Mary Rigdon assisted by Robert Beddor
School of Arts and Sciences Rutgers University University – New New Brunswick
May 4, 2015
1
1: INTRODUCTION 1.1 Objective
Game theory has been a popular area of research due to its wide application in fields that concern social interactions between interdependent agents, and consequently has been repeatedly studied by countless economists, computer scientists, cogni cog nitive tive scientists, and military military strategists. The purpose of this paper is to gather and explore e xplore recent research on advanced models of game theory, with emphasis on complex and incomplete information information scenarios in games, methods for predicting strategies, reinforcement learning, and co st of computations and its effect on behavior. 1.2 Basic Background of Game Theory
Game theory is the analysis of interdependent relationships between co mpetitive agents and the evaluation o f strategies strategies for dealing with t hese situations. Games between agents depend heavily on how the each agent acts and reacts, and thus a decision-making algorithm that considers the potential decisions that the opponent can make is essential in solutions so lutions to a game theory problem. Also, an important aspect of game theory to note is that these agents are considered to be rational agents, which means that they have clear preferences and always aim to choose the decision that will most benefit themselves. Rational agents can be anything or anyone that can make decisions based upon the scenario and information given to them, and can be automata (machines) or humans. A classic example of a game theory problem is that of the prisoner ’s dilemma, which was first developed by Merrill Flood and Melvin Dresher in 1950 . In the scenario, two t wo suspected criminals referred to as A and B were caught for a minor offense o ffense such as vandalism and face small amount of jail time. However, the t he investigator suspects that the criminals committed other
2
crimes previously and wants to arrest them for a major offense such as robbery (assuming they did commit both crimes). The investigator tells each of the criminal of their possible options: if they both confess, they both get three years; if one confesses but the other denies, the one who confesses get one year, but the other gets seven years; if they both stay silent, they both receive two years. In here, the optimal outcome for both criminals two years if both stay silent. However, knowing each of the possible choices (and assuming these criminals are rational agents), each criminal will act according to his own self-interest. Then, given B’s possible options (confess or deny), A will try to confess every time since it works for his favor regardless of what B chooses to do: if B was to confess, A can deny (receive seven years) or confess (receive three years); if B was to deny, A can deny (receive two years) or confess (receive one year). Thus, regardless of what the other criminal chooses to do, the action that the criminal will always follow through is to confess, and will not deviate from this action in all subsequent cases since it do es not benefit them to do so. This outcome is called the Nash equilibrium, where players have noth ing to gain from deviating from their current strategy after considering the opponents’ actions. 1.3 Types of Game theory and Common Terminologies
Different types of game theory can alter the amount of information shared, the extent to which one agent can influence another, and the possible rewards or consequences from each agents’ actions. Game theory is typically split into two branches, cooperative and non-
cooperative. Cooperative games mean that the agents in the game can communicate with another, while non-cooperative games do not. The prisoner’s dilemma, as explained above, is a non-cooperative game, since they cannot communicate with each other. Other examples of games with game theoretic framework include Rock -Paper-Scissors, Ultimatum Games, Dictator Games, Peace War Games, Dollar Auction Game, and Trust Games, to name a few.
3
Another important game type to note is a negotiation game. Negotiation is a game in which each player tries to come up with a mutually beneficial solution, but at the same time acting as a rational agent (thus attempts to exploit the other agents). There are terms in negotiations that are frequently used in the sources within this paper, which will be briefly defined. An agent’s preference, regarding economics, is how an agent values each alternative or
outcome based on the amount of utility the outcome provides or how much the agent feel like they are being rewarded with what they want (e.g., happiness, satisfaction). Utility is how useful the outcome of the negotiation will be to the agent. In economics, it is equivalent to how much people would pay for a company’s product. Test domain consists of two parts, as defined by Chen and Weiss (2015, p. 2295): competitiveness and domain size. Competitiveness describes the distances between the issues that each agents have, meaning, the greater the competitiveness, the harder to reach an agreement. Domain size refer to the number of possible agreements, and as such, the efficiency of negotiation tactics is crucial if the agent wants to resolve multiple issues. Discounting factor is the factor in which the value o f the potential profit in a negotiation decreases as time passes. The discounting factor starts at 1 and goes to 0, and can only decrease the maximum utility that can be gained from the negotiation. A typical explanation of the discounting factor is that “a dollar today is worth more t han a dollar tomorrow.” Discounting
factor becomes irrelevant for issues that does not concern the future (factor stays at 1 for the whole negotiation). Concession (or giving into an agreement) in co mplex negotiations is strictly dependent on the type of agreement and the amount of time that has elapsed since the negotiation began. As
4
explained by Williams, Robu, Gerding, and Jennings (2011, p.432), “there exists a trade-off
between conceding quickly, thereby reaching an agreement early with a low utility, versus a more patient approach, that concedes slowly, therefore reaching an agreement later, but potentially with a higher utility.” As such, discounting factors play a significant role in influencing the agents’ behavior, depending o n their intention for higher or lower utility
outcome. 1.4 Evaluation of Efficiency of Proposed Strategies, Mechanisms, and Algorithms
Since the introduction of Generic Environment for Negotiation with Intelligent multi purpose Usage Simulation (GENIUS) by Hindriks et al. (2009), almost all new negotiation studies evaluated their agents’ effectiveness through their program. With GENIUS,
experimenters can configure negotiation domains, preference profiles, agent (both human a nd automata), and scenarios and provides analytic feedback such as Nash equilibriums, visualization of data and optimal solutions. The purpose of GENIUS is to facilitate research of game theory applications in complex negotiations and practical environments.
5
2: RISING RESEARCH TOPICS 2.1 Model Prediction with Incomplete Information
Previous research of game theory applications consists mostly of simple models or scenarios with predetermined information and preferences of opponent agents in restricted environments. Since applying game theory to dynamic and interacting games can be very difficult, researchers often used to over-simplify the games’ conditions and environment to gather data on the effectiveness of game theory. However, these environments are seldom encountered in real-world situations, and “real-world decision-makers mistrust these oversimplifications and thus lack confidence in any presented results” (Collins and Thomas, 2012).
Thus, recent studies have moved on to focusing on experiments with highly complex scenarios to mimic behavior and to gather experimental data of applied game theory in realistic environments. As defined by Chen and Weiss (2015, p. 2287), a complex multi-agent system has three assumptions made about the setting. First, agents have no prior knowledge or information about their opponents such as their preferences or strategies, and thus cannot use external sources to configure or adjust their strategies. Second, the negotiation is restricted with time and the agents do not have information on when the negotiation will end. Thus the agents must consider their remaining chances to offer exchanges and also realize the fact that the profit or reward through agreement of the negotiation decreases as t ime passes by a discounting factor. Third, each agent has its own private reservation value in which an o ffer below the threshold will not be accepted, and can at least achieve that amount by ending the negotiation before the timediscounting effect diminishes the profit to the reservation value. T hese assumptions create complicated interaction models between the agents, but also serve to create more accurate datasets for realistic situations.
6
The study of complex and multi-agent negotiations is becoming a growing area for research in game theory since the inception of negotiation tactics (i.e., Raiffa, 1982). As such, a difficult problem that frequently arises when facing complex scenarios is how to compensate for the lack of prior knowledge of the opponent agent in choosing one’s strategy. Without knowing the opponents’ strategies, preferences, or reservation values, an agent cannot rely on a dominant
strategy to gain advantages and instead must present an adaptive behavior throughout the game. Consequently, recent research have been focused on finding methods to model the opponent’s behavior within the negotiation itself, along with its preferences and reservation
values. For instance, Lin, Kraus, Wilkenfield, and Barry (2008) proposed an automated negotiating agent that can achieve significantly better results in agreements and in its utility capabilities than a human agent playing that role. Lin et al. (2008) based the agent’s model on Bayesian learning algorithms to refine a reasoning model based o n a belief and decision updating mechanism. For the purpose of this paper, the algorithm will be explained in a grossly simplified perspective. The main algorithm for the agent is split into two major components – “the decision making valuation component” and the “Bayesian updating rule component” (Lin et al., 2008, p.
827). The former inputs the agent’s utility function, i.e., a function to calculate utility or potential, as well as a formed conjecture of the type of opponent that the agent is facing. Both data are used to generate or accept an offer, based on how the inputs compare to a set of probabilities describing if the rival will accept them, or how the potential reward and loss relate to the reservation value. The ladder component updates the agent’s beliefs about the opponent as well as itself based on Bayes’ theorem of conditional probabilities. The two combine to create a working design mechanism that shows consistently successful results. Lin et al. (2008) experimented their mechanism in three experiments in which they matched their automated agent
7
against humans, a different automated agent following an equilibrium strategy, and itself. The results demonstrate a superior performance of the agent against humans in any given scenario in relation to their utility values, and thus the researchers claimed to have succeeded in developing an agent that can negotiate efficiently with humans with incomplete information. Similar line of research have been taken by others to develop agent mechanisms that can learn the opponent’s strategy in complex negotiations. Williams, Robu, Gerding, and Jennings (2011) aimed to develop a profitable concession strategy using Gaussian processes to find optimal expected utility in a complex (as defined above) and realistic setting. The algorithm for the strategy is separated into three stages – 1) predicting the opponent’s concession, 2) setting the concession rate for optimal expected utility, and 3) selecting an offer to generate or choose (Williams et al., 2011, p. 434). The estimation process for the opponent’s concession or expected utility is handled by a Gaussian process with covariance and mean functions, which provide predictions and a measure of confidence of that prediction within real-time time constraints. The process is evaluated for each particular time window of some duration to reduce both the effect of noise and amount of input data (Williams et al., 2011, p.434). After the prediction is generated, the strategy utilizes Gaussian processes to set the concession rate by simulating the probability that an offer of a given utility will be accepted by the opponent. When a target utility is generated after the concession rate is set, the strategy creates a range from the target utility, for instance, [µ – 0.025, µ +0.025], and chooses a random value within the range as an offer. If the offer is not available within the range, the range o f the offer increases until an offer is accepted (Williams et al., 2011, p. 435). The results, after being compared to the performance of the negotiating agents from the 2010 Automated Negotiating Agents Competition, indicate that their new strategy is superior in terms of accuracy and e fficiency. For concrete numbers, an analysis
8
done in self-play scores show that this new strategy achieved 91.9 ± 0.7 % of the maximum score possible, with the next distinct score achieved by IAMhaggler2010 (previous leading strategy developed by Williams et al. in 2010) with 71.1 ± 3.4 % of the maximum score (Williams et al., 2011, p. 437). However, in these previous research attempts to solve complex negotiation scenarios, there are cases where inacceptable assumptions were made which may have overestimated the agents’ true effectiveness in real-life negotiations. In their most recent work, Chen and Weiss
(2015) pointed out two issues that were not fully addressed in previous studies. One issue was that the way that the agents learn their opponents’ strategies is flawed, and the other issue was that there is no effective existing decision-making algorithm for complex negotiations to concede to the opponent’s offer s. Previous researchers undermined or overlooked the first issue as their
methods of modeling was computationally cost-ineffective (which will be discussed in another section) or they made impractical assumptions about the opponent before the negotiation began. For example, in the research performed by Lin et al. (2008), the proposed Bayesian learning method started with out-of-scope assumptions, as Chen and Weiss (2015, p. 2288) pointed out that the experiment “assumed that the set of possible opponent profiles is known as a pr iori.” This convenient assumption allowed comparative analysis to be made within the negotiation to determine which profile the opponent resembles, but such assumption goes against the conditions set as a complex scenario in which no prior knowledge about the opponent is brought into the negotiation. Furthermore, the method fails to compensate for profiles that are not contained within the premade set, and thus only produces results based on the model that is the most similar to one of the set profiles. However, for realistic applications, the model proposed by Lin et al. (2008) can be deemed appropriate, since a human agent could have prior experience
9
with other agents and have learned their possible profiles. Hao et al. (2014, p. 46) also pointed out that Saha et al. (2005) made their strategy under the assumption that agents negotiate over a single item, rather than multiple items like in practical negot iation settings, which diminished the complexity of the setting. In addition, in the experiment done by Williams et al. (2011), Chen and Weiss (2015) noticed that while optimizing its own expected utility based on the opponents’ strategy provides strong results, the strategy suffers from the problem of “irrational concession.” Most likely, the time window factor creates opportunity for errors, as Williams et al (2011, p. 434) stated, “small change in utility for the opponent can be observed as a large change by our agent.” Thus, the agent can misinterpret the opponent agent’s intentions (especially against sophisticated agents),
which can lead to the formation of high risk behavior that can potentially backfire on the agent by conceding too quickly. Therefore, Chen and Weiss (2015) claimed that the current methods of learning opponent models is too inflexible or inefficient, and proposed a new adaptive model called the OMAC* (opponent modeling and adaptive concession) that carefully accounts for all the aforementioned errors. The OMAC* model contains an improved version of Gaussian processing algorithm with the addition of discrete wavelet transformation (DWT) analysis for its oppo nent-modeling component. Discrete wave transformation analysis allows noise filtering and function approximation among others (Ruch and Fleet, 2009) through repeated representation of a signal and its frequency. In relevance to the OMAC*, DWT allows the agent to extract “ previous main trends of an ongoing negotiation” with smoother curves and more accurate mean approximations as time elapses (Chen and Weiss, 2015, p. 2290). Combined with Gaussian processes, DWT analysis have shown one of the highest capabilities to predict opponent’s behavior in real-time.
10
As for the OMAC*’s decision-making component, the researchers implemented an adaptive concession-making mechanism which relies on the learnt opponent model. The component handles how the agent will behave in response to counter-offers and also decides when the agent should withdraw from the negotiation. The results (Chen and Weiss, 2015, p. 2298), through experimental and empirical game theoretical analysis, suggest that the OMAC* out performs all previous agents, including IAMhaggler2011 (the new agent by Williams et al.) by a large margin – OMAC* (rank 1, normalized score = 0.667 ± 0.01) versus IAMhaggler2011 (rank 9,
normalized score = 0.511 ± 0.005) (Chen and Weiss, 2015, p. 2298). The strength of OMAC* also lies in its stable robustness, as the agent was able to perform well against in non-tournament (non-fixed) settings where agents can switch strategies in an ongoing nego tiation. As seen from the results, r esearch in the agents’ performance in complex settings is advancing and evolving at a promising rate. Future research on the topic can focus on even more efficient learning techniques, more adaptive behaviors (perhaps on par with human flexibility), and more fluent predictive abilities using Gaussian processes, DWT, and ut ility or mass functions. 2.2 Reinforcement Learning Applications in Game Theory
Reinforcement learning, or learning through a system of reward and punishment, is an effective strategy in training agents to perform repeated t asks to maximize their predictive capabilities. As research on complex scenarios arose, reinforcement- learning based models became increasingly popular due to its dynamic and adaptive ability to observe and calculate trends and offer predictions to maximize the agents’ rewards. As an example, Collins and Thomas (2012) utilized reinforcement learning to search for Nash equilibriums in games using a dynamic airline pricing game case study. The idea was that
11
through episodes, or repeated play of the game, the agents within the game can estimate and store the values of possible strategies and actions until enough data has been acquired to form a policy that models the opponent’s behavior . The case study observed three types of reinforcement learning techniques: SARSA, Q-learning, and Monte Carlo. As described by Collins and Thomas (2012, p. 1167), the differences between the three methods are in how they interpret and use data, and how they evaluate policies relevant to the scenario. SARSA and Qlearning are called bootstrapping methods since they use the current estimated data to update other estimates in a self-sustaining manner . Monte Carlo and SARSA are implemented as “on policy” methods where the current policy is evaluated, compared to Q-learning which is implemented as an “off - policy” method that does not consider the current policy when
evaluating the algorithm. All three methods use Q-value function that estimate the quality of an action a given state s, and choose the most optimal option (i.e., the one with the maximum award) for each state. This algorithm is repeated constantly, in which the old Q-value is assumed and then updated depending on new information (Wiering, 2005). An important finding from the implementation of these learning models is that after each batch was ran 10 million episodes, there appeared four pattern-like stages or phases that were consistent within all three models. First there was the random po licy stage, with expected random behavior. Then a near optimal policy stage, in which the model seemed to approach optimal values. However, the algorithm deviates from the near optimal policy and enters a myopic policy phase, where the agents began to play myopically due to either changes in learning rate or chaos within action selection mechanisms (Collins and Thomas, 2012, p. 1171). And after very large number of episode runs (~70,000 episodes) the algorithm moved towards the expect solution policy, and 9 million more episode runs confirmed the stability of phase.
12
These four stages, due to their consistency, present a new problem of finding stable trends within a reasonable amount of test cases, and thus further research should be done to ensure that the problem is existent in other experimental settings. In addition, the study showcased how reinforcement learning can be applied to solve simple systems with complex opponents, which implies that further research can be done to find non-traditional solutions for complex games. Hao et al. (2014) proposed another successful complex automated negotiation technique with non-exploitation time points and reinforcement-learning based appro ach to predict the optimal offers for the opponent. This strategy, named ABiNeS, incorporates a non-exploitation point λ that determines the degree to which the agent exploits its opponent. The strategy also
includes a reinforcement-learning algorithm to calculate the o ptimal proposal setting from the current negotiation history to create an efficient and robust agent. In detail, the ABiNeS can be viewed as four basic decision components working together (Hao et al., 2014, p. 47). The first is the acceptance-threshold (AT) component which estimates its own agent’s minimum acceptance threshold at a specific time. The threshold function considers maximum utility over the negotiation, the discounting factor, and the time passed t ∈ [0, 1]. To maximize the potential profit, the agent should try to exploit its opponent as much as possible. However, this greedy algorithm can potentially prevent any negotiations from being made, resulting in a t = 1 negotiation in which the discounting factor forces the agents to end up with zero utility. Thus, Hao et al. (2014, p. 48) introduced a non-exploitation point λ that prevents the agent from exploiting the opponent after that time point is reached to ensure maximum profit. The next-bid (NB) component determines what the ABiNeS will propose to the opponent using acceptance threshold and estimation of the opponent’s reservation value. To predict the best possible
negotiation offer for the opponent agent, the algorithm used a model-free reinforcement method
13
with the one assumption that the opponent is rational, which is true for these cases (Hao et al., 2014, p. 49). The acceptance-condition (AC) component determines if the propo sal made by the opponent is acceptable, which essentially checks if the proposed value is higher than the current acceptance threshold at time t , and if so, accepts the offer. Finally, the termination-condition (TC) component determines when to terminate the negotiation to receive the reservation value, which runs the same algorithm as the acceptance-condition except with the reservation value as comparison (Hao et al., 2014, p. 50). The experimental evaluation and analysis of ABiNeS demonstrate that the agent obtained the highest average payoff compared to other agents from ANAC 2012 and also that the agent is robust for both bilateral and tour nament negotiations (Hao et al., 2014, p. 56). Since the ABiNeS strategy lacks efficiency in modeling and estimation, the researchers suggest that incorporating such algorithms from other high performing agents can be very beneficial to its performance and stability. 2.3 Computational Cost and its Effect on Behavior
Traditional game theory does not consider computational cost as a factor of variance for the utility of each agents’ strategies. As an attempt to resolve this issue, past experimental
research split into two major approaches. Abraham Neyman (1985) initiated one line of research by treating the players in the game as finite state automata, and claiming that a Nash equilibrium can be reached with sufficiently enough games. Neyman (1985, p. 228) proved that, even though a game is non-cooperative (e.g., prisoner’s dilemma), a payoff that resembles an outcome from a cooperative game can exist depending on the method used and the amount of games played (in his work, he used Tit for Tat in the prisoner’s dilemma, in which agents copy the opponents’ previous move).
14
Ariel Rubinstein (1986) initiated the other, more prominent line of research which focuses on the effect of costly computation on an agent’s utility and performance. Rubinstein experimented with the idea that players decide which finite automata to use rather than which strategy to use, and hypothesized that the player’s performance is depended on the actions of the
chosen automaton and its complexity level. To continue such line o f research, Halpern and Pass (2015) reexamined Rubinstein’s work with a Turing machine, which has unlimited memory capacity, in place of a finite automaton. Their research showed that a Nash equilibrium does not necessarily have to exist, and a psychologically appealing explanation can be provided for its absence. The significance of recognizing the effect of costly computation can be explained through an example. Consider a scenario, as described by Halpern and Pass (2015, p. 246-247), consisting of a mathematical problem with multiple basic operations. For each problem an agent is able to solve, the agent earns a small amount of money, say $2. However, outputting an incorrect answer results in a penalty of $1000. The agent also have the ability to forfeit, and be rewarded with $1. In traditional game theory – which does not consider the cost of computation to succeed – the given solution would be to keep solving the problem to earn the maximum reward. Such behavior is determined to be the Nash equilibrium of this type of problem, despite the size and complexity of the equation. But in real-world situations, the cost of computation must be considered since it potentially can affect the agent’s utility or strategy. In this example, for a trivial mathematical problem, solving the problem by hand is reasonable and a reward can be earned. However, for more computationally unpleasant problems, most agents or players will give up, since they realize that the dollar reward from giving up o utweighs the cost of solving the problem through efficient means (e.g., with the help of powerful but expensive calculators). As
15
shown, the change in strategy due to computational cost can beco me a significant factor in deciding the agent’s strategy. If the previous researchers, such as Lin et al. (2008) and Collins
and Thomas (2012), considered cost of computation as part of their analysis, their results would be judged as an inefficient method due to the heavy processing power of Bayesian updating algorithms relative to real-time negotiations, as pointed out by Chen a nd Weiss (2015, p. 2288). To take these issues into account, Halpern and Pass (2015, p. 249) studied Bayesian machine games, which are Bayesian games (incomplete information, single-turn, information updating game) with the exception that computations are not free of cost. This cost includes the cost of run time (the time it takes for an algorithm to complete), the cost of rethinking and updating an agent’s strategy given information at a certain state, and the general complexity of
the machine. Adding the weight of cost produces interesting outcomes, one of which is that Nash equilibria become more intuitively reasonable. Since the cost prevents agents from switching strategies due to the cost of changing, the agents fall into a Nash equilibrium much quicker (Halpern and Pass, 2015, p. 251). Halpern and Pass also claims that for games that charge for randomizations (instead of it being free) charge may prevent an equilibrium from being reached (2015, p. 253). In games with mixed strategies such as Rock-Paper-Scissor, the traditional game theory equilibrium states that uniform randomization of the available cho ices (rock, paper, or scissor) as a strategy is known to be the equilibrium. An equal randomized cost to play each strategy (such as deciding to use a certain strategy depending on the combination of how two coins land) results in the existence of a “deterministic best reply that can be co mputed quickly” (Halpern and Pass, 2015, p. 252). With
these conditions, a uniform distribution of the three strategies (performed finite number o f times) can be approximated. Since the outcome relies on a systematic way of playing a certain strategy
16
depending on the cost to play (i.e., flipping a coin to choo se rather than specifically choosing rock, paper, or scissor), the traditional Nash equilibrium of randomizing the t hree strategies cannot be applied.
17
3: CONCLUSION
This paper aims to synthesize recent research in regards to progress in game theory and its applied uses in complex, incomplete scenarios as defined previously. Three major issues that frequently appears on the sources were identified and reviewed: the making of strategies and models and the impractical assumptions that often serve as their foundation; reinforcement learning based approach that significantly increases trend and offer predictions i n incomplete information games; and a computational analysis of how the cost of utilizing and updating strategies can impact the agent’s utility. All these issues tackle on the problems that researchers face when trying to incorporate game theoretic framework in realistic settings. Although previous researchers made several “incorrect” assumptions in relation to the
strict set of rules defining a complex scenario, the proposed novel strategies can be used as practical applications in specific negotiation situations. The developing methods of constructing efficient and robust algorithms, such as Gaussian processes, Bayesian updating, and discrete wavelet transformations, can be used as significant stepping ston es in the way of future technological development in artificial intelligence. Previous research have proved that reinforcement learning has great potential in its Bayesian applic ations, and several of these experiments show that the combination of these methods seem to produce high-yielding results. Though the study of computational cost is not fully recognized by all researchers due to its untraditional claims about the lack of Nash equilibriums through randomizations, the analysis provided a new insight and depth in which future researchers should consider in creating strategies that are efficient for real-time applications. Through further development of automated agent negotiations and those alike, these adaptive agents can then guide humans to better understand their own behaviors and decision-
18
making processes during negotiations or even in any applicable game. And as evaluated by this literature review, the evidence seems to point towards a positive and strong direction in the advancements of game theory and its applications.
19
Bibliography
Chen, S., & Weiss, G. (2015). An approach to complex agent-based negotiations via effectively modeling unknown opponents. Expert Systems with Applications, 2287-2304. http://www.sciencedirect.com/science/article/pii/S0957417414006800 Collins, A., & Thomas, L. (2012). Comparing reinforcement learning approaches for solving game theoretic models: A dynamic airline pricing game example. Journal of the Operational Research Society, 1165-1173.
http://www.palgravejournals.com/jors/journal/v63/n8/full/jors201194a.html Hao, J., Song, S., Leung, H., & Ming, Z. (2014). An efficient and robust negotiating strategy in bilateral negotiations over multiple items. Engineering Applications of Artificial Intelligence, 45-57.
http://www.sciencedirect.com/science/article/pii/S0952197614001067 Halpern, J., & Pass, R. (2015). Algorithmic rationality: Game theory with costly computation. Journal of Economic Theory, 246-268.
http://www.sciencedirect.com/science/article/pii/S0022053114000611 Hindriks, K., Jonker, C., Kraus, S., Lin, R., & Tykhonov, D. (2009). GENIUS – Negotiation Environment for Heterogeneous Agents. Proc. Of 8th Int. Conf. on Autonomous Agents and Multiagent Systems, 1397-1398.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.149.7751&rep=rep1&type=pdf Lin, R., Kraus, S., Wilkenfeld, J., & Barry, J. (2008). Negotiating with bounded rational agents in environments with incomplete information using an automated agent. Artificial Intelligence, 823-851.
http://www.sciencedirect.com/science/article/pii/S0004370207001518
20
Neyman, A. (1985). Bounded complexity justifies cooperation in finitely repeated prisoner’s dilemma. Economics Letters 19, 227-229. http://www.sciencedirect.com/science/article/pii/0165176585900266# Raiffa, H. (1982). The art and science of negotiation. Cambridge, Mass.: Belknap Press of Harvard University Press. Rubinstein, A. (1986). Finite Automata play the repeated prisoner’s dilemma. Journal of Economic Theory, 83-96 . http://ac.els-cdn.com/0022053186900219/1-s2.0-
0022053186900219-main.pdf?_tid=9dcd94de-f23e-11e4-911800000aab0f02&acdnat=1430731327_0d34e38015f429f4438c8f979a631f97 Ruch, D., & Fleet, P. (2011). Wavelet Theory an Elementary Approach with Applications. Chicester: Wiley.
Saha, S., Biswas, A., & Sen, S. Modeling opponent decision in repeated one-shot negotiations. AAMAS’05,
397-403. http://delivery.acm.org/10.1145/1090000/1082534/p397-
saha.pdf?ip=128.6.36.193&id=1082534&acc=ACTIVE%20SERVICE&key=777711629 8C9657D%2E56792D02161BECDE%2E4D4702B0C3E38B35%2E4D4702B0C3E38B3 5&CFID=669812475&CFTOKEN=45695084&__acm__=1430743135_59b6f21fd48591 8402fadf4475c79278 Williams, C., Robu, V., Gerding, E., & Jennings, N. (2011). Using Gaussian Processes to Optimise Concession in Complex Negotiations against Unknown Opponents. Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence, 432-438.
http://ijcai.org/papers11/Papers/IJCAI11-080.pdf
21
Wiering, M. A. (2005). QV(λ)-learning: A New On-policy Reinforcement Learning Algorithm. Proceedings of the 7th European Workshop on Reinforcement Learning, 17-18.
https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact =8&ved=0CB8QFjAA&url=http%3A%2F%2Fwww.researchgate.net%2Fprofile%2FMa rco_Wiering%2Fpublication%2F228339324_QV_%2528%2529-learning_A_new_on policy_reinforcement_learning_algorithm%2Flinks%2F02e7e51948462777e9000000.pd f &ei=lWdHVcLDEYjYggTR84HADQ&usg=AFQjCNE25WamNDJuJthd65X528RibBdz _w&sig2=xoyoGxPccP0c2ndwPB3DPQ&bvm=bv.92291466,d.eXY
22
