---
title: "Human-Like Agents for Repeated Negotiation"
authors:
  - "Johnathan Mell"
doi: "10.24963/ijcai.2017/754"
url: "https://ijcai.org/proceedings/2017/754"
pdf: "https://ijcai.org/proceedings/2017/0754.pdf"
venue: "Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence"
year: 2017
pages: "5189-5190"
track: "Doctoral Consortium"
keywords:
  - "Artificial Intelligence: computer science"
  - "Artificial Intelligence: agents and multi-agent systems"
---

# Human-Like Agents for Repeated Negotiation

Johnathan Mell

*Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence*, pages 5189-5190. DOI: [10.24963/ijcai.2017/754](https://doi.org/10.24963/ijcai.2017/754)

## Abstract

Virtual agents have been used as tools in negotiation—from acting as mediators to manifesting as full-fledged conversational partners. Virtual agents are a powerful tool for teaching negotiation skills, but require an accurate model of human behavior to perform well both as partners and teachers. The work proposed here aims to expand the current horizon of virtual negotiating agents to utilize human-like strategies. Further agents developed using this framework should be cognizant of the social factors influencing negotiation, including reputation effects and the implications of long-term repeated relationships. A roadmap of current efforts to develop agent platforms and future expansions is discussed.

## Full text

## 1 Introduction and Related Work

Virtual agents as developed for negotiation purposes are the subjects of several areas of research. The use of agents as mediators is well-established [Guttman and Maes, 1998], and robust models of agent-agent negotiation have been developed [Faratin et al., 1998]. Indeed, several tools for developing and improving agents in agent-agent competition have been developed [Hindriks et al., 2009]. These agents typically participate in games that are characterized by rapid offer exchange (sometimes hundreds of offers per second) and strict protocols (alternating offers only, e.g.). Communication of preferences is often disallowed, or at best, restricted to certain phases of the negotiation. Emotional exchange is rarely used. While these models are useful for many applications, they are poor comparisons to human negotiation. In human negotiation, other factors like trust, rapport, and emotion are impactful. Different tactics and different models are required for agents that are adequate negotiation partners for humans or (better yet) can teach humans negotiation [Core et al., 2006].

Negotiator relationships are recognized as being as important as the quality of offers exchanged. Thus, trust and deception are key tools in an effective negotiatorÕs arsenal [Olekalns and Smith, 2009]. Further, emotions can be used to affect the outcome of negotiations, particularly through the use of threats or anger [Van Kleef et al., 2004]. To this end, agents that negotiate using human-like strategies have been The next generation of virtual agents needs to be able to both reason effectively about strategy as current agents do, and also embody the strategies popular in human negotiation. These agents need to model opponent preferences, log information about players over time, and utilize emotion effectively to both perform as player partners and, eventually, as effective trainers for humans seeking negotiation skills.

## 2 Completed Work

To address the problems of developing effective, human-like negotiating agents, our work has taken a three-step approach. First, empirical work with human subjects helps to identify areas in which agents can develop more human-like qualities. For example, in previous work [Mell et al., 2015], we identified a particular strategy that is effective in human negotiation called Òfavors and ledgersÓ. This strategy allows negotiators to accept unfair outcomes in the short term with the expectation that these favors will be repaid over time, thus unlocking greater shared value (Ògrowing the pieÓ). Similar work has been conducted, focusing on trust [Fulmer and Gelfand, 2013] and social dependencies [Grosz et al., 2004].

This empirically-based first step is critical as human participants often behave similarlyÑbut not identicallyÑwith agent partners as they do with human partners. When designing agents for teaching negotiation, understanding these effects based on empirical data is thus essential. While most of these data are analyzed using traditional, offline methods, automatic, real-time analysis of negotiation performance is of particular future interest since it allows feedback to be generated in training contexts on-the-fly.

For the second step of development, a platform for developing human-like agents is required. This platform is the Interactive Arbitration Guide Online (IAGO) [Mell and Gratch,

2017]. IAGO prioritizes communication across three channels: offers, messages, and emotions. This allows agents to be developed that can communicate preferences, lie about their intentions, and intimidate opponents through anger or other emotions. Finally, IAGO allows metrics on negotiation performance to be logged for validation purposes.

## 3 Proposed Work

The final step in the furtherance of human-like negotiating agents is to actually develop these agents and subsequently validate them. Using IAGO, we will continue to develop agents that build rapport and trust, utilize deception effectively, and maintain models of opponentsÕ preferences and strategies. However, our current models of human-agent interaction are largely based on dyadic interaction. While oneto-one negotiation still presents many challenges, it also makes many simplifications to truly human-like negotiation. Humans that develop relationships in negotiation often do so over time, and choose partners based on past interactions. To this end, we propose to develop agents that are specifically targeted to perform within a marketplace of agents, and that are intended to maximize long-term value. Agents that build trust over time may be expected to succeed in the long-term over more short-sighted and greedy strategies.

Our next generation of agents also needs to have a greater awareness of social reputation in order to perform in this new marketplace. Previous work has indicated that humans may be very aware of the types of strategies and tactics that their partners (and themselves) employ [Robinson et al., 2000]. As such, overuse of tactics like misrepresentation or hardline offers may come at a cost to reputation.

As we develop these new agents, we will base our models on empirical data gathered from numerous human-human and human-agent studies. By continuing to prototype agents based on real-world data and validating them over the web via IAGO, we can be assured that our agents continue to perform as they expand into ever-more complicated domains and adopt ever-more human-like strategies and features.

## References

1. [Castelfranchi and Falcone, 1998] Cristiano Castelfranchi and Rino Falcone. Principles of trust for MAS: Cognitive anatomy, social importance, and quantification. In Proceedings of the International Conference on Multi Agent Systems, 1998., pp. 72-79. IEEE, 1998.
2. [Core et al., 2006] Mark Core et al.. Teaching negotiation skills through practice and reflection with virtual humans. Simulation 82, no. 11pp. 685-701, 2006.
3. [de Melo et al., 2011] Celso M. de Melo, Peter Carnevale, and Jonathan Gratch. The effect of expression of anger and happiness in computer agents on negotiations with humans. In The 10th International Conference on Autonomous Agents and Multiagent Systems, 2011.
4. [Faratin et al., 1998] Peyman Faratin, Carles Sierra, and Nick R. Jennings. Negotiation decision functions for autonomous agents. In Robotics and Autonomous Systems 24, no. 3-4 pp. 159-182, 1998.
5. [Fulmer and Gelfand, 2013] Ashley C. Fulmer and Michele J. Gelfand. How do I trust thee? Dynamic trust patterns and their individual and social contextual determinants. In Models for intercultural collaboration and negotiation, pp. 97-131. Springer Netherlands, 2013.
6. [Guttman and Maes, 1998] Robert H. Guttman and Pattie Maes. Agent-mediated integrative negotiation for retail electronic commerce. In International Workshop on Agent-Mediated Electronic Trading, pp. 70-90, 1998.
7. [Grosz et al., 2004] Barbara J. Grosz et al. The influence of social dependencies on decision-making: Initial investigations with a new game. In Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems-Volume 2, pp. 782-789. IEEE Computer Society, 2004.
8. [Haim et al., 2017] Galit Haim et al. Human-computer negotiation in a three player market setting. Artificial Intelligence 246 pp. 34-52, 2017.
9. [Hindriks et al., 2009] Koen Hindriks, et al. Genius: negotiation environment for heterogeneous agents. In Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems-Volume 2, 2009.
10. [Mell et al., 2015] Johnathan Mell, Gale Lucas, and Jonathan Gratch. An effective conversation tactic for creating value over repeated negotiations. In Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems, 2015.
11. [Mell and Gratch, 2017] Johnathan Mell and Jonathan Gratch. Grumpy & Pinocchio: Answering Human-Agent Negotiation Questions through Realistic Agent Design. In Proceedings of the 16th Conference on Autonomous Agents and Multiagent Systems, 2017.
12. [Olekalns and Smith, 2009] Mara Olekalns and Philip L. Smith. Mutually dependent: Power, trust, affect and the use of deception in negotiation. Journal of Business Ethics 85, no. 3 (2009): 347-365.
13. [Robinson et al., 2000] Robert J. Robinson, Roy J. Lewicki, and Eileen M. Donahue. Extending and testing a five factor model of ethical and unethical bargaining tactics: Introducing the SINS scale. Journal of Organizational Behavior (2000): 649-664.
14. [Rosenfeld et al., 2016] Avi Rosenfeld, Inon Zuckerman, Erel Segal-Halevi, Osnat Drein, and Sarit Kraus. NegoChat-A: a chat-based negotiation agent with bounded rationality. Autonomous Agents and Multi- Agent Systems 30, no. 1, pp. 60-81, 2016.
15. [Van Kleef et al., 2004] Gerben A. Van Kleef, Carsten KW De Dreu, and Antony SR Manstead. The interpersonal effects of emotions in negotiations: a motivated information processing approach. Journal of personality and social psychology 87, no. 4 p. 510, 2004.