---
title: "NEGOTIATOR: A Comprehensive Framework for Human-Agent Negotiation Integrating Preferences, Interaction, and Emotion"
authors:
  - "Mehmet Onur Keskin"
  - "Berk Buzcu"
  - "Berkecan Koçyiğit"
  - "Umut Çakan"
  - "Anıl Doğru"
  - "Reyhan Aydoğan"
doi: "10.24963/ijcai.2024/1012"
url: "https://ijcai.org/proceedings/2024/1012"
pdf: "https://ijcai.org/proceedings/2024/1012.pdf"
venue: "Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence"
year: 2024
pages: "8700-8703"
track: "Demo Track"
keywords:
  - "Agent-based and Multi-agent Systems: MAS: Human-agent interaction"
  - "Agent-based and Multi-agent Systems: MAS: Agreement technologies: Negotiation and contract-based systems"
---

# NEGOTIATOR: A Comprehensive Framework for Human-Agent Negotiation Integrating Preferences, Interaction, and Emotion

Mehmet Onur Keskin, Berk Buzcu, Berkecan Koçyiğit, Umut Çakan, Anıl Doğru, Reyhan Aydoğan

*Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence*, pages 8700-8703. DOI: [10.24963/ijcai.2024/1012](https://doi.org/10.24963/ijcai.2024/1012)

## Abstract

The paper introduces a comprehensive human-agent negotiation framework designed to facilitate the development and evaluation of research studies on human-agent negotiation without building each component from scratch. Leveraging the interoperability and reusability of its components, this framework offers various functionalities, including speech-to-text conversion, emotion recognition, a repository of negotiation strategies, and an interaction manager capable of managing gestures designed for Nao, Pepper, and QT, and coordinating message exchanges in a turn-taking fashion. This framework aims to lower the entry barrier for researchers in human-agent negotiation by providing a versatile platform that supports a wide range of research directions, including affective computing, natural language processing, decision-making, and non-verbal communication.

## Full text

## 1 Introduction

Agent-based agreement technologies have become effective tools for resolving conflicts among diverse stakeholders [Fatima et al., 2014; Marsa-Maestre et al., 2014]. While offering a versatile environment for researchers to design and assess negotiation mechanisms such as protocols and strategies, it is crucial to design and implement the software architecture in a modular fashion for seamless integration of new components while still providing a user-friendly interface. In this aspect, GENIUS (General Environment for Negotiation with Intelligent multi-purpose Usage Simulation) [Hindriks et al., 2009] and NegMAS (Negotiation MultiAgent System) [Mohammad et al., 2020] are leading automated negotiation platforms utilized in the Automated Negotiating Agents Competition(ANAC) [Jonker et al., 2017] to streamline the research in agent-based negotiation. Another simulation environment called NegoSim [Ebrahimnezhad and Fujita, 2023] extends the traditional bidding, opponent model, and acceptance strategy components with preference elicitation to facilitate the development of customized agents and protocols. All these platforms mainly focus on automated negotiations where intelligent agents negotiate on their users' behavior.

In recent years, there has been a growing demand for AI systems capable of interacting with humans, particularly in human-agent negotiation systems. Ficici et al. introduced Colored Trails (CT), a situated multiagent game environment designed to accommodate players who can be humans, agents, or a combination of both [Ficici et al., 2008]. CT offers customizability and extensibility, making it a valuable tool for the research community investigating multiagent decision-making processes. Avi et al. proposed the "NegoChat" framework, enabling negotiation between a negotiating agent and a human negotiator through bid exchanges in English [Rosenfeld et al., 2014]. Interactive Arbitration Guide Online (IAGO) introduces a Web-based platform for human-agent negotiation, which also allows exchanging arguments in natural language (e.g., preference statements) and emotional states via emojis [Mell and Gratch, 2016]. The platform enables the agent developers to change the negotiation scenarios but mainly focuses on resource allocation problems. Recognizing the significance of communication mediums in human interactions, Divekar et al. introduced a framework where a virtual agent communicates with human negotiators through speech, further enhancing the negotiation experience [Divekar et al., 2019]. All of those platforms are designed for either particular problems or settings.

While human-agent negotiation, particularly in humanrobot contexts, presents intriguing research challenges, prospective researchers are deterred by the initial setup overhead. Existing platforms mentioned earlier can be valuable if their research questions align with those platforms' capabilities. However, pursuing diverse research directions and specific needs often necessitates developing a tailored human-agent platform, which can be time-consuming. Thus, there is an urgent need for a versatile human-agent negotiation platform that supports various research directions, such as emotional awareness [Keskin et al., 2021], natural language usage [Chawla et al., 2023; Lewis et al., 2017], embodiment/appearance impact [C¸ akan et al., 2023; Keskin et al., 2024], and non-verbal communication such as facial expressions and gestures [Aydo˘gan et al., 2022].

Figure 1: NEGOTIATIOR System Architecture & Modules

![Image](https://ijcai.org/proceedings/2024/fulltext/1012/image_000000_d133a722e85b660430c3f0e04244b314abc9004c5ab8d13a47fc68f1bfbd1e43.png)

Some of these directions require functionalities like speech recognition, language processing, video processing, and emotion/gesture recognition. Addressing this gap, our framework offers a comprehensive and modular approach to developing general-purpose human-agent negotiation sessions. Researchers can leverage this tool to craft innovative negotiation/interaction strategies and evaluate them through user experiments. This platform aims to lower the barrier to entry for newcomers and streamline the development process for seasoned researchers in human-agent negotiation. This framework particularly explores how physical embodiment and behavior-based negotiation strategies, augmented by robotic gestures, can significantly impact human responses and perceptions during negotiation processes.

## [2 NEGOTIATOR: Proposed Framework](https://github.com/berkbuzcu/HumanRobotNego)

There is a growing trend for human-agent interaction research, where a user should be able to engage with a human user effectively. A human-agent interaction generally should be able to accommodate the following abilities: (i) receiving verbal/nonverbal input from the user, (ii) processing verbal/nonverbal messages, (iii) reasoning on the given input in the underlying state, (iv) generating a response, and (v) communicating with the user. Our framework aims to build the necessary components and establish their interaction smoothly so researchers do not waste time developing their framework from scratch.

## 2.1 Framework Components

Figure 1 illustrates the components of NEGOTIATOR 12 , where each box, except for the core and the configuration manager, can be theoretically unplugged or swapped easily as long as the related messages are provided to the NegoCore, which is responsible for the orchestration of the messages exchanges among components. Along with NegoCore, Config Manager is the core system component that decides the respective configuration of the negotiation (e.g., the domain information, the deadline duration) and which components should be running. We focus on bilateral negotiations where a human and an agent negotiate in a range of joint decision-making scenarios spanning from resource allocation (i.e., zero-sum games) to multi-issue negotiation scenarios (i.e., nonzero-sum games). The core first starts with health checks of each module to ensure they have started without issue, then proceeds to send the underlying negotiation information to the relevant modules. It coordinates the message exchanges between respective modules by following the interaction rules governed by the negotiation protocol [Aydo˘gan et al., 2017].

[1 NEGOTIATOR GitHub Repo: github.com/HumanRobotNego](https://github.com/berkbuzcu/HumanRobotNego)

[2 You can access the demonstration video here.](https://drive.google.com/drive/folders/11_Nil1Ea5w2_vZt35CzgBU4N0cWXN8G_?usp=sharing)

- Speech-to-Text: Through this module, the system utilizes speech recognition, transforming audible inputs received from human negotiators into textual inputs
- Text-to-Offer: By parsing the textual data and/or employing NLP techniques, this module extracts a structured offer content.
- Emotional Modelling: From facial expressions or given textual data, this component recognizes the human negotiator's emotional state and passes the recognized emotion to the Negotiation Core. Such facial emotion modeling is comprised of categorical [Li and Deng, 2019], dimensional [Barros et al., 2020], and continual models [Churamani and Gunes, 2020].
- Agent: The agent component consists of three subcomponents: Offering Strategy, Opponent Model, and Acceptance Strategy. The main agent component receives the offer, emotional feedback, and argument inputs from the Core and sends the relevant information to the subcomponents. Here, the offering strategy generates the coming offer, whereas the acceptance strategy determines whether or not to accept the human negotiator's offer. Opponent modeling is an optional component, which might maintain a model of the human negotiator's preferences or strategy. The agent transfers the agent's underlying emotional state, Mood, as well as the offer content. Note that it could also send an acceptance.
- Interaction Manager: It acquires the agent's current offer and mood to express. For instance, if the agent wishes to express the "anger" emotion, the corresponding mood of "anger" is sent to the interaction manager,

Table 1: Types of Default Negotiation Configuration Matrix

| User Input   | Speech        | Text            | Audio + Visual   |
|--------------|---------------|-----------------|------------------|
| Interaction  | GUI Only      | Robot Only      | Robot + GUI      |
| Agent        | Emotion Aware | Behavior + Time | Behavior-Based   |
| Emotion      | Categorical   | Dimensional     | Personalized     |
| Negotiation  | Zero-Sum      | Non Zero-Sum    |                  |

and the respective embodied response or textual argument is given to the human negotiator. This component currently consists of the following components: Gesture Generator in the case of an embodied agent, Argument Generator, and Text to Speech in the case of speech-based communication is desired.

- GUI: This is an optional component where the human negotiator interacts with the agent via a user interface. All inputs from the agent, such as offer sentences, remaining time, and so on, are passed to this component.

Our open-source framework provides components specified in Table 1. The researchers can use speech/text/video to communicate with the human negotiators. For instance, a humanoid robot with a camera and microphone can utilize video to receive human inputs during the negotiation. There are three negotiation strategies: emotion-aware, a mix of time and behavior strategy [Keskin et al., 2021], and behaviorbased strategy. They can make their agent negotiate over multiple issues or a resource allocation.

## 2.2 SystemDesign

We build the system as a set of modules running Docker containers and other parallelized architectures to ensure that interaction between these modules remains as independent processes, giving the agent designers full autonomy in system requirements (e.g., the user can program their agent programming language independently as long as the messages are sent accordingly to core requests). An independent message queue provider (RabbitMQ) and respective message queues are adopted for each module to communicate within the system. We define a message as M<queue, sender, context, body, status> where the body comprises module-related input and outputs according to the direction.

## 2.3 Use Cases and Applications

The NEGOTIATOR has been utilized in several research projects to date. For instance, to explore the impact of gestures, we conducted experiments where human negotiators interacted with the humanoid Nao robot, both with and without gestures [Aydo˘gan et al., 2022]. Subsequently, our research team devised an emotion-aware negotiation strategy, necessitating emotion recognition through live video processing. This strategy was integrated into the framework to investigate the role of emotions in negotiation dynamics [Keskin et al., 2021]. Additionally, we sought to examine the influence of physical embodiment in negotiation scenarios [C¸ akan et al., 2023]. We required a virtual agent with a negotiation setting identical to our Nao robot to achieve this. The interoperability feature of the NEGOTIATOR significantly shortened development time, enabling us to leverage common models across both settings. Furthermore, we collaborated with another university to study the effect of robots' appearance (e.g., NAO, QT, Pepper) in negotiation [Keskin et al., 2024]. Notably, the straightforward setup process of both experimental configurations was facilitated by the user-friendly interface of the NEGOTIATOR, as shown in Figure 2(b). Apart from the robot interface, the framework could be used for chatbased negotiation where the human interacts with the agent via the Web interface, as shown in Figure 2(a). Its versatile design allows adaptation to meet various research demands, thereby establishing The NEGOTIATOR as a valuable asset for human-agent negotiation research.

Figure 2: Gui Only & Robot + GUI Interaction Examples

![Image](https://ijcai.org/proceedings/2024/fulltext/1012/image_000001_777a37b9042c9b01850c68396a4381bdca1f909f9ee5d9e605fe20edbbca7017.png)

## 3 Conclusion and FutureWork

The insights from this research highlight the potential for developing a more sophisticated, socially aware agent capable of engaging in negotiations with humans more naturally and effectively. This work further expands the horizon of negotiation frameworks, illustrating the importance of incorporating social and psychological dimensions into collective decisionmaking, thereby enriching the dialogue on human-agent collaboration. Incorporating these insights, the NEGOTIATOR aims to synthesize advancements from diverse negotiation frameworks, including dynamic scenario modeling, negotiation strategies, emotional intelligence, non-verbal/verbal communication, and human-robot interaction dynamics. This holistic approach ensures that the NEGOTIATOR provides automated negotiations' technical and strategic demands and embraces the complexities of human-like interactions, setting a new standard for future negotiation frameworks.

## Ethical Statement

All user experiments on this framework were conducted under the approval of ¨Ozye˘gin the University Ethics Committee.

## Acknowledgments

This work is partially supported by the Horizon Europe project SmartEdge under grant agreement No. 101092908 and by the Chist-Era grant CHIST-ERA-19-XAI-005, and by the Scientific and Research Council of Turkey (T ¨UB˙ITAK, G.A. 120N680).

## References

1. [Aydogan et al., 2017] Reyhan Aydogan, David Festen, Koen V. Hindriks, and Catholijn M. Jonker. Alternating Offers Protocols for Multilateral Negotiation, pages 153- 167. Springer International Publishing, Cham, 2017.
2. [Aydogan et al., 2022] Reyhan Aydogan, Onur Keskin, and Umut C ¸akan. Would you imagine yourself negotiating with a robot, jennifer? why not? IEEE Transactions on Human-Machine Systems, 52(1):41-51, 2022.
3. [Barros et al., 2020] Pablo Barros, Nikhil Churamani, and Alessandra Sciutti. The facechannel: A fast and furious deep neural network for facial expression recognition. SN Computer Science, 1(6):321, Oct 2020.
4. [C ¸akan et al., 2023] Umut C ¸akan, M. Onur Keskin, and Rey- han Aydogan. Effects of agent's embodiment in human- agent negotiations. In Proceedings of the 23rd ACM In- ternational Conference on Intelligent Virtual Agents, IVA '23, New York, NY, USA, 2023. Association for Comput- ing Machinery.
5. [Chawla et al., 2023] Kushal Chawla, Weiyan Shi, Jingwen Zhang, Gale Lucas, Zhou Yu, and Jonathan Gratch. So- cial influence dialogue systems: A survey of datasets and models for social influence tasks. In Andreas Vlachos and Isabelle Augenstein, editors, Proceedings of the 17th Con- ference of the European Chapter of the Association for Computational Linguistics, pages 750-766, Dubrovnik, Croatia, May 2023.
6. [Churamani and Gunes, 2020] Nikhil Churamani and Hatice Gunes. Clifer: Continual learning with imagination for facial expression recognition. 05 2020.
7. [Divekar et al., 2019] Rahul R. Divekar, Jeffrey O. Kephart, Xiangyang Mou, Lisha Chen, and Hui Su. You talkin' to me? A practical attention-aware embodied agent. In Human-Computer Interaction -INTERACT, pages 760- 780, Paphos, Cyprus, 2019.
8. [Ebrahimnezhad and Fujita, 2023] Arash Ebrahimnezhad and Katsuhide Fujita. Negosim: A modular and ex- tendable automated negotiation simulation platform considering euboa. Applied Sciences, 13(1), 2023.
9. [Fatima et al., 2014] Shaheen Fatima, Sarit Kraus, and Michael Wooldridge. Principles of automated negotiation. Cambridge University Press, 2014.
10. [Ficici et al., 2008] Sevan G. Ficici, Avi Pfeffer, Ya'akov Gal, Barbara J. Grosz, and Stuart M. Shieber. Colored trails: a multiagent system testbed for decision-making re- search. In AAMAS, pages 1661-1662, Estoril, Portugal, 2008.
11. [Hindriks et al., 2009] Koen Hindriks, Catholijn M. Jonker, Sarit Kraus, Raz Lin, and Dmytro Tykhonov. Genius: ne- gotiation environment for heterogeneous agents. AAMAS '09, page 1397-1398, Richland, SC, 2009.
12. [Jonker et al., 2017] Catholijn M. Jonker, Reyhan Aydogan, Tim Baarslag, Katsuhide Fujita, Takayuki Ito, and Koen Hindriks. Automated negotiating agents competition (ANAC). In Proceedings of the Thirty-First AAAI Con- ference on Artificial Intelligence (AAAI-17), pages 5070- 5072, Melbourne, 2017. AAAI Press.
13. [Keskin et al., 2021] Mehmet Onur Keskin, Umut C ¸akan, and Reyhan Aydogan. Solver agent: Towards emotional and opponent-aware agent for human-robot negotiation. 05 2021.
14. [Keskin et al., 2024] M Onur Keskin, Selen Akay, Ayse Dogan, Berkecan Koc ¸yigit, Junko Kanero, and Reyhan Aydogan. You look nice, but i am here to negotiate: The influence of robot appearance on negotiation dynamics. In Companion of the 2024 ACM/IEEE International Confer- ence on Human-Robot Interaction (HRI '24 Companion), Boulder, CO, USA, 2024. ACM.
15. [Lewis et al., 2017] Mike Lewis, Denis Yarats, Yann Dauphin, Devi Parikh, and Dhruv Batra. Deal or no deal? end-to-end learning of negotiation dialogues. In Proceedings of the 2017 Conference on Empirical Meth- ods in Natural Language Processing, pages 2443-2453, Copenhagen, Denmark, September 2017. Association for Computational Linguistics.
16. [Li and Deng, 2019] Shan Li and Weihong Deng. Reliable crowdsourcing and deep locality-preserving learning for unconstrained facial expression recognition. IEEE Trans- actions on Image Processing, 28(1):356-370, 2019.
17. [Marsa-Maestre et al., 2014] Ivan Marsa-Maestre, Mark Klein, Catholijn M. Jonker, and Reyhan Aydogan. From problems to protocols: Towards a negotiation handbook. Decision Support Systems, 60:39 -54, 2014. Automated Negotiation Technologies and their Applications.
18. [Mell and Gratch, 2016] Johnathan Mell and Jonathan Gratch. Iago: Interactive arbitration guide online (demon- stration). In Adaptive Agents and Multi-Agent Systems, AAMAS '16, page 1510-1512, Richland, SC, 2016.
19. [Mohammad et al., 2020] Yasser Mohammad, Shinji Nakadai, and Amy Greenwald. Negmas: A platform for automated negotiations. In PRIMA 2020: Principles and Practice of Multi-Agent Systems: 23rd International Conference, page 343-351, Berlin, Heidelberg, 2020.
20. [Springer-Verlag. Rosenfeld et al., 2014] Avi Rosenfeld, Inon Zuckerman, Erel Segal, Osnat Drein, and Sarit Kraus. Negochat: a chat-based negotiation agent. In AAMAS, pages 525-532, 2014.