{"type":"ScholarlyArticle","title":"Transfer Learning Based Adaptive Automated Negotiating Agent Framework","authors":[{"name":"Ayan Sengupta","slug":"ayan-sengupta","given":"Ayan","family":"Sengupta","affiliation":"NEC Corporation , Japan"},{"name":"Shinji Nakadai","slug":"shinji-nakadai","given":"Shinji","family":"Nakadai","affiliation":"NEC Corporation , Japan; National Institute of Advanced Industrial Science and Technology (AIST) , Japan"},{"name":"Yasser Mohammad","slug":"yasser-mohammad","given":"Yasser","family":"Mohammad","affiliation":"NEC Corporation , Japan; National Institute of Advanced Industrial Science and Technology (AIST) , Japan; Assiut University , Egypt"}],"abstract":"With the availability of domain specific historical negotiation data,  the practical applications of machine learning techniques can prove to be increasingly effective in the field of automated negotiation. Yet a large portion of the literature focuses on domain independent negotiation and thus passes the possibility of leveraging any domain specific insights from historical data. Moreover, during sequential negotiation, utility functions may alter due to various reasons including market demand, partner agreements, weather conditions, etc. This poses a unique set of challenges and one can easily infer that one strategy that fits all is rather impossible in such scenarios. In this work, we present a simple yet effective method of learning an end-to-end negotiation strategy from historical negotiation data.  Next, we show that transfer learning based solutions are effective in designing adaptive strategies when underlying utility functions of agents change. Additionally, we also propose an online method of detecting and measuring such changes in the utility functions. Combining all three contributions we propose an adaptive automated negotiating agent framework that enables the automatic creation of transfer learning based negotiating agents capable of adapting to changes in utility functions. Finally, we present the results of an agent generated using our framework in different ANAC domains with 100 different utility functions each and show that our agent outperforms the benchmark score by domain independent agents by 6%.","doi":"10.24963/ijcai.2022/67","url":"https://ijcai.org/proceedings/2022/67","pdf":"https://ijcai.org/proceedings/2022/0067.pdf","markdown":"https://ijcai.org/proceedings/2022/67.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/67","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Agent-based and Multi-agent Systems","track":"Main Track","number":67,"pages":{"first":468,"last":474},"keywords":["Agent-based and Multi-agent Systems: Agreement Technologies: Negotiation and Contract-Based Systems","Agent-based and Multi-agent Systems: Economic Paradigms, Auctions and Market-Based Systems"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38986749","length_sec":87,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38984892","length_sec":262,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Abadi et al., 2015] Martín Abadi, Ashish Agarwal, Paul Barham, and ET al. TensorFlow: Large-scale ma- chine learning on heterogeneous systems. https://www. tensorflow.org/, 2015. Accessed: 2022-05-17.","title":"TensorFlow: Large-scale ma-chine learning on heterogeneous systems","authors":"Martín Abadi, Ashish Agarwal, Paul Barham","year":"2022","venue":null,"doi":null,"url":"https://www.tensorflow.org/,2015","internal_url":null},{"position":2,"raw":"[Aydogan et al., 2017] Reyhan Aydogan, David Festen, Koen V Hindriks, and Catholijn M Jonker. Alternating offers protocols for multilateral negotiation. In Modern Approaches to Agent-based Complex Automated Negotia- tion, pages 153-167. Springer, Berlin, Germany, 2017.","title":"Alternating offers protocols for multilateral negotiation","authors":"Reyhan Aydogan, David Festen, Koen V Hindriks, Catholijn M Jonker","year":"2017","venue":"Modern Approaches to Agent-based Complex Automated Negotiation","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Cao et al., 2015] Mukun Cao, Xudong Luo, Xin Robert Luo, and Xiaopei Dai. Automated negotiation for e- commerce decision making: a goal deliberated agent ar- chitecture for multi-strategy selection. Decision Support Systems, 73:1-14, 2015.","title":"Automated negotiation for ecommerce decision making: a goal deliberated agent architecture for multi-strategy selection","authors":"Cao","year":"2015","venue":"Decision Support Systems","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Chang, 2021] Ho-Chun Herbert Chang. Multi-issue negoti- ation with deep reinforcement learning. Knowledge-Based Systems, 211:106544, 2021.","title":"Multi-issue negotiation with deep reinforcement learning","authors":"Chun Ho, Chang Herbert","year":"2021","venue":"Knowledge-Based Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Chater et al., 2018] Nick Chater, Jennifer Misyak, Derrick Watson, Nathan Griffiths, and Alex Mouzakitis. Nego- tiating the traffic: Can cognitive science help make au- tonomous vehicles a reality? Trends in Cognitive Sciences, 22(2):93-95, 2018.","title":"Negotiating the traffic: Can cognitive science help make autonomous vehicles a reality","authors":"Nick Chater, Jennifer Misyak, Derrick Watson, Nathan Griffiths, Alex Mouzakitis","year":"2018","venue":"Trends in Cognitive Sciences","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen and Guestrin, 2016] Tianqi Chen and Carlos Guestrin. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16, pages 785-794, New York, NY, USA, 2016. ACM.","title":"XGBoost: A scalable tree boosting system","authors":"Tianqi Chen, Carlos Guestrin","year":"2016","venue":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2012] Siqi Chen, Haitham Bou Ammar, Karl Tuyls, and Gerhard Weiss. Transfer learning for bilat- eral multi-issue negotiation. In Proceedings of the 24th Benelux Conference on Artificial Intelligence (BNAIC), pages 59-66, 2012.","title":"Transfer learning for bilateral multi-issue negotiation","authors":"Siqi Chen, Haitham Bou Ammar, Karl Tuyls, Gerhard Weiss","year":"2012","venue":"Proceedings of the 24th Benelux Conference on Artificial Intelligence (BNAIC)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chen et al., 2016] Siqi Chen, Shuang Zhou, Gerhard Weiss, and Karl Tuyls. Using transfer learning to model unknown opponents in automated negotiations. In Recent Advances in Agent-based Complex Automated Negotiation, pages 175-192. Springer, 2016.","title":"Using transfer learning to model unknown opponents in automated negotiations","authors":"Siqi Chen, Shuang Zhou, Gerhard Weiss, Karl Tuyls","year":"2016","venue":"Recent Advances in Agent-based Complex Automated Negotiation","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Etukudor et al., 2020] Christie Etukudor, Benoit Couraud, Valentin Robu, Wolf-Gerrit Früh, David Flynn, and Chi- nonso Okereke. Automated negotiation for peer-to-peer electricity trading in local energy markets. Energies, 13(4):920, 2020.","title":"Automated negotiation for peer-to-peer electricity trading in local energy markets","authors":"Christie Etukudor, Benoit Couraud, Valentin Robu, Wolf-Gerrit Früh, David Flynn, Chinonso Okereke","year":"2020","venue":"Energies","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fiedler and Sackmann, 2021] Alexandra Fiedler and Dirk Sackmann. Automated negotiation for supply chain fi- nance. In Martijn Mes, Eduardo Lalla-Ruiz, and Ste- fan Voß, editors, Computational Logistics, pages 130-141, Cham, 2021. Springer International Publishing.","title":"Automated negotiation for supply chain finance","authors":"Sackmann Fiedler, Alexandra Fiedler, Dirk Sackmann","year":"2021","venue":"Computational Logistics","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Goodfellow et al., 2016] Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016.","title":"Deep Learning","authors":"Ian Goodfellow, Yoshua Bengio, Aaron Courville","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hochreiter and Schmidhuber, 1997] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735-1780, 1997.","title":"Long short-term memory","authors":"Schmidhuber Hochreiter, Sepp Hochreiter, Jürgen Schmidhuber","year":"1997","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Krainin et al., 2007] Michael Krainin, Bo An, and Victor Lesser. An application of automated negotiation to dis- tributed task allocation. In 2007 IEEE/WIC/ACM In- ternational Conference on Intelligent Agent Technology (IAT'07), pages 138-145. IEEE, 2007.","title":"An application of automated negotiation to distributed task allocation","authors":"Bo Michael Krainin, Victor An, Lesser","year":"2007","venue":"IEEE/WIC/ACM International Conference on Intelligent Agent Technology (IAT'07)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Kulis et al., 2011] Brian Kulis, Kate Saenko, and Trevor Darrell. What you saw is not what you get: Domain adap- tation using asymmetric kernel transforms. In CVPR 2011, pages 1785-1792. IEEE, 2011.","title":"What you saw is not what you get: Domain adaptation using asymmetric kernel transforms","authors":"Kulis","year":"2011","venue":"CVPR 2011","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Levenshtein and others, 1966] Vladimir I Levenshtein et al. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady, volume 10, pages 707-710. Soviet Union, 1966.","title":"Binary codes capable of correcting deletions, insertions, and reversals","authors":"Vladimir I Levenshtein","year":"1966","venue":"Soviet physics doklady","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Lewis et al., 2017] Mike Lewis, Denis Yarats, Yann N Dauphin, Devi Parikh, and Dhruv Batra. Deal or no deal? end-to-end learning for negotiation dialogues. arXiv preprint arXiv:1706.05125, 2017.","title":"Deal or no deal? end-to-end learning for negotiation dialogues","authors":"Mike Lewis, Denis Yarats, Devi Yann N Dauphin, Dhruv Parikh, Batra","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lin et al., 2014] Raz Lin, Sarit Kraus, Tim Baarslag, Dmytro Tykhonov, Koen Hindriks, and Catholijn M. Jonker. Genius: An integrated environment for support- ing the design of generic automated negotiators. Compu- tational Intelligence, 30(1):48-70, 2014.","title":"Genius: An integrated environment for supporting the design of generic automated negotiators","authors":"Raz Lin, Sarit Kraus, Tim Baarslag, Dmytro Tykhonov, Koen Hindriks, Catholijn M Jonker","year":"2014","venue":"Computational Intelligence","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Liu et al., 2019] Ruijun Liu, Yuqian Shi, Changjiang Ji, and Ming Jia. A survey of sentiment analysis based on transfer learning. IEEE Access, 7:85401-85412, 2019.","title":"A survey of sentiment analysis based on transfer learning","authors":"Liu","year":"2019","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Liu et al., 2020] Tingwei Liu, Zheng Zheng, et al. Negotia- tion assistant bot of pricing prediction based on machine learning. International Journal of Intelligence Science, 10(02):9, 2020.","title":"Negotiation assistant bot of pricing prediction based on machine learning","authors":"Liu","year":"2020","venue":"International Journal of Intelligence Science","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mohammad et al., 2021] Yasser Mohammad, Shinji Nakadai, and Amy Greenwald. Negmas: A platform for automated negotiations. In PRIMA 2020: Principles and Practice of Multi-Agent Systems: 23rd International Conference, Nagoya, Japan, November 18-20, 2020, Proceedings 23, pages 343-351. Springer International Publishing, 2021.","title":"Negmas: A platform for automated negotiations","authors":"Mohammad","year":"2020","venue":"PRIMA 2020: Principles and Practice of Multi-Agent Systems: 23rd International Conference","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Ramdas et al., 2015] Aaditya Ramdas, Nicolas Garcia, and Marco Cuturi. On wasserstein two sample testing and re- lated families of nonparametric tests. https://arxiv.org/abs/ 1509.02237, 2015.","title":"On wasserstein two sample testing and related families of nonparametric tests","authors":"Aaditya Ramdas, Nicolas Garcia, Marco Cuturi","year":"2015","venue":null,"doi":null,"url":"https://arxiv.org/abs/1509.02237","internal_url":null},{"position":22,"raw":"[Sengupta et al., 2021] Ayan Sengupta, Yasser Mohammad, and Shinji Nakadai. An autonomous negotiating agent framework with reinforcement learning based strategies and adaptive strategy switching mechanism. In Pro- ceedings of the 20th International Conference on Au- tonomous Agents and MultiAgent Systems, AAMAS '21, page 1163-1172, Richland, SC, 2021. International Foun- dation for Autonomous Agents and Multiagent Systems.","title":"An autonomous negotiating agent framework with reinforcement learning based strategies and adaptive strategy switching mechanism","authors":"Ayan Sengupta, Yasser Mohammad, Shinji Nakadai","year":"2021","venue":"Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS '21","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Székely, 2003] Gábor J Székely. E-statistics: The energy of statistical samples. Bowling Green State University, De- partment of Mathematics and Statistics Technical Report, 3(05):1-18, 2003.","title":"E-statistics: The energy of statistical samples","authors":"J Gábor, Székely","year":"2003","venue":"Bowling Green State University, Department of Mathematics and Statistics Technical Report","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Zhou et al., 2014] Joey Tianyi Zhou, Sinno Jialin Pan, Ivor W Tsang, and Yan Yan. Hybrid heterogeneous trans- fer learning through deep learning. In Twenty-eighth AAAI conference on artificial intelligence, 2014.","title":"Hybrid heterogeneous transfer learning through deep learning","authors":"Joey Tianyi Zhou, Sinno Jialin Pan, Ivor W Tsang, Yan Yan","year":"2014","venue":"Twenty-eighth AAAI conference on artificial intelligence","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Zhu et al., 2011] Yin Zhu, Yuqiang Chen, Zhongqi Lu, Sinno Jialin Pan, Gui-Rong Xue, Yong Yu, and Qiang Yang. Heterogeneous transfer learning for image classi- fication. In Twenty-Fifth AAAI Conference on Artificial Intelligence, 2011.","title":"Heterogeneous transfer learning for image classification","authors":"Yin Zhu, Yuqiang Chen, Zhongqi Lu, Sinno Jialin Pan, Gui-Rong Xue, Yong Yu, Qiang Yang","year":"2011","venue":"Twenty-Fifth AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}