{"type":"ScholarlyArticle","title":"ComVas: Contextual Moral Values Alignment System","authors":[{"name":"Inkit Padhi","slug":"inkit-padhi","given":"Inkit","family":"Padhi","affiliation":"IBM Research"},{"name":"Pierre Dognin","slug":"pierre-dognin","given":"Pierre","family":"Dognin","affiliation":"IBM Research"},{"name":"Jesus Rios","slug":"jesus-rios","given":"Jesus","family":"Rios","affiliation":"IBM Research"},{"name":"Ronny Luss","slug":"ronny-luss","given":"Ronny","family":"Luss","affiliation":"IBM Research"},{"name":"Swapnaja Achintalwar","slug":"swapnaja-achintalwar","given":"Swapnaja","family":"Achintalwar","affiliation":"IBM Research"},{"name":"Matthew Riemer","slug":"matthew-riemer","given":"Matthew","family":"Riemer","affiliation":"IBM Research"},{"name":"Miao Liu","slug":"miao-liu","given":"Miao","family":"Liu","affiliation":"IBM Research"},{"name":"Prasanna Sattigeri","slug":"prasanna-sattigeri","given":"Prasanna","family":"Sattigeri","affiliation":"IBM Research"},{"name":"Manish Nagireddy","slug":"manish-nagireddy","given":"Manish","family":"Nagireddy","affiliation":"IBM Research"},{"name":"Kush R. Varshney","slug":"kush-r-varshney","given":"Kush R.","family":"Varshney","affiliation":"IBM Research"},{"name":"Djallel Bouneffouf","slug":"djallel-bouneffouf","given":"Djallel","family":"Bouneffouf","affiliation":"IBM Research"}],"abstract":"In contemporary society, the integration of artificial intelligence (AI) systems into various aspects of daily life raises significant ethical concerns. One critical aspect is to ensure that AI systems align with the moral values of the endusers. To that end, we introduce the Contextual Moral Value Alignment System, ComVas. Unlike traditional AI systems which have moral values predefined, ComVas empowers users to dynamically select and customize the desired moral values thereby guiding the system’s decision-making process. Through a user-friendly interface, individuals can specify their preferred morals, allowing the system to steer the model’s responses and actions accordingly. ComVas utilizes advanced natural language processing techniques to engage with the users in a meaningful dialogue, understanding their preferences, and reasoning about moral dilemmas in diverse contexts. This demo article showcases the functionality of ComVas, illustrating its potential to foster ethical decision-making in AI systems while respecting individual autonomy and promoting user-centric design principles.","doi":"10.24963/ijcai.2024/1026","url":"https://ijcai.org/proceedings/2024/1026","pdf":"https://ijcai.org/proceedings/2024/1026.pdf","markdown":"https://ijcai.org/proceedings/2024/1026.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/1026","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Demo Track","track":"Demo Track","number":1026,"pages":{"first":8759,"last":8762},"keywords":["Natural Language Processing: NLP: Language models","AI Ethics, Trust, Fairness: ETF: Moral decision making"],"videos":[],"references":[{"position":1,"raw":"[Abid et al., 2019] Abubakar Abid, Ali Abdalla, Ali Abid, Dawood Khan, Abdulrahman Alfozan, and James Zou. Gradio: Hassle-free sharing and testing of ml models in the wild, 2019.","title":"Gradio: Hassle-free sharing and testing of ml models in the wild","authors":"Abubakar Abid, Ali Abdalla, Ali Abid, Dawood Khan, Abdulrahman Alfozan, James Zou","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Brown et al., 2021] Daniel S. Brown, Jordan Schneider, Anca Dragan, and Scott Niekum. Value alignment ver- ification. In Proceedings of the 38th International Con- ference on Machine Learning (ICML 2021), pages 1105- 1115, 2021.","title":"Value alignment verification","authors":"S Daniel, Jordan Brown, Anca Schneider, Scott Dragan, Niekum","year":"2021","venue":"Proceedings of the 38th International Conference on Machine Learning (ICML 2021)","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Forbes et al., 2020] Maxwell Forbes, Jena D. Hwang, Vered Shwartz, Maarten Sap, and Yejin Choi. Social chem- istry 101: Learning to reason about social and moral norms. In Proceedings of the 2020 Conference on Empir- ical Methods in Natural Language Processing (EMNLP), pages 653-670, 2020.","title":"Social chemistry 101: Learning to reason about social and moral norms","authors":"Maxwell Forbes, Jena D Hwang, Vered Shwartz, Maarten Sap, Yejin Choi","year":"2020","venue":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Gabriel, 2020] I. Gabriel. Artificial intelligence, values, and alignment. In Minds & Machines, volume 30, pages 411- 437, 2020.","title":"Artificial intelligence, values, and alignment","authors":"I Gabriel","year":"2020","venue":"Minds & Machines","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Hendrycks et al., 2020] Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. Aligning ai with shared human values. arXiv preprint arXiv:2008.02275, 2020.","title":"Aligning ai with shared human values","authors":"Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, Jacob Steinhardt","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Ji et al., 2023] Jiaming Ji, Tianyi Qiu, Boyuan Chen, Borong Zhang, Hantao Lou, Kaile Wang, Yawen Duan, Zhonghao He, Jiayi Zhou, Zhaowei Zhang, et al. Ai alignment: A comprehensive survey. arXiv preprint arXiv:2310.19852, 2023.","title":"Ai alignment: A comprehensive survey","authors":"Jiaming Ji, Tianyi Qiu, Boyuan Chen, Borong Zhang, Hantao Lou, Kaile Wang, Yawen Duan, Zhonghao He, Jiayi Zhou, Zhaowei Zhang","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Köpf et al., 2023] Andreas Köpf, Yannic Kilcher, Dim- itri von Rütte, Sotiris Anagnostidis, Zhi-Rui Tam, Keith Stevens, Abdullah Barhoum, Nguyen Minh Duc, Oliver Stanley, Richárd Nagyfi, et al. Openassistant conversations-democratizing large language model align- ment. arXiv preprint arXiv:2304.07327, 2023.","title":"Openassistant conversations-democratizing large language model alignment","authors":"Andreas Köpf, Yannic Kilcher, Sotiris Dimitri Von Rütte, Zhi-Rui Anagnostidis, Keith Tam, Abdullah Stevens, Barhoum, Minh Nguyen, Oliver Duc, Richárd Stanley, Nagyfi","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lin, 2004] Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81, 2004.","title":"Rouge: A package for automatic evaluation of summaries","authors":"Chin-Yew Lin","year":"2004","venue":"Text summarization branches out","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Rodriguez-Soto et al., 2024] Manel Rodriguez-Soto, Nar- dine Osman, Carles Sierra, Paula Sánchez Veja, Ro- cio Cintas Garcia, Cristina Farriols Danes, Montser- rat Garcia Retortillo, and Silvia Minguez Maso. Towards value awareness in the medical field. In 16th International Conference on Agents and Artificial Intelligence-ICAART, volume 2024, 2024.","title":"Towards value awareness in the medical field","authors":"Manel Rodriguez-Soto, Nardine Osman, Carles Sierra, Paula Sánchez Veja, Rocio Cintas Garcia, Cristina Farriols Danes, Montserrat Garcia Retortillo, Silvia Minguez, Maso","year":"2024","venue":"16th International Conference on Agents and Artificial Intelligence-ICAART","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Sun et al., 2023] Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Cox, Yiming Yang, and Chuang Gan. Principle-driven self- alignment of language models from scratch with minimal human supervision. In Thirty-seventh Conference on Neu- ral Information Processing Systems, 2023.","title":"Principle-driven selfalignment of language models from scratch with minimal human supervision","authors":"Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Cox, Yiming Yang, Chuang Gan","year":"2023","venue":"Thirty-seventh Conference on Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Sun et al., 2024] Lichao Sun, Yue Huang, Haoran Wang, Siyuan Wu, Qihui Zhang, Chujie Gao, Yixin Huang, Wen- han Lyu, Yixuan Zhang, Xiner Li, et al. Trustllm: Trust- worthiness in large language models. arXiv preprint arXiv:2401.05561, 2024.","title":"Trustllm: Trustworthiness in large language models","authors":"Lichao Sun, Yue Huang, Haoran Wang, Siyuan Wu, Qihui Zhang, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[von Werra et al., 2020] Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, and Shengyi Huang. Trl: Transformer reinforcement learning. https://github.com/huggingface/trl, 2020.","title":"Trl: Transformer reinforcement learning","authors":"Von Werra","year":"2020","venue":null,"doi":null,"url":"https://github.com/huggingface/trl,2020","internal_url":null},{"position":13,"raw":"[Ziems et al., 2022] Caleb Ziems, Jane Yu, Yi-Chia Wang, Alon Halevy, and Diyi Yang. The moral integrity corpus: A benchmark for ethical dialogue systems. In Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pages 3755- 3773, 2022.","title":"The moral integrity corpus: A benchmark for ethical dialogue systems","authors":"Caleb Ziems, Jane Yu, Yi-Chia Wang, Alon Halevy, Diyi Yang","year":"2022","venue":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}