{"type":"ScholarlyArticle","title":"An Ethical Dataset from Real-World Interactions Between Users and Large Language Models","authors":[{"name":"Masahiro Kaneko","slug":"masahiro-kaneko","given":"Masahiro","family":"Kaneko","affiliation":""},{"name":"Danushka Bollegala","slug":"danushka-bollegala","given":"Danushka","family":"Bollegala","affiliation":""},{"name":"Timothy Baldwin","slug":"timothy-baldwin","given":"Timothy","family":"Baldwin","affiliation":""}],"abstract":"Recent studies have demonstrated that Large Language Models (LLMs) have ethical-related problems such as social biases, lack of moral reasoning, and generation of offensive content.\r\n    The existing evaluation metrics and methods to address these ethical challenges use datasets intentionally created by instructing humans to create instances including ethical problems.\r\n    Therefore, the data does not sufficiently include comprehensive prompts that users actually provide when using LLM services in everyday contexts and outputs that LLMs generate.\r\n    There may be different tendencies between unethical instances intentionally created by humans and actual user interactions with LLM services, which could result in a lack of comprehensive evaluation.\r\n    To investigate the difference, we create Eagle datasets extracted from actual interactions between ChatGPT and users that exhibit social biases, opinion biases, toxicity, and immoral problems.\r\n    Our experiments show that Eagle captures complementary aspects, not covered by existing datasets proposed for evaluation and mitigation.\r\n    We argue that using both existing and proposed datasets leads to a more comprehensive assessment of the ethics.","doi":"10.24963/ijcai.2025/1082","url":"https://ijcai.org/proceedings/2025/1082","pdf":"https://ijcai.org/proceedings/2025/1082.pdf","markdown":"https://ijcai.org/proceedings/2025/1082.md","bibtex":"https://ijcai.org/proceedings/2025/bibtex/1082","volume":{"slug":"2025","title":"Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2025","url":"https://ijcai.org/proceedings/2025/","year":2025,"location":"Montreal, Canada","dates":"16-22 August 2025","editors":"James Kwok","isbn":"978-1-956792-06-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2025"},"section":"AI and Social Good","track":"AI and Social Good","number":1082,"pages":{"first":9737,"last":9745},"keywords":["AI Ethics, Trust, Fairness: General","Humans and AI: General"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}