{"type":"ScholarlyArticle","title":"Artificial Intelligence, Bias, and Ethics","authors":[{"name":"Aylin Caliskan","slug":"aylin-caliskan","given":"Aylin","family":"Caliskan","affiliation":"University of Washington"}],"abstract":"Although ChatGPT attempts to mitigate bias, when instructed to translate the gender-neutral Turkish sentences “O bir doktor. O bir hemşire” to English, the outcome is biased: “He is a doctor. She is a nurse.” In 2016, we have demonstrated that language representations trained via unsupervised learning automatically embed implicit biases documented in social cognition through the statistical regularities in language corpora. Evaluating embedding associations in language, vision, and multi-modal language-vision models reveals that large-scale sociocultural data is a source of implicit human biases regarding gender, race or ethnicity, skin color, ability, age, sexuality, religion, social class, and intersectional associations. The study of gender bias in language, vision, language-vision, and generative AI has highlighted the sexualization of women and girls in AI, while easily accessible generative AI models such as text-to-image generators amplify bias at scale. As AI increasingly automates tasks that determine life’s outcomes and opportunities, the ethics of AI bias has significant implications for human cognition, society, justice, and the future of AI. Thus, it is necessary to advance our understanding of the depth, prevalence, and complexities of bias in AI to mitigate it both in machines and society.","doi":"10.24963/ijcai.2023/799","url":"https://ijcai.org/proceedings/2023/799","pdf":"https://ijcai.org/proceedings/2023/0799.pdf","markdown":"https://ijcai.org/proceedings/2023/799.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/799","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Early Career","track":"Early Career","number":799,"pages":{"first":7007,"last":7013},"keywords":["EC: Trustworthy AI","EC: Trustworthy Machine Learning, Fairness In Machine Learning","EC: Algorithmic Fairness"],"videos":[],"references":[{"position":1,"raw":"[Bianchi et al., 2023] Federico Bianchi, Pratyusha Kalluri, Esin Durmus, Faisal Ladhak, Myra Cheng, Debora Nozza, Tatsunori Hashimoto, Dan Jurafsky, James Zou, and Aylin Caliskan. Easily accessible text-to-image generation am- plifies demographic stereotypes at large scale. Proceed- ings of the 2023 ACM Conference on Fairness, Account- ability, and Transparency (ACM FAccT), 2023.","title":"Easily accessible text-to-image generation amplifies demographic stereotypes at large scale","authors":"Federico Bianchi, Pratyusha Kalluri, Esin Durmus, Faisal Ladhak, Myra Cheng, Debora Nozza, Tatsunori Hashimoto, Dan Jurafsky, James Zou, Aylin Caliskan","year":"2023","venue":"Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Caliskan and Lewis, 2021] Aylin Caliskan and Molly Lewis. Social biases in word embeddings and their relation to human cognition. Handbook of Language Analysis in Psychology. Guilford Press, 2021.","title":"Social biases in word embeddings and their relation to human cognition","authors":"Aylin Caliskan, Molly Lewis","year":"2021","venue":"Handbook of Language Analysis in Psychology","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Caliskan and Steed, 2022] Aylin Caliskan and Ryan Steed. Managing the risks of inevitably biased visual artificial in- telligence systems. Brookings Institution, 2022.","title":"Managing the risks of inevitably biased visual artificial intelligence systems","authors":"Aylin Caliskan, Ryan Steed","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Caliskan et al., 2016a] Aylin Caliskan, Joanna Bryson, and Arvind Narayanan. A story of discrimination and unfairness: Implicit bias embedded in language models. 9th Hot Topics in Privacy Enhancing Technologies (HotPETs 2016), Accepted on May 20, 2016. https: //www.gffz.de/fileadmin/user upload/LAKOF/informatik/ A Story of Discrimination and Unfairness.pdf.","title":"A story of discrimination and unfairness: Implicit bias embedded in language models. 9th Hot Topics in Privacy Enhancing Technologies","authors":"Caliskan","year":"2016","venue":"HotPETs 2016)","doi":null,"url":"https://www.gffz.de/fileadmin/userupload/LAKOF/informatik/AStoryofDiscriminationandUnfairness.pdf","internal_url":null},{"position":5,"raw":"[Caliskan et al., 2016b] Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. Semantics derived automatically from language corpora necessarily contain human biases. arXiv preprint arXiv:1608.07187, pages 1-14, 2016.","title":"Semantics derived automatically from language corpora necessarily contain human biases","authors":"Caliskan","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Caliskan et al., 2017] Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183-186, 2017.","title":"Semantics derived automatically from language corpora contain human-like biases","authors":"Aylin Caliskan, Joanna J Bryson, Arvind Narayanan","year":"2017","venue":"Science","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Caliskan et al., 2022] Aylin Caliskan, Pimparkar Parth Ajay, Tessa Charlesworth, Robert Wolfe, and Mahzarin R. Banaji. Gender bias in word embeddings: A comprehen- sive analysis of frequency, syntax, and semantics. In Pro- ceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society (AAAI/ACM AIES), 2022.","title":"Gender bias in word embeddings: A comprehensive analysis of frequency, syntax, and semantics","authors":"Aylin Caliskan, Pimparkar Parth Ajay, Tessa Charlesworth, Robert Wolfe, Mahzarin R Banaji","year":"2022","venue":"Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society (AAAI/ACM AIES)","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Caliskan, 2021] Aylin Caliskan. Detecting and mitigating bias in natural language processing. Brookings Institution, 2021.","title":"Detecting and mitigating bias in natural language processing","authors":"Aylin Caliskan","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Charlesworth et al., 2022] Tessa Charlesworth, Aylin Caliskan, and Mahzarin R. Banaji. Historical repre- sentations of social groups across 200 years of word embeddings from google books. Proceedings of the National Academy of Sciences (PNAS), 2022.","title":"Historical representations of social groups across 200 years of word embeddings from google books","authors":"Tessa Charlesworth, Aylin Caliskan, Mahzarin R Banaji","year":"2022","venue":"Proceedings of the National Academy of Sciences","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ghosh and Caliskan, 2023] Sourojit Ghosh and Aylin Caliskan. Chatgpt perpetuates gender bias in machine translation and ignores non-gendered pronouns: Findings across bengali and five other low-resource languages. Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society (AAAI/ACM AIES), 2023.","title":"Chatgpt perpetuates gender bias in machine translation and ignores non-gendered pronouns: Findings across bengali and five other low-resource languages","authors":"Caliskan Ghosh, Sourojit Ghosh, Aylin Caliskan","year":"2023","venue":"Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society (AAAI/ACM AIES)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Greenwald and Banaji, 1995] Anthony G Greenwald and Mahzarin R Banaji. Implicit social cognition: atti- tudes, self-esteem, and stereotypes. Psychological review, 102(1):4, 1995.","title":"Implicit social cognition: attitudes, self-esteem, and stereotypes","authors":"G Anthony, Mahzarin R Greenwald, Banaji","year":"1995","venue":"Psychological review","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Greenwald et al., 1998] Anthony G Greenwald, Debbie E McGhee, and Jordan LK Schwartz. Measuring individ- ual differences in implicit cognition: the implicit associ- ation test. Journal of personality and social psychology, 74(6):1464, 1998.","title":"Measuring individual differences in implicit cognition: the implicit association test","authors":"Debbie E Anthony G Greenwald, Jordan Lk Mcghee, Schwartz","year":"1998","venue":"Journal of personality and social psychology","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Guo and Caliskan, 2021] Wei Guo and Aylin Caliskan. De- tecting emergent intersectional biases: Contextualized word embeddings contain a distribution of human-like bi- ases. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AAAI/ACM AIES), pages 122- 133, 2021.","title":"Detecting emergent intersectional biases: Contextualized word embeddings contain a distribution of human-like biases","authors":"Wei Guo, Aylin Caliskan","year":"2021","venue":"Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AAAI/ACM AIES)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Mei et al., 2023] Katelyn X. Mei, Sonia Fereidooni, and Aylin Caliskan. Bias against 93 stigmatized groups in masked language models and downstream sentiment clas- sification tasks. Proceedings of the 2023 ACM Confer- ence on Fairness, Accountability, and Transparency (ACM FAccT), 2023.","title":"Bias against 93 stigmatized groups in masked language models and downstream sentiment classification tasks","authors":"Katelyn X Mei, Sonia Fereidooni, Aylin Caliskan","year":"2023","venue":"Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Omrani Sabbaghi and Caliskan, 2022] Shiva Omrani Sab- baghi and Aylin Caliskan. Measuring gender bias in word embeddings of gendered languages requires disen- tangling grammatical gender signals. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Soci- ety (AAAI/ACM AIES), 2022.","title":"Measuring gender bias in word embeddings of gendered languages requires disentangling grammatical gender signals","authors":"Omrani Sabbaghi, Caliskan ; Shiva, Omrani Sabbaghi, Aylin Caliskan","year":"2022","venue":"Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society (AAAI/ACM AIES)","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Omrani Sabbaghi et al., 2023] Shiva Omrani Sabbaghi, Robert Wolfe, and Aylin Caliskan. Evaluating biased atti- tude associations of language models in an intersectional context. Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society (AAAI/ACM AIES), 2023.","title":"Evaluating biased attitude associations of language models in an intersectional context","authors":"Omrani Sabbaghi","year":"2023","venue":"Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society (AAAI/ACM AIES)","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Pandey and Caliskan, 2021] Akshat Pandey and Aylin Caliskan. Disparate impact of artificial intelligence bias in ridehailing economy's price discrimination algorithms. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AAAI/ACM AIES), pages 822-833, 2021.","title":"Disparate impact of artificial intelligence bias in ridehailing economy's price discrimination algorithms","authors":"Caliskan Pandey, Akshat Pandey, Aylin Caliskan","year":"2021","venue":"Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AAAI/ACM AIES)","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Steed and Caliskan, 2021a] Ryan Steed and Aylin Caliskan. Image representations learned with unsupervised pre- training contain human-like biases. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT), pages 701-713, 2021.","title":"Image representations learned with unsupervised pretraining contain human-like biases","authors":"Caliskan Steed, Ryan Steed, Aylin Caliskan","year":"2021","venue":"Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT)","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Steed and Caliskan, 2021b] Ryan Steed and Aylin Caliskan. A set of distinct facial traits learned by machines is not predictive of appearance bias in the wild. AI and Ethics, 1(3):249-260, 2021.","title":"A set of distinct facial traits learned by machines is not predictive of appearance bias in the wild","authors":"Caliskan Steed, Ryan Steed, Aylin Caliskan","year":"2021","venue":"AI and Ethics","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Toney et al., 2021] Autumn Toney, Akshat Pandey, Wei Guo, David Broniatowski, and Aylin Caliskan. Auto- matically characterizing targeted information operations through biases present in discourse on Twitter. In 2021 IEEE 15th International Conference on Semantic Comput- ing (ICSC), pages 82-83. IEEE, 2021.","title":"Automatically characterizing targeted information operations through biases present in discourse on Twitter","authors":"Toney","year":"2021","venue":"IEEE 15th International Conference on Semantic Computing (ICSC)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Toney-Wails and Caliskan, 2021] Autumn Toney-Wails and Aylin Caliskan. Valnorm quantifies semantics to reveal consistent valence biases across languages and over cen- turies. Empirical Methods in Natural Language Process- ing (EMNLP), 2021.","title":"Autumn Toney-Wails and Aylin Caliskan. Valnorm quantifies semantics to reveal consistent valence biases across languages and over centuries","authors":"Toney-Wails, Caliskan","year":"2021","venue":"Empirical Methods in Natural Language Processing (EMNLP)","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Wolfe and Caliskan, 2021] Robert Wolfe and Aylin Caliskan. Low frequency names exhibit bias and over- fitting in contextualizing language models. Proceedings of the 2021 conference on empirical methods in natural language processing (EMNLP), 2021.","title":"Low frequency names exhibit bias and over-fitting in contextualizing language models","authors":"Caliskan Wolfe, Robert Wolfe, Aylin Caliskan","year":"2021","venue":"Proceedings of the 2021 conference on empirical methods in natural language processing","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Wolfe and Caliskan, 2022a] Robert Wolfe and Aylin Caliskan. American==white in multimodal language- and-image ai. In Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society (AAAI/ACM AIES), 2022.","title":"American==white in multimodal languageand-image ai","authors":"Caliskan Wolfe, Robert Wolfe, Aylin Caliskan","year":"2022","venue":"Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society (AAAI/ACM AIES)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wolfe and Caliskan, 2022b] Robert Wolfe and Aylin Caliskan. Contrastive visual semantic pretraining mag- nifies the semantics of natural language representations. Association for Computational Linguistics (ACL), 2022.","title":"Contrastive visual semantic pretraining magnifies the semantics of natural language representations","authors":"Caliskan Wolfe, Robert Wolfe, Aylin Caliskan","year":"2022","venue":"Association for Computational Linguistics (ACL)","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Wolfe and Caliskan, 2022c] Robert Wolfe and Aylin Caliskan. Detecting emerging associations and behaviors using regional and diachronic word embeddings. In 2022 IEEE 16th International Conference on Semantic Computing (ICSC). IEEE, 2022.","title":"Detecting emerging associations and behaviors using regional and diachronic word embeddings","authors":"Caliskan Wolfe, Robert Wolfe, Aylin Caliskan","year":"2022","venue":"IEEE 16th International Conference on Semantic Computing (ICSC)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Wolfe and Caliskan, 2022d] Robert Wolfe and Aylin Caliskan. Markedness in visual semantic ai. In Pro- ceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT), 2022.","title":"Markedness in visual semantic ai","authors":"Caliskan Wolfe, Robert Wolfe, Aylin Caliskan","year":"2022","venue":"Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Wolfe and Caliskan, 2022e] Robert Wolfe and Aylin Caliskan. Vast: The valence-assessing semantics test for contextualizing language models. In Proceedings of the 36th AAAI Conference on Artificial Intelligence (AAAI), 2022.","title":"Vast: The valence-assessing semantics test for contextualizing language models","authors":"Caliskan Wolfe, Robert Wolfe, Aylin Caliskan","year":"2022","venue":"Proceedings of the 36th AAAI Conference on Artificial Intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Wolfe et al., 2022] Robert Wolfe, Mahzarin R. Banaji, and Aylin Caliskan. Evidence for hypodescent in visual se- mantic ai. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT), 2022.","title":"Evidence for hypodescent in visual semantic ai","authors":"Robert Wolfe, Mahzarin R Banaji, Aylin Caliskan","year":"2022","venue":"Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Wolfe et al., 2023] Robert Wolfe, Yiwei Yang, Bill Howe, and Aylin Caliskan. Contrastive language-vision ai models pretrained on web-scraped multimodal data exhibit sexual objectification bias. Proceedings of the 2023 ACM Confer- ence on Fairness, Accountability, and Transparency (ACM FAccT), 2023.","title":"Contrastive language-vision ai models pretrained on web-scraped multimodal data exhibit sexual objectification bias","authors":"Robert Wolfe, Yiwei Yang, Bill Howe, Aylin Caliskan","year":"2023","venue":"Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}