{"type":"ScholarlyArticle","title":"Addressing Weak Decision Boundaries in Image Classification by Leveraging Web Search and Generative Models","authors":[{"name":"Preetam Prabhu Srikar Dammu","slug":"preetam-prabhu-srikar-dammu","given":"Preetam Prabhu Srikar","family":"Dammu","affiliation":"University of Washington , Seattle , WA , USA"},{"name":"Yunhe Feng","slug":"yunhe-feng","given":"Yunhe","family":"Feng","affiliation":"University of Washington , Seattle , WA , USA; University of North Texas , Denton , TX , USA"},{"name":"Chirag Shah","slug":"chirag-shah","given":"Chirag","family":"Shah","affiliation":"University of Washington , Seattle , WA , USA"}],"abstract":"Machine learning (ML) technologies are known to be riddled with ethical and operational problems, however, we are witnessing an increasing thrust by businesses to deploy them in sensitive applications. One major issue among many is that ML models do not perform equally well for underrepresented groups. This puts vulnerable populations in an even disadvantaged and unfavorable position. We propose an approach that leverages the power of web search and generative models to alleviate some of the shortcomings of discriminative models. We demonstrate our method on an image classification problem using ImageNet's People Subtree subset, and show that it is effective in enhancing robustness and mitigating bias in certain classes that represent vulnerable populations (e.g., female doctor of color). Our new method is able to (1) identify weak decision boundaries for such classes; (2) construct search queries for Google as well as text for generating images through DALL-E 2 and Stable Diffusion; and (3) show how these newly captured training samples could alleviate population bias issue. While still improving the model's overall performance considerably, we achieve a significant reduction (77.30%) in the model's gender accuracy disparity. In addition to these improvements, we observed a notable enhancement in the classifier's decision boundary, as it is characterized by fewer weakspots and an increased separation between classes. Although we showcase our method on vulnerable populations in this study, the proposed technique is extendable to a wide range of problems and domains.","doi":"10.24963/ijcai.2023/659","url":"https://ijcai.org/proceedings/2023/659","pdf":"https://ijcai.org/proceedings/2023/0659.pdf","markdown":"https://ijcai.org/proceedings/2023/659.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/659","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"AI for Good","track":"AI for Good","number":659,"pages":{"first":5941,"last":5949},"keywords":["AI for Good: AI Ethics, Trust, Fairness","AI for Good: Humans and AI","AI for Good: Machine Learning","AI for Good: Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Baeza-Yates, 2018] Ricardo Baeza-Yates. Bias on the web. Communications of the ACM, 61(6):54-61, 2018.","title":"Bias on the web","authors":"Ricardo Baeza-Yates","year":"2018","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Bernhardt et al., 2022] Mélanie Bernhardt, Charles Jones, and Ben Glocker. Potential sources of dataset bias compli- cate investigation of underdiagnosis by machine learning algorithms. Nature Medicine, 28(6):1157-1158, 2022.","title":"Potential sources of dataset bias complicate investigation of underdiagnosis by machine learning algorithms","authors":"Mélanie Bernhardt, Charles Jones, Ben Glocker","year":"2022","venue":"Nature Medicine","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Buda et al., 2018] Mateusz Buda, Atsuto Maki, and Ma- ciej A Mazurowski. A systematic study of the class im- balance problem in convolutional neural networks. Neural networks, 106:249-259, 2018.","title":"A systematic study of the class imbalance problem in convolutional neural networks","authors":"Mateusz Buda, Atsuto Maki, Maciej A Mazurowski","year":"2018","venue":"Neural networks","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Buolamwini and Gebru, 2018] Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy dispari- ties in commercial gender classification. In Conference on fairness, accountability and transparency, pages 77- 91. PMLR, 2018.","title":"Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification","authors":"Gebru Buolamwini","year":"2018","venue":"Conference on fairness, accountability and transparency","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Carlini and Wagner, 2017] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pages 39-57. Ieee, 2017.","title":"Towards evaluating the robustness of neural networks","authors":"Wagner Carlini, Nicholas Carlini, David Wagner","year":"2017","venue":"ieee symposium on security and privacy (sp)","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chapelle et al., 1999] Olivier Chapelle, Patrick Haffner, and Vladimir N Vapnik. Support vector machines for histogram-based image classification. IEEE transactions on Neural Networks, 10(5):1055-1064, 1999.","title":"Support vector machines for histogram-based image classification","authors":"Olivier Chapelle, Patrick Haffner, Vladimir N Vapnik","year":"1999","venue":"IEEE transactions on Neural Networks","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Chen et al., 2022] Yiwen Chen, Xue Li, Sheng Guo, Xian Yao Ng, and Marcelo Ang. Real2sim or sim2real: Robotics visual insertion using deep reinforcement learn- ing and real2sim policy adaptation. arXiv preprint arXiv:2206.02679, 2022.","title":"Real2sim or sim2real: Robotics visual insertion using deep reinforcement learning and real2sim policy adaptation","authors":"Yiwen Chen, Xue Li, Sheng Guo, Xian Yao Ng, Marcelo Ang","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"['Alessandro et al., 2017] Brian d'Alessandro, Cathy O'Neil, and Tom LaGatta. Conscientious classifica- tion: A data scientist's guide to discrimination-aware classification. Big data, 5(2):120-134, 2017.","title":"Conscientious classification: A data scientist's guide to discrimination-aware classification","authors":"Alessandro","year":"2017","venue":"Big data","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Dosovitskiy et al., 2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Min- derer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.","title":"An image is worth 16x16 words: Transformers for image recognition at scale","authors":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Feng and Shah, 2022] Yunhe Feng and Chirag Shah. Has ceo gender bias really been fixed? adversarial attacking and improving gender fairness in image search. In Pro- ceedings of the AAAI conference on artificial intelligence (AAAI), 2022.","title":"Has ceo gender bias really been fixed? adversarial attacking and improving gender fairness in image search","authors":"Shah Feng, Yunhe Feng, Chirag Shah","year":"2022","venue":"Proceedings of the AAAI conference on artificial intelligence (AAAI)","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Goodfellow et al., 2014] Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014.","title":"Explaining and harnessing adversarial examples","authors":"Ian J Goodfellow, Jonathon Shlens, Christian Szegedy","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770-778, 2016.","title":"Deep residual learning for image recognition","authors":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun","year":"2016","venue":"Proceedings of the IEEE conference on computer vision and pattern recognition","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[He et al., 2017] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961-2969, 2017.","title":"Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn","authors":"He","year":"2017","venue":"Proceedings of the IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hendricks et al., 2018] Lisa Anne Hendricks, Kaylee Burns, Kate Saenko, Trevor Darrell, and Anna Rohrbach. Women also snowboard: Overcoming bias in captioning models. In Proceedings of the European Conference on Computer Vision (ECCV), pages 771-787, 2018.","title":"Women also snowboard: Overcoming bias in captioning models","authors":"Anne Lisa, Kaylee Hendricks, Kate Burns, Trevor Saenko, Anna Darrell, Rohrbach","year":"2018","venue":"Proceedings of the European Conference on Computer Vision (ECCV)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hendrycks et al., 2021] Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15262-15271, 2021.","title":"Natural adversarial examples","authors":"Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, Dawn Song","year":"2021","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Hu and Li, 2019] Mengxiao Hu and Jinlong Li. Exploring bias in gan-based data augmentation for small samples. arXiv preprint arXiv:1905.08495, 2019.","title":"Exploring bias in gan-based data augmentation for small samples","authors":"Li ; Hu, Mengxiao Hu, Jinlong Li","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Iosifidis and Ntoutsi, 2018] Vasileios Iosifidis and Eirini Ntoutsi. Dealing with bias via data augmentation in super- vised learning scenarios.","title":"Vasileios Iosifidis and Eirini Ntoutsi. Dealing with bias via data augmentation in supervised learning scenarios","authors":"Ntoutsi Iosifidis","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"Jo Bates Paul D. Clough Robert Jäschke, 24, 2018.","title":null,"authors":"Jo Bates, Paul D Clough, Robert Jäschke","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Jaipuria et al., 2020] Nikita Jaipuria, Xianling Zhang, Ro- han Bhasin, Mayar Arafa, Punarjay Chakravarty, Shubham Shrivastava, Sagar Manglani, and Vidya N Murali. Deflat- ing dataset bias using synthetic data augmentation. In Pro- ceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition Workshops, pages 772-773, 2020.","title":"Deflating dataset bias using synthetic data augmentation","authors":"Nikita Jaipuria, Xianling Zhang, Rohan Bhasin, Mayar Arafa, Punarjay Chakravarty, Shubham Shrivastava, Sagar Manglani, Vidya, Murali","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Johnson et al., 2019] Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with gpus. IEEE Transactions on Big Data, 7(3):535-547, 2019.","title":"Billion-scale similarity search with gpus","authors":"Jeff Johnson, Matthijs Douze, Hervé Jégou","year":"2019","venue":"IEEE Transactions on Big Data","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Kim et al., 2021] Youmin Kim, AFM Shahab Uddin, and Sung-Ho Bae. Local augment: Utilizing local bias prop- erty of convolutional neural networks for data augmenta- tion. IEEE Access, 9:15191-15199, 2021.","title":"Local augment: Utilizing local bias property of convolutional neural networks for data augmentation","authors":"Kim","year":"2021","venue":"IEEE Access","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lam et al., 2018] Onyi Lam, Brian Broderick, Stefan Woj- cik, and Adam Hughes. Gender and jobs in online image searches. Pew Social Trends. Retrieved March, 14:2020, 2018.","title":"Gender and jobs in online image searches","authors":"Lam","year":"2018","venue":"Pew Social Trends","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Liu et al., 2008] Xu-Ying Liu, Jianxin Wu, and Zhi-Hua Zhou. Exploratory undersampling for class-imbalance learning. IEEE Transactions on Systems, Man, and Cy- bernetics, Part B (Cybernetics), 39(2):539-550, 2008.","title":"Exploratory undersampling for class-imbalance learning","authors":"Xu-Ying Liu, Jianxin Wu, Zhi-Hua Zhou","year":"2008","venue":"IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Madry et al., 2017] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083, 2017.","title":"Towards deep learning models resistant to adversarial attacks","authors":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, Adrian Vladu","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Mikołajczyk and Grochowski, 2018] Agnieszka Mikołajczyk and Michał Grochowski. Data augmentation for improving deep learning in image classification problem. In 2018 international interdisciplinary PhD workshop (IIPhDW), pages 117-122. IEEE, 2018.","title":"Data augmentation for improving deep learning in image classification problem","authors":"Grochowski Mikołajczyk, Agnieszka Mikołajczyk, Michał Grochowski","year":"2018","venue":"2018 international interdisciplinary PhD workshop (IIPhDW)","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Minot et al., 2022] Joshua R. Minot, Nicholas Cheney, Marc E. Maier, Danne C. Elbers, Christopher M. Dan- forth, and Peter Sheridan Dodds. Interpretable bias mit- igation for textual data: Reducing genderization in patient notes while maintaining classification performance. ACM Transactions on Computing for Healthcare, 2022.","title":"Interpretable bias mitigation for textual data: Reducing genderization in patient notes while maintaining classification performance","authors":"Joshua R Minot, Nicholas Cheney, Marc E Maier, Danne C Elbers, Christopher M Danforth, Peter Sheridan Dodds","year":"2022","venue":"ACM Transactions on Computing for Healthcare","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Plumb et al., 2022] Gregory Plumb, Marco Tulio Ribeiro, and Ameet Talwalkar. Finding and fixing spurious patterns with explanations. Transactions on Machine Learning Re- search, 2022.","title":"Finding and fixing spurious patterns with explanations","authors":"Gregory Plumb, Marco Tulio Ribeiro, Ameet Talwalkar","year":"2022","venue":"Transactions on Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Radford et al., 2019] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.","title":"Language models are unsupervised multitask learners","authors":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever","year":"2019","venue":"OpenAI blog","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Ramesh et al., 2021] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International Conference on Machine Learning, pages 8821-8831. PMLR, 2021.","title":"Zero-shot text-to-image generation","authors":"Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, Ilya Sutskever","year":"2021","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Rombach et al., 2022] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-10695, 2022.","title":"High-resolution image synthesis with latent diffusion models","authors":"Rombach","year":"2022","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Selvaraju et al., 2017] Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on com- puter vision, pages 618-626, 2017.","title":"Grad-cam: Visual explanations from deep networks via gradient-based localization","authors":"Michael Ramprasaath R Selvaraju, Abhishek Cogswell, Ramakrishna Das, Devi Vedantam, Dhruv Parikh, Batra","year":"2017","venue":"Proceedings of the IEEE international conference on computer vision","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Seyyed-Kalantari et al., 2021] Laleh Seyyed-Kalantari, Haoran Zhang, Matthew McDermott, Irene Y Chen, and Marzyeh Ghassemi. Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nature medicine, 27(12):2176-2182, 2021.","title":"Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations","authors":"Seyyed-Kalantari","year":"2021","venue":"Nature medicine","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Sharma et al., 2020] Shubham Sharma, Yunfeng Zhang, Jesús M Ríos Aliaga, Djallel Bouneffouf, Vinod Muthusamy, and Kush R Varshney. Data augmentation for discrimination prevention and bias disambiguation. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pages 358-364, 2020.","title":"Data augmentation for discrimination prevention and bias disambiguation","authors":"Shubham Sharma, Yunfeng Zhang, Jesús M Ríos Aliaga, Djallel Bouneffouf, Vinod Muthusamy, Kush R Varshney","year":"2020","venue":"Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Singla and Feizi, 2022] Sahil Singla and Soheil Feizi. Salient imagenet: How to discover spurious features in deep learning? In International Conference on Learning Representations, 2022.","title":"Salient imagenet: How to discover spurious features in deep learning?","authors":"Feizi Singla, Sahil Singla, Soheil Feizi","year":"2022","venue":"International Conference on Learning Representations","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Szegedy et al., 2013] Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199, 2013.","title":"Intriguing properties of neural networks","authors":"Szegedy","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Słowik and Bottou, 2021] Agnieszka Słowik and Léon Bot- tou. Algorithmic bias and data bias: Understanding the relation between distributionally robust optimization and data curation. ArXiv, abs/2106.09467, 2021.","title":"Agnieszka Słowik and Léon Bottou. Algorithmic bias and data bias: Understanding the relation between distributionally robust optimization and data curation","authors":"Bottou Słowik","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Taori et al., 2020] Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, and Ludwig Schmidt. Measuring robustness to natural distribution shifts in image classification. Advances in Neural Infor- mation Processing Systems, 33:18583-18599, 2020.","title":"Measuring robustness to natural distribution shifts in image classification","authors":"Rohan Taori, Achal Dave, Vaishaal Shankar, Nicholas Carlini, Benjamin Recht, Ludwig Schmidt","year":"2020","venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Wang and Culotta, 2021] Zhao Wang and Aron Culotta. Robustness to spurious correlations in text classification via automatically generated counterfactuals. In Proceed- ings of the AAAI Conference on Artificial Intelligence, vol- ume 35, pages 14024-14031, 2021.","title":"Robustness to spurious correlations in text classification via automatically generated counterfactuals","authors":"Wang, Culotta ; ] Zhao Wang, Aron Culotta","year":"2021","venue":"Proceedings of the AAAI Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Wang et al., 2019] Tianlu Wang, Jieyu Zhao, Mark Yatskar, Kai-Wei Chang, and Vicente Ordonez. Balanced datasets are not enough: Estimating and mitigating gender bias in deep image representations. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5310-5319, 2019.","title":"Balanced datasets are not enough: Estimating and mitigating gender bias in deep image representations","authors":"Wang","year":"2019","venue":"Proceedings of the IEEE/CVF International Conference on Computer Vision","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Wang et al., 2021a] Shuo Wang, L. Lyu, Surya Nepal, Carsten Rudolph, Marthie Grobler, and Kristen Moore. Robust training using natural transformation. ArXiv, abs/2105.04070, 2021.","title":"Robust training using natural transformation","authors":"Wang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":41,"raw":"[Wang et al., 2021b] Tianlu Wang, Diyi Yang, and Xuezhi Wang. Identifying and mitigating spurious correlations for improving robustness in nlp models. arXiv preprint arXiv:2110.07736, 2021.","title":"Identifying and mitigating spurious correlations for improving robustness in nlp models","authors":"Wang","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":42,"raw":"[Yang et al., 2020] Kaiyu Yang, Klint Qinami, Li Fei-Fei, Jia Deng, and Olga Russakovsky. Towards fairer datasets: Filtering and balancing the distribution of the people sub- tree in the imagenet hierarchy. In Proceedings of the 2020 conference on fairness, accountability, and transparency, pages 547-558, 2020.","title":"Towards fairer datasets: Filtering and balancing the distribution of the people subtree in the imagenet hierarchy","authors":"Kaiyu Yang, Klint Qinami, Li Fei-Fei, Jia Deng, Olga Russakovsky","year":"2020","venue":"Proceedings of the 2020 conference on fairness, accountability, and transparency","doi":null,"url":null,"internal_url":null},{"position":43,"raw":"[Yucer et al., 2020] Seyma Yucer, Samet Akc ¸ay, Noura Al- Moubayed, and Toby P Breckon. Exploring racial bias within face recognition via per-subject adversarially- enabled data augmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pages 18-19, 2020.","title":"Exploring racial bias within face recognition via per-subject adversariallyenabled data augmentation","authors":"Yucer","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops","doi":null,"url":null,"internal_url":null},{"position":44,"raw":"[Zhao et al., 2017] Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. Men also like shop- ping: Reducing gender bias amplification using corpus- level constraints. arXiv preprint arXiv:1707.09457, 2017.","title":"Men also like shopping: Reducing gender bias amplification using corpuslevel constraints","authors":"Zhao","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":45,"raw":"[Zhou et al., 2017] Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence, 40(6):1452-1464, 2017.","title":"Places: A 10 million image database for scene recognition","authors":"Zhou","year":"2017","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}