{"type":"ScholarlyArticle","title":"Dataset Augmentation in Papyrology with Generative Models: A Study of Synthetic Ancient Greek Character Images","authors":[{"name":"Matthew I. Swindall","slug":"matthew-i-swindall","given":"Matthew I.","family":"Swindall","affiliation":"Middle Tennessee State University"},{"name":"Timothy Player","slug":"timothy-player","given":"Timothy","family":"Player","affiliation":"University of Tennessee , Knoxville"},{"name":"Ben Keener","slug":"ben-keener","given":"Ben","family":"Keener","affiliation":"University of Kentucky"},{"name":"Alex C. Williams","slug":"alex-c-williams","given":"Alex C.","family":"Williams","affiliation":"Amazon"},{"name":"James H. Brusuelas","slug":"james-h-brusuelas","given":"James H.","family":"Brusuelas","affiliation":"University of Kentucky"},{"name":"Federica Nicolardi","slug":"federica-nicolardi","given":"Federica","family":"Nicolardi","affiliation":"Università degli Studi di Napoli Federico II"},{"name":"Marzia D'Angelo","slug":"marzia-d-angelo","given":"Marzia","family":"D'Angelo","affiliation":"Istituto Papirologico 'G. Vitelli' ( Università degli Studi di Firenze )"},{"name":"Claudio Vergara","slug":"claudio-vergara","given":"Claudio","family":"Vergara","affiliation":"Università di Pisa"},{"name":"Michael McOsker","slug":"michael-mcosker","given":"Michael","family":"McOsker","affiliation":"Institut für Altertumskunde Universität zu Köln"},{"name":"John F. Wallin","slug":"john-f-wallin","given":"John F.","family":"Wallin","affiliation":"Middle Tennessee State University"}],"abstract":"Character recognition models rely substantially on image datasets that maintain a balance of class samples. However, achieving a balance of classes is particularly challenging for ancient manuscript contexts as character instances may be significantly limited. In this paper, we present findings from a study that assess the efficacy of using synthetically generated character instances to augment an existing dataset of ancient Greek character images for use in machine learning models. We complement our model exploration by engaging professional papyrologists to better understand the practical opportunities afforded by synthetic instances. Our results suggest that synthetic instances improve model performance for limited character classes, and may have unexplored effects on character classes more generally. We also find that trained papyrologists are unable to distinguish between synthetic and non-synthetic images and regard synthetic instances as valuable assets for professional and educational contexts. We conclude by discussing the practical implications of our research.","doi":"10.24963/ijcai.2022/689","url":"https://ijcai.org/proceedings/2022/689","pdf":"https://ijcai.org/proceedings/2022/0689.pdf","markdown":"https://ijcai.org/proceedings/2022/689.md","bibtex":"https://ijcai.org/proceedings/2022/bibtex/689","volume":{"slug":"2022","title":"Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2022","url":"https://ijcai.org/proceedings/2022/","year":2022,"location":"Vienna","dates":"23-29 July 2022","editors":"Luc De Raedt","isbn":"978-1-956792-00-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2022"},"section":"Research Papers","track":"AI and Arts","number":689,"pages":{"first":4973,"last":4979},"keywords":["Application domains: Other domains of art or creativity","Theory and philosophy of arts and creativity in AI systems: Support of human creativity","Methods and resources: Datasets, knowledge bases and ontologies","Application domains: Text, literature and creative language"],"videos":[{"label":"Video #1","url":"https://slideslive.com/38987167","length_sec":84,"provider":"slideslive"},{"label":"Video #2","url":"https://slideslive.com/38985310","length_sec":590,"provider":"slideslive"}],"references":[{"position":1,"raw":"[Ast and Bagnall, 2012] Rodney Ast and Roger Bagnall. Digital corpus of greek and latin literary papyri. https: //caa.hcommons.org/deposits/item/hc:12447/, 2012. Ac- cessed: 2022-01-15.","title":"Digital corpus of greek and latin literary papyri","authors":"Bagnall Ast, Rodney Ast, Roger Bagnall","year":"2012","venue":null,"doi":null,"url":"https://caa.hcommons.org/deposits/item/hc:12447/,2012.Ac-cessed:2022-01-15","internal_url":null},{"position":2,"raw":"[Bowles et al., 2018] Christopher Bowles, Liang Chen, Ri- cardo Guerrero, Paul Bentley, Roger Gunn, Alexan- der Hammers, David Alexander Dickie, Maria Valdés Hernández, Joanna Wardlaw, and Daniel Rueckert. Gan augmentation: Augmenting training data using generative adversarial networks. arXiv preprint arXiv:1810.10863, 2018.","title":"Gan augmentation: Augmenting training data using generative adversarial networks","authors":"Christopher Bowles, Liang Chen, Ricardo Guerrero, Paul Bentley, Roger Gunn, Alexander Hammers, David Alexander Dickie, Maria Valdés Hernández, Joanna Wardlaw, Daniel Rueckert","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Braun and Clarke, 2012] Virginia Braun and Victoria Clarke. Thematic analysis. In H. Cooper, P. M. Camic, D. L. Long, A. T. Panter, D. Rindskopf, K. J. Sher (Eds.), APA handbook of research methods in psychology, Vol.","title":"Virginia Braun and Victoria Clarke. Thematic analysis","authors":"Clarke Braun","year":"2012","venue":"APA handbook of research methods in psychology","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"Research designs: Quantitative, qualitative, neu- ropsychological, and biological, pages 57-71. American Psychology Association, 2012.","title":"Research designs: Quantitative, qualitative, neuropsychological, and biological","authors":null,"year":"2012","venue":null,"doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Cronin et al., 2020] Neil J. Cronin, Taija Finni, and Olivier Seynnes. Using deep learning to generate synthetic b-mode musculoskeletal ultrasound images. Computer Methods and Programs in Biomedicine, 196:105583, 2020.","title":"Using deep learning to generate synthetic b-mode musculoskeletal ultrasound images","authors":"J Neil, Taija Cronin, Olivier Finni, Seynnes","year":"2020","venue":"Computer Methods and Programs in Biomedicine","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Deng, 2012] Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Processing Magazine, 29(6):141-142, 2012.","title":"The mnist database of handwritten digit images for machine learning research [best of the web","authors":"Li Deng","year":"2012","venue":"IEEE Signal Processing Magazine","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Elgammal et al., 2017] Ahmed Elgammal, Bingchen Liu, Mohamed Elhoseiny, and Marian Mazzone. Can: Creative adversarial networks, generating 'art' by learning about styles and deviating from style norms. arXiv preprint arXiv:1706.07068, 2017.","title":"Can: Creative adversarial networks, generating 'art' by learning about styles and deviating from style norms","authors":"Ahmed Elgammal, Bingchen Liu, Mohamed Elhoseiny, Marian Mazzone","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Fischer et al., 2011] Andreas Fischer, Volkmar Frinken, Alicia Fornés, and Horst Bunke. Transcription alignment of latin manuscripts using hidden markov models. New York, NY, USA, 2011. Association for Computing Ma- chinery.","title":"Transcription alignment of latin manuscripts using hidden markov models","authors":"Andreas Fischer, Volkmar Frinken, Alicia Fornés, Horst Bunke","year":"2011","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Frid-Adar et al., 2018] Maayan Frid-Adar, Idit Diamant, Eyal Klang, Michal Amitai, Jacob Goldberger, and Hayit Greenspan. Gan-based synthetic medical image augmen- tation for increased cnn performance in liver lesion classi- fication. Neurocomputing, 321:321-331, 2018.","title":"Gan-based synthetic medical image augmentation for increased cnn performance in liver lesion classification","authors":"Maayan Frid-Adar, Idit Diamant, Eyal Klang, Michal Amitai, Jacob Goldberger, Hayit Greenspan","year":"2018","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Goodfellow et al., 2014] Ian J. Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Gen- erative adversarial networks. In Advances in Neural In- formation Processing Systems, volume 27.","title":"Generative adversarial networks","authors":"Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":null,"venue":"Advances in Neural Information Processing Systems","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Karras et al., 2020] Tero Karras, Samuli Laine, Miika Ait- tala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. An- alyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8110- 8119. IEEE, 2020.","title":"Analyzing and improving the image quality of stylegan","authors":"Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, Timo Aila","year":"2020","venue":"Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Shrivastava et al., 2017] Ashish Shrivastava, Tomas Pfister, Oncel Tuzel, Joshua Susskind, Wenda Wang, and Russell Webb. Learning from simulated and unsupervised images through adversarial training. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, July 2017.","title":"Learning from simulated and unsupervised images through adversarial training","authors":"Ashish Shrivastava, Tomas Pfister, Oncel Tuzel, Joshua Susskind, Wenda Wang, Russell Webb","year":"2017","venue":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Swindall et al., 2021] Matthew I. Swindall, Gregory Crois- dale, Chase C. Hunter, Ben Keener, Alex C. Williams, James H. Brusuelas, Nita Krevans, Melissa Sellew, Lucy Fortson, and John F. Wallin. Exploring learning ap- proaches for ancient greek character recognition with cit- izen science data. In 2021 17th International Conference on eScience (eScience), pages 128-137. IEEE, 2021.","title":"Exploring learning approaches for ancient greek character recognition with citizen science data","authors":"Matthew I Swindall, Gregory Croisdale, Chase C Hunter, Ben Keener, Alex C Williams, James H Brusuelas, Nita Krevans, Melissa Sellew, Lucy Fortson, John F Wallin","year":"2021","venue":"17th International Conference on eScience (eScience)","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Tremblay et al., 2018] Jonathan Tremblay, Aayush Prakash, David Acuna, Mark Brophy, Varun Jam- pani, Cem Anil, Thang To, Eric Cameracci, Shaad Boochoon, and Stan Birchfield. Proceedings of the ieee conference on computer vision and pattern recognition (cvpr) workshops. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, June 2018.","title":"Proceedings of the ieee conference on computer vision and pattern recognition (cvpr) workshops","authors":"Jonathan Tremblay, Aayush Prakash, David Acuna, Mark Brophy, Varun Jampani, Cem Anil, Thang To, Eric Cameracci, Shaad Boochoon, Stan Birchfield","year":"2018","venue":"IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Williams et al., 2014] Alex C. Williams, John F. Wallin, Haoyu Yu, Marco Perale, Hyrum D. Carroll, Anne- Francoise Lamblin, Lucy Fortson, Dirk Obbink, Chris J. Lintott, and James H. Brusuelas. A computational pipeline for crowdsourced transcriptions of ancient greek papyrus fragments. In 2014 IEEE International Conference on Big Data (Big Data), pages 100-105. IEEE, 2014.","title":"A computational pipeline for crowdsourced transcriptions of ancient greek papyrus fragments","authors":"Alex C Williams, John F Wallin, Haoyu Yu, Marco Perale, D Hyrum, Anne-Francoise Carroll, Lucy Lamblin, Dirk Fortson, Chris J Obbink, James H Lintott, Brusuelas","year":"2014","venue":"IEEE International Conference on Big Data (Big Data)","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}