{"type":"ScholarlyArticle","title":"Harnessing Code Switching to Transcend the Linguistic Barrier","authors":[{"name":"Ashiqur R. KhudaBukhsh","slug":"ashiqur-r-khudabukhsh","given":"Ashiqur R.","family":"KhudaBukhsh","affiliation":"School of Computer Science , Carnegie Mellon University"},{"name":"Shriphani Palakodety","slug":"shriphani-palakodety","given":"Shriphani","family":"Palakodety","affiliation":"School of Computer Science , Carnegie Mellon University"},{"name":"Jaime G. Carbonell","slug":"jaime-g-carbonell","given":"Jaime G.","family":"Carbonell","affiliation":"School of Computer Science , Carnegie Mellon University"}],"abstract":"Code mixing (or code switching) is a common phenomenon observed in social-media content generated by a linguistically diverse user-base. Studies show that in the Indian sub-continent, a substantial fraction of social media posts exhibit code switching. While the difficulties posed by code mixed documents to further downstream analyses are well-understood, lending visibility to code mixed documents under certain scenarios may have utility that has been previously overlooked. For instance, a document written in a mixture of multiple languages can be partially accessible to a wider audience; this could be particularly useful if a considerable fraction of the audience lacks fluency in one of the component languages. In this paper, we provide a systematic approach to sample code mixed documents leveraging a polyglot embedding based method that requires minimal supervision. In the context of the 2019 India-Pakistan conflict triggered by the Pulwama terror attack, we demonstrate an untapped potential of harnessing code mixing for human well-being: starting from an existing hostility diffusing hope speech classifier solely trained on English documents, code mixed documents are utilized to perform cross-lingual sampling and retrieve hope speech content written in a low-resource but widely used language - Romanized Hindi. Our proposed pipeline requires minimal supervision and holds promise in substantially reducing web moderation efforts. A further exploratory study on a new COVID-19 data set introduced in this paper demonstrates the generalizability of our cross-lingual sampling technique.","doi":"10.24963/ijcai.2020/602","url":"https://ijcai.org/proceedings/2020/602","pdf":"https://ijcai.org/proceedings/2020/0602.pdf","markdown":"https://ijcai.org/proceedings/2020/602.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/602","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Special track on AI for CompSust and Human well-being","track":"Special track on AI for CompSust and Human well-being","number":602,"pages":{"first":4366,"last":4374},"keywords":["Natural Language Processing: Natural Language Processing","Data Mining: Mining Text, Web, Social Media","Natural Language Processing: Embeddings","Natural Language Processing: Information Retrieval"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/26877/file_26877.m3u8","length_sec":750,"provider":"hls"}],"references":[{"position":1,"raw":"[Auer, 2013] Peter Auer. Code-switching in conversation: Language, interaction and identity. Routledge, 2013.","title":"Code-switching in conversation: Language, interaction and identity","authors":"Peter Auer, Auer","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Butzkamm, 1998] Wolfgang Butzkamm. Code-switching in a bilingual history lesson: The mother tongue as a con- versational lubricant. International Journal of Bilingual Education and Bilingualism, 1(2):81-99, 1998.","title":"Wolfgang Butzkamm. Code-switching in a bilingual history lesson: The mother tongue as a conversational lubricant","authors":"Butzkamm","year":"1998","venue":"International Journal of Bilingual Education and Bilingualism","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Celli et al., 2016] Fabio Celli, Evgeny Stepanov, Massimo Poesio, and Giuseppe Riccardi. Predicting brexit: Classi- fying agreement is better than sentiment and pollsters. In Proceedings of the Workshop on Computational Modeling of People's Opinions, Personality, and Emotions in Social Media (PEOPLES), pages 110-118, 2016.","title":"Predicting brexit: Classifying agreement is better than sentiment and pollsters","authors":"Fabio Celli, Evgeny Stepanov, Massimo Poesio, Giuseppe Riccardi","year":"2016","venue":"Proceedings of the Workshop on Computational Modeling of People's Opinions, Personality, and Emotions in Social Media (PEOPLES)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Cieri et al., 2016] Christopher Cieri, Mike Maxwell, Stephanie Strassel, and Jennifer Tracey. Selection criteria for low resource language programs. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pages 4543-4549, 2016.","title":"Selection criteria for low resource language programs","authors":"Christopher Cieri, Mike Maxwell, Stephanie Strassel, Jennifer Tracey","year":"2016","venue":"Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Das and Gambäck, 2014] Amitava Das and Björn Gambäck. Identifying languages at the word level in code-mixed indian social media text. In Proceedings of the 11th International Conference on Natural Language Processing, pages 378-387, 2014.","title":"Identifying languages at the word level in code-mixed indian social media text","authors":"Amitava Das, Björn Gambäck","year":"2014","venue":"Proceedings of the 11th International Conference on Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Demszky et al., 2019] Dorottya Demszky, Nikhil Garg, Rob Voigt, James Zou, Jesse Shapiro, Matthew Gentzkow, and Dan Jurafsky. Analyzing polarization in social media: Method and application to tweets on 21 mass shootings. In Proceedings of NAACL-HLT 2019, pages 2970-3005.","title":"Analyzing polarization in social media: Method and application to tweets on 21 mass shootings","authors":"Dorottya Demszky, Nikhil Garg, Rob Voigt, James Zou, Jesse Shapiro, Matthew Gentzkow, Dan Jurafsky","year":null,"venue":"Proceedings of NAACL-HLT 2019","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"ACL, June 2019.","title":null,"authors":"Acl","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Dimovski et al., 2018] Mladen Dimovski, Claudiu Musat, Vladimir Ilievski, Andreea Hossmann, and Michael Baeriswyl. Submodularity-inspired data selection for goal-oriented chatbot training based on sentence embed- dings. In Proceedings of the 27th International Joint Con- ference on Artificial Intelligence, pages 4019-4025. AAAI Press, 2018.","title":"Submodularity-inspired data selection for goal-oriented chatbot training based on sentence embeddings","authors":"Mladen Dimovski, Claudiu Musat, Vladimir Ilievski, Andreea Hossmann, Michael Baeriswyl","year":"2018","venue":"Proceedings of the 27th International Joint Conference on Artificial Intelligence","doi":null,"url":null,"internal_url":"/proceedings/2018/559"},{"position":9,"raw":"[Elfardy and Diab, 2013] Heba Elfardy and Mona T. Diab. Sentence level dialect identification in arabic. In ACL, 2013.","title":"Sentence level dialect identification in arabic","authors":"Heba Elfardy, Mona T Diab","year":"2013","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gella et al., 2014] Spandana Gella, Kalika Bali, and Mono- jit Choudhury. \"ye word kis lang ka hai bhai?\" testing the limits of word level language identification. In Pro- ceedings of the 11th International Conference on Natural Language Processing, pages 368-377. NLP Association of India, December 2014.","title":"ye word kis lang ka hai bhai?\" testing the limits of word level language identification","authors":"Gella","year":"2014","venue":"Proceedings of the 11th International Conference on Natural Language Processing","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Gumperz, 1982] John J Gumperz. Discourse strategies, vol- ume 1. Cambridge University Press, 1982.","title":"Discourse strategies","authors":"J John, Gumperz","year":"1982","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Johns Hopkins, 2020] CSSE Johns Hopkins. Coronavirus covid-19 global cases, 2020.","title":"Coronavirus covid-19 global cases","authors":"Johns Hopkins","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kumar et al., 2019] Ashutosh Kumar, Satwik Bhattamishra, Manik Bhandari, and Partha Talukdar. Submodular optimization-based diverse paraphrasing and its effective- ness in data augmentation. In Proceedings of NAACL-HLT 2019, pages 3609-3619, 2019.","title":"Submodular optimization-based diverse paraphrasing and its effectiveness in data augmentation","authors":"Ashutosh Kumar, Satwik Bhattamishra, Manik Bhandari, Partha Talukdar","year":"2019","venue":"Proceedings of NAACL-HLT 2019","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Maaten and Hinton, 2008] Laurens van der Maaten and Ge- offrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(Nov):2579-2605, 2008.","title":"Laurens van der Maaten and Geoffrey Hinton","authors":"Hinton Maaten","year":"2008","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Mate et al., 2020] Aditya Mate, Jackson A Killian, Bryan Wilder, Marie Charpignon, Ananya Awasthi, Milind Tambe, and Maimuna S Majumder. Evaluating covid- 19 lockdown policies for india: A preliminary model- ing assessment for individual states. Available at SSRN 3575207, 2020.","title":"Evaluating covid-19 lockdown policies for india: A preliminary modeling assessment for individual states","authors":"Aditya Mate, Jackson A Killian, Bryan Wilder, Marie Charpignon, Ananya Awasthi, Milind Tambe, Maimuna S Majumder","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Myers-Scotton, 1993] Carol Myers-Scotton. Dueling lan- guages: Grammatical structure in code-switching. clare- don, 1993.","title":"Dueling languages: Grammatical structure in code-switching","authors":"Myers-Scotton","year":"1993","venue":"Carol Myers-Scotton","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Nguyen and Dogruoz, 2013] Dong-Phuong Nguyen and A. Seza Dogruoz. Word level language identification in online multilingual communication. In Proceedings of EMNLP 2013, pages 857-862. ACL, 2013.","title":"Word level language identification in online multilingual communication","authors":"Dong-Phuong Nguyen, A Seza Dogruoz","year":"2013","venue":"Proceedings of EMNLP 2013","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Palakodety et al., 2020a] Shriphani Palakodety, Ashiqur R. KhudaBukhsh, and Jaime G. Carbonell. Hope speech de- tection: A computational analysis of the voice of peace. In Proceedings of ECAI 2020, page To appear, 2020.","title":"Hope speech detection: A computational analysis of the voice of peace","authors":"Palakodety","year":"2020","venue":"Proceedings of ECAI 2020","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Palakodety et al., 2020b] Shriphani Palakodety, Ashiqur R. KhudaBukhsh, and Jaime G. Carbonell. Mining insights from large-scale corpora using fine-tuned language mod- els. In Proceedings of the Twenty-Fourth European Con- ference on Artificial Intelligence (ECAI-2020), page To ap- pear, 2020.","title":"Mining insights from large-scale corpora using fine-tuned language models","authors":"Palakodety","year":"2020","venue":"Proceedings of the Twenty-Fourth European Conference on Artificial Intelligence (ECAI-2020)","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Palakodety et al., 2020c] Shriphani Palakodety, Ashiqur R. KhudaBukhsh, and Jaime G. Carbonell. Voice for the voiceless: Active sampling for finding comments support- ing the rohingyas. In Proceedings of AAAI 2020, page To appear, 2020.","title":"Voice for the voiceless: Active sampling for finding comments supporting the rohingyas","authors":"Palakodety","year":"2020","venue":"Proceedings of AAAI 2020","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Rijhwani et al., 2017a] Shruti Rijhwani, Royal Sequiera, Monojit Choudhury, Kalika Bali, and Chandra Shekhar Maddila. Estimating code-switching on twitter with a novel generalized word-level language detection tech- nique. In Proceedings of ACL 2017, pages 1971-1982, 2017.","title":"Estimating code-switching on twitter with a novel generalized word-level language detection technique","authors":"Rijhwani","year":"2017","venue":"Proceedings of ACL 2017","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Rijhwani et al., 2017b] Shruti Rijhwani, Royal Sequiera, Monojit Choudhury, Kalika Bali, and Chandra Shekhar Maddila. Estimating code-switching on twitter with a novel generalized word-level language detection tech- nique. In Proceedings of the ACL 2017, pages 1971-1982. ACL, July 2017.","title":"Estimating code-switching on twitter with a novel generalized word-level language detection technique","authors":"Rijhwani","year":"2017","venue":"Proceedings of the ACL 2017","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Rudra et al., 2016] Koustav Rudra, Shruti Rijhwani, Rafiya Begum, Kalika Bali, Monojit Choudhury, and Niloy Gan- guly. Understanding language preference for expression of opinion and sentiment: What do hindi-english speak- ers do on twitter? In Proceedings of EMNLP 2016, pages 1131-1141, 2016.","title":"Understanding language preference for expression of opinion and sentiment: What do hindi-english speakers do on twitter?","authors":"Rudra","year":"2016","venue":"Proceedings of EMNLP 2016","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Toon et al., 2019] Owen B Toon, Charles G Bardeen, Alan Robock, Lili Xia, Hans Kristensen, Matthew McKinzie, RJ Peterson, Cheryl S Harrison, Nicole S Lovenduski, and Richard P Turco. Rapidly expanding nuclear arsenals in pakistan and india portend regional and global catastrophe. Science Advances, 5(10):eaay5478, 2019.","title":"Rapidly expanding nuclear arsenals in pakistan and india portend regional and global catastrophe","authors":"Owen B Toon, Alan Charles G Bardeen, Lili Robock, Hans Xia, Matthew Kristensen, Mckinzie, Cheryl S Peterson, Nicole S Harrison, Richard P Lovenduski, Turco","year":"2019","venue":"Science Advances","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yoder et al., 2017] Michael Yoder, Shruti Rijhwani, Car- olyn Rosé, and Lori Levin. Code-switching as a social act: The case of Arabic Wikipedia talk pages. In Proceed- ings of the Second Workshop on NLP and Computational Social Science, pages 73-82. ACL, August 2017.","title":"Code-switching as a social act: The case of Arabic Wikipedia talk pages","authors":"Michael Yoder, Shruti Rijhwani, Carolyn Rosé, Lori Levin","year":"2017","venue":"Proceedings of the Second Workshop on NLP and Computational Social Science","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zeitzoff, 2017] Thomas Zeitzoff. How social media is changing conflict. Journal of Conflict Resolution, 61(9):1970-1991, 2017.","title":"How social media is changing conflict","authors":"Thomas Zeitzoff","year":"2017","venue":"Journal of Conflict Resolution","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}