{"type":"ScholarlyArticle","title":"ADELT: Transpilation between Deep Learning Frameworks","authors":[{"name":"Linyuan Gong","slug":"linyuan-gong","given":"Linyuan","family":"Gong","affiliation":"University of California , Berkeley"},{"name":"Jiayi Wang","slug":"jiayi-wang","given":"Jiayi","family":"Wang","affiliation":"University of California , Berkeley"},{"name":"Alvin Cheung","slug":"alvin-cheung","given":"Alvin","family":"Cheung","affiliation":"University of California , Berkeley"}],"abstract":"We propose the Adversarial DEep Learning Transpiler (ADELT), a novel approach to source-to-source transpilation between deep learning frameworks. ADELT uniquely decouples code skeleton transpilation and API keyword mapping. For code skeleton transpilation, it uses few-shot prompting on large language models (LLMs), while for API keyword mapping, it uses contextual embeddings from a code-specific BERT. These embeddings are trained in a domain-adversarial setup to generate a keyword translation dictionary. ADELT is trained on an unlabeled web-crawled deep learning corpus, without relying on any hand-crafted rules or parallel data. It outperforms state-of-the-art transpilers, improving pass@1 rate by 16.2 pts and 15.0 pts for PyTorch-Keras and PyTorch-MXNet transpilation pairs respectively. We provide open access to our code at  https://github.com/gonglinyuan/adelt","doi":"10.24963/ijcai.2024/694","url":"https://ijcai.org/proceedings/2024/694","pdf":"https://ijcai.org/proceedings/2024/0694.pdf","markdown":"https://ijcai.org/proceedings/2024/694.md","bibtex":"https://ijcai.org/proceedings/2024/bibtex/694","volume":{"slug":"2024","title":"Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2024","url":"https://ijcai.org/proceedings/2024/","year":2024,"location":"Jeju, Korea","dates":"3-9 August 2024","editors":"Kate Larson","isbn":"978-1-956792-04-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2024"},"section":"Natural Language Processing","track":"Main Track","number":694,"pages":{"first":6279,"last":6287},"keywords":["Natural Language Processing: NLP: Applications","Machine Learning: ML: Adversarial machine learning","Natural Language Processing: NLP: Language models","Natural Language Processing: NLP: Machine translation and multilinguality"],"videos":[],"references":[{"position":1,"raw":"[Agashe et al., 2019] Rajas Agashe, Srinivasan Iyer, and Luke Zettlemoyer. Juice: A large scale distantly supervised dataset for open domain context-based code generation. arXiv:1910.02216 [cs], Oct 2019. arXiv: 1910.02216.","title":"Juice: A large scale distantly supervised dataset for open domain context-based code generation","authors":"Rajas Agashe, Srinivasan Iyer, Luke Zettlemoyer","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Ahmad and Cheung, 2018] Maaz Bin Safeer Ahmad and Alvin Cheung. Automatically leveraging mapreduce frame- works for data-intensive applications. In Gautam Das, Christopher M. Jermaine, and Philip A. Bernstein, edi- tors, Proceedings of the 2018 International Conference on Management of Data, SIGMOD Conference 2018, Houston, TX, USA, June 10-15, 2018, pages 1205-1220. ACM, 2018.","title":"Automatically leveraging mapreduce frameworks for data-intensive applications","authors":"Ahmad, Cheung ; ] Maaz Bin, Safeer Ahmad, Alvin Cheung","year":"2018","venue":"Proceedings of the 2018 International Conference on Management of Data, SIGMOD Conference 2018","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Ahmad et al., 2019] Maaz Bin Safeer Ahmad, Jonathan Ragan-Kelley, Alvin Cheung, and Shoaib Kamil. Auto- matically translating image processing libraries to halide. ACM Trans. Graph., 38(6):204:1-204:13, 2019.","title":"Automatically translating image processing libraries to halide","authors":"Maaz Bin, Safeer Ahmad, Jonathan Ragan-Kelley, Alvin Cheung, Shoaib Kamil","year":"2019","venue":"ACM Trans. Graph","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Ahmad et al., 2022] Maaz Bin Safeer Ahmad, Alexander J. Root, Andrew Adams, Shoaib Kamil, and Alvin Cheung. Vector instruction selection for digital signal processors us- ing program synthesis. In Babak Falsafi, Michael Ferdman, Shan Lu, and Thomas F. Wenisch, editors, ASPLOS '22: 27th ACM International Conference on Architectural Sup- port for Programming Languages and Operating Systems, Lausanne, Switzerland, 28 February 2022 -4 March 2022, pages 1004-1016. ACM, 2022.","title":"Vector instruction selection for digital signal processors using program synthesis","authors":"Maaz Bin, Safeer Ahmad, Alexander J Root, Andrew Adams, Shoaib Kamil, Alvin Cheung","year":"2022","venue":"ASPLOS '22: 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Artetxe et al., 2018] Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. Unsupervised neural ma- chine translation. arXiv:1710.11041 [cs], Feb 2018. arXiv: 1710.11041.","title":"Unsupervised neural machine translation","authors":"Mikel Artetxe, Gorka Labaka, Eneko Agirre, Kyunghyun Cho","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Bhatia et al., 2023] Sahil Bhatia, Sumer Kohli, Sanjit A. Se- shia, and Alvin Cheung. Building code transpilers for domain-specific languages using program synthesis. pages 30 pages, 1247897 bytes, 2023.","title":"Building code transpilers for domain-specific languages using program synthesis","authors":"Sahil Bhatia, Sumer Kohli, A Sanjit, Alvin Seshia, Cheung","year":"2023","venue":null,"doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Brown et al., 2020] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christo- pher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. arXiv:2005.14165 [cs], Jul 2020. arXiv: 2005.14165.","title":"Language models are few-shot learners","authors":"B Tom, Benjamin Brown, Nick Mann, Melanie Ryder, Jared Subbiah, Prafulla Kaplan, Arvind Dhariwal, Pranav Neelakantan, Girish Shyam, Amanda Sastry, Sandhini Askell, Ariel Agarwal, Gretchen Herbert-Voss, Tom Krueger, Rewon Henighan, Aditya Child, Daniel M Ramesh, Jeffrey Ziegler, Clemens Wu, Christopher Winter, Mark Hesse, Eric Chen, Mateusz Sigler, Scott Litwin, Benjamin Gray, Jack Chess, Christopher Clark, Sam Berner, Alec Mccandlish, Ilya Radford, Dario Sutskever, Amodei","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Chen et al., 2021] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavar- ian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Mu- rati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. Evaluating large language models trained on code. arXiv:2107.03374 [cs], Jul 2021. arXiv: 2107.03374.","title":"Evaluating large language models trained on code","authors":"Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob Mcgrew, Dario Amodei, Sam Mccandlish, Ilya Sutskever, Wojciech Zaremba","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Cheung et al., 2013] Alvin Cheung, Armando Solar- Lezama, and Samuel Madden. Optimizing database-backed applications with query synthesis. In Hans-Juergen Boehm and Cormac Flanagan, editors, ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI '13, Seattle, WA, USA, June 16-19, 2013, pages 3-14. ACM, 2013.","title":"Optimizing database-backed applications with query synthesis","authors":"Alvin Cheung, Armando Solar-Lezama, Samuel Madden","year":"2013","venue":"ACM SIGPLAN Conference on Programming Language Design and Implementation","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Conneau et al., 2018] Alexis Conneau, Guillaume Lample, Marc'Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. Word translation without parallel data. arXiv:1710.04087 [cs], Jan 2018. arXiv: 1710.04087.","title":"Word translation without parallel data","authors":"Alexis Conneau, Guillaume Lample, Marc'aurelio Ranzato, Ludovic Denoyer, Hervé Jégou","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Devlin et al., 2019] Jacob Devlin, Ming-Wei Chang, Ken- ton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805 [cs], May 2019. arXiv: 1810.04805.","title":"Bert: Pre-training of deep bidirectional transformers for language understanding","authors":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova","year":"2019","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Feng et al., 2020] Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. Code- bert: A pre-trained model for programming and natural languages. arXiv:2002.08155 [cs], Sep 2020. arXiv: 2002.08155.","title":"Codebert: A pre-trained model for programming and natural languages","authors":"Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, Ming Zhou","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Ganin et al., 2016] Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lem- pitsky. Domain-adversarial training of neural net- works. arXiv:1505.07818 [cs, stat], May 2016. arXiv: 1505.07818.","title":"Domain-adversarial training of neural networks","authors":"Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, Victor Lempitsky","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Gong et al., 2024] Linyuan Gong, Jiayi Wang, and Alvin Cheung. ADELT: Transpilation between deep learn- ing frameworks. (arXiv:2303.03593), April 2024. arXiv:2303.03593 [cs.","title":"ADELT: Transpilation between deep learning frameworks","authors":"Gong","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Goodfellow et al., 2014] Ian J. Goodfellow, Jean Pouget- Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sher- jil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. arXiv:1406.2661 [cs, stat], Jun 2014. arXiv: 1406.2661.","title":"Generative adversarial networks","authors":"Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Kanade et al., 2020] Aditya Kanade, Petros Maniatis, Gogul Balakrishnan, and Kensen Shi. Learning and evaluating contextual embedding of source code. arXiv:2001.00059 [cs], Aug 2020. arXiv: 2001.00059.","title":"Learning and evaluating contextual embedding of source code","authors":"Aditya Kanade, Petros Maniatis, Gogul Balakrishnan, Kensen Shi","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Karaivanov et al., 2014] Svetoslav Karaivanov, Veselin Ray- chev, and Martin Vechev. Phrase-based statistical trans- lation of programming languages. In Proceedings of the 2014 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming & Software, Onward! 2014, page 173-184. Association for Computing Machinery, Oct 2014.","title":"Phrase-based statistical trans-lation of programming languages","authors":"Svetoslav Karaivanov, Veselin Raychev, Martin Vechev","year":"2014","venue":"Proceedings of the 2014 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming & Software","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lachaux et al., 2020] Marie-Anne Lachaux, Baptiste Roziere, Lowik Chanussot, and Guillaume Lample. Unsupervised translation of programming languages. arXiv:2006.03511 [cs], Sep 2020. arXiv: 2006.03511.","title":"Unsupervised translation of programming languages","authors":"Marie-Anne Lachaux, Baptiste Roziere, Lowik Chanussot, Guillaume Lample","year":"2020","venue":null,"doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Levenshtein, 1966] Vladimir Iosifovich Levenshtein. Binary codes capable of correcting deletions, insertions and re- versals. Soviet Physics Doklady, 10:707, Feb 1966. ADS Bibcode: 1966SPhD...10..707L.","title":"Binary codes capable of correcting deletions, insertions and reversals","authors":"Vladimir Iosifovich, Levenshtein","year":"1966","venue":"Soviet Physics Doklady","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Mikolov et al., 2013] Tomas Mikolov, Kai Chen, Greg Cor- rado, and Jeffrey Dean. Efficient estimation of word repre- sentations in vector space. Jan 2013.","title":"Efficient estimation of word representations in vector space","authors":"Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean","year":"2013","venue":null,"doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Nguyen et al., 2013] Anh Tuan Nguyen, Tung Thanh Nguyen, and Tien N. Nguyen. Lexical statistical machine translation for language migration. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engi- neering, ESEC/FSE 2013, page 651-654. Association for Computing Machinery, Aug 2013.","title":"Lexical statistical machine translation for language migration","authors":"Anh Tuan Nguyen, Tung, Thanh Nguyen, Tien N Nguyen","year":"2013","venue":"Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2013","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Qiu et al., 2024] Jie Qiu, Colin Cai, Sahil Bhatia, Niranjan Hasabnis, Sanjit A. Seshia, and Alvin Cheung. Tenspiler: A verified lifting-based compiler for tensor operations. (arXiv:2404.18249), April 2024. arXiv:2404.18249 [cs].","title":"Tenspiler: A verified lifting-based compiler for tensor operations","authors":"Jie Qiu, Colin Cai, Sahil Bhatia, Niranjan Hasabnis, A Sanjit, Alvin Seshia, Cheung","year":"2024","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Quaranta et al., 2021] Luigi Quaranta, Fabio Calefato, and Filippo Lanubile. Kgtorrent: A dataset of python jupyter notebooks from kaggle. 2021 IEEE/ACM 18th Interna- tional Conference on Mining Software Repositories (MSR), page 550-554, May 2021. arXiv: 2103.10558.","title":"Kgtorrent: A dataset of python jupyter notebooks from kaggle","authors":"Luigi Quaranta, Fabio Calefato, Filippo Lanubile","year":"2021","venue":"IEEE/ACM 18th International Conference on Mining Software Repositories (MSR)","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Roziere et al., 2021] Baptiste Roziere, Marie-Anne Lachaux, Marc Szafraniec, and Guillaume Lample. Dobf: A deobfuscation pre-training objective for programming languages. arXiv:2102.07492 [cs], Oct 2021. arXiv: 2102.07492.","title":"Dobf: A deobfuscation pre-training objective for programming languages","authors":"Marie-Anne Baptiste Roziere, Marc Lachaux, Guillaume Szafraniec, Lample","year":"2021","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Roziere et al., 2022] Baptiste Roziere, Jie M. Zhang, Fran- cois Charton, Mark Harman, Gabriel Synnaeve, and Guil- laume Lample. Leveraging automated unit tests for un- supervised code translation. arXiv:2110.06773 [cs], Feb 2022. arXiv: 2110.06773.","title":"Leveraging automated unit tests for unsupervised code translation","authors":"Jie M Baptiste Roziere, Francois Zhang, Mark Charton, Gabriel Harman, Guillaume Synnaeve, Lample","year":"2022","venue":null,"doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Sennrich et al., 2016] Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. arXiv:1508.07909 [cs], Jun 2016. arXiv: 1508.07909.","title":"Neural machine translation of rare words with subword units","authors":"Sennrich","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Sutskever et al., 2014] Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. Sequence to sequence learning with neu- ral networks. arXiv:1409.3215 [cs], Dec 2014. arXiv: 1409.3215.","title":"Le. Sequence to sequence learning with neural networks","authors":"Sutskever","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Vaswani et al., 2017] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Jun 2017.","title":"Attention is all you need","authors":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, Illia Polosukhin","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}