{"type":"ScholarlyArticle","title":"Learning to Learn Programs from Examples: Going Beyond Program Structure","authors":[{"name":"Kevin Ellis","slug":"kevin-ellis","given":"Kevin","family":"Ellis","affiliation":"MIT"},{"name":"Sumit Gulwani","slug":"sumit-gulwani","given":"Sumit","family":"Gulwani","affiliation":"Microsoft"}],"abstract":"Programming-by-example technologies let end users construct and run\r\nnew programs by providing examples of the intended program behavior.\r\nBut, the few provided examples seldom uniquely determine the intended\r\nprogram.  Previous approaches to picking a program used a bias toward\r\nshorter or more naturally structured programs.  Our work here gives a\r\nmachine learning approach for learning to learn programs that departs\r\nfrom previous work by relying upon features that are independent of\r\nthe program structure, instead relying upon a learned bias over\r\nprogram behaviors, and more generally over program execution\r\ntraces. Our approach leverages abundant unlabeled data for\r\nsemisupervised learning, and incorporates simple kinds of world\r\nknowledge for common-sense reasoning during program induction. These\r\ntechniques are evaluated in two programming-by-example domains,\r\nimproving the accuracy of program learners.","doi":"10.24963/ijcai.2017/227","url":"https://ijcai.org/proceedings/2017/227","pdf":"https://ijcai.org/proceedings/2017/0227.pdf","markdown":"https://ijcai.org/proceedings/2017/227.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/227","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":227,"pages":{"first":1638,"last":1645},"keywords":["Machine Learning: Semi-Supervised Learning","Multidisciplinary Topics and Applications: Intelligent User Interfaces"],"videos":[],"references":[{"position":1,"raw":"[Balog et al., 2016] Matej Balog, Alexander L Gaunt, Marc Brockschmidt, Sebastian Nowozin, and Daniel Tarlow. Deepcoder: Learning to write programs. arXiv preprint arXiv:1611.01989, 2016.","title":"Deepcoder: Learning to write programs","authors":"Matej Balog, Alexander L Gaunt, Marc Brockschmidt, Sebastian Nowozin, Daniel Tarlow","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"Cheema et al., ] Salman Cheema, Sumit Gulwani, and Joseph LaViola. Quickdraw: improving drawing experi- ence for geometric diagrams. In SIGCHI.","title":"Quickdraw: improving drawing experience for geometric diagrams","authors":"Cheema","year":null,"venue":"SIGCHI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Devlin et al., 2017] Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdel-rahman Mohamed, and Pushmeet Kohli. Robustfill: Neural program learning under noisy i/o. arXiv preprint arXiv:1703.07469, 2017.","title":"Robustfill: Neural program learning under noisy i/o","authors":"Jacob Devlin, Jonathan Uesato, Surya Bhupatiraju, Rishabh Singh, Abdel-Rahman Mohamed, Pushmeet Kohli","year":"2017","venue":null,"doi":null,"url":null,"internal_url":null},{"position":4,"raw":"Ellis et al., ] Kevin Ellis, Armando Solar-Lezama, and Josh Tenenbaum. Unsupervised learning by program synthesis. In NIPS.","title":"Unsupervised learning by program synthesis","authors":"Ellis","year":null,"venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Feng et al., 2016] Yu Feng, Ruben Martins, Jacob Van Gef- fen, Isil Dillig, and Swarat Chaudhuri. Component-based synthesis of table consolidation and transformation tasks from examples. PLDI, 2016.","title":"Component-based synthesis of table consolidation and transformation tasks from examples","authors":"Yu Feng, Ruben Martins, Jacob Van Geffen, Isil Dillig, Swarat Chaudhuri","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"Ganchev et al., ] Kuzman Ganchev, Jennifer Gillenwater, Ben Taskar, et al. Posterior regularization for structured latent variable models. JMLR.","title":"Posterior regularization for structured latent variable models","authors":"Ganchev","year":null,"venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gershman and Blei, 2012] Samuel J Gershman and David M Blei. A tutorial on bayesian nonparametric mod- els. Journal of Mathematical Psychology, 56(1):1-12, 2012.","title":"A tutorial on bayesian nonparametric models","authors":"Blei ; ] Gershman, J Samuel, David M Gershman, Blei","year":"2012","venue":"Journal of Mathematical Psychology","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gulwani et al., 2015] Sumit Gulwani, Jose Hernandez- Orallo, Emanuel Kitzelmann, Stephen Muggleton, Ute Schmid, and Ben Zorn. Inductive programming meets the real world. Commun. ACM, 2015.","title":"Inductive programming meets the real world","authors":"Sumit Gulwani, Jose Hernandez-Orallo, Emanuel Kitzelmann, Stephen Muggleton, Ute Schmid, Ben Zorn","year":"2015","venue":"Commun. ACM","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Gulwani, 2011] Sumit Gulwani. Automating string process- ing in spreadsheets using input-output examples. In ACM SIGPLAN Notices, volume 46, pages 317-330. ACM, 2011.","title":"Automating string processing in spreadsheets using input-output examples","authors":"Sumit Gulwani","year":"2011","venue":"ACM SIGPLAN Notices","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Gulwani, 2014] Sumit Gulwani. Example-based learning in computer-aided stem education. Communications of the ACM, 57(8):70-80, 2014.","title":"Example-based learning in computer-aided stem education","authors":"Sumit Gulwani","year":"2014","venue":"Communications of the ACM","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hofstadter et al., 1994] Douglas R Hofstadter, Melanie Mitchell, et al. The copycat project: A model of mental fluidity and analogy-making. 1994.","title":"The copycat project: A model of mental fluidity and analogy-making","authors":"Melanie Douglas R Hofstadter, Mitchell","year":"1994","venue":null,"doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Lau, 2001] Tessa Lau. Programming by demonstration: a machine learning approach. PhD thesis, University of Washington, 2001.","title":"Programming by demonstration: a machine learning approach","authors":"Tessa Lau","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Le and Gulwani, 2014] Vu Le and Sumit Gulwani. Flashex- tract: a framework for data extraction by examples. In ACM SIGPLAN Notices, volume 49, pages 542-553. ACM, 2014.","title":"Vu Le and Sumit Gulwani. Flashextract: a framework for data extraction by examples","authors":"Gulwani Le","year":"2014","venue":"ACM SIGPLAN Notices","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Le et al., 2013] Vu Le, Sumit Gulwani, and Zhendong Su. Smartsynth: Synthesizing smartphone automation scripts from natural language. In Proceeding of the 11th annual international conference on Mobile systems, applications, and services, pages 193-206. ACM, 2013.","title":"Smartsynth: Synthesizing smartphone automation scripts from natural language","authors":"Sumit Vu Le, Zhendong Gulwani, Su","year":"2013","venue":"Proceeding of the 11th annual international conference on Mobile systems, applications, and services","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Liang et al., 2010] Percy Liang, Michael I. Jordan, and Dan Klein. Learning programs: A hierarchical bayesian ap- proach. In Johannes Fürnkranz and Thorsten Joachims, editors, ICML, pages 639-646. Omnipress, 2010.","title":"Learning programs: A hierarchical bayesian approach","authors":"Percy Liang, Michael I Jordan, Dan Klein","year":"2010","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Liang et al., 2011] Percy Liang, Michael I. Jordan, and Dan Klein. Learning dependency-based compositional seman- tics. In ACL, pages 590-599, 2011.","title":"Learning dependency-based compositional semantics","authors":"Percy Liang, Michael I Jordan, Dan Klein","year":"2011","venue":"ACL","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Lieberman, 2001] Henry Lieberman. Your Wish is My Com- mand: Programming by Example. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2001.","title":"Your Wish is My Command: Programming by Example","authors":"Henry Lieberman","year":"2001","venue":null,"doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Lin et al., 2014] Dianhuan Lin, Eyal Dechter, Kevin Ellis, Joshua B. Tenenbaum, and Stephen Muggleton. Bias re- formulation for one-shot function induction. In ECAI 2014, pages 525-530, 2014.","title":"Bias reformulation for one-shot function induction","authors":"Dianhuan Lin, Eyal Dechter, Kevin Ellis, Joshua B Tenenbaum, Stephen Muggleton","year":"2014","venue":"ECAI 2014","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Mayer et al., 2015] Mikaël Mayer, Gustavo Soares, Maxim Grechkin, Vu Le, Mark Marron, Oleksandr Polozov, Rishabh Singh, Benjamin Zorn, and Sumit Gulwani. User interaction models for disambiguation in programming by example. In Proceedings of the 28th Annual ACM Sympo- sium on User Interface Software & Technology, UIST '15, pages 291-301, New York, NY, USA, 2015. ACM.","title":"User interaction models for disambiguation in programming by example","authors":"Mikaël Mayer, Gustavo Soares, Maxim Grechkin, Vu Le, Mark Marron, Oleksandr Polozov, Rishabh Singh, Benjamin Zorn, Sumit Gulwani","year":"2015","venue":"Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology, UIST '15","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Menon et al., 2013] Aditya Menon, Omer Tamuz, Sumit Gulwani, Butler Lampson, and Adam Kalai. A machine learning framework for programming by example. In ICML, pages 187-195, 2013.","title":"A machine learning framework for programming by example","authors":"Aditya Menon, Omer Tamuz, Sumit Gulwani, Butler Lampson, Adam Kalai","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Muggleton et al., 2015] Stephen H Muggleton, Dianhuan Lin, and Alireza Tamaddoni-Nezhad. Meta-interpretive learning of higher-order dyadic datalog: Predicate inven- tion revisited. Machine Learning, 100(1):49-73, 2015.","title":"Meta-interpretive learning of higher-order dyadic datalog: Predicate invention revisited","authors":"Dianhuan Stephen H Muggleton, Alireza Lin, Tamaddoni-Nezhad","year":"2015","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Parisotto et al., 2016] Emilio Parisotto, Abdel-rahman Mo- hamed, Rishabh Singh, Lihong Li, Dengyong Zhou, and Pushmeet Kohli. Neuro-symbolic program synthesis. arXiv preprint arXiv:1611.01855, 2016.","title":"Neuro-symbolic program synthesis","authors":"Emilio Parisotto, Abdel-Rahman Mohamed, Rishabh Singh, Lihong Li, Dengyong Zhou, Pushmeet Kohli","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Polozov and Gulwani, 2015] Oleksandr Polozov and Sumit Gulwani. Flashmeta: A framework for inductive program synthesis. ACM SIGPLAN Notices, 50(10):107-126, 2015.","title":"Flashmeta: A framework for inductive program synthesis","authors":"Gulwani Polozov, Oleksandr Polozov, Sumit Gulwani","year":"2015","venue":"ACM SIGPLAN Notices","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Prince and Smolensky, 2008] Alan Prince and Paul Smolen- sky. Optimality Theory: Constraint interaction in genera- tive grammar. John Wiley & Sons, 2008.","title":"Optimality Theory: Constraint interaction in generative grammar","authors":"Smolensky Prince","year":"2008","venue":null,"doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Raza et al., 2015] Mohammad Raza, Sumit Gulwani, and Natasa Milic-Frayling. Compositional program synthesis from natural language and examples. In IJCAI, 2015.","title":"Compositional program synthesis from natural language and examples","authors":"Mohammad Raza, Sumit Gulwani, Natasa Milic-Frayling","year":"2015","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2015/117"},{"position":26,"raw":"Schmidhuber, ] Jürgen Schmidhuber. The speed prior: a new simplicity measure yielding near-optimal computable predictions. In COLT.","title":"The speed prior: a new simplicity measure yielding near-optimal computable predictions","authors":"] Jürgen Schmidhuber, Schmidhuber","year":null,"venue":"COLT","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"Singh and Gulwani, a] Rishabh Singh and Sumit Gulwani. Predicting a correct program in programming by example. In CAV.","title":"Predicting a correct program in programming by example","authors":"Gulwani Singh, Rishabh Singh, Sumit Gulwani","year":null,"venue":"CAV","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"Singh and Gulwani, b] Rishabh Singh and Sumit Gulwani. Synthesizing number transformations from input-output examples. In CAV.","title":"Synthesizing number transformations from input-output examples","authors":"Gulwani Singh, Rishabh Singh, Sumit Gulwani","year":null,"venue":"CAV","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Singh and Gulwani, 2012] Rishabh Singh and Sumit Gul- wani. Learning semantic string transformations from ex- amples. Proceedings of the VLDB Endowment, 5(8):740- 751, 2012.","title":"Learning semantic string transformations from examples","authors":"Rishabh Singh, Sumit Gulwani","year":"2012","venue":"Proceedings of the VLDB Endowment","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Singh, 2016] Rishabh Singh. Blinkfill: Semi-supervised programming by example for syntactic string transforma- tions. Proceedings of the VLDB Endowment, 2016.","title":"Blinkfill: Semi-supervised programming by example for syntactic string transformations","authors":"Rishabh Singh","year":"2016","venue":"Proceedings of the VLDB Endowment","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Solomonoff, 1964] Ray J Solomonoff. A formal theory of inductive inference. Information and control, 7(1):1-22, 1964.","title":"A formal theory of inductive inference","authors":"J Ray, Solomonoff","year":"1964","venue":"Information and control","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Št'ava et al., 2010] Ondrej Št'ava, Bedrich Beneš, Radomir Měch, Daniel G Aliaga, and Peter Krištof. Inverse proce- dural modeling by automatic generation of l-systems. In Computer Graphics Forum, volume 29, pages 665-674. Wiley Online Library, 2010.","title":"Inverse procedural modeling by automatic generation of l-systems","authors":"Št'ava","year":"2010","venue":"Computer Graphics Forum","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Tieleman and Hinton, 2014] Tijmen Tieleman and Geoffrey Hinton. RMSprop Gradient Optimization. 2014.","title":"RMSprop Gradient Optimization","authors":"Tijmen Tieleman, Geoffrey Hinton","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}