{"type":"ScholarlyArticle","title":"Lessons from the Amazon Picking Challenge: Four Aspects of Building Robotic Systems","authors":[{"name":"Clemens Eppner","slug":"clemens-eppner","given":"Clemens","family":"Eppner","affiliation":"Robotics and Biology Laboratory , Technische Universität Berlin , Germany"},{"name":"Sebastian Höfer","slug":"sebastian-hofer","given":"Sebastian","family":"Höfer","affiliation":"Robotics and Biology Laboratory , Technische Universität Berlin , Germany"},{"name":"Rico Jonschkowski","slug":"rico-jonschkowski","given":"Rico","family":"Jonschkowski","affiliation":"Robotics and Biology Laboratory , Technische Universität Berlin , Germany"},{"name":"Roberto Martín-Martín","slug":"roberto-martin-martin","given":"Roberto","family":"Martín-Martín","affiliation":"Robotics and Biology Laboratory , Technische Universität Berlin , Germany"},{"name":"Arne Sieverling","slug":"arne-sieverling","given":"Arne","family":"Sieverling","affiliation":"Robotics and Biology Laboratory , Technische Universität Berlin , Germany"},{"name":"Vincent Wall","slug":"vincent-wall","given":"Vincent","family":"Wall","affiliation":"Robotics and Biology Laboratory , Technische Universität Berlin , Germany"},{"name":"Oliver Brock","slug":"oliver-brock","given":"Oliver","family":"Brock","affiliation":"Robotics and Biology Laboratory , Technische Universität Berlin , Germany"}],"abstract":"We describe the winning entry to the Amazon Picking Challenge 2015. From the experience of building this system and competing, we derive several conclusions: (1) We suggest to characterize robotic system building along four key aspects, each of them spanning a spectrum of solutions - modularity vs. integration, generality vs. assumptions, computation vs. embodiment, and planning vs. feedback. (2) To understand which region of each spectrum most adequately addresses which robotic problem, we must explore the full spectrum of possible approaches. (3) For manipulation problems in unstructured environments, certain regions of each spectrum match the problem most adequately, and should be exploited further. This is supported by the fact that our solution deviated from the majority of the other challenge entries along each of the spectra.\r\nThis is an abridged version of a conference publication.","doi":"10.24963/ijcai.2017/676","url":"https://ijcai.org/proceedings/2017/676","pdf":"https://ijcai.org/proceedings/2017/0676.pdf","markdown":"https://ijcai.org/proceedings/2017/676.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/676","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Best Sister Conferences","track":"Best Sister Conferences","number":676,"pages":{"first":4831,"last":4835},"keywords":["Artificial Intelligence: robotics"],"videos":[],"references":[{"position":1,"raw":"[Aloimonos et al., 1988] J. Aloimonos, I. Weiss, and A. Bandy- opadhyay. Active vision. International Journal of Computer Vision, 1(4):333-356, 1988.","title":"Bandyopadhyay","authors":"J Aloimonos, I Weiss, A","year":"1988","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Atkeson et al., 2015] C. G. Atkeson, B. P. W. Babu, N. Banerjee, D. Berenson, C. P. Bove, X. Cui, M. DeDonato, R. Du, S. Feng, P. Franklin, et al. No falls, no resets: Reliable humanoid behavior in the DARPA robotics challenge. In Humanoids, pages 623-630, 2015.","title":"No falls, no resets: Reliable humanoid behavior in the DARPA robotics challenge","authors":"C G Atkeson, B P W Babu, N Banerjee, D Berenson, C P Bove, X Cui, M Dedonato, R Du, S Feng, P Franklin","year":"2015","venue":"Humanoids","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bohg et al., 2016] J. Bohg, K. Hausman, B. Sankaran, O. Brock, D. Kragic, S. Schaal, and G. Sukhatme. Interactive perception: Leveraging action in perception and perception in action. IEEE Transactions on Robotics, 2016. Under review.","title":"Interactive perception: Leveraging action in perception and perception in action","authors":"J Bohg, K Hausman, B Sankaran, O Brock, D Kragic, S Schaal, G Sukhatme","year":"2016","venue":"IEEE Transactions on Robotics","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Brooks, 1990] R. A. Brooks. Elephants don't play chess. Robotics and Autonomous Systems, 6(1):3-15, 1990.","title":"Elephants don't play chess","authors":"R A Brooks","year":"1990","venue":"Robotics and Autonomous Systems","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Buehler et al., 2007] M. Buehler, K. Iagnemma, and S. Singh. The 2005 DARPA grand challenge: the great robot race, volume 36. Springer Science & Business Media, 2007.","title":"The 2005 DARPA grand challenge: the great robot race","authors":"M Buehler, K Iagnemma, S Singh","year":"2007","venue":null,"doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Cohen, 1996] P. R. Cohen. Empirical methods for artificial intelli- gence. IEEE Intelligent Systems, 11(6):88, 1996.","title":"Empirical methods for artificial intelligence","authors":"P R Cohen","year":"1996","venue":"IEEE Intelligent Systems","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Coleman, 2015] D. Coleman. MoveIt! strengths, weaknesses, and developer insight. Presentation at ROSCon, 2015.","title":"MoveIt! strengths, weaknesses, and developer insight","authors":"D Coleman","year":"2015","venue":"Presentation at ROSCon","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Correll et al., 2016] N. Correll, K. E. Bekris, D. Berenson, O. Brock, A. Causo, K. Hauser, K. Okada, A. Rodriguez, J. M. Romano, and P. R. Wurman. Lessons from the Amazon Picking Challenge. ArXiv e-prints, January 2016.","title":"Lessons from the Amazon Picking Challenge","authors":"N Correll, K E Bekris, D Berenson, O Brock, A Causo, K Hauser, K Okada, A Rodriguez, J M Romano, P R Wurman","year":"2016","venue":null,"doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Deimel and Brock, 2016] R. Deimel and O. Brock. A novel type of compliant and underactuated robotic hand for dexterous grasp- ing. The International Journal of Robotics Research, 35:161- 185, 2016.","title":"A novel type of compliant and underactuated robotic hand for dexterous grasping","authors":"R Deimel, O Brock","year":"2016","venue":"The International Journal of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Dollar and Howe, 2010] A. M. Dollar and R. D. Howe. The highly adaptive SDM hand: Design and performance evaluation. The In- ternational Journal of Robotics Research, 29(5):585-597, 2010.","title":"The highly adaptive SDM hand: Design and performance evaluation","authors":"A M Dollar, R D Howe","year":"2010","venue":"The International Journal of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Egerstedt, 2000] M. Egerstedt. Behavior based robotics using hy- brid automata. In Hybrid Systems: computation and control, pages 103-116. Springer, 2000.","title":"Behavior based robotics using hybrid automata","authors":"M Egerstedt","year":"2000","venue":"Hybrid Systems: computation and control","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Eppner et al., 2015] C. Eppner, R. Deimel, J. Álvarez Ruiz, M. Maertens, and O. Brock. Exploitation of environmental con- straints in human and robotic grasping. The International Journal of Robotics Research, 34(7):1021-1038, 2015.","title":"Exploitation of environmental constraints in human and robotic grasping","authors":"C Eppner, R Deimel, J Álvarez Ruiz, M Maertens, O Brock","year":"2015","venue":"The International Journal of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Eppner et al., 2016] C. Eppner, S. Höfer, R. Jonschkowski, R. Martın-Martın, A. Sieverling, V. Wall, and O. Brock. Lessons from the amazon picking challenge: Four aspects of building robotic systems. Proc. of RSS, 2016.","title":"Lessons from the amazon picking challenge: Four aspects of building robotic systems","authors":"C Eppner, S Höfer, R Jonschkowski, R Martın-Martın, A Sieverling, V Wall, O Brock","year":"2016","venue":"Proc. of RSS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Espiau et al., 1992] B. Espiau, F. Chaumette, and P. Rives. A new approach to visual servoing in robotics. IEEE Transactions on Robotics and Automation, 8(3):313-326, 1992.","title":"A new approach to visual servoing in robotics","authors":"B Espiau, F Chaumette, P Rives","year":"1992","venue":"IEEE Transactions on Robotics and Automation","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hastie et al., 2005] T. Hastie, R. Tibshirani, J. Friedman, and J. Franklin. The elements of statistical learning: data min- ing, inference and prediction. The Mathematical Intelligencer, 27(2):83-85, 2005.","title":"The elements of statistical learning: data mining, inference and prediction","authors":"T Hastie, R Tibshirani, J Friedman, J Franklin","year":"2005","venue":"The Mathematical Intelligencer","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Hawes et al., 2010] N. Hawes, M. Zillich, and P. Jensfelt. Lessons learnt from scenario-based integration. In Cognitive Systems, pages 423-438. Springer, 2010.","title":"Lessons learnt from scenario-based integration","authors":"N Hawes, M Zillich, P Jensfelt","year":"2010","venue":"Cognitive Systems","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Johnson et al., 2015] M. Johnson, B. Shrewsbury, S. Bertrand, T. Wu, D. Duran, M. Floyd, P. Abeles, D. Stephen, N. Mertins, A. Lesman, et al. Team IHMC's lessons learned from the DARPA robotics challenge trials. Journal of Field Robotics, 32(2):192- 208, 2015.","title":"Team IHMC's lessons learned from the DARPA robotics challenge trials","authors":"M Johnson, B Shrewsbury, S Bertrand, T Wu, D Duran, M Floyd, P Abeles, D Stephen, N Mertins, A Lesman","year":"2015","venue":"Journal of Field Robotics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Jonschkowski and Brock, 2015] R. Jonschkowski and O. Brock. Learning state representations with robotic priors. Autonomous Robots, 39(3):407-428, 2015.","title":"Learning state representations with robotic priors","authors":"R Jonschkowski, O Brock","year":"2015","venue":"Autonomous Robots","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Jonschkowski et al., 2016] R. Jonschkowski, C. Eppner, S. Höfer, R. Martín-Martín, and O. Brock. Probabilistic multi-class seg- mentation for the Amazon Picking Challenge. In IEEE/RSJ ICRA, 2016.","title":"Probabilistic multi-class segmentation for the Amazon Picking Challenge","authors":"R Jonschkowski, C Eppner, S Höfer, R Martín-Martín, O Brock","year":"2016","venue":"IEEE/RSJ ICRA","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Katz and Brock, 2011] D. Katz and O. Brock. A factorization ap- proach to manipulation in unstructured environments. In ISRR 2009, pages 285-300. Springer, 2011.","title":"A factorization approach to manipulation in unstructured environments","authors":"D Katz, O Brock","year":"2011","venue":"ISRR 2009","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[LaValle, 2006] S. M. LaValle. Planning Algorithms. Cambridge University Press, New York, NY, USA, 2006.","title":"Planning Algorithms","authors":"S M Lavalle","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Lozano-Perez et al., 1984] T. Lozano-Perez, M. T. Mason, and R. H. Taylor. Automatic synthesis of fine-motion strategies for robots. The International Journal of Robotics Research, 3(1):3- 24, 1984.","title":"Automatic synthesis of fine-motion strategies for robots","authors":"T Lozano-Perez, M T Mason, R H Taylor","year":"1984","venue":"The International Journal of Robotics Research","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Marr, 1982] D. Marr. Vision: A computational approach. Freeman & Co., San Francisco, 1982.","title":"Vision: A computational approach","authors":"D Marr","year":"1982","venue":null,"doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Martín-Martín and Brock, 2014] R. Martín-Martín and O. Brock. Online interactive perception of articulated objects with multi- level recursive estimation based on task-specific priors. In IROS, pages 2494-2501, 2014.","title":"Online interactive perception of articulated objects with multilevel recursive estimation based on task-specific priors","authors":"R Martín-Martín, O Brock","year":"2014","venue":"IROS","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Miller and Allen, 2004] A. T. Miller and P. K. Allen. GraspIt! a versatile simulator for robotic grasping. IEEE Robotics & Au- tomation Magazine, 11(4):110-122, 2004.","title":"GraspIt! a versatile simulator for robotic grasping","authors":"A T Miller, P K Allen","year":"2004","venue":"IEEE Robotics & Automation Magazine","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Papadimitriou and Tsitsiklis, 1987] C. H. Papadimitriou and J. N. Tsitsiklis. The complexity of Markov decision processes. Math- ematics of operations research, 12(3):441-450, 1987.","title":"The complexity of Markov decision processes","authors":"C H Papadimitriou, J N Tsitsiklis","year":"1987","venue":"Mathematics of operations research","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Pfeifer and Gómez, 2009] R. Pfeifer and G. Gómez. Morpholog- ical computation-connecting body, brain, and environment. In Creating Brain-Like Intelligence, pages 66-83. Springer, 2009.","title":"Morphological computation-connecting body, brain, and environment","authors":"R Pfeifer, G Gómez","year":"2009","venue":"Creating Brain-Like Intelligence","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Quigley et al., 2009] M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs, R. Wheeler, and A. Y. Ng. ROS: an open- source robot operating system. In ICRA workshop on open source software, 2009.","title":"ROS: an opensource robot operating system","authors":"M Quigley, K Conley, B Gerkey, J Faust, T Foote, J Leibs, R Wheeler, A Y Ng","year":"2009","venue":"ICRA workshop on open source software","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Rickert et al., 2014] M. Rickert, A. Sieverling, and O. Brock. Bal- ancing exploration and exploitation in sampling-based motion planning. IEEE Transactions on Robotics, 30(6):1305-1317, 2014.","title":"Balancing exploration and exploitation in sampling-based motion planning","authors":"M Rickert, A Sieverling, O Brock","year":"2014","venue":"IEEE Transactions on Robotics","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Scholz et al., 2014] J. Scholz, M. Levihn, C. Isbell, and D. Wingate. A physics-based model prior for object-oriented MDPs. In Proc. of ICML-14, pages 1089-1097, 2014.","title":"A physics-based model prior for object-oriented MDPs","authors":"J Scholz, M Levihn, C Isbell, D Wingate","year":"2014","venue":"Proc. of ICML-14","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Schwaber, 2004] K. Schwaber. Agile project management with Scrum. Microsoft Press, 2004.","title":"Agile project management with Scrum","authors":"K Schwaber","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Sucan and Chitta, 2016] I. I. Sucan and S. Chitta. MoveIt. http: //moveit.ros.org, 2016.","title":"MoveIt","authors":"Chitta Sucan, I I Sucan, S Chitta","year":"2016","venue":null,"doi":null,"url":"http://moveit.ros.org","internal_url":null},{"position":33,"raw":"[Thrun et al., 2005] S. Thrun, W. Burgard, and D. Fox. Probabilis- tic robotics. MIT Press, 2005.","title":"Probabilistic robotics","authors":"S Thrun, W Burgard, D Fox","year":"2005","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[van Gigch, 1991] J. P. van Gigch. System Design Modeling and Metamodeling. Springer Science & Business Media, 1991.","title":"System Design Modeling and Metamodeling","authors":"J P Van Gigch","year":"1991","venue":null,"doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wolpert and Macready, 1997] D. H. Wolpert and W. G. Macready. No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 1(1):67-82, 1997.","title":"No free lunch theorems for optimization","authors":"D H Wolpert, W G Macready","year":"1997","venue":"IEEE Transactions on Evolutionary Computation","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Wolpert, 1996] D. H. Wolpert. The lack of a priori distinctions between learning algorithms. Neural computation, 8(7):1341- 1390, 1996.","title":"The lack of a priori distinctions between learning algorithms","authors":"D H Wolpert","year":"1996","venue":"Neural computation","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}