{"type":"ScholarlyArticle","title":"Reinforcement Learning for Sparse-Reward Object-Interaction Tasks in a First-person Simulated 3D Environment","authors":[{"name":"Wilka Carvalho","slug":"wilka-carvalho","given":"Wilka","family":"Carvalho","affiliation":""},{"name":"Anthony Liang","slug":"anthony-liang","given":"Anthony","family":"Liang","affiliation":""},{"name":"Kimin Lee","slug":"kimin-lee","given":"Kimin","family":"Lee","affiliation":""},{"name":"Sungryull Sohn","slug":"sungryull-sohn","given":"Sungryull","family":"Sohn","affiliation":""},{"name":"Honglak Lee","slug":"honglak-lee","given":"Honglak","family":"Lee","affiliation":""},{"name":"Richard Lewis","slug":"richard-lewis","given":"Richard","family":"Lewis","affiliation":""},{"name":"Satinder Singh","slug":"satinder-singh","given":"Satinder","family":"Singh","affiliation":""}],"abstract":"Learning how to execute complex tasks involving multiple objects in a 3D world is challenging when there is no ground-truth information about the objects or any demonstration to learn from.\r\nWhen an agent only receives a signal from task-completion, this makes it challenging to learn the object-representations which support learning the correct object-interactions needed to complete the task.\r\nIn this work, we formulate learning an attentive object dynamics model as a classification problem, using random object-images to define incorrect labels for our object-dynamics model. \r\nWe show empirically that this enables object-representation learning that captures an object's category (is it a toaster?), its properties (is it on?), and object-relations (is something inside of it?).\r\nWith this, our core learner (a relational RL agent) receives the dense training signal it needs to rapidly learn object-interaction tasks.\r\nWe demonstrate results in the 3D AI2Thor simulated kitchen environment with a range of challenging food preparation tasks.\r\nWe compare our method's performance to several related approaches and against the performance of an oracle: an agent that is supplied with ground-truth information about objects in the scene.\r\nWe find that our agent achieves performance closest to the oracle in terms of both learning speed and maximum success rate.","doi":"10.24963/ijcai.2021/306","url":"https://ijcai.org/proceedings/2021/306","pdf":"https://ijcai.org/proceedings/2021/0306.pdf","markdown":"https://ijcai.org/proceedings/2021/306.md","bibtex":"https://ijcai.org/proceedings/2021/bibtex/306","volume":{"slug":"2021","title":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2021","url":"https://ijcai.org/proceedings/2021/","year":2021,"location":"Montreal","dates":"19-27 August 2021","editors":"Prof. Zhi-Hua Zhou","isbn":"978-0-9992411-9-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2021"},"section":"Machine Learning","track":"Main Track","number":306,"pages":{"first":2219,"last":2226},"keywords":["Machine Learning: Deep Reinforcement Learning","Machine Learning: Deep Learning","Machine Learning: Relational Learning","Machine Learning: Reinforcement Learning"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}