{"type":"ScholarlyArticle","title":"CogTree: Cognition Tree Loss for Unbiased Scene Graph Generation","authors":[{"name":"Jing Yu","slug":"jing-yu","given":"Jing","family":"Yu","affiliation":""},{"name":"Yuan Chai","slug":"yuan-chai","given":"Yuan","family":"Chai","affiliation":""},{"name":"Yujing Wang","slug":"yujing-wang","given":"Yujing","family":"Wang","affiliation":""},{"name":"Yue Hu","slug":"yue-hu","given":"Yue","family":"Hu","affiliation":""},{"name":"Qi Wu","slug":"qi-wu","given":"Qi","family":"Wu","affiliation":""}],"abstract":"Scene  graphs  are  semantic  abstraction  of  images that encourage visual understanding and reasoning. However,  the  performance  of  Scene  Graph  Generation  (SGG)  is  unsatisfactory  when  faced  with biased data in real-world scenarios.  Conventional debiasing  research  mainly  studies  from  the  view of balancing data distribution or learning unbiased models  and  representations,  ignoring  the  correlations among the biased classes. In this work, we analyze this problem from a novel cognition perspective:  automatically  building  a  hierarchical  cognitive structure from the biased predictions and navigating  that  hierarchy  to  locate  the  relationships, making  the  tail  relationships  receive  more  attention in a coarse-to-fine mode.  To this end, we propose a novel debiasing Cognition Tree (CogTree) loss  for  unbiased  SGG.  We  first  build  a  cognitive structure CogTree to organize the relationships based  on  the  prediction  of  a  biased  SGG  model. The CogTree distinguishes remarkably different relationships  at  first  and  then  focuses  on  a  small portion  of  easily  confused  ones.   Then,  we  propose  a  debiasing  loss  specially  for  this  cognitive structure, which supports coarse-to-fine distinction for  the  correct  relationships.   The  loss  is  model-agnostic and consistently boosting the performance of  several  state-of-the-art  models. The  code  is available at:  https://github.com/CYVincent/Scene-Graph-Transformer-CogTree.","doi":"10.24963/ijcai.2021/176","url":"https://ijcai.org/proceedings/2021/176","pdf":"https://ijcai.org/proceedings/2021/0176.pdf","markdown":"https://ijcai.org/proceedings/2021/176.md","bibtex":"https://ijcai.org/proceedings/2021/bibtex/176","volume":{"slug":"2021","title":"Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2021","url":"https://ijcai.org/proceedings/2021/","year":2021,"location":"Montreal","dates":"19-27 August 2021","editors":"Prof. Zhi-Hua Zhou","isbn":"978-0-9992411-9-6","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2021"},"section":"Computer Vision II","track":"Main Track","number":176,"pages":{"first":1274,"last":1280},"keywords":["Computer Vision: Language and Vision"],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}