{"type":"ScholarlyArticle","title":"Learning deep structured network for weakly supervised change detection","authors":[{"name":"Salman Khan","slug":"salman-khan","given":"Salman","family":"Khan","affiliation":"Data61- CSIRO , Canberra , Australia; Australian National University , Canberra , Australia"},{"name":"Xuming He","slug":"xuming-he","given":"Xuming","family":"He","affiliation":"ShanghaiTech University , Shanghai , China"},{"name":"Fatih Porikli","slug":"fatih-porikli","given":"Fatih","family":"Porikli","affiliation":"Australian National University , Canberra , Australia"},{"name":"Mohammed Bennamoun","slug":"mohammed-bennamoun","given":"Mohammed","family":"Bennamoun","affiliation":"The University of Western Australia , Perth , Australia"},{"name":"Ferdous Sohel","slug":"ferdous-sohel","given":"Ferdous","family":"Sohel","affiliation":"Murdoch University , Perth , Australia"},{"name":"Roberto Togneri","slug":"roberto-togneri","given":"Roberto","family":"Togneri","affiliation":"The University of Western Australia , Perth , Australia"}],"abstract":"Conventional change detection methods require a large number of images to learn background models or depend on tedious pixel-level labeling by humans. In this paper, we present a weakly supervised approach that needs only image-level labels to simultaneously detect and localize changes in a pair of images. To this end, we employ a deep neural network with DAG topology to learn patterns of change from image-level labeled training data. On top of the initial CNN activations, we define a CRF model to incorporate the local differences and context with the dense connections between individual pixels. We apply a constrained mean-field algorithm to estimate the pixel-level labels, and use the estimated labels to update the parameters of the CNN in an iterative EM framework. This enables imposing global constraints on the observed foreground probability mass function. Our evaluations on four benchmark datasets demonstrate superior detection and localization performance.","doi":"10.24963/ijcai.2017/279","url":"https://ijcai.org/proceedings/2017/279","pdf":"https://ijcai.org/proceedings/2017/0279.pdf","markdown":"https://ijcai.org/proceedings/2017/279.md","bibtex":"https://ijcai.org/proceedings/2017/bibtex/279","volume":{"slug":"2017","title":"Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2017","url":"https://ijcai.org/proceedings/2017/","year":2017,"location":"Melbourne, Australia","dates":"19-25 August 2017","editors":"Carles Sierra, IIIA-CSIC","isbn":"978-0-9992411-0-3","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2017"},"section":"Machine Learning A-R","track":"Main track","number":279,"pages":{"first":2008,"last":2015},"keywords":["Machine Learning: Machine Learning","Robotics and Vision: Vision and Perception","Machine Learning: Semi-Supervised Learning"],"videos":[],"references":[{"position":1,"raw":"[Adams et al., 2010] Andrew Adams, Jongmin Baek, and Myers Abraham Davis. Fast high-dimensional filtering using the permutohedral lattice. In Computer Graphics Forum, volume 29, pages 753-762. Wiley Online Library, 2010.","title":"Fast high-dimensional filtering using the permutohedral lattice","authors":"Andrew Adams, Jongmin Baek, Myers Abraham, Davis","year":"2010","venue":"Computer Graphics Forum","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Badrinarayanan et al., 2013] Vijay Badrinarayanan, Ignas Budvytis, and Roberto Cipolla. Semi-supervised video segmentation using tree structured graphical models. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 35(11):2751-2764, 2013.","title":"Semi-supervised video segmentation using tree structured graphical models","authors":"Vijay Badrinarayanan, Ignas Budvytis, Roberto Cipolla","year":"2013","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Bourdis et al., 2011] Nicolas Bourdis, Denis Marraud, and Hichem Sahbi. Constrained optical flow for aerial image change detection. In IGARSS, pages 4176-4179. IEEE, 2011.","title":"Constrained optical flow for aerial image change detection","authors":"Bourdis","year":"2011","venue":"IGARSS","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Boykov et al., 2001] Yuri Boykov, Olga Veksler, and Ramin Zabih. Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(11):1222-1239, 2001.","title":"Fast approximate energy minimization via graph cuts","authors":"Yuri Boykov, Olga Veksler, Ramin Zabih","year":"2001","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Fan et al., 2008] Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. Liblinear: A library for large linear classification. The Journal of Ma- chine Learning Research, 9:1871-1874, 2008.","title":"Liblinear: A library for large linear classification","authors":"Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, Chih-Jen Lin","year":"2008","venue":"The Journal of Machine Learning Research","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Girshick et al., 2014] Ross Girshick, Jeff Donahue, Trevor Darrell, and Jagannath Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR, pages 580-587. IEEE, 2014.","title":"Rich feature hierarchies for accurate object detection and semantic segmentation","authors":"Ross Girshick, Jeff Donahue, Trevor Darrell, Jagannath Malik","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Gueguen and Hamid, 2015] Lionel Gueguen and Raffay Hamid. Large-scale damage detection using satellite im- agery. CVPR, 2(2):3, 2015.","title":"Large-scale damage detection using satellite imagery","authors":"Hamid ; Gueguen, Lionel Gueguen, Raffay Hamid","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Jain and Grauman, 2013] Suyog Dutt Jain and Kristen Grauman. Predicting sufficient annotation strength for in- teractive foreground segmentation. In ICCV, pages 1313- 1320. IEEE, 2013.","title":"Predicting sufficient annotation strength for interactive foreground segmentation","authors":"Suyog Dutt, Jain, Kristen Grauman","year":"2013","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Khan et al., 2015] Salman H Khan, Mohammed Ben- namoun, Ferdous Sohel, and Roberto Togneri. Cost sen- sitive learning of deep feature representations from imbal- anced data. arXiv preprint arXiv:1508.03422, 2015.","title":"Cost sensitive learning of deep feature representations from imbalanced data","authors":"Mohammed Salman H Khan, Ferdous Bennamoun, Roberto Sohel, Togneri","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Khan et al., 2016] Salman H Khan, Mohammed Ben- namoun, Ferdous Sohel, and Roberto Togneri. Automatic shadow detection and removal from a single image. IEEE transactions on pattern analysis and machine intelligence, 38(3):431-446, 2016.","title":"Automatic shadow detection and removal from a single image","authors":"Mohammed Salman H Khan, Ferdous Bennamoun, Roberto Sohel, Togneri","year":"2016","venue":"IEEE transactions on pattern analysis and machine intelligence","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Khan et al., 2017] Salman H Khan, Xuming He, Fatih Porikli, and Mohammed Bennamoun. Forest change de- tection in incomplete satellite images with deep neural net- works. IEEE Transactions on Geosciences and Remote Sensing, 2017.","title":"Forest change detection in incomplete satellite images with deep neural networks","authors":"Xuming Salman H Khan, Fatih He, Mohammed Porikli, Bennamoun","year":"2017","venue":"IEEE Transactions on Geosciences and Remote Sensing","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Kohli et al., 2007] Pushmeet Kohli, M Pawan Kumar, and Philip HS Torr. P3 & beyond: Solving energies with higher order cliques. In CVPR, pages 1-8. IEEE, 2007.","title":"P3 & beyond: Solving energies with higher order cliques","authors":"Kohli","year":"2007","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Krähenbühl and Koltun, 2011] Philipp Krähenbühl and Vladlen Koltun. Efficient inference in fully connected crfs with gaussian edge potentials. In NIPS, pages 109-117, 2011.","title":"Philipp Krähenbühl and Vladlen Koltun. Efficient inference in fully connected crfs with gaussian edge potentials","authors":"Koltun Krähenbühl","year":"2011","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Krähenbühl and Koltun, 2013] Philipp Krähenbühl and Vladlen Koltun. Parameter learning and convergent infer- ence for dense random fields. In ICML, pages 513-521, 2013.","title":"Philipp Krähenbühl and Vladlen Koltun. Parameter learning and convergent inference for dense random fields","authors":"Koltun Krähenbühl","year":"2013","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lin et al., 2014] Tsung-Yi Lin, Michael Maire, Serge Be- longie, James Hays, Pietro Perona, Deva Ramanan, Pi- otr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, pages 740-755. Springer, 2014.","title":"Microsoft coco: Common objects in context","authors":"Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, C Lawrence, Zitnick","year":"2014","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Long et al., 2015] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In CVPR, pages 3431-3440. IEEE, 2015.","title":"Fully convolutional networks for semantic segmentation","authors":"Jonathan Long, Evan Shelhamer, Trevor Darrell","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Papandreou et al., 2015] George Papandreou, Liang-Chieh Chen, Kevin P Murphy, and Alan L Yuille. Weakly-and semi-supervised learning of a deep convolutional network for semantic image segmentation. In CVPR, pages 1742- 1750. IEEE, 2015.","title":"Weakly-and semi-supervised learning of a deep convolutional network for semantic image segmentation","authors":"George Papandreou, Liang-Chieh Chen, Kevin P Murphy, Alan L Yuille","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Pinheiro and Collobert, 2015] Pedro O Pinheiro and Ronan Collobert. From image-level to pixel-level labeling with convolutional networks. In CVPR, pages 1713-1721. IEEE, 2015.","title":"From image-level to pixel-level labeling with convolutional networks","authors":"O Pedro, Ronan Pinheiro, Collobert","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Russakovsky et al., 2015] Olga Russakovsky, Amy L Bear- man, Vittorio Ferrari, and Fei-Fei Li. What's the point: Semantic segmentation with point supervision. arXiv preprint arXiv:1506.02106, 2015.","title":"What's the point: Semantic segmentation with point supervision","authors":"Olga Russakovsky, Amy L Bearman, Vittorio Ferrari, Fei-Fei Li","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Sakurada and Okatani, 2015] Ken Sakurada and Takayuki Okatani. Change detection from a street image pair us- ing cnn features and superpixel segmentation. In BMVC, 2015.","title":"Change detection from a street image pair using cnn features and superpixel segmentation","authors":"Ken Sakurada, Takayuki Okatani","year":"2015","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Simonyan and Zisserman, 2014] Karen Simonyan and An- drew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.","title":"Very deep convolutional networks for large-scale image recognition","authors":"Karen Simonyan, Andrew Zisserman","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Song et al., 2015] Shuran Song, Samuel P Lichtenberg, and Jianxiong Xiao. Sun rgb-d: A rgb-d scene understanding benchmark suite. In CVPR, pages 567-576, 2015.","title":"Sun rgb-d: A rgb-d scene understanding benchmark suite","authors":"Song","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Vineet et al., 2014] Vibhav Vineet, Jonathan Warrell, and Philip HS Torr. Filter-based mean-field inference for ran- dom fields with higher-order terms and product label- spaces. International Journal of Computer Vision, 110(3):290-307, 2014.","title":"Filter-based mean-field inference for random fields with higher-order terms and product labelspaces","authors":"Vibhav Vineet, Jonathan Warrell, Philip Hs Torr","year":"2014","venue":"International Journal of Computer Vision","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Wang et al., 2014] Yi Wang, Pierre-Marc Jodoin, Fatih Porikli, Janusz Konrad, Yannick Benezeth, and Prakash Ishwar. Cdnet 2014: An expanded change detection benchmark dataset. In CVPR Workshops, pages 393-400. IEEE, 2014.","title":"Cdnet 2014: An expanded change detection benchmark dataset","authors":"Yi Wang, Pierre-Marc Jodoin, Fatih Porikli, Janusz Konrad, Yannick Benezeth, Prakash Ishwar","year":"2014","venue":"CVPR Workshops","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yuille and Rangarajan, 2003] Alan L Yuille and Anand Rangarajan. The concave-convex procedure. Neural com- putation, 15(4):915-936, 2003.","title":"The concave-convex procedure","authors":"Alan L Yuille, Anand Rangarajan","year":"2003","venue":"Neural computation","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zagoruyko and Komodakis, 2015] Sergey Zagoruyko and Nikos Komodakis. Learning to compare image patches via convolutional neural networks. arXiv preprint arXiv:1504.03641, 2015.","title":"Learning to compare image patches via convolutional neural networks","authors":"Komodakis Zagoruyko, Sergey Zagoruyko, Nikos Komodakis","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}