{"type":"ScholarlyArticle","title":"Enhanced-alignment Measure for Binary Foreground Map Evaluation","authors":[{"name":"Deng-Ping Fan","slug":"deng-ping-fan","given":"Deng-Ping","family":"Fan","affiliation":"College of Computer and Control Engineering , Nankai University"},{"name":"Cheng Gong","slug":"cheng-gong","given":"Cheng","family":"Gong","affiliation":"College of Computer and Control Engineering , Nankai University"},{"name":"Yang Cao","slug":"yang-cao","given":"Yang","family":"Cao","affiliation":"College of Computer and Control Engineering , Nankai University"},{"name":"Bo Ren","slug":"bo-ren","given":"Bo","family":"Ren","affiliation":"College of Computer and Control Engineering , Nankai University"},{"name":"Ming-Ming Cheng","slug":"ming-ming-cheng","given":"Ming-Ming","family":"Cheng","affiliation":"College of Computer and Control Engineering , Nankai University"},{"name":"Ali Borji","slug":"ali-borji","given":"Ali","family":"Borji","affiliation":"Center for Research in Computer Vision , Central Florida University"}],"abstract":"The existing binary foreground map (FM) measures address various types of errors in either pixel-wise or structural ways. These measures consider pixel-level match or image-level information independently, while cognitive vision studies have shown that human vision is highly sensitive to both global information and local details in scenes. In this paper, we take a detailed look at current binary FM evaluation measures and propose a novel and effective E-measure (Enhanced-alignment measure). Our measure combines local pixel values with the image-level mean value in one term, jointly capturing image-level statistics and local pixel matching information. We demonstrate the superiority of our measure over the available measures on 4 popular datasets via 5 meta-measures, including ranking models for applications, demoting generic, random Gaussian noise maps, ground-truth switch, as well as human judgments. We find large improvements in almost all the meta-measures. For instance, in terms of application ranking, we observe improvement ranging from 9.08% to 19.65% compared with other popular measures.","doi":"10.24963/ijcai.2018/97","url":"https://ijcai.org/proceedings/2018/97","pdf":"https://ijcai.org/proceedings/2018/0097.pdf","markdown":"https://ijcai.org/proceedings/2018/97.md","bibtex":"https://ijcai.org/proceedings/2018/bibtex/97","volume":{"slug":"2018","title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2018","url":"https://ijcai.org/proceedings/2018/","year":2018,"location":"Stockholm","dates":"13-19 July 2018","editors":"Jérôme Lang","isbn":"978-0-9992411-2-7","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2018"},"section":"Computer Vision","track":"Main track","number":97,"pages":{"first":698,"last":704},"keywords":["Computer Vision: Computer Vision","Computer Vision: 2D and 3D Computer Vision"],"videos":[],"references":[{"position":1,"raw":"[Achanta et al., 2009] R. Achanta, S. Hemami, F. Estrada, and S. Süsstrunk. Frequency-tuned salient region detec- tion. In CVPR. IEEE, 2009.","title":"Frequency-tuned salient region detection","authors":"R Achanta, S Hemami, F Estrada, S Süsstrunk","year":"2009","venue":"CVPR. IEEE","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Arbelaez et al., 2011] Pablo Arbelaez, Michael Maire, Charless Fowlkes, and Jitendra Malik. Contour detec- tion and hierarchical image segmentation. IEEE TPAMI, 33(5):898-916, 2011.","title":"Contour detection and hierarchical image segmentation","authors":"Arbelaez","year":"2011","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Best and Roberts, 1975] DJ Best and DE Roberts. Algorith- m as 89: the upper tail probabilities of spearman's rho. Journal of the Royal Statistical Society. Series C (Applied Statistics), 24(3):377-379, 1975.","title":"DJ Best and DE Roberts. Algorithm as 89: the upper tail probabilities of spearman's rho","authors":"Roberts Best","year":"1975","venue":"Journal of the Royal Statistical Society. Series C (Applied Statistics)","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Blake et al., 2004] Andrew Blake, Carsten Rother, Matthew Brown, Patrick Perez, and Philip Torr. Interactive image segmentation using an adaptive GMMRF model. In EC- CV, pages 428-441. Springer, 2004.","title":"Interactive image segmentation using an adaptive GMMRF model","authors":"Andrew Blake, Carsten Rother, Matthew Brown, Patrick Perez, Philip Torr","year":"2004","venue":"EC-CV","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Borji et al., 2015] Ali Borji, Ming-Ming Cheng, Huaizu Jiang, and Jia Li. Salient object detection: A benchmark. IEEE TIP, 24(12):5706-5722, 2015.","title":"Salient object detection: A benchmark","authors":"Ali Borji, Ming-Ming Cheng, Huaizu Jiang, Jia Li","year":"2015","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chen et al., 2016] Tianshui Chen, Liang Lin, Lingbo Liu, Xiaonan Luo, and Xuelong Li. Disc: Deep image saliency computing via progressive representation learning. IEEE T Neur. Net. Lear., 27(6):1135-1149, 2016.","title":"Disc: Deep image saliency computing via progressive representation learning","authors":"Chen","year":"2016","venue":"IEEE T Neur. Net. Lear","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Cheng et al., 2015] Ming-Ming Cheng, Niloy J. Mitra, Xi- aolei Huang, Philip H. S. Torr, and Shi-Min Hu. Glob- al contrast based salient region detection. IEEE TPAMI, 37(3):569-582, 2015.","title":"Global contrast based salient region detection","authors":"Ming-Ming Cheng, Niloy J Mitra, Xiaolei Huang, H S Philip, Shi-Min Torr, Hu","year":"2015","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Csurka et al., 2013] Gabriela Csurka, Diane Larlus, Florent Perronnin, and France Meylan. What is a good evalua- tion measure for semantic segmentation? In BMVC, vol- ume 27, page 2013. Citeseer, 2013.","title":"What is a good evaluation measure for semantic segmentation?","authors":"Gabriela Csurka, Diane Larlus, Florent Perronnin, France Meylan","year":"2013","venue":"BMVC","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Everingham et al., 2010] Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisser- man. The pascal visual object classes (voc) challenge. I- JCV, 88(2):303-338, 2010.","title":"The pascal visual object classes (voc) challenge","authors":"Mark Everingham, Luc Van Gool, K I Christopher, John Williams, Andrew Winn, Zisserman","year":"2010","venue":"I-JCV","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Fan et al., 2017] Deng-Ping Fan, Ming-Ming Cheng, Yun Liu, Tao Li, and Ali Borji. Structure-measure: A new way to evaluate foreground maps. In ICCV, pages 4550-4557. IEEE, 2017.","title":"Structure-measure: A new way to evaluate foreground maps","authors":"Deng-Ping Fan, Ming-Ming Cheng, Yun Liu, Tao Li, Ali Borji","year":"2017","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hou et al., 2018] Qibin Hou, Ming-Ming Cheng, Xiaowei Hu, Ali Borji, Zhuowen Tu, and Philip Torr. Deeply su- pervised salient object detection with short connections. IEEE TPAMI, 2018.","title":"Deeply supervised salient object detection with short connections","authors":"Hou","year":"2018","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Jaccard, 1901] Paul Jaccard. Étude comparative de la distri- bution florale dans une portion des alpes et des jura. Bull Soc Vaudoise Sci Nat, 37:547-579, 1901.","title":"Étude comparative de la distribution florale dans une portion des alpes et des jura","authors":"Paul Jaccard","year":"1901","venue":"Bull Soc Vaudoise Sci Nat","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Kanan and Cottrell, 2010] Christopher Kanan and Garrison Cottrell. Robust classification of objects, faces, and flow- ers using natural image statistics. In CVPR, pages 2472- 2479. IEEE, 2010.","title":"Robust classification of objects, faces, and flowers using natural image statistics","authors":"Cottrell Kanan","year":"2010","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Lee et al., 2016] Gayoung Lee, Yu-Wing Tai, and Junmo Kim. Deep saliency with encoded low level distance map and high level features. In CVPR, pages 660-668. IEEE, 2016.","title":"Deep saliency with encoded low level distance map and high level features","authors":"Lee","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Lew et al., 2000] Michael S. Lew, Nicu Sebe, Chabane D- jeraba, and Ramesh Jain. Content-based multimedia infor- mation retrieval: State of the art and challenges. ACM T Multim. Comput., 2(1):1-19, 2000.","title":"Content-based multimedia information retrieval: State of the art and challenges","authors":"Michael S Lew, Nicu Sebe, Chabane Djeraba, Ramesh Jain","year":"2000","venue":"ACM T Multim. Comput","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Li and Yu, 2015] Guanbin Li and Yizhou Yu. Visual salien- cy based on multiscale deep features. In CVPR, pages 5455-5463. IEEE, 2015.","title":"Visual saliency based on multiscale deep features","authors":"Yu ; Li, Guanbin Li, Yizhou Yu","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Li and Yu, 2016] Guanbin Li and Yizhou Yu. Deep contrast learning for salient object detection. In CVPR, pages 478- 487. IEEE, 2016.","title":"Deep contrast learning for salient object detection","authors":"Yu ; ] Guanbin Li, Yizhou Li, Yu","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Li et al., 2013] Xiaohui Li, Huchuan Lu, Lihe Zhang, Xi- ang Ruan, and Ming-Hsuan Yang. Saliency detection via dense and sparse reconstruction. In ICCV, pages 2976- 2983. IEEE, 2013.","title":"Saliency detection via dense and sparse reconstruction","authors":"Xiaohui Li, Huchuan Lu, Lihe Zhang, Xiang Ruan, Ming-Hsuan Yang","year":"2013","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Li et al., 2014] Yin Li, Xiaodi Hou, Christof Koch, James M Rehg, and Alan L Yuille. The secrets of salien- t object segmentation. In CVPR, pages 280-287. IEEE, 2014.","title":"The secrets of salient object segmentation","authors":"Yin Li, Xiaodi Hou, Christof Koch, James M Rehg, Alan L Yuille","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Liu and Fan, 2013] Guanghai Liu and Dengping Fan. A model of visual attention for natural image retrieval. In Information Science and Cloud Computing Companion (ISCC-C), pages 728-733. IEEE, 2013.","title":"Guanghai Liu and Dengping Fan. A model of visual attention for natural image retrieval","authors":"Fan Liu","year":"2013","venue":"Information Science and Cloud Computing Companion (ISCC-C)","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Liu and Han, 2016] Nian Liu and Junwei Han. DHSNet: Deep hierarchical saliency network for salient object de- tection. In CVPR, pages 678-686. IEEE, 2016.","title":"DHSNet: Deep hierarchical saliency network for salient object detection","authors":"Nian Liu, Junwei Han","year":"2016","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Liu et al., 2011] Tie Liu, Zejian Yuan, Jian Sun, Jingdong Wang, Nanning Zheng, Xiaoou Tang, and Heung-Yeung Shum. Learning to detect a salient object. IEEE TPAMI, 33(2):353-367, 2011.","title":"Learning to detect a salient object","authors":"Liu","year":"2011","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Liu et al., 2014] Zhi Liu, Wenbin Zou, and Olivier Le Meur. Saliency tree: A novel saliency detection framework. IEEE TIP, 23(5):1937-1952, 2014.","title":"Saliency tree: A novel saliency detection framework","authors":"Zhi Liu, Wenbin Zou, Olivier Le Meur","year":"2014","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Margolin et al., 2014] Ran Margolin, Lihi Zelnik-Manor, and Ayellet Tal. How to evaluate foreground maps? In CVPR, pages 248-255. IEEE, 2014.","title":"How to evaluate foreground maps?","authors":"Ran Margolin, Lihi Zelnik-Manor, Ayellet Tal","year":"2014","venue":"CVPR","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Martin et al., 2001] David Martin, Charless Fowlkes, Doron Tal, and Jitendra Malik. A database of human segmented natural images and its application to evaluat- ing segmentation algorithms and measuring ecological statistics. In ICCV, pages 416-423. IEEE, 2001.","title":"A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics","authors":"Martin","year":"2001","venue":"ICCV","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[McGuinness and O'connor, 2010] Kevin McGuinness and Noel E O'connor. A comparative evaluation of interactive segmentation algorithms. Pattern Recognition, 43(2):434- 444, 2010.","title":"A comparative evaluation of interactive segmentation algorithms","authors":"O Mcguinness, Kevin Mcguinness, E O' Noel, Connor","year":"2010","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Movahedi and Elder, 2010] Vida Movahedi and James H Elder. Design and perceptual validation of performance measures for salient object segmentation. In IEEE CVPRW, pages 49-56, 2010.","title":"Design and perceptual validation of performance measures for salient object segmentation","authors":"Movahedi, ; Elder, Vida Movahedi, James H Elder","year":"2010","venue":"IEEE CVPRW","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Pont-Tuset and Marques, 2016] Jordi Pont-Tuset and Fer- ran Marques. Supervised evaluation of image segmen- tation and object proposal techniques. IEEE TPAMI, 38(7):1465-1478, 2016.","title":"Jordi Pont-Tuset and Ferran Marques. Supervised evaluation of image segmentation and object proposal techniques","authors":"Pont-Tuset, Marques","year":"2016","venue":"IEEE TPAMI","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Qin et al., 2014] Chanchan Qin, Guoping Zhang, Yicong Zhou, Wenbing Tao, and Zhiguo Cao. Integration of the saliency-based seed extraction and random walks for im- age segmentation. Neurocomputing, 129:378-391, 2014.","title":"Integration of the saliency-based seed extraction and random walks for image segmentation","authors":"Qin","year":"2014","venue":"Neurocomputing","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Rutishauser et al., 2004] Ueli Rutishauser, Dirk Walther, Christof Koch, and Pietro Perona. Is bottom-up attention useful for object recognition? In CVPR 2004. Proceed- ings of the 2004 IEEE Computer Society Conference on, volume 2, pages II-37. IEEE, 2004.","title":"Is bottom-up attention useful for object recognition?","authors":"Ueli Rutishauser, Dirk Walther, Christof Koch, Pietro Perona","year":"2004","venue":"Proceedings of the 2004 IEEE Computer Society Conference on","doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Shi et al., 2015] Ran Shi, King Ngi Ngan, Songnan Li, Raveendran Paramesran, and Hongliang Li. Visual quality evaluation of image object segmentation: Subjective as- sessment and objective measure. IEEE TIP, 24(12):5033- 5045, 2015.","title":"Visual quality evaluation of image object segmentation: Subjective assessment and objective measure","authors":"Ran Shi, King Ngi Ngan, Songnan Li, Raveendran Paramesran, Hongliang Li","year":"2015","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Villegas and Marichal, 2004] Paulo Villegas and Xavier Marichal. Perceptually-weighted evaluation criteria for segmentation masks in video sequences. IEEE TIP, 13(8):1092-1103, 2004.","title":"Perceptually-weighted evaluation criteria for segmentation masks in video sequences","authors":"Marichal Villegas, Paulo Villegas, Xavier Marichal","year":"2004","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[Wang et al., 2004] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessmen- t: from error visibility to structural similarity. IEEE TIP, 13(4):600-612, 2004.","title":"Image quality assessment: from error visibility to structural similarity","authors":"Zhou Wang, Alan C Bovik, Hamid R Sheikh, Eero P Simoncelli","year":"2004","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Wang et al., 2016] Linzhao Wang, Lijun Wang, Huchuan Lu, Pingping Zhang, and Xiang Ruan. Saliency detec- tion with recurrent fully convolutional networks. In ECCV, pages 825-841. Springer, 2016.","title":"Saliency detection with recurrent fully convolutional networks","authors":"Wang","year":"2016","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wang et al., 2017] Jingdong Wang, Huaizu Jiang, Zejian Yuan, Ming-Ming Cheng, Xiaowei Hu, and Nanning Zheng. Salient object detection: A discriminative region- al feature integration approach. IJCV, 123(2):251-268, 2017.","title":"Salient object detection: A discriminative regional feature integration approach","authors":"Jingdong Wang, Huaizu Jiang, Zejian Yuan, Ming-Ming Cheng, Xiaowei Hu, Nanning Zheng","year":"2017","venue":"IJCV","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Xie et al., 2013] Yulin Xie, Huchuan Lu, and Ming-Hsuan Yang. Bayesian saliency via low and mid level cues. IEEE TIP, 22(5):1689-1698, 2013.","title":"Bayesian saliency via low and mid level cues","authors":"Yulin Xie, Huchuan Lu, Ming-Hsuan Yang","year":"2013","venue":"IEEE TIP","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhao et al., 2015] Rui Zhao, Wanli Ouyang, Hongsheng Li, and Xiaogang Wang. Saliency detection by multi-context deep learning. In CVPR, pages 1265-1274. IEEE, 2015.","title":"Saliency detection by multi-context deep learning","authors":"Rui Zhao, Wanli Ouyang, Hongsheng Li, Xiaogang Wang","year":"2015","venue":"CVPR","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}