{"type":"ScholarlyArticle","title":"Weakly-Supervised Multi-view Multi-instance Multi-label Learning","authors":[{"name":"Yuying Xing","slug":"yuying-xing","given":"Yuying","family":"Xing","affiliation":"College of Computer and Information Sciences , Southwest University , Chongqing , China"},{"name":"Guoxian Yu","slug":"guoxian-yu","given":"Guoxian","family":"Yu","affiliation":"College of Computer and Information Sciences , Southwest University , Chongqing , China; CEMSE , King Abdullah University of Science and Technology , Thuwal , SA"},{"name":"Jun Wang","slug":"jun-wang","given":"Jun","family":"Wang","affiliation":"College of Computer and Information Sciences , Southwest University , Chongqing , China"},{"name":"Carlotta Domeniconi","slug":"carlotta-domeniconi","given":"Carlotta","family":"Domeniconi","affiliation":"Department of Computer Science , George Mason University , VA , USA"},{"name":"Xiangliang Zhang","slug":"xiangliang-zhang","given":"Xiangliang","family":"Zhang","affiliation":"CEMSE , King Abdullah University of Science and Technology , Thuwal , SA"}],"abstract":"Multi-view, Multi-instance, and Multi-label Learning (M3L) can model complex objects (bags), which are represented with different feature views, made of diverse instances, and annotated with discrete non-exclusive labels. Existing M3L approaches assume a complete correspondence between  bags and  views, and also assume a complete annotation for training. However, in practice, neither the correspondence between bags, nor the bags' annotations are complete. To tackle such a weakly-supervised M3L task, a solution called WSM3L is introduced. WSM3L adapts multimodal dictionary learning to learn a shared dictionary (representational space) across views and individual encoding vectors of bags for each view. The label similarity and feature similarity of encoded bags are jointly used to match bags across views. In addition, it replenishes the annotations of a bag based on the annotations of its neighborhood bags, and introduces a dispatch and aggregation term to dispatch bag-level annotations to instances and to reversely aggregate instance-level annotations to bags. WSM3L unifies these objectives and processes in a joint objective function to predict the instance-level and bag-level annotations in a coordinated fashion, and it further introduces an alternative solution for the objective function optimization. Extensive experimental results show the effectiveness of WSM3L on benchmark datasets.","doi":"10.24963/ijcai.2020/432","url":"https://ijcai.org/proceedings/2020/432","pdf":"https://ijcai.org/proceedings/2020/0432.pdf","markdown":"https://ijcai.org/proceedings/2020/432.md","bibtex":"https://ijcai.org/proceedings/2020/bibtex/432","volume":{"slug":"2020","title":"Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2020","url":"https://ijcai.org/proceedings/2020/","year":2020,"location":"Yokohama","dates":"January 2021 (postponed from 11-17 July 2020)","editors":"Christian Bessiere","isbn":"978-0-9992411-6-5","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2020"},"section":"Machine Learning","track":"Main track","number":432,"pages":{"first":3124,"last":3130},"keywords":["Machine Learning: Classification","Machine Learning: Multi-instance;Multi-label;Multi-view learning"],"videos":[{"label":"Video","url":"https://videos.ijcai20.org/26929/file_26929.m3u8","length_sec":806,"provider":"hls"}],"references":[{"position":1,"raw":"[Blei et al., 2003] David M Blei, Andrew Y Ng, and Michael I Jordan. Latent dirichlet allocation. JMLR, 3:993- 1022, 2003.","title":"Latent dirichlet allocation","authors":"Andrew Y David M Blei, Michael I Ng, Jordan","year":"2003","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Boyd and Vandenberghe, 2004] Stephen Boyd and Lieven Vandenberghe. Convex optimization. Cambridge University Press, 2004.","title":"Convex optimization","authors":"Vandenberghe Boyd, Stephen Boyd, Lieven Vandenberghe","year":"2004","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Briggs et al., 2012] Forrest Briggs, Xiaoli Z Fern, and Ra- viv Raich. Rank-loss support instance machines for miml instance annotation. In KDD, pages 534-542, 2012.","title":"Rank-loss support instance machines for miml instance annotation","authors":"Forrest Briggs, Raviv Xiaoli Z Fern, Raich","year":"2012","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Carbonneau et al., 2018] Marc-André Carbonneau, Veroni- ka Cheplygina, Eric Granger, and Ghyslain Gagnon. Multi- ple instance learning: A survey of problem characteristics and applications. Pattern Recognition, 77:329-353, 2018.","title":"Multiple instance learning: A survey of problem characteristics and applications","authors":"Marc-André Carbonneau, Veronika Cheplygina, Eric Granger, Ghyslain Gagnon","year":"2018","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Demšar, 2006] Janez Demšar. Statistical comparisons of classifiers over multiple data sets. JMLR, 7(1):1-30, 2006.","title":"Janez Demšar. Statistical comparisons of classifiers over multiple data sets","authors":"Demšar","year":"2006","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gao et al., 2015] Zan Gao, Hua Zhang, GP Xu, YB Xue, and Alexander G Hauptmann. Multi-view discriminative and structured dictionary learning with group sparsity for human action recognition. Signal Processing, 112:83-97, 2015.","title":"Multi-view discriminative and structured dictionary learning with group sparsity for human action recognition","authors":"Zan Gao, Hua Zhang, G P Xu, Alexander G Yb Xue, Hauptmann","year":"2015","venue":"Signal Processing","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Huang et al., 2019] Shengjun Huang, Wei Gao, and Zhihua Zhou. Fast multi-instance multi-label learning. TPAMI, 99(1):1-14, 2019.","title":"Fast multi-instance multi-label learning","authors":"Shengjun Huang, Wei Gao, Zhihua Zhou","year":"2019","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lampert and Krömer, 2010] Christoph H Lampert and Oliv- er Krömer. Weakly-paired maximum covariance analysis for multimodal dimensionality reduction and transfer learn- ing. In ECCV, pages 566-579, 2010.","title":"Weakly-paired maximum covariance analysis for multimodal dimensionality reduction and transfer learning","authors":"H Christoph, Oliver Lampert, Krömer","year":"2010","venue":"ECCV","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Li et al., 2017] Bing Li, Chunfeng Yuan, Weihua Xiong, Weiming Hu, Houwen Peng, Xinmiao Ding, and Steve Maybank. Multi-view multi-instance learning based on joint sparse representation and multi-view dictionary learn- ing. TPAMI, 39(12):2554-2560, 2017.","title":"Multi-view multi-instance learning based on joint sparse representation and multi-view dictionary learning","authors":"Bing Li, Chunfeng Yuan, Weihua Xiong, Weiming Hu, Houwen Peng, Xinmiao Ding, Steve Maybank","year":"2017","venue":"TPAMI","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Liu et al., 2018a] Huaping Liu, Fuchun Sun, Bin Fang, and Shan Lu. Multi-modal measurements fusion for surface material categorization. IEEE Transactions on Instrumen- tation and Measurement, 67(2):246-256, 2018.","title":"Multi-modal measurements fusion for surface material categorization","authors":"Liu","year":"2018","venue":"IEEE Transactions on Instrumentation and Measurement","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Liu et al., 2018b] Huaping Liu, Feng Wang, Xinyu Zhang, and Fuchun Sun. Weakly-paired deep dictionary learn- ing for cross-modal retrieval. Pattern Recognition Letters, 99(1):1-8, 2018.","title":"Weakly-paired deep dictionary learning for cross-modal retrieval","authors":"Liu","year":"2018","venue":"Pattern Recognition Letters","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Mandal and Biswas, 2016] Devraj Mandal and Soma Biswas. Generalized coupled dictionary learning ap- proach with applications to cross-modal matching. TIP, 25(8):3826-3837, 2016.","title":"Generalized coupled dictionary learning approach with applications to cross-modal matching","authors":"Biswas Mandal, Devraj Mandal, Soma Biswas","year":"2016","venue":"TIP","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Monaci et al., 2007] Gianluca Monaci, Philippe Jost, Pierre Vandergheynst, Boris Mailhé, Sylvain Lesage, and Rémi Gribonval. Learning multi-modal dictionaries. TIP, 16(9):2272-2283, 2007.","title":"Learning multi-modal dictionaries","authors":"Gianluca Monaci, Philippe Jost, Pierre Vandergheynst, Boris Mailhé, Sylvain Lesage, Rémi Gribonval","year":"2007","venue":"TIP","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Nguyen et al., 2013] Cam Tu Nguyen, De Chuan Zhan, and Zhi Hua Zhou. Multi-modal image annotation with multi- instance multi-label lda. In IJCAI, pages 1558-1564, 2013.","title":"Multi-modal image annotation with multiinstance multi-label lda","authors":"Cam Tu Nguyen, De Chuan Zhan, Zhi Hua Zhou","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Nguyen et al., 2014] Cam Tu Nguyen, Xiaoliang Wang, Jing Liu, and Zhihua Zhou. Labeling complicated objects: multi- view multi-instance multi-label learning. In AAAI, pages 2013-2019, 2014.","title":"Labeling complicated objects: multiview multi-instance multi-label learning","authors":"Nguyen","year":"2014","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Rubinstein et al., 2010] Ron Rubinstein, Alfred M Bruck- stein, and Michael Elad. Dictionaries for sparse representa- tion modeling. Proceedings of the IEEE, 98(6):1045-1057, 2010.","title":"Dictionaries for sparse representation modeling","authors":"Ron Rubinstein, Alfred M Bruckstein, Michael Elad","year":"2010","venue":"Proceedings of the IEEE","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Tan et al., 2018] Qiaoyu Tan, Guoxian Yu, Carlotta Domeni- coni, Jun Wang, and Zili Zhang. Incomplete multi-view weak-label learning. In IJCAI, pages 2703-2709, 2018.","title":"Incomplete multi-view weak-label learning","authors":"Tan","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2018/375"},{"position":18,"raw":"[Wu et al., 2016] Fei Wu, Xiaoyuan Jing, Xinge You, Dong Yue, Ruimin Hu, and Jingyu Yang. Multi-view low-rank dictionary learning for image classification. Pattern Recog- nition, 50:143-154, 2016.","title":"Multi-view low-rank dictionary learning for image classification","authors":"Fei Wu, Xiaoyuan Jing, Xinge You, Dong Yue, Ruimin Hu, Jingyu Yang","year":"2016","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Xing et al., 2018] Yuying Xing, Guoxian Yu, Carlotta Domeniconi, Jun Wang, and Zhang Zili. Multi-label co- training. In IJCAI, pages 2882-2888, 2018.","title":"Multi-label cotraining","authors":"Yuying Xing, Guoxian Yu, Carlotta Domeniconi, Jun Wang, Zhang Zili","year":"2018","venue":"IJCAI","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Xing et al., 2019] Yuying Xing, Guoxian Yu, Carlotta Domeniconi, Jun Wang, Zili Zhang, and Maozu Guo. Multi- view multi-instance multi-label learning based on collab- orative matrix factorization. In AAAI, pages 5508-5515, 2019.","title":"Multiview multi-instance multi-label learning based on collaborative matrix factorization","authors":"Yuying Xing, Guoxian Yu, Carlotta Domeniconi, Jun Wang, Zili Zhang, Maozu Guo","year":"2019","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Xu and Zhou, 2017] Miao Xu and Zhihua Zhou. Incomplete label distribution learning. In IJCAI, pages 3175-3181, 2017.","title":"Incomplete label distribution learning","authors":"Miao Xu, Zhihua Zhou","year":"2017","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2017/443"},{"position":22,"raw":"[Yang et al., 2013] Shujun Yang, Yuan Jiang, and Zhihua Zhou. Multi-instance multi-label learning with weak label. In IJCAI, pages 1862-1868, 2013.","title":"Multi-instance multi-label learning with weak label","authors":"Shujun Yang, Yuan Jiang, Zhihua Zhou","year":"2013","venue":"IJCAI","doi":null,"url":null,"internal_url":"/proceedings/2013/275"},{"position":23,"raw":"[Yang et al., 2018] Yang Yang, Yifeng Wu, Dechuan Zhan, Zhibin Liu, and Yuan Jiang. Complex object classification: A multi-modal multi-instance multi-label deep network with optimal transport. In KDD, pages 2594-2603, 2018.","title":"Complex object classification: A multi-modal multi-instance multi-label deep network with optimal transport","authors":"Yang Yang, Yifeng Wu, Dechuan Zhan, Zhibin Liu, Yuan Jiang","year":"2018","venue":"KDD","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Yang et al., 2019] Yang Yang, Zhaoyang Fu, Dechuan Zhan, Zhibin Liu, and Yuan Jiang. Semi-supervised multi-modal multi-instance multi-label deep network with optimal trans- port. TKDE, 2019.","title":"Semi-supervised multi-modal multi-instance multi-label deep network with optimal transport","authors":"Yang Yang, Zhaoyang Fu, Dechuan Zhan, Zhibin Liu, Yuan Jiang","year":"2019","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Yu et al., 2020] Guoxian Yu, Keyao Wang, Carlotta Domeni- coni, Maozu Guo, and Jun Wang. Isoform function predic- tion based on bi-random walks on a heterogeneous network. Bioinformatics, 36(1):303-310, 2020.","title":"Isoform function prediction based on bi-random walks on a heterogeneous network","authors":"Yu","year":"2020","venue":"Bioinformatics","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Zhang and Zhou, 2014] Minling Zhang and Zhihua Zhou. A review on multi-label learning algorithms. TKDE, 26(8):1819-1837, 2014.","title":"A review on multi-label learning algorithms","authors":"Zhou Zhang, Minling Zhang, Zhihua Zhou","year":"2014","venue":"TKDE","doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Zhang et al., 2015] Xianchao Zhang, Linlin Zong, Xinyue Liu, and Hong Yu. Constrained nmf-based multi-view clustering on unmapped data. In AAAI, pages 3174-3180, 2015.","title":"Constrained nmf-based multi-view clustering on unmapped data","authors":"Zhang","year":"2015","venue":"AAAI","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Zhou and Zhang, 2007] Zhihua Zhou and Minling Zhang. Solving multi-instance problems with classifier ensemble based on constructive clustering. KAIS, 11(2):155-170, 2007.","title":"Solving multi-instance problems with classifier ensemble based on constructive clustering","authors":"Zhihua Zhou, Minling Zhang","year":"2007","venue":"KAIS","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Zhou et al., 2012] Zhihua Zhou, Minling Zhang, Shengjun Huang, and Yufeng Li. Multi-instance multi-label learning. Artificial Intelligence, 176(1):2291-2320, 2012.","title":"Multi-instance multi-label learning","authors":"Zhihua Zhou, Minling Zhang, Shengjun Huang, Yufeng Li","year":"2012","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}