{"type":"ScholarlyArticle","title":"Cross-Media Shared Representation by Hierarchical Learning with Multiple Deep Networks","authors":[{"name":"Yuxin Peng","slug":"yuxin-peng","given":"Yuxin","family":"Peng","affiliation":""},{"name":"Xin Huang","slug":"xin-huang","given":"Xin","family":"Huang","affiliation":""},{"name":"Jinwei Qi","slug":"jinwei-qi","given":"Jinwei","family":"Qi","affiliation":""}],"abstract":"Inspired by the progress of deep neural network (DNN) in single-media retrieval, the researchers have applied the DNN to cross-media retrieval. These methods are mainly two-stage learning: the first stage is to generate the separate representation for each media type, and the existing methods only model the intra-media information but ignore the inter-media correlation with the rich complementary context to the intra-media information. The second stage is to get the shared representation by learning the cross-media correlation, and the existing methods learn the shared representation through a shallow network structure, which cannot fully capture the complex cross-media correlation. For addressing the above problems, we propose the cross-media multiple deep network (CMDN) to exploit the complex cross-media correlation by hierarchical learning. In the first stage, CMDN jointly models the intra-media and inter-media information for getting the complementary separate representation of each media type. In the second stage, CMDN hierarchically combines the inter-media and intra-media representations to further learn the rich cross-media correlation by a deeper two-level network strategy, and finally get the shared representation by a stacked network style. Experiment results show that CMDN achieves better performance comparing with several state-of-the-art methods on 3 extensively used cross-media datasets.","doi":null,"url":"https://ijcai.org/proceedings/2016/541","pdf":"https://ijcai.org/proceedings/2016/0541.pdf","markdown":"https://ijcai.org/proceedings/2016/541.md","bibtex":"https://ijcai.org/proceedings/2016/bibtex/541","volume":{"slug":"2016","title":"Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2016","url":"https://ijcai.org/proceedings/2016/","year":2016,"location":"New York, NY, USA","dates":"9–15 July 2016","editors":"Subbarao Kambhampati","isbn":"978-1-57735-770-4; 978-1-57735-771-1","publisher":"AAAI Press / International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":null,"track":"","number":541,"pages":{"first":3846,"last":3852},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}