Unsupervised Multi-view Learning

1 Introduction

Due to the rapid development of information technology, an increasing amount of multi-view data has been generated and collected. The basic property of multi-view data is that the same set of data objects possesses multiple representations in different views, which characterize the objects from various perspectives. According to the data representation, there are two major types of multi-view data, namely the multiview feature data and the multi-view network data. For the multi-view feature data, different views encode different feature representation spaces (of possibly different dimensions) for the same set of objects. For the multi-view network data, which is also called multi-layer network [Huang et al., 2018a], multi-layered network [Chen et al., 2017], multiplex network [Mucha et al., 2010], etc., different views encode different topological structures (which may be coupled with weighted cross-view coupling) for the same set of nodes.

Although it might be sufficient to learn from individual views, as shown in the previous studies [Mucha et al., 2010; Wang et al., 2016], properly combining information from different views will improve the learning performance. Therefore, multi-view learning has emerged as a hot research topic, among which unsupervised multi-view learning has gained an increasing amount of attention due to its challenging but attractive issues that need to be addressed. The typical unsupervised multi-view learning tasks are multi-view data clustering and multi-view network community detection, which are respectively associated with multi-view feature data and multi-view network data. The goal is to partition the objects (nodes) into groups such that the objects (nodes) in the same group are similar while the objects (nodes) in different groups are dissimilar.

2 Contributions

In our research works, we have focused on addressing the following challenging issues encountered by unsupervised multi-view learning, namely leveraging higher-order structure in multi-view network [Huang et al., 2018a], capturing view consistency of community structure in multi-view network [Huang et al., 2018b] and overcoming view insufficiency in multi-view feature data [Huang et al., 2019a].

Despite significant success, the existing multi-view network community detection approaches mainly rely on the lower-order connectivity structure at the level of individual nodes and edges. As revealed in [Benson et al., 2016], the higher-order connectivity structure plays the essential role as the building block for complex networks, which, compared with edges, may contain better signature of community. One representative higher-order connectivity structure is motif, which is defined to be a dense subgraph occurring in complex networks at numbers that are significantly higher than those in randomized networks preserving the same degree of nodes [Benson et al., 2016]. To leverage motifs, we have developed a novel higher-order multi-view network community detection approach termed HM-Modularity. The basic idea is to propose a novel higher-order structure, termed harmonic motif, which is a dense subgraph having on average the largest statistical significance in each view. Based on the harmonic motif, a primary view is constructed by integrating higher-order structural information from multiple views. Additionally, the higher-order structural information of each individual view is taken as the auxiliary information. A coupling is established between the primary view and each auxiliary view. Accordingly, a harmonic motif modularity is designed to generate the community structure. The results presented in [Huang et al., 2018a] have confirmed the superiority of the proposed HM-Modularity approach in various types of multi-view networks.

The second challenging issue we have addressed is capturing view consistency of community structure in multiview network [Huang et al., 2018b]. It is believed that although different views encode various topological structures of the same set of nodes, the consistency degree of community structures across different views should coincide with the consistency degree of the topological structures across different views. For measuring the consistency of community structures across different views, the Jaccard similarity is adopted to measure the first-order structural consistency of one node across different views, based on which a cross-view community consistency model is established. Additionally, a community membership strength vector is introduced for each node in each view, based on which a network generative model is designed to measure the within-view community quality. Therefore, we have developed a novel multi-view network community detection approach termed oComm. The results presented in [Huang et al., 2018b] have confirmed the superiority of the proposed oComm approach in multi-view brain network.

Another challenging issue we have addressed is overcoming view insufficiency in multi-view feature data [Huang et al., 2019a]. The basic assumption is that some views may contain only a small portion of the features, which are insufficient for characterizing the property of the objects and hence fail to reveal the underlying cluster structure. One naive approach for addressing the view insufficiency issue is to separately perform clustering on the latent intact space representation learned by the multi-view intact space learning [Xu et al., 2015]. However, the restored latent intact space does not consider the cluster structure, and hence may fail in discovering meaningful clusters. To address this issue, our study proposes a multi-view intact space clustering (MVIC), which is able to simultaneously recover the latent intact space from multiple insufficient views and discover the cluster structure from the intact space. Additionally, the proposed method designs an interplay manner for recovering the latent intact space and discovering the cluster structure, and therefore is able to restore the latent intact space representation with better cluster structure. The results presented in [Huang et al., 2019a] have confirmed the superiority of the proposed MVIC approach.

3 OngoingWork

The first challenge that remains to be addressed is the multiview data stream clustering. Although many efforts have been made in developing multi-view clustering methods and data stream clustering methods, multi-view data stream clustering remains largely unsolved. In addition to the many issues encountered by the conventional single-view data stream clustering such as capturing cluster evolution and discovering clusters of arbitrary shapes under the limited computational resources, the main challenge of multi-view data stream clustering lies in integrating information from multiple views in a streaming manner and abstracting summary statistics from the integrated features simultaneously. Therefore, multi-view data stream clustering is a challenging task.

Another effort we are making is to further develop some other types of higher-order multi-view network community detection approaches that are different fromHM-Modularity. For instance, one ongoing work was presented as Student Abstract in AAAI 2019 [Huang et al., 2019b]. In this ongoing work, we attempt to develop a higher-order multiview network community detection approach that meets the two requirements, namely Intra-view Higher-order Community Quality and Inter-view Higher-order Community Consistency. In the first requirement, for each view, the number of motif instances within each community should be as large as possible compared with the expected number of motifs within community. In the second requirement, for each node, if the motif structures containing this node are consistent across two views, the community label should also be consistent. Currently, experiments are conducted for evaluating the performance.

4 Conclusions and FutureWork

In our research works, we have proposed several unsupervised multi-view learning approaches, which address some challenging issues from various perspectives. In our future work, one interesting topic is to extract in an unsupervised manner the multi-view proximity matrixes that reflect the underlying characteristics of the multi-view data.

Acknowledgments

This work was supported by NSFC (61876193) and Guangdong Natural Science Funds for Distinguished Young Scholar (2016A030306014).