{"type":"ScholarlyArticle","title":"Latent Processes Identification From Multi-View Time Series","authors":[{"name":"Zenan Huang","slug":"zenan-huang","given":"Zenan","family":"Huang","affiliation":"Qiushi Academy for Advanced Studies (QAAS) , Zhejiang University; College of Computer Science and Technology , Zhejiang University"},{"name":"Haobo Wang","slug":"haobo-wang","given":"Haobo","family":"Wang","affiliation":"College of Computer Science and Technology , Zhejiang University"},{"name":"Junbo Zhao","slug":"junbo-zhao","given":"Junbo","family":"Zhao","affiliation":"College of Computer Science and Technology , Zhejiang University"},{"name":"Nenggan Zheng","slug":"nenggan-zheng","given":"Nenggan","family":"Zheng","affiliation":"Qiushi Academy for Advanced Studies (QAAS) , Zhejiang University; The State Key Lab of Brain-Machine Intelligence , Zhejiang University; CCAI by MOE and Zhejiang Provincial Government (ZJU ); College of Computer Science and Technology , Zhejiang University"}],"abstract":"Understanding the dynamics of time series data typically requires identifying the unique latent factors for data generation, a.k.a., latent processes identification. Driven by the independent assumption, existing works have made great progress in handling single-view data. However, it is a non-trivial problem that extends them to multi-view time series data because of two main challenges: (i) the complex data structure, such as temporal dependency, can result in violation of the independent assumption; (ii) the factors from different views are generally overlapped and are hard to be aggregated to a complete set. In this work, we propose a novel framework MuLTI that employs the contrastive learning technique to invert the data generative process for enhanced identifiability. Additionally, MuLTI integrates a permutation mechanism that merges corresponding overlapped variables by the establishment of an optimal transport formula. Extensive experimental results on synthetic and real-world datasets demonstrate the superiority of our method in recovering identifiable latent variables on multi-view time series. The code is available on https://github.com/lccurious/MuLTI.","doi":"10.24963/ijcai.2023/428","url":"https://ijcai.org/proceedings/2023/428","pdf":"https://ijcai.org/proceedings/2023/0428.pdf","markdown":"https://ijcai.org/proceedings/2023/428.md","bibtex":"https://ijcai.org/proceedings/2023/bibtex/428","volume":{"slug":"2023","title":"Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2023","url":"https://ijcai.org/proceedings/2023/","year":2023,"location":"Macao, SAR","dates":"19-25 August 2023","editors":"Edith Elkind","isbn":"978-1-956792-03-4","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2023"},"section":"Machine Learning","track":"Main Track","number":428,"pages":{"first":3848,"last":3856},"keywords":["Machine Learning: ML: Multi-view learning","Machine Learning: ML: Causality"],"videos":[],"references":[{"position":1,"raw":"[Akaho, 2006] Shotaro Akaho. A kernel method for canoni- cal correlation analysis. arXiv preprint cs/0609071, 2006.","title":"A kernel method for canonical correlation analysis","authors":"Shotaro Akaho","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Altun et al., 2010] Kerem Altun, Billur Barshan, and Orkun Tunc ¸el. Comparative study on classifying human activi- ties with miniature inertial and magnetic sensors. Pattern Recognition, 2010.","title":"Comparative study on classifying human activities with miniature inertial and magnetic sensors","authors":"Altun","year":"2010","venue":"Pattern Recognition","doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Andrew et al., 2013] Galen Andrew, Raman Arora, Jeff Bilmes, and Karen Livescu. Deep canonical correlation analysis. In International Conference on Machine Learn- ing, pages 1247-1255. PMLR, 2013.","title":"Deep canonical correlation analysis","authors":"Galen Andrew, Raman Arora, Jeff Bilmes, Karen Livescu","year":"2013","venue":"International Conference on Machine Learning","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Belouchrani et al., 1997] Adel Belouchrani, Karim Abed- Meraim, J-F Cardoso, and Eric Moulines. A blind source separation technique using second-order statistics. IEEE Transactions on signal processing, 1997.","title":"A blind source separation technique using second-order statistics","authors":"Adel Belouchrani, Karim Abed-Meraim, J-F Cardoso, Eric Moulines","year":"1997","venue":"IEEE Transactions on signal processing","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Bianchi et al., 2021] Filippo Maria Bianchi, Simone Scar- dapane, Sigurd Løkse, and Robert Jenssen. Reservoir Computing Approaches for Representation and Classifica- tion of Multivariate Time Series. IEEE Transactions on Neural Networks and Learning Systems, 2021.","title":"Reservoir Computing Approaches for Representation and Classification of Multivariate Time Series","authors":"Maria Filippo, Simone Bianchi, Sigurd Scardapane, Robert Løkse, Jenssen","year":"2021","venue":"IEEE Transactions on Neural Networks and Learning Systems","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Chickering, 2002] David Maxwell Chickering. Optimal structure identification with greedy search. Journal of ma- chine learning research, 2002.","title":"Optimal structure identification with greedy search","authors":"David Maxwell, Chickering","year":"2002","venue":"Journal of machine learning research","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Ding et al., 2006] Mingzhou Ding, Yonghong Chen, and Steven L Bressler. Granger causality: Basic theory and application to neuroscience. Handbook of time series anal- ysis: recent theoretical developments and applications, 2006.","title":"Handbook of time series analysis: recent theoretical developments and applications","authors":"Ding","year":"2006","venue":null,"doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Gresele et al., 2019] Luigi Gresele, Paul K. Rubenstein, Arash Mehrjou, Francesco Locatello, and Bernhard Schölkopf. The incomplete rosetta stone problem: Iden- tifiability results for multi-view nonlinear ICA. In Proc. of UAI, 2019.","title":"The incomplete rosetta stone problem: Identifiability results for multi-view nonlinear ICA","authors":"Luigi Gresele, Paul K Rubenstein, Arash Mehrjou, Francesco Locatello, Bernhard Schölkopf","year":"2019","venue":"Proc. of UAI","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[He et al., 2021] Guoliang He, Han Wang, Shenxiang Liu, and Bo Zhang. CSMVC: A Multiview Method for Mul- tivariate Time-Series Clustering. IEEE Transactions on Cybernetics, 2021.","title":"CSMVC: A Multiview Method for Multivariate Time-Series Clustering","authors":"Guoliang He, Han Wang, Shenxiang Liu, Bo Zhang","year":"2021","venue":"IEEE Transactions on Cybernetics","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Higgins et al., 2017] Irina Higgins, Loïc Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained varia- tional framework. In Proc. of ICLR, 2017.","title":"beta-vae: Learning basic visual concepts with a constrained variational framework","authors":"Irina Higgins, Loïc Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, Alexander Lerchner","year":"2017","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Hotelling, 1992] Harold Hotelling. Relations Between Two Sets of Variates. In Breakthroughs in Statistics. Springer New York, 1992.","title":"Harold Hotelling. Relations Between Two Sets of Variates","authors":"Hotelling","year":"1992","venue":"Breakthroughs in Statistics","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Hoyer et al., 2008] Patrik O. Hoyer, Dominik Janzing, Joris M. Mooij, Jonas Peters, and Bernhard Schölkopf. Nonlinear causal discovery with additive noise models. In Proc. of NeurIPS, 2008.","title":"Nonlinear causal discovery with additive noise models","authors":"Patrik O Hoyer, Dominik Janzing, M Joris, Jonas Mooij, Bernhard Peters, Schölkopf","year":"2008","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Huang et al., 2018] Xun Huang, Ming-Yu Liu, Serge Be- longie, and Jan Kautz. Multimodal unsupervised image- to-image translation. In Proc. of ECCV, 2018.","title":"Multimodal unsupervised imageto-image translation","authors":"Xun Huang, Ming-Yu Liu, Serge Belongie, Jan Kautz","year":"2018","venue":"Proc. of ECCV","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Hwang et al., 2020] HyeongJoo Hwang, Geon-Hyeong Kim, Seunghoon Hong, and Kee-Eung Kim. Variational interaction information maximization for cross-domain disentanglement. In Proc. of NeurIPS, 2020.","title":"Variational interaction information maximization for cross-domain disentanglement","authors":"Hwang","year":"2020","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Hyvärinen and Morioka, 2016] Aapo Hyvärinen and Hi- roshi Morioka. Unsupervised feature extraction by time- contrastive learning and nonlinear ICA. In Proc. of NeurIPS, 2016.","title":"Unsupervised feature extraction by timecontrastive learning and nonlinear ICA","authors":"Morioka Hyvärinen, Aapo Hyvärinen, Hiroshi Morioka","year":"2016","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Hyvärinen and Morioka, 2017] Aapo Hyvärinen and Hi- roshi Morioka. Nonlinear ICA of temporally dependent stationary sources. In Proc. of AISTATS, 2017.","title":"Nonlinear ICA of temporally dependent stationary sources","authors":"Morioka Hyvärinen, Aapo Hyvärinen, Hiroshi Morioka","year":"2017","venue":"Proc. of AISTATS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Hyvärinen and Pajunen, 1999] Aapo Hyvärinen and Petteri Pajunen. Nonlinear independent component analysis: Ex- istence and uniqueness results. Neural Networks, 1999.","title":"Nonlinear independent component analysis: Existence and uniqueness results","authors":"Pajunen ; ] Aapo Hyvärinen, Petteri Hyvärinen, Pajunen","year":"1999","venue":"Neural Networks","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Hyvärinen et al., 2019] Aapo Hyvärinen, Hiroaki Sasaki, and Richard E. Turner. Nonlinear ICA using auxiliary variables and generalized contrastive learning. In Proc. of AISTATS, 2019.","title":"Nonlinear ICA using auxiliary variables and generalized contrastive learning","authors":"Aapo Hyvärinen, Hiroaki Sasaki, Richard E Turner","year":"2019","venue":"Proc. of AISTATS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Khemakhem et al., 2020] Ilyes Khemakhem, Diederik P. Kingma, Ricardo Pio Monti, and Aapo Hyvärinen. Varia- tional autoencoders and nonlinear ICA: A unifying frame- work. In Proc. of AISTATS, 2020.","title":"Variational autoencoders and nonlinear ICA: A unifying framework","authors":"Ilyes Khemakhem, P Diederik, Ricardo Pio Kingma, Aapo Monti, Hyvärinen","year":"2020","venue":"Proc. of AISTATS","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Klindt et al., 2021] David A. Klindt, Lukas Schott, Yash Sharma, Ivan Ustyuzhaninov, Wieland Brendel, Matthias Bethge, and Dylan M. Paiton. Towards nonlinear disen- tanglement in natural data with temporal sparse coding. In Proc. of ICLR, 2021.","title":"Towards nonlinear disentanglement in natural data with temporal sparse coding","authors":"David A Klindt, Lukas Schott, Yash Sharma, Ivan Ustyuzhaninov, Wieland Brendel, Matthias Bethge, Dylan M Paiton","year":"2021","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Lai and Fyfe, 2000] P. L. Lai and C. Fyfe. Kernel and non- linear canonical correlation analysis. International Jour- nal of Neural Systems, 2000.","title":"Kernel and nonlinear canonical correlation analysis","authors":"P L Lai, C Fyfe","year":"2000","venue":"International Journal of Neural Systems","doi":null,"url":null,"internal_url":null},{"position":22,"raw":"[Li et al., 2016] Sheng Li, Yaliang Li, and Yun Fu. Multi- view time series classification: A discriminative bilinear projection approach. In Proc. of CIKM, 2016.","title":"Multiview time series classification: A discriminative bilinear projection approach","authors":"Sheng Li, Yaliang Li, Yun Fu","year":"2016","venue":"Proc. of CIKM","doi":null,"url":null,"internal_url":null},{"position":23,"raw":"[Li et al., 2020] Yunzhu Li, Antonio Torralba, Anima Anandkumar, Dieter Fox, and Animesh Garg. Causal discovery in physical systems from videos. In Proc. of NeurIPS, 2020.","title":"Causal discovery in physical systems from videos","authors":"Li","year":"2020","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":24,"raw":"[Luo et al., 2022] Dixin Luo, Hongteng Xu, and Lawrence Carin. Differentiable Hierarchical Optimal Transport for Robust Multi-View Learning. IEEE Transactions on Pat- tern Analysis and Machine Intelligence, 2022.","title":"Differentiable Hierarchical Optimal Transport for Robust Multi-View Learning","authors":"Luo","year":"2022","venue":"IEEE Transactions on Pattern Analysis and Machine Intelligence","doi":null,"url":null,"internal_url":null},{"position":25,"raw":"[Lyu et al., 2021] Qi Lyu, Xiao Fu, Weiran Wang, and Song- tao Lu. Understanding latent correlation-based multiview learning and self-supervision: An identifiability perspec- tive. In Proc. of ICLR, 2021.","title":"Understanding latent correlation-based multiview learning and self-supervision: An identifiability perspective","authors":"Qi Lyu, Xiao Fu, Weiran Wang, Songtao Lu","year":"2021","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":26,"raw":"[Pearl, 2000] Judea Pearl. Causality: Models, Reasoning, and Inference. Cambridge University Press, Cambridge, U.K. ; New York, 2000.","title":"Judea Pearl. Causality: Models, Reasoning, and Inference","authors":"Pearl","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":27,"raw":"[Peyré and Cuturi, 2019] Gabriel Peyré and Marco Cuturi. Computational optimal transport. Found. Trends Mach. Learn., 11(5-6):355-607, 2019.","title":"Computational optimal transport","authors":"Gabriel Peyré, Marco Cuturi","year":"2019","venue":"Found. Trends Mach. Learn","doi":null,"url":null,"internal_url":null},{"position":28,"raw":"[Richard et al., 2021] Hugo Richard, Pierre Ablin, Bertrand Thirion, Alexandre Gramfort, and Aapo Hyvarinen. Shared Independent Component Analysis for Multi- Subject Neuroimaging. In Proc. of NeurIPS, 2021.","title":"Shared Independent Component Analysis for Multi-Subject Neuroimaging","authors":"Richard","year":"2021","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":29,"raw":"[Sorrenson et al., 2020] Peter Sorrenson, Carsten Rother, and Ullrich Köthe. Disentanglement by nonlinear ICA with general incompressible-flow networks (GIN). In Proc. of ICLR, 2020.","title":"Disentanglement by nonlinear ICA with general incompressible-flow networks (GIN)","authors":"Peter Sorrenson, Carsten Rother, Ullrich Köthe","year":"2020","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":30,"raw":"[Spirtes et al., 2000] Peter Spirtes, Clark N. Glymour, Richard Scheines, and David Heckerman. Causation, Pre- diction, and Search. MIT Press, 2000.","title":"Causation, Prediction, and Search","authors":"Peter Spirtes, Clark N Glymour, Richard Scheines, David Heckerman","year":"2000","venue":null,"doi":null,"url":null,"internal_url":null},{"position":31,"raw":"[Tsamardinos et al., 2006] Ioannis Tsamardinos, Laura E. Brown, and Constantin F. Aliferis. The max-min hill- climbing Bayesian network structure learning algorithm. Machine Learning, 2006.","title":"The max-min hillclimbing Bayesian network structure learning algorithm","authors":"Ioannis Tsamardinos, Laura E Brown, Constantin F Aliferis","year":"2006","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":32,"raw":"[Urai et al., 2022] Anne E. Urai, Brent Doiron, Andrew M. Leifer, and Anne K. Churchland. Large-scale neural recordings call for new insights to link brain and behav- ior. Nature Neuroscience, 2022.","title":"Large-scale neural recordings call for new insights to link brain and behavior","authors":"Anne E Urai, Brent Doiron, Andrew M Leifer, Anne K Churchland","year":"2022","venue":"Nature Neuroscience","doi":null,"url":null,"internal_url":null},{"position":33,"raw":"[van den Oord et al., 2018] Aäron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with con- trastive predictive coding. CoRR, abs/1807.03748, 2018.","title":"Representation learning with contrastive predictive coding","authors":"Oord Van Den","year":"2018","venue":null,"doi":null,"url":null,"internal_url":null},{"position":34,"raw":"[Wang et al., 2015] Weiran Wang, Raman Arora, Karen Livescu, and Jeff A. Bilmes. On deep multi-view repre- sentation learning. In Proc. of ICML, 2015.","title":"On deep multi-view representation learning","authors":"Wang","year":"2015","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null},{"position":35,"raw":"[Wang et al., 2020] Heyuan Wang, Tengjiao Wang, and Yi Li. Incorporating expert-based investment opinion sig- nals in stock prediction: A deep learning framework. In Proc. of AAAI, 2020.","title":"Incorporating expert-based investment opinion signals in stock prediction: A deep learning framework","authors":"Wang","year":"2020","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":36,"raw":"[Yao et al., 2021] Weiran Yao, Yuewen Sun, Alex Ho, Changyin Sun, and Kun Zhang. Learning Temporally Causal Latent Processes from General Temporal Data. In Proc. of ICLR, 2021.","title":"Learning Temporally Causal Latent Processes from General Temporal Data","authors":"Yao","year":"2021","venue":"Proc. of ICLR","doi":null,"url":null,"internal_url":null},{"position":37,"raw":"[Zhang, 2008] Jiji Zhang. On the completeness of orienta- tion rules for causal discovery in the presence of latent confounders and selection bias. Artificial Intelligence, 2008.","title":"On the completeness of orientation rules for causal discovery in the presence of latent confounders and selection bias","authors":"Jiji Zhang, Zhang","year":"2008","venue":"Artificial Intelligence","doi":null,"url":null,"internal_url":null},{"position":38,"raw":"[Zhao et al., 2017] Handong Zhao, Zhengming Ding, and Yun Fu. Multi-view clustering via deep matrix factoriza- tion. In Proc. of AAAI, 2017.","title":"Multi-view clustering via deep matrix factorization","authors":"Handong Zhao, Zhengming Ding, Yun Fu","year":"2017","venue":"Proc. of AAAI","doi":null,"url":null,"internal_url":null},{"position":39,"raw":"[Zheng et al., 2018] Xun Zheng, Bryon Aragam, Pradeep Ravikumar, and Eric P. Xing. Dags with NO TEARS: continuous optimization for structure learning. In Proc. of NeurIPS, 2018.","title":"Dags with NO TEARS: continuous optimization for structure learning","authors":"Xun Zheng, Bryon Aragam, Pradeep Ravikumar, Eric P Xing","year":"2018","venue":"Proc. of NeurIPS","doi":null,"url":null,"internal_url":null},{"position":40,"raw":"[Zimmermann et al., 2021] Roland S. Zimmermann, Yash Sharma, Steffen Schneider, Matthias Bethge, and Wieland Brendel. Contrastive learning inverts the data generating process. In Proc. of ICML, 2021.","title":"Contrastive learning inverts the data generating process","authors":"Roland S Zimmermann, Yash Sharma, Steffen Schneider, Matthias Bethge, Wieland Brendel","year":"2021","venue":"Proc. of ICML","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}