{"type":"ScholarlyArticle","title":"Scalable Bayesian Non-linear Matrix Completion","authors":[{"name":"Xiangju Qin","slug":"xiangju-qin","given":"Xiangju","family":"Qin","affiliation":"Aalto University , 00076 Espoo , Finland; Currently at Institute for Molecular Medicine Finland (FIMM) , University of Helsinki."},{"name":"Paul Blomstedt","slug":"paul-blomstedt","given":"Paul","family":"Blomstedt","affiliation":"Department of Computer Science , Helsinki Institute for Information Technology HIIT ,"},{"name":"Samuel Kaski","slug":"samuel-kaski","given":"Samuel","family":"Kaski","affiliation":"Department of Computer Science , Helsinki Institute for Information Technology HIIT ,"}],"abstract":"Matrix completion aims to predict missing elements in a partially observed data matrix which in typical applications, such as collaborative filtering, is large and extremely sparsely observed. A standard solution is matrix factorization, which predicts unobserved entries as linear combinations of latent variables. We generalize to non-linear combinations in massive-scale matrices. Bayesian approaches have been proven beneficial in linear matrix completion, but not applied in the more general non-linear case, due to limited scalability. We introduce a Bayesian non-linear matrix completion algorithm, which is based on a recent Bayesian formulation of Gaussian process latent variable models. To solve the challenges regarding scalability and computation, we propose a data-parallel distributed computational  approach with a restricted communication scheme.  We evaluate our method on challenging out-of-matrix prediction tasks using both simulated and real-world data.","doi":"10.24963/ijcai.2019/454","url":"https://ijcai.org/proceedings/2019/454","pdf":"https://ijcai.org/proceedings/2019/0454.pdf","markdown":"https://ijcai.org/proceedings/2019/454.md","bibtex":"https://ijcai.org/proceedings/2019/bibtex/454","volume":{"slug":"2019","title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2019","url":"https://ijcai.org/proceedings/2019/","year":2019,"location":"Macao","dates":"10-16 August 2019","editors":"Sarit Kraus","isbn":"978-0-9992411-4-1","publisher":"International Joint Conferences on Artificial Intelligence Organization","doi_prefix":"10.24963/ijcai.2019"},"section":"Machine Learning M-Z","track":"Main track","number":454,"pages":{"first":3275,"last":3281},"keywords":["Machine Learning: Probabilistic Machine Learning","Machine Learning: Recommender Systems","Machine Learning Applications: Big data ; Scalability"],"videos":[],"references":[{"position":1,"raw":"[Cao and Fleet, 2014] Yanshuai Cao and David J. Fleet. Generalized product of experts for automatic and prin- cipled fusion of Gaussian process predictions. In Mod- ern Nonparametrics 3: Automating the Learning Pipeline workshop at NIPS, 2014.","title":"Generalized product of experts for automatic and principled fusion of Gaussian process predictions","authors":"Fleet Cao, Yanshuai Cao, David J Fleet","year":"2014","venue":"Modern Nonparametrics 3: Automating the Learning Pipeline workshop at NIPS","doi":null,"url":null,"internal_url":null},{"position":2,"raw":"[Dai et al., 2014] Zhenwen Dai, Andreas Damianou, James Hensman, and Neil Lawrence. Gaussian process models with parallelization and GPU acceleration. arXiv preprint arXiv:1410.4984, 2014.","title":"Gaussian process models with parallelization and GPU acceleration","authors":"Dai","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":3,"raw":"[Damianou et al., 2012] Andreas Damianou, Carl Henrik Ek, Michalis K. Titsias, and Neil D. Lawrence. Manifold relevance determination. In ICML, 2012.","title":"Manifold relevance determination","authors":"Andreas Damianou, Carl Henrik Ek, Michalis K Titsias, Neil D Lawrence","year":"2012","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":4,"raw":"[Damianou et al., 2016] Andreas C. Damianou, Michalis K. Titsias, and Neil D. Lawrence. Variational inference for la- tent variables and uncertain inputs in Gaussian processes. JMLR, 17(42):1-62, 2016.","title":"Variational inference for latent variables and uncertain inputs in Gaussian processes","authors":"Andreas C Damianou, K Michalis, Neil D Titsias, Lawrence","year":"2016","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":5,"raw":"[Deisenroth and Ng, 2015] Marc Deisenroth and Jun Wei Ng. Distributed gaussian processes. In ICML, pages 1481- 1490, 2015.","title":"Distributed gaussian processes","authors":"Ng Deisenroth, Marc Deisenroth, Jun Wei Ng","year":"2015","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":6,"raw":"[Gal et al., 2014] Yarin Gal, Mark van der Wilk, and Carl Edward Rasmussen. Distributed variational inference in sparse Gaussian process regression and latent variable models. In NIPS, pages 3257-3265, 2014.","title":"Distributed variational inference in sparse Gaussian process regression and latent variable models","authors":"Yarin Gal, Carl Edward Mark Van Der Wilk, Rasmussen","year":"2014","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":7,"raw":"[Girard et al., 2003] Agathe Girard, Carl Edward Ras- mussen, Joaquin Quinonero Candela, and Roderick Murray-Smith. Gaussian process priors with uncertain in- puts application to multiple-step ahead time series fore- casting. In NIPS, pages 545-552, 2003.","title":"Gaussian process priors with uncertain inputs application to multiple-step ahead time series forecasting","authors":"Agathe Girard, Carl Edward Rasmussen, Joaquin Quinonero Candela, Roderick Murray-Smith","year":"2003","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":8,"raw":"[Lawrence and Urtasun, 2009] Neil D. Lawrence and Raquel Urtasun. Non-linear matrix factorization with Gaussian processes. In ICML, pages 601-608, 2009.","title":"Non-linear matrix factorization with Gaussian processes","authors":"Neil D Lawrence, Raquel Urtasun","year":"2009","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":9,"raw":"[Lawrence, 2005] Neil Lawrence. Probabilistic non-linear principal component analysis with Gaussian process latent variable models. JMLR, 6(Nov):1783-1816, 2005.","title":"Probabilistic non-linear principal component analysis with Gaussian process latent variable models","authors":"Neil Lawrence","year":"2005","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":10,"raw":"[Ng and Deisenroth, 2014] Jun W. Ng and Marc P. Deisen- roth. Hierarchical mixture-of-experts model for large- scale Gaussian process regression. arXiv preprint arXiv:1412.3078, 2014.","title":"Hierarchical mixture-of-experts model for largescale Gaussian process regression","authors":"Jun W Ng, Marc P Deisenroth","year":"2014","venue":null,"doi":null,"url":null,"internal_url":null},{"position":11,"raw":"[Qin et al., 2019] Xiangju Qin, Paul Blomstedt, Eemeli Leppäaho, Pekka Parviainen, and Samuel Kaski. Dis- tributed Bayesian matrix factorization with limited com- munication. Machine Learning, pages 1-26, 2019.","title":"Distributed Bayesian matrix factorization with limited communication","authors":"Qin","year":"2019","venue":"Machine Learning","doi":null,"url":null,"internal_url":null},{"position":12,"raw":"[Quiñonero Candela and Rasmussen, 2005] Joaquin Quiñonero Candela and Carl Edward Rasmussen. A unifying view of sparse approximate gaussian process regression. JMLR, 6:1939-1959, 2005.","title":"Joaquin Quiñonero Candela and Carl Edward Rasmussen. A unifying view of sparse approximate gaussian process regression","authors":"Quiñonero Candela, Rasmussen","year":"2005","venue":"JMLR","doi":null,"url":null,"internal_url":null},{"position":13,"raw":"[Salakhutdinov and Mnih, 2008a] Ruslan Salakhutdinov and Andriy Mnih. Bayesian probabilistic matrix factorization using Markov chain Monte Carlo. In ICML, pages 880- 887, 2008.","title":"Ruslan Salakhutdinov and Andriy Mnih. Bayesian probabilistic matrix factorization using Markov chain Monte Carlo","authors":"Mnih Salakhutdinov","year":"2008","venue":"ICML","doi":null,"url":null,"internal_url":null},{"position":14,"raw":"[Salakhutdinov and Mnih, 2008b] Ruslan Salakhutdinov and Andriy Mnih. Probabilistic matrix factorization. In NIPS, 2008.","title":"Ruslan Salakhutdinov and Andriy Mnih. Probabilistic matrix factorization","authors":"Mnih Salakhutdinov","year":"2008","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":15,"raw":"[Simm et al., 2015] Jaak Simm, Adam Arany, Pooya Zakeri, Tom Haber, Jörg K. Wegner, Vladimir Chupakhin, Hugo Ceulemans, and Yves Moreau. Macau: Scalable bayesian multi-relational factorization with side information using mcmc. arXiv preprint arXiv:1509.04610, 2015.","title":"Macau: Scalable bayesian multi-relational factorization with side information using mcmc","authors":"Jaak Simm, Adam Arany, Pooya Zakeri, Tom Haber, Jörg K Wegner, Vladimir Chupakhin, Hugo Ceulemans, Yves Moreau","year":"2015","venue":null,"doi":null,"url":null,"internal_url":null},{"position":16,"raw":"[Snelson and Ghahramani, 2006] Edward Snelson and Zoubin Ghahramani. Sparse Gaussian processes using pseudo-inputs. In NIPS, pages 1257-1264. MIT Press, 2006.","title":"Sparse Gaussian processes using pseudo-inputs","authors":"Ghahramani Snelson, Edward Snelson, Zoubin Ghahramani","year":"2006","venue":"NIPS","doi":null,"url":null,"internal_url":null},{"position":17,"raw":"[Sun et al., 2017] Jiangming Sun, Nina Jeliazkova, Vladimir Chupakhin, Jose-Felipe Golib-Dzib, Ola Engkvist, Lars Carlsson, Jörg Wegner, Hugo Ceulemans, Ivan Georgiev, Vedrin Jeliazkov, Nikolay Kochev, Thomas J. Ashby, and Hongming Chen. ExCAPE-DB: an integrated large scale dataset facilitating big data analysis in chemogenomics. Journal of Cheminformatics, 9(1):17:1-17:9, 2017.","title":"ExCAPE-DB: an integrated large scale dataset facilitating big data analysis in chemogenomics","authors":"Sun","year":"2017","venue":"Journal of Cheminformatics","doi":null,"url":null,"internal_url":null},{"position":18,"raw":"[Titsias and Lawrence, 2010] Michalis Titsias and Neil Lawrence. Bayesian Gaussian process latent variable model. In AISTATS, pages 844-851, 2010.","title":"Michalis Titsias and Neil Lawrence. Bayesian Gaussian process latent variable model","authors":"Lawrence Titsias","year":"2010","venue":"AISTATS","doi":null,"url":null,"internal_url":null},{"position":19,"raw":"[Tresp, 2000] Volker Tresp. A Bayesian committee machine. Neural Computation, 12(11):2719-2741, 2000.","title":"A Bayesian committee machine","authors":"Tresp Volker","year":"2000","venue":"Neural Computation","doi":null,"url":null,"internal_url":null},{"position":20,"raw":"[Vander Aa et al., 2016] Tom Vander Aa, Imen Chakroun, and Tom Haber. Distributed Bayesian probabilistic ma- trix factorization. In 2016 IEEE International Conference on Cluster Computing, pages 346-349, 2016.","title":"Distributed Bayesian probabilistic matrix factorization","authors":"Aa Vander","year":"2016","venue":"IEEE International Conference on Cluster Computing","doi":null,"url":null,"internal_url":null},{"position":21,"raw":"[Yao et al., 2011] Angela Yao, Juergen Gall, Luc V Gool, and Raquel Urtasun. Learning probabilistic non-linear la- tent variable models for tracking complex activities. In NIPS, pages 1359-1367, 2011.","title":"Learning probabilistic non-linear latent variable models for tracking complex activities","authors":"Angela Yao, Juergen Gall, Luc V Gool, Raquel Urtasun","year":"2011","venue":"NIPS","doi":null,"url":null,"internal_url":null}],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}