{"type":"ScholarlyArticle","title":"Scalable Gaussian Process Regression Using Deep Neural Networks","authors":[{"name":"Wen-bing Huang","slug":"wen-bing-huang","given":"Wen-bing","family":"Huang","affiliation":""},{"name":"Deli Zhao","slug":"deli-zhao","given":"Deli","family":"Zhao","affiliation":""},{"name":"Fuchun Sun","slug":"fuchun-sun","given":"Fuchun","family":"Sun","affiliation":""},{"name":"Huaping Liu","slug":"huaping-liu","given":"Huaping","family":"Liu","affiliation":""},{"name":"Edward Y. Chang","slug":"edward-y-chang","given":"Edward Y.","family":"Chang","affiliation":""}],"abstract":"We propose a scalable Gaussian process model for regression by applying a deep neural network as the feature-mapping function. We first pretrain the deep neural network with a stacked denoising auto-encoder in an unsupervised way. Then, we perform a Bayesian linear regression on the top layer of the pre-trained deep network. The resulting model, Deep-Neural-Network-based Gaussian Process (DNN-GP), can learn much more meaningful representation of the data by the finite-dimensional but deep-layered feature-mapping function. Unlike standard Gaussian processes, our model scales well with the size of the training set due to the avoidance of kernel matrix inversion. Moreover, we present a mixture of DNN-GPs to further improve the regression performance. For the experiments on three representative large datasets, our proposed models significantly outperform the state-of-the-art algorithms of Gaussian process regression.","doi":null,"url":"https://ijcai.org/proceedings/2015/503","pdf":"https://ijcai.org/proceedings/2015/0503.pdf","markdown":"https://ijcai.org/proceedings/2015/503.md","bibtex":"https://ijcai.org/proceedings/2015/bibtex/503","volume":{"slug":"2015","title":"Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence","short_title":"IJCAI 2015","url":"https://ijcai.org/proceedings/2015/","year":2015,"location":"Buenos Aires, Argentina","dates":"25–31 July 2015","editors":"Qiang Yang and Michael Wooldridge","isbn":"978-1-57735-738-4","publisher":"International Joint Conferences on Artificial Intelligence","doi_prefix":""},"section":"Special Track on Machine Learning","track":"Special Track on Machine Learning","number":503,"pages":{"first":3576,"last":3581},"keywords":[],"videos":[],"references":[],"license_note":"Copyright International Joint Conferences on Artificial Intelligence."}