---
title: "Scalable Gaussian Process Regression Using Deep Neural Networks"
authors:
  - "Wen-bing Huang"
  - "Deli Zhao"
  - "Fuchun Sun"
  - "Huaping Liu"
  - "Edward Y. Chang"
doi: ""
url: "https://ijcai.org/proceedings/2015/503"
pdf: "https://ijcai.org/proceedings/2015/0503.pdf"
venue: "Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence"
year: 2015
pages: "3576-3581"
track: "Special Track on Machine Learning"
keywords:
---

# Scalable Gaussian Process Regression Using Deep Neural Networks

Wen-bing Huang, Deli Zhao, Fuchun Sun, Huaping Liu, Edward Y. Chang

*Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence*, pages 3576-3581.

## Abstract

We propose a scalable Gaussian process model for regression by applying a deep neural network as the feature-mapping function. We first pretrain the deep neural network with a stacked denoising auto-encoder in an unsupervised way. Then, we perform a Bayesian linear regression on the top layer of the pre-trained deep network. The resulting model, Deep-Neural-Network-based Gaussian Process (DNN-GP), can learn much more meaningful representation of the data by the finite-dimensional but deep-layered feature-mapping function. Unlike standard Gaussian processes, our model scales well with the size of the training set due to the avoidance of kernel matrix inversion. Moreover, we present a mixture of DNN-GPs to further improve the regression performance. For the experiments on three representative large datasets, our proposed models significantly outperform the state-of-the-art algorithms of Gaussian process regression.