Worst-Case Bounds for Gaussian Process Models

Citation:

S. M. Kakade, M. W. Seeger, and D. P. Foster, Worst-Case Bounds for Gaussian Process Models. Advances in Neural Information Processing Systems 18 (NIPS 2005): , 2006.

Abstract:

We present a competitive analysis of some non-parametric Bayesian al- gorithms in a worst-case online learning setting, where no probabilistic assumptions about the generation of the data are made. We consider models which use a Gaussian process prior (over the space of all func- tions) and provide bounds on the regret (under the log loss) for com- monly used non-parametric Bayesian algorithms — including Gaussian regression and logistic regression — which show how these algorithms can perform favorably under rather general conditions. These bounds ex- plicitly handle the infinite dimensionality of these non-parametric classes in a natural way. We also make formal connections to the minimax and minimum description length (MDL) framework. Here, we show precisely how Bayesian Gaussian regression is a minimax strategy.

Publisher's Version

See also: 2006
Last updated on 10/14/2021