Publications
14 results found
L'Huillier A, Travis L, Castillo I, et al., 2023, Semiparametric inference using fractional posteriors, Journal of Machine Learning Research, ISSN: 1532-4435
We establish a general Bernstein–von Mises theorem for approximately linear semiparametric functionals of fractional posterior distributions based on nonparametric priors. This is illustrated in a number of nonparametric settings and for different classes of prior distributions, including Gaussian process priors. We show that fractional posterior credible sets can provide reliable semiparametric uncertainty quantification, but have inflated size. To remedy this, we further propose a shifted-and-rescaled fractional posterior set that is an efficient confidence set having optimal size under regularity conditions. As part of our proofs,we also refine existing contraction rate results for fractional posteriors by sharpening the dependence of the rate on the fractional exponent.
Travis L, Ray K, 2023, Pointwise uncertainty quantification for sparse variational Gaussian process regression with a Brownian motion prior, 37th Conference on Neural Information Processing Systems (NeurIPS 2023), ISSN: 1049-5258
We study pointwise estimation and uncertainty quantification for a sparse variational Gaussian process method with eigenvector inducing variables. For a rescaledBrownian motion prior, we derive theoretical guarantees and limitations for the frequentist size and coverage of pointwise credible sets. For sufficiently many inducing variables, we precisely characterize the asymptotic frequentist coverage, deducing when credible sets from this variational method are conservative and when overconfident/misleading. We numerically illustrate the applicability of our results and discuss connections with other common Gaussian process priors.
Giordano M, Ray K, Schmidt-Hieber J, 2022, On the inability of Gaussian process regression to optimally learn compositional functions, 36th Conference on Neural Information Processing Systems (NeurIPS 2022), Pages: 1-13, ISSN: 1049-5258
We rigorously prove that deep Gaussian process priors can outperform Gaussian process priors if the target function has a compositional structure. To this end, we study information-theoretic lower bounds for posterior contraction rates for Gaussian process regression in a continuous regression model. We show that if the true function is a generalized additive function, then the posterior based on any mean-zero Gaussian process can only recover the truth at a rate that is strictly slower than the minimax rate by a factor that is polynomially suboptimal in the sample size n.
Wang J, Ray K, Brito-Parada P, et al., 2022, A Bayesian approach for the modelling of material stocks and flows with incomplete data, ArXiv
Material Flow Analysis (MFA) is used to quantify and understand the lifecycles of materials from production to end of use, which enables environmental,social and economic impacts and interventions. MFA is challenging as availabledata is often limited and uncertain, giving rise to an underdetermined systemwith an infinite number of solutions when attempting to calculate the values ofall stocks and flows in the system. Bayesian statistics is an effective way toaddress these challenges as it rigorously quantifies uncertainty in the dataand propagates it in a system flow model to provide the probabilitiesassociated with model solutions. Furthermore, the Bayesian approach provides anatural way to incorporate useful domain knowledge about the system through theelicitation of the prior distribution. This paper presents a novel Bayesian approach to MFA. We propose a mass basedframework that directly models the flow and change in stock variables in thesystem, including systems with simultaneous presence of stocks anddisaggregation of processes. The proposed approach is demonstrated on a globalaluminium cycle, under a scenario where there is a shortage of data, coupledwith weakly informative priors that only require basic information on flows andchange in stocks. Bayesian model checking helps to identify inconsistencies inthe data, and the posterior distribution is used to identify the variables inthe system with the most uncertainty, which can aid data collection. Wenumerically investigate the properties of our method in simulations, and showthat in limited data settings, the elicitation of an informative prior cangreatly improve the performance of Bayesian methods, including for bothestimation accuracy and uncertainty quantification.
Giordano M, Ray K, 2022, Nonparametric Bayesian inference for reversible multi-dimensional diffusions, Annals of Statistics, Vol: 50, Pages: 2872-2898, ISSN: 0090-5364
We study nonparametric Bayesian models for reversible multidimensional diffusions with periodic drift. For continuous observation paths, reversibility is exploited to prove a general posterior contraction rate theoremfor the drift gradient vector field under approximation-theoretic conditions onthe induced prior for the invariant measure. The general theorem is appliedto Gaussian priors and p-exponential priors, which are shown to converge tothe truth at the optimal nonparametric rate over Sobolev smoothness classesin any dimension.
Ray K, Szabó B, 2022, Variational Bayes for high-dimensional linear regression with sparse priors, Journal of the American Statistical Association, Vol: 117, Pages: 1270-1281, ISSN: 0162-1459
We study a mean-field spike and slab variational Bayes (VB) approximation to Bayesian model selection priors in sparse high-dimensional linear regression. Under compatibility conditions on the design matrix, oracle inequalities are derived for the mean-field VB approximation, implying that it converges to the sparse truth at the optimal rate and gives optimal prediction of the response vector. The empirical performance of our algorithm is studied, showing that it works comparably well as other state-of-the-art Bayesian variable selection methods. We also numerically demonstrate that the widely used coordinate-ascent variational inference (CAVI) algorithm can be highly sensitive to the parameter updating order, leading to potentially poor performance. To mitigate this, we propose a novel prioritized updating scheme that uses a data-driven updating order and performs better in simulations. The variational algorithm is implemented in the R package sparsevb.
Komodromos M, Aboagye EO, Evangelou M, et al., 2022, Variational Bayes for high-dimensional proportional hazards models with applications within gene expression, BIOINFORMATICS, Vol: 38, Pages: 3918-3926, ISSN: 1367-4803
Motivation:Few Bayesian methods for analyzing high-dimensional sparse survival data provide scalable variable selection, effect estimation and uncertainty quantification. Such methods often either sacrifice uncertainty quantification by computing maximum a posteriori estimates, or quantify the uncertainty at high (unscalable) computational expense.Results:We bridge this gap and develop an interpretable and scalable Bayesian proportional hazards model for prediction and variable selection, referred to as SVB. Our method, based on a mean-field variational approximation, overcomes the high computational cost of MCMC whilst retaining useful features, providing a posterior distribution for the parameters and offering a natural mechanism for variable selection via posterior inclusion probabilities. The performance of our proposed method is assessed via extensive simulations and compared against other state-of-the-art Bayesian variable selection methods, demonstrating comparable or better performance. Finally, we demonstrate how the proposed method can be used for variable selection on two transcriptomic datasets with censored survival outcomes, and how the uncertainty quantification offered by our method can be used to provide an interpretable assessment of patient risk.Availability and implementation:our method has been implemented as a freely available R package survival.svb (https://github.com/mkomod/survival.svb).
Ray K, van der Vaart A, 2021, On the Bernstein-von Mises theorem for the Dirichlet process, Electronic Journal of Statistics, Vol: 15, Pages: 2224-2246, ISSN: 1935-7524
We establish that Laplace transforms of the posterior Dirichlet process converge to those of the limiting Brownian bridge process in a neighbourhood about zero, uniformly over Glivenko-Cantelli function classes. For real-valued random variables and functions of bounded variation, we strengthen this result to hold for all real numbers. This last result is proved via an explicit strong approximation coupling inequality.
Ray K, Szabo B, Clara G, 2020, Spike and slab variational Bayes for high dimensional logistic regression, 34th Conference on Neural Information Processing Systems
Variational Bayes (VB) is a popular scalable alternative to Markov chain Monte Carlo for Bayesian inference. We study a mean-field spike and slab VB approximation of widely used Bayesian model selection priors in sparse high-dimensional logistic regression. We provide non-asymptotic theoretical guarantees for the VB posterior in both ℓ2 and prediction loss for a sparse truth, giving optimal (minimax) convergence rates. Since the VB algorithm does not depend on the unknown truth to achieve optimality, our results shed light on effective prior choices. We confirm the improved performance of our VB algorithm over common sparse VB approaches in a numerical study.
Ray K, van der Vaart A, 2020, Semiparametric Bayesian causal inference, Annals of Statistics, Vol: 48, Pages: 2999-3020, ISSN: 0090-5364
We develop a semiparametric Bayesian approach for estimatingthe mean response in a missing data model with binary outcomesand a nonparametrically modelled propensity score. Equivalently, weestimate the causal effect of a treatment, correcting nonparamet-rically for confounding. We show that standard Gaussian processpriors satisfy a semiparametric Bernstein–von Mises theorem undersmoothness conditions. We further propose a novel propensity score-dependent prior that provides efficient inference under strictly weakerconditions. We also show that it is theoretically preferable to modelthe covariate distribution with a Dirichlet process or Bayesian boot-strap, rather than modelling its density.
Nickl R, Ray K, 2020, Nonparametric statistical inference for drift vector fields of multi-dimensional diffusions, Annals of Statistics, Vol: 48, Pages: 1383-1408, ISSN: 0090-5364
The problem of determining a periodic Lipschitz vector fieldb=(b1,...,bd) from an observed trajectory of the solution (Xt: 0≤t≤T) of the multi-dimensional stochastic differential equationdXt=b(Xt)dt+dWt, t≥0,whereWtis a standardd-dimensional Brownian motion, is consid-ered. Convergence rates of a penalised least squares estimator, whichequals the maximum a posteriori (MAP) estimate corresponding to ahigh-dimensional Gaussian product prior, are derived. These resultsare deduced from corresponding contraction rates for the associatedposterior distributions. The rates obtained are optimal up to log-factors inL2-loss in any dimension, and also for supremum norm losswhend≤4. Further, whend≤3, nonparametric Bernstein-von Misestheorems are proved for the posterior distributions ofb. From this wededuce functional central limit theorems for the implied estimatorsof the invariant measureμb. The limiting Gaussian process distribu-tions have a covariance structure that is asymptotically optimal froman information-theoretic point of view.
Mariucci E, Ray K, Szabó B, 2020, A Bayesian nonparametric approach to log-concave density estimation, Bernoulli, Vol: 26, Pages: 1070-1097, ISSN: 1350-7265
The estimation of a log-concave density on R is a canonical problem in the area of shape-constrained nonparametric inference. We present a Bayesian nonparametric approach to this problem based on an exponentiated Dirichlet process mixture prior and show that the posterior distribution converges to the log-concave truth at the (near-) minimax rate in Hellinger distance. Our proof proceeds by establishing a general contraction result based on the log-concave maximum likelihood estimator that prevents the need for further metric entropy calculations. We further present computationally more feasible approximations and both an empirical and hierarchical Bayes approach. All priors are illustrated numerically via simulations.
Ray K, Schmidt-Hieber J, 2018, The Le Cam distance between density estimation, Poisson processes and Gaussian white noise, Mathematical Statistics and Learning, Vol: 1, Pages: 101-170, ISSN: 2520-2316
It is well known that density estimation on the unit interval is asymptotically equivalent to a Gaussian white noise experiment, provided the densities have Hölder smoothness larger than 1/2 and are uniformly bounded away from zero. We derive matching lower and constructive upper bounds for the Le Cam deficiencies between these experiments, with explicit dependence on both the sample size and the size of the densities in the parameter space. As a consequence, we derive sharp conditions on how small the densities can be for asymptotic equivalence to hold. The related case of Poisson intensity estimation is also treated.
Ray K, 2017, Adaptive Bernstein–von Mises theorems in Gaussian white noise, Annals of Statistics, Vol: 45, Pages: 2511-2536, ISSN: 0090-5364
We investigate Bernstein–von Mises theorems for adaptive nonparametric Bayesian procedures in the canonical Gaussian white noise model. Weconsider both a Hilbert space and multiscale setting with applications in L2and L∞, respectively. This provides a theoretical justification for plug-in procedures, for example the use of certain credible sets for sufficiently smoothlinear functionals. We use this general approach to construct optimal frequentist confidence sets based on the posterior distribution. We also provide simulations to numerically illustrate our approach and obtain a visual representation of the geometries involved.
This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.