Search or filter publications

Filter by type:

Filter by publication type

Filter by year:



  • Showing results for:
  • Reset all filters

Search results

    Filippi S, Holmes C, 2016,

    A Bayesian nonparametric approach to testing for dependence between random variables

    , Bayesian Analysis, Vol: 12, Pages: 919-938, ISSN: 1931-6690

    Nonparametric and nonlinear measures of statistical dependence between pairsof random variables are important tools in modern data analysis. In particularthe emergence of large data sets can now support the relaxation of linearityassumptions implicit in traditional association scores such as correlation.Here we describe a Bayesian nonparametric procedure that leads to a tractable,explicit and analytic quantification of the relative evidence for dependence vsindependence. Our approach uses Polya tree priors on the space of probabilitymeasures which can then be embedded within a decision theoretic test fordependence. Polya tree priors can accommodate known uncertainty in the form ofthe underlying sampling distribution and provides an explicit posteriorprobability measure of both dependence and independence. Well known advantagesof having an explicit probability measure include: easy comparison of evidenceacross different studies; encoding prior information; quantifying changes independence across different experimental conditions, and; the integration ofresults within formal decision analysis.

    Filippi S, Holmes CC, Nieto-Barajas LE, 2016,

    Scalable Bayesian nonparametric measures for exploring pairwise dependence via Dirichlet Process Mixtures

    , Electronic Journal of Statistics, Vol: 10, Pages: 3338-3354, ISSN: 1935-7524

    In this article we propose novel Bayesian nonparametric methods using Dirichlet Process Mixture (DPM) models for detecting pairwise dependence between random variables while accounting for uncertainty in the form of the underlying distributions. A key criteria is that the procedures should scale to large data sets. In this regard we find that the formal calculation of the Bayes factor for a dependent-vs.-independent DPM joint probability measure is not feasible computationally. To address this we present Bayesian diagnostic measures for characterising evidence against a “null model” of pairwise independence. In simulation studies, as well as for a real data analysis, we show that our approach provides a useful tool for the exploratory nonparametric Bayesian analysis of large multivariate data sets.

    Flaxman S, Sejdinovic D, Cunningham JP, Filippi Set al., 2016,

    Bayesian Learning of Kernel Embeddings

    , UAI'16

    Kernel methods are one of the mainstays of machine learning, but the problemof kernel learning remains challenging, with only a few heuristics and verylittle theory. This is of particular importance in methods based on estimationof kernel mean embeddings of probability measures. For characteristic kernels,which include most commonly used ones, the kernel mean embedding uniquelydetermines its probability measure, so it can be used to design a powerfulstatistical testing framework, which includes nonparametric two-sample andindependence tests. In practice, however, the performance of these tests can bevery sensitive to the choice of kernel and its lengthscale parameters. Toaddress this central issue, we propose a new probabilistic model for kernelmean embeddings, the Bayesian Kernel Embedding model, combining a Gaussianprocess prior over the Reproducing Kernel Hilbert Space containing the meanembedding with a conjugate likelihood function, thus yielding a closed formposterior over the mean embedding. The posterior mean of our model is closelyrelated to recently proposed shrinkage estimators for kernel mean embeddings,while the posterior uncertainty is a new, interesting feature with variouspossible applications. Critically for the purposes of kernel learning, ourmodel gives a simple, closed form marginal pseudolikelihood of the observeddata given the kernel hyperparameters. This marginal pseudolikelihood caneither be optimized to inform the hyperparameter choice or fully Bayesianinference can be used.

    Gyorgy A, Szcpesvari C, 2016,

    Shifting regret, mirror descent, and matrices

    , Pages: 4324-4332

    © 2016 by the author(s). We consider the problem of online prediction in changing environments. In this framework the performance of a predictor is evaluated as the loss relative to an arbitrarily changing predictor, whose individual components come from a base class of predictors. Typical results in the literature consider different base classes (experts, linear predictors on the simplex, etc.) separately. Introducing an arbitrary mapping inside the mirror decent algorithm, we provide a framework that unifies and extends existing results. As an example, we prove new shifting regret bounds for matrix prediction problems.

    Gyorgy A, Szepesvari C, 2016,

    Shifting Regret, Mirror Descent, and Matrices

    , International Conference on Machine Learning, Publisher: Journal of Machine Learning Research, Pages: 2943-2951, ISSN: 1532-4435

    We consider the problem of online prediction inchanging environments. In this framework theperformance of a predictor is evaluated as theloss relative to an arbitrarily changing predictor,whose individual components come from a baseclass of predictors. Typical results in the literatureconsider different base classes (experts, linearpredictors on the simplex, etc.) separately.Introducing an arbitrary mapping inside the mirrordecent algorithm, we provide a frameworkthat unifies and extends existing results. As anexample, we prove new shifting regret bounds formatrix prediction problems.

    Jamisola RS, Kormushev PS, Roberts RG, Caldwell DGet al., 2016,

    Task-Space Modular Dynamics for Dual-Arms Expressed through a Relative Jacobian

    , JOURNAL OF INTELLIGENT & ROBOTIC SYSTEMS, Vol: 83, Pages: 205-218, ISSN: 0921-0296
    Joulani P, Gyorgy A, Szepesvari C, 2016,

    Delay-Tolerant Online Convex Optimization: Unified Analysis and Adaptive-Gradient Algorithms

    , Thirtieth AAAI Conference on Artificial Intelligence (AAAI-16), Publisher: AAAI

    We present a unified, black-box-style method for developingand analyzing online convex optimization (OCO) algorithmsfor full-information online learning in delayed-feedback environments.Our new, simplified analysis enables us to substantiallyimprove upon previous work and to solve a numberof open problems from the literature. Specifically, we developand analyze asynchronous AdaGrad-style algorithmsfrom the Follow-the-Regularized-Leader (FTRL) and MirrorDescentfamily that, unlike previous works, can handle projectionsand adapt both to the gradients and the delays, withoutrelying on either strong convexity or smoothness of theobjective function, or data sparsity. Our unified frameworkbuilds on a natural reduction from delayed-feedback to standard(non-delayed) online learning. This reduction, togetherwith recent unification results for OCO algorithms, allows usto analyze the regret of generic FTRL and Mirror-Descent algorithmsin the delayed-feedback setting in a unified mannerusing standard proof techniques. In addition, the reduction isexact and can be used to obtain both upper and lower boundson the regret in the delayed-feedback setting.

    Joulani P, Gyorgy A, Szepesvari C, 2016,

    A unified modular analysis of online and stochastic optimization: adaptivity, optimism, non-convexity

    , 9th NIPS Workshop on Optimization for Machine Learning

    We present a simple unified analysis of adaptive Mirror Descent (MD) and Follow-the-Regularized-Leader (FTRL) algorithms for online and stochastic optimizationin (possibly infinite-dimensional) Hilbert spaces. The analysis is modular inthe sense that it completely decouples the effect of possible assumptions on theloss functions (such as smoothness, strong convexity, and non-convexity) andon the optimization regularizers (such as strong convexity, non-smooth penaltiesin composite-objective learning, and non-monotone step-size sequences). Wedemonstrate the power of this decoupling by obtaining generalized algorithms andimproved regret bounds for the so-called “adaptive optimistic online learning” set-ting. In addition, we simplify and extend a large body of previous work, includingseveral various AdaGrad formulations, composite-objective and implicit-updatealgorithms. In all cases, the results follow as simple corollaries within few linesof algebra. Finally, the decomposition enables us to obtain preliminary globalguarantees for limited classes of non-convex problems.

    Kern T, Gyorgy A, 2016,

    SVRG++ with non-uniform sampling

    , 9th NIPS Workshop on Optimization for Machine Learning, Publisher: Neural Information Processing Systems Foundation, Inc.

    SVRG++ is a recent randomized optimization algorithm designed to solve non-strongly convex smooth composite optimization problems in the large data regime.In this paper we combine SVRG++ with non-uniform sampling of the data points(already present in the original SVRG algorithm), leading to an algorithm with thebest sample complexity to date and state-of-the art empirical performance. Whilethe combination and the analysis of the algorithm is admittedly straightforward,our experimental results show significant improvement over the original SVRG++method with the new method outperforming all competitors on datasets where thesmoothness of the components varies. This demonstrates that, despite its simplicityand limited novelty, this extension is important in practice.

    Kormushev P, Ahmadzadeh SR, 2016,

    Robot Learning for Persistent Autonomy

    , Handling Uncertainty and Networked Structure in Robot Control, Editors: Busoniu, Tamás, Publisher: Springer International Publishing, Pages: 3-28, ISBN: 978-3-319-26327-4

This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.

Request URL: Request URI: /respub/WEB-INF/jsp/search-t4-html.jsp Query String: id=954&limit=10&page=5&respub-action=search.html Current Millis: 1531910066147 Current Time: Wed Jul 18 11:34:26 BST 2018