Publications
59 results found
Yang L, Sun X, Hamzi B, et al., 2024, Learning dynamical systems from data: A simple cross-validation perspective, Part V: Sparse Kernel Flows for 132 chaotic dynamical systems, Physica D: Nonlinear Phenomena, Vol: 460, ISSN: 0167-2789
Learning equations of complex dynamical systems from data is one of the principal problems in scientific machine learning. The method of Kernel Flows (KFs) has offered an effective learning strategy that interpolates the vector-field of dynamical system with a data-adapted kernel. It is based on the premise that a kernel is good if the number of interpolation points can be halved without significant loss in accuracy. However, KFs is limited by the choice of a base kernel. In this paper, we introduce the method of Sparse Kernel Flows in order to learn the “best” kernel starting from a preset kernel library. First, we design a parameterized base kernel that is a linear combination of several existing kernel functions. Then, under the assumption of KFs that a kernel is good if the number of interpolation points can be halved without significant loss in accuracy, we design the kernel learning loss function by incorporating ℓ1 regularization. Then, Least absolute shrinkage and selection operator (LASSO) is performed to extract the fewest active terms from the base overdetermined set of candidate kernel functions, thereby estimating the weight coefficient. Furthermore, we apply our proposal to a benchmark dataset including 132 chaotic dynamical systems.
Hamzi B, Owhadi H, Kevrekidis Y, 2023, Learning dynamical systems from data: A simple cross-validation perspective, part IV: Case with partial observations, Physica D: Nonlinear Phenomena, Vol: 454, ISSN: 0167-2789
A simple and interpretable way to learn a dynamical system from data is to interpolate its governing equations with a kernel. In particular, this strategy is highly efficient (both in terms of accuracy and complexity) when the kernel is data-adapted using Kernel Flows (KF) (Owhadi and Yoo, 2019), (which uses gradient-based optimization to learn a kernel based on the premise that a kernel is good if there is no significant loss in accuracy if half of the data is used for interpolation). In this work, we extend previous work on learning dynamical systems using Kernel Flows (Hamzi and Owhadi, 2021; Darcy et al. 2021; Lee et al. 2023; Darcy et al. 2023; Owhadi and Romit Maulik, 2021) to the case of learning vector-valued dynamical systems from time-series observations that are partial/incomplete in the state space. The method combines Kernel Flows with Computational Graph Completion.
Gazor M, Hamzi B, Shoghi A, 2023, The infinite level normal forms for non-resonant double Hopf singularities, SYSTEMS & CONTROL LETTERS, Vol: 176, ISSN: 0167-6911
Hamzi B, Owhadi H, Paillet L, 2023, A note on microlocal kernel design for some slow-fast stochastic differential equations with critical transitions and application to EEG signals, PHYSICA A-STATISTICAL MECHANICS AND ITS APPLICATIONS, Vol: 616, ISSN: 0378-4371
Darcy M, Hamzi B, Livieri G, et al., 2023, One-shot learning of stochastic differential equations with data adapted kernels, Physica D: Nonlinear Phenomena, Vol: 444, ISSN: 0167-2789
We consider the problem of learning Stochastic Differential Equations of the formdXt = f(Xt)dt + σ(Xt)dWt from one sample trajectory. This problem is more challengingthan learning deterministic dynamical systems because one sample trajectory only providesindirect information on the unknown functions f, σ, and stochastic process dWt representingthe drift, the diffusion, and the stochastic forcing terms, respectively. We propose a methodthat combines Computational Graph Completion [46] and data adapted kernels learned via anew variant of cross validation. Our approach can be decomposed as follows: (1) Represent thetime-increment map Xt → Xt+dt as a Computational Graph in which f, σ and dWt appearas unknown functions and random variables. (2) Complete the graph (approximate unknownfunctions and random variables) via Maximum a Posteriori Estimation (given the data) withGaussian Process (GP) priors on the unknown functions. (3) Learn the covariance functions(kernels) of the GP priors from data with randomized cross-validation. Numerical experimentsillustrate the efficacy, robustness, and scope of our method.
Dingle K, Kamal R, Hamzi B, 2023, A note on a priori forecasting and simplicity bias in time series, PHYSICA A-STATISTICAL MECHANICS AND ITS APPLICATIONS, Vol: 609, ISSN: 0378-4371
- Author Web Link
- Cite
- Citations: 4
Lee J, De Brouwer E, Hamzi B, et al., 2023, Learning dynamical systems from data: A simple cross-validation perspective, Part III: Irregularly-sampled time series, PHYSICA D-NONLINEAR PHENOMENA, Vol: 443, ISSN: 0167-2789
Smirnov A, Hamzi B, Owhadi H, 2022, Mean-field limits of trained weights in deep learning: A dynamical systems perspective, DOLOMITES RESEARCH NOTES ON APPROXIMATION, Vol: 15, Pages: 125-145, ISSN: 2035-6803
- Author Web Link
- Cite
- Citations: 1
Haasdonk B, Hamzi B, Santin G, et al., 2021, Kernel methods for center manifold approximation and a weak data-based version of the Center Manifold Theorem, PHYSICA D-NONLINEAR PHENOMENA, Vol: 427, ISSN: 0167-2789
- Author Web Link
- Cite
- Citations: 6
Hamzi B, Maulik R, Owhadi H, 2021, Simple, low-cost and accurate data-driven geophysical forecasting with learned kernels, PROCEEDINGS OF THE ROYAL SOCIETY A-MATHEMATICAL PHYSICAL AND ENGINEERING SCIENCES, Vol: 477, ISSN: 1364-5021
- Author Web Link
- Cite
- Citations: 10
Hamzi B, Owhadi H, 2021, Learning dynamical systems from data: A simple cross-validation perspective, part I: Parametric kernel flows, PHYSICA D-NONLINEAR PHENOMENA, Vol: 421, ISSN: 0167-2789
- Author Web Link
- Cite
- Citations: 19
Bittracher A, Klus S, Hamzi B, et al., 2021, Dimensionality Reduction of Complex Metastable Systems via Kernel Embeddings of Transition Manifolds, Publisher: SPRINGER
- Author Web Link
- Cite
- Citations: 6
Colonius F, Hamzi B, 2020, Entropy for practical stabilization, Publisher: arXiv
For deterministic continuous-time nonlinear control systems,epsilon-practical stabilization entropy and practical stabilization entropy areintroduced. Here the rate of attraction is specified by a KL-function. Upperand lower bounds for the diverse entropies are proved, with special attentionto exponential KL-functions. Two scalar examples are analyzed in detail.
Klus S, Nueske F, Hamzi B, 2020, Kernel-based approximation of the koopman generator and schrodinger operator, Entropy: international and interdisciplinary journal of entropy and information studies, Vol: 22, Pages: 1-22, ISSN: 1099-4300
Many dimensionality and model reduction techniques rely on estimating dominant eigenfunctions of associated dynamical operators from data. Important examples include the Koopman operator and its generator, but also the Schrödinger operator. We propose a kernel-based method for the approximation of differential operators in reproducing kernel Hilbert spaces and show how eigenfunctions can be estimated by solving auxiliary matrix eigenvalue problems. The resulting algorithms are applied to molecular dynamics and quantum chemistry examples. Furthermore, we exploit that, under certain conditions, the Schrödinger operator can be transformed into a Kolmogorov backward operator corresponding to a drift-diffusion process and vice versa. This allows us to apply methods developed for the analysis of high-dimensional stochastic differential equations to quantum mechanical systems.
Giesl P, Hamzi B, Rasmussen M, et al., 2020, Approximation of Lyapunov functions from noisy data, Journal of Computational Dynamics, Vol: 7, Pages: 57-81, ISSN: 2158-2491
Methods have previously been developed for the approximation of Lyapunovfunctions using radial basis functions. However these methods assume that theevolution equations are known. We consider the problem of approximating a givenLyapunov function using radial basis functions where the evolution equationsare not known, but we instead have sampled data which is contaminated withnoise. We propose an algorithm in which we first approximate the underlyingvector field, and use this approximation to then approximate the Lyapunovfunction. Our approach combines elements of machine learning/statisticallearning theory with the existing theory of Lyapunov function approximation.Error estimates are provided for our algorithm.
Haasdonk B, Hamzi B, Santin G, et al., 2020, Greedy kernel methods for center manifold approximation, Pages: 95-106, ISSN: 1439-7358
For certain dynamical systems it is possible to significantly simplify the study of stability by means of the center manifold theory. This theory allows to isolate the complicated asymptotic behavior of the system close to a non-hyperbolic equilibrium point, and to obtain meaningful predictions of its behavior by analyzing a reduced dimensional problem. Since the manifold is usually not known, approximation methods are of great interest to obtain qualitative estimates. In this work, we use a data-based greedy kernel method to construct a suitable approximation of the manifold close to the equilibrium. The data are collected by repeated numerical simulation of the full system by means of a high-accuracy solver, which generates sets of discrete trajectories that are then used to construct a surrogate model of the manifold. The method is tested on different examples which show promising performance and good accuracy.
Hamzi B, Abed EH, 2020, Local modal participation analysis of nonlinear systems using Poincare linearization, NONLINEAR DYNAMICS, Vol: 99, Pages: 803-811, ISSN: 0924-090X
- Author Web Link
- Cite
- Citations: 9
Haasdonk B, Hamzi B, Santin G, et al., 2020, Greedy Kernel Methods for Center Manifold Approximation, Publisher: Springer International Publishing
<jats:title>Abstract</jats:title><jats:p>For certain dynamical systems it is possible to significantly simplify the study of stability by means of the center manifold theory. This theory allows to isolate the complicated asymptotic behavior of the system close to a non-hyperbolic equilibrium point, and to obtain meaningful predictions of its behavior by analyzing a reduced dimensional problem. Since the manifold is usually not known, approximation methods are of great interest to obtain qualitative estimates. In this work, we use a data-based greedy kernel method to construct a suitable approximation of the manifold close to the equilibrium. The data are collected by repeated numerical simulation of the full system by means of a high-accuracy solver, which generates sets of discrete trajectories that are then used to construct a surrogate model of the manifold. The method is tested on different examples which show promising performance and good accuracy.</jats:p>
Hamzi B, Colonius F, 2019, Kernel methods for the approximation of discrete-time linear autonomous and control systems, SN Applied Sciences, Vol: 1, ISSN: 2523-3963
Methods from learning theory are used in the state space of linear dynamical and control systems in order to estimate relevant matrices and some relevant quantities such as the topological entropy. An application to stabilization via algebraic Riccati equations is included by viewing a control system as an autonomous system in an extended space of states and control inputs. Kernel methods are the main techniques used in this paper and the approach is illustrated via a series of numerical examples. The advantage of using kernel methods is that they allow to perform function approximation from data and, as illustrated in this paper, allow to approximate linear discrete-time autonomous and control systems from data.
Hamzi B, Kuehn C, Mohamed S, 2019, A note on kernel methods for multiscale systems with critical transitions, MATHEMATICAL METHODS IN THE APPLIED SCIENCES, Vol: 42, Pages: 907-917, ISSN: 0170-4214
Hamzi B, Colonius F, 2019, Kernel Methods for Discrete-Time Linear Equations, 19th Annual International Conference on Computational Science (ICCS), Publisher: SPRINGER INTERNATIONAL PUBLISHING AG, Pages: 178-191, ISSN: 0302-9743
Hamzi B, AlOtaiby TN, AlShebeili S, et al., 2018, Kernel Methods and the Maximum Mean Discrepancy for Seizure Detection, 1st International Conference on Computer Applications and Information Security (ICCAIS), Publisher: IEEE
Bouvrie J, Hamzi B, 2017, KERNEL METHODS FOR THE APPROXIMATION OF NONLINEAR SYSTEMS, SIAM JOURNAL ON CONTROL AND OPTIMIZATION, Vol: 55, Pages: 2460-2492, ISSN: 0363-0129
Bouvrie J, Hamzi B, 2017, Kernel methods for the approximation of some key quantities of nonlinear systems, Journal of Computational Dynamics, Vol: 4
We introduce a data-based approach to estimating key quantities which arise in the study of nonlinear control systems and random nonlinear dynamical systems. Our approach hinges on the observation that much of the existing linear theory may be readily extended to nonlinear systems - with a reasonable expectation of success - once the nonlinear system has been mapped into a high or infinite dimensional feature space. In particular, we embed a nonlinear system in a reproducing kernel Hilbert space where linear theory can be used to develop computable, non-parametric estimators approximating controllability and observability energy functions for nonlinear systems. In all cases the relevant quantities are estimated from simulated or observed data. It is then shown that the controllability energy estimator provides a key means for approximating the invariant measure of an ergodic, stochastically forced nonlinear system.
Hamzi B, Lamb JSW, Lewis D, 2015, A Characterization of Normal Forms for Control Systems, JOURNAL OF DYNAMICAL AND CONTROL SYSTEMS, Vol: 21, Pages: 273-284, ISSN: 1079-2724
- Author Web Link
- Cite
- Citations: 8
Hamzi B, Praly L, 2015, Ignored input dynamics and a new characterization of control Lyapunov functions, Pages: 2626-2631
Our objective in this paper is to extend as much as possible the dissipativity approach for the study of robustness of stability in the presence of known/unknown but ignored input dynamics. This leads us to: • give a new characterization of control Lyapunov functions (CLF) where LfV is upper-bounded by a function of LgV, • define the dissipativity approach as : - assuming the ignored dynamics are dissipative with storage function W and (known) supply rate w, - analyzing closed-loop stability with the sum of the storage function W and a CLF for the nominal part. Stability margin is given in terms of an inequality the supply rate should satisfy. Unfortunately this extension of the dissipativity approach cannot still cope with ignored dynamics which have non zero relative degree or are non minimum phase.
Hamzi B, Abed EH, 2014, Local Mode-in-State Participation Factors for Nonlinear Systems, 53rd IEEE Annual Conference on Decision and Control (CDC), Publisher: IEEE, Pages: 43-48, ISSN: 0743-1546
- Author Web Link
- Cite
- Citations: 3
Bouvrie J, Hamzi B, 2012, Empirical Estimators for Stochastically Forced Nonlinear Systems: Observability, Controllability and the Invariant Measure, Proc. American Control Conference (ACC)
We introduce a data-based approach to estimating key quantities which arise in the study of nonlinear control systems and random nonlinear dynamical systems. Our approach hinges on the observation that much of the existing linear theory may be readily extended to nonlinear systems - with a reasonable expectation of success - once the nonlinear system has been mapped into a high or infinite dimensional feature space. In particular, we develop computable, non-parametric estimators approximating controllability and observability energy functions for nonlinear systems, and study the ellipsoids they induce. In all cases the relevant quantities are estimated from simulated or observed data. It is then shown that the controllability energy estimator provides a key means for approximating the invariant measure of an ergodic, stochastically forced nonlinear system.
Bouvrie J, Hamzi B, 2012, Empirical estimators for stochastically forced nonlinear systems: Observability, controllability and the invariant measure, Pages: 4142-4148, ISSN: 0743-1619
We introduce a data-based approach to estimating key quantities which arise in the study of nonlinear control systems and random nonlinear dynamical systems. Our approach hinges on the observation that much of the existing linear theory may be readily extended to nonlinear systems with a reasonable expectation of success once the nonlinear system has been mapped into a high or infinite dimensional feature space. In particular, we develop computable, non-parametric estimators approximating controllability and observability energy functions for nonlinear systems, and study the ellipsoids they induce. In all cases the relevant quantities are estimated from simulated or observed data. It is then shown that the controllability energy estimator provides a key means for approximating the invariant measure of an ergodic, stochastically forced nonlinear system. © 2012 AACC American Automatic Control Council).
Bouvrie J, Hamzi B, 2011, Model Reduction for Nonlinear Control Systems using Kernel Subspace Methods
We introduce a data-driven order reduction method for nonlinear control systems, drawing on recent progress in machine learning and statistical dimensionality reduction. The method rests on the assumption that the nonlinear system behaves linearly when lifted into a high (or infinite) dimensional feature space where balanced truncation may be carried out implicitly. This leads to a nonlinear reduction map which can be combined with a representation of the system belonging to a reproducing kernel Hilbert space to give a closed, reduced order dynamical system which captures the essential input-output characteristics of the original model.Empirical simulations illustrating the approach are also provided.
This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.