Publications
118 results found
Folch JP, Lee RM, Shafei B, et al., 2023, Combining multi-fidelity modelling and asynchronous batch Bayesian Optimization, Computers and Chemical Engineering, Vol: 172, ISSN: 0098-1354
Bayesian Optimization is a useful tool for experiment design. Unfortunately, the classical, sequential setting of Bayesian Optimization does not translate well into laboratory experiments, for instance battery design, where measurements may come from different sources and their evaluations may require significant waiting times. Multi-fidelity Bayesian Optimization addresses the setting with measurements from different sources. Asynchronous batch Bayesian Optimization provides a framework to select new experiments before the results of the prior experiments are revealed. This paper proposes an algorithm combining multi-fidelity and asynchronous batch methods. We empirically study the algorithm behaviour, and show it can outperform single-fidelity batch methods and multi-fidelity sequential methods. As an application, we consider designing electrode materials for optimal performance in pouch cells using experiments with coin cells to approximate battery performance.
Odgers J, Kappatou C, Misener R, et al., 2023, Probabilistic Predictions for Partial Least Squares using Bootstrap, AIChE Journal, ISSN: 0001-1541
Folch JP, Lee RM, Shafei B, et al., 2022, Combining multi-fidelity modelling and asynchronous batch bayesian optimization, Publisher: arXiv
Bayesian Optimization is a useful tool for experiment design. Unfortunately,the classical, sequential setting of Bayesian Optimization does not translatewell into laboratory experiments, for instance battery design, wheremeasurements may come from different sources and their evaluations may requiresignificant waiting times. Multi-fidelity Bayesian Optimization addresses thesetting with measurements from different sources. Asynchronous batch BayesianOptimization provides a framework to select new experiments before the resultsof the prior experiments are revealed. This paper proposes an algorithmcombining multi-fidelity and asynchronous batch methods. We empirically studythe algorithm behavior, and show it can outperform single-fidelity batchmethods and multi-fidelity sequential methods. As an application, we considerdesigning electrode materials for optimal performance in pouch cells usingexperiments with coin cells to approximate battery performance.
Ceccon F, Jalving J, Haddad J, et al., 2022, OMLT: Optimization & Machine Learning Toolkit, Journal of Machine Learning Research, Vol: 23, ISSN: 1532-4435
The optimization and machine learning toolkit (OMLT) is an open-source software package incorporating neural network and gradient-boosted tree surrogate models, which have been trained using machine learning, into larger optimization problems. We discuss the advances in optimization technology that made OMLT possible and show how OMLT seamlessly integrates with the algebraic modeling language Pyomo. We demonstrate how to use OMLT for solving decision-making problems in both computer science and engineering.
Campos JS, Misener R, Parpas P, 2022, Partial lasserre relaxation for sparse max-cut, Optimization and Engineering, ISSN: 1389-4420
A common approach to solve or find bounds of polynomial optimization problems like Max-Cut is to use the first level of the Lasserre hierarchy. Higher levels of the Lasserre hierarchy provide tighter bounds, but solving these relaxations is usually computationally intractable. We propose to strengthen the first level relaxation for sparse Max-Cut problems using constraints from the second order Lasserre hierarchy. We explore a variety of approaches for adding a subset of the positive semidefinite constraints of the second order sparse relaxation obtained by using the maximum cliques of the graph’s chordal extension. We apply this idea to sparse graphs of different sizes and densities, and provide evidence of its strengths and limitations when compared to the state-of-the-art Max-Cut solver BiqCrunch and the alternative sparse relaxation CS-TSSOS.
Thebelt A, Wiebe J, Kronqvist JPF, et al., 2022, Maximizing information from chemical engineering data sets: Applications to machine learning, Chemical Engineering Science, Vol: 252, Pages: 1-14, ISSN: 0009-2509
It is well-documented how artificial intelligence can have (and already is having) a big impact on chemical engineering. But classical machine learning approaches may be weak for many chemical engineering applications. This review discusses how challenging data characteristics arise in chemical engineering applications. We identify four characteristics of data arising in chemical engineering applications that make applying classical artificial intelligence approaches difficult: (1) high variance, low volume data, (2) low variance, high volume data, (3) noisy / corrupt / missing data, and (4) restricted data with physics-based limitations. For each of these four data characteristics, we discuss applications where these data characteristics arise and show how current chemical engineering research is extending the fields of data science and machine learning to incorporate these challenges. Finally, we identify several challenges for future research.
Thebelt A, Tsay C, Lee R, et al., 2022, Multi-objective constrained optimization for energy applications via tree ensembles, Applied Energy, Vol: 306, Pages: 1-15, ISSN: 0306-2619
Energy systems optimization problems are complex due to strongly non-linear system behavior and multiple competing objectives, e.g. economic gain vs. environmental impact. Moreover, a large number of input variables and different variable types, e.g. continuous and categorical, are challenges commonly present in real-world applications. In some cases, proposed optimal solutions need to obey explicit input constraints related to physical properties or safety-critical operating conditions. This paper proposes a novel data-driven strategy using tree ensembles for constrained multi-objective optimization of black-box problems with heterogeneous variable spaces for which underlying system dynamics are either too complex to model or unknown. In an extensive case study comprised of synthetic benchmarks and relevant energy applications we demonstrate the competitive performance and sampling efficiency of the proposed algorithm compared to other state-of-the-art tools, making it a useful all-in-one solution for real-world applications with limited evaluation budgets.
Ceccon F, Jalving J, Haddad J, et al., 2022, Presentation abstract: Optimization formulations for machine learning surrogates, Computer Aided Chemical Engineering, Pages: 57-58
In many process systems engineering applications, we seek to integrate surrogate models, e.g. already-trained neural network and gradient-boosted tree models, into larger decision-making problems. This presentation explores different ways to automatically take the machine learning surrogate model and produce an optimization formulation. Our goal is to automate the entire workflow of decision-making with surrogate models from input data to optimization formulation. This presentation discusses our progress towards this goal, gives examples of previous successes, and elicits a conversation with colleagues about the path forward.
Mistry M, Letsios D, Krennrich G, et al., 2021, Mixed-integer convex nonlinear optimization with gradient-boosted trees embedded, Informs Journal on Computing, Vol: 33, Pages: 837-1257, C2, ISSN: 1091-9856
Decision trees usefully represent sparse, high-dimensional, and noisy data. Having learned a function from these data, we may want to thereafter integrate the function into a larger decision-making problem, for example, for picking the best chemical process catalyst. We study a large-scale, industrially relevant mixed-integer nonlinear nonconvex optimization problem involving both gradient-boosted trees and penalty functions mitigating risk. This mixed-integer optimization problem with convex penalty terms broadly applies to optimizing pretrained regression tree models. Decision makers may wish to optimize discrete models to repurpose legacy predictive models or they may wish to optimize a discrete model that accurately represents a data set. We develop several heuristic methods to find feasible solutions and an exact branch-and-bound algorithm leveraging structural properties of the gradient-boosted trees and penalty functions. We computationally test our methods on a concrete mixture design instance and a chemical catalysis industrial instance.
Mistry M, Letsios D, Krennrich G, et al., 2021, Mixed-Integer Convex Nonlinear Optimization with Gradient-Boosted Trees Embedded, Publisher: INFORMS
- Author Web Link
- Open Access Link
- Cite
- Citations: 6
Wiebe J, Misener R, 2021, ROmodel: Modeling robust optimization problems in Pyomo, Publisher: arXiv
This paper introduces ROmodel, an open source Python package extending themodeling capabilities of the algebraic modeling language Pyomo to robustoptimization problems. ROmodel helps practitioners transition fromdeterministic to robust optimization through modeling objects which allowformulating robust models in close analogy to their mathematical formulation.ROmodel contains a library of commonly used uncertainty sets which can begenerated using their matrix representations, but it also allows users todefine custom uncertainty sets using Pyomo constraints. ROmodel supportsadjustable variables via linear decision rules. The resulting models can besolved using ROmodels solvers which implement both the robust reformulation andcutting plane approach. ROmodel is a platform to implement and compare customuncertainty sets and reformulations. We demonstrate ROmodel's capabilities byapplying it to six case studies. We implement custom uncertainty sets based on(warped) Gaussian processes to show how ROmodel can integrate data-drivenmodels with optimization.
Ceccon F, Misener R, 2021, Solving the pooling problem at scale with extensible solver GALINI, Publisher: arXiv
This paper presents a Python library to model pooling problems, a class ofnetwork flow problems with many engineering applications. The libraryautomatically generates a mixed-integer quadratically-constrained quadraticoptimization problem from a given network structure. The library additionallyuses the network structure to build 1) a convex linear relaxation of thenon-convex quadratic program and 2) a mixed-integer linear restriction of theproblem. We integrate the pooling network library with galini, an open-sourceextensible global solver for quadratic optimization. We demonstrate galini'sextensible characteristics by using the pooling library to develop two galiniplug-ins: 1) a cut generator plug-in that adds valid inequalities in the galinicut loop and 2) a primal heuristic plug-in that uses the mixed-integer linearrestriction. We test galini on large scale pooling problems and show that,thanks to the good upper bound provided by the mixed-integer linear restrictionand the good lower bounds provided by the convex relaxation, we obtainoptimality gaps that are competitive with Gurobi 9.1 on the largest probleminstances.
Letsios D, Bradley JT, Suraj G, et al., 2021, Approximate and robust bounded job start scheduling for Royal Mail delivery offices, Journal of Scheduling, Vol: 24, Pages: 237-258, ISSN: 1094-6136
Motivated by mail delivery scheduling problems arising in Royal Mail, we study a generalization of the fundamental makespan scheduling P||Cmax problem which we call the bounded job start scheduling problem. Given a set of jobs, each specified by an integer processing time pj, that have to be executed non-preemptively by a set of m parallel identical machines, the objective is to compute a minimum makespan schedule subject to an upper bound g≤m on the number of jobs that may simultaneously begin per unit of time. With perfect input knowledge, we show that Longest Processing Time First (LPT) algorithm is tightly 2-approximate. After proving that the problem is strongly NP-hard even when g=1, we elaborate on improving the 2-approximation ratio for this case. We distinguish the classes of long and short instances satisfying pj≥m and pj<m, respectively, for each job j. We show that LPT is 5/3-approximate for the former and optimal for the latter. Then, we explore the idea of scheduling long jobs in parallel with short jobs to obtain tightly satisfied packing and bounded job start constraints. For a broad family of instances excluding degenerate instances with many very long jobs, we derive a 1.985-approximation ratio. For general instances, we require machine augmentation to obtain better than 2-approximate schedules. In the presence of uncertain job processing times, we exploit machine augmentation and lexicographic optimization, which is useful for P||Cmax under uncertainty, to propose a two-stage robust optimization approach for bounded job start scheduling under uncertainty aiming in a low number of used machines. Given a collection of schedules of makespan ≤D, this approach allows distinguishing which are the more robust. We substantiate both the heuristics and our recovery approach numerically using Royal Mail data. We show that for the Royal Mail application, machine augmentation, i.e., short-term van rental, is especially relevant.
Pistikopoulos EN, Barbosa-Povoa A, Lee JH, et al., 2021, Process systems engineering - The generation next?, COMPUTERS & CHEMICAL ENGINEERING, Vol: 147, ISSN: 0098-1354
- Author Web Link
- Cite
- Citations: 47
Tsay C, Kronqvist J, Thebelt A, et al., 2021, Partition-based formulations for mixed-integer optimization of trained ReLU neural networks, Publisher: arXiv
This paper introduces a class of mixed-integer formulations for trained ReLUneural networks. The approach balances model size and tightness by partitioningnode inputs into a number of groups and forming the convex hull over thepartitions via disjunctive programming. At one extreme, one partition per inputrecovers the convex hull of a node, i.e., the tightest possible formulation foreach node. For fewer partitions, we develop smaller relaxations thatapproximate the convex hull, and show that they outperform existingformulations. Specifically, we propose strategies for partitioning variablesbased on theoretical motivations and validate these strategies using extensivecomputational experiments. Furthermore, the proposed scheme complements knownalgorithmic approaches, e.g., optimization-based bound tightening capturesdependencies within a partition.
Olofsson S, Schultz ES, Mhamdi A, et al., 2021, Design of dynamic experiments for black-box model discrimination, Publisher: arXiv
Diverse domains of science and engineering require and use mechanisticmathematical models, e.g. systems of differential algebraic equations. Suchmodels often contain uncertain parameters to be estimated from data. Consider adynamic model discrimination setting where we wish to chose: (i) what is thebest mechanistic, time-varying model and (ii) what are the best model parameterestimates. These tasks are often termed modeldiscrimination/selection/validation/verification. Typically, several rivalmechanistic models can explain data, so we incorporate available data and alsorun new experiments to gather more data. Design of dynamic experiments formodel discrimination helps optimally collect data. For rival mechanistic modelswhere we have access to gradient information, we extend existing methods toincorporate a wider range of problem uncertainty and show that our proposedapproach is equivalent to historical approaches when limiting the types ofconsidered uncertainty. We also consider rival mechanistic models as dynamicblack boxes that we can evaluate, e.g. by running legacy code, but wheregradient or other advanced information is unavailable. We replace theseblack-box models with Gaussian process surrogate models and thereby extend themodel discrimination setting to additionally incorporate rival black-box model.We also explore the consequences of using Gaussian process surrogates toapproximate gradient-based methods.
Kronqvist J, Misener R, Tsay C, 2021, Between steps: Intermediate relaxations between big-M and convex hull formulations, Publisher: arXiv
This work develops a class of relaxations in between the big-M and convexhull formulations of disjunctions, drawing advantages from both. The proposed"P-split" formulations split convex additively separable constraints into Ppartitions and form the convex hull of the partitioned disjuncts. Parameter Prepresents the trade-off of model size vs. relaxation strength. We examine thenovel formulations and prove that, under certain assumptions, the relaxationsform a hierarchy starting from a big-M equivalent and converging to the convexhull. We computationally compare the proposed formulations to big-M and convexhull formulations on a test set including: K-means clustering, P_ball problems,and ReLU neural networks. The computational results show that the intermediateP-split formulations can form strong outer approximations of the convex hullwith fewer variables and constraints than the extended convex hullformulations, giving significant computational advantages over both the big-Mand convex hull.
Wiebe J, Misener R, 2021, ROmodel: A Python Robust Optimization Modeling Toolbox, Computer Aided Chemical Engineering, Pages: 683-688
We introduce ROmodel, a Python package that extends the modeling capabilities of the popular modeling language Pyomo to robust optimization problems. ROmodel contains a library of commonly used uncertainty sets which can be generated using their matrix representations, but it also allows the definition of custom uncertainty sets using Pyomo constraints. The resulting models can be solved using ROmodels solvers which implement both the robust reformulation and cutting plane approach. We apply the problem to a number of instances of three case studies and show some results.
Kronqvist J, Misener R, 2020, A disjunctive cut strengthening technique for convex MINLP, OPTIMIZATION AND ENGINEERING, Vol: 22, Pages: 1315-1345, ISSN: 1389-4420
- Author Web Link
- Cite
- Citations: 5
Wiebe J, Cecílio I, Dunlop J, et al., 2020, A robust approach to warped Gaussian process-constrained optimization, Publisher: arXiv
Optimization problems with uncertain black-box constraints, modeled by warpedGaussian processes, have recently been considered in the Bayesian optimizationsetting. This work introduces a new class of constraints in which the sameblack-box function occurs multiple times evaluated at different domain points.Such constraints are important in applications where, e.g., safety-criticalmeasures are aggregated over multiple time periods. Our approach, which usesrobust optimization, reformulates these uncertain constraints intodeterministic constraints guaranteed to be satisfied with a specifiedprobability, i.e., deterministic approximations to a chance constraint. Thisapproach extends robust optimization methods from parametric uncertainty touncertain functions modeled by warped Gaussian processes. We analyze convexityconditions and propose a custom global optimization strategy for non-convexcases. A case study derived from production planning and an industriallyrelevant example from oil well drilling show that the approach effectivelymitigates uncertainty in the learned curves. For the drill scheduling example,we develop a custom strategy for globally optimizing integer decisions.
Čyras K, Karamlou A, Lee M, et al., 2020, AI-assisted schedule explainer for nurse rostering, AAMAS, Pages: 2101-2103, ISSN: 1548-8403
We present an argumentation-supported explanation generating system, called Schedule Explainer, that assists with makespan scheduling. Our stand-alone generic tool explains to a lay user why a resource allocation schedule is good or not, and offers actions to improve the schedule given the user's constraints. Schedule Explainer provides actionable textual explanations via an interactive graphical interface. We illustrate our system with a proof-of-concept application tool in a nurse rostering scenario whereby a shift-lead nurse aims to account for unexpected events by rescheduling some patient procedures to nurses and is aided by the system to do so.
Thebelt A, Kronqvist J, Mistry M, et al., 2020, ENTMOOT: A framework for optimization over ensemble tree models, Publisher: arXiv
Gradient boosted trees and other regression tree models perform well in awide range of real-world, industrial applications. These tree models (i) offerinsight into important prediction features, (ii) effectively manage sparsedata, and (iii) have excellent prediction capabilities. Despite theiradvantages, they are generally unpopular for decision-making tasks andblack-box optimization, which is due to their difficult-to-optimize structureand the lack of a reliable uncertainty measure. ENTMOOT is our new frameworkfor integrating (already trained) tree models into larger optimizationproblems. The contributions of ENTMOOT include: (i) explicitly introducing areliable uncertainty measure that is compatible with tree models, (ii) solvingthe larger optimization problems that incorporate these uncertainty aware treemodels, (iii) proving that the solutions are globally optimal, i.e. no bettersolution exists. In particular, we show how the ENTMOOT approach allows asimple integration of tree models into decision-making and black-boxoptimization, where it proves as a strong competitor to commonly-usedframeworks.
Letsios D, Baltean-Lugojan R, Ceccon F, et al., 2020, Approximation algorithms for process systems engineering, COMPUTERS & CHEMICAL ENGINEERING, Vol: 132, ISSN: 0098-1354
- Author Web Link
- Cite
- Citations: 2
Botoeva E, Kouvaros P, Kronqvist J, et al., 2020, Efficient Verification of ReLU-Based Neural Networks via Dependency Analysis, 34th AAAI Conference on Artificial Intelligence / 32nd Innovative Applications of Artificial Intelligence Conference / 10th AAAI Symposium on Educational Advances in Artificial Intelligence, Publisher: ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE, Pages: 3291-3299, ISSN: 2159-5399
- Author Web Link
- Cite
- Citations: 29
Thebelt A, Kronqvist J, Lee RM, et al., 2020, Global Optimization with Ensemble Machine Learning Models, Editors: Pierucci, Manenti, Bozzano, Manca, Publisher: ELSEVIER SCIENCE BV, Pages: 1981-1986
- Author Web Link
- Cite
- Citations: 3
Sedgwick R, Goertz J, Stevens M, et al., 2020, Design of Experiments for Verifying Biomolecular Networks
Wiebe J, Cecílio I, Misener R, 2019, Data-driven optimization of processes with degrading equipment, Industrial & Engineering Chemistry Research, Vol: 57, Pages: 17177-17191, ISSN: 0888-5885
In chemical and manufacturing processes, unit failures due to equipment degradation can lead to process downtime and significant costs. In this context, finding an optimal maintenance strategy to ensure good unit health while avoiding excessive expensive maintenance activities is highly relevant. We propose a practical approach for the integrated optimization of production and maintenance capable of incorporating uncertain sensor data regarding equipment degradation. To this end, we integrate data-driven stochastic degradation models from Condition-based Maintenance into a process level mixed-integer optimization problem using Robust Optimization. We reduce computational expense by utilizing both analytical and data-based approximations and optimize the Robust optimization parameters using Bayesian Optimization. We apply our framework to five instances of the State-Task-Network and demonstrate that it can efficiently compromise between equipment availability and cost of maintenance.
Letsios D, Bradley JT, Suraj G, et al., 2019, Approximating bounded job start scheduling with application in Royal Mail deliveries under uncertainty, Publisher: arXiv
Motivated by mail delivery scheduling problems arising in Royal Mail, westudy a generalization of the fundamental makespan scheduling P||Cmax problemwhich we call the "bounded job start scheduling problem". Given a set of jobs,each specified by an integer processing time p_j, that have to be executednon-preemptively by a set of m parallel identical machines, the objective is tocompute a minimum makespan schedule subject to an upper bound g<=m on thenumber of jobs that may simultaneously begin per unit of time. We show thatLongest Processing Time First (LPT) algorithm is tightly 2-approximate. Afterproving that the problem is strongly NP-hard even when g=1, we elaborate onimproving the 2-approximation ratio for this case. We distinguish the classesof long and short instances satisfying p_j>=m and p_j<m, respectively, for eachjob j. We show that LPT is 5/3-approximate for the former and optimal for thelatter. Then, we explore the idea of scheduling long jobs in parallel withshort jobs to obtain tightly satisfied packing and bounded job startconstraints. For a broad family of instances excluding degenerate instanceswith many very long jobs, we derive a 1.985-approximation ratio. For generalinstances, we require machine augmentation to obtain better than 2-approximateschedules. Finally, we exploit machine augmentation and lexicographicoptimization, which is useful for P||Cmax under uncertainty, to propose atwo-stage robust optimization approach for bounded job start scheduling underuncertainty aiming in good trade-offs in terms of makespan and number of usedmachines. We substantiate this approach numerically using Royal Mail data.
Kouyialis G, Wang X, Misener R, 2019, Symmetry Detection for Quadratic Optimization Using Binary Layered Graphs, PROCESSES, Vol: 7
Campos JS, Misener R, Parpas P, 2019, A multilevel analysis of the Lasserre hierarchy, European Journal of Operational Research, Vol: 277, Pages: 32-41, ISSN: 0377-2217
This paper analyzes the relation between different orders of the Lasserre hierarchy for polynomial optimization (POP). Although for some cases solving the semidefinite programming relaxation corresponding to the first order of the hierarchy is enough to solve the underlying POP, other problems require sequentially solving the second or higher orders until a solution is found. For these cases, and assuming that the lower order semidefinite programming relaxation has been solved, we develop prolongation operators that exploit the solutions already calculated to find initial approximations for the solution of the higher order relaxation. We can prove feasibility in the higher order of the hierarchy of the points obtained using the operators, as well as convergence to the optimal as the relaxation order increases. Furthermore, the operators are simple and inexpensive for problems where the projection over the feasible set is “easy” to calculate (for example integer {0, 1} and {−1,1} POPs). Our numerical experiments show that it is possible to extract useful information for real applications using the prolongation operators. In particular, we illustrate how the operators can be used to increase the efficiency of an infeasible interior point method by using them as an initial point. We use this technique to solve quadratic integer {0, 1} problems, as well as MAX-CUT and integer partition problems.
This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.