Imperial College London

ProfessorStephenMuggleton

Faculty of EngineeringDepartment of Computing

Royal Academy Chair in Machine Learning
 
 
 
//

Contact

 

+44 (0)20 7594 8307s.muggleton Website

 
 
//

Assistant

 

Mrs Bridget Gundry +44 (0)20 7594 1245

 
//

Location

 

407Huxley BuildingSouth Kensington Campus

//

Summary

 

Publications

Publication Type
Year
to

260 results found

Ai L, Muggleton SH, Hocquette C, Gromowski M, Schmid Uet al., 2021, Beneficial and harmful explanatory machine learning, MACHINE LEARNING, Vol: 110, Pages: 695-721, ISSN: 0885-6125

Journal article

Patsantzis S, Muggleton SH, 2021, Top program construction and reduction for polynomial time Meta-Interpretive learning, MACHINE LEARNING, Vol: 110, Pages: 755-778, ISSN: 0885-6125

Journal article

Dai W-Z, Hallett L, Muggleton SH, Baldwin GSet al., 2021, Automated Biodesign Engineering by Abductive Meta-Interpretive Learning

The application of Artificial Intelligence (AI) to synthetic biology willprovide the foundation for the creation of a high throughput automated platformfor genetic design, in which a learning machine is used to iteratively optimisethe system through a design-build-test-learn (DBTL) cycle. However, mainstreammachine learning techniques represented by deep learning lacks the capabilityto represent relational knowledge and requires prodigious amounts of annotatedtraining data. These drawbacks strongly restrict AI's role in synthetic biologyin which experimentation is inherently resource and time intensive. In thiswork, we propose an automated biodesign engineering framework empowered byAbductive Meta-Interpretive Learning ($Meta_{Abd}$), a novel machine learningapproach that combines symbolic and sub-symbolic machine learning, to furtherenhance the DBTL cycle by enabling the learning machine to 1) exploit domainknowledge and learn human-interpretable models that are expressed by formallanguages such as first-order logic; 2) simultaneously optimise the structureand parameters of the models to make accurate numerical predictions; 3) reducethe cost of experiments and effort on data annotation by actively generatinghypotheses and examples. To verify the effectiveness of $Meta_{Abd}$, we havemodelled a synthetic dataset for the production of proteins from a three geneoperon in a microbial host, which represents a common synthetic biologyproblem.

Journal article

Patsantzis S, Muggleton SH, 2021, Meta-Interpretive Learning as Metarule Specialisation., CoRR, Vol: abs/2106.07464

Journal article

Patsantzis S, Muggleton SH, 2021, Top program construction and reduction for polynomial time Meta-Interpretive learning., Mach. Learn., Vol: 110, Pages: 755-778

Journal article

Ai L, Muggleton SH, Hocquette C, Gromowski M, Schmid Uet al., 2021, Beneficial and harmful explanatory machine learning., Mach. Learn., Vol: 110, Pages: 695-721

Journal article

Cropper A, Dumancic S, Evans R, Muggleton SHet al., 2021, Inductive logic programming at 30., CoRR, Vol: abs/2102.10556

Journal article

Cropper A, Morel R, Muggleton S, 2020, Learning higher-order logic programs, Machine Learning, Vol: 109, Pages: 1289-1322, ISSN: 0885-6125

A key feature of inductive logic programming is its ability to learn first-order programs, which are intrinsically more expressive than propositional programs. In this paper, we introduce techniques to learn higher-order programs. Specifically, we extend meta-interpretive learning (MIL) to support learning higher-order programs by allowing for higher-order definitions to be used as background knowledge. Our theoretical results show that learning higher-order programs, rather than first-order programs, can reduce the textual complexity required to express programs, which in turn reduces the size of the hypothesis space and sample complexity. We implement our idea in two new MIL systems: the Prolog system Metagol ho and the ASP system HEXMIL ho. Both systems support learning higher-order programs and higher-order predicate invention, such as inventing functions for map/3 and conditions for filter/3. We conduct experiments on four domains (robot strategies, chess playing, list transformations, and string decryption) that compare learning first-order and higher-order programs. Our experimental results support our theoretical claims and show that, compared to learning first-order programs, learning higher-order programs can significantly improve predictive accuracies and reduce learning times.

Journal article

Dai W-Z, Muggleton SH, 2020, Abductive Knowledge Induction From Raw Data

For many reasoning-heavy tasks involving raw inputs, it is challenging todesign an appropriate end-to-end learning pipeline. Neuro-Symbolic Learning,divide the process into sub-symbolic perception and symbolic reasoning, tryingto utilise data-driven machine learning and knowledge-driven reasoningsimultaneously. However, they suffer from the exponential computationalcomplexity within the interface between these two components, where thesub-symbolic learning model lacks direct supervision, and the symbolic modellacks accurate input facts. Hence, most of them assume the existence of astrong symbolic knowledge base and only learn the perception model whileavoiding a crucial problem: where does the knowledge come from? In this paper,we present Abductive Meta-Interpretive Learning ($Meta_{Abd}$) that unitesabduction and induction to learn neural networks and induce logic theoriesjointly from raw data. Experimental results demonstrate that $Meta_{Abd}$ notonly outperforms the compared systems in predictive accuracy and dataefficiency but also induces logic programs that can be re-used as backgroundknowledge in subsequent learning tasks. To the best of our knowledge,$Meta_{Abd}$ is the first system that can jointly learn neural networks fromscratch and induce recursive first-order logic theories with predicateinvention.

Journal article

Cropper A, Dumancic S, Muggleton SH, 2020, Turning 30: New ideas in inductive logic programming, IJCAI International Joint Conference on Artificial Intelligence, Vol: 2021-January, Pages: 4833-4839, ISSN: 1045-0823

Common criticisms of state-of-the-art machine learning include poor generalisation, a lack of interpretability, and a need for large amounts of training data. We survey recent work in inductive logic programming (ILP), a form of machine learning that induces logic programs from data, which has shown promise at addressing these limitations. We focus on new methods for learning recursive programs that generalise from few examples, a shift from using hand-crafted background knowledge to learning background knowledge, and the use of different technologies, notably answer set programming and neural networks. As ILP approaches 30, we also discuss directions for future research.

Journal article

Cropper A, Morel R, Muggleton SH, 2020, Learning Higher-Order Programs through Predicate Invention, 34th AAAI Conference on Artificial Intelligence / 32nd Innovative Applications of Artificial Intelligence Conference / 10th AAAI Symposium on Educational Advances in Artificial Intelligence, Publisher: ASSOC ADVANCEMENT ARTIFICIAL INTELLIGENCE, Pages: 13655-13658, ISSN: 2159-5399

Conference paper

Hocquette C, Muggleton SH, 2020, Complete bottom-up predicate invention in meta-interpretive learning, Pages: 2312-2318, ISSN: 1045-0823

Predicate Invention in Meta-Interpretive Learning (MIL) is generally based on a top-down approach, and the search for a consistent hypothesis is carried out starting from the positive examples as goals. We consider augmenting top-down MIL systems with a bottom-up step during which the background knowledge is generalised with an extension of the immediate consequence operator for second-order logic programs. This new method provides a way to perform extensive predicate invention useful for feature discovery. We demonstrate this method is complete with respect to a fragment of dyadic datalog. We theoretically prove this method reduces the number of clauses to be learned for the top-down learner, which in turn can reduce the sample complexity. We formalise an equivalence relation for predicates which is used to eliminate redundant predicates. Our experimental results suggest pairing the state-of-the-art MIL system Metagol with an initial bottom-up step can significantly improve learning performance.

Conference paper

Cropper A, Dumancic S, Muggleton SH, 2020, Turning 30: New Ideas in Inductive Logic Programming., Publisher: ijcai.org, Pages: 4833-4839

Conference paper

Ichise R, Muggleton S, Kozaki K, Lecue F, Zhao D, Kawamura Tet al., 2019, Special Issue on Semantic Technology, New Generation Computing, Vol: 37, Pages: 359-360, ISSN: 0288-3635

Journal article

Muggleton SH, Hocquette C, 2019, Machine discovery of comprehensible strategies for simple games using Meta-interpretive Learning, New Generation Computing, Vol: 37, Pages: 203-217, ISSN: 0288-3635

Recently, world-class human players have been outperformed in a number of complex two-person games (Go, Chess, Checkers) by Deep Reinforcement Learning systems. However, the data efficiency of the learning systems is unclear given that they appear to require far more training games to achieve such performance than any human player might experience in a lifetime. In addition, the resulting learned strategies are not in a form which can be communicated to human players. This contrasts to earlier research in Behavioural Cloning in which single-agent skills were machine learned in a symbolic language, facilitating their being taught to human beings. In this paper, we consider Machine Discovery of human-comprehensible strategies for simple two-person games (Noughts-and-Crosses and Hexapawn). One advantage of considering simple games is that there is a tractable approach to calculating minimax regret. We use these games to compare Cumulative Minimax Regret for variants of both standard and deep reinforcement learning against two variants of a new Meta-interpretive Learning system called MIGO. In our experiments, tested variants of both normal and deep reinforcement learning have consistently worse performance (higher cumulative minimax regret) than both variants of MIGO on Noughts-and-Crosses and Hexapawn. In addition, MIGO’s learned rules are relatively easy to comprehend, and are demonstrated to achieve significant transfer learning in both directions between Noughts-and-Crosses and Hexapawn.

Journal article

Hocquette C, Muggleton SH, 2019, Can meta-interpretive learning outperform deep reinforcement learning of evaluable game strategies?, Publisher: arXiv

World-class human players have been outperformed in a number of complex two person games (Go, Chess, Checkers) by Deep Reinforcement Learning systems. However, owing to tractability considerations minimax regret of a learning system cannot be evaluated in such games. In this paper we consider simple games (Noughts-and-Crosses and Hexapawn) in which minimax regret can be efficiently evaluated. We use these games to compare Cumulative Minimax Regret for variants of both standard and deep reinforcement learning against two variants of a new Meta-Interpretive Learning system called MIGO. In our experiments all tested variants of both normal and deep reinforcement learning have worse performance (higher cumulative minimax regret) than both variants of MIGO on Noughts-and-Crosses and Hexapawn. Additionally, MIGO's learned rules are relatively easy to comprehend, and are demonstrated to achieve significant transfer learning in both directions between Noughts-and-Crosses and Hexapawn.

Working paper

Cropper A, Muggleton SH, 2019, Learning efficient logic programs., Mach. Learn., Vol: 108, Pages: 1063-1083

Journal article

Raedt LD, Evans R, Muggleton SH, Schmid Uet al., 2019, Approaches and Applications of Inductive Programming (Dagstuhl Seminar 19202)., Dagstuhl Reports, Vol: 9, Pages: 58-88

Journal article

Muggleton S, Dai WZ, Sammut C, Tamaddoni-Nezhad A, Wen J, Zhou ZHet al., 2018, Meta-Interpretive Learning from noisy images, Machine Learning, Vol: 107, Pages: 1097-1118, ISSN: 0885-6125

Statistical machine learning is widely used in image classification. However, most techniques (1) require many images to achieve high accuracy and (2) do not provide support for reasoning below the level of classification, and so are unable to support secondary reasoning, such as the existence and position of light sources and other objects outside the image. This paper describes an Inductive Logic Programming approach called Logical Vision which overcomes some of these limitations. LV uses Meta-Interpretive Learning (MIL) combined with low-level extraction of high-contrast points sampled from the image to learn recursive logic programs describing the image. In published work LV was demonstrated capable of high-accuracy prediction of classes such as regular polygon from small numbers of images where Support Vector Machines and Convolutional Neural Networks gave near random predictions in some cases. LV has so far only been applied to noise-free, artificially generated images. This paper extends LV by (a) addressing classification noise using a new noise-telerant version of the MIL system Metagol, (b) addressing attribute noise using primitive-level statistical estimators to identify sub-objects in real images, (c) using a wider class of background models representing classical 2D shapes such as circles and ellipses, (d) providing richer learnable background knowledge in the form of a simple but generic recursive theory of light reflection. In our experiments we consider noisy images in both natural science settings and in a RoboCup competition setting. The natural science settings involve identification of the position of the light source in telescopic and microscopic images, while the RoboCup setting involves identification of the position of the ball. Our results indicate that with real images the new noise-robust version of LV using a single example (i.e. one-shot LV) converges to an accuracy at least comparable to a thirty-shot statistical machine learner on bot

Journal article

Muggleton SH, Schmid U, Zeller C, Tamaddoni-Nezhad A, Besold Tet al., 2018, Ultra-strong machine learning: comprehensibility of programs learned with ILP, Machine Learning, Vol: 107, Pages: 1119-1140, ISSN: 0885-6125

During the 1980s Michie defined Machine Learning in terms of two orthogonal axes of performance: predictive accuracy and comprehensibility of generated hypotheses. Since predictive accuracy was readily measurable and comprehensibility not so, later definitions in the 1990s, such as Mitchell’s, tended to use a one-dimensional approach to Machine Learning based solely on predictive accuracy, ultimately favouring statistical over symbolic Machine Learning approaches. In this paper we provide a definition of comprehensibility of hypotheses which can be estimated using human participant trials. We present two sets of experiments testing human comprehensibility of logic programs. In the first experiment we test human comprehensibility with and without predicate invention. Results indicate comprehensibility is affected not only by the complexity of the presented program but also by the existence of anonymous predicate symbols. In the second experiment we directly test whether any state-of-the-art ILP systems are ultra-strong learners in Michie’s sense, and select the Metagol system for use in humans trials. Results show participants were not able to learn the relational concept on their own from a set of examples but they were able to apply the relational definition provided by the ILP system correctly. This implies the existence of a class of relational concepts which are hard to acquire for humans, though easy to understand given an abstract explanation. We believe improved understanding of this class could have potential relevance to contexts involving human learning, teaching and verbal interaction.

Journal article

Cropper A, Muggleton SH, 2018, Learning efficient logic programs, Machine Learning, Pages: 1-21, ISSN: 0885-6125

© 2018 The Author(s) When machine learning programs from data, we ideally want to learn efficient rather than inefficient programs. However, existing inductive logic programming (ILP) techniques cannot distinguish between the efficiencies of programs, such as permutation sort (n!) and merge sort (Formula presented.). To address this limitation, we introduce Metaopt, an ILP system which iteratively learns lower cost logic programs, each time further restricting the hypothesis space. We prove that given sufficiently large numbers of examples, Metaopt converges on minimal cost programs, and our experiments show that in practice only small numbers of examples are needed. To learn minimal time-complexity programs, including non-deterministic programs, we introduce a cost function called tree cost which measures the size of the SLD-tree searched when a program is given a goal. Our experiments on programming puzzles, robot strategies, and real-world string transformation problems show that Metaopt learns minimal cost programs. To our knowledge, Metaopt is the first machine learning approach that, given sufficient numbers of training examples, is guaranteed to learn minimal cost logic programs, including minimal time-complexity programs.

Journal article

Ichise R, Lecue F, Kawamura T, Zhao D, Muggleton S, Kozaki Ket al., 2018, Preface, ISBN: 9783030042837

Book

Hocquette C, Muggleton S, 2018, How Much Can Experimental Cost Be Reduced in Active Learning of Agent Strategies?, Pages: 38-53, ISSN: 0302-9743

In science, experiments are empirical observations allowing for the arbitration of competing hypotheses and knowledge acquisition. For a scientist that aims at learning an agent strategy, performing experiments involves costs. To that extent, the efficiency of a learning process relies on the number of experiments performed. We study in this article how the cost of experimentation can be reduced with active learning to learn efficient agent strategies. We consider an extension of the meta-interpretive learning framework that allocates a Bayesian posterior distribution over the hypothesis space. At each iteration, the learner queries the label of the instance with maximum entropy. This produces the maximal discriminative over the remaining competing hypotheses, and thus achieves the highest shrinkage of the version space. We study the theoretical framework and evaluate the gain on the cost of experimentation for the task of learning regular grammars and agent strategies: our results demonstrate the number of experiments to perform to reach an arbitrary accuracy level can at least be halved.

Conference paper

Dai W-Z, Muggleton S, Wen J, Tamaddoni-Nezhad A, Zhou Z-Het al., 2018, Logical Vision: One-Shot Meta-Interpretive Learning from Real Images, 27th International Conference on Inductive Logic Programming (ILP), Publisher: SPRINGER INTERNATIONAL PUBLISHING AG, Pages: 46-62, ISSN: 0302-9743

Conference paper

Conn H, Muggleton SH, 2018, The effect of predicate order on curriculum learning in ILP, 27th International Conference on Inductive Logic Programming, Pages: 17-21, ISSN: 1613-0073

© by the paper's authors. Development of effective methods for learning large programs is arguably one of the hardest unsolved problems within ILP. The most obvious approach involves learning a sequence of predicate definitions incrementally. This approach is known as Curriculum Learning. However, Quinlan and Cameron-Jones' paper from 1993 indicates difficulties in this approach since the predictive accuracy of ILP systems, such as FOIL, rapidly degrades given a growing set of learned background predicates, even when a reasonable ordering over the predicate sequence is chosen. Limited progress was made in this problem until the recent advent of bias-reformulation methods within Meta-Interpretive Learning. In this paper we show empirically that given a well-ordered predicate sequence, relatively large sets of dyadic predicates can be learned incrementally using a state-of-the-art Meta-Interpretive Learning system which employs a Universal Set of metarules. However, further experiments show how progressive random permutations of the sequence rapidly degrades performance in a fashion comparable to Quinlan and Cameron-Jones's results. On the basis of these results we propose the need for further identification of methods for identifying well-ordered predicate sequences to address this issue.

Conference paper

Schmid U, Muggleton SH, Singh R, 2017, Approaches and Applications of Inductive Programming (Dagstuhl Seminar 17382)., Dagstuhl Seminar 17382, Pages: 86-108

Conference paper

Tamaddoni-Nezhad A, Besold T, Schmid U, Zeller C, Muggleton SHet al., 2017, How does Predicate Invention affect Human Comprehensibility?, Inductive Logic Programming, 26th International Conference

Conference paper

Muggleton SH, 2017, Meta-Interpretive Learning: Achievements and Challenges, 1st International Joint Conference on Rules and Reasoning (RuleML+RR), Publisher: SPRINGER INTERNATIONAL PUBLISHING AG, Pages: 1-6, ISSN: 0302-9743

Conference paper

Schmid U, Zeller C, Besold T, Tamaddoni-Nezhad A, Muggleton Set al., 2017, How does predicate invention affect human comprehensibility?, Pages: 52-67, ISSN: 0302-9743

During the 1980s Michie defined Machine Learning in terms of two orthogonal axes of performance: predictive accuracy and comprehensibility of generated hypotheses. Since predictive accuracy was readily measurable and comprehensibility not so, later definitions in the 1990s, such as that of Mitchell, tended to use a one-dimensional approach to Machine Learning based solely on predictive accuracy, ultimately favouring statistical over symbolic Machine Learning approaches. In this paper we provide a definition of comprehensibility of hypotheses which can be estimated using human participant trials. We present the results of experiments testing human comprehensibility of logic programs learned with and without predicate invention. Results indicate that comprehensibility is affected not only by the complexity of the presented program but also by the existence of anonymous predicate symbols.

Conference paper

Cropper A, Tamaddoni-Nezhad A, Muggleton SH, 2016, Meta-interpretive learning of data transformation programs, 25th International Conference, ILP 2015, Publisher: Springer, Pages: 46-59, ISSN: 0302-9743

Data transformation involves the manual construction of large numbers of special-purpose programs. Although typically small, such programs can be complex, involving problem decomposition, recursion, and recognition of context. Building such programs is common in commercial and academic data analytic projects and can be labour intensive and expensive, making it a suitable candidate for machine learning. In this paper, we use the meta-interpretive learning framework (MIL) to learn recursive data transformation programs from small numbers of examples. MIL is well suited to this task because it supports problem decomposition through predicate invention, learning recursive programs, learning from few examples, and learning from only positive examples. We apply Metagol, a MIL implementation, to both semi-structured and unstructured data. We conduct experiments on three real-world datasets: medical patient records, XML mondial records, and natural language taken from ecological papers. The experimental results suggest that high levels of predictive accuracy can be achieved in these tasks from small numbers of training examples, especially when learning with recursion.

Conference paper

This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.

Request URL: http://wlsprd.imperial.ac.uk:80/respub/WEB-INF/jsp/search-html.jsp Request URI: /respub/WEB-INF/jsp/search-html.jsp Query String: respub-action=search.html&id=00334581&limit=30&person=true