Logic-Based Learning

Module aims

Logic-based learning is a particular area of Machine Learning that aims at learning rule-based knowledge, called hypotheses, from observations (positive and negative), using existing background knowledge and integrity constraints. Learned hypotheses should be able to explain positive observations, classify them from negative observations, and because of their generalization, accurately predict unseen observations. A key characteristic of logic-based learning is that logic is used as the underlying unifying representation language for observations, background knowledge and hypotheses. Various approaches have been developed in AI since the 1980 and recent advances have seen also an increased application to real world problems in domains such as bioinformatics, privacy and security and software engineering in general. In application domains where observations are noisy and concepts to be learned can be fuzzy, probabilistic inference can be combined with logic-based learning. The integration of these two different worlds has recently inspired new research directions and broadened the applicability of logic-based learning. This course aims to provide an in-depth presentation of the current state-of-the-art of logic-based learning, starting from its key foundation concepts and principles and moving to most recent advances with a particular emphasis on learning answer set programs. The course include also an introudction to probabilistic logic programming and probabilistic inference for both paramter learning and inductive learning of declarative models. The aim of this course are:

  1. Acquire foundation knowledge and basic principles of logic-based learning.
  2. Familiarize with state-of-the-art learning algorithms and heuristics, and develop basic skills for formalizing a logic-based learning task to solve a given learning problem.
  3. Explore how probabilistic and logic-based learning can be integrated.
  4. Develop an in-depth understanding of the applicability of these algorithms through analysis of real-case studies in some application domains.

Learning outcomes

Knowledge and Understanding

Upon successful completion of the course, students will develop an in-depth knowledge of

  1. basic concepts of the three classes of logic-based reasoning: deductive, abductive, inductive and what their main differences are.
  2. fundamental characteristics and principles of different logic-based approaches to logic-based learning: bottom-up, top-down, meta-level learning, and predicate invention.
  3. basic semantics frameworks for logic-based learning: monotonic versus non-monotonic learning; brave versus caution induction.
  4. logic-based learning algorithms and their properties: soundness, completeness expressive power, and complexity.  
  5. principles of probabilistic inference and its related models: distribution semantics models.
  6. understanding the scope of applicability of (probabilistic) logic-based learning.

Intellectual and practical skills

Upon successful completion of the course, students will develop a wide range of skills necessary for modeling problem domains in terms of learning tasks. In particular, students will

  • Acquire in-depth familiarity with several state-of-the-art logic-based learning systems.

    • Apply logic-based learning to a variety of problems with the objective of automatically developing underling models
    • Set-up experimental context for evaluating a logic-based learning approach, e.g., cross-validation, sensitivity and specificity metrics, heuristics for improving performance.
    •  Experiment with different parameter learning mechanisms

    Module syllabus

    The course stresses formal and pragmatic skills in logic-based and probabilistic logic-based learning. It teaches the basic principles and techniques for logic-based learning, modeling learning tasks in terms of logic-based learning and key challenges in real world applications. In particular the course covers the following main themes and associated topics:

    Formal reasoning:
    Introduction of deductive, abductive and inductive reasoning, with particular emphasis on abductive reasoning for explanation generation, Bayesian reasoning techniques and ASP computational environment.

    Logic-based learning:
    Logic-based learning as a form of inductive reasoning. Bottom-up approaches to learning: inverse entailment, bottom clause and bottom generalization. Top-down learning techniques, hypothesis specialization and search heuristics. Meta-level logic-based learning as abductive search. Semantic approaches to logic-based learning, with particular emphasis on monotonic and non-monotonic learning, brave and cautious induction.

    Logic-based learning algorithms and systems:
    Brief summary of logic programming and answer set programming. Key learning algorithms with particular emphasis on Progol, TopLog, Metagol, TAL, Imparo, ASPAL and ILASP. Soundness and completeness of these learning algorithms. Probabilistic/stochastic inductive learning.  

    Probabilitic learning:
    Brief introduction of probabilisticv logic programming and ProbLog to support the computation of likely consequences and likely explanations, extension of this approach to probabilistic inductive learning for the paramter adn structural learning. We will look at systems such as ProbLog and ProbILP. 

    Pre-requisites

    1. Prerequisites include Logic, and preferebly Declarative Programming (Prolog), but not required.

    Teaching methods

    3 hours lectures and 1 hour tutorial per week. Students will be able to use exiting learning systems through a dedciatd Web portal. The course will have a single assessed coursework and will be assessed with a written exam in Spring term consisting of 3 questions out of a choice of 4.

    Assessments

    *This is a level 6/H course

    2 small assessed courseworks and 1 written exam paper.

    Reading list

    Module leaders

    Professor Alessandra Russo