Publications
435 results found
Albini E, Rago A, Baroni P, et al., 2021, Influence-Driven Explanations for Bayesian Network Classifiers., Publisher: Springer, Pages: 88-100
Cyras K, Rago A, Albini E, et al., 2021, Argumentative XAI: A Survey., Publisher: ijcai.org, Pages: 4392-4399
Lauren S, Belardinelli F, Toni F, 2020, Aggregating Bipolar Opinions, 20th International Conference on Autonomous Agents and Multiagent Systems
Kotonya N, Toni F, 2020, Explainable Automated Fact-Checking: A Survey, Barcelona. Spain, 28th International Conference on Computational Linguistics (COLING 2020), Publisher: International Committee on Computational Linguistics, Pages: 5430-5443
A number of exciting advances have been made in automated fact-checkingthanks to increasingly larger datasets and more powerful systems, leading toimprovements in the complexity of claims which can be accurately fact-checked.However, despite these advances, there are still desirable functionalitiesmissing from the fact-checking pipeline. In this survey, we focus on theexplanation functionality -- that is fact-checking systems providing reasonsfor their predictions. We summarize existing methods for explaining thepredictions of fact-checking systems and we explore trends in this topic.Further, we consider what makes for good explanations in this specific domainthrough a comparative analysis of existing fact-checking explanations againstsome desirable properties. Finally, we propose further research directions forgenerating fact-checking explanations, and describe how these may lead toimprovements in the research area.v
Kotonya N, Toni F, 2020, Explainable Automated Fact-Checking for Public Health Claims, 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP(1) 2020), Publisher: ACL, Pages: 7740-7754
Fact-checking is the task of verifying the veracity of claims by assessing their assertions against credible evidence. The vast major-ity of fact-checking studies focus exclusively on political claims. Very little research explores fact-checking for other topics, specifically subject matters for which expertise is required. We present the first study of explainable fact-checking for claims which require specific expertise. For our case study we choose the setting of public health. To support this case study we construct a new datasetPUBHEALTHof 11.8K claims accompanied by journalist crafted, gold standard explanations(i.e., judgments) to support the fact-check la-bels for claims1. We explore two tasks: veracity prediction and explanation generation. We also define and evaluate, with humans and computationally, three coherence properties of explanation quality. Our results indicate that,by training on in-domain data, gains can be made in explainable, automated fact-checking for claims which require specific expertise.
Lertvittayakumjorn P, Specia L, Toni F, 2020, FIND: human-in-the-loop debugging deep text classifiers
Since obtaining a perfect training dataset (i.e., a dataset which isconsiderably large, unbiased, and well-representative of unseen cases) ishardly possible, many real-world text classifiers are trained on the available,yet imperfect, datasets. These classifiers are thus likely to have undesirableproperties. For instance, they may have biases against some sub-populations ormay not work effectively in the wild due to overfitting. In this paper, wepropose FIND -- a framework which enables humans to debug deep learning textclassifiers by disabling irrelevant hidden features. Experiments show that byusing FIND, humans can improve CNN text classifiers which were trained underdifferent types of imperfect datasets (including datasets with biases anddatasets with dissimilar train-test distributions).
Lertvittayakumjorn P, Specia L, Toni F, 2020, FIND: Human-in-the-loop debugging deep text classifiers, 2020 Conference on Empirical Methods in Natural Language Processing, Publisher: ACL
Since obtaining a perfect training dataset (i.e., a dataset which is considerably large, unbiased, and well-representative of unseen cases)is hardly possible, many real-world text classifiers are trained on the available, yet imperfect, datasets. These classifiers are thus likely to have undesirable properties. For instance, they may have biases against some sub-populations or may not work effectively in the wild due to overfitting. In this paper, we propose FIND–a framework which enables humans to debug deep learning text classifiers by disabling irrelevant hidden features. Experiments show that by using FIND, humans can improve CNN text classifiers which were trained under different types of imperfect datasets (including datasets with biases and datasets with dissimilar train-test distributions).
Albini E, Baroni P, Rago A, et al., 2020, PageRank as an Argumentation Semantics, Biennial International Conference on Computational Models of Argument (COMMA), Publisher: IOS PRESS, Pages: 55-66, ISSN: 0922-6389
Cocarascu O, Stylianou A, Cyras K, et al., 2020, Data-empowered argumentation for dialectically explainable predictions, 24th European Conference on Artificial Intelligence (ECAI 2020), Publisher: IOS Press, Pages: 2449-2456
Today’s AI landscape is permeated by plentiful data anddominated by powerful data-centric methods with the potential toimpact a wide range of human sectors. Yet, in some settings this po-tential is hindered by these data-centric AI methods being mostlyopaque. Considerable efforts are currently being devoted to defin-ing methods for explaining black-box techniques in some settings,while the use of transparent methods is being advocated in others,especially when high-stake decisions are involved, as in healthcareand the practice of law. In this paper we advocate a novel transpar-ent paradigm of Data-Empowered Argumentation (DEAr in short)for dialectically explainable predictions. DEAr relies upon the ex-traction of argumentation debates from data, so that the dialecticaloutcomes of these debates amount to predictions (e.g. classifications)that can be explained dialectically. The argumentation debates con-sist of (data) arguments which may not be linguistic in general butmay nonetheless be deemed to be ‘arguments’ in that they are dialec-tically related, for instance by disagreeing on data labels. We illus-trate and experiment with the DEAr paradigm in three settings, mak-ing use, respectively, of categorical data, (annotated) images and text.We show empirically that DEAr is competitive with another transpar-ent model, namely decision trees (DTs), while also providing natu-rally dialectical explanations.
Paulino-Passos G, Toni F, 2020, Cautious Monotonicity in Case-Based Reasoning with Abstract Argumentation
Recently, abstract argumentation-based models of case-based reasoning($AA{\text -}CBR$ in short) have been proposed, originally inspired by thelegal domain, but also applicable as classifiers in different scenarios,including image classification, sentiment analysis of text, and in predictingthe passage of bills in the UK Parliament. However, the formal properties of$AA{\text -}CBR$ as a reasoning system remain largely unexplored. In thispaper, we focus on analysing the non-monotonicity properties of a regularversion of $AA{\text -}CBR$ (that we call $AA{\text -}CBR_{\succeq}$).Specifically, we prove that $AA{\text -}CBR_{\succeq}$ is not cautiouslymonotonic, a property frequently considered desirable in the literature ofnon-monotonic reasoning. We then define a variation of $AA{\text-}CBR_{\succeq}$ which is cautiously monotonic, and provide an algorithm forobtaining it. Further, we prove that such variation is equivalent to using$AA{\text -}CBR_{\succeq}$ with a restricted casebase consisting of all"surprising" cases in the original casebase.
Baroni P, Toni F, Verheij B, 2020, On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games: 25 years later Foreword, ARGUMENT & COMPUTATION, Vol: 11, Pages: 1-14, ISSN: 1946-2166
Čyras K, Karamlou A, Lee M, et al., 2020, AI-assisted schedule explainer for nurse rostering, AAMAS, Pages: 2101-2103, ISSN: 1548-8403
We present an argumentation-supported explanation generating system, called Schedule Explainer, that assists with makespan scheduling. Our stand-alone generic tool explains to a lay user why a resource allocation schedule is good or not, and offers actions to improve the schedule given the user's constraints. Schedule Explainer provides actionable textual explanations via an interactive graphical interface. We illustrate our system with a proof-of-concept application tool in a nurse rostering scenario whereby a shift-lead nurse aims to account for unexpected events by rescheduling some patient procedures to nurses and is aided by the system to do so.
Albini E, Rago A, Baroni P, et al., 2020, Relation-Based Counterfactual Explanations for Bayesian Network Classifiers, The 29th International Joint Conference on Artificial Intelligence (IJCAI 2020)
Cocarascu O, Cabrio E, Villata S, et al., 2020, A dataset independent set of baselines for relation prediction in argument mining., Publisher: arXiv
Argument Mining is the research area which aims at extracting argument components and predicting argumentative relations (i.e.,support and attack) from text. In particular, numerous approaches have been proposed in the literature to predict the relations holding between the arguments, and application-specific annotated resources were built for this purpose. Despite the fact that these resources have been created to experiment on the same task, the definition of a single relation prediction method to be successfully applied to a significant portion of these datasets is an open research problem in Argument Mining. This means that none of the methods proposed in the literature can be easily ported from one resource to another. In this paper, we address this problem by proposing a set of dataset independent strong neural baselines which obtain homogeneous results on all the datasets proposed in the literature for the argumentative relation prediction task. Thus, our baselines can be employed by the Argument Mining community to compare more effectively how well a method performs on the argumentative relation prediction task.
Cocarascu O, Toni F, 2020, Deploying Machine Learning Classifiers for Argumentative Relations “in the Wild”, Argumentation Library, Pages: 269-285
Argument Mining (AM) aims at automatically identifying arguments and components of arguments in text, as well as at determining the relations between these arguments, on various annotated corpora using machine learning techniques (Lippi & Torroni, 2016).
Cocarascu O, Rago A, Toni F, 2020, Explanation via Machine Arguing, Pages: 53-84, ISBN: 9783030600662
As AI becomes ever more ubiquitous in our everyday lives, its ability to explain to and interact with humans is evolving into a critical research area. Explainable AI (XAI) has therefore emerged as a popular topic but its research landscape is currently very fragmented. Explanations in the literature have generally been aimed at addressing individual challenges and are often ad-hoc, tailored to specific AIs and/or narrow settings. Further, the extraction of explanations is no simple task; the design of the explanations must be fit for purpose, with considerations including, but not limited to: Is the model or a result being explained? Is the explanation suited to skilled or unskilled explainees? By which means is the information best exhibited? How may users interact with the explanation? As these considerations rise in number, it quickly becomes clear that a systematic way to obtain a variety of explanations for a variety of users and interactions is much needed. In this tutorial we will overview recent approaches showing how these challenges can be addressed by utilising forms of machine arguing as the scaffolding underpinning explanations that are delivered to users. Machine arguing amounts to the deployment of methods from computational argumentation in AI with suitably mined argumentation frameworks, which provide abstractions of “debates”. Computational argumentation has been widely used to support applications requiring information exchange between AI systems and users, facilitated by the fact that the capability of arguing is pervasive in human affairs and arguing is core to a multitude of human activities: humans argue to explain, interact and exchange information. Our lecture will focus on how machine arguing can serve as the driving force of explanations in AI in different ways, namely: by building explainable systems with argumentative foundations from linguistic data focusing on reviews), or by extracting argumentative reasoning from existin
Kotonya N, Toni F, 2020, Explainable Automated Fact-Checking for Public Health Claims., Publisher: Association for Computational Linguistics, Pages: 7740-7754
Jha R, Belardinelli F, Toni F, 2020, Formal Verification of Debates in Argumentation Theory., CoRR, Vol: abs/1912.05828
Jha R, Belardinelli F, Toni F, 2020, Formal verification of debates in argumentation theory., Publisher: ACM, Pages: 940-947
Cocarascu O, Cabrio E, Villata S, et al., 2020, Dataset Independent Baselines for Relation Prediction in Argument Mining., Publisher: IOS Press, Pages: 45-52
Toni F, 2020, From Computational Argumentation to Explanation., Publisher: CEUR-WS.org, Pages: 1-1
Altuncu MT, Sorin E, Symons JD, et al., 2019, Extracting information from free text through unsupervised graph-based clustering: an application to patient incident records
The large volume of text in electronic healthcare records often remainsunderused due to a lack of methodologies to extract interpretable content. Herewe present an unsupervised framework for the analysis of free text thatcombines text-embedding with paragraph vectors and graph-theoretical multiscalecommunity detection. We analyse text from a corpus of patient incident reportsfrom the National Health Service in England to find content-based clusters ofreports in an unsupervised manner and at different levels of resolution. Ourunsupervised method extracts groups with high intrinsic textual consistency andcompares well against categories hand-coded by healthcare personnel. We alsoshow how to use our content-driven clusters to improve the supervisedprediction of the degree of harm of the incident based on the text of thereport. Finally, we discuss future directions to monitor reports over time, andto detect emerging trends outside pre-existing categories.
Lertvittayakumjorn P, Toni F, 2019, Human-grounded evaluations of explanation methods for text classification, 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Publisher: ACL Anthology, Pages: 5195-5205
Due to the black-box nature of deep learning models, methods for explaining the models’ results are crucial to gain trust from humans and support collaboration between AIsand humans. In this paper, we consider several model-agnostic and model-specific explanation methods for CNNs for text classification and conduct three human-grounded evaluations, focusing on different purposes of explanations: (1) revealing model behavior, (2)justifying model predictions, and (3) helping humans investigate uncertain predictions.The results highlight dissimilar qualities of thevarious explanation methods we consider andshow the degree to which these methods couldserve for each purpose.
Schulz C, Toni F, 2019, On the responsibility for undecisiveness in preferred and stable labellings in abstract argumentation (extended abstract), IJCAI International Joint Conference on Artificial Intelligence, Pages: 6382-6386, ISSN: 1045-0823
© 2019 International Joint Conferences on Artificial Intelligence. All rights reserved. Different semantics of abstract Argumentation Frameworks (AFs) provide different levels of decisiveness for reasoning about the acceptability of conflicting arguments. The stable semantics is useful for applications requiring a high level of decisiveness, as it assigns to each argument the label “accepted” or the label “rejected”. Unfortunately, stable labellings are not guaranteed to exist, thus raising the question as to which parts of AFs are responsible for the non-existence. In this paper, we address this question by investigating a more general question concerning preferred labellings (which may be less decisive than stable labellings but are always guaranteed to exist), namely why a given preferred labelling may not be stable and thus undecided on some arguments. In particular, (1) we give various characterisations of parts of an AF, based on the given preferred labelling, and (2) we show that these parts are indeed responsible for the undecisiveness if the preferred labelling is not stable. We then use these characterisations to explain the non-existence of stable labellings.
Čyras K, Birch D, Guo Y, et al., 2019, Explanations by arbitrated argumentative dispute, Expert Systems with Applications, Vol: 127, Pages: 141-156, ISSN: 0957-4174
Explaining outputs determined algorithmically by machines is one of the most pressing and studied problems in Artificial Intelligence (AI) nowadays, but the equally pressing problem of using AI to explain outputs determined by humans is less studied. In this paper we advance a novel methodology integrating case-based reasoning and computational argumentation from AI to explain outcomes, determined by humans or by machines, indifferently, for cases characterised by discrete (static) features and/or (dynamic) stages. At the heart of our methodology lies the concept of arbitrated argumentative disputesbetween two fictitious disputants arguing, respectively, for or against a case's output in need of explanation, and where this case acts as an arbiter. Specifically, in explaining the outcome of a case in question, the disputants put forward as arguments relevant cases favouring their respective positions, with arguments/cases conflicting due to their features, stages and outcomes, and the applicability of arguments/cases arbitrated by the features and stages of the case in question. We in addition use arbitrated dispute trees to identify the excess features that help the winning disputant to win the dispute and thus complement the explanation. We evaluate our novel methodology theoretically, proving desirable properties thereof, and empirically, in the context of primary legislation in the United Kingdom (UK), concerning the passage of Bills that may or may not become laws. High-level factors underpinning a Bill's passage are its content-agnostic features such as type, number of sponsors, ballot order, as well as the UK Parliament's rules of conduct. Given high numbers of proposed legislation (hundreds of Bills a year), it is hard even for legal experts to explain on a large scale why certain Bills pass or not. We show how our methodology can address this problem by automatically providing high-level explanations of why Bills pass or not, based on the given Bills and the
Cocarascu O, Rago A, Toni F, 2019, From formal argumentation to conversational systems, 1st Workshop on Conversational Interaction Systems (WCIS 2019)
Arguing is amenable to humans and argumentation serves as anatural form of interaction in many settings. Several formal mod-els of argumentation have been proposed in the AI literature asabstractions of various forms of debates. We show how these mod-els can serve as the backbone of conversational systems that canexplain machine-computed outputs. These systems can engage inconversations with humans following templates instantiated onargumentation models that are automatically obtained from thedata analysis underpinning the machine-computed outputs. Asan illustration, we consider one such argumentation-empoweredconversational system and exemplify its use and benefits in twodifferent domains, for recommending movies and hotels based onthe aggregation of information drawn from reviews.
Čyras K, Letsios D, Misener R, et al., 2019, Argumentation for explainable scheduling, Thirty-Third AAAI Conference on Artificial Intelligence (AAAI-19), Publisher: AAAI, Pages: 2752-2759
Mathematical optimization offers highly-effective tools for finding solutions for problems with well-defined goals, notably scheduling. However, optimization solvers are often unexplainable black boxes whose solutions are inaccessible to users and which users cannot interact with. We define a novel paradigm using argumentation to empower the interaction between optimization solvers and users, supported by tractable explanations which certify or refute solutions. A solution can be from a solver or of interest to a user (in the context of 'what-if' scenarios). Specifically, we define argumentative and natural language explanations for why a schedule is (not) feasible, (not) efficient or (not) satisfying fixed user decisions, based on models of the fundamental makespan scheduling problem in terms of abstract argumentation frameworks (AFs). We define three types of AFs, whose stable extensions are in one-to-one correspondence with schedules that are feasible, efficient and satisfying fixed decisions, respectively. We extract the argumentative explanations from these AFs and the natural language explanations from the argumentative ones.
Karamlou A, Cyras K, Toni F, 2019, Complexity results and algorithms for bipolar argumentation, International Conference on Autonomous Agents and MultiAgent Systems, Publisher: ACM, Pages: 1713-1721
Bipolar Argumentation Frameworks (BAFs) admit several interpretations of the support relation and diverging definitions of semantics. Recently, several classes of BAFs have been captured as instances of bipolar Assumption-Based Argumentation, a class of Assumption-Based Argumentation (ABA). In this paper, we establish the complexity of bipolar ABA, and consequently of several classes of BAFs. In addition to the standard five complexity problems, we analyse the rarely-addressed extension enumeration problem too. We also advance backtracking-driven algorithms for enumerating extensions of bipolar ABA frameworks, and consequently of BAFs under several interpretations. We prove soundness and completeness of our algorithms, describe their implementation and provide a scalability evaluation. We thus contribute to the study of the as yet uninvestigated complexity problems of (variously interpreted) BAFs as well as of bipolar ABA, and provide the lacking implementations thereof.
Karamlou A, Cyras K, Toni F, 2019, Deciding the winner of a debate using bipolar argumentation, International Conference on Autonomous Agents and MultiAgent Systems, Publisher: IFAAMAS / ACM, Pages: 2366-2368, ISSN: 2523-5699
Bipolar Argumentation Frameworks (BAFs) are an important class of argumentation frameworks useful for capturing, reasoning with, and deriving conclusions from debates. They have the potential to make solid contributions to real-world multi-agent systems and human-agent interaction in domains such as legal reasoning, healthcare and politics. Despite this fact, practical systems implementing BAFs are largely lacking. In this demonstration, we provide a software system implementing novel algorithms for calculating extensions (winning sets of arguments) of BAFs. Participants in the demonstration will be able to input their own debates into our system, and watch a graphical representation of the algorithms as they process information and decide which sets of arguments are winners of the debate.
Cocarascu O, Rago A, Toni F, 2019, Extracting dialogical explanations for review aggregations with argumentative dialogical agents, International Conference on Autonomous Agents and MultiAgent Systems (AAMAS), Publisher: International Foundation for Autonomous Agents and Multiagent Systems
The aggregation of online reviews is fast becoming the chosen method of quality control for users in various domains, from retail to entertainment. Consequently, fair, thorough and explainable aggregation of reviews is increasingly sought-after. We consider the movie review domain, and in particular Rotten Tomatoes' ubiquitous (and arguably over-simplified) aggregation method, the Tomatometer Score (TS). For a movie, this amounts to the percentage of critics giving the movie a positive review. We define a novel form of argumentative dialogical agent (ADA) for explaining the reasoning within the reviews. ADA integrates: 1.) NLP with reviews to extract a Quantitative Bipolar Argumentation Framework (QBAF) for any chosen movie to provide the underlying structure of explanations, and 2.) gradual semantics for QBAFs for deriving a dialectical strength measure for movies, as an alternative to the TS, satisfying desirable properties for obtaining explanations. We evaluate ADA using some prominent NLP methods and gradual semantics for QBAFs. We show that they provide a dialectical strength which is comparable with the TS, while at the same time being able to provide dialogical explanations of why a movie obtained its strength via interactions between the user and ADA.
This data is extracted from the Web of Science and reproduced under a licence from Thomson Reuters. You may not copy or re-distribute this data in whole or in part without the written consent of the Science business of Thomson Reuters.