Explainable AI and Its Application to Machine Teaching

Overview

The black-box nature of deep learning systems has motivated a large number of work on Explainable AI (XAI), which complements network predictions with human-understandable explanations. Unlike other literature that tries to interpret the behavior of neural networks or design interpretable networks, we instead in this project aim to take advantage of the powerfulness of XAI results for other machine learning problems. For this purpose, we propose a new family of explanations, denoted deliberative, proposed to address the ''why'' question. Deliberative explanations are a visualization technique that aims to expose the deliberations carried by the network to arrive at a prediction, by exposing the network insecurities about this decision. These are ambiguous regions, which could belong to two or more classes. We also focus on the class of counterfactual explanations, which have been shown to address the ''why not'' question but have so far been generated with procedures that are computationally inefficient, ineffective for complicated imagery, or both. A new approach (SCOUT) is proposed to their computation via a new class of discriminative explanations, which produce heatmaps that attribute high scores to image regions informative of a classifier prediction but not of a counter class.

We work on machine teaching, a young but interesting machine learning task. Its goal is to design systems that can teach real students new concepts efficiently and automatically, for example, designing a machine that teaches human learners to discriminate between different fine-grained image classes, via a carefully selected ordered images. We first correct a plausible hypothesis of current machine teaching algorithms and then propose a new MaxGrad by maximazing gradient of the risk. The counterfactual explanation is merged to the teaching algorithm and shows significant better results. Our method (MEMORABLE) also suggests a new idea to solve the fine-grained expert-domain semi-supervised learning problem where data annotation is hard and expensive, unlike the everyday objects which can be scalably annotated by crowd source workers.

Projects

A Machine Teaching Framework for Scalable Recognition

We consider the scalable recognition problem in the finegrained expert domain where large-scale data collection is easy whereas annotation is difficult. Existing solutions are typically based on semi-supervised or self-supervised learning. We propose an alternative new framework, MEMORABLE, based on machine teaching and online crowdsourcing platforms. A small amount of data is first labeled by experts and then used to teach online annotators for the classes of interest, who finally label the entire dataset. Preliminary studies show that the accuracy of classifiers trained on the final dataset is a function of the accuracy of the student annotators. A new machine teaching algorithm, CMaxGrad, is then proposed to enhance this accuracy by introducing explanations in a state-of-the-art machine teaching algorithm. For this, CMaxGrad leverages counterfactual explanations, which take into account student predictions, thereby proving feedback that is studentspecific, explicitly addresses the causes of student confusion, and adapts to the level of competence of the student. Experiments show that both MEMORABLE and CMaxGrad outperform existing solutions to their respective problems.

paper code

Gradient-Based Algorithms for Machine Teaching

The problem of machine teaching is considered. A new formulation is proposed under the assumption of an optimal student, where optimality is defined in the usual machine learning sense of empirical risk minimization. This is a sensible assumption for machine learning students and for human students in crowdsourcing platforms, who tend to perform at least as well as machine learning systems. It is shown that, if allowed unbounded effort, the optimal student always learns the optimal predictor for a classification task. Hence, the role of the optimal teacher is to select the teaching set that minimizes student effort. This is formulated as a problem of functional optimization where, at each teaching iteration, the teacher seeks to align the steepest descent directions of the risk of (1) the teaching set and (2) entire example population. The optimal teacher, denoted MaxGrad, is then shown to maximize the gradient of the risk on the set of new examples selected per iteration. MaxGrad teaching algorithms are finally provided for both binary and multiclass tasks, and shown to have some similarities with boosting algorithms. Experimental evaluations demonstrate the effectiveness of MaxGrad, which outperforms previous algorithms on the classification task, for both machine learning and human students from MTurk, by a substantial margin.

paper code

SCOUT: Self-aware Discriminant Counterfactual Explanations

The problem of machine teaching is considered. A new formulation is proposed under the assumption of an optimal student, where optimality is defined in the usual machine learning sense of empirical risk minimization. This is a sensible assumption for machine learning students and for human students in crowdsourcing platforms, who tend to perform at least as well as machine learning systems. It is shown that, if allowed unbounded effort, the optimal student always learns the optimal predictor for a classification task. Hence, the role of the optimal teacher is to select the teaching set that minimizes student effort. This is formulated as a problem of functional optimization where, at each teaching iteration, the teacher seeks to align the steepest descent directions of the risk of (1) the teaching set and (2) entire example population. The optimal teacher, denoted MaxGrad, is then shown to maximize the gradient of the risk on the set of new examples selected per iteration. MaxGrad teaching algorithms are finally provided for both binary and multiclass tasks, and shown to have some similarities with boosting algorithms. Experimental evaluations demonstrate the effectiveness of MaxGrad, which outperforms previous algorithms on the classification task, for both machine learning and human students from MTurk, by a substantial margin.

paper code

Deliberative Explanations: visualizing network insecurities

A new approach to explainable AI, denoted deliberative explanations, is proposed. Deliberative explanations are a visualization technique that aims to go beyond the simple visualization of the image regions (or, more generally, input variables) responsible for a network prediction. Instead, they aim to expose the deliberations carried by the network to arrive at that prediction, by uncovering the insecurities of the network about the latter. The explanation consists of a list of insecurities, each composed of 1) an image region (more generally, a set of input variables), and 2) an ambiguity formed by the pair of classes responsible for the network uncertainty about the region. Since insecurity detection requires quantifying the difficulty of network predictions, deliberative explanations combine ideas from the literature on visual explanations and assessment of classification difficulty. More specifically, the proposed implementation combines attributions with respect to both class predictions and a difficulty score. An evaluation protocol that leverages object recognition (CUB200) and scene classification (ADE20K) datasets that combine part and attribute annotations is also introduced to evaluate the accuracy of deliberative explanations. Finally, an experimental evaluation shows that the most accurate explanations are achieved by combining non self-referential difficulty scores and second-order attributions. The resulting insecurities are shown to correlate with regions of attributes shared by different classes. Since these regions are also ambiguous for humans, deliberative explanations are intuitive, suggesting that the deliberative process of modern networks correlates with human reasoning.

paper code

Towards Realistic Predictors

A new class of predictors, denoted realistic predictors, is defined. These are predictors that, like humans, assess the difficulty of examples, reject to work on those that are deemed too hard, but guarantee good performance on the ones they operate on. In this paper, we talk about a particular case of it, realistic classifiers. The central problem in realistic classification, the design of an inductive predictor of hardness scores, is considered. It is argued that this should be a predictor independent of the classifier itself, but tuned to it, and learned without explicit supervision, so as to learn from its mistakes. A new architecture is proposed to accomplish these goals by complementing the classifier with an auxiliary hardness prediction network (HP-Net). Sharing the same inputs as classifiers, the HP-Net outputs the hardness scores to be fed to the classifier as loss weights. Alternatively, the output of classifiers is also fed to HP-Net in a new defined loss, variant of cross entropy loss. The two networks are trained jointly in an adversarial way where, as the classifier learns to improve its predictions, the HP-Net refines its hardness scores. Given the learned hardness predictor, a simple implementation of realistic classifiers is proposed by rejecting examples with large scores. Experimental results not only provide evidence in support of the effectiveness of the proposed architecture and the learned hardness predictor, but also show that the realistic classifier always improves performance on the examples that it accepts to classify, performing better on these examples than an equivalent nonrealistic classifier. All of these make it possible for realistic classifiers to guarantee a good performance.

paper code

Publications

A Machine Teaching Framework for Scalable Recognition
Pei Wang, Nuno Vasconcelos
IEEE International Conference on Computer Vision (ICCV), 2021.

Gradient-Based Algorithms for Machine Teaching
Pei Wang, Kabir Nagrecha, Nuno Vasconcelos
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021.

SCOUT: Self-aware Discriminant Counterfactual Explanations
Pei Wang and Nuno Vasconcelos
IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.

Deliberative Explanations: visualizing network insecurities
Pei Wang and Nuno Vasconcelos
Conference on Neural Information Processing Systems (NeurIPS), 2019.

Towards Realistic Predictors
Pei Wang and Nuno Vasconcelos
European Conference on Computer Vision (ECCV), 2018.

Acknowledgements

This work was partially funded by NSF awards IIS-1924937, IIS-2041009, and gifts from NVIDIA, Amazon and Qualcomm. We also acknowledge and thank the use of the Nautilus platform for some of the experiments in the papers above.