Teaching Categories to Human Learners With Visual Explanations

Oisin Mac Aodha, Shihan Su, Yuxin Chen, Pietro Perona, Yisong Yue; The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 3820-3828

Abstract


We study the problem of computer-assisted teaching with explanations. Conventional approaches for machine teaching typically only provide feedback at the instance level e.g., the category or label of the instance. However, it is intuitive that clear explanations from a knowledgeable teacher can significantly improve a student's ability to learn a new concept. To address these existing limitations, we propose a teaching framework that provides interpretable explanations as feedback and models how the learner incorporates this additional information. In the case of images, we show that we can automatically generate explanations that highlight the parts of the image that are responsible for the class label. Experiments on human learners illustrate that, on average, participants achieve better test set performance on challenging categorization tasks when taught with our interpretable approach compared to existing methods.

Related Material


[pdf] [arXiv]
[bibtex]
@InProceedings{Aodha_2018_CVPR,
author = {Mac Aodha, Oisin and Su, Shihan and Chen, Yuxin and Perona, Pietro and Yue, Yisong},
title = {Teaching Categories to Human Learners With Visual Explanations},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2018}
}