Exemplary Natural Images Explain CNN Activations Better than State-of-the-Art Feature Visualizations

Judy Borowski*
University of Tübingen & IMPRS-IS
Roland S. Zimmermann*
University of Tübingen & IMPRS-IS
Judith Schepers
University of Tübingen
Robert Geirhos
University of Tübingen & IMPRS-IS
Thomas S. A. Wallis
Technical University of Darmstadt
Matthias Bethge
University of Tübingen
Wieland Brendel
University of Tübingen

tl;dr: Using human psychophysical experiments, we show that natural images can be significantly more informative for interpreting neural network activations than synthetic feature visualizations.

News

May '21 Poster presentation at ICLR 2021.
January '21 The paper was accepted for a poster presentation at ICLR 2021.
December '20 A shorter workshop version of the paper was accepted for a poster presentation at the Shared Visual Representations in Human & Machine Intelligence Workshop at NeurIPS 2020.
October '20 The pre-print is now available on arXiv.

Abstract

Overview: How useful are synthetic compared to natural images for interpreting neural network activations? Given extremely activating reference images (either synthetic or natural), a human participant chooses which out of two query images is also a strongly activating image. Synthetic images were generated via feature visualization (Olah et al. (2017)).

Why

Feature visualizations such as synthetic maximally activating images are a widely used explanation method. They are used to better understand convolutional neural networks (CNNs) as they grant insights into their learned features. At the same time, there are concerns that these visualizations might not accurately represent CNNs’ inner workings. Here, we investigate how much extremely activating images help humans to predict CNN activations.

What we did

Using a well-controlled psychophysical paradigm, we measure the informativeness of synthetic feature visualizations by Olah et al. (2017). Here’s an example trial that participants did in our lab:

Based on minimally and maximally activating feature visualizations for a certain feature map shown at the sides, a participant is asked to select the image from the center that also strongly activates that feature map.

On the right hand side, we’re seeing maximally activating images; while on the left hand side, we’re seeing minimally activating images. The question always concerns the two images at the center of the screen: Which of them is also a strongly activating image? When breaking this down, all we’re really asking is which image at the center is more similar to those at the right side.[1]

As you can imagine, this task will give us some performance value. In order to set this into context besides the chance level of 50% (if you were to randomly guess at the above task, you would on average get 50% correct), we tested another condition that we intended as a baseline: natural images. This time, instead of showing extremely activating synthetic images at the sides, we displayed natural images that elicit either very low or very high activations. The task regarding the images at the center of the screen still remains the same: Which of the two is also a strongly activating image?

Example trial in psychophysical experiments to measure the baseline performance of natural images.

What we found

First of all, we found that synthetic feature visualizations are indeed helpful: In 82±4% of the trials (that is many more than the chance proportion of 50%), participants chose the truly strongly activating image (blue bar). Surprisingly, though, participants answered the trials even more often correctly when they were given natural reference images at the sides, namely in 92±2% of the trials.

Given synthetic reference images, participants are well above chance (proportion correct: 82 ± 4%), but even better for natural reference images (92 ± 2%).

To see the results of many more conditions, such as performance of lay and expert participants, a comparison between hand-selected and randomly sampled feature maps, check out our paper.

What we take from this

Most importantly, our data show that feature visualizations are indeed helpful for humans - and that natural images can be even more helpful. On a higher level, we take from this that thorough human quantitative evaluations of feature visualizations are needed and that example natural images might provide a surprisingly challenging baseline for understanding CNN activations.

We’re continuing our work and investigating other aspects of the helpfulness of feature visualizations. Stay tuned!

Acknowledgements & Funding

BibTeX

Please cite our paper as follows:

@inproceedings{borowski2021exemplary,
  author = {
    Borowski, Judy and
    Zimmermann, Roland S. and
    Schepers, Judith and
    Geirhos, Robert and
    Wallis, Thomas S. A., and
    Bethge, Matthias and
    Brendel, Wieland
  },
  title = {
    Exemplary Natural Images Explain
    CNN Activations Better than
    State-of-the-Art Feature Visualization
  },
  booktitle = {Ninth International Conference on
    Learning Representations (ICLR 2021)},
  year = {2021},
}
Webpage designed using Bootstrap 4.5.