Abstract
Post-hoc importance attribution methods are a popular tool for "explaining"
Deep Neural Networks (DNNs) and are inherently based on the assumption that the
explanations can be applied independently of how the models were trained.
Contrarily, in this work we bring forward empirical evidence that challenges
this very notion. Surprisingly, we discover a strong dependency on and
demonstrate that the training details of a pre-trained model's classification
layer (less than 10 percent of model parameters) play a crucial role, much more
than the pre-training scheme itself. This is of high practical relevance: (1)
as techniques for pre-training models are becoming increasingly diverse,
understanding the interplay between these techniques and attribution methods is
critical; (2) it sheds light on an important yet overlooked assumption of
post-hoc attribution methods which can drastically impact model explanations
and how they are interpreted eventually. With this finding we also present
simple yet effective adjustments to the classification layers, that can
significantly enhance the quality of model explanations. We validate our
findings across several visual pre-training frameworks (fully-supervised,
self-supervised, contrastive vision-language training) and analyse how they
impact explanations for a wide range of attribution methods on a diverse set of
evaluation metrics.
BibTeX
@online{Gairola2503.00641,
TITLE = {How to Probe: Simple Yet Effective Techniques for Improving Post-hoc Explanations},
AUTHOR = {Gairola, Siddhartha and B{\"o}hle, Moritz and Locatello, Francesco and Schiele, Bernt},
LANGUAGE = {eng},
URL = {https://www.arxiv.org/abs/2503.00641},
EPRINT = {2503.00641},
EPRINTTYPE = {arXiv},
YEAR = {2025},
MARGINALMARK = {$\bullet$},
ABSTRACT = {Post-hoc importance attribution methods are a popular tool for "explaining"<br>Deep Neural Networks (DNNs) and are inherently based on the assumption that the<br>explanations can be applied independently of how the models were trained.<br>Contrarily, in this work we bring forward empirical evidence that challenges<br>this very notion. Surprisingly, we discover a strong dependency on and<br>demonstrate that the training details of a pre-trained model's classification<br>layer (less than 10 percent of model parameters) play a crucial role, much more<br>than the pre-training scheme itself. This is of high practical relevance: (1)<br>as techniques for pre-training models are becoming increasingly diverse,<br>understanding the interplay between these techniques and attribution methods is<br>critical; (2) it sheds light on an important yet overlooked assumption of<br>post-hoc attribution methods which can drastically impact model explanations<br>and how they are interpreted eventually. With this finding we also present<br>simple yet effective adjustments to the classification layers, that can<br>significantly enhance the quality of model explanations. We validate our<br>findings across several visual pre-training frameworks (fully-supervised,<br>self-supervised, contrastive vision-language training) and analyse how they<br>impact explanations for a wide range of attribution methods on a diverse set of<br>evaluation metrics.<br>},
}Endnote
%0 Report %A Gairola, Siddhartha %A Böhle, Moritz %A Locatello, Francesco %A Schiele, Bernt %+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society External Organizations External Organizations Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society %T How to Probe: Simple Yet Effective Techniques for Improving Post-hoc Explanations : %G eng %U http://hdl.handle.net/21.11116/0000-0010-DACE-1 %U https://www.arxiv.org/abs/2503.00641 %D 2025 %X Post-hoc importance attribution methods are a popular tool for "explaining"<br>Deep Neural Networks (DNNs) and are inherently based on the assumption that the<br>explanations can be applied independently of how the models were trained.<br>Contrarily, in this work we bring forward empirical evidence that challenges<br>this very notion. Surprisingly, we discover a strong dependency on and<br>demonstrate that the training details of a pre-trained model's classification<br>layer (less than 10 percent of model parameters) play a crucial role, much more<br>than the pre-training scheme itself. This is of high practical relevance: (1)<br>as techniques for pre-training models are becoming increasingly diverse,<br>understanding the interplay between these techniques and attribution methods is<br>critical; (2) it sheds light on an important yet overlooked assumption of<br>post-hoc attribution methods which can drastically impact model explanations<br>and how they are interpreted eventually. With this finding we also present<br>simple yet effective adjustments to the classification layers, that can<br>significantly enhance the quality of model explanations. We validate our<br>findings across several visual pre-training frameworks (fully-supervised,<br>self-supervised, contrastive vision-language training) and analyse how they<br>impact explanations for a wide range of attribution methods on a diverse set of<br>evaluation metrics.<br> %K Computer Science, Computer Vision and Pattern Recognition, cs.CV