Sukrut Rao (PhD Student)

Sukrut Sridhar Rao

Address: Max-Planck-Institut für Informatik
Saarland Informatics Campus
Campus E1 4
66123 Saarbrücken
Standort: E1 4 - 628
Telefon: +49 681 9325 2146
Fax: +49 681 9325 2099
E-mail: Get email via email

Personal Information

Web Pages: Personal, MPI
Publication Profiles: Google Scholar, DBLP, ORCiD
Social: LinkedIn, Twitter
Code: GitHub

Publications

2024

Article

S. Rao, M. Boehle, and B. Schiele

“Better Understanding Differences in Attribution Methods via Systematic Evaluations,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 6, 2024.

mehr

BibTeX

@article{RaoTPAMI24,
TITLE = {Better Understanding Differences in Attribution Methods via Systematic Evaluations},
AUTHOR = {Rao, Sukrut and Boehle, Moritz and Schiele, Bernt},
LANGUAGE = {eng},
DOI = {10.1109/TPAMI.2024.3353528},
PUBLISHER = {IEEE},
ADDRESS = {Piscataway, NJ},
YEAR = {2024},
DATE = {2024},
JOURNAL = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
VOLUME = {46},
NUMBER = {6},
PAGES = {4090--4101},
}

Endnote

%0 Journal Article
%A Rao, Sukrut
%A Boehle, Moritz
%A Schiele, Bernt
%+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T Better Understanding Differences in Attribution Methods via Systematic Evaluations : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-5532-9
%R 10.1109/TPAMI.2024.3353528
%7 2024
%D 2024
%J IEEE Transactions on Pattern Analysis and Machine Intelligence
%V 46
%N 6
%& 4090
%P 4090 - 4101
%I IEEE
%C Piscataway, NJ

Paper

A. Parchami-Araghi, M. D. Boehle, S. S. Rao, and B. Schiele

“Good Teachers Explain: Explanation-Enhanced Knowledge Distillation,” 2024. [Online]. Available: https://arxiv.org/abs/2402.03119.

mehr

Abstract

Knowledge Distillation (KD) has proven effective for compressing large
teacher models into smaller student models. While it is well known that student
models can achieve similar accuracies as the teachers, it has also been shown
that they nonetheless often do not learn the same function. It is, however,
often highly desirable that the student's and teacher's functions share similar
properties such as basing the prediction on the same input features, as this
ensures that students learn the 'right features' from the teachers. In this
work, we explore whether this can be achieved by not only optimizing the
classic KD loss but also the similarity of the explanations generated by the
teacher and the student. Despite the idea being simple and intuitive, we find
that our proposed 'explanation-enhanced' KD (e$^2$KD) (1) consistently provides
large gains in terms of accuracy and student-teacher agreement, (2) ensures
that the student learns from the teacher to be right for the right reasons and
to give similar explanations, and (3) is robust with respect to the model
architectures, the amount of training data, and even works with 'approximate',
pre-computed explanations.

BibTeX

@online{ParchamiAraghi2402.03119,
TITLE = {Good Teachers Explain: Explanation-Enhanced Knowledge Distillation},
AUTHOR = {Parchami-Araghi, Amin and Boehle, Moritz Daniel and Rao, Sukrut Sridhar and Schiele, Bernt},
LANGUAGE = {eng},
URL = {https://arxiv.org/abs/2402.03119},
EPRINT = {2402.03119},
EPRINTTYPE = {arXiv},
YEAR = {2024},
ABSTRACT = {Knowledge Distillation (KD) has proven effective for compressing large<br>teacher models into smaller student models. While it is well known that student<br>models can achieve similar accuracies as the teachers, it has also been shown<br>that they nonetheless often do not learn the same function. It is, however,<br>often highly desirable that the student's and teacher's functions share similar<br>properties such as basing the prediction on the same input features, as this<br>ensures that students learn the 'right features' from the teachers. In this<br>work, we explore whether this can be achieved by not only optimizing the<br>classic KD loss but also the similarity of the explanations generated by the<br>teacher and the student. Despite the idea being simple and intuitive, we find<br>that our proposed 'explanation-enhanced' KD (e$^2$KD) (1) consistently provides<br>large gains in terms of accuracy and student-teacher agreement, (2) ensures<br>that the student learns from the teacher to be right for the right reasons and<br>to give similar explanations, and (3) is robust with respect to the model<br>architectures, the amount of training data, and even works with 'approximate',<br>pre-computed explanations.<br>},
}

Endnote

%0 Report
%A Parchami-Araghi, Amin
%A Boehle, Moritz Daniel
%A Rao, Sukrut Sridhar
%A Schiele, Bernt
%+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T Good Teachers Explain: Explanation-Enhanced Knowledge Distillation : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000F-5534-7
%U https://arxiv.org/abs/2402.03119
%D 2024
%X   Knowledge Distillation (KD) has proven effective for compressing large<br>teacher models into smaller student models. While it is well known that student<br>models can achieve similar accuracies as the teachers, it has also been shown<br>that they nonetheless often do not learn the same function. It is, however,<br>often highly desirable that the student's and teacher's functions share similar<br>properties such as basing the prediction on the same input features, as this<br>ensures that students learn the 'right features' from the teachers. In this<br>work, we explore whether this can be achieved by not only optimizing the<br>classic KD loss but also the similarity of the explanations generated by the<br>teacher and the student. Despite the idea being simple and intuitive, we find<br>that our proposed 'explanation-enhanced' KD (e$^2$KD) (1) consistently provides<br>large gains in terms of accuracy and student-teacher agreement, (2) ensures<br>that the student learns from the teacher to be right for the right reasons and<br>to give similar explanations, and (3) is robust with respect to the model<br>architectures, the amount of training data, and even works with 'approximate',<br>pre-computed explanations.<br>
%K Computer Science, Computer Vision and Pattern Recognition, cs.CV,Computer Science, Artificial Intelligence, cs.AI,Computer Science, Learning, cs.LG

2023

Conference paper

S. Rao, M. Böhle, A. Parchami-Araghi, and B. Schiele

“Studying How to Efficiently and Effectively Guide Models with Explanations,” in IEEE/CVF International Conference on Computer Vision (ICCV 2023), Paris, France, 2023.

mehr

BibTeX

@inproceedings{Rao_ICCV23,
TITLE = {Studying How to Efficiently and Effectively Guide Models with Explanations},
AUTHOR = {Rao, Sukrut and B{\"o}hle, Moritz and Parchami-Araghi, Amin and Schiele, Bernt},
LANGUAGE = {eng},
ISBN = {979-8-3503-0718-4},
DOI = {10.1109/ICCV51070.2023.00184},
PUBLISHER = {IEEE},
YEAR = {2023},
MARGINALMARK = {$\bullet$},
DATE = {2023},
BOOKTITLE = {IEEE/CVF International Conference on Computer Vision (ICCV 2023)},
PAGES = {1922--1933},
ADDRESS = {Paris, France},
}

Endnote

%0 Conference Proceedings
%A Rao, Sukrut
%A B&#246;hle, Moritz
%A Parchami-Araghi, Amin
%A Schiele, Bernt
%+ Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T Studying How to Efficiently and Effectively Guide Models with Explanations : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000D-CA7B-6
%R 10.1109/ICCV51070.2023.00184
%D 2023
%B IEEE/CVF International Conference on Computer Vision
%Z date of event: 2023-10-02 - 2023-10-06
%C Paris, France
%B IEEE/CVF International Conference on Computer Vision
%P 1922 - 1933
%I IEEE
%@ 979-8-3503-0718-4

2022

Conference paper

D4D2

S. Rao, M. Böhle, and B. Schiele

“Towards Better Understanding Attribution Methods,” in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2022), New Orleans, LA, USA, 2022.

mehr

BibTeX

@inproceedings{Rao_CVPR2022,
TITLE = {Towards Better Understanding Attribution Methods},
AUTHOR = {Rao, Sukrut and B{\"o}hle, Moritz and Schiele, Bernt},
LANGUAGE = {eng},
ISBN = {978-1-6654-6946-3},
DOI = {10.1109/CVPR52688.2022.00998},
PUBLISHER = {IEEE},
YEAR = {2022},
MARGINALMARK = {$\bullet$},
BOOKTITLE = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2022)},
PAGES = {10213--10222},
ADDRESS = {New Orleans, LA, USA},
}

Endnote

%0 Conference Proceedings
%A Rao, Sukrut
%A B&#246;hle, Moritz
%A Schiele, Bernt
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T Towards Better Understanding Attribution Methods : 
%G eng
%U http://hdl.handle.net/21.11116/0000-000A-6F91-6
%R 10.1109/CVPR52688.2022.00998
%D 2022
%B 35th IEEE/CVF Conference on Computer Vision and Pattern Recognition
%Z date of event: 2022-06-19 - 2022-06-24
%C New Orleans, LA, USA
%B IEEE/CVF Conference on Computer Vision and Pattern Recognition
%P 10213 - 10222
%I IEEE
%@ 978-1-6654-6946-3

2020

Conference paper

D4D2

S. Rao, D. Stutz, and B. Schiele

“Adversarial Training Against Location-Optimized Adversarial Patches,” in Computer Vision -- ECCV Workshops 2020, Glasgow, UK, 2021.

mehr

BibTeX

@inproceedings{DBLP:conf/eccv/RaoSS20,
TITLE = {Adversarial Training Against Location-Optimized Adversarial Patches},
AUTHOR = {Rao, Sukrut and Stutz, David and Schiele, Bernt},
LANGUAGE = {eng},
ISBN = {978-3-030-68237-8},
DOI = {10.1007/978-3-030-68238-5_32},
PUBLISHER = {Springer},
YEAR = {2020},
MARGINALMARK = {$\bullet$},
DATE = {2021},
BOOKTITLE = {Computer Vision -- ECCV Workshops 2020},
EDITOR = {Bartoli, Adrian and Fusiello, Andrea},
PAGES = {429--448},
SERIES = {Lecture Notes in Computer Science},
VOLUME = {12539},
ADDRESS = {Glasgow, UK},
}

Endnote

%0 Conference Proceedings
%A Rao, Sukrut
%A Stutz, David
%A Schiele, Bernt
%+ Computer Graphics, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
Computer Vision and Machine Learning, MPI for Informatics, Max Planck Society
%T Adversarial Training Against Location-Optimized Adversarial Patches : 
%G eng
%U http://hdl.handle.net/21.11116/0000-0008-1662-1
%R 10.1007/978-3-030-68238-5_32
%D 2021
%B 16th European Conference on Computer Vision
%Z date of event: 2020-08-23 - 2020-08-28
%C Glasgow, UK
%B Computer Vision -- ECCV Workshops 2020 
%E Bartoli, Adrian; Fusiello, Andrea
%P 429 - 448
%I Springer
%@ 978-3-030-68237-8
%B Lecture Notes in Computer Science
%N 12539