Research map / Label-efficient and robust learning / Recognition and long-tail

Gnothi Seauton: Empowering Faithful Self-Interpretability in Black-Box Transformers

ICLR 2025 · Label-efficient and robust learning › Recognition and long-tail · approach: Post-hoc explanation analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AutoGnothi, a side-tuning pipeline that adds a small side network to a frozen black-box transformer so it can predict and produce Shapley-value explanations in one inference pass, with theoretical convergence/error bounds.

Problem

Post-hoc Shapley explainers for transformers are costly in training, memory and inference (two separate models/passes), while self-interpretable models often sacrifice accuracy; the paper seeks self-interpretability for black-box transformers without hurting prediction accuracy.

Approach

Attach an additive side network (ladder side-tuning, width scaled by 1/r) to a frozen ViT/BERT so that the side branch first acts as a masked-input surrogate and then as a Shapley-value explainer trained with the Shapley kernel and efficiency constraint, letting prediction and explanation share one forward pass while the original classifier is left unchanged.

Key results

• On ImageNette ViT-base, AutoGnothi explainer reduces trainable parameters by 98% and training memory by 77% relative to ViT Shapley while achieving insertion 0.9874 and deletion 0.7954, the best among 12+ baselines.
• Self-interpretable AutoGnothi model reduces inference FLOPs by 54% and parameters by 54% versus separate classifier plus explainer on ViT-base.
• AutoGnothi surrogate keeps ViT-base accuracy at 0.9959 on masked inputs versus 0.9939 for full fine-tuning, with 97% fewer trainable parameters.

Measured effects reported in the paper

Surprising findings

• Full fine-tuning to add self-interpretability (Duo pipeline) reduced ViT-base prediction accuracy by 4.0%, with gradient conflicts between prediction and explanation tasks.
• The Froyo pipeline, which freezes the encoder and trains only an explanation head, degrades explanation quality despite no change in accuracy.

Limitations

• Evaluated only on ViT and BERT with a small set of image and sentiment datasets.
• Explanation quality is measured via insertion/deletion metrics using the surrogate, not human evaluation.
• Relies on causal attention masking for masked inputs, which may differ from other masking schemes.

Experimental setup

Datasets: ImageNette MURA Oxford-IIIT Pets Yelp Review Polarity

Benchmarks: ImageNette MURA Oxford-IIIT Pets Yelp Review Polarity

Metrics: insertion AUC deletion AUC accuracy trainable parameters training memory FLOPs inference time

Baselines: ViT Shapley (Covert et al.) KernelSHAP Attention rollout Attention last GradCAM Vanilla Gradients IntGrad SmoothGrad VarGrad LRP Leave-one-out RISE

Builds on: ViT Shapley Ladder Side-Tuning FastSHAP Shapley Kernel

Closest papers by meaning

• Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable? (ICLR 2025 · similarity 0.86)
• Beyond Topological Self-Explainable GNNs: A Formal Explainability Perspective (ICML 2025 · similarity 0.85)
• Q-SENN: Quantized Self-Explaining Neural Networks (AAAI 2024 · similarity 0.85)
• Self-Interpretable Graph Learning with Sufficient and Necessary Explanations (AAAI 2024 · similarity 0.85)
• GraphTrail: Translating GNN Predictions into Human-Interpretable Logical Rules (NeurIPS 2024 · similarity 0.85)
• Towards Modeling Uncertainties of Self-Explaining Neural Networks via Conformal Prediction (AAAI 2024 · similarity 0.85)