Research map / Probabilistic and scientific modeling / Molecules and proteins
SOInter: A Novel Deep Energy-Based Interpretation Method for Explaining Structured Output Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SOInter, an energy-based, globally trained local interpreter that selects the k most important input features for one target output of a black-box structured output model, accounting for correlations among output variables.
Problem
Explaining structured output models: a feature may affect a target output indirectly through other correlated outputs, so per-output instance-wise feature attribution ignores important structure in the black-box.
Approach
Train an interpreter network (MLP + Gumbel-Softmax top-k selection) whose loss compares energy values of the black-box's structure, using an energy network pre-trained on black-box input-output pairs and fine-tuned jointly with a structured hinge loss, so that the target output is preserved while other outputs are free to change.
Key results
- On multi-label text datasets, SOInter achieves post-hoc F1 0.60 on Bibtex versus 0.54 for Lime and 0.13 for L2X, with relative post-hoc F1 0.68.
- On Enron, SOInter reaches relative post-hoc F1 0.57 versus 0.35 for Lime and 0.06 for L2X.
- SOInter has near-constant explaining time across input sizes and is faster than Lime and Kernel-Shap in the synthetic experiments.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add structural information from other output variables during interpreter training (ablation) | SOInter · Bibtex | post-hoc F1: 0.56 -> 0.60 | Helped |
| Add structural information from other output variables during interpreter training (ablation) | SOInter · Bibtex | relative post-hoc F1: 0.64 -> 0.68 | Helped |
| Add structural information from other output variables during interpreter training (ablation) | SOInter · Enron | post-hoc F1: 0.21 -> 0.24 | Helped |
| Add structural information from other output variables during interpreter training (ablation) | SOInter · Enron | relative post-hoc F1: 0.51 -> 0.57 | Helped |
| Add structural information from other output variables during interpreter training (ablation) | SOInter · IMDB | post-hoc F1: 0.05 -> 0.05 | No clear effect |
| Add structural information from other output variables during interpreter training (ablation) | SOInter · IMDB | relative post-hoc F1: 0.31 -> 0.24 | Hurt |
| Increase the number of selected features K to 100 | SOInter · Weizmann-horses | relative post-hoc F1 | Helped |
| Increase K beyond 100 | SOInter · Weizmann-horses | relative post-hoc F1 | Helped |
| Vary the Gumbel-Softmax temperature τ | SOInter · Synthetic datasets | interpretation performance | Mixed |
Surprising findings
- On IMDB, SOInter-1 (without structural information) achieves higher relative post-hoc F1 (0.31) than the full SOInter (0.24), and Lime's post-hoc F1 is tied at 0.05.
Limitations
- Zeroing unselected features may confuse the black-box when zero has a special meaning.
- Two neural networks increase parameter count and deployment memory compared with baselines.
- Selects features independently rather than higher-level conceptual elements.
Experimental setup
Datasets: Synthetic energy-function data (E1, E2) Bibtex Enron IMDB Weizmann-horses
Benchmarks: Synthetic energy-function datasets Bibtex Enron IMDB Weizmann-horses
Metrics: accuracy of selected features median rank plain F1 post-hoc F1 relative post-hoc F1
Baselines: Lime Kernel-Shap L2X SOInter-1
Builds on: Structured prediction energy networks Gumbel-Softmax L2X Lime Kernel-Shap
Closest papers by meaning
- Deep Structured Prediction with Nonlinear Output Transformations (NeurIPS 2018 · similarity 0.86)
- Adversarial Localized Energy Network for Structured Prediction (AAAI 2020 · similarity 0.85)
- Structured Energy Network As a Loss (NeurIPS 2022 · similarity 0.85)
- Faithful Model Explanations through Energy-Constrained Conformal Counterfactuals (AAAI 2024 · similarity 0.83)
- Explainable Neural Networks with Guarantee: A Sparse Estimation Approach (AAAI 2025 · similarity 0.83)
- CoE: Chain-of-Explanation via Automatic Visual Concept Circuit Description and Polysemanticity Quantification (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.