Research map / Architectures and efficiency / Attention and transformers
Decomposing and Editing Predictions by Modeling Model Computation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces component modeling and component attribution, and COAR, a regression-based method that estimates linear attributions of model components from random ablations, then uses them for ablation-based model editing across five tasks.
Problem
Understanding how collective ablations of internal model components (e.g., conv filters, attention rows) change individual predictions, in a way that is interpretable and scalable to large models.
Approach
Cast component counterfactual prediction as supervised linear regression: sample random fractional ablations of components, measure the correct-class margin for each, and fit a per-example linear model whose coefficients are component attributions; use these attributions with a t-test-based selection to pick ablations for model editing.
Key results
- On ImageNet ResNet-50 at 5% ablation, COAR attributions reach 0.65 average correlation with ground-truth counterfactuals versus 0.34 for the best baseline (LOO).
- Ablating eight of 22,720 components drops ImageNet 'chain-link fence' class accuracy from 66% to 20% while preserving overall accuracy.
- Ablating 26 components raises CelebA worst-subpopulation accuracy from 47% to 85%; ablating 300 components lifts CLIP accuracy under synthetic typographic attacks from 51% to 89%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Coar instead of the LOO baseline for component attribution | ImageNet ResNet-50 · 5% component ablation | Pearson correlation: 0.34 -> 0.65 | Helped |
| Increase test ablation fraction from 10% to 12.5% | Coar · CIFAR-10 ResNet-18 | Pearson correlation: 0.74 -> 0.70 | Hurt |
| Increase test ablation fraction from 10% to 15% | Coar · CIFAR-10 ResNet-18 | Pearson correlation: 0.74 -> 0.68 | Hurt |
| Use attribution fit with α_train=0.05 rather than α_train=0.10 (ablation) | Coar · ImageNet ResNet-50, small test ablation fractions | Pearson correlation | Helped |
| Use attribution fit with α_train=0.10 rather than α_train=0.05 (ablation) | Coar · ImageNet ResNet-50, large test ablation fractions | Pearson correlation | Helped |
| Reduce component-counterfactual samples by 5× (ablation) | Coar on CIFAR-10 ResNet-18 · CIFAR-10 | Pearson correlation: 0.7 -> 0.65 | Hurt |
| Ablate 210 components selected by Coar-Edit | Waterbirds model · Waterbirds worst-subpopulation accuracy | worst-subpopulation accuracy: 64% -> 83% | Helped |
| Ablate 26 components selected by Coar-Edit | CelebA model · CelebA worst-subpopulation accuracy | worst-subpopulation accuracy: 47% -> 85% | Helped |
| Ablate 26 components selected by Coar-Edit | CelebA model · CelebA average-subpopulation accuracy | average-subpopulation accuracy: 84% -> 90% | Helped |
| Ablate 8 components selected by Coar-Edit | ImageNet ResNet-50 · Chain-link fence class | class accuracy: 66% -> 20% | Hurt |
| Ablate 25 components selected by Coar-Edit | Backdoor-trained CIFAR-10 ResNet-18 · Test examples with blue-square trigger | accuracy: 37% -> 84% | Helped |
| Ablate 300 components selected by Coar-Edit | Zero-shot CLIP ViT-B/16 · Held-out images with synthetic typographic attacks | accuracy: 51% -> 89% | Helped |
Surprising findings
- Simply zero-ablating a few components from standard ERM-trained models achieves worst-subpopulation gains comparable to specialized robust training methods.
- Despite strong non-linearity of deep networks, linear component attributions predict component counterfactuals with substantial correlation.
Limitations
- Requires many forward passes to construct component datasets, a computational bottleneck.
- Linear attributions are less predictive for out-of-distribution ablation fractions, suggesting linear form is limited.
- Ablation-only editing is coarse-grained and likely not the most effective editing method.
Experimental setup
Datasets: CIFAR-10 ImageNet Waterbirds CelebA TinyStories BoolQ ImageNet-Sketch ImageNet* MIMIC-CXR typographic attacks dataset
Benchmarks: CIFAR-10 ImageNet Waterbirds CelebA TinyStories BoolQ ImageNet-Sketch ImageNet* typographic attacks dataset
Metrics: Pearson correlation between ground-truth component counterfa worst-subpopulation accuracy class accuracy backdoor trigger accuracy typographic attack accuracy
Baselines: leave-one-out gradient-times-parameter neuron conductance internal influence
Builds on: integrated gradients datamodeling activation patching attribution patching
Closest papers by meaning
- ModHiFi: Identifying High Fidelity predictive components for Model Modification (NeurIPS 2025 · similarity 0.84)
- DisCEdit: Model Editing by Identifying Discriminative Components (NeurIPS 2024 · similarity 0.84)
- Unveiling Concept Attribution in Diffusion Models (NeurIPS 2025 · similarity 0.83)
- DePass: Unified Feature Attributing by Simple Decomposed Forward Pass (NeurIPS 2025 · similarity 0.83)
- Factor Decorrelation Enhanced Data Removal from Deep Predictive Models (NeurIPS 2025 · similarity 0.82)
- Provably Better Explanations with Optimized Aggregation of Feature Attributions (ICML 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.