Research map / Learning theory and algorithms / Prediction and decision losses
DisCEdit: Model Editing by Identifying Discriminative Components
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DisCEdit, which uses witness-function lower bounds on total variation distance to find discriminative filters for structured pruning (DisCEdit-SP) and class unlearning (DisCEdit-U) without training data or loss function.
Problem
Identifying which filters or components in trained CNNs are important for predictions, for structured pruning and selective class forgetting, without access to training data or loss function and without Gaussianity assumptions on feature distributions.
Approach
Define filter discriminative ability via Bayes error (equivalently TV distance between class-conditional feature distributions), and derive distribution-free witness-function lower bounds on TV from finitely many moments, yielding saliency scores that prune non-discriminative filters (SP) or discriminative filters for one class (U).
Key results
- On ImageNet ResNet50, DisCEdit-SP reaches 40.8% sparsity with a 2.58% accuracy drop after fine-tuning, versus 2.74% for TVSPrune and 2.76% for CHIP.
- DisCEdit-U reduces forget-class accuracy by about 83% on average (VGG16 CIFAR10 FA 8.7% after pruning 5.4% of weights) with small remain-set drops, without fine-tuning.
- On CIFAR10 VGG16 with fine-tuning, DisCEdit-SP reaches 61.2% sparsity with a -0.37% change versus TVSPrune's -0.98%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DISCEDIT-SP rather than TVSPrune | VGG16 · CIFAR-10, 61.2% sparsity | accuracy change: -0.98% -> -0.37% | Helped |
| use DISCEDIT-SP rather than TVSPrune | VGG19 · CIFAR-10, 72.4% sparsity | accuracy change: -0.16% -> -0.12% | Helped |
| use DISCEDIT-SP rather than TVSPrune | ResNet50 · ImageNet, 40.8% sparsity | accuracy change: -2.74% -> -2.58% | Helped |
| prune 40.8% of parameters with DISCEDIT-SP | ResNet50 · ImageNet | accuracy drop: 2.58% | Hurt |
| prune the most class-discriminative filters | DISCEDIT-U · CIFAR-10 and CIFAR-100 | forget-class accuracy drop: 83% | Helped |
| prune the most class-discriminative filters | DISCEDIT-U · CIFAR-10 and CIFAR-100 | remaining-class accuracy change: 1.2% | Mixed |
| fine-tune for 1 epoch and increase the pruning ratio (ablation) | DISCEDIT-U on VGG16 · CIFAR-10 | forget accuracy drop: 8.7% -> 3.7% | Hurt |
| fine-tune for 1 epoch and increase the pruning ratio (ablation) | DISCEDIT-U on ResNet56 · CIFAR-10 | forget accuracy drop: 16.3% -> 9.7% | Hurt |
| fine-tune for 1 epoch and increase the pruning ratio (ablation) | DISCEDIT-U on ResNet20 · CIFAR-10 | forget accuracy drop: 9.4% -> 6.0% | Hurt |
| fine-tune for 1 epoch and increase the pruning ratio (ablation) | DISCEDIT-U on ViT · CIFAR-10 | forget accuracy drop: 16.5% -> 11.0% | Hurt |
| use quadratic witness features instead of TVSPrune's approach (ablation) | DISCEDIT-SP on VGG16 · CIFAR-10, 70% sparsity in layers 1-3 | accuracy: 6.6% | Helped |
Surprising findings
- Fine-tuning on the remain set raises forget-class accuracy unless many more filters are pruned.
- Class-selective components generally cannot be found when the number of classes is large; the method fails for ResNet56 on CIFAR100.
Limitations
- Lower bounds are not tight in general; tractable exact TV only for Gaussian case.
- Class unlearning is ineffective when the number of classes exceeds network width (ResNet56 on CIFAR100).
- Fine-tuning on the remain set can restore forget-class accuracy unless many more filters are pruned.
Experimental setup
Datasets: CIFAR10 CIFAR100 ImageNet
Benchmarks: CIFAR10 CIFAR100 ImageNet
Metrics: accuracy drop sparsity forget accuracy remain accuracy
Baselines: TVSPrune CHIP L1-norm pruning GA IU
Builds on: discriminative filters hypothesis (TVSPrune) Fisher discriminant minimax probability machine Selective forgetting via pruning (Shah et al.)
Closest papers by meaning
- ModHiFi: Identifying High Fidelity predictive components for Model Modification (NeurIPS 2025 · similarity 0.86)
- Learning Where to Edit Vision Transformers (NeurIPS 2024 · similarity 0.85)
- Discriminative Score Function: Turning Pretrained Models into Functional Generative Priors (NeurIPS 2026 · similarity 0.85)
- Adversarial Representation Engineering: A General Model Editing Framework for Large Language Models (NeurIPS 2024 · similarity 0.84)
- Attribution Analysis Meets Model Editing: Advancing Knowledge Correction in Vision Language Models with VisEdit (AAAI 2025 · similarity 0.84)
- ImgEdit: A Unified Image Editing Dataset and Benchmark (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.