Research map / Probabilistic and scientific modeling / Molecules and proteins
Navigating Neural Space: Revisiting Concept Activation Vectors to Overcome Directional Divergence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows filter-based CAVs (SVM/linear classifier weights) diverge from true concept direction due to distractor signals, and proposes pattern-based CAVs (covariance/difference-of-means style estimator) that align with the true concept direction and improve TCAV and ClArC model correction.
Problem
Concept Activation Vectors are usually computed as weight vectors of linear classifiers optimized for class separability, which can diverge from the true concept signal direction when distractor directions are present.
Approach
Replace separability-oriented filter CAVs with pattern-based CAVs obtained from a linear regression of activations on concept labels (Haufe-style patterns), which equal the difference of cluster means for binary concepts and are invariant to feature scaling and robust to noise.
Key results
- Pattern-CAVs achieve significantly higher cosine alignment with the true concept direction than filter-CAVs across all VGG16 conv layers on ISIC2019, Bone Age and FunnyBirds, while filter-CAVs show higher separability.
- In RR-ClArC model correction on VGG16 (Bone Age), pattern-CAV raises biased-test accuracy to 0.75 from 0.50 for vanilla and reduces artifact relevance to 0.26 vs 0.62 for vanilla.
- For FunnyBirds TCAV, pattern-CAVs achieve a perfect delta-TCAV of 0.5 for VGG16 and EfficientNet-B0, while filter-CAVs do not fully reflect the model's concept dependence.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Pattern-CAV instead of filter-based CAVs for RRClArC correction | VGG16 · ISIC2019 controlled timestamp | accuracy (biased): 0.28 -> 0.69 | Helped |
| Use Pattern-CAV instead of filter-based CAVs for RRClArC correction | VGG16 · ISIC2019 controlled timestamp | artifact relevance: 0.55 -> 0.26 | Helped |
| Use Pattern-CAV instead of Logistic-CAV for RRClArC correction | VGG16 · ISIC2019 controlled timestamp | ΔTCAV ground truth: 0.07 -> 0.10 | Hurt |
| Use Pattern-CAV instead of filter-based CAVs for RRClArC correction | VGG16 · ISIC2019 real band-aid | accuracy (biased): 0.75 -> 0.79 | Helped |
| Use Pattern-CAV instead of SVM-CAV for RRClArC correction | VGG16 · ISIC2019 real band-aid | artifact relevance: 0.46 -> 0.31 | Helped |
| Use Pattern-CAV instead of Logistic-CAV for RRClArC correction | VGG16 · ISIC2019 real band-aid | ΔTCAV ground truth: 0.09 -> 0.03 | Helped |
| Use Pattern-CAV instead of filter-based CAVs for RRClArC correction | VGG16 · Pediatric Bone Age | accuracy (biased): 0.50 -> 0.75 | Helped |
| Use Pattern-CAV instead of SVM-CAV for RRClArC correction | VGG16 · Pediatric Bone Age | ΔTCAV ground truth: 0.24 -> 0.14 | Helped |
| Use pattern-based rather than filter-based CAVs | CAV concept representation · ISIC2019, Pediatric Bone Age, and FunnyBirds controlled datasets | cosine similarity with true concept direction | Helped |
| Use filter-based rather than pattern-based CAVs | CAV concept representation · ISIC2019, Pediatric Bone Age, and FunnyBirds controlled datasets | concept separability | Helped |
| Use Pattern-CAV rather than filter-CAV under feature scaling or distractor rotation | CAV concept representation · 2D toy experiment | alignment with true concept direction | Helped |
| Use Pattern-CAV rather than filter-CAV | TCAV · 2D toy experiment | TCAV sensitivity: 0.5 | Helped |
Surprising findings
- Filter-based CAVs from unpenalized logistic regression scale anti-proportionally with feature scaling, so they misalign with the concept direction even under mild feature rescaling.
- All CAV variants achieve a perfect TCAV score for ResNet18, which the authors attribute to poorly localized concepts.
Limitations
- Filter-based CAVs remain better for class-separability-driven applications such as concept bottleneck models.
- Pattern-CAVs are restricted to binary concept labels in the current formulation.
- Evaluation is limited to controlled artificial concepts and a few real artifacts in ISIC2019.
Experimental setup
Datasets: ISIC2019 Pediatric Bone Age FunnyBirds ImageNet
Benchmarks: ISIC2019 Pediatric Bone Age FunnyBirds
Metrics: cosine similarity to true concept direction AUC TCAV score accuracy on clean and biased test sets artifact relevance
Baselines: SVM CAV lasso CAV logistic CAV ridge CAV Vanilla model
Builds on: TCAV ClArC RR-ClArC Haufe et al. patterns
Closest papers by meaning
- FastCAV: Efficient Computation of Concept Activation Vectors for Explaining Deep Neural Networks (ICML 2025 · similarity 0.89)
- LG-CAV: Train Any Concept Activation Vector with Language Guidance (NeurIPS 2024 · similarity 0.85)
- From Causal to Concept-Based Representation Learning (NeurIPS 2024 · similarity 0.85)
- The SuperActivator Mechanism: Transformers Concentrate Reliable Concept Signals in the Tail (NeurIPS 2026 · similarity 0.85)
- Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometry (NeurIPS 2025 · similarity 0.84)
- Learning Discrete Concepts in Latent Hierarchical Models (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.