Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes

Neuron Explanations for Conformal Prediction (Student Abstract)

AAAI 2025 · Probabilistic and scientific modeling › Uncertainty and Gaussian processes · approach: Conformal prediction methods · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a neuron-level explanation method for conformal prediction that finds neurons driving uncertainty sets and uses LLMs and CLIP to auto-generate their semantic descriptions.

Problem

Existing conformal prediction methods give guaranteed uncertainty sets but do not explain where predictive uncertainty comes from; neuron-level explanations for CP are unexplored and hard because CP is discrete and non-differentiable and semantic annotation is labor-intensive.

Approach

Identify the minimal set of neurons in a layer whose masking removes a target label from the conformal set via an empirical search, then label those neurons automatically by pairing LLM-generated class concepts with CLIP-aligned activated image patches.

Key results

• Masking top-ranked neurons in ResNet-50 layer 4 causes accuracy drops up to roughly 100% on some classes, with the ranking shown across the 200 sorted units.
• Generated neuron descriptions are reported as highly aligned with human annotations on qualitative examples (e.g., unit 1762 linked to 'school bus' text on the side of the vehicle).
• The method identifies neurons influencing CP prediction sets for HPS and APS without human annotation.

Measured effects reported in the paper

Limitations

• Optimization is an empirical search rather than an exact solution to the discrete, non-differentiable problem.
• Evaluation is limited to qualitative examples and an accuracy-drop plot; the full quantitative details are deferred to an unavailable full paper.
• Only convolutional layers of ResNet-50 and AlexNet on two image datasets are studied.

Experimental setup

Datasets: ImageNet Places365

Benchmarks: ImageNet Places365

Metrics: accuracy drop (%) after neuron masking qualitative agreement with human annotations

Baselines: human annotations HPS APS

Builds on: conformal prediction Network Dissection

Closest papers by meaning

• Towards Modeling Uncertainties of Self-Explaining Neural Networks via Conformal Prediction (AAAI 2024 · similarity 0.90)
• Evaluating Neuron Explanations: A Unified Framework with Sanity Checks (ICML 2025 · similarity 0.86)
• Explaining, Fast and Slow: Abstraction and Refinement of Provable Explanations (ICML 2025 · similarity 0.86)
• Approximating Full Conformal Prediction for Neural Network Regression with Gauss-Newton Influence (ICLR 2025 · similarity 0.85)
• Verifiably Robust Conformal Prediction (NeurIPS 2024 · similarity 0.85)
• Reliable Uncertainty Quantification in Machine Learning via Conformal Prediction (AAAI 2025 · similarity 0.84)