Research map / Adversarial robustness and security / Certified and empirical robustness

The Effect of Intrinsic Dataset Properties on Generalization: Unraveling Learning Differences Between Natural and Medical Images

ICLR 2024 · Adversarial robustness and security › Certified and empirical robustness · approach: Robust overfitting and regularization · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes label sharpness (a Lipschitz-based intrinsic property of a dataset) and derives generalization scaling laws in dataset and representation intrinsic dimension, showing medical image datasets have higher label sharpness, correlating with lower adversarial robustness.

Problem

Generalization scaling with intrinsic dimension differs drastically between natural and medical (radiology) image domains, with no theoretical explanation; also adversarial robustness differences between domains are unexplained.

Approach

Introduces label sharpness K_F, the empirical Lipschitz constant of the ground-truth labeling function on the data manifold, and derives a generalization scaling law L ~ K_L K_F N^{-1/d_data} that explains the domain gap; also links robustness radius to K_F and proves d_repr upper bounded by d_data.

Key results

• Medical datasets have label sharpness around 2.5e-4 versus about 1e-4 for natural image datasets, and models trained on medical data show a steeper generalization-loss scaling with d_data.
• Label sharpness is negatively correlated with FGSM robustness; e.g., Pearson r = 0.70 for ResNet-18 at epsilon 2/255 and 0.64 for VGG-16.
• Log likelihood ratio favoring the label-sharpness-aware scaling law is positive for all six architectures (e.g., 13.5 for ResNet-18).

Measured effects reported in the paper

Surprising findings

• Some ISIC-based models end up close to natural image models in robustness, not uniformly in between the domains.

Limitations

• Difficult to establish that label sharpness causally explains the natural vs medical generalization gap.
• Theoretical results rely on Lipschitz/smoothness assumptions and asymptotic arguments; bounds are approximate.
• Analysis limited to binary classification with CNNs (ResNet and VGG) on radiology and natural image datasets.

Experimental setup

Datasets: BraTS DBC Prostate MRI RSNA-IH-CT CheXpert MURA OAI ImageNet CIFAR10 SVHN MNIST ISIC

Benchmarks: BraTS DBC Prostate MRI RSNA-IH-CT CheXpert MURA OAI ImageNet CIFAR10 SVHN MNIST ISIC

Metrics: test loss test accuracy Pearson correlation log-likelihood ratio FGSM loss penalty

Builds on: manifold hypothesis MLE intrinsic dimension estimator TwoNN FGSM

Closest papers by meaning

• What Variables Affect Out-of-Distribution Generalization in Pretrained Models? (NeurIPS 2024 · similarity 0.86)
• Can Biases in ImageNet Models Explain Generalization? (CVPR 2024 · similarity 0.86)
• TinyMIG: Transferring Generalization from Vision Foundation Models to Single-Domain Medical Imaging (ICML 2025 · similarity 0.85)
• Benchmarking Out-of-Distribution Generalization Capabilities of DNN-based Encoding Models for the Ventral Visual Cortex. (NeurIPS 2024 · similarity 0.85)
• Explaining the role of Intrinsic Dimensionality in Adversarial Training (ICML 2025 · similarity 0.85)
• Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Data (NeurIPS 2024 · similarity 0.85)