Research map / Label-efficient and robust learning / Self-supervised and contrastive

Learning Low-Rank Feature for Thorax Disease Classification

NeurIPS 2024 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Self-supervised neuroimaging representations · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Low-Rank Feature Learning (LRFL), which adds a separable approximation of the truncated nuclear norm to fine-tuning of MAE-pretrained CNNs/ViTs for thorax disease classification, reporting new mAUC records on NIH ChestX-ray14 and CheXpert and accuracy on COVIDx.

Problem

Thorax disease classification on chest radiographs is hampered by noise and non-disease background in features; existing SSL-pretrained CNN/ViT pipelines lack principled ways to suppress these high-frequency, non-informative components.

Approach

Add a truncated nuclear norm penalty on the feature matrix (approximated by a separable form using eigenvectors refreshed every t0 epochs) to the classifier/fine-tuning loss so that features are low-rank, backed by a generalization bound showing the benefit of a small truncated nuclear norm.

Key results

• ViT-B-LR reaches 83.4 mAUC on NIH ChestX-ray14 versus 83.0 for MAE ViT-B, a new reported record among the compared methods.
• ViT-B-LR achieves 97.0% top-1 accuracy on COVIDx (3-class test), up from 95.3% for ViT-B.
• ViT-B-LR reaches 89.8 mAUC on CheXpert, improving ViT-B by 0.5 points.

Measured effects reported in the paper

Surprising findings

• Fine-tuning the same model for 20 epochs without low-rank regularization does not improve mAUC (82.27 to 82.26 for ViT-S), suggesting the gain comes from the regularizer rather than extra training.
• Mix-up, label smoothing and EMA reduce performance on NIH ChestX-ray14 relative to the base model in the reported comparison.

Limitations

• Gains over base models are small (about 0.4-0.5 mAUC points on most benchmarks) and no variance or significance testing is reported.
• Rank ratio and eta are tuned by cross-validation per dataset, adding hyperparameter search cost.
• Theoretical bound is derived for linear networks/gradient descent, and its extension to deep fine-tuning is only heuristically motivated.

Experimental setup

Datasets: ImageNet-1k X-rays (0.3M/0.5M) NIH ChestX-ray14 COVIDx CheXpert

Benchmarks: NIH ChestX-ray14 COVIDx CheXpert

Metrics: mAUC top-1 accuracy Covid-19 sensitivity

Baselines: MAE ViT-B/16 MAE ViT-S/16 MoCo v2 DenseNet-121 COVIDNet-CXR Mix-up Label Smoothing EMA

Builds on: masked autoencoders Vision Transformer truncated nuclear norm

Closest papers by meaning

• Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification (AAAI 2021 · similarity 0.88)
• G2D: From Global to Dense Radiography Representation Learning via Vision-Language Pre-training (NeurIPS 2024 · similarity 0.84)
• Large-scale and Fine-grained Vision-language Pre-training for Enhanced CT Image Understanding (ICLR 2025 · similarity 0.84)
• Novel Class Discovery in Chest X-rays via Paired Images and Text (AAAI 2024 · similarity 0.83)
• NaFV-Net: An Adversarial Four-view Network for Mammogram Classification (AAAI 2025 · similarity 0.83)
• Bootstrapping Chest CT Image Understanding by Distilling Knowledge from X-ray Expert Models (CVPR 2024 · similarity 0.82)