Research map / Adversarial robustness and security / Certified and empirical robustness

Attack-inspired Calibration Loss for Calibrating Crack Recognition

AAAI 2025 · Adversarial robustness and security › Certified and empirical robustness · approach: Adversarial training with feature consistency · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AICL, a plug-and-play calibration loss that uses adversarial feature attack rounds to estimate sample correctness and enforces distribution-level confidence ordering, and releases the BRIDGE2024 crack dataset.

Problem

DNNs for crack recognition in safety-critical structural health monitoring are poorly calibrated, showing both under-confidence and over-confidence, and existing calibration methods do not fit this setting.

Approach

Estimate each sample's correctness degree by the number of PGD steps on its features needed to flip the initial prediction (ACE), then enforce, at the distribution level, that the expected confidence of higher-correctness groups exceeds that of lower-correctness groups (Correctness-aware Distribution Guidance), added to cross-entropy as a plug-and-play loss.

Key results

• On BRIDGE2024 with ResNet-50, AICL reduces ECE to 2.10 from 5.98-9.56 for crack classifiers, with ACC 97.77 and F1 95.88.
• Distribution-level guidance improves F1 from 96.54 to 99.84 and ECE from 4.19 to 0.20 on CRACK2019 versus sample-level guidance.
• Combining AICL with temperature scaling reduces BRIDGE2024 ECE from 2.10 to 1.90.

Measured effects reported in the paper

Surprising findings

• Crack recognition DNNs are both over-confident and under-confident, unlike general classification where over-confidence dominates.
• Sample-level ordering of confidence by estimated correctness performs worse than the distribution-level version, because per-sample correctness estimates are unreliable.

Limitations

• Calibration gains on general classification are reported only briefly; the method is evaluated mainly on binary crack datasets.
• Sensitivity to hyperparameters K and lambda requires tuning; burn-in of 10 epochs needed.
• Adversarial attack-based correctness estimate may be noisy when classifier is poorly trained.

Experimental setup

Datasets: CRACK2019 SDNET2018 BRIDGE2024

Benchmarks: CRACK2019 SDNET2018 BRIDGE2024

Metrics: AUC ACC SEN SPEC F1 ECE

Baselines: SDNET SilvaNet AliNet DCrack IVGG FLSD AR-AdaLS MbLS DualFocalLoss ACEnt DECE Temperature scaling

Builds on: PGD Label smoothing calibration (MbLS) DECE AR-AdaLS ACE (entropy calibration)

Closest papers by meaning

• Towards Certification of Uncertainty Calibration under Adversarial Attacks (ICLR 2025 · similarity 0.84)
• Uncertainty Weighted Gradients for Model Calibration (CVPR 2025 · similarity 0.82)
• Detecting Brittle Decisions for Free: Leveraging Margin Consistency in Deep Robust Classifiers (NeurIPS 2024 · similarity 0.82)
• Exploring the Relationship Between Samples and Masks for Robust Defect Localization (AAAI 2025 · similarity 0.82)
• A Benchmark Study on Calibration (ICLR 2024 · similarity 0.82)
• Learning Better Certified Models from Empirically-Robust Teachers (NeurIPS 2026 · similarity 0.81)