Research map / Architectures and efficiency / RNNs and normalization
Tilt and Average : Geometric Adjustment of the Last Layer for Recalibration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Tilt and Average (TNA), which recalibrates a trained classifier by randomly rotating the last-layer class vectors to a target mean angle (mRC) and averaging rotations, combinable with calibration-map methods.
Problem
Neural network classifiers are overconfident; post-hoc recalibration usually fits a calibration map on logits, while the geometry of the final linear layer is left unexploited.
Approach
Instead of fitting a calibration map, rotate the class weight vectors of the last linear layer by a controlled angle (Givens-rotation compositions, with mean rotation over classes mRC as the intensity), then average multiple tilted weights to recover accuracy; this relaxes confidence and is orthogonal to calibration-map methods.
Key results
- On CIFAR100-WideResNet28x10, None+TNA reduces ECE from 5.91% to 4.11% with accuracy change under 0.3%.
- On ImageNet-ViT-H/14, TNA(comp.) with ETS reduces ECE from 0.63% to 0.60%, and on ImageNet-ResNet50 TS+TNA(comp.) reduces ECE from 2.09% to 1.93%.
- TNA combined with calibration-map methods improves ECE over the calibration map alone in most dataset-model settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply TNA to the final-layer weights | Uncalibrated classifier · CIFAR10, WideResNet28x10 | ECE (%): 1.66_{.1} -> 0.66_{.14} | Helped |
| Apply TNA to the final-layer weights | Uncalibrated classifier · CIFAR100, MobileNetV2 | ECE (%): 10.03_{.19} -> 1.54_{.09} | Helped |
| Apply TNA to the final-layer weights | Uncalibrated classifier · ImageNet-1k, ResNet50 | ECE (%): 3.83_{.09} -> 1.73_{.06} | Helped |
| Add TNA to Temperature Scaling | Temperature Scaling · CIFAR10, WideResNet28x10 | ECE (%): 0.84_{.05} -> 0.78_{.13} | Helped |
| Add TNA to Temperature Scaling | Temperature Scaling · CIFAR100, MobileNetV2 | ECE (%): 1.82_{.33} -> 1.41_{.15} | Helped |
| Add TNA to Temperature Scaling | Temperature Scaling · ImageNet-1k, ResNet50 | ECE (%): 2.09_{.3} -> 2.02_{.27} | Helped |
| Reduce the penultimate-feature dimension from 640 to 10 (ablation) | TNA with WideResNet28x10 · CIFAR100 | ECE (%): 4.11 -> 11.04 | Hurt |
| Reduce the penultimate-feature dimension from 640 to 10 (ablation) | TNA with WideResNet28x10 · CIFAR10 | ECE (%): 0.66 -> 1.48 | Hurt |
| Average multiple tilted weights (increase ensemble members) (ablation) | TNA · ImageNet-1k, ResNet50 | accuracy | Helped |
| Use lower penultimate-feature dimension (ablation) | TNA · CIFAR10, WideResNet28x10 | accuracy | Hurt |
Surprising findings
- Applying TNA alone (no calibration map) outperforms temperature scaling on several settings, e.g., CIFAR10-WideResNet28x10 ECE 0.66% vs TS 0.84%.
Limitations
- Accuracy degrades as the rotation angle grows, requiring averaging to compensate; the theory relies on assumptions like equal class-wise angle changes and high dimensionality.
- Effectiveness diminishes when the penultimate feature dimension is small, as shown by ablation.
- Theoretical results rely on assumptions that are only partially validated experimentally.
Experimental setup
Datasets: CIFAR10 CIFAR100 SVHN ImageNet-1k
Benchmarks: CIFAR10 CIFAR100 SVHN ImageNet-1k
Metrics: accuracy ECE AdaECE
Baselines: Temperature Scaling IROvA Ensemble Temperature Scaling Atypicality-Aware Recalibration
Builds on: temperature scaling Givens rotation IROvA Ensemble Temperature Scaling Atypicality-Aware Recalibration
Closest papers by meaning
- Balancing Two Classifiers via A Simplex ETF Structure for Model Calibration (CVPR 2025 · similarity 0.85)
- Calibration Bottleneck: Over-compressed Representations are Less Calibratable (ICML 2024 · similarity 0.85)
- Improving Multi-Class Calibration through Normalization-Aware Isotonic Techniques (ICML 2025 · similarity 0.85)
- Confidence Calibration of Classifiers with Many Classes (NeurIPS 2024 · similarity 0.84)
- Optimal and Provable Calibration in High-Dimensional Binary Classification: Angular Calibration and Platt Scaling (NeurIPS 2025 · similarity 0.84)
- Reassessing How to Compare and Improve the Calibration of Machine Learning Models (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.