Research map / Label-efficient and robust learning / Domain adaptation and generalization
Merge-Friendly Post-Training Quantization for Multi-Target Domain Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HDRQ, a post-training quantization method that uses noise-based Hessian regularization, weight-distance regularization, and noise-sampling rounding so quantized domain-adapted models can be merged for multi-target domain adaptation.
Problem
Quantizing target-adapted models degrades their weight-space merging for multi-target domain adaptation, because quantization noise and discretization create error barriers between models.
Approach
Analyzes quantization's effect on merging via error barriers, then combines noise-injection training (flattening the Hessian), a penalty on distance between domain-adapted weights and the source weight, and noise-sampled merging with cosine-similarity selection to produce merge-friendly PTQ.
Key results
- On semantic segmentation W4A4, HDRQ's merged model reaches 45.75 harmonic-mean mIoU versus 41.54 for QDrop (+4.21).
- On Office-Home R→A,C,P at W3A3, HDRQ merged accuracy is 64.70 vs 62.99 for QDrop.
- HDRQ's single-model quantization accuracy is comparable to QDrop on segmentation (W6A6 mIoU 61.54 vs 61.32).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HDRQ instead of QDrop quantization before merging | GTA→Cityscapes + Indian Driving, W6A6 · GTA→Cityscapes + Indian Driving, W6A6 merged | mIoU harmonic mean: +1.19 mIoU | Helped |
| Use HDRQ instead of QDrop quantization before merging | GTA→Cityscapes + Indian Driving, W4A4 · GTA→Cityscapes + Indian Driving, W4A4 merged | mIoU harmonic mean: +4.21 mIoU | Helped |
| Use HDRQ instead of QDrop quantization before merging | Office-Home · Office-Home, R→A,C,P, W3A3 merged | harmonic mean accuracy: 62.99 -> 64.70 | Helped |
| Use HDRQ instead of QDrop quantization before merging | Office-Home · Office-Home, P→R,A,C, W3A3 merged | harmonic mean accuracy: 61.24 -> 61.55 | Helped |
| Add noise-based quantization to the QDrop baseline (ablation) | HDRQ · Office-Home, R→A,C,P, W3A3 merged | accuracy: 62.99 -> 64.21; 1.22% | Helped |
| Add distance regularization to noise-based quantization (ablation) | HDRQ · Office-Home, R→A,C,P, W3A3 merged | accuracy: 64.21 -> 64.70; 0.49% | Helped |
Surprising findings
- Standard PTQ methods such as BRECQ fail to merge at low bit-widths, with W4A4 segmentation merged harmonic mIoU of 31.95 for BRECQ.
Limitations
- Merging is restricted to midpoint averaging with batch norm folded, and the theoretical analysis relies on assumptions such as zero loss barrier and Lipschitz Hessians.
- Noise sampling for rounding requires selecting among 30 samples by cosine similarity, and the selection relies on access to target-domain weights.
Experimental setup
Datasets: GTA Cityscapes Indian Driving Dataset Office-Home
Benchmarks: Cityscapes Indian Driving Dataset Office-Home
Metrics: mIoU accuracy harmonic mean
Baselines: BRECQ QDrop
Builds on: QDrop BRECQ HRDA SHOT error barrier analysis
Closest papers by meaning
- QT-DoG: Quantization-Aware Training for Domain Generalization (ICML 2025 · similarity 0.89)
- Quant Experts: Token-aware Adaptive Error Reconstruction with Mixture of Experts for Large Vision-Language Models Quantization (CVPR 2026 · similarity 0.87)
- Modeling Multi-Task Model Merging as Adaptive Projective Gradient Descent (ICML 2025 · similarity 0.87)
- Model Merging in Pre-training of Large Language Models (NeurIPS 2025 · similarity 0.86)
- Model Merging by Uncertainty-Based Gradient Matching (ICLR 2024 · similarity 0.86)
- Whoever Started the interference Should End It: Guiding Data-Free Model Merging via Task Vectors (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.