Research map / Image generation / Image restoration
On Disentangled Training for Nonlinear Transform in Learned Image Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AuxT, a lightweight linear bypass transform built from wavelet-based linear shortcuts that disentangles energy compaction from nonlinear transform learning, roughly doubling LIC training speed while keeping or improving BD-rate.
Problem
Learned image compression models converge very slowly (over two weeks to train a state-of-the-art model), largely because nonlinear transforms struggle to learn energy compaction (feature decorrelation and uneven energy modulation) during training.
Approach
Add a parallel linear auxiliary transform (AuxT) of wavelet-based linear shortcuts (Haar DWT, subband-aware scaling, orthogonal linear projection with an orthogonality regularizer) so that coarse energy compaction is handled linearly and the nonlinear transform only fits fine details.
Key results
- AuxT with TCM-small reaches -6.0% BD-rate on Kodak at 1M iterations versus -5.3% for the anchor at 2M iterations, with training time cut from 240 to 125 GPU hours (-48%).
- STF+AuxT at 0.6M iterations matches STF at 2M iterations (-3.2% BD-rate) with 68% less training time (36 vs 116 GPU hours).
- AuxT adds about 1M parameters and 1 to 10 GMACs, and the paper reports the increase in inference latency as negligible.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add AuxT to mbt2018mean | mbt2018mean · Kodak | BD-rate (%) with PSNR: 25.5 -> 22.6 | Helped |
| Add AuxT to ELIC | ELIC · Kodak | BD-rate (%) with PSNR: -4.5 -> -5.7 | Helped |
| Add AuxT to STF | STF · Kodak | BD-rate (%) with PSNR: -3.2 -> -5.7 | Helped |
| Add AuxT to TCM-small | TCM-small · Kodak | BD-rate (%) with PSNR: -5.3 -> -6.0 | Helped |
| Add AuxT to TCM-large | TCM-large · Kodak | BD-rate (%) with PSNR: -10.6 -> -11.3 | Helped |
| Remove the orthogonality constraint for OLP (ablation) | TCM-small + AuxT · Kodak | rate-distortion performance | Hurt |
| Remove subband-aware scaling (ablation) | TCM-small + AuxT · Kodak | rate-distortion performance | Hurt |
| Replace DWT with average pooling and remove subband-aware scaling (ablation) | TCM-small + AuxT · Kodak | rate-distortion performance | Hurt |
| Replace DWT with a stride-2 convolution and remove subband-aware scaling (ablation) | TCM-small + AuxT · Kodak | rate-distortion performance | Hurt |
| Replace Haar wavelet with db4 or bior2.2 (ablation) | TCM-small + AuxT · Kodak | rate-distortion performance | Hurt |
| Apply AuxT in the synthesis transform (ablation) | TCM-small + AuxT · Training convergence | convergence gain | No clear effect |
| Add ReLU after OLP (ablation) | TCM-small + AuxT · Kodak | rate-distortion performance | Hurt |
Surprising findings
- Replacing the nonlinear sub-sampling and up-sampling layers of TCM with WLS/iWLS degrades performance, so the bypass branch design matters more than the wavelet components alone.
- Adding a nonlinearity (ReLU or GDN) to AuxT significantly hurts its performance, indicating linearity is essential to the auxiliary branch.
Limitations
- Gains in BD-rate at the reduced iteration budget are modest, and some anchors show trivial gain (e.g., STF at 0.6M iterations shows no BD-rate change).
- Evaluated only on image compression with a fixed set of LIC architectures; the pairing with newer or different entropy models is not explored.
- The orthogonal constraint and wavelet choice (Haar) were selected empirically, with other bases performing worse.
Experimental setup
Datasets: ImageNet-1k
Benchmarks: Kodak Tecnick CLIC Professional Validation
Metrics: BD-rate (PSNR) BD-rate (MS-SSIM) training GPU hours GMACs parameter count inference latency
Baselines: mbt2018mean ELIC STF TCM-small TCM-large VTM-18.0
Builds on: TCM mbt2018 (joint autoregressive hyperprior) STF ELIC
Closest papers by meaning
- Learned Image Compression via Sparse Attention and Adaptive Frequency (CVPR 2026 · similarity 0.88)
- Efficient Learned Image Compression without Entropy Coding (ICML 2026 · similarity 0.87)
- Learned Image Compression With Mixed Transformer-CNN Architectures (CVPR 2023 · similarity 0.87)
- Balanced Rate-Distortion Optimization in Learned Image Compression (CVPR 2025 · similarity 0.87)
- Few-Shot Domain Adaptation for Learned Image Compression (AAAI 2025 · similarity 0.87)
- Frequency-Aware Transformer for Learned Image Compression (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.