Research map / Image generation / Autoregressive and latent generation
SSC-VAE: Structured Sparse Coding Based Variational Autoencoder for Detail Preserved Image Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends sparse-coding VAE (SC-VAE) with a two-step learned threshold controller (conv inference plus CBAM-style refinement) that makes LISTA thresholds vectorized and structure-aware, improving detail-preserving image reconstruction and denoising.
Problem
Discrete latent representations in VAEs (VQ-VAE, SC-VAE) treat latent codes independently, ignoring spatial correlations across patches and internal correlations across dictionary atoms, which yields coarse reconstructions and loses local detail.
Approach
Replace SC-VAE's scalar sparsity threshold with a vector of per-element thresholds inferred by a residual conv network, then refine them with CBAM-style channel and spatial attention before feeding them into an unrolled LISTA sparse coder, so that thresholds encode within-code and across-code structure.
Key results
- On the ultrasound dataset SSC-VAE reaches SSIM 0.94, PSNR 35.78 dB, LPIPS 0.05 and FID 17.83, versus SC-VAE at 0.87, 32.53 dB, 0.20 and 63.21.
- On BSDS500 SSC-VAE improves over SC-VAE (SSIM 0.94 vs 0.87, PSNR 31.19 vs 28.97 dB, FID 16.40 vs 36.22).
- Ablation on BSDS500 shows the inference-only variant (SCVAE+In) reaches SSIM 0.90 and FID 28.37, with the refinement module adding further gains; computational overhead is about 2% more FLOPs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add structured sparsity controller inference and refinement to SC-VAE | SC-VAE · Ultrasound test | SSIM: 0.87 -> 0.94 | Helped |
| add structured sparsity controller inference and refinement to SC-VAE | SC-VAE · Ultrasound test | PSNR: 32.53 -> 35.78 | Helped |
| add structured sparsity controller inference and refinement to SC-VAE | SC-VAE · Ultrasound test | LPIPS: 0.20 -> 0.05 | Helped |
| add structured sparsity controller inference and refinement to SC-VAE | SC-VAE · Ultrasound test | FID: 63.21 -> 17.83 | Helped |
| add structured sparsity controller inference and refinement to SC-VAE | SC-VAE · BSDS500 test | SSIM: 0.87 -> 0.94 | Helped |
| add structured sparsity controller inference and refinement to SC-VAE | SC-VAE · BSDS500 test | PSNR: 28.97 -> 31.19 | Helped |
| add structured sparsity controller inference and refinement to SC-VAE | SC-VAE · BSDS500 test | LPIPS: 0.08 -> 0.02 | Helped |
| add structured sparsity controller inference and refinement to SC-VAE | SC-VAE · BSDS500 test | FID: 36.22 -> 16.40 | Helped |
| add structured sparsity controller inference and refinement to SC-VAE | SC-VAE · mini-ImageNet test | SSIM: 0.93 -> 0.97 | Helped |
| add structured sparsity controller inference and refinement to SC-VAE | SC-VAE · mini-ImageNet test | PSNR: 31.20 -> 35.48 | Helped |
| add structured sparsity controller inference and refinement to SC-VAE | SC-VAE · mini-ImageNet test | LPIPS: 0.02 -> 0.01 | Helped |
| add structured sparsity controller inference and refinement to SC-VAE | SC-VAE · mini-ImageNet test | FID: 1.53 -> 0.80 | Helped |
Limitations
- Gains are reported on a small set of datasets and mainly against self-trained baselines.
- Denoising evaluated only with additive Gaussian noise on ultrasound images.
Experimental setup
Datasets: Ultrasound (MICCAI 2023 Ultrasound Image Enhancement Challenge) BSDS500 mini-Imagenet
Benchmarks: Ultrasound BSDS500 mini-Imagenet
Metrics: SSIM PSNR LPIPS FID
Baselines: SC-VAE VQ-VAE VQ-GAN VAE
Builds on: SC-VAE LISTA VQ-VAE CBAM VQGAN
Closest papers by meaning
- Be a Goldfish: Forgetting Bad Conditioning in Sparse Linear Regression via Variational Autoencoders (ICML 2025 · similarity 0.88)
- Sparse Autoencoders, Again? (ICML 2025 · similarity 0.87)
- Improved Video VAE for Latent Video Diffusion Model (CVPR 2025 · similarity 0.87)
- LMD: Faster Image Reconstruction with Latent Masking Diffusion (AAAI 2024 · similarity 0.86)
- DreamUHD: Frequency Enhanced Variational Autoencoder for Ultra-High-Definition Image Restoration (AAAI 2025 · similarity 0.86)
- SeNM-VAE: Semi-Supervised Noise Modeling with Hierarchical Variational Autoencoder (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.