Research map / Detection and segmentation / CNN features

DenoiseRep: Denoising Model for Representation Learning

NeurIPS 2024 · Detection and segmentation › CNN features · approach: Convolution-transformer backbones · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DenoiseRep, which treats each backbone embedding layer as a denoising step, trains label-free denoising layers on frozen backbone features, and fuses them into the embedding weights so inference adds no latency.

Problem

Denoising diffusion models are mainly used for generation, and their use for discriminative representation learning is little explored; naively applying recursive feature denoising adds inference latency.

Approach

Treat each cascaded embedding layer of a backbone as a denoising layer with a fixed step index instead of a dynamic diffusion timestep, train per-layer denoising modules on frozen backbone features, and merge their weights into the existing embedding layers via a derived closed-form reparameterization so that denoising is computation-free at inference.

Key results

• On MSMT17 with TransReID-SSL ViT-small, DenoiseRep raises mAP from 66.30% to 67.33% and label-augmented training reaches 67.33% vs. 66.30% baseline, with inference time unchanged at 0.34s.
• On ImageNet-1k, SwinV2-T top-1 improves from 81.82% to 82.13% with no added parameters.
• On COCO with Mask-RCNN SwinV2-T, AP improves from 42.8% to 44.3%.

Measured effects reported in the paper

Surprising findings

• Denoising only the final-layer features (DenoiseRep-minus) adds about 15% inference latency, while per-layer fused denoising adds none.
• The authors report smaller gains for larger backbones, attributing this to less denoising headroom in strong features.

Limitations

• Gains are modest (typically under 1-2 points) and the authors note limited improvement on general tasks.
• Denoising is restricted to one or two steps per layer and denoising layers cannot exceed the number of backbone layers, limiting denoising strength.
• Derivation and experiments are mostly on ReID; the fusion equivalence is stated for linear layers and its general validity for CNNs is only empirically shown.

Experimental setup

Datasets: Market-1501 DukeMTMC-reID MSMT17 CUHK-03 vehicleID ImageNet-1k CUB200 Oxford-Pet Flowers COCO ADE20K Cifar-10

Benchmarks: Market-1501 DukeMTMC-reID MSMT17 CUHK-03 vehicleID ImageNet-1k CUB200 Oxford-Pet Flowers COCO ADE20K Cifar-10

Metrics: mAP Rank-1 acc@1 acc@5 AP mIoU B-IoU inference time

Baselines: TransReID-SSL TransReID CLIP-REID MGN ABD-Net Nformer DenoiseRep-minus (final-layer denoising)

Builds on: DDPM TransReID-SSL Swin Transformer Vision Transformer

Closest papers by meaning

• ADDP: Learning General Representations for Image Recognition and Generation with Alternating Denoising Diffusion Process (ICLR 2024 · similarity 0.88)
• Exploring Diffusion Time-steps for Unsupervised Representation Learning (ICLR 2024 · similarity 0.86)
• Deconstructing Denoising Diffusion Models for Self-Supervised Learning (ICLR 2025 · similarity 0.86)
• Denoising Autoregressive Representation Learning (ICML 2024 · similarity 0.86)
• Rethinking Reconstruction and Denoising in the Dark: New Perspective, General Architecture and Beyond (CVPR 2025 · similarity 0.86)
• Denoising as Adaptation: Noise-Space Domain Adaptation for Image Restoration (ICLR 2025 · similarity 0.85)