Research map / Image generation / Autoregressive and latent generation
TFCustom: Customized Image Generation with Time-Aware Frequency Feature Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TFCustom, a fine-tuning-free subject-driven image generation framework that adds a synchronized ReferenceNet, a time-aware high/low-frequency feature refinement module, and a DINOv2 reward loss.
Problem
Current ReferenceNet-based customization injects reference features at a constant intensity across denoising timesteps and mainly as latent-level features, which limits image consistency and multi-object identity preservation.
Approach
Noise the reference image to match each denoising timestep so ReferenceNet features are time-aligned, split reference features into high- and low-frequency components via learnable Krisch and Gaussian convolutions, and modulate them with time embeddings (AdaLN) for coarse-to-fine injection, plus a DINOv2-based reward loss on one-step x0 predictions.
Key results
- Zero-shot single-subject on DreamBench: CLIP-I 84.1, DINO 71.4, CLIP-T 33.9, beating MS-Diffusion (DINO 67.1) and Kosmos-G (CLIP-I 82.2).
- Multi-subject on MS-Bench: CLIP-I 74.3, DINO 43.8, M-DINO 12.9, CLIP-T 36.9, above MS-Diffusion (M-DINO 10.8).
- Ablation: removing all components drops CLIP-I from 85.9 to 81.3 on DreamBench.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add synchronized denoising (Lref) to TFCustom (ablation) | TFCustom · DreamBench, FT | CLIP-I: 81.3 -> 83.2 | Helped |
| Add the time-aware frequency module to synchronized denoising (ablation) | TFCustom · DreamBench, FT | CLIP-I: 83.2 -> 85.9 | Helped |
| Add the reward model loss to synchronized denoising and frequency refinement (ablation) | TFCustom · DreamBench, FT | CLIP-I: 72.4 -> 73.7 | Helped |
| Use 1 M rather than 0.5 M training samples (ablation) | TFCustom · DreamBench, FT | CLIP-I: 80.1 -> 84.0 | Helped |
| Use 1.5 M rather than 1 M training samples (ablation) | TFCustom · DreamBench, FT | CLIP-I: 84.0 -> 84.9 | Helped |
| Use 2.3 M rather than 1.5 M training samples (ablation) | TFCustom · DreamBench, FT | CLIP-I: 84.9 -> 85.9 | Helped |
| Replace DINOv2 with DeepLabv3-MBv2 as the reward model (ablation) | TFCustom · DreamBench, FT | CLIP-I: 85.9 -> 84.1 | Hurt |
| Replace DINOv2 with FCN-R101 as the reward model (ablation) | TFCustom · DreamBench, FT | CLIP-I: 85.9 -> 84.3 | Hurt |
| Replace DINOv2 with UperNet-R50 as the reward model (ablation) | TFCustom · DreamBench, FT | CLIP-I: 85.9 -> 85.4 | Hurt |
| Use TFCustom rather than Kosmos-G | single-subject generation, No-FT · DreamBench | CLIP-I: 82.2 -> 84.1 | Helped |
| Use TFCustom rather than MS-Diffusion | single-subject generation, No-FT · DreamBench | DINO: 67.1 -> 71.4 | Helped |
| Use TFCustom rather than MS-Diffusion | single-subject generation, FT · DreamBench | CLIP-T: 31.3 -> 35.1 | Helped |
Limitations
- Training relies on a 2.3M-pair in-house dataset that is not released.
- Reward loss is only applied for early timesteps because one-step x0 estimation error grows with t.
- Evaluation covers only DreamBench and MS-Bench with automatic metrics and no human study.
Experimental setup
Datasets: in-house image pairs (2.3M) DreamBench MS-Bench
Benchmarks: DreamBench MS-Bench
Metrics: CLIP-I DINO CLIP-T M-DINO
Baselines: BLIP-Diffusion IP-Adapter Kosmos-G Emu2 λ-ECLIPSE SSR-Encoder MS-Diffusion DreamBooth Textual Inversion
Builds on: ReferenceNet AnimateAnyone DiT DreamBooth
Closest papers by meaning
- Disentangling Dual Image References in Frequency Aware Diffusion Models for Personalized Generation (NeurIPS 2026 · similarity 0.87)
- RefDrop: Controllable Consistency in Image or Video Generation via Reference Feature Guidance (NeurIPS 2024 · similarity 0.85)
- Refine-by-Align: Reference-Guided Artifacts Refinement through Semantic Alignment (ICLR 2025 · similarity 0.85)
- Resolving Multi-Condition Confusion for Finetuning-Free Personalized Image Generation (AAAI 2025 · similarity 0.84)
- DreamCache: Finetuning-Free Lightweight Personalized Image Generation via Feature Caching (CVPR 2025 · similarity 0.84)
- Energy-Guided Optimization for Personalized Image Editing with Pretrained Text-to-Image Diffusion Models (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.