Research map / 3D vision / Face and shape
HyperNet Fields: Efficiently Training Hypernetworks without Ground Truth by Learning Weight Trajectories
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Hypernetwork Fields, which learn the entire weight-space optimization trajectory of a task network by conditioning on a timestep and matching numerical weight updates to task gradients, removing the need for precomputed converged weights.
Problem
Training hypernetworks normally requires precomputed ground-truth converged weights for every training sample, which is expensive to produce at scale (e.g., 50 GPU-days for HyperDreamBooth) and assumes a one-to-one condition-to-weights mapping.
Approach
Treat the hypernetwork as a neural field over (condition, optimization step) and supervise it only by matching its finite-difference weight updates to the task network's gradient steps, so the hypernetwork learns the full convergence trajectory without any per-sample ground-truth weights.
Key results
- On CelebA-HQ, the method reaches DINO 0.605 and CLIP-I 0.639 versus HyperDreamBooth's 0.473 and 0.577, but lower face recognition (0.325 vs 0.655) and CLIP-T (0.268 vs 0.286).
- On AFHQ, the method achieves DINO 0.664 and CLIP-I 0.807, outperforming DreamBooth (0.560, 0.763) and Textual Inversion (0.596, 0.780).
- Training takes about 1.5 days on 8 A100 GPUs (~12 GPU-days), less than a quarter of HyperDreamBooth's 50 GPU-days of precomputation; inference takes 0.3 s versus 2 s for HyperDreamBooth.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HyperNet Fields instead of DreamBooth | Personalized image generation · CelebA HQ | DINO: 0.441 -> 0.605 | Helped |
| Use HyperNet Fields instead of DreamBooth | Personalized image generation · CelebA HQ | CLIP-I: 0.546 -> 0.639 | Helped |
| Use HyperNet Fields instead of DreamBooth | Personalized image generation · CelebA HQ | Face Rec.: 0.618 -> 0.325 | Hurt |
| Use HyperNet Fields instead of DreamBooth | Personalized image generation · CelebA HQ | CLIP-T: 0.282 -> 0.268 | Hurt |
| Use HyperNet Fields instead of DreamBooth | Personalized image generation · AFHQ | DINO: 0.560 -> 0.664 | Helped |
| Use HyperNet Fields instead of DreamBooth | Personalized image generation · AFHQ | CLIP-I: 0.763 -> 0.807 | Helped |
| Use HyperNet Fields instead of DreamBooth | Personalized image generation · AFHQ | CLIP-T: 0.268 -> 0.277 | Helped |
| Use HyperNet Fields instead of Textual Inversion | Personalized image generation · CelebA HQ | DINO: 0.289 -> 0.605 | Helped |
| Use HyperNet Fields instead of Textual Inversion | Personalized image generation · AFHQ | DINO: 0.596 -> 0.664 | Helped |
Surprising findings
- The hypernetwork field outperforms DreamBooth on all AFHQ metrics, which the authors attribute to learning a general trend of how DreamBooth weights behave.
- Outputs after only 100 training iterations already produce images visually similar to the conditioning images on AFHQ.
Limitations
- Output layer must have as many neurons as target network weights, giving suboptimal memory use.
- Face recognition identity preservation is substantially lower than HyperDreamBooth and DreamBooth.
- 3D image-conditioned experiment is limited to overfitting a 128-shape Objaverse subset.
Experimental setup
Datasets: CelebA-HQ AFHQ ShapeNet Objaverse
Benchmarks: CelebA-HQ AFHQ Objaverse ShapeNet
Metrics: DINO CLIP-I CLIP-T Face Recognition similarity IoU training GPU-days inference time
Baselines: HyperDreamBooth DreamBooth Textual Inversion
Builds on: HyperDreamBooth DreamBooth HyperDiffusion consistency models
Closest papers by meaning
- Foundation Models Secretly Understand Neural Network Weights: Enhancing Hypernetwork Architectures with Foundation Models (ICLR 2025 · similarity 0.89)
- Magnitude Invariant Parametrizations Improve Hypernetwork Learning (ICLR 2024 · similarity 0.88)
- Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit (ICLR 2024 · similarity 0.87)
- Towards Scalable and Versatile Weight Space Learning (ICML 2024 · similarity 0.87)
- RECAST: Reparameterized, Compact weight Adaptation for Sequential Tasks (ICLR 2025 · similarity 0.86)
- HyperFields: Towards Zero-Shot Generation of NeRFs from Text (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.