Research map / 3D vision / Face and shape
Scaling Optimization-Oriented Hypernetworks for Implicit Neural Representations
Lulu Cai, Hanxiang Ren, Siyan Dong, Li Sun, Jiefeng Wu, Youyi Zheng, Yi Ma, Yanchao Yang
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a hypernetwork that generates INR parameters for a signal in one forward pass, preserving the chain-rule structure of test-time optimization while scaling better than prior quartic-parameter designs.
Problem
Generating implicit neural representation (INR) parameters for a given signal without test-time optimization; existing hypernetworks either ignore the chain-rule structure of test-time optimization or instantiate it with MLPs whose parameter count grows quartically in target size.
Approach
A hypernetwork that mirrors the chain-rule structure of optimization-based INR fitting, so that the generated parameters follow an optimization-oriented update, while avoiding the quartic parameter growth of prior MLP instantiations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| tokenize weights by input neuron (column) instead of output neuron (row) (ablation) | hypernetwork · CelebA-HQ | PSNR (dB): 25.3 -> 27.7 | Improved |
| use output-neuron-centric tokenization (ablation) | hypernetwork · ImageNet | PSNR (dB): 19.8 -> 22.9 | Improved |
| fix cross-attention QKV roles by information-flow consistency (ablation) | hypernetwork · NeRF (weights & images) | PSNR (dB): 22.9 -> 27.8 | Improved |
| fix QKV roles by information-flow consistency (ablation) | hypernetwork · CelebA-HQ | PSNR (dB): 23.5 -> 27.7 | Improved |
| replace MLP body with attention in optimization-oriented hypernetwork | Ours vs HyPoGen · ImageNet | PSNR (dB): 13.2 -> 22.9 | Improved |
| add optimization-oriented structure to attention hypernetwork | Ours vs LDMI · CelebA-HQ | PSNR (dB): 24.8 -> 27.7 | Improved |
| add optimization-oriented structure under weights-only supervision | Ours vs LDMI · Pre-trained NeRF (weights only) | PSNR (dB): 19.3 -> 22.0 | Improved |
| MLP-based optimization-oriented body vs attention-only body (HyPoGen vs LDMI) | HyPoGen · Pre-trained NeRF (weights only) | PSNR (dB): 19.9 vs 19.3 | Mixed |
| train without ground-truth INR weights (images only) | Ours · Pre-trained NeRF | PSNR (dB): 27.8 -> 28.8 | Improved |
| increase NeRF training data from 10% to 100% (ablation) | Ours · Pre-trained NeRF | PSNR (dB): +3.2 dB (LDMI +2.5, HyPoGen +1.8) | Improved |
| increase target INR width from 128 to 1024 (ablation) | Ours · CelebA-HQ | PSNR (dB): 26.5 -> 28.8 | Improved |
| increase target INR width to 1024 (ablation) | HyPoGen · CelebA-HQ | PSNR (dB) | Hurt |
Experimental setup
Builds on: optimization-oriented hypernetworks
Closest papers by meaning
- Foundation Models Secretly Understand Neural Network Weights: Enhancing Hypernetwork Architectures with Foundation Models (ICLR 2025 · similarity 0.89)
- Principled Architecture-aware Scaling of Hyperparameters (ICLR 2024 · similarity 0.88)
- Magnitude Invariant Parametrizations Improve Hypernetwork Learning (ICLR 2024 · similarity 0.88)
- HyperNet Fields: Efficiently Training Hypernetworks without Ground Truth by Learning Weight Trajectories (CVPR 2025 · similarity 0.88)
- Towards Scalable and Versatile Weight Space Learning (ICML 2024 · similarity 0.87)
- Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
