Research map / Large language models / Reasoning and chain of thought
Implicit Models: Expressive Power Scales with Test-Time Compute
Jialin Liu, Lisang Ding, Stanley Osher, Wotao Yin
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies implicit models (weight-tied networks iterated to a fixed point), arguing their expressive power scales with test-time compute.
Problem
Implicit models are known empirically to match larger explicit models with constant training memory, but the source of their expressive power is not well understood.
Approach
Frames implicit models as infinite-depth weight-tied networks whose expressive power grows with the amount of test-time computation spent iterating to a fixed point.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase test-time fixed-point iterations of the implicit operator (ablation) | Implicit FNO · 2D steady Navier-Stokes | relative error: 1.1 -> 0.0784 (t=1 to t=50) | Improved |
| increase test-time iterations (ablation) | Implicit FNO · 2D steady Navier-Stokes | empirical Lipschitz of iterate map: 23.1 -> 367 | Improved |
| increase test-time iterations (ablation) | Implicit GNN · Linear programming | relative error: 0.575 -> 0.146 | Improved |
| increase test-time iterations (ablation) | implicit image deblurring (PGD, HQS) · BSDS500 deblurring | PSNR and Lipschitz: L_t from 0.140/0.436 to about 5.0, PSNR rises and stabilizes | Improved |
| use implicit HQS model instead of explicit DRUNet | image deblurring · BSDS500 deblurring test set | PSNR (dB): 26.97 -> 29.18 | Improved |
| use implicit FNO instead of explicit FNO at same size | FNO · 2D steady Navier-Stokes test set | relative error: 0.179 -> 0.078 | Improved |
| use implicit GNN instead of explicit GNN | LP GNN · Linear programming test | relative error: 0.273 -> 0.177 (emb 8) | Improved |
| smaller implicit model versus larger explicit model | LP GNN · Linear programming train | relative error: 0.233 -> 0.203 and 0.183 -> 0.162 | Improved |
| widen explicit GNN from 8 to 32 (ablation) | explicit GNN · Linear programming test | relative error: 0.273 -> 0.318 | Hurt |
| widen implicit GNN from 16 to 32 (ablation) | implicit GNN · Linear programming test | relative error: 0.152 -> 0.156 | No clear effect |
| increase looped-transformer recurrent iterations (ablation) | pretrained looped transformer (Geiping et al.) · LLM reasoning, charge/voltage vs charge/pay prompts | qualitative output separation | Improved |
Closest papers by meaning
- Kinetics: Rethinking Test-Time Scaling Law (NeurIPS 2025 · similarity 0.88)
- Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck (NeurIPS 2026 · similarity 0.87)
- Rethinking Fine-Tuning when Scaling Test-Time Compute: Limiting Confidence Improves Mathematical Reasoning (NeurIPS 2025 · similarity 0.86)
- On the Expressive Power and Limitations of Multi-Layer SSMs (NeurIPS 2026 · similarity 0.86)
- What If We Allocate Test-Time Compute Adaptively? (ICML 2026 · similarity 0.86)
- Sample, Scrutinize and Scale: Effective Inference-Time Search by Scaling Verification (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
