Research map / Architectures and efficiency / RNNs and normalization
Recursive Inference Scaling: A Winning Path to Scalable Inference in Language and Multimodal Systems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Recursive INference Scaling (RINS), which recursively applies the first half of a transformer several times at fixed parameter count, and shows compute-matched gains in language modeling and SigLIP vision-language pretraining, plus a no-regret variant using stochastic depth and linear adapters.
Problem
How to scale inference compute with a fixed model size and training budget; prior recursive/parameter-sharing works compared models on equal tokens rather than equal training FLOPs.
Approach
Split a network into two blocks and apply the first block recursively r times before the second block (signature A^r B), trained under matched training FLOPs; stochastic depth over recursion rounds plus per-round linear adapters allow running without recursion at test time with little loss.
Key results
- SigLIP-RINS-B/16 improves 0-shot ImageNet from 77.3% to 79.6% and CIFAR100 from 70.3% to 80.7% at matched training FLOPs.
- On 600M-parameter LMs, RINS improves both the scaling exponent and the asymptotic log-perplexity limit over the non-recursive baseline and RAO.
- With linear adapters and stochastic depth (p_s=0.5 or 0.8), RINS models trained with recursion disabled at inference match or beat the baseline at 1x inference cost.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the baseline with RINS | 600M language model · OpenBookQA (0-shot) | accuracy: 37.1±0.8 -> 39.0±0.3; RAO: 40.0±0.6 | Hurt |
| Replace the baseline with RINS | 600M language model · BoolQ, PIQA, SIQA, HellaSwag, CommonSenseQA (0-shot) | accuracy: BoolQ 53.1±4.7 -> 59.5±0.8; PIQA 65.8±3.4 -> 69.8±0.3; SIQA 40.2±0.5 -> 40.9±0.4; HellaSwag 46.0±1.0 -> 50.2±0.5; Common | Helped |
| Enable RINS recursion during pretraining (p_s=0) | SigLIP-B/16 · ImageNet, CIFAR100, Pet (0-shot) | accuracy: ImageNet 77.3 -> 79.0; CIFAR100 70.3 -> 78.5; Pet 92.6 -> 94.8 | Helped |
| Use stochastic RINS with p_s=1/4 (ablation) | SigLIP-B/16 · ImageNet, CIFAR100, cultural diversity, XM3600, COCO, Flickr | benchmark scores: ImageNet 77.3 -> 79.6; CIFAR100 70.3 -> 80.7; XM3600 img2txt@1 48.4 -> 53.5; COCO img2txt@1 67.6 -> 70.0 | Helped |
| Use stochastic RINS with p_s=1/2 (ablation) | SigLIP-B/16 · CIFAR100 and several overtraining benchmarks | benchmark scores: CIFAR100 70.3 -> 81.7; Pet 92.6 -> 92.7; ImageNet 77.3 -> 79.2 | Mixed |
| Increase recursion rounds from one to two in SigLIP-B/16 (ablation) | SigLIP-B/16 · ImageNet, CIFAR100, Pet, COCO and Flickr | benchmark scores: ImageNet 73.4 -> 74.1; CIFAR100 68.9 -> 72.2; Pet 90.4 -> 90.0 | Mixed |
| Train on 336-resolution images instead of 224-resolution images (ablation) | ViT-B/16 · ImageNet-ILSRCV2012, 300 epochs | Avg accuracy: 72.1 -> 73.0 | Helped |
| Use recursive parameter sharing instead of the 336-resolution baseline (ablation) | ViT-B/16 · ImageNet-ILSRCV2012, 1,000 epochs | Avg accuracy: A@336 75.0; recursive variants 73.7, 73.3, 73.2, 72.8 | Hurt |
| Increase RINS skip probability during training (ablation) | 1B language model · Language modeling, inference at 1x cost | perplexity | Helped |
| Increase RINS skip probability during training (ablation) | 1B language model · Language modeling, scaled inference | perplexity | Hurt |
| Add lightweight linear adapters to stochastic RINS (ablation) | 600M language model · Language modeling, 1x inference cost | perplexity | Helped |
| Add KV cache sharing during RINS recursion (ablation) | RINS language model · Language modeling | perplexity | Hurt |
Surprising findings
- For ViT image classification, recursive architectures including RINS gave no advantage, and a non-recursive baseline at higher resolution beat all parameter-sharing variants.
- For moderately sized models (over 1B), RAO gains can be matched by simply training the baseline longer at equal compute.
Limitations
- Gains in vision-language and language results depend on the number of recursion rounds and inference compute; at fixed inference scaling, benefits shrink with higher stochastic skip probability.
- KV cache sharing reduces but diminishes the gain.
- Vision classification experiments show no benefit, so the fractal-language explanation is not directly verified.
Experimental setup
Datasets: C4 SlimPajama image-text pairs (English and multilingual mixture) ImageNet ILSVRC2012
Benchmarks: OpenBookQA BoolQ PIQA SIQA HellaSwag CommonSenseQA ImageNet CIFAR100 Pet COCO Flickr30k XM3600
Metrics: perplexity 0-shot accuracy recall@K scaling exponent asymptotic log-perplexity
Baselines: non-recursive baseline MobileLLM repeat-all-over (RAO) latent recurrent thinking long-sequence baseline SigLIP-B/16
Builds on: Universal Transformers ALBERT MobileLLM stochastic depth SigLIP
Closest papers by meaning
- Scaling Inference-Efficient Language Models (ICML 2025 · similarity 0.91)
- Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for LLM Problem-Solving (ICLR 2025 · similarity 0.90)
- Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs (ICLR 2026 · similarity 0.89)
- Entropy-Gated Latent Recursion (NeurIPS 2026 · similarity 0.89)
- Recursive Language Models (NeurIPS 2026 · similarity 0.89)
- The Sparsity Whisperer (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.