Research map / Large language models / Reasoning and chain of thought

Tina: Tiny Reasoning Models via LoRA

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Tina applies LoRA during GRPO-style RL to the 1.5B DeepSeek-R1-Distill-Qwen model, reaching competitive reasoning scores on six math/science benchmarks for about $9 of post-training and evaluation cost.

Problem

Reasoning RL for language models is resource-intensive; the paper asks how cheaply strong reasoning can be instilled in small models via RL.

Approach

Apply LoRA (rank 32, alpha 128) during GRPO-style RL on a tiny 1.5B reasoning-distilled base model, using open datasets and minimal hardware, and hypothesize LoRA mainly adapts the reasoning output format while preserving knowledge.

Key results

• Tina-Open-RS2 reaches 50.60% average across six benchmarks vs 41.60% for its full-parameter baseline Open-RS2, with 43.33% Pass@1 on AIME24.
• The best Tina checkpoint costs about $9 for training plus evaluation, against an estimated 260x reduction in cost claimed in the abstract.
• Tina-Open-RS3-DrGRPO reaches its best checkpoint at 17% of an epoch and scores 49.53% average, similar to GRPO's 49.45%.

Surprising findings

• Tina models trained on a 7k-example dataset outperform those trained on the 93.7k OpenR1 set (50.60% vs 49.26% average).
• In LoRA runs, more training compute tends to reduce performance, unlike full-parameter runs.

Limitations

• Only a 1.5B base model was studied, so the reasoning ceiling may be lower than for larger models.
• Evaluation focuses on math and science benchmarks; coding and other domains are not tested.
• Hyperparameters were kept near defaults with little tuning, so gains from tuning are unexplored.

Experimental setup

Datasets: STILL-3 (33k) DeepScaleR (40.3k) Open-S1 (18.6k) Open-RS (7k) LIMR (1.39k) OpenR1 (93.7k) OpenThoughts (66.1k)

Benchmarks: AIME24 AIME25 AMC23 MATH500 GPQA Diamond Minerva

Metrics: pass@1 average accuracy training FLOPs cost in USD

Baselines: STILL-3-1.5B-preview DeepScaleR-1.5B-Preview Open-RS1 Open-RS2 Open-RS3 DeepSeek-R1-Distill-Qwen-1.5B

Builds on: GRPO LoRA Open-R1 DeepScaleR Open-RS STILL

Closest papers by meaning

• Hybrid-LoRA: Bridging Full Fine-Tuning and Low-Rank Adaptation for Post-Training (NeurIPS 2026 · similarity 0.92)
• RL of Thoughts: Navigating LLM Reasoning with Inference-time Reinforcement Learning (ICLR 2026 · similarity 0.91)
• Reasoning Under 1 Billion: Memory-Augmented Reinforcement Learning for Large Language Models (NeurIPS 2026 · similarity 0.91)
• LoongRL: Reinforcement Learning for Advanced Reasoning over Long Contexts (ICLR 2026 · similarity 0.91)
• RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMs (ICLR 2026 · similarity 0.91)
• On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Models (ICML 2026 · similarity 0.91)