Research map / Vision-language and multimodal / Multimodal LLMs

Mimicking the Physicist's Eye: A VLM-centric Approach for Physics Formula Discovery

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes VIPER-R1, a Qwen2.5-VL-based multimodal model that induces physics equations from phase portraits and trajectories via SFT with causal CoT, GRPO-based structural reward calibration, and an agentic symbolic-regression residual step (SR²), evaluated on a new 5,000-instance synthetic PhysSymbol corpus.

Problem

Automated discovery of governing equations from data relies on numerical tokens or symbolic regression and ignores visual phase-space and trajectory evidence that physicists use; existing VLMs perform poorly at plot-based formula hypothesis generation.

Approach

Train a VLM in two stages (SFT with causal chain-of-thought, then GRPO with a parameter-agnostic structural Jaccard reward) to propose symbolic ansatzes from plots, then let the model call a symbolic regression tool on the residual between the ansatz and measured accelerations.

Key results

• VIPER-R1-7B reaches structural score 0.812 and accuracy 0.487, vs best VLM baseline Claude-4-Sonnet at 0.518 structural and GPT-5 at 0.363 accuracy (paper claims 56.7% and 45.4% relative gains).
• After SR², VIPER-R1-7B achieves post-SR² MSE 0.032 vs best baseline 0.091 (Claude-4-Sonnet).
• Ablation on 7B: base Qwen2.5-VL structural 0.096 → MSI-only 0.554 → MSI+RGSC 0.812.

Surprising findings

• Grok 3 scores near zero structural (0.026) and accuracy (0.019) on PhysSymbol.
• Gemini 2.5 Pro has a low structural score (0.302) despite a competitive post-SR² MSE (0.107).

Limitations

• Evaluated only on synthetic simple mechanical systems with 2-5 term equations generated from a fixed term library.
• The SR² step relies on an external symbolic regression tool and uses MSE on the same test trajectories.
• Ablation and baseline comparisons mix base models without matched prompting details; no real experimental video data.

Experimental setup

Datasets: PhysSymbol

Benchmarks: PhysSymbol test set

Metrics: structural score (Jaccard) accuracy score (exact symbolic match) post-SR² MSE

Baselines: GPT-5 GPT-5 mini GPT-4o GPT-4o mini GPT-o3 Grok 3 Claude-4 Sonnet Claude-3.7 Sonnet Qwen-VL-Max Qwen-VL-2.5-72B-Instruct Gemini 2.5 Pro

Builds on: GRPO Qwen2.5-VL LLM-SR PySR Open-R1

Closest papers by meaning

• Active Symbolic Discovery of Ordinary Differential Equations via Phase Portrait Sketching (AAAI 2025 · similarity 0.88)
• LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Models (ICML 2025 · similarity 0.87)
• LLM-SR: Scientific Equation Discovery via Programming with Large Language Models (ICLR 2025 · similarity 0.86)
• Symmetry-Informed Governing Equation Discovery (NeurIPS 2024 · similarity 0.85)
• Beyond Static Vision: Scene Dynamic Field Unlocks Intuitive Physics Understanding in Multi-modal Large Language Models (ICLR 2026 · similarity 0.85)
• SNIP: Bridging Mathematical Symbolic and Numeric Realms with Unified Pre-training (ICLR 2024 · similarity 0.85)