Research map / Video / Video segmentation and tracking
Learning Physics From Video: Unsupervised Physical Parameter Estimation for Continuous Dynamical Systems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a decoder-free, unsupervised method that estimates physical parameters of known second-order ODEs from single videos using a latent-space loss (prediction MSE plus KL regularizer), and releases the real-world Delfys75 dataset of 75 videos.
Problem
Unsupervised estimation of physical parameters of known continuous governing equations from video; prior unsupervised methods rely on frame reconstruction, are slow, unstable to initialization, limited to motion dynamics, and mostly tested on synthetic data.
Approach
Replace frame reconstruction with a latent-space loss: a small MLP encoder maps frames to a low-dimensional state, a differentiable Euler-step physics block with learnable ODE coefficients predicts the next latent state, and a KL-divergence term toward N(0,1) (computed over batch statistics) prevents trivial collapse; learning rates scale with initial parameter magnitude for robustness.
Key results
- On Delfys75 motion experiments, the proposed method's estimates are closer to ground truth than PAIG and NIRPI in the reported tables (e.g., pendulum L estimates 0.51 vs GT 0.45 for ours, versus 1.01 for PAIG).
- The model has 75.42K parameters and 0.11 s per epoch, versus 5.27M parameters and 252.72 s per epoch for PAIG, and 4.19M parameters and 0.95 s per epoch for NIRPI.
- On the spring-MNIST dataset, the proposed model converges to the ground-truth k=2 from initializations of 1.0 and 10.0, whereas PAIG and NIRPI fail to converge consistently.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed method instead of PAIG or NIRPI | Delfys75 parameter estimation · Delfys75 motion experiments | parameter estimation accuracy | Mixed |
| Use a decoder-free latent-space model instead of PAIG | MNIST spring synthetic dataset · MNIST spring synthetic dataset | time per epoch [s]: 252.72 -> 0.95 | Helped |
| Use a decoder-free latent-space model instead of PAIG w/o U-Net | MNIST spring synthetic dataset · MNIST spring synthetic dataset | time per epoch [s]: 80.56 -> 0.95 | Helped |
| Use a decoder-free latent-space model instead of NIRPI | MNIST spring synthetic dataset · MNIST spring synthetic dataset | time per epoch [s]: 0.11 -> 0.95 | Hurt |
| Initialize the estimated parameter k at different values | Ours, PAIG, and NIRPI · MNIST spring synthetic dataset | parameter estimation robustness | Helped |
| Change the latent prior from N(0, 1) to N(1, 0.2) | Ours · Delfys75 Torricelli | parameter estimation accuracy | Helped |
Surprising findings
- PAIG estimates in the real-world experiments stay close to the initialization value (1.0) rather than the ground truth.
- Ours is not uniformly most accurate: on the Torricelli experiment, the estimates still deviate notably from ground truth for some settings (e.g., 1.07 vs 0.90 and 1.30 vs 1.50 for the pendulum/sliding block columns) and the abstract's claim is backed by mixed numerical gaps.
Limitations
- Limited to autonomous continuous ODEs; fluids and more complex equations are not handled.
- Multiple interacting objects require masks, and the model does not yet handle independent dynamics of multiple objects in a scene.
- Discretization limits sub-pixel accuracy, so performance depends on resolution (notably scale dynamics).
Experimental setup
Datasets: Synthetic pendulum, intensity, and scale datasets Spring-MNIST dataset from PAIG Delfys75
Benchmarks: Delfys75 synthetic pendulum/intensity/scale datasets Spring-MNIST (PAIG dataset)
Metrics: estimated parameter mean and standard deviation vs ground tr convergence across initializations model size training time per epoch
Baselines: PAIG NIRPI
Builds on: Physics-as-Inverse-Graphics (PAIG) NIRPI variational autoencoder
Closest papers by meaning
- Physics from Video: Identifiability of Time-Invariant Second-Order ODEs under Minimal Trajectory Conditions (ICML 2026 · similarity 0.90)
- Learning Physics Constrained Dynamics Using Autoencoders (NeurIPS 2022 · similarity 0.89)
- Physics-as-Inverse-Graphics: Unsupervised Physical Parameter Estimation from Video (ICLR 2020 · similarity 0.88)
- PhysInOne: Visual Physics Learning and Reasoning in One Suite (CVPR 2026 · similarity 0.88)
- PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generation (NeurIPS 2025 · similarity 0.87)
- FreeGave: 3D Physics Learning from Dynamic Videos by Gaussian Velocity (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.