ExPhy: A Benchmark for Explicit Physical Property Learning in Multi-Object Trajectory Forecasting

arXiv:2608.200092026-08-21

Predicting where objects go doesn't mean a model actually knows their weight, friction, or bounciness

The authors built ExPhy, a benchmark of 24,000 physics-simulated scenes where multiple objects move and collide, each labeled not only with future trajectories but also with ground-truth mass, friction, and restitution for every object. The benchmark separates in-distribution testing from two out-of-distribution settings, one with unseen physical values and one with unseen starting positions/velocities. Their proposed model PhyODE, which estimates physical properties from observed motion and feeds them into a physics-based rollout, cuts trajectory error by over 30% versus the best prior method on the hardest long-horizon, unseen-initial-state setting.

Original abstract (English)

Understanding object dynamics requires not only predicting future trajectories but also examining whether a model captures the physical properties that govern motion. However, existing benchmarks rarely expose object-level physical properties as explicit evaluation targets alongside trajectory forecasting. To address this gap, we introduce \emph{ExPhy}, a multi-object trajectory forecasting benchmark containing 24,000 simulated physical scenes with explicit object-level labels for mass, friction, and restitution. ExPhy provides observed and future trajectories together with an in-distribution (ID) split and two out-of-distribution (OOD) splits over physical parameters (OOD-Parameter) and initial states (OOD-Initial) for jointly evaluating trajectory forecasting and physical property estimation. We further instantiate \textsc{PhyODE}, a physics-guided model with an explicit property interface that estimates physical properties from observed trajectories and uses them for differentiable future rollout. On the long-horizon OOD-Initial setting, \textsc{PhyODE} reduces ADE and FDE by 33.1\% and 31.0\%, respectively, compared with the strongest baseline. Zero-shot evaluation on ComPhy further assesses cross-benchmark transfer. Property-level analyses reveal that accurate trajectory forecasting does not necessarily imply accurate recovery of the underlying physical properties. Code and data are available at https://github.com/Zest86/ExPhy.

Authors · Rui Wang, Yeteng Wu, Xianlin Zhang, Mengshi Qi

Figures: Rui Wang et al., arXiv:2608.20009, CC BY 4.0