PhyMetric
NeurIPS 2026

Diagnosing physical plausibility in text-to-video generation

Which physical law did the video break?

PhyMetric replaces one opaque plausibility score with verifiable, scene-grounded questions. Every scene carries eight balanced yes/no questions and a structured physical prior, so a score decomposes into the specific physical consequences a generator got right and wrong.

Authors · Siwei Meng, Xuerou Cai, Rui Hu, Ping Liu  |  University of Nevada, Reno

Paper Code Dataset Browse taxonomy
1,115physical scenes
8,920balanced binary questions
50physical categories
6physics domains

Abstract

The protocol on three real scenes

4 s · 120 frames

A generator receives the caption alone. Each question targets a consequence that caption implies, is answered from frames sampled around its own temporal anchors, and is scored against a reference answer fixed before any video exists. These are the frames the judge actually saw.

Taxonomy

Fifty physical categories across six domains, each defined by a hand-authored card: the governing law in closed form, the object and environment pools scenes are drawn from, and the diagnostic dimensions its questions exercise. Select a category to read its card.

Results

Accuracy % · best per judge highlighted

Four open-source text-to-video models under three VLM judges. Weighted combines question difficulty and dimension weight. The ranking is stable across judges, and the gap between the strongest and weakest model shows the benchmark is not saturated.

Citation

@inproceedings{phymetric2026,
  title     = {PhyMetric: Diagnosing Physical Plausibility in
               Text-to-Video Generation via Scene-Level QA},
  author    = {Siwei Meng, Xuerou Cai, Rui Hu, Ping Liu},
  booktitle = {NeurIPS Evaluations and Datasets Track},
  year      = {2026}
}