A source-verified, agent-assisted audit of SARS-CoV-2 main-protease (Mpro) molecular docking — the first protocol of the reproduce engine, run by hand.
We gathered 236 open-access docking papers — around 12,466 individual ligand results — and audited every one against a sixteen-field reporting standard built for the job. The picture was bleak: only 8.1% of papers reported enough to run their docking from the methods alone, nearly half were blocked outright by a single missing parameter, and not one of the 236 was fully reproducible.
Executability class predicts reproducibility: cleanly-reported papers re-docked within a median 0.36 kcal/mol of the published number, while execution-blocked papers couldn’t be reconstructed at all. And the grading itself only holds up when it’s rule-based — a holistic AI-only pass agreed with expert reviewers barely better than chance, while a deterministic classifier reading the same pinned parameters matched two independent experts almost exactly.
Release — Mpro-DockExec: an open, corpus-scale executability benchmark, scoring engine, re-execution scripts, and provenance package.