Single-group studies in leading physical therapy journals commonly make inappropriate conclusions regarding treatment effect modifiers. A systematic review.

The reviewers had no previously published tool for judging whether reporting was inappropriate, so they wrote their own criteria. Two reviewers agreed on 47 studies they both rated (Kappa 0.90), but a different set of criteria would produce a different proportion. The verdict was about the words a paper used, not about whether its analysis was done well or its results were correct. Only the eight highest-impact physical therapy journals were searched, and the authors state the results should not be generalised to all physical therapy journals. The search used a single database. The authors checked all 980 articles published in the Journal of Physiotherapy since 2000 and found none their search had missed, but the other seven journals were not checked the same way. The four time periods were chosen to be roughly equal in length; the authors note that different time windows could change the results. The review cannot say why inappropriate reporting is common, or why it peaked in 2006-2011 and again from 2018.

The authors declare no conflicts of interest. The text supplied does not report who funded the work.

Do not read this as a verdict on whether the treatments studied work, or on whether the analyses behind those papers were done correctly. The reviewers judged only the words each paper used, and they judged them by their own newly written criteria. A cohort study can still tell you which patients are likely to do well. What it cannot tell you is which patients respond to one particular treatment rather than another.