Reliability of the PEDro scale: comparison between trials published in predatory and non-predatory journals.
Matteo Paci, Claudio Bianchini, Marco Baccini
PMID 35354496WHAT IT FOUND
Raters agreed less on the PEDro quality score for trials in predatory journals than in reputable ones.
For predatory trials, the agreement was too weak to trust a single score. This makes judging bias risk harder when relying on those publications.
Key findings
01Inter-rater reliability for the total PEDro score was lower for trials published in predatory journals (ICC 0.537) than for non-predatory journals (ICC 0.729).
02The agreement for predatory journal trials did not meet the minimum acceptable level for assessing individual trials, whereas non-predatory trials did.
03Raters had the most difficulty agreeing on baseline similarity, dropout rates, and intention-to-treat analysis, especially in predatory journals.
STILL TO COME
How it was doneWhat they foundWhat it means for PTsWhat it means for OTsWhat it means for SLPs
Read the rest of this summary
You get three full summaries a month, free, and we do not ask for a card. Search, the TL;DRs and your library stay unlimited either way.
What it does not show
Raters were not blinded to the journal source. They could see which journal the article was from, which might have influenced their scores. The definition of 'predatory' journals is not universally agreed upon. The study used the Beall's list, but other lists exist and classifications can be inconsistent. The sample of raters was mixed, including students and experienced researchers, which might have affected the agreement levels. The study did not calculate the sample size beforehand; it depended on the number of trials available from the selected journals.
Declared interests
The text does not explicitly state funding sources or conflicts of interest for the authors of this specific analysis.
The easy way to misread this
Do not assume that a PEDro score for a trial from a predatory journal is as accurate as one from a reputable journal. The reliability for predatory trials was too low to trust for individual assessments, meaning the score might reflect poor reporting quality rather than just the trial's actual methodological flaws.