Adding a fourth rater to three had little impact in pre-linguistic outcome classification.
Christina Persson, Elizabeth J Conroy, Carrol Gamble and 3 others
PMID 32372661WHAT IT FOUND
Adding a fourth rater to pre-linguistic speech assessments changed classification outcomes in only 1 of 4 video samples.
Three raters were sufficient for canonical babbling presence, ratio, and syllable inventory size in most cases. A fourth rater did not improve stability enough to justify the extra time.
Key findings
01For three of the four video recordings, adding a fourth rater had no impact on the classification of canonical babbling presence or absence.
02In the one recording where a fourth rater mattered, the infant had a low canonical babbling ratio (mean 0.22) near the cut-off for non-canonical status, suggesting difficult cases drive disagreement.
03The maximum absolute difference in canonical babbling ratio between three and four raters was small (0.06 to 0.13), indicating high stability for this measure.
STILL TO COME
How it was doneWhat they foundWhat it means for SLPs
Read the rest of this summary
You get three full summaries a month, free, and we do not ask for a card. Search, the TL;DRs and your library stay unlimited either way.
What it does not show
The study only included four video samples, all from infants deemed to have some level of canonical babbling; it did not include clearly non-canonical infants, so results may not generalize to those cases. One language group (Brazilian Portuguese) was excluded because no recordings met the utterance count requirement. Intra-rater reliability was tested on the same day, which may have influenced memory-based outcomes like syllable inventory size.
Declared interests
The authors report no conflict of interest. The study was funded by the National Institute of Dental and Craniofacial Research (NIDCR).
The easy way to misread this
Do not assume three raters are always sufficient. The study found that infants with low canonical babbling ratios (near the cut-off) or those producing immature sounds like glottal stops caused significant disagreement between raters, where a fourth rater or arbitrator was necessary to resolve the classification.