Assess the Fidelity, Validity and Responsiveness of the Brief Observation of Social Communication Change (BOSCC) to Measure Changes in the Social Interactions of Preschool Children With Autism Spectrum Disorder.

The clinician ratings used to test whether the tool detects change were made from the same 12-minute videos the BOSCC was scored from, so the two are not independent and their agreement is inflated. The trial these children came from found no clear treatment effect, so the researchers could not test whether the tool detects change produced by an intervention that works. Agreement between two assessments a month apart was poor (0.43) in children with a nonverbal developmental quotient below the median, and that subgroup was small, so scores for these children may be unstable. Only two people coded the videos, which limits how far the reliability figures can be generalised. Four items (facial expression, sensory interest, mannerisms and vocalisation) showed floor or ceiling effects above the 15% threshold, so they may not detect change. Fifty children had no usable video because of recording quality or parental objection, and the number rated fell from 139 at entry to 131 at two years.

The original randomised trial was funded by the French Direction Générale de l'Offre de Soins (PREPS 14-0533) and the Fondation de France. This reanalysis was funded by the Pôle de Recherche et de Valorisation at CH le Vinatier, Bron. The authors declared no conflicts of interest.

Do not read this as evidence that early intervention works, or that the BOSCC is better than the ADOS at detecting change. The trial these children came from found no clear treatment effect, and the strongest result, the agreement with clinician-rated change, was tested against ratings made from the same videos the BOSCC was scored from.