Assessing Quality of Program Environments for Children and Youth with Autism: Autism Program Environment Rating Scale (APERS).

The NPDC sample did not have inter-rater agreement data because of resource and geographic limitations. In CSESA, two-rater agreement on total scores was moderate: the ICC was .56 and the correlation between mean total ratings was .54, even though item-level agreement within one point was high. The factor fit index RMSEA was .13 for NPDC and .11 or .10 for CSESA, while the paper says values above .08 are not acceptable and may be inflated by smaller samples. The samples were convenience samples from professional development or research projects, not randomly selected schools. For a scale with 64 or 66 items, the number of programs or schools was small for some analyses, and the authors said they approached being under-powered. All researchers were members of a single research group, and the paper calls for replication by independent researchers. Racial and ethnic data were not available for the NPDC sample. The fall-to-spring APERS increase was not tested against a control group, so it cannot separate the effect of using the APERS from other school improvement activities. The APERS requires 3-4 days of training and 2 days of staff time per school, so it is not a quick off-the-shelf check.

The NPDC work was funded by the U.S. Department of Education, Office of Special Education Programs. The CSESA work was funded by the U.S. Department of Education, Institute of Education Sciences. No other funding or conflict statements appear in the supplied text.

Do not conclude that using the APERS improves autism programs or student outcomes. The study tested whether the scale was consistent and whether scores changed after a non-randomized improvement plan; the paper says randomized studies are needed to confirm program improvement.