Autism Knowledge Assessments: A Closer Examination of Validity by Autism Experts.
Camilla M McMahon, Maryellen Brunson McClain, Savannah Wells and 2 others
PMID 38583097WHAT IT FOUND
Autism experts flagged 34% of questions in four autism knowledge assessments as potentially inaccurate and 17% as ambiguous.
Six questions had most experts disagreeing with the authors' answer. Use these tools cautiously in research.
Key findings
01Autism experts flagged 34% of questions as potentially problematic for accuracy.
02Six questions had mean accuracy below 50%, so most experts disagreed with the authors' correct answer.
0353% of questions were rated extremely ambiguous by at least one expert.
STILL TO COME
How it was doneWhat they found
Read the rest of this summary
You get three full summaries a month, free, and we do not ask for a card. Search, the TL;DRs and your library stay unlimited either way.
What it does not show
Only 14.8% of eligible journal editorial board members participated. The analysed sample was 69 experts, and three consented participants were excluded before analysis. Each question had about 30 expert ratings. Participants were limited to journal editorial board members, so autistic individuals, parents, teachers, and clinicians without editorial board roles were not included. Experts reported most familiarity with Severity Level 1 and least familiarity with Severity Level 3, so ratings may reflect Level 1 more than Level 3. No cognitive interviewing with general population respondents was conducted. The four assessments differ in difficulty, intended audience, and response format, so ratings cannot be compared directly.
The easy way to misread this
Do not conclude that these autism knowledge assessments are invalid or should be abandoned. The study flagged 34% of questions for accuracy and 17% for ambiguity, and the authors report that 62% of questions did not raise concerns. It assessed research measurement items, not clinical outcomes.