A Tool for Automatic Scoring of Spelling Performance.
Charalambos Themistocleous, Kyriaki Neophytou, Brenda Rapp and 1 others
PMID 33151810WHAT IT FOUND
An automated scoring tool matched manual spelling scores closely, with correlations of .99 for words and .95 for nonwords.
It processed the entire dataset in under one second, replacing 120 hours of manual work while reducing scorer disagreement.
Key findings
01The normalized Damerau–Levenshtein distance metric correlated highly with manual scoring for both real words and nonwords.
02Automated scoring took less than one second for the whole dataset, whereas manual scoring required approximately 120 hours.
03Manual scoring of nonwords showed lower agreement between clinicians than scoring of real words, highlighting the need for a more reliable system.
STILL TO COME
How it was doneWhat they foundWhat it means for SLPs
Read the rest of this summary
You get three full summaries a month, free, and we do not ask for a card. Search, the TL;DRs and your library stay unlimited either way.
What it does not show
The study used data only from patients with PPA, so its performance in other populations (e.g., children, stroke-induced aphasia) is inferred rather than tested. The automated method provides item-level scores but does not identify error types (e.g., semantic vs. phonological errors). The algorithm relies on English pronunciation rules, which may not generalize well to patients with different accents or dialects without adjustment. The study did not test the tool's impact on clinical decision-making or patient outcomes, only its correlation with manual scores.
Declared interests
The study was supported by the Science of Learning Institute at Johns Hopkins University and the National Institute on Deafness and Other Communication Disorders (Grant R01 DC014475). No commercial conflicts of interest were reported.
The easy way to misread this
Do not assume this tool identifies the type of spelling error (e.g., whether a patient confused a word's meaning or just its sounds). It only provides a score for how close the response is to the target, so you still need to analyze error patterns manually if you want to understand the underlying language deficit.