Reliability of the Brief Assessment of Transactional Success in Communication in Aphasia.
Jacquie Kurland, Anna Liu, Vishnupriya Varadharaju and 2 others
PMID 40160198WHAT IT FOUND
The BATS story-retell tool showed high agreement between automated and edited transcripts, acceptable agreement between human main-concept scorers, and acceptable stability over 7±3 days.
It does not yet prove treatment change.
Key findings
01Automated transcripts produced topic-similarity scores almost identical to human-edited transcripts, with correlation 0.99 and ICC 0.99.
02Two human raters agreed acceptably on main concept scoring, with Pearson correlation 0.87 and overall ICC 0.86.
03Test-retest data were collected within 7±3 days, and reliability was generally higher for identical stories than for different stories of the same type, but differences were not statistically significant.
STILL TO COME
How it was doneWhat they foundWhat it means for SLPs
Read the rest of this summary
You get three full summaries a month, free, and we do not ask for a card. Search, the TL;DRs and your library stay unlimited either way.
What it does not show
Only 24 people with aphasia were analysed, and they were a convenience subset of a larger test-development study. Most participants were chronically aphasic, with an average time poststroke of 6.6 years, so results may not apply to very early aphasia. The study tested reliability, not whether the tool can detect treatment change or directly measure interactional and transactional conversation. Automated transcripts were less reliable when speech was sparse, mostly nonverbal, severely dysarthric or apraxic, or when a non-native English accent was pronounced. Main concept scoring still required trained human raters and could involve implicit judgments, so it is not fully automated. Test-retest used different unfamiliar partners and no familiar partners to reduce learning effects, which added variability.
Declared interests
The authors report no competing interests to declare.
The easy way to misread this
Do not conclude that BATS has been validated as a treatment outcome measure or that it will improve communication. This paper tested score consistency, not therapy effects, and the authors state the study did not directly measure the interactional and transactional aspects of conversation.