Inter-Rater Reliability of Novice Linkers Using an Innovative Sequential Iterative Linking Method to Link Prosthetic Outcomes to The International Classification of Functioning, Disability and Health.
Leigh Clarke, Emily Ridgewell, Xia Li and 1 others
PMID 36892440WHAT IT FOUND
Two novice linkers achieved 91.7% agreement on prosthetic outcomes after five iterative rounds, matching expert-level reliability.
The method improved agreement from 74.5% initially, suggesting novices can reliably link outcomes to the ICF with training and consensus.
Key findings
01Novice linkers achieved 91.7% agreement on the linking end-point in the final round.
02Baseline agreement in round 1 was 74.5%, which improved significantly over subsequent rounds.
03The sequential iterative method allowed novices to achieve agreement similar to expert linkers.
STILL TO COME
How it was doneWhat they found
Read the rest of this summary
You get three full summaries a month, free, and we do not ask for a card. Search, the TL;DRs and your library stay unlimited either way.
What it does not show
The study involved only two linkers, both with subject-matter expertise in prosthetics, limiting generalizability to other disciplines or larger groups. The method ensures reliability (agreement between raters) but does not guarantee accuracy (correctness of the link). Linkers did not attend face-to-face ICF workshops, relying instead on online and study-specific training. Outcomes were extracted from systematic reviews, requiring linkers to infer the 'meaningful concept' from titles and abstracts, which can be ambiguous.
Declared interests
The authors have no conflicts of interest to declare.
The easy way to misread this
Do not assume this method guarantees accurate linking. The study demonstrates that novices can agree with each other (reliability) after iterative training, but it does not prove they are linking outcomes to the correct ICF categories (accuracy).