Testing the interrater reliability of integrated interpretations using a problem-solving model.
Kentaro Kato, Ai Ishii, Kengo Sasagawa and 1 others
PMID 39092418WHAT IT FOUND
Ten experienced PTs rated student problem-solving maps with high agreement (ICC 0.88).
This suggests the structured mapping method makes clinical reasoning evaluation more consistent than subjective text review, though the small sample limits generalizability.
Key findings
01Inter-rater reliability for assessing student integrated interpretation using the problem-solving model was 0.88.
02The structured nature of the problem-solving model allowed supervisors to assess appropriateness more easily, leading to high agreement.
03The study included only 3 students and 10 supervisors, which the authors state is not sufficient to demonstrate effectiveness.
STILL TO COME
How it was doneWhat they foundWhat it means for PTs
Read the rest of this summary
You get three full summaries a month, free, and we do not ask for a card. Search, the TL;DRs and your library stay unlimited either way.
What it does not show
The sample size was extremely small, involving only three students and ten supervisors. The authors explicitly state this sample size is not sufficient to demonstrate effectiveness. The results were not compared with other assessment methods or reports. The case scenarios were fictitious and created by a co-researcher, which may not reflect real-world clinical complexity.
Declared interests
The authors declare that there are no conflicts of interest.
The easy way to misread this
Do not assume this method is proven effective for general clinical practice. The high reliability score comes from a very small pilot with only three students, and the authors themselves note this is insufficient to demonstrate effectiveness.