Performance of the Large Language Model ChatGPT on the National Nurse Examinations in Japan: Evaluation Study.
Kazuya Taira, Takahiro Itaya, Ayame Hanada
PMID 37368470WHAT IT FOUND
ChatGPT passed the 2019 Japanese nurse exam but failed 2020-2023, scoring 75.1% on basic knowledge and 64.5% on general questions.
It struggled with complex situations and pharmacology, showing it cannot yet safely replace clinical judgment.
Key findings
01ChatGPT met the passing criteria only for the 2019 examination, not for 2020-2023.
02Average correct answers were 75.1% for basic knowledge and 64.5% for general questions.
03Performance was lower in pharmacology, social welfare, law, endocrinology, and dermatology.
STILL TO COME
How it was doneWhat they foundWhat it means for RNs
Read the rest of this summary
You get three full summaries a month, free, and we do not ask for a card. Search, the TL;DRs and your library stay unlimited either way.
What it does not show
Questions with images, graphs, or clinical photos were excluded because GPT-3.5 cannot process them. The study used simple prompts without advanced engineering or follow-up dialogue, which may have underestimated the AI's potential. ChatGPT's training data ended in 2021, so it lacked knowledge for the 2022 and 2023 exams. The AI sometimes provided false answers without warning or misaligned its responses with the number of choices.
Declared interests
None declared.
The easy way to misread this
Do not assume ChatGPT is safe for clinical decision-making. It failed the most recent exams, struggled with complex patient situations, and provided false answers without warning, particularly in pharmacology and law.