Explainability in AI-enabled medical neurotechnology: a scoping review.
Laura Schopp, Georg Starke, Marcello Ienca
PMID 41639858WHAT IT FOUND
Only 14 of 161 studies on closed-loop brain devices used explainable AI methods.
Clinicians cannot currently rely on these tools to show why a device made a specific decision, as most algorithms remain opaque black boxes.
Key findings
01Just 14 of the 161 reviewed articles employed explainable AI methods, representing approximately 9% of the literature.
02Hardware constraints like limited battery life and processing power, along with the trade-off between model accuracy and interpretability, are the primary barriers to adopting explainable AI in these devices.
03Current explainable AI methods are largely designed for developers rather than clinicians, meaning they rarely provide the type of transparency needed for clinical decision-making or informed consent.
STILL TO COME
How it was doneWhat they foundWhat it means for PTsWhat it means for OTsWhat it means for SLPs
Read the rest of this summary
You get three full summaries a month, free, and we do not ask for a card. Search, the TL;DRs and your library stay unlimited either way.
What it does not show
The review was restricted to articles in English and German, potentially missing relevant studies from other regions. It only included research up to January 2025, so very recent developments in explainable AI might not be captured. Scoping reviews do not assess the quality of the included studies, so the reliability of the 161 papers was not evaluated. Many of the included studies had very small sample sizes, making their findings on device performance and explainability fragile. The review focused only on clinical neurology and psychiatry, excluding other potential applications like general rehabilitation without neural interfacing.
Declared interests
The authors declared no conflicts of interest. The study was not funded by a specific grant listed in the provided text.
The easy way to misread this
Do not assume that the absence of explainable AI in these studies means the devices are unsafe or ineffective. The low adoption rate reflects technical barriers and a focus on developer-oriented metrics, not necessarily a failure of the technology itself.
Summarised by AI from the full paper, without a clinician reviewing it. Check it against the source before it changes what you do. Read it on PubMed →