Alberta Rating Index for Apps: Study of Reliability and Validity.
Peyman Azad-Khaneghah, Mary Roduta Roberts, Lili Liu
PMID 35294312WHAT IT FOUND
A new tool for rating mental health apps showed moderate to high reliability for occupational therapists and adults with mental health conditions, but low reliability for older adults.
Therapists rated apps stricter than other groups. The tool correlates moderately with existing scales.
Key findings
01The Alberta Rating Index for Apps (ARIA) showed high reliability for occupational therapists (G=0.95) and moderate reliability for adults with mental health conditions (G=0.88), but low reliability for older adults (G=0.60).
02Occupational therapists rated mental health apps significantly lower than both older adults and adults with mental health conditions.
03ARIA scores had a low to moderate positive correlation with the User version of the Mobile Application Rating Scale (U-MARS), suggesting they measure similar constructs but with minimal overlap in domains.
STILL TO COME
How it was doneWhat they foundWhat it means for OTs
Read the rest of this summary
You get three full summaries a month, free, and we do not ask for a card. Search, the TL;DRs and your library stay unlimited either way.
What it does not show
The sample size was very small (10 raters completing the study). The study only included mental health apps; ARIA has not been tested on other types of health apps. Apps were selected from a curated directory, so they were similar in quality, which may limit how well the tool distinguishes between poor and good apps in the wild. The tool is not yet validated for older adults, who showed low reliability in ratings. The study did not compare ARIA to a gold standard because none exists for app quality.
Declared interests
The study received ethics approval from the University of Alberta. No specific funding sources or conflicts of interest were detailed in the provided text beyond the institutional affiliation.
The easy way to misread this
Do not assume ARIA is ready for use with older adult clients or patients. The reliability coefficient for older adults was 0.60, which is below the recommended threshold of 0.80 for clinical or research tools, indicating their ratings were too variable to be trusted as a consistent measure of app quality.