Erroneous inference based on a lack of preference within one group: Autism, mice, and the social approach task.
Kayla R Nygaard, Susan E Maloney, Joseph D Dougherty
PMID 31187603WHAT IT FOUND
A common statistical shortcut in mouse autism studies can falsely label a mutant group as socially impaired.
Directly comparing groups with a social preference score kept the false positive rate at 5% in simulations.
Key findings
01In simulations where two mouse groups truly did not differ, using within-group tests only produced a 25% false positive rate at 16 mice per group.
02Among 29 reviewed Social Approach Task papers, within-group-only comparisons were used in 13 of 29 sociability studies and 11 of 25 social novelty studies.
03A direct between-group comparison of a social preference index with a t-test held the false positive rate at 5% regardless of group size.
STILL TO COME
How it was doneWhat they found
Read the rest of this summary
You get three full summaries a month, free, and we do not ask for a card. Search, the TL;DRs and your library stay unlimited either way.
What it does not show
This work concerns mouse behavioral assays and statistical interpretation, not human therapy outcomes, so it does not directly change physical, occupational, or speech therapy practice. The simulations were based on parameters from one laboratory's mouse data and a set of 29 reviewed papers, so false positive rates in other settings may differ. The review counted how papers analyzed data, but it did not reanalyze primary data for every study, so some studies that used within-group-only comparisons might still have true group differences. The power estimates assume a complete loss of sociability in mutant mice; partial deficits would require more animals. Published social preference values may be inflated because positive results are more likely to be published.
Declared interests
The paper lists NIH extramural and U.S. government non-PHS research support. No separate conflict-of-interest statement is provided.
The easy way to misread this
Do not read a non-significant within-group preference in mutant mice as proof that the mutant differs from controls. The simulations show that two groups drawn from the same distribution can still produce a false positive 25% of the time at 16 mice per group.