The studies that cannot prove anything
A pilot, a feasibility study, a case report and a protocol each answer a real question. None of those questions is whether the treatment works — and reading them as if it were is the commonest way to misread the literature.
Updated 9 September 2026 · Free to read
Some studies are not designed to show that a treatment works, and reading them as if they were is the most common way to misread the literature. A pilot, a feasibility study, a case report and a protocol each answer a real question. None of those questions is “does this work”.
What can each of these designs actually tell you?
A feasibility study asks whether a trial could be run at all — could you recruit, would people turn up, would they finish. A pilot is a small dress rehearsal of a larger trial, run to find the problems before they are expensive. A case report describes what happened to one patient, and a case series to a handful. A study protocol is the written plan for research that has not happened yet.
| Design | The question it answers | What it cannot tell you |
|---|---|---|
| Feasibility | Can this study be run? Will people enrol, tolerate it, and finish? | Whether the treatment helps. Improvement in a feasibility study is a by-product, not a result. |
| Pilot | Does the procedure work as intended, and what should the full trial change? | Whether the effect is real. It is deliberately too small to find out. |
| Case report / series | Is this possible? Has anyone seen this before? | Whether it would happen to anyone else. There is no comparison and no denominator. |
| Study protocol | What is about to be studied, and how — registered in advance so the plan cannot change quietly. | Anything at all about the outcome. No data exists yet. |
Feasibility: the outcome is the process
A feasibility study succeeds when it tells you the trial can proceed, or that it cannot. Both are useful answers. What it never establishes is benefit, because it has no comparison group and was not sized to detect one.
Feasibility · occupational therapy
Occupational Therapy International, 2026 · pilot study
High attendance, high satisfaction, no dropouts. Every one of those is a feasibility outcome and every one is good news for whoever runs the next trial. With five children and no control group, none of them is evidence that positioning helped, and the paper says so.
The useful reading is the operational detail. How many people were invited and how many finished tells you what recruitment looks like in your setting. A survey feasibility paper in the corpus reports 156 of 215 invited adults completing it — a number worth more to someone planning a service evaluation than any effect size in the same paper.
Pilot: small on purpose, and the p-values are a trap
A pilot is undersized by design. That has a consequence people forget: it is far more likely to miss a real effect than a full trial, and far more likely to produce a large-looking one by chance. Both errors point the same way — do not act on the number.
The warning sign: a significance test in a study too small to run one
When a pilot reports p-values across several outcomes and one of them comes out below .05, that is close to what you would expect from chance alone.
Pilot · physical therapy
Frontiers in Rehabilitation Sciences, 2026 · pilot study
Ten patients per group, three weeks, and an advantage for interval cycling on one cognitive flexibility test — and not on the other four. The title reports the one. Five tests, one positive, twenty patients: that is the shape of a chance finding, and the honest conclusion is that a full trial is worth running, not that interval training improves cognition.
This is also why “positive” means something different on a pilot. Our summaries record the direction a paper’s findings point, and on a pilot that direction describes what happened in twenty people over three weeks. It is a reason to look again, not a reason to change anyone’s programme.
Case report: an existence proof, not a rate
A case report tells you something is possible. It cannot tell you how often, in whom, or better than what — there is no comparison group and no denominator. The effect size is frequently enormous, which is exactly what makes it seductive.
Case report · physical therapy · free to read
International Journal of Sports Physical Therapy, 2026 · case report
One sprinter, twenty years of hamstring activation failure, roughly a fortyfold increase in muscle activity in a single session. Read as a rate, that is a miracle treatment. Read as what it is, it is one person who responded, with no way to know whether the next person would, and a strong argument for someone to run a trial.
A case series is the same logic with a slightly larger n, and the honest ones show you the variation.
Case series · occupational therapy
Cognitive-Functional Intervention for Adults With ADHD (Cog-Fun-A): an extreme case series analysis
OTJR: Occupation, Participation and Health, 2026 · case series
Three adult men, fifteen weekly sessions. Two improved meaningfully, one barely changed. That third man is the most informative person in the paper, and he is the one who would have disappeared into a group mean.
Protocol: a plan, and worth reading anyway
A study protocol contains no results, because the study has not been done. On this site those pages say exactly that and carry no findings section.
Protocol · multidisciplinary
Early rehabilitation for critically ill adult patients: protocol for an umbrella review
2026 · study protocol
A plan for a future review. It contains nothing about whether early rehabilitation helps or harms ICU patients, and nothing in it changes what you do on Monday.
They are still worth knowing about for two reasons. A protocol tells you a question is being answered, so you know to look again in two years. And because it is registered before the data exists, it is the document that makes it possible to notice later if the outcomes quietly changed between the plan and the paper.
How to read one of these without misreading it
- Read the design before the result. The design decides which sentences in the abstract can carry weight.
- Treat every effect size as an illustration. In these designs the size of the change tells you almost nothing about the size of the effect.
- Look for what the authors say they will do next. That sentence is usually the real finding, and it is usually honest.
- Ask what it changes on Monday. For most of these the answer is nothing, and that is not a criticism of the paper. It is what the paper is for.
All of these designs rate very low on this site, which is the whole basis of the certainty rating — and very low certainty is not the same as a small effect or a bad paper. If you are weighing whether a change on an outcome measure is real, the companion piece on MCID and MDC covers that. You can also browse every paper we have read by design.
Applied Evidence summarises peer-reviewed therapy research from the full paper. Browse the index.