Pilot, feasibility, protocol, case report: what each one is for

What can each of these designs actually tell you?

A feasibility study asks whether a trial could be run at all — could you recruit, would people turn up, would they finish. A pilot is a small dress rehearsal of a larger trial, run to find the problems before they are expensive. A case report describes what happened to one patient, and a case series to a handful. A study protocol is the written plan for research that has not happened yet.

Feasibility: the outcome is the process

A feasibility study succeeds when it tells you the trial can proceed, or that it cannot. Both are useful answers. What it never establishes is benefit, because it has no comparison group and was not sized to detect one.

The useful reading is the operational detail. How many people were invited and how many finished tells you what recruitment looks like in your setting. A survey feasibility paper in the corpus reports 156 of 215 invited adults completing it — a number worth more to someone planning a service evaluation than any effect size in the same paper.

Pilot: small on purpose, and the p-values are a trap

A pilot is undersized by design. That has a consequence people forget: it is far more likely to miss a real effect than a full trial, and far more likely to produce a large-looking one by chance. Both errors point the same way — do not act on the number.

The warning sign: a significance test in a study too small to run one

When a pilot reports p-values across several outcomes and one of them comes out below .05, that is close to what you would expect from chance alone.

This is also why “positive” means something different on a pilot. Our summaries record the direction a paper’s findings point, and on a pilot that direction describes what happened in twenty people over three weeks. It is a reason to look again, not a reason to change anyone’s programme.

Case report: an existence proof, not a rate

A case report tells you something is possible. It cannot tell you how often, in whom, or better than what — there is no comparison group and no denominator. The effect size is frequently enormous, which is exactly what makes it seductive.

A case series is the same logic with a slightly larger n, and the honest ones show you the variation.

Protocol: a plan, and worth reading anyway

A study protocol contains no results, because the study has not been done. On this site those pages say exactly that and carry no findings section.

They are still worth knowing about for two reasons. A protocol tells you a question is being answered, so you know to look again in two years. And because it is registered before the data exists, it is the document that makes it possible to notice later if the outcomes quietly changed between the plan and the paper.

How to read one of these without misreading it

• Read the design before the result. The design decides which sentences in the abstract can carry weight.
• Treat every effect size as an illustration. In these designs the size of the change tells you almost nothing about the size of the effect.
• Look for what the authors say they will do next. That sentence is usually the real finding, and it is usually honest.
• Ask what it changes on Monday. For most of these the answer is nothing, and that is not a criticism of the paper. It is what the paper is for.

All of these designs rate very low on this site, which is the whole basis of the certainty rating — and very low certainty is not the same as a small effect or a bad paper. If you are weighing whether a change on an outcome measure is real, the companion piece on MCID and MDC covers that. You can also browse every paper we have read by design.