Critically Appraising Quantitative Evidence

Part A was levels and bias in the abstract. This part is the designs themselves — trials and systematic reviews at the top of the hierarchy, then the observational designs, each with the questions you ask of it. Two things carry the lecture. Randomisation distributes the factors nobody thought of, which is exactly why it licenses a causal claim and why nothing else does. And a systematic review is defined by its method rather than by its title — a paper calling itself systematic may not be one. By the end you can say what random assignment does that nothing else does; name the four requirements that make a review systematic; rank a review of trials against a mixed review and a narrative review; ask the three trial-appraisal questions; and say what randomisation does not fix.

Two videos on the design that sits at the top. What matters is the mechanism: randomisation distributes the factors nobody thought of.

The strongest single design.

EBP 0786 True Experimental Designs

A design with an intervention and random allocation. This is the one that supports a causal claim.

What randomisation actually does.

EBP 0787 What Random Assignment Does

It distributes the factors you did not think of across both groups, which is why it handles unknown confounders.

Six videos on the four requirements. The one that does the most work is prespecification: criteria set before the studies are seen cannot be shaped by the results.

The definition, precisely.

EBP 0788 What a Systematic Review Is

A review that answers one question against the whole body, by a stated method someone else could repeat.

The first requirement.

EBP 0789 Prespecified Eligibility Criteria

The criteria for including a study are set before the studies are seen, so they cannot be shaped by the results.

The second.

EBP 0790 An Explicit, Reproducible Method

The method is written out so another team could repeat it and reach the same set.

The third.

EBP 0791 A Search That Attempts Everything

The search tries to find everything, including work that was never published, because missing studies bias the answer.

The fourth.

EBP 0792 Risk of Bias Assessment

Every included study is assessed for bias, so the review weighs rather than merely collects.

A useful reframing.

EBP 0793 The Sample in a Review Is Studies

In a systematic review the unit of analysis is the study, not the patient. That is why review methods look like study methods.

Six videos on where different reviews sit, ending in the warning that a paper calling itself systematic may not be.

Where a review of trials sits.

EBP 0794 A Review of Trials Is Level I

A systematic review of randomised trials is the highest level for an intervention question.

What a mixed body costs.

EBP 0795 Mixed Reviews Rank Lower

A review that includes non-randomised studies cannot rank as high, because its inputs admit more bias.

Where narrative reviews sit.

EBP 0796 Narrative and Literature Reviews Are Level VII

A narrative review is an expert writing about a literature, and it sits at the bottom of the hierarchy.

The warning that follows.

EBP 0797 Labelled Systematic Is Not Systematic

A paper calling itself a systematic review may not be one. Check for prespecified criteria and a reproducible method.

A specific technique.

EBP 0798 Meta-Analysis

Statistically combining results across studies to produce a single pooled estimate.

A broader form.

EBP 0799 Integrative Review

A review combining diverse designs, including qualitative work, to address a broader question.

Three videos, three questions. The second is the one that catches most trials: people who vanish from a study take the result with them.

Appraising a trial, question one.

EBP 0800 Appraising a Trial: Was Randomisation Real?

Whether allocation was genuinely random and concealed, because a predictable sequence is not randomisation.

Question two.

EBP 0801 Appraising a Trial: Were All Accounted For?

Whether every participant randomised is accounted for at the end. People who vanish take the result with them.

Question three.

EBP 0802 Appraising a Trial: Equal Treatment

Whether the groups were treated the same apart from the intervention. Any other difference is a rival explanation.

Seven videos on cohorts, case-control, case reports, and improvement projects — each with what to ask of it.

The next design.

EBP 0803 Cohort Studies

Groups followed forward over time to see what happens. Good for prognosis questions.

How to appraise one.

EBP 0804 Appraising a Cohort Study

Ask whether the groups were comparable at the start and whether follow-up was long enough and complete enough.

The next design.

EBP 0805 Case-Control Studies

People with an outcome compared to people without, looking backward for exposures.

How to appraise one.

EBP 0806 Appraising a Case-Control Study

Ask how cases and controls were selected, and whether exposure was ascertained the same way for both.

The smallest designs.

EBP 0807 Case Studies and Case Series

One patient, or a handful, described in detail. No comparison group at all.

What they are actually for.

EBP 0808 What Case Reports Are For

They alert, describe the rare, and generate hypotheses. They do not establish that anything works.

Appraising a project.

EBP 0809 Appraising Implementation and QI Projects

Improvement projects are appraised for what they claim locally, rather than judged as failed research.

Three videos. The second is the most seductive, because randomisation genuinely does solve one problem and people extend it to all of them.

The first error.

EBP 0810 Misconception: A Level Describes a Study

A level describes a design. Two studies at the same level can differ enormously in quality.

The second.

EBP 0811 Misconception: Randomisation Fixes Everything

Randomisation handles allocation. It does not fix attrition, unequal treatment, biased measurement or selective reporting.

The third.

EBP 0812 Misconception: Significant Means Important

Statistical significance says a result is unlikely to be chance, not that it is large enough to matter.

One procedure taking a quantitative study from the three questions through its design-specific checks.

The strand assembled.

EBP 0813 Appraising a Quantitative Study

One quantitative study taken through the three questions and its design-specific checks to a judgement.

That is the designs. Against the outcomes. Random assignment distributes the factors you did not think of across both groups. That is the whole mechanism, and it is why randomisation handles unknown confounders when nothing else can. A systematic review is made systematic by four things: prespecified eligibility criteria, an explicit reproducible method, a search that attempts everything including the unpublished, and a risk-of-bias assessment of every included study. A review of randomised trials is level one for an intervention question. A mixed review ranks lower because its inputs admit more bias. A narrative review is level seven — an expert writing about a literature. Three questions of a trial: was randomisation real and concealed, is every participant accounted for at the end, and were the groups treated the same apart from the intervention. And randomisation fixes allocation only. It does not fix attrition, unequal treatment between groups, biased measurement, or selective reporting — which is why the three trial questions exist at all. Part C goes underneath the results, to the numbers themselves.