The Role of Evidence-Based Quality Improvement

Unit 9 established that implementation is its own activity with its own skills. This unit is the machinery: how you actually change something locally, and how you know whether it worked. One idea runs through all of it. A measure that moved is not proof that your change moved it — seasonal variation, case mix and the sheer attention that any new project attracts all move measures on their own. Everything here about run charts and balancing measures exists to tell those apart from a real improvement. By the end you can tell structure, process and outcome measures apart and say what a balancing measure is for; state the Model for Improvement's three questions; say why a run chart beats a before-and-after pair; say why QI results not generalising is a feature; and say what decides whether a project is QI or research.

Five videos. The distinction that matters is placement: improvement work that consults the literature afterwards is quality improvement, and work built on appraised evidence is EBQI.

The activity, defined.

EBP 1351 Quality Improvement

Systematic work to improve local processes and outcomes.

The combination this unit is about.

EBP 1352 Evidence-Based Quality Improvement

Quality improvement with appraised external evidence inside the plan.

A distinction worth making early.

EBP 1353 EBQI Is Not Outcomes Research

EBQI improves this setting. Outcomes research studies outcomes to produce generalisable knowledge.

What it produces.

EBP 1354 EBQI Produces Internal Evidence

The data an EBQI project generates are internal evidence, which is one of the three sources of a decision.

How the loop closes.

EBP 1355 Internal Evidence Raises Clinical Questions

Those data raise the next PICOT question, which is how the cycle continues rather than ending.

Six videos on Donabedian's categories. Watch the balancing measure — omitting it is how a project reports success while harming something nobody was watching.

The framework everything hangs on.

EBP 1356 Donabedian's Three Categories

Structure, process and outcome. Three kinds of measure, and confusing them is the commonest error in improvement work.

The word, defined.

EBP 1357 What an Outcome Is

The result for the patient, which is what the work is ultimately for.

What can and cannot be moved.

EBP 1358 Modifiable and Nonmodifiable Factors

Some factors affecting an outcome can be changed and some cannot. Only the first are targets.

The first kind.

EBP 1359 Structure Measures

What exists — staffing, equipment, systems. The easiest to measure and the furthest from the patient.

The second.

EBP 1360 Process Measures

What was done — whether the intended action actually happened. This is adherence rather than effect.

The third, and the one people omit.

EBP 1361 Balancing Measures

What might have got worse while the target got better. Omitting it is how a project reports success while harming something unwatched.

Five videos. Benchmarking answers against what, and value-based purchasing answers why an institution pays for any of this.

The comparison.

EBP 1362 Benchmarking

Comparing against a standard or against comparable units, because a number alone says little.

Why any of this is funded.

EBP 1363 Value-Based Purchasing

Payment tied to quality measures, which is why improvement work has an institutional sponsor at all.

How that payment is scored.

EBP 1364 Four Domains, Equally Weighted

The scheme weights four domains equally, so no single measure dominates the result.

What makes a measure usable.

EBP 1365 What a Good Measure Requires

It has to be defined precisely enough that two people counting get the same number.

And who has to use it.

EBP 1366 Data People Can Actually Use

Data returned to the people doing the work, in a form they can act on during a shift.

Five videos on outcomes management and improvement science. Rapid-cycle testing is the idea to hold: a failure should cost one shift, not one quarter.

The apparatus, in four phases.

EBP 1367 Outcomes Management, in Four Phases

Define, identify, compare, analyse. The same sequence unit 1 introduced, now with the machinery attached.

The instruments it uses.

EBP 1368 Structured Care Methods

Pathways, protocols and order sets that make the intended practice the default one.

The field, named.

EBP 1369 The Science of Improvement

A body of method for improving systems, with its own techniques and its own literature.

Its defining technique.

EBP 1370 Rapid-Cycle Testing

Small changes tested quickly, so a failure costs one shift rather than one quarter.

The model most used.

EBP 1371 The Model for Improvement

Three questions and a testing cycle. The next four videos are the questions.

Five videos. The second question is the hardest and the one projects skip — the measures are chosen before the change, and that is where the balancing measure belongs.

The first question.

EBP 1372 Question One — What Are We Trying to Accomplish?

The aim, stated so specifically that you could tell whether you reached it.

The second, and the hardest.

EBP 1373 Question Two — How Will We Know a Change Is an Improvement?

The measures, chosen before the change. This is where the balancing measure belongs.

The third.

EBP 1374 Question Three — What Change Can We Make?

The change itself, drawn from the appraised evidence and from what this setting can deliver.

The cycle itself.

EBP 1375 Plan-Do-Study-Act

Plan the test, do it small, study what happened, act on what you learned. Then repeat.

The artefact the first question produces.

EBP 1376 The Aim Statement

A written aim with a number and a date, which is what stops a project drifting.

Nine videos. The models differ in emphasis and share their tools, so the tools matter more than choosing between them. The run chart is the one to remember.

Two more models.

EBP 1377 Six Sigma and DMAIC

A method aimed at reducing variation, with a five-phase cycle.

And a third.

EBP 1378 Lean

A method aimed at removing waste from a process.

The point that matters more than the models.

EBP 1379 The Models Share Their Tools

The models differ in emphasis and share their tools. The next six videos are those tools.

The first tool.

EBP 1380 The Process Map

The steps as they actually happen, which is reliably different from how anyone describes them.

The second.

EBP 1381 The Cause-and-Effect Diagram

Sorting possible causes into categories, so the team argues about evidence rather than about instinct.

The third.

EBP 1382 The Pareto Chart

Ranking causes by frequency, so effort goes where the volume is.

The fourth, and the one to remember.

EBP 1383 The Run Chart

Data plotted over time, which is the only way to tell a trend from ordinary variation. Two points cannot do it.

The fifth.

EBP 1384 Dashboards and Scorecards

Standing displays of the measures that matter, kept current.

The sixth.

EBP 1385 Daily Huddles

Short daily conversations where the data are looked at by the people who can act on them.

Six videos. QI results do not generalise, and that is a feature rather than a limitation — improving this setting was the objective.

Now the statistics of a project.

EBP 1386 Effect Size in a QI Project

How large a change the project produced, which is a different question from whether it is statistically detectable.

Why small projects find nothing.

EBP 1387 Power and the Size of the Project

A small project may be unable to detect a real change, so absence of a finding is not absence of an effect.

The limitation, stated plainly.

EBP 1388 QI Results Do Not Generalise

What worked here may not work elsewhere, because the setting was part of the intervention.

And why that is a feature.

EBP 1389 Which Makes the Setting the Point

That is not a weakness of the method. Improving this setting was the objective.

The question that decides the obligations.

EBP 1390 Is It QI or Is It Research?

The distinction turns on intent: local improvement, or generalisable knowledge.

And the answer that matters.

EBP 1391 The Ethics Do Not Depend on the Label

Whatever the label, people are affected and the obligations to them are the same.

Three videos. The third is the seductive one, because a measure that moves feels like proof.

The first error.

EBP 1392 Misconception: QI and Research Are the Same Work

They differ in intent, method and what the results may claim. Treating them as one produces projects that overreach.

The second.

EBP 1393 Misconception: QI Needs No Evidence

Improvement without appraised evidence rediscovers what the literature already settled, slowly and locally.

The third, and the seductive one.

EBP 1394 Misconception: The Measure Moved, So It Worked

Seasonal variation, case mix and the attention any new project attracts all move measures. That is what the run chart and the balancing measure are for.

One procedure taking a project from aim statement to sustained change.

The unit assembled.

EBP 1395 Running an Evidence-Based Quality Improvement Project

One project from aim statement through PDSA cycles to a sustained change, with balancing measures throughout.

That is evidence-based quality improvement. Against the outcomes. Three kinds of measure. Structure is what exists, process is whether the intended action happened, outcome is the result for the patient. And the balancing measure is what might have got worse — omitting it is how a project reports success while harming something nobody watched. Three questions: what are we trying to accomplish, how will we know a change is an improvement, and what change can we make. The second is hardest, because the measures are chosen before the change rather than after the results arrive. A run chart rather than a before-and-after pair, because two points cannot separate a trend from ordinary variation. Most units' rates swing on their own, and a project that measures twice will find whatever the swing was doing. QI results do not generalise, and that is a feature. The setting was part of the intervention, and improving this setting was the objective. A project that claimed to generalise would have been research done badly. Intent decides whether it is QI or research: local improvement, or generalisable knowledge. And the ethics do not depend on the label, because people are affected either way and the obligations to them are identical. Unit 11 turns to competence itself, and to what it means to claim someone can do this.