Critically Appraising Qualitative and Mixed Methods Evidence
The narration between entries paraphrases; it never reads an entry aloud before the entry plays. Every entry in this lecture plays exactly as it does on its own page — same file, same speed, nothing laid over it.
What you can do after this
- Apply the trustworthiness criteria to a qualitative study
- Say why an absent detail is not automatically a methodological failure
- Say what metasynthesis provides that individual studies do not
- Tell mixed methods research from running two studies side by side
- Answer the three misconceptions that keep qualitative evidence out of decisions
What it covers
27 entries in 6 topics. Each number links to that entry on its own.
-
1. The appraisal questions
-
2. What you can reasonably expect
-
3. Metasynthesis
-
4. Mixed methods
-
5. The three canonical errors
-
6. A study, appraised
Check yourself
What does it mean that analysis procedures must be described rather than asserted?
Saying that themes emerged from the data describes nothing — it names an outcome, not a process. An appraisable study says how coding was done, how categories were derived, and how disagreement was handled, so a reader can judge the route from data to interpretation.
Why is a missing detail not automatically a methodological failure?
Because journals impose word limits, and qualitative reports lose method detail first. The question is whether what is absent matters for the judgement you are making, not whether the paper satisfied every criterion.
What does metasynthesis provide?
An interpretation across studies that none of them reached alone. It synthesises findings rather than pooling numbers, so the product is a new reading rather than a summary statistic.
How do you tell mixed methods from two studies in one paper?
By whether the strands are integrated by design. In genuine mixed methods one strand scaffolds the other, or the two run in parallel and are assimilated into a single account. A survey and some interviews reported side by side is multiple methods, not mixed methods.
How would you answer someone who says qualitative evidence is weak?
That it is weak only against criteria built for a different question. Hierarchies are question-specific, and for a meaning question qualitative work is not the weakest evidence available — it is the only evidence there is. Small samples do not invalidate it, because saturation rather than size is the criterion.
Lecture transcript
The narration as recorded, in order. Where an entry plays, the transcript links to it rather than reprinting it — the entry plays from its own file, unaltered, and says the same thing on its own page.
Part A gave you the traditions. This part is what you actually do with a qualitative paper in front of you — the criteria you apply, and then mixed methods. One criterion in this list has no counterpart anywhere else in the course: whether the account produces recognition in a reader who already knows the situation. That is a real appraisal question here, and it would be meaningless applied to a trial. By the end you can apply the trustworthiness criteria; say why a missing detail may be a word limit rather than a failure; say what metasynthesis provides; tell genuine mixed methods from two studies run side by side; and answer the three misconceptions that keep this evidence out of clinical decisions.
Eleven videos, one per criterion. Watch the ninth — whether the account produces recognition in a reader who knows the situation. There is no quantitative equivalent of that criterion anywhere in this course.
The question above the rest.
EBP 0931 Are the Results Trustworthy and Credible?
The overarching appraisal question, which the ones that follow break down.
The first specific question.
EBP 0932 How Participants Were Chosen
Were participants chosen in a way that fits the phenomenon — people who lived it, rather than a random draw.
The second.
EBP 0933 Is the Phenomenon Plainly Identified?
Whether the study says plainly what it is studying. Vagueness here makes everything after it unappraisable.
The third.
EBP 0934 Does the Approach Fit the Purpose?
Whether the tradition chosen suits the question asked. Phenomenology on a question about process is a mismatch.
The fourth.
EBP 0935 Are Data Collection Procedures Clear?
Whether you can follow how the data were gathered well enough to judge them.
The fifth.
EBP 0936 Are Analysis Procedures Described?
Whether the analytic process is described rather than asserted. Themes emerged is not a description of analysis.
The sixth.
EBP 0937 Are the Findings Well Presented?
Whether findings are presented so a reader can see the relationship between data and interpretation.
The seventh.
EBP 0938 Are Meanings Described in Context?
Whether meanings are given in context rather than as quotations floating free of the situation that produced them.
A criterion with no quantitative equivalent.
EBP 0939 The Effect on the Reader
Whether the account produces recognition in a reader who knows the situation. This has no quantitative counterpart.
The eighth.
EBP 0940 Are the Implications Stated?
Whether the study says what follows from it for practice.
The last, and the familiar one.
EBP 0941 Will the Results Help My Patients?
The applicability question, unchanged from every other appraisal in this course.
Two videos on realistic expectations. An absent detail may be a word limit rather than a methodological failure.
A realistic expectation.
EBP 0942 No Study Contains Everything
No study satisfies every criterion. The judgement is whether what is missing matters here.
A structural reason for gaps.
EBP 0943 Journals Constrain Reports
Word limits force omissions, so an absent detail may be a publishing constraint rather than a methodological failure.
Two videos on synthesis across qualitative studies, which is interpretation rather than pooling.
The qualitative counterpart to synthesis.
EBP 0944 Metasynthesis
Synthesising findings across qualitative studies, which is not the same as pooling numbers.
What it gives you.
EBP 0945 What Metasynthesis Provides
An interpretation across studies that none of them reached alone.
Eight videos. The last one is the practical test: running a survey and some interviews is not mixed methods unless the strands are integrated by design.
The third kind of study.
EBP 0946 Mixed Methods Research
Research combining qualitative and quantitative work within a single design.
The rule that governs it.
EBP 0947 Match the Method to the Research Question
The method follows the question, which is why a mixed question needs a mixed design.
The word for its parts.
EBP 0948 Strands
The qualitative and quantitative components of a mixed methods study.
How the strands support each other.
EBP 0949 Scaffolding
One strand building on the other, so that the second is shaped by what the first found.
A named arrangement.
EBP 0950 Convergent Parallel
Both strands run at once and are brought together at the end.
How they are combined.
EBP 0951 Data Assimilation
The work of bringing the two kinds of finding into one account.
Questions that arise mid-study.
EBP 0952 Emergent Questions
Questions arising during the study that the design allows you to pursue.
A distinction worth holding.
EBP 0953 Mixed Methods Is Not Multiple Methods
Running a survey and some interviews is not mixed methods. The strands have to be integrated by design.
Three videos, and these three are why qualitative evidence stays out of clinical decisions. Each is answered by something you now know.
The first, and the most consequential.
EBP 0954 Misconception: Qualitative Evidence Is Weak Evidence
It is weak only against criteria built for a different question. On its own question it is the only evidence there is.
The second.
EBP 0955 Misconception: Small Samples Invalidate It
The study was never drawing a sample to generalise from. Saturation is the criterion, not size.
The third, and the subtlest.
EBP 0956 Misconception: Researcher Influence Is Contamination
The researcher's presence is acknowledged and examined rather than scrubbed out. That is what bracketing is, and it is the opposite of pretending to have no assumptions.
One procedure taking a qualitative study through the trustworthiness criteria to a judgement.
The unit assembled.
EBP 0957 Appraising a Qualitative Study
One qualitative study taken through the trustworthiness criteria to a judgement about whether it informs this decision.
That completes unit 7. Against the outcomes. The trustworthiness criteria ask how participants were chosen, whether the phenomenon is plainly identified, whether the approach fits the purpose, and whether collection and analysis are described rather than asserted. Themes emerged is not a description of analysis. No study contains everything, and journals impose word limits, so an absent detail may be a publishing constraint. The judgement is whether what is missing matters for your decision. Metasynthesis produces an interpretation across studies that none of them reached alone. It is interpretation rather than the pooling of numbers. Mixed methods integrates its strands by design. A survey plus some interviews, reported separately, is two studies in one paper. And the three misconceptions. It is weak evidence only against criteria built for another question. Small samples do not invalidate it, because saturation rather than size is the criterion. And researcher influence is acknowledged and examined rather than scrubbed out — which is what bracketing means. Unit 8 is the last of Part 1: appraising clinical practice guidelines.
This is free, and stays free. If it helped, here is what keeps it running.