Community Medicine

Evidence-based Medicine: Critical Appraisal on Therapy and Prevention

Cheryl Villamor-Red, MD, CFP
May 15, 2021

OUTLINE Steps in EBM

PRIMER ............................................................................................................ 1 ASK Convert your clinical dilemma from the patient
Objectives ........................................................................................................................ 1 encounter to a focused research question.
Disclaimer ........................................................................................................................ 1 ACQUIRE Search the literature and online databases for
EVIDENCE-BASED MEDICINE ........................................................................ 1 applicable research data.
What is Evidence-Based Medicine? ............................................................................... 1 APPRAISE Critically appraise the evidence gathered with
Examples of Clinical Dilemma in Patient Encounters ................................................... 1 regards to its validity and relevance.
Steps in EBM.................................................................................................................... 1
APPLY Integrate appraisal knowledge about the unique
EBM IN APPLICATION (APPRAISAL)............................................................. 1 aspects of the patient (including the patient’s
Case Scenario ................................................................................................................. 1
Step 1: Ask ....................................................................................................................... 2
preferences about the possible outcomes).
Step 2: Acquire ................................................................................................................ 2 ASSESS Interview and examine the patient.
Step 3: Appraise .............................................................................................................. 3 Table 1. Steps in EBM
Steps 4 & 5: Applicability & Assess................................................................................ 6
Resolution of the Problem Scenario .............................................................................. 6
EBM IN APPLICATION (APPRAISAL)
REFERENCES .................................................................................................. 6
Case Scenario
- S.C.
- 32/F
PRIMER - Single
Objectives - Roman Catholic
1. To outline the steps in performing EBM - Negros Occidental
2. To critically appraise a sample article on Therapeutics & Prevention - HPI
o AM PTC, sudden onset of dry cough and dyspnea aggravated
by dust and smoke exposure. No fever, no chest pain.
Disclaimer - Past Medical History
- EBM Is not a perfect paradigm. o Known Asthmatic since childhood with maintenance of
- It is one paradigm. Salmeterol+Fluticasone (Seretide) diskus 250/25mcg 2 puffs
- The challenge is to balance judicious use of evidence to our clinical twice a day and Montelukast 10mg tab once a day at bedtime
skills and humane attention to patient needs. o Last asthma attack: 3 months PTC
- The slides have been replicated from the lectures done by my ▪ Admitted at private hospital
o Non-Hypertensive
mentors during Family Medicine Residency.
o Non-diabetic
EVIDENCE-BASED MEDICINE AT THE EMERGENCY ROOM:
What is Evidence-Based Medicine? - Personal and Social History
Evidence-based medicine is the integration of best research evidence o works as an office clerk
with clinical expertise and patient values – Dave Sackett o Known allergies: pet danders, cigarette smoke, seafoods, and
crustaceans
o No known drug allergies
- Pertinent PE:
o Conscious, coherent, talks in sentences, Mild respiratory
distress
o BP: 130/70
o RR: 24cpm
o PR: 92bpm
o Temperature: 37.1 degrees Celsius
o O2 Sat: 95%
o C/L: Supraclavicular retractions, equal chest expansion,
wheezing on all lung fields
- Impression: Bronchial Asthma in Acute Exacerbation
- Plans
o Advised admission
o Opted HAMA (No watcher, no immediate family/relative in
Figure 1. EBM Triad CDO)
o Home medications:
Examples of Clinical Dilemma in Patient Encounters ▪ 1. Salmeterol+Fluticasone (Seretide) diskus 250/25mcg
- Your auntie asked if Vitamin B-complex is effective in treating 2 puffs twice a day
▪ 2. Salbutamol+Ipratropium nebulizer every 8 hours as
numbness since it was mention in commercials
needed
- Your yaya who has put on some weight over the summer asked if
▪ 3.Prednisone 20mg tab 1 tab 2x a day for 1 week
drinking anything with L-carnitine would make her lose weight ▪ 4. Montelukast 10mg tab once a day at bedtime
▪ Advised to avoid allergens
▪ Rest for 5-7 days
o Prior to discharge:

1 of 6
 Community Medicine | Evidence-based Medicine: Critical Appraisal on Therapy and Prevention
▪ Patient expresses concern of financial constraints and
would like to go back to work as soon as possible
▪ She admits that she has not been compliant to her
maintenance of Montelukast 10mg tablet.

She asks if taking Montelukast tablet will benefit in her recovery.

Step 1: Ask
- Therapy and Prevention
- Diagnostic Test
- Prognosis
- Harm
- Others: Differential diagnosis, Systematic review or Meta-analysis, Table 3. Selected tools for finding the evidence in EBM.
Clinical practice guidelines
Searching the Literature
Clinical Question www.pubmed.gov (Free)
- Patient problem
o Describe the ones that come into contact with and are relevant
to your practice
- Intervention
o Interventions can be types of dressings, drug therapies,
placebos or counselling. Also they can be about the provision
of differing environmental factors or deal with the way in which
information is given to patients. They can also be diagnostic
tests.
- Comparison Intervention
o This can be a standard therapy, though it may also be no
intervention a placebo or an alternative treatment, exposure Figure 2. Search the literature
or diagnostic test.
- Outcomes Search Terms
o Make a distinction between the outcome which is relevant to - Use keywords only
your patient or problem and the outcome measures deployed - adding “AND” will include searching for titles with both words needed
in the studies. Spend some time working out exactly what via Boolean phrasing.
outcome is important to you, your patient, and the time-frame
which is appropriate.
- Methodology
o Study designs that best answers the questions

Application to Case
- Clinical Dilemma: She asks if taking Montelukast tab will benefit in
her early recovery
o Population: Adult patients
o Intervention: Oral Montelukast
o Comparison: Placebo
o Outcome: Faster recovery in asthma exacerbation
o Methodology: Randomized Controlled Trial

1. Will addition of oral montelukast (I) procide faster recovery in

asthma exacerbation (O) in adult patients (P)?
2. In adult patients (P), will oral montelukast (I) compared to placebo
(C), provide faster recovery in asthma exacerbation (O) in a
randomized control trial (M)?

Types of Study Design according to the Types of Clinical Question

Types of Question Types of Study
Etiology Case-control or cohort study
Diagnosis Diagnostic validation study
Prognosis Inception cohort study
Therapy Randomized controlled trial
Cost-effectiveness Economic Evaluation
Quality of Life Qualitative Study

Step 2: Acquire
- Search the Literature and online databases for applicable research
data

2 of 6
 Community Medicine | Evidence-based Medicine: Critical Appraisal on Therapy and Prevention
Validity Did the study use valid methods to address this
question?
Results - What are the results?
- Are the valid results of this study important?
Applicability - Are these valid, important results applicable to
my patient or population?
- Will the results be helpful to me?
Each component has Questions answerable by: YES, NO, or UNCLEAR.

Relevance
- Appraising directness
- is the objective of the article comparing therapeutic interventions
similar to your clinical dilemma?

Validity
Primary Validity
- Was the assignment of patients to treatment groups randomized?
- Was follow-up rate adequate?
- Were all patients analyzed in the group to which they were originally
randomized?

Secondary Validity
- Were baseline characteristics similar at the start of the trial?
- Were patients blinded to treatment assignment?
- Were clinicians “blind” to treatment?
- Was the study personnel “blind” to treatment?
- Aside from the experimental intervention, were the groups treated
equally?

To Validate:
- Overall, is the study valid?
- YES if there is a yes answer to at least 3 of the validity questions:
o 1 primary validity
o 2 secondary validity

Results
- How large was the treatment effect?
- How precise was the estimate of treatment effect?

Step 3: Appraise Event Rates:

- Computation is necessary in order to determine to magnitude of the
- Critically appraise the evidence gathered with regards to its validity
treatment effect.
and relevance.
- Risk in control (Rc): death in control / # of patients in control
- In reporting an entire journal article, there will be a lot of unanswered
o those negative bad outcomes under the control group.
questions, so the best way to do that is to appraise the article.
o Mostly these will be reported as either death, sepsis,
- Appraise means to critic the article as third person, to point flaws,
respiratory failure, non-responders, or all of the above.
using the 4 components of appraisal (RVRA).
- Risk in treatment (Rt): death in treatment / # of px in treatment
o This is the application of the rules of evidence to a study, to
o Same with Rc but for those in therapy drug
assess the completeness of reporting methods and
- Relative risk (RR): Rt / Rc
procedures, conclusions, compliance with ethical standards.
o the risk of events or bad outcome in the treatment group
o This is also the application principles of analysis to identify
relative to the placebo or control group; measure of
unbiased and valid studies reported in literature.
effectiveness of your treatment
- Through appraisal, we can exclude research studies that are poorly-
- Absolute risk reduction (ARR): Rc – Rt
designed.
o absolute difference between the proportion who died in the
placebo or control group compared to the treatment group.
Critical Appraisal o For the most part, the article may not report about death in both
- Journal appraisal: used to test the effectiveness of the treatment groups, so you will not be able to compute for the ARR.
(whether the medication is beneficial, has no effect or is harmful) - Relative risk reduction (RRR): 1 – RR
- Validity must be measured to ensure experiment is proper and o tells you by how much the treatment reduced the risk of bad
reasonable. outcomes relative to the control group, who did not have the
treatment.
4 Components of Appraisal TOOLS - Number needed to treat (NNT): 100 / ARR
R-V-R-A o number of patients needed to treat to prevent 1 adverse
Relevance - Does this study address a clearly focused outcome; the closer to 1 → more effective, the higher above
question? 1 → less effective
- Does this study answer a particular question o if result is closer to 1 → more effective the treatment
similar to my own? o If result is higher than 1 → less effective the treatment

3 of 6
 Community Medicine | Evidence-based Medicine: Critical Appraisal on Therapy and Prevention
▪ A drop-out rate of 20% or more is usually declared as
Estimation of Treatment Effect: substantial.
- Because a study nearly estimates the true effect of a drug, in order o It means you have to criticize if the drop-out rate is reasonable;
to simplify this, researchers report an interval estimate, which if there are a lot of drop-out rates because of the side-effects
provides a range of possible values of the treatment effect. or death, then we already know that this therapy has a lot of
- 95% confidence interval issues.
- By convention, interval estimates are estimated at a 95% level of
confidence. 3) Were all the patients analyzed in the group which they were
o Thus, when we state 95% confidence interval, it means that we originally randomized?
are 95% sure that the true effect of the treatment lies within - From the article (refer to same table above
this range. - Answer to the question: YES
o So basically, to answer this parameter, we take a look at the o Intention not to treat not stated.
table where you can find the 95% confidence interval. - Intention to treat:
- The closer these values, the more precise your estimate. o Another way of checking whether the follow-up is complete is
- P value </= 0.10 to check whether an intention to treat analysis was mentioned
o If confidence interval is not reported, then check the P value. anywhere in the article.
o Anywhere less than or equal to 0.10, is acceptable. o If they mentioned intention to treat analysis was done, then you
o According to some books for EBM, the P value should be less can safely assume that the follow-up was complete.
than or equal to 0.05 so that it will coincide with 95% o Going through the results in table 2, the baseline parameters
confidence interval. were initially taken from the start to the end of the trial and the
o Some books also state that </= 0.10 is acceptable or 90% can same number of patients were accounted.
also be okay. However, for the most part/most books on EBM, ▪ Since there is no intention to treat statement, it was clear
recommend 95% C.I. and P Value </=0.10. that patients of both groups were actually all accounted
and hence answer for this question is yes.
Application to Given Case
Let’s apply the RVRA approach to the given case. Secondary Validity
1) Were the baseline characteristics similar at the start of the trial?
Relevance - From the article: pg. 4 Table 1
Does the study answer a particular question similar to my own?
- Our Research question: Will addition of Oral Montelukast (I) provide
faster recovery in asthma exacerbation (O) in Adults patients (P)?
- From the article: (p.2 3rd paragraph):
o In our study, we tested the hypothesis that treatment with a
Leukotriene Receptor Antagonist (LTRA), Montelukast sodium
would improve airway obstruction and clinical outcomes in
Acute Asthma Exacerbation and would subsequently
decrease the duration of hospital stay.
- Answer to the question: YES

Validity
Primary Validity
1) Was the assignment of patients to treatment randomized?
- From the article: pg.2 Study Design First Paragraph
o “This was a randomized, double-blind, placebo controlled
parallel group drug trial conducted over a period of two years
from February 2006 to February 2008.”
- Answer to question: YES

2) Was the follow-up rate adequate?

- From the article p.5 table 2:

- Answer to the question: YES

o Because of randomization, the baseline characteristics of the
treatment groups tend to be very similar.
o Sometimes, there is inequality between treatment group and
the placebo group
▪ this may be due to chance alone or other factors.
o By checking the baseline characteristics, this checks whether
the randomization is successful.
- Answer to the question: YES o You have to make sure that
o For numbers 2 and 3, this is best checked by looking at the ▪ (1) there is randomization
number of patients enrolled at the outset and comparing this ▪ (2) the characteristics during randomization has to be at
to the number of patients reported in the results table. least similar between placebo and treatment group (age
o Every patient who entered the trial should be accounted for, at groups). (see table for example)
an institution.
o If substantial numbers are lost to follow-up, the validity of the 2) Were the patient’s blinded to treatment assignment?
conclusion are open to question. - From the article: page 3 1st paragraph

4 of 6
 Community Medicine | Evidence-based Medicine: Critical Appraisal on Therapy and Prevention
o “AKUH pharmacy was the key person to maintain o There should be no bias; both groups should be treated
randomization and blinding and was the only one to know equally.
the treatment status of the patient. Allocation numbers were
generated and assigned to each patient found eligible to be Overall Validity
enrolled in the study. Patients were distributed to each arm - Overall, is the study valid?
based on the allocated code. Neither the evaluators, nor the
on-call admitting team were made aware of the actual
treatment allocations.”
- Answer to question: YES
o According to the statement (from the article), both the doctor
and the participants were blinded of the treatment so the
answer would be yes.
o Blinding is very important.
o Blinding is the process by which the intervention being given is
concealed from the patient, clinicians, and the one who
analyzes the data.
o Patients, clinicians, and data-analysts are likely to have an
opinion regarding the experimental treatment, and this
opinion can distort reporting of treatment outcomes.
▪ to avoid this bias, blinding is very necessary.

3) Were the clinicians blind to the “treatment”?

- From the article: pg. 3 1st paragraph
o “Neither the evaluators, nor the on-call admitting team were - Most of our questions were answered by yes.
made aware of the actual treatment allocations.” - Note that you have to have at least:
- Answer to the question: YES o 1 primary validity +
o 2 secondary validity.
4) Was the study personnel “blind” to the treatment? - For this article, there are a lot of “yes”, it has already met the criteria,
- Randomization and Blinding so this article is valid.
o The AKUH pharmacy played a pivotal role in the randomization
of patients. The trial coordinator at the AKUH pharmacy was Results
the key person to maintain randomization and blinding and 1) How large was the treatment effect?
was the only one to know the treatment status of the patient. o For the appraisal of results, computations will be done.
- Answer to the question: UNCLEAR / NOT STATED o We have to compute for the treatment outcomes; these are
o Going over the article, you did not come across the statement what we call the measures of effect.
where the data analysts were blinded. ▪ Remember the following event rates and the
corresponding formula.
5) Aside from the experimental intervention, were the groups treated o In our article, the treatment is montelukast; we’ll use these
equally? parameters to measure the effectiveness of montelukast.
- From the article page 3 Figure 2; Intervention first paragraph - To start with our computation:
o Intervention: “All patients with acute asthma underwent o According to our article, 1 patient from each group (1 from
clinical assessment for severity of attack and received placebo group, 1 from treatment group) developed
standard therapy with oxygen and inhaled respiratory failure (this means the bad outcome)
bronchodilators via jet nebulizer with Salbutamol 2.5mg o there were no recorded deaths from each group.
and Ipratropium bromide 500mcg mixed with 2cc of o There 50 participants in the montelukast (treatment) group
Normal Saline every 15-30 minutes. The duration of the high and 50 participants in the placebo. So,
dose bronchodilator therapy was variable (1-4 hours) and
subsequently tapered to every 6 hours depending on the Rt: 1/50 = 0.02
symptomatic response to therapy. A dose of systemic Rc:1/50=0.02
corticosteroids in the form of hydrocortisone 200mg IV Then proceed to RR, which is Rt/Rc = 0.02/0.02 = 1
was administered in the ER followed by 100mg every 6 For the RRR, which is 1-RR = 0
hours. The steroids were subsequently changed to oral For NNT, which is 100/ARR = 100/0 = 0
prednisolone 0.5mg/kg/day for 7 days. The trial design *ARR = Rc-Rt = 0.02-0.02 = 0
followed in the ER is shown in Figure 2.”

- To interpret computations, use the table on Interpretation of

- Answer to the question: YES
treatment to determine if beneficial, no effect or harmful.
o If there are other interventions, aside from the one in the study
o According to our computation, treatment has no effect (for RR,
we call it as co-interventions.
ARR, RRR)
o In this case, all of the patients were given nebulization and
o Thus, answer to question is NO EFFECT
corticosteroids, regardless being in the treatment or placebo
group → this is an example of co-intervention.
5 of 6
 Community Medicine | Evidence-based Medicine: Critical Appraisal on Therapy and Prevention

2) How precise was the estimate of treatment effect?

- To answer the question: NO EFFECT

Steps 4 & 5: Applicability & Assess

Step 4: Applicability
- Can the results be applied to my patient care?
- Were all clinically important outcomes considered?
- Are the likely treatment worth the potential harm and costs?

Step 5: Assess
Resolution of the problem in the Scenario.

Application to the Given Case

Applicability
- Can the results be applied to my patient care? → YES
- Were all clinically important outcomes considered? → YES
- Is the likely treatment worth the potential harm and costs?
o NO, treatment has no effect

Resolution of the Problem Scenario

- Given that we have gone through all the steps in how to apply
evidence-based medicine, we can now resolve the case problem
given.
- Resolution: Addition of Oral Montelukast to standard
management of Acute Asthma exacerbation has no beneficial
effect that will accelerate recovery.

REFERENCES
- Dans, A. L, Dans, L. F., & Silvestre, M.A. (2008). Painless Evidence-
based Medicine. West SussesPO109 8SQ, England: John Wiley &
Sons, Ltd.
- Espallardo, N.L. (2008). Evidence Based Medicine. Self-Instruction
Manual. Department of Clinical Epidemiology (p.102). UP College of
Medicine and Philippine General Hospital
- Leopando, Z.E., Bienvenido Alip, Jr, A.J., La-Rose-Fernandez, T.,
Lazaro-Hipol, C., Yu-Maglonzo, E., Olazo, R.A., Serrano-Tinio, C.
(2016). Textbook of Family Medicine Volume 1: Principles,
Concepts, Practice, and Concept. Quezon City: C&E Publishing, Inc.

6 of 6