October 21, 2008
Course Overview:
Medical Information
for Decision Making
HuBio 590, Fall 2008
Peter Tarczy-Hornoch MD
Head and Professor, Division of Biomedical & Health Informatics
Professor, Division of Neonatology
Adjunct Professor, Computer Science and Engineering
Course Objectives
1. Describe the value of high quality medical information
for clinical care
2. Describe the range of factors that influence the clinical
decision making process
3. Translate a clinical scenario into a searchable question
4. Describe advantages and limitations of various medical
information resources and types of documents
5. Find documents from one or more medical information
resource(s) that may address the clinical situation
6. Assess systematically the relevance and validity of a
given document with respect to the clinical situation
7. Compare relevance and validity across two documents
Course Overview
 Session 1, Tuesday October 21
 “Medical Information and Medical Decision Making” &
“Statistics 101” (1:30-2:30; Learning Objectives 1,2,3)
 Small Group: Introductions (2:40-3:20)
 Session 2, Wednesday October 22
 Small Group: Translating a clinical question into a searchable
one (2:00-2:50; Learning Objective 3)
 “Finding Medical Information in a Clinical Context” (33:50; Learning Objectives 4,5; L. St. Anna)
 Session 3, Monday October 27
 Small Group: Hands on session searching on-line databases
(2:00-2:50; Learning Objectives 4,5; Librarians join groups)
 “Assessing a Document on Treatment” (3:00-3:50;
Learning Objective 6 with focus on treatment)
 Session 4, Wednesday October 29
 Small Group: Practice assessing document on treatment
Course Overview
 Session 4, Wednesday October 29
 Small Group: Review of sample problems and discussion of
real world examples of interpreting articles on treatment
(2:00-2:50; Learning Objective 6 with focus on treatment)
 “Assessing a Document on Diagnosis”(3:00-3:50; Learning
Objective 6 with focus on diagnosis)
 Session 5, Monday November 3
 Small Group: Review of sample problems and discussion of
real world examples of interpreting articles on diagnosis
(2:00-2:50; Learning Objective 6 with focus on diagnosis)
 “Assessing Multiple Studies” (3:00-3:50; Learning
Objectives 6,7 with focus on systematic reviews)
 Session 6, Wednesday November 5
 Small Group: Review of sample problems (2:00-2:50)
 “Applying MIDM Concepts in the Real World” (3:00-3:50)
Course Logistics
 Lectures: T-435
Lectures cover key content in the syllabus (see
courses.washington.edu/midm “Syllabus” for content
outline for each session). Assignments permit practice
of material presented prior to small group session.
 Small Groups:
T538, T540, T543, T546, T548, T547, T549
Focus on application and discussion of lecture material
See web page “Small Group Assignments” (note user
name/password information e-mailed earlier)
 Office Hours
Peter Tarczy-Hornoch, I264A
By arrangement – e-mail [email protected]
Grading & Class Attendance
 To pass the course the following is required:
Attendance at all six small group sessions
If you miss a small group session then a makeup assignment
will be required.
Assignments will be given after Lectures 1-5 to prepare
for small group sessions but these assignments will not
be graded
Passing the final exam
70% is a pass
 Multiple choice and fill in the blanks
 Take-home, open book, web administered
 Available at 5P Wed Nov 5th; due 5P Wed Nov 12th
 The complete syllabus (PDF) contains the
following sections:
Course Description
WWAMI Course Chairs
Seattle Course Chair and Small Group Leads
Learning Objectives
Course Organization
Grading and Class Attendance
Required Textbook/Readings
Schedule for 2008-9
Content outline for each of the six sessions
October 21, 2008
Session 1a: Medical Information and
Medical Decision Making
Peter Tarczy-Hornoch MD
Head and Professor, Division of BHI
Professor, Division of Neonatology
Adjunct Professor, Computer Science and Engineering
Medical Information
and Medical Decision Making
 Medical Decision Making
 Nature of Medical Information
 Reducing Errors & Improving Quality
 Finding Knowledge and Evidence
Medical Decision Making Requires
Integrating Information
Diagnostic Testing
(What is it?)
(Session 4)
Patient Data &
(What do I do for it?)
(Session 3)
Case specific
decision making
General Information & Knowledge (MIDM – finding/assessing)
Patient Information Comes
From Diverse Sources
System (drugs)
System (X-rays)
(test results)
Billing System
- Stay / Visit / Cost
- Diagnoses / Treatments
Past Visit Info:
- Paper chart(s)
- History
- Physical
- Family history
- Problem lists
- etc.
Current Visit Info:
- Symptoms
- History
- Findings
Medical Knowledge Acquired
From Diverse Sources
CD ROM books,
CME, etc.
Medical Decision Making Requires
Integrating Information
 Patient Specific Knowledge:
6 year old boy
History of chicken pox exposure
Currently on steroids for asthma
Exam showing “dewdrop on a rose petal”
 General Medical Knowledge:
Diagnosis and management of chicken pox
Management of asthma
Risk of steroids and chicken pox
 Therapy Decision: Stop steroids, treat with Acyclovir
Clinical Encounters Generate Questions
 “…conservative to conclude that every interaction
between a patient and a doctor is likely on average
to generate at least one question” R. Smith, BMJ, 1996
 Types of information needed (data to knowledge)
Patient specific (laboratory, radiology, immunization)
Guidelines, policies, standards (national or local)
Drug information (dosage, interactions, side effects)
Medical literature (textbooks, reference books, journals)
 Information needed in the context of a specific
encounter (e.g. immunization)
 Known/unknown & met/unmet information needs
Medical Information
and Medical Decision Making
 Medical Decision Making
 Nature of Medical Information
 Reducing Errors & Improving Quality
 Finding Knowledge and Evidence
Managing Medical Information a
Longstanding Challenge
 “The Art is long, life is short, opportunity fleeting,
experience delusive, judgment difficult”
Hippocrates ~400 B.C.
 “While the continuing gains in medical knowledge
and the accompanying ability of doctors to treat the
sick have been real, the passage of time has too often
proven the espoused remedies of one era to be of
limited value or frankly harmful in the next…How
much of what we embrace as truth today will suffer
this fate over the ensuing decades”
LC Epstein 1997
“the Art is long”: Information Overload
 “if the most conscientious physician were to attempt to keep up
with the literature by reading two articles per day in one year
this individual would be 800 years behind” O. Barnett, 1990
 2/3 of primary care practitioners surveyed: “the current volume
of scientific literature is unmanageable” J.W.Williamson, 1989
 “Although well over 1 million clinical trials have been
conducted, hundreds of thousands remain unpublished or are
hard to find and may be in various languages. In the unlikely
event that the physician finds all the relevant trials of a
treatment, these are rarely accompanied by any comprehensive
systematic review attempting to assess and make sense of the
evidence” Bero & Rennie, JAMA 1995
“experience delusive”: unproven treatments
 Best evidence may not be as good as you’d wish
 n=2500 treatments in BMJ Clinical Evidence
See http://www.clinicalevidence.com/ceweb/about/knowledge.jsp
“experience delusive”: data vs. opinion
“Types” of medical knowledge
 Compiled formal/scientific knowledge (best)
E.g. systematic reviews, evidence based medicine,
some Up To Date entries
 Uncompiled formal/scientific knowledge (good)
E.g. “raw” PubMed search, a single randomized trial
 Compiled informal/experiential knowledge (ok)
E.g. consensus statements, “standard of care”, books,
“Spiral” manuals, some Up To Date entries
 Uncompiled informal/experiential knowledge (ok)
 E.g. opinions/customs of experts and consultants
“judgment difficult”
Uncertainty Impacts Decision Making
 Diagnostic uncertainty
Horses vs. Zebras (infection vs. genetic problem)
Availability bias (last patient I saw with this had X)
 Therapeutic uncertainty
Attributes of patient vs. study population
Study drug vs. class of drugs
Population vs. individual (genes + drugs)
 “Islands of certainty in seas of uncertainty”
Studies vs. experience/judgment
“judgment difficult”
Biases Impact Decision Making
 Bias: 2a. A preference or an inclination, especially
one that inhibits impartial judgment. 3. A statistical
sampling or testing error caused by systematically
favoring some outcomes over others. (American
Heritage Dictionary)
 Recall bias: inaccurate recollection of information
 Availability bias: recent or memorable
information/decisions easier to remember
 Sampling bias: personal experience around
information/decision making not representative
 Publication bias: “negative” studies hard to publish
Biomedical Informatics Studies the Use
and Nature of Medical Information
 Doctors use medical knowledge for clinical problem
solving and decision making
 Biomedical informatics focuses on the general issues of
biomedical knowledge capture, retrieval, and application
 “the scientific field that deals with biomedical
information, data, and knowledge – their storage, retrieval,
and optimal use for problem solving and decision making”
Shortliffe, E.H., 2006
 Important discipline in the context of medical information
for decision making
Division of Biomedical and Health Informatics at UW
“Just in time information”: unmet need
 “At the bedside or in the office, physicians should
have instantaneous, up-to-date assistance from an
affordable, universally available database of
systematic reviews of the best evidence from
clinical trials” Bero and Rennie, JAMA, 1995
 “New information tools are needed: they are likely
to be electronic, portable, fast, easy to use,
connected to both a large valid database of
medical knowledge and the patient record”
R. Smith, BMJ, 1996
Medical Information
and Medical Decision Making
 Medical Decision Making
 Nature of Medical Information
 Reducing Errors & Improving Quality
 Finding Knowledge and Evidence
Do these challenges around
medical information and
decision making matter?
Hippocratic Oath
 Hippocratic Oath has evolved since 400 BC as
societal values and standards have changed
 Two aspects particularly relevant to HuBio 590
Medical Information for Decision Making:
“To practice and prescribe to the best of my ability for
the good of my patients, and to try to avoid harming
them.” => finding and applying the best information
“To keep the good of the patient as the highest priority”
=> integrating patient specific information
 How successful are we at this today?
 Press release
 “…medical errors kill some
44,000 people in U.S. hospitals
each year. Another study puts
the number much higher, at
98,000 ”
 “Even using the lower estimate,
more people die from medical
mistakes each year than from
highway accidents, breast
cancer, or AIDS.”
 Some errors are unavoidable
mistakes, some errors are due
to missing or incorrect data or
 Press release
 “The nation's health care
industry has foundered in its
ability to provide safe, highquality care consistently to all
 Studies estimate 3-4% of
hospitalizations result in adverse
 Recommendation 2 “…six
major aims: specifically, health
care should be safe, effective,
patient-centered, timely,
efficient, and equitable”
 Press release
 “Medication errors are among the most
common medical errors, harming at
least 1.5 million people every year,
says a new report from the Institute of
Medicine of the National
Academies. The extra medical costs of
treating drug-related injuries occurring
in hospitals alone conservatively
amount to $3.5 billion a year”
 Recommendation 3: “All health care
organizations should immediately
make complete patient-information
and decision-support tools available
to clinicians and patients. Health care
systems should capture information on
medication safety and use this
information to improve the safety of
their care delivery systems.”
Economic & Legal Context
 Rising healthcare costs
2005: $2 Trillion, 16% GDP, 6.9% (2 x inflation)
Each diagnostic & therapeutic decision has a cost
If 47% of treatments are “of unknown effectiveness”
then what is the cost implication of this?
 Malpractice
Malpractice “the provider failed to conform to the
relevant standard of care.”
Historically standard of care = “reasonable person”
Evolution of standard of care = “what is the evidence”
Medical Information
and Medical Decision Making
 Medical Decision Making
 Nature of Medical Information
 Reducing Errors & Improving Quality
 Finding Knowledge and Evidence
First Randomized Controlled Trial - 1948
Evidence Based Medicine – 1992-2008
 “Evidence-Based Medicine: A New Approach to
Teaching the Practice of Medicine”, JAMA 1992
(classic article, see MIDM website)
 “Evidence-based medicine (EBM) requires the
integration of the best research evidence with
our clinical expertise and our patient’s unique
values and circumstances” in Evidence Based
Medicine: How to Practice and Teach EBM,
Straus et al 2005 (the definitive textbook)
 “Progress in Evidence-Based Medicine”, JAMA
2008 (reviews 1992 article, see MIDM website)
Evidence Based Medicine - Caveat
 Parachute use to prevent
death and major trauma
related to gravitational
challenge: systematic
review of randomised
controlled trials. Smith
and Pell, BMJ, Dec 2003
Where to get “best research evidence”?
CD ROM books,
CME, etc.
No single source for “best evidence”
 Ideal: continually updated, synthesized, expert
authored, peer reviewed, electronic knowledge
 Cochrane collaboration (www.cochrane.org) and
similar databases are closest we have but…
Restricted to areas with sufficient literature
Updated episodically, not real time
 Developing tools for finding evidence is an active
area of biomedical informatics research nationally
So What Do You Do?
 Use Evidence Based Medicine Resources
 Learn General Standard of Care
National policy statements, e.g. American Academy of Pediatrics
 Learn Local Standards of Care
Policies/guidelines, e.g. UW
Prevailing practice – conferences/grand rounds
 Keep Up to Date on New Clinical Studies
Journal abstracting/summarizing services
 Learn to search databases of medical knowledge…
Steps to Finding & Assessing Information
1. Translate your clinical situation into a formal
framework to get a searchable question (today)
2. Choose source(s) to search (Session 2)
3. Search your source(s) (Session 2)
4. Assess the resulting articles (documents)
Therapy documents (Session 3)
Diagnosis documents (Session 4)
Systematic reviews/comparing documents (Session 5)
5. Decide if you have enough information to make a
decision, repeat 1-4 as needed (ICM, clinical rotations,
internship, residency) (Session 6)
Step 1: Frame the question (I)
 Translate clinical question to searchable question
(PPICONSS framework for assessing a document,
PPICOS framework for formulating a
search/finding information)
 P P : Problem
 P P : Patient/Population
 I I : Intervention
 C C : Comparison
 O O : Outcome
 N
: Number of Subjects
 S S : Study Design/Type/Statistics
 S
: Sponsor
Step 1: Frame the question (II)
 Translate clinical question to searchable question (PPICOS)
 P: Problem
P: Patient
Alternative diagnosis(es)/treatment(s) (of secondary interest)
E.g. “Over the counter ointment vs. prescription ointment vs. pill”
O: Outcome
Diagnosis/treatment, which one is of primary interest/preferred a priori
E.g. “Treatment with over the counter ointment”
C: Comparison
Demographics (e.g. gender/age range), condition, disease
E.g. “Healthy female college athlete with skin/nails affected”
I: Intervention
What is the question of interest?
E.g. “How to treat athletes foot?”
Diagnostic accuracy, complication, death, cost, etc.
E.g. “Cheapest and safest cure since no insurance” => Cost, how often does
each alternative cure it, what are side effects of each treatment
S: Study Design/Type
Ideally what type of study/document are you looking for
E.g. “Systematic Review”
Compare Results to Search (I)
Problem at hand
Problem studied
Are they really the same?
Patient characteristics
Population characteristics
Intervention most relevant
to patient/provider
Comparison – other
alternatives considered
Outcomes – those
important to pat/prov
Intervention studied
(primary one)
Comparison – alternatives
Outcomes – those looked
at by study
Number of subjects
Is patient similar enough to
population studied?
Are they the same?
Study design hoped for
Statistics – study design
and statistical results
Sponsor – who paid for
Are alternatives studied
those of interest to you?
Are outcomes studied
those of interest to you?
Does study have enough
subjects to trust results?
Is study design good?
What do results mean?
Is there potential bias?
Compare Results to Search (II)
Problem: Athletes foot of
Problem studied: Athletes
foot of skin/nails
Patient: young adult,
female, healthy
Intervention relevant:
treatment with over
counter cream
Comparison: over counter
cream, prescription cream,
Outcomes: desire cheapest
& safest cure
Population characteristics: May not be similar enough
healthy elderly males
Intervention studied:
treatment with new pill
Study design hoped for:
“Systematic Review”
Comparisons studied:
prescription cream vs. old
pill vs. new pill
Outcomes: cure rate, side
They are the same
Pretty close but no over
counter comparison
Ok since can look up cost
elsewhere but side effects
in patient vs. population?
Statistics – a single clinical We wanted a review of all
available studies
Conclusion: probably not what we want, look for another document
October 21, 2008
Session 1b:
Statistics 101
Peter Tarczy-Hornoch MD
Head and Professor, Division of BHI
Professor, Division of Neonatology
Adjunct Professor, Computer Science and Engineering
Statistics 101: Mean, Standard Deviation
 *Population: weights of all medical students in the class
 *Sample: weights of 10 randomly chosen students:
50, 53, 56, 60, 65, 67, 70, 73, 73, 75 kg
 Sample Mean:
Mean=sum{50,53,…75}/10=64.2 kg
 Sample Variance
s2=sum{(50-64.2)2,...(75-64.2)2}/(10-1) =80.6
 Sample Standard Deviation
s=sqrt(80.6)=8.97 kg
Statistics 101: Normal Distribution
*Sample 1=X
*Sample 2=O
 68% of values are +/- one  from the mean
 95.4% of values are +/- two  from the mean
 99.6% of values are +/- three  from the mean
 Sample standard deviation and mean estimate
population  and mean
Statistics 101 – Sample Sizes
 Mean:
Increasing n does not impact mean
 Sample Variance
Increasing n decreases sample variance
 Sample sizes
Larger sample sizes decrease variance and allow you to
see smaller differences between groups
Rule of thumb for sample size for a strong study n=400
Statistics 101: p values
 “Treatment 1 was better than treatment 2 (P<0.05)”
P<0.05 roughly means less than 5% (0.05) chance treatment 1 and 2
are the same
 “Treatment 1 was better than treatment 2, P=0.001”
P=0.001 roughly means 1/1000 (0.001) chance treatment 1 and 2 are
the same
 p=0.04 vs. p=0.05 vs. p=0.06
All roughly the same, choice of “p<0.05” as “statistically significant” is
 Study 1: mortality cut by 50% with p=0.04 vs Study 2:
mortality cut by 1% with p=0.01
Cutting mortality by 50% clinically more significant than by 1%
P=0.01 is statistically more significant than P=0.04
See http://www.acponline.org/journals/ecp/julaug01/primer.pdf
Also http://www.acponline.org/journals/ecp/primers.htm
Statistical  Clinical Significance
 Statistical Significance: are the treatments the same?
 Clinical Significance: if they are different then do we
care about the difference?
 Examples:
Duration of pharyngitis: 8.1 days to 7.4 days
Weight: 279 lbs to 266 lbs after 3 months
Survival increased from 4.5 mos to 5.2 mos with 100%
mortality at 12 months
Claudication: Increase in walking distance by 34 ft.
Small Group Sessions 1 & 2
 S1: Small group leads to introduce themselves
Name, where they are from, where they went to medical school,
their clinical practice, interesting fact about their background
 S1: Students to introduce themselves
Name, where they are from, interesting fact about their
background, what they hope to get from small group sessions
 S1: Small group leads to give examples of translating
clinical situations/scenarios into a searchable question
(using PPICOS framework)
 S2: Assignment for 10/22 (tomorrow) for students
Come up with one clinical “situation” you have wondered about
or been asked about and translate it into PPICOS framework
Work through 4 scenarios “Formulating a searchable question”
Bring paper or electronic copy of your completed assignment
Small groups to identify two PPICOS scenarios for assignment
for small group on 10/27
 * Reminder: students please sign in, group leads please
turn in sign in sheets to Donna Rowe, Box 357240
 Medical Decision Making
 Nature of Medical Information
 Reducing Errors & Improving Quality
 Finding Knowledge and Evidence
 Statistics 101
 Small Group Session
 When done with Q/A we will all go to our
assigned small groups

No Slide Title