Readability Assessment of British
Internet Information Resources on
Diabetes Mellitus Targeting Laypersons
Maged N. Kamel Boulos, Ph.D.
School for Health, University of Bath
[email protected]
HDL-ECDL’04, 16 September 2004, Bath, UK
Agenda
•
•
•
•
•
Introduction
Material and Methods
Results
Discussion and Recommendations
Conclusions and Useful Web Resources
HDL-ECDL’04, 16 September 2004, Bath, UK
Introduction
HDL-ECDL’04, 16 September 2004, Bath, UK
On Diabetes Mellitus
• Diabetes mellitus, with its different clinical types, is
an important public health condition in the UK and
elsewhere, accounting for a sizeable percentage of
the annual NHS expenditure.
• The education of members of the public, patients
and their carers is widely recognised as an essential
component of chronic disease care and effective
health promotion.
• Appropriate education can minimise the risk of
diabetes mellitus and its complications.
• The Internet could be an extremely useful and costeffective medium in this respect.
HDL-ECDL’04, 16 September 2004, Bath, UK
On Readability
• Readability is an attempt to match the reading
level of written material to the ‘reading with
understanding’ level of the reader.
• Readability (or understandability) is a frequently
overlooked aspect of health information quality
and accessibility.
• For example, a recent evaluation by
Thakurdesai et al. (2004) of the quality and
contents of diabetes mellitus patient education
on the Internet failed to test the reviewed sites
for readability.
HDL-ECDL’04, 16 September 2004, Bath, UK
On Readability – Cont’d
• Nearly half of all American adults or
90 million people have limited
English and math skills (most are
native-born English speakers),
making it difficult for them to fully
and safely understand and act upon
health information (US Institute of
Medicine, 2004).
• Comparable proportions have been
reported in the UK according to the
Office for National Statistics.
HDL-ECDL’04, 16 September 2004, Bath, UK
IOM Report Cover
Readability Testing Methods
• More costly and time-consuming, but also
more precise methods:
– A comprehension test based on the document of
interest administered to a group of readers of known
reading ability.
– Measuring readability by the judgment of a ‘literacy
expert’.
• Quick and rough approximations:
– Using readability formulae: mathematical equations that
estimate the reading level of a document based on the
words that are used and the lengths of sentences.
– Dozens of different readability formulae (e.g., Fry, SMOG and
Flesch/Flesch-Kincaid formulae) have been developed over the
years and many are implemented today in software.
HDL-ECDL’04, 16 September 2004, Bath, UK
Material and Methods
HDL-ECDL’04, 16 September 2004, Bath, UK
Sample Selection
• We used Google (http://www.google.co.uk/ - UK
pages) to locate British Internet information
resources on diabetes mellitus targeting
laypersons.
• The ‘links’ sections of some of these resources
were also explored to pick further UK sites.
• In total, we selected 15 trustworthy English
language sites/providers for inclusion in this
study (all of UK provenance).
HDL-ECDL’04, 16 September 2004, Bath, UK
Readability Formulae and Software Used
• We tested the readability of 20 representative
pages from these 15 sites, providing consumer
information on diabetes mellitus.
• The full text from each of these pages was
opened in Microsoft Word 2000 (any page
navigation text was removed), and a readability
level was generated using the Flesch Reading
Ease score and Flesch-Kincaid Grade Level
score.
HDL-ECDL’04, 16 September 2004, Bath, UK
Readability Statistics in MS Word
Readability statistics in Microsoft Word
for the text of
http://www.diabetes.org.uk/infocentre/pubs/Whatc.doc
The dialogue box on the left shows
where to enable this function (‘Show
readability statistics’) in Microsoft Word.
HDL-ECDL’04, 16 September 2004, Bath, UK
Calculation of Flesch Reading Ease
and Flesch-Kincaid Grade Level scores
HDL-ECDL’04, 16 September 2004, Bath, UK
Transforming Results to Equivalent UK
Reading Age and Computing Statistics
• The reading level in US school grade (FleschKincaid) was then transformed to the equivalent
UK reading age.
• For example, a Flesch-Kincaid Grade Level
score of 8.0 equates to the UK reading age of an
educated 13 year old person.
• Mean and median readability scores of all
sites/pages and other statistics were computed
in Microsoft Excel 2000.
HDL-ECDL’04, 16 September 2004, Bath, UK
Results
HDL-ECDL’04, 16 September 2004, Bath, UK
Required Reading Age of Tested Material
• The average and median UK reading ages
of the sites/pages tested were 14.2 and
14.65 years of age respectively (range: 1116.9 years).
• These figures are well above the
estimated reading age of the UK
population in general (9 years or Year
4/5 at UK school).
HDL-ECDL’04, 16 September 2004, Bath, UK
Results – Cont’d
• Only 7 pages (out of 20 tested, i.e., 35% of all tested
pages) from 4 sites/providers (out of 15 included in
this study, i.e., 26.66% of all sites/providers
evaluated in this study) had a Flesch Reading Ease
score above 60 (range: 60.4-73.3) and a FleschKincaid Grade Level of 8.0 or lower (range: 6.0-8.0)
corresponding to a UK reading age of 11-13 years.
• The remaining sites/pages in this study scored
worse (Flesch Reading Ease score range: 31.6-57.1
and Flesch-Kincaid Grade Level range: 9.2-11.9),
with a corresponding UK reading age of 14.2-16.9
years.
HDL-ECDL’04, 16 September 2004, Bath, UK
Detailed Results
HDL-ECDL’04, 16 September 2004, Bath, UK
And the Winner Is…
• NHS Direct Online health encyclopaedia
(http://www.nhsdirect.nhs.uk/) scored among the worst
sites in this study, while the patients’ section of
BestTreatments (BMJ Publishing Group/NHS Direct http://www.besttreatments.co.uk/) came out as the best,
and most readable site in our study.
• As expected, the two readability scoring formulae used
in our study were found to be very highly correlated (a
high negative correlation of -0.92124, measured using
Microsoft Excel CORREL function; when Flesch Reading
Ease increases, Flesch-Kincaid Grade Level decreases
and vice versa).
HDL-ECDL’04, 16 September 2004, Bath, UK
Discussion and
Recommendations
HDL-ECDL’04, 16 September 2004, Bath, UK
How Does This Study Compare With
Previous Studies?
• Our results echo those of similar studies conducted over
the past few years, and point to a chronic and often
forgotten information quality problem.
• Berland et al. (2001) reviewed 25 English and Spanish-language
health Web sites and concluded that “much of the health
information available on the Internet is beyond the
comprehension of many consumers”.
• Kusec et al. (2003) studied the readability levels of 99 diabetesrelated Web sites displaying the HONcode logo
(http://www.hon.ch/) and found that 86.9% of the tested material
would be too difficult to read for the average adult population.
• Chestnutt (2004) evaluated the readability of 100 patient-related
Internet sites presenting information on common oral pathologies
and reported a mean UK reading age of the evaluated sites of
10-11 years, which is higher than the UK national reading age.
HDL-ECDL’04, 16 September 2004, Bath, UK
Limitations of Readability Formulae
• There are many factors that affect readability and
understandability of online health information, which
are not measured by readability formulae.
• Rudolf Franz Flesch, who created the Flesch
readability formula, warned that “Some readers, I am
afraid, will expect a magic formula for good writing
and will be disappointed with my simple yardstick.
Others, with a passion for accuracy, will wallow in the
little rules and computations but lose sight of the
principles of plain English. What I hope for are
readers who won't take the formula too seriously and
won't expect from it more than a rough estimate”.
HDL-ECDL’04, 16 September 2004, Bath, UK
But They Remain Better Than Nothing...
• Nevertheless, we recommend that any
health information quality benchmarking
scheme or checklist in use should include
(or be supplemented with) some explicit
testing for content readability.
• Readability testing results could be also
displayed on the tested pages to inform
readers (‘a readability seal’).
HDL-ECDL’04, 16 September 2004, Bath, UK
Making Online Consumer Health Information
Easier to Understand
• Ensure that written material is pitched at a level
appropriate to the intended lay audience (aim to
match the general reading level of the UK
population: nine years or Year 4/5 at UK school).
• Write shorter sentences and paragraphs.
• Use the active voice.
• Avoid jargon and use simpler, common words.
• A picture is worth thousand words: include real
world examples and easy-to-understand
illustrations (where applicable).
HDL-ECDL’04, 16 September 2004, Bath, UK
Practical Example: (Re-)writing for Readability
Quoted from
http://www.pfizerhealthliteracy.com/improving_empathy.html
Original text - Flesch Reading Ease 14.5: A living will is a written
declaration directing your doctor to provide, withhold or withdraw
lifesaving procedures should you be diagnosed as having a
terminal condition or exist in a vegetative state and you lack the
capacity to make the decision. (Source: Planning for Incapacity, Legal
Council for the Elderly)
Revised text in an easy-to-read format - Flesch Reading Ease 68.1
(^ = better): A living will is a paper you sign telling your doctor
whether you want treatments that could keep you alive if you
have a terminal disease or are in a coma. This lets your doctor
know how you want to be treated if you are not able to make the
decision yourself.
HDL-ECDL’04, 16 September 2004, Bath, UK
Making Online Consumer Health Information
Easier to Understand – Cont’d
• Properly structure and organise information into
sections and subsections with meaningful
headings and subheadings.
• Use bulleted lists (where applicable).
• An inverted pyramid writing style is sometimes
also helpful (start the article by telling the reader
the conclusion/action(s) to take, followed by the
most important supporting information, and end
by giving the background).
HDL-ECDL’04, 16 September 2004, Bath, UK
Making Online Consumer Health Information
Easier to Understand – Cont’d
• Consider the specific cultural needs and socioeconomic levels of different ethnic groups. For
example, health education material should describe
alternative options in case of lack of economic
resources, when referring to processes that imply
large expenses, e.g., some costly diabetic diet
recipes.
• Also pay attention to the visual style and
presentation of online health information
material, including issues like text/background
colour and font type and size. These are equally
important to improve information accessibility.
HDL-ECDL’04, 16 September 2004, Bath, UK
Consider Other Means of Delivery
• However, even the most readable documents will
remain difficult to fully and properly understand for a
sizeable proportion of the population.
• For this reason, in addition to written online material,
health information providers should also consider
other means of delivering health education and
promotion programmes like face-to-face education
and plain English videos, so that no one is left
behind.
• The videos could be also made available online and
provided in additional languages spoken by ethnic
minorities/immigrants.
HDL-ECDL’04, 16 September 2004, Bath, UK
Conclusions and Useful Web
Resources
HDL-ECDL’04, 16 September 2004, Bath, UK
Not All Patients Are Experts!
• During recent years, there has been a growing trend of treating
medical knowledge as a single blob that is relevant to both
professionals and laypersons, and so should be made
accessible to all groups without any distinction.
• Supporters of this trend believe that patients should be
empowered and given more information and control of their
conditions. Indeed, laypersons sometimes show more
knowledge and understanding of their own conditions than their
treating doctors do.
• In spite of that, it seems sensible to continue writing different
information sets for the healthcare professional, the highly
educated and the average layperson, and to label online health
information according to its intended primary audience.
HDL-ECDL’04, 16 September 2004, Bath, UK
Conclusions – Cont’d
• Besides the fundamental requirement to check the
medical correctness or soundness, completeness,
accuracy, and currency of health information targeting
laypersons, online health information providers also
need to ensure that their written material is pitched at a
level appropriate to their intended lay audience.
• This is an equally important requirement as it affects the
final utility and reliability of published online material.
• Public and patient health information that is difficult to
understand or liable to misunderstanding by the lay
consumer could result in serious consequences.
HDL-ECDL’04, 16 September 2004, Bath, UK
Useful Web Resources
• Pfizer Clear Health
Communication http://www.pfizerhealthliteracy.com/
and their handbook http://www.pfizerhealthliteracy.com/4548_Health_literacy_all.pdf
• The Center for Health Care
Strategies’ Fact Sheets on
Health Literacy - http://www.chcs.org/
HDL-ECDL’04, 16 September 2004, Bath, UK
Descargar

Document