Scholarship in Support of
Evidence-Based Practice:
Generating and disseminating
evidence to enhance service
quality
Gary Kielhofner
The State of Evidence-Based Practice
Increasing demand for practice based on evidence
Leaders in the field and external forces asking
practitioners to based their practice decisions on
evidence
Practitioners expected not only to use but to locate and
evaluate evidence
Concerns expressed by practitioners:
Relevance of evidence (does not always fit practice
circumstances)
Unavailability of evidence for necessary practice decisions
Accessibility of evidence
Difficulty of evaluating evidence
Some guiding principles
• The “right evidence” depends on what
practice decision is being informed
For example:
• Given a specific population, what needs or problems
should I address?
• What assessments should I use to evaluate this
population
• What kinds of services are possible to give in my
context (eg. acute care, community-, home- or schoolbased)
• What interventions are likely to be effective for this
population?
Each question requires a different type of evidence
Some guiding principles
Relevant evidence for practice is multifaceted
and may include:
– Clinical knowledge (perspectives and
experiences of experts)
– Perspectives of clients (focus groups and
qualitative studies)
– Applied studies (psychometric, outcome)
– Basic studies that contribute to
understanding phenomena address in
practice and provide evidence about
theories used in practice
The most relevant evidence will emerge
from a Scholarship of Practice approach
Theory &
Research
Academics,
practitioners and
clients
together
generate
knowledge
about what
can/should be
done in practice
Practice
Conceptual practice models can be
effective knowledge-creating systems,
bringing together:
– Theory development
– Development of practical resources (e.g.,
assessments, case examples, intervention
protocols, programs of service)
– Expert clinical reasoning of therapists who
use the theory and resources
– Basic and applied research that tests
theory and evaluates resources
Guide
Refine
Theory
Provide
feedback
Raises
questions
Tests,
improves
Is there evidence for the
proposed concepts?
Is there evidence for the
propositions concerning
their relationships?
What does new evidence say
about expanding/changing
concepts or generating new
concepts?
Develop resources (e.g.
assessments,
procedures, equipment,
programs)
Raise
question
s
Tests,
improves
Do assessments work in
practice?
How do concepts influence
therapeutic reasoning?
How does use of concepts
shape what happens in
therapy?
What are outcomes?
Typical Basic and Applied Research Questions
MOHO:
Two examples of
traditions of research
developing evidence
to support practice:
UIC project with
persons who have
HIV/AIDS
UKCORE project in
Mental Health
Enabling Self
Determination
Controlled study of the
effectiveness of an
intervention program
designed to enhance
productive participation of
persons with HIV/AIDS
 in 4 transitional living
facilities in greater Chicago
Enabling Self Determination
compared 2 groups receiving ESD
or standard care program
Logistics ruled out a conventional
randomized design and an
interrupted times series design
The most rigorous design that
could be implemented for the study
was a non-randomized two-group
design (2 facilities served as the
ESD sites and 2 as standard care
sites)
The ESD program:
 Based on the Model of Human Occupation
 Provided group and individual interventions
designed to enhance volition for and support the
development of routines habits and skills that would
support the choice and enactment of new productive
occupational roles.
 Elements of the program
were incorporated from EO
 Informed by focus groups
with residents and staff in
transitional living facilities
Demographics of participants in the ESD and
Standard Care programs (n=65) Note: no
statistically significant differences
ESD
Program
Standard Care
Program
Mean Age [SD]
Gender
42.68 [8.00]
42.17 [7.54]
Male
Female
Ethnicity/Race
Caucasian
31 ( 82%)
7 (18%)
Variable
African-American
Hispanic/Latino
Other
8 (21%)
27 (71%)
2 ( 5%
1 ( 3%)
21 (78%)
6 (22%)
6
20
1
0
(22%)
(74%)
( 4%)
(0.0%)
Demographics of participants in the ESD and
Standard Care programs (continued)
History of mental illness, substance abuse, or
felony conviction
Mental Illness
23 (61%)
15 (56%)
History of Substance
Abuse
30 (79%)
17 (63%)
Felony conviction
15 (40%)
10 (37%)
Mean Symptom Intensity
31.33
21.86
Mean Symptom Total
18.73
13.60
Impairment
Comparison of productive outcomes among
participants in the ESD and standard care programs at
3, 6, and 9 months post intervention (n=45)
ESD Program
Participants
with
Productive
Participation
Standard Care
Participants
with
Productive
Participation
ChiSquare
Odds
Ratio
f/n (%)
f/n (%)
3 months
20/28 (71.4%)
6/17 (35.3%)
5.66
.019
4.58
6 months
21/27 (77.8%)
5/18 (27.8%)
11.07
.001
9.10
9 months
18/25 (72.0%)
5/14 (35.7%)
4.88
.031
5.66
•1-tailed Fisher’s Exact Test
Value
Pvalue*
Clients’ and Therapists’
Perceptions of the Occupational
Performance History Interview – II
(OPHI - II)
Study Aims
• To understand from the perspectives of
therapists how they used the interview
and narrative slope, and what value and
limitations they saw in them
• To examine how clients understood and
experienced the interview and narrative
slope and whether they found it useful for
themselves
Methods: Design
Anthropological Rapid Ethnographic
Approach (Scrimshaw and Hurtado, 1987;
Bernard,1994)
Two groups of participants:
A convenience sample consisting of the
occupational therapists providing services
Purposive sample of client participants
who were enrolled in the ESD program
Client Participants’ Perspectives on the
Interview Process
• positive and useful process
• Impact of the Interview Process:
– Increased communication and trust with
the therapist
– Personal Insights that emanated from the
interview
– Clarified their own thoughts and feelings
– Help to see life more positively
– Clarified thinking about the future
Client Perspectives on the
Narrative Interview as a
Positive and Useful Process
“I could see where I am, you know,
where’s my strengths, where’s my
weakness, then you know, cause it gave
time to reflect after the interview, like when
I finished talking, then I could see how I
feel about this, when I am all by myself, I
could reflect on what I had said” – Client
Client Participants’ Perspectives
on Impact of the Interview
Process
“That interview made me realize that I
was wasting a lot of time not doing
anything, and now after the interview I am
doing things that I should have been
doing more aggressively, than just sitting
back and waiting on it to come to me, so I
woke up and started working towards
those ideas”

“negative events below the line serve as an index of
what needs to be changed and that the overall upward
direction of the narrative slope encourages me”
Supportive living
and recovery
Growing up with
family in Chicago
Good
Moves out, has
various jobs
Works in
restaurant, 1 year
sober
Living and
recovery
Residential
treatment program
Narrative
Slope
Marries and has 1st
child
Bad
Growing up with
Supportive
family in Chicago
Deep into drug use
Significant other
passes, relapse
Suicide attempt
Client Participants’ Perspectives on
the Narrative Slope
“It should be given to clients, its good to
know, because the 1st thing which I would do is
to hang it on my wall, it will show me what I am
accomplishing where I am failing, where to go. .
. this [narrative slope] will help me keep
focused”
Client Participants’ Perspectives
on the Narrative Slope
“I think the clients should draw the line.
They have a good idea of where they are at
with their life. It would be more helpful for
the client that way. Therapist drawing is not
so helpful, unless the client wants the
therapist to help. The client should be given
choice. For me it would have been good, I
would include some more events”
Predictive value of Narrative slope-2 studies
Good
Regressive
Stability
Progressive
Bad
PROGRESSION OF TIME
Prospective study of 129 clients by type
of narrative slope showed it was the only
significant predictor of outcomes
Regressive
(n=49)
Dropped
out
Completedno work or
school
Competed, work or
school
Progressive
(n=41)
Stability
(n=32)
39%
15%
25%
29%
27%
31%
32%
58%
44%
Analysis of Narrative in 65
participants from ESD study
No Significant* differences by outcomes were
found across any demographic variables
•
•
•
•
•
•
•
age,
gender,
education,
recent work history,
history of mental illness
history of substance abuse
level of impairment
*Chi-square with Bonferroni correction for multiple tests
was used
Relationships Between Narrative Slope at Baseline
and Engagement in Employment and Other
Productive Activity at Follow-up (Kendall’s tau-b)
Follow-up Period
Employment
tau-b
p
Other Productive
Activity
tau-b
p
Discharge
.35
.003 .23
.006
Three Months
.40
.009 .50
.001
Six Months
.50
.001 .31
.037
Nine Months
.21
.297 .33
.054
Payment by Results (PbR)
• Funding mechanism used to pay for
acute hospital services in England
•Reimburse for services on the basis of
21 clusters of clients according to need
DECISION TREE
(Relationship of Care Groups to each other)
Working-aged Adults and Older People with Mental health Problems
A
NonPsychotic
a.
Mild/mod/
severe
1
2
3
B
Psychosis
b.
Very Severe
and Complex
4
5
6
7
c.
Substance
Misuse
8
9
a.
First
episode
b.
Severe
ongoing
10
11 12 13
C
Organic
c.
Psychotic
crisis
14
15
d.
Very severe
engagement
16
17
a.
Cognitive
Impairment
18 19 20 21
UIC team is part of the process of identfiying
the OT care package for each of the 21
clusters (to be completed by October 2010):
3 phased study Empirical identification of
1) occupational profilers of service-users within
the clusters
2) Survey of OT’s in England to elicit expert
opinion about clinical aims
3) Participatory research project with consumers
and therapists to identify care pathways
Method
Retrospective, descriptive study made use
of data extracted from clinical records from:
• South West London & St Georges
Mental Health NHS Trust (SWSTG)
and
• South West Yorkshire Partnership
Foundation NHS Trust (SWYPFT)
• Instruments: HoNOS-PBR & MOHOST
MOHOST Rating Scale,
Subscales, and Items
Data were retrieved on 645 participants
• 356 females (55.2%)
• 289 males (44.8%).
• 18 to 96 years old (M= 58.52;
SD=21.99).
Employment data available on 384:
• 31 ( 8.1%) employed,
• 211 (54.9%) of employment age
and unemployed,
•142 (36.0 %) were retired.
C
om
un
i
m
ca
t io
n
Pa
n
&
of
fo
r
En
ot
or
ss
en
t
m
Sk
ills
Sk
ills
vir
on
M
ce
Sk
ills
tio
n
tio
n
3
Pr
o
n
cc
up
a
cc
up
a
2.45
ct
io
O
O
In
te
ra
tte
rn
at
io
ot
iv
M
Mean MOHOST score (N=645)
4
3.10
2.92
2.58
2.77
2.32
2
1
Average Percent of Subscale items with Ratings of
Interfere or Restrict
Volition
51.7 %
Habituation
60.3 %
Communication/Interaction
Skills
Process skills
30.0 % *
Motor Skills
24.7 % **
Environment
38.4%)
46.6 %
* Item reflecting relationships = 43.8 %
** Correlation with HoNOS-PbR physical
illness/disability item (r=.43 p<.001)
4
Volition
Habituation
3
CI Skills
Process
2
Motor Skills
Environment
1
Common
Mental Health
Problems
(Low Severity
& Low
Severity with
greater need)
NonPsychotic
(Moderate
Severity)
NonPsychotic
(Severe)
NonPsychotic
(Very Severe)
Mean
Age
Percent
Male Female
Common Mental Health Problems
50.19
23.8
76.2
Non-Psychotic (Moderate Severity)
59.10
24.5
75.5
Non-Psychotic (Severe)
61.73
32.7
67.3
Non-Psychotic (Very Severe)
62.27
42.3
57.7
HoNOS PbR Items
(1=no problem-5=very severe
problem)
Cluster
Cognitive
problems
Common
mental health problems
Non-psychotic (moderate
severity)
Non-psychotic (severe)
.24
Non-psychotic (Very
severe
Problems
with living
condition
.33
.55
.43
.79
.84
.46
.77
higher scores in volition, habituation & environment may
reflect longer engagement in services.
4
Volition
3
Habituation
CI Skills
Process
Motor Skills
2
Environment
1
Non-Psychotic
Disorders of
overvalued ideas
Enduring NonPsychotic
Disorders (High
Disability)
Cluster
Non-Psychotic
Chaotic &
challenging
disorder
Mean Age
% male
%
female
Non-Psychotic Disorders of overvalued ideas
37.09
42.9
57.1
Enduring Non-Psychotic Disorders
56.40
60.0
40.0
Non-Psychotic Chaotic & challenging disorder
46.00
43.8
56.3
4
3
Volition
Habituation
CI Skills
2
Process
Motor Skills
Environment
1
0
Substance Misuse
Dual Diagnosis
Cluster
Age M
Percent
Male
Percent
Female
Substance Misuse
49.25
80.0
20.0
Dual Diagnosis
39.64
90.9
9.1
1
Psychosis and Affective Disorder
difficult to engage
Severe Psychotic Depression
Psychotic Crisis
Ongoing or recurrent psychosis
(High symptom and Disability)
Ongoing or recurrent psychosis
(High Disability)
Recurrent Psychosis (Low
symptoms)
First episode Psychosis
4
3
Volition
Habituation
CI Skills
2
Process
Motor Skills
Environment
4
Volition
Habituation
3
CI Skills
Process
2
Motor Skills
Environment
1
Cognitive
Impairment (low
need)
Cognitive
impairment or
dementia
complicated
(moderate need)
Cognitive
Impairment or
dementia
complicated (high
need & High
physical or
engagement)
Cluster
Age M
% % Female
Male
Cognitive Impairment (low need)
80.05
24.4
75.6
Cognitive impairment or dementia
complicated (moderate need)
80.65
28.7
71.3
Cognitive Impairment or dementia
complicated ( high need)
73.76
46.3
53.7
Discussion
results indicate:
• there are qualitatively distinct occupational profiles for
each cluster
• these profiles make sense given the composition of
service-users in the clusters:
•Clusters with non psychotic problems had occupational
profiles that showed the least dysfunction and greatest
strengths.
•Clusters involving psychosis have occupational profiles
indicating more occupational dysfunction
•Clusters involving cognitive impairment demonstrated
the most difficulty with skills
Differences across the various clusters suggest
different emphases in care pathways:
`
• several
clusters showed mostly strength in skills
and may need less service related to skill
problems.
•the psychotic and cognitive impairment clusters
show more difficulty with skills and thus will need
services to address these problems
•those with more severe habituation and/or volition
challenges may need additional assessment and
will need treatment goals and interventions to
address these occupational problems.
MOHO:
evidence to support
practice
- Approx 500 articles/chapters
published
- 245 published studies
Comparison of Research-Based
Publications Across Occupation-Based
Models Lee (In Press)
260
240
220
200
180
160
140
120
100
80
60
40
20
0
245
1
EHP
MOHO
10
13
OA
PEOP
Web-based Efforts to organize
evidence for practitioners
MOHO website: www.moho.uic.edu
MOHO List Serv
–Way to share clinical expertise
–Archived discussions allows for future
access:
http://www.moho.uic.edu/archived_list_
serv.html
Archived listserv Discussions
Example: How to Access
MOHO List Serv Discussion
Evidence-based search engine on MOHO website
Locates all forms of evidence including those not readily
available through other searches such as a case
discussed in a chapter or list serve discussions.
Evidence-based Search Engine:
-identifies references and archived list
serve discussions
X
X
X
X
Example results from search:
Search Engine Example:
Results for children + assessment + PVQ
Evidence-Briefs:
• Available for ongoing research/ in press
research, also for published articles (including
non-English articles).
• Access through reference list; click hypertext
Evidence Briefs
(downloadable
from website)
– Summarize
studies noting
the research
questions,
population,
methods and
findings
– Discuss practice
implications
Making evidence-based resources
readily available:
EO and ESD programs are described in
detailed manuals downloadable free of charge
Several MOHO based assessments are also
downloadable
Other MOHO-based assessments/interventions
can be purchased from MOHO e-store (include
manuals and cases so instruments can be selftaught)
(Manual purchasers can download additoinal
resources such as reporting forms that have
been developed by practitioners)
Evidence for Practice: MOHO Website
MOHO Tools
• Provides information about all MOHO-related
assessments, MOHO-based programs and
interventions, as well as sample pages from various
assessments.
How to Access
Evidence for Practice: MOHO Website
Additional Resources
• This page provides information to request or download
free resources, MOHO related web links, and contacts for
MOHO assessments translated into other languages.
Questions?
Descargar

Empirical Examination of Theoretical Constructs