Scholarship in Support of Evidence-Based Practice: Generating and disseminating evidence to enhance service quality Gary Kielhofner The State of Evidence-Based Practice Increasing demand for practice based on evidence Leaders in the field and external forces asking practitioners to based their practice decisions on evidence Practitioners expected not only to use but to locate and evaluate evidence Concerns expressed by practitioners: Relevance of evidence (does not always fit practice circumstances) Unavailability of evidence for necessary practice decisions Accessibility of evidence Difficulty of evaluating evidence Some guiding principles • The “right evidence” depends on what practice decision is being informed For example: • Given a specific population, what needs or problems should I address? • What assessments should I use to evaluate this population • What kinds of services are possible to give in my context (eg. acute care, community-, home- or schoolbased) • What interventions are likely to be effective for this population? Each question requires a different type of evidence Some guiding principles Relevant evidence for practice is multifaceted and may include: – Clinical knowledge (perspectives and experiences of experts) – Perspectives of clients (focus groups and qualitative studies) – Applied studies (psychometric, outcome) – Basic studies that contribute to understanding phenomena address in practice and provide evidence about theories used in practice The most relevant evidence will emerge from a Scholarship of Practice approach Theory & Research Academics, practitioners and clients together generate knowledge about what can/should be done in practice Practice Conceptual practice models can be effective knowledge-creating systems, bringing together: – Theory development – Development of practical resources (e.g., assessments, case examples, intervention protocols, programs of service) – Expert clinical reasoning of therapists who use the theory and resources – Basic and applied research that tests theory and evaluates resources Guide Refine Theory Provide feedback Raises questions Tests, improves Is there evidence for the proposed concepts? Is there evidence for the propositions concerning their relationships? What does new evidence say about expanding/changing concepts or generating new concepts? Develop resources (e.g. assessments, procedures, equipment, programs) Raise question s Tests, improves Do assessments work in practice? How do concepts influence therapeutic reasoning? How does use of concepts shape what happens in therapy? What are outcomes? Typical Basic and Applied Research Questions MOHO: Two examples of traditions of research developing evidence to support practice: UIC project with persons who have HIV/AIDS UKCORE project in Mental Health Enabling Self Determination Controlled study of the effectiveness of an intervention program designed to enhance productive participation of persons with HIV/AIDS in 4 transitional living facilities in greater Chicago Enabling Self Determination compared 2 groups receiving ESD or standard care program Logistics ruled out a conventional randomized design and an interrupted times series design The most rigorous design that could be implemented for the study was a non-randomized two-group design (2 facilities served as the ESD sites and 2 as standard care sites) The ESD program: Based on the Model of Human Occupation Provided group and individual interventions designed to enhance volition for and support the development of routines habits and skills that would support the choice and enactment of new productive occupational roles. Elements of the program were incorporated from EO Informed by focus groups with residents and staff in transitional living facilities Demographics of participants in the ESD and Standard Care programs (n=65) Note: no statistically significant differences ESD Program Standard Care Program Mean Age [SD] Gender 42.68 [8.00] 42.17 [7.54] Male Female Ethnicity/Race Caucasian 31 ( 82%) 7 (18%) Variable African-American Hispanic/Latino Other 8 (21%) 27 (71%) 2 ( 5% 1 ( 3%) 21 (78%) 6 (22%) 6 20 1 0 (22%) (74%) ( 4%) (0.0%) Demographics of participants in the ESD and Standard Care programs (continued) History of mental illness, substance abuse, or felony conviction Mental Illness 23 (61%) 15 (56%) History of Substance Abuse 30 (79%) 17 (63%) Felony conviction 15 (40%) 10 (37%) Mean Symptom Intensity 31.33 21.86 Mean Symptom Total 18.73 13.60 Impairment Comparison of productive outcomes among participants in the ESD and standard care programs at 3, 6, and 9 months post intervention (n=45) ESD Program Participants with Productive Participation Standard Care Participants with Productive Participation ChiSquare Odds Ratio f/n (%) f/n (%) 3 months 20/28 (71.4%) 6/17 (35.3%) 5.66 .019 4.58 6 months 21/27 (77.8%) 5/18 (27.8%) 11.07 .001 9.10 9 months 18/25 (72.0%) 5/14 (35.7%) 4.88 .031 5.66 •1-tailed Fisher’s Exact Test Value Pvalue* Clients’ and Therapists’ Perceptions of the Occupational Performance History Interview – II (OPHI - II) Study Aims • To understand from the perspectives of therapists how they used the interview and narrative slope, and what value and limitations they saw in them • To examine how clients understood and experienced the interview and narrative slope and whether they found it useful for themselves Methods: Design Anthropological Rapid Ethnographic Approach (Scrimshaw and Hurtado, 1987; Bernard,1994) Two groups of participants: A convenience sample consisting of the occupational therapists providing services Purposive sample of client participants who were enrolled in the ESD program Client Participants’ Perspectives on the Interview Process • positive and useful process • Impact of the Interview Process: – Increased communication and trust with the therapist – Personal Insights that emanated from the interview – Clarified their own thoughts and feelings – Help to see life more positively – Clarified thinking about the future Client Perspectives on the Narrative Interview as a Positive and Useful Process “I could see where I am, you know, where’s my strengths, where’s my weakness, then you know, cause it gave time to reflect after the interview, like when I finished talking, then I could see how I feel about this, when I am all by myself, I could reflect on what I had said” – Client Client Participants’ Perspectives on Impact of the Interview Process “That interview made me realize that I was wasting a lot of time not doing anything, and now after the interview I am doing things that I should have been doing more aggressively, than just sitting back and waiting on it to come to me, so I woke up and started working towards those ideas” “negative events below the line serve as an index of what needs to be changed and that the overall upward direction of the narrative slope encourages me” Supportive living and recovery Growing up with family in Chicago Good Moves out, has various jobs Works in restaurant, 1 year sober Living and recovery Residential treatment program Narrative Slope Marries and has 1st child Bad Growing up with Supportive family in Chicago Deep into drug use Significant other passes, relapse Suicide attempt Client Participants’ Perspectives on the Narrative Slope “It should be given to clients, its good to know, because the 1st thing which I would do is to hang it on my wall, it will show me what I am accomplishing where I am failing, where to go. . . this [narrative slope] will help me keep focused” Client Participants’ Perspectives on the Narrative Slope “I think the clients should draw the line. They have a good idea of where they are at with their life. It would be more helpful for the client that way. Therapist drawing is not so helpful, unless the client wants the therapist to help. The client should be given choice. For me it would have been good, I would include some more events” Predictive value of Narrative slope-2 studies Good Regressive Stability Progressive Bad PROGRESSION OF TIME Prospective study of 129 clients by type of narrative slope showed it was the only significant predictor of outcomes Regressive (n=49) Dropped out Completedno work or school Competed, work or school Progressive (n=41) Stability (n=32) 39% 15% 25% 29% 27% 31% 32% 58% 44% Analysis of Narrative in 65 participants from ESD study No Significant* differences by outcomes were found across any demographic variables • • • • • • • age, gender, education, recent work history, history of mental illness history of substance abuse level of impairment *Chi-square with Bonferroni correction for multiple tests was used Relationships Between Narrative Slope at Baseline and Engagement in Employment and Other Productive Activity at Follow-up (Kendall’s tau-b) Follow-up Period Employment tau-b p Other Productive Activity tau-b p Discharge .35 .003 .23 .006 Three Months .40 .009 .50 .001 Six Months .50 .001 .31 .037 Nine Months .21 .297 .33 .054 Payment by Results (PbR) • Funding mechanism used to pay for acute hospital services in England •Reimburse for services on the basis of 21 clusters of clients according to need DECISION TREE (Relationship of Care Groups to each other) Working-aged Adults and Older People with Mental health Problems A NonPsychotic a. Mild/mod/ severe 1 2 3 B Psychosis b. Very Severe and Complex 4 5 6 7 c. Substance Misuse 8 9 a. First episode b. Severe ongoing 10 11 12 13 C Organic c. Psychotic crisis 14 15 d. Very severe engagement 16 17 a. Cognitive Impairment 18 19 20 21 UIC team is part of the process of identfiying the OT care package for each of the 21 clusters (to be completed by October 2010): 3 phased study Empirical identification of 1) occupational profilers of service-users within the clusters 2) Survey of OT’s in England to elicit expert opinion about clinical aims 3) Participatory research project with consumers and therapists to identify care pathways Method Retrospective, descriptive study made use of data extracted from clinical records from: • South West London & St Georges Mental Health NHS Trust (SWSTG) and • South West Yorkshire Partnership Foundation NHS Trust (SWYPFT) • Instruments: HoNOS-PBR & MOHOST MOHOST Rating Scale, Subscales, and Items Data were retrieved on 645 participants • 356 females (55.2%) • 289 males (44.8%). • 18 to 96 years old (M= 58.52; SD=21.99). Employment data available on 384: • 31 ( 8.1%) employed, • 211 (54.9%) of employment age and unemployed, •142 (36.0 %) were retired. C om un i m ca t io n Pa n & of fo r En ot or ss en t m Sk ills Sk ills vir on M ce Sk ills tio n tio n 3 Pr o n cc up a cc up a 2.45 ct io O O In te ra tte rn at io ot iv M Mean MOHOST score (N=645) 4 3.10 2.92 2.58 2.77 2.32 2 1 Average Percent of Subscale items with Ratings of Interfere or Restrict Volition 51.7 % Habituation 60.3 % Communication/Interaction Skills Process skills 30.0 % * Motor Skills 24.7 % ** Environment 38.4%) 46.6 % * Item reflecting relationships = 43.8 % ** Correlation with HoNOS-PbR physical illness/disability item (r=.43 p<.001) 4 Volition Habituation 3 CI Skills Process 2 Motor Skills Environment 1 Common Mental Health Problems (Low Severity & Low Severity with greater need) NonPsychotic (Moderate Severity) NonPsychotic (Severe) NonPsychotic (Very Severe) Mean Age Percent Male Female Common Mental Health Problems 50.19 23.8 76.2 Non-Psychotic (Moderate Severity) 59.10 24.5 75.5 Non-Psychotic (Severe) 61.73 32.7 67.3 Non-Psychotic (Very Severe) 62.27 42.3 57.7 HoNOS PbR Items (1=no problem-5=very severe problem) Cluster Cognitive problems Common mental health problems Non-psychotic (moderate severity) Non-psychotic (severe) .24 Non-psychotic (Very severe Problems with living condition .33 .55 .43 .79 .84 .46 .77 higher scores in volition, habituation & environment may reflect longer engagement in services. 4 Volition 3 Habituation CI Skills Process Motor Skills 2 Environment 1 Non-Psychotic Disorders of overvalued ideas Enduring NonPsychotic Disorders (High Disability) Cluster Non-Psychotic Chaotic & challenging disorder Mean Age % male % female Non-Psychotic Disorders of overvalued ideas 37.09 42.9 57.1 Enduring Non-Psychotic Disorders 56.40 60.0 40.0 Non-Psychotic Chaotic & challenging disorder 46.00 43.8 56.3 4 3 Volition Habituation CI Skills 2 Process Motor Skills Environment 1 0 Substance Misuse Dual Diagnosis Cluster Age M Percent Male Percent Female Substance Misuse 49.25 80.0 20.0 Dual Diagnosis 39.64 90.9 9.1 1 Psychosis and Affective Disorder difficult to engage Severe Psychotic Depression Psychotic Crisis Ongoing or recurrent psychosis (High symptom and Disability) Ongoing or recurrent psychosis (High Disability) Recurrent Psychosis (Low symptoms) First episode Psychosis 4 3 Volition Habituation CI Skills 2 Process Motor Skills Environment 4 Volition Habituation 3 CI Skills Process 2 Motor Skills Environment 1 Cognitive Impairment (low need) Cognitive impairment or dementia complicated (moderate need) Cognitive Impairment or dementia complicated (high need & High physical or engagement) Cluster Age M % % Female Male Cognitive Impairment (low need) 80.05 24.4 75.6 Cognitive impairment or dementia complicated (moderate need) 80.65 28.7 71.3 Cognitive Impairment or dementia complicated ( high need) 73.76 46.3 53.7 Discussion results indicate: • there are qualitatively distinct occupational profiles for each cluster • these profiles make sense given the composition of service-users in the clusters: •Clusters with non psychotic problems had occupational profiles that showed the least dysfunction and greatest strengths. •Clusters involving psychosis have occupational profiles indicating more occupational dysfunction •Clusters involving cognitive impairment demonstrated the most difficulty with skills Differences across the various clusters suggest different emphases in care pathways: ` • several clusters showed mostly strength in skills and may need less service related to skill problems. •the psychotic and cognitive impairment clusters show more difficulty with skills and thus will need services to address these problems •those with more severe habituation and/or volition challenges may need additional assessment and will need treatment goals and interventions to address these occupational problems. MOHO: evidence to support practice - Approx 500 articles/chapters published - 245 published studies Comparison of Research-Based Publications Across Occupation-Based Models Lee (In Press) 260 240 220 200 180 160 140 120 100 80 60 40 20 0 245 1 EHP MOHO 10 13 OA PEOP Web-based Efforts to organize evidence for practitioners MOHO website: www.moho.uic.edu MOHO List Serv –Way to share clinical expertise –Archived discussions allows for future access: http://www.moho.uic.edu/archived_list_ serv.html Archived listserv Discussions Example: How to Access MOHO List Serv Discussion Evidence-based search engine on MOHO website Locates all forms of evidence including those not readily available through other searches such as a case discussed in a chapter or list serve discussions. Evidence-based Search Engine: -identifies references and archived list serve discussions X X X X Example results from search: Search Engine Example: Results for children + assessment + PVQ Evidence-Briefs: • Available for ongoing research/ in press research, also for published articles (including non-English articles). • Access through reference list; click hypertext Evidence Briefs (downloadable from website) – Summarize studies noting the research questions, population, methods and findings – Discuss practice implications Making evidence-based resources readily available: EO and ESD programs are described in detailed manuals downloadable free of charge Several MOHO based assessments are also downloadable Other MOHO-based assessments/interventions can be purchased from MOHO e-store (include manuals and cases so instruments can be selftaught) (Manual purchasers can download additoinal resources such as reporting forms that have been developed by practitioners) Evidence for Practice: MOHO Website MOHO Tools • Provides information about all MOHO-related assessments, MOHO-based programs and interventions, as well as sample pages from various assessments. How to Access Evidence for Practice: MOHO Website Additional Resources • This page provides information to request or download free resources, MOHO related web links, and contacts for MOHO assessments translated into other languages. Questions?