The Role of Professional Judgment and Client Values, Preferences, and Context: A TUTORIAL. Overview. Evidence-based Practice 6/8/13

Transcription

1 The Role of Professional Judgment and Client Values, Preferences, and Context: A TUTORIAL Trina D. Spencer, PhD, BCBA-D, LBA INSTITUTE for Human Development Overview Ø Explore the research to practice gap. Ø Client values, preferences, and context ü Strain, Barton, & Dunlap, 2012 ü Recommendations Ø Professional judgment ü Spencer, Petersen, & Gillam, 2008 ü Recommendations Ø Ethical guidelines Evidence-based Practice Ø Evidence-based practice is a means for narrowing the research to practice gap. Ø But what caused/causes this gap? 1

2 Research to Practice Gaps (Carnine, 1995) Ø Trustworthiness ü Confidence practitioner has of the evidence Ø Usability ü Demands required to implement Ø Accessibility ü Ease of contacting research recommendations Best Available Evidence Progress Ø Trustworthiness ü Noticeable improvement in quality E.g., journals, published standards, funders Ø Usability ü More attention on feasibility and usability E.g., implementation science, IES requirements Ø Accessibility ü Practitioner friendly materials E.g., practice guides, research summaries, briefs EBP is more than BAE Ø Although we have made progress in the best available evidence, we have not made parallel progress in the selection, implementation, and sustainability of empirically supported interventions. ü Treatment fidelity = weak link (Detrich & Lewis, 2012). Ø We know little about what to do when the best available evidence is not ü as high quality as we would like ü as plentiful as we would like ü relevant to our consumer(s) 2

3 EBP Areas of Need Ø Values, Preferences, and Context ü Cultural and personal variables ü Professional and organization variables ü Interprofessionalism Ø ü What is it and why do we care? ü Implementation and sustainability ü Decision making as a skill Values, Preferences, and Context Client or Consumer Ø Individual who is the focus of the behavior change Ø Individual(s) who are invested in outcomes and/or are critical to the behavior change process ü Parents and family members, teachers, mentors, colleagues, employers ü Organization, society 3

4 Contextual Fit (Albin et al 1996) Ø Values Ø Preferences Ø Characteristics Ø Culture Ø Goals Ø Skills Ø Barriers Ø There is no reason to assume that an intervention will be effective in all contexts with all consumers. Ø Context and fit need to be intentionally included in the decision making process. Historical Perspective Ø Baer, Wolf & Risley (1968) ü Immediate and important change in behavior that has practical value as determined by the interest which society shows in the problems. (p. 92) Ø Social Validity, Wolf (1978) ü Consumers selection of intervention targets. Social meaning ü Consumers compatibility with intervention tactics. Treatment acceptability ü Consumers evaluation of intervention impact. Consumer satisfaction Decision Making Framework Client and Context! Spencer, Detrich, & Slocum,

5 Strain, Barton & Dunlap (2012) Ø Delineate the importance of values, preferences, and context for evidencebased practice. Ø Consumer-oriented perspective revealed unique information that on occasion was contrary to conventional wisdom. Ø Demonstrate that choosing an intervention with prior efficacy data does not ensure effectiveness with the current case. Selection of Targets Example Ø LEAP ü behavioral skill training for families ü young children with autism Ø Clinicians ü self help skills, communication Ø Families ü completing routines in a timely fashion Ø Comfort-level and confidence as outcomes Ø Families achieved behavior change long before they reported feeling comfortable. Ø Continue support until families are confident. Ø Sustainability. 5

6 Selection of Strategies Example Ø Prevent Teach Reinforce (PBIS Model) Ø Input from primary intervention agent throughout every step of the process. Ø Strategy menu at least one from ü Prevention strategies ü Teaching strategies ü Reinforcement strategies Ø Choose based on beliefs about effectiveness, assessment data, and congruence with preferences, values, and routines of implementer. Findings Ø Ratings of teacher attitudes regarding feasibility and efficacy ü 1-5 scale on 15 items ü Average score: 4.16 Ø Fidelity of implementation ü 80% of the teachers implemented PTR plans accurately. Consumer Satisfaction Example Ø Incredible Years (IY) caregiver training Ø Positive parenting behaviors ü Responsive play ü Descriptive praise ü Limit setting ü Appropriate responses to problem behavior Ø Results parenting behaviors only moderately improved Ø Caregivers reported improvements in their children s behavior and highly satisfied with the intervention, despite modest change in their own behavior Ø Satisfied parents are likely to continue with intervention and eventually improve parenting 6

7 RECOMMENDATION Use the EBP decision making framework to establish reciprocity with consumers, families, and professionals. EBP as glue and goal Ø In EBP, decisions are not ours alone!!! Ø Most human service organizations have adopted the broad definition of evidence-based practice. Ø EBP can be the process (glue) for working effectively with people who have different perspectives. Ø EBP is readily accepted as the goal of any team-based effort. Posture of Cultural Reciprocity Ø A process of introspection and inquiry that not only questions the assumptions of the field (e.g., ABA, SLP, SPED) but also forces professionals to confront the contradictions between their values and practices. Ø reflective practice ü A way of life not a checklist to perform ü Interactive and respectful relationship between people with different perspectives (i.e., from different cultures) ü Don t have to be from their culture to work effectively with them. 7

8 Posture of Cultural Reciprocity Step 1: Identify the values that are embedded in your recommendations. Step 2: Find out whether the family, individual being served, and other professionals recognize and value these assumptions and, if not, how their view differs from yours. Step 3: Acknowledge and give explicit respect to any differences identified, and fully explain the basis of your assumption. Work to establish common goals. Step 4: Through discussion and collaboration, set about determining the most effective way of adapting recommendations in consideration of the family s (or professional s) value system so that the common goals can be achieved. Harry, Rueda, & Kalyanpur, 1999 Posture of Cultural Reciprocity Ø All parties are engaged in dialogue where each learns from the other. Ø A framework for working through diversity as long as we have the client s (not ours) best interest in mind. What does this look like with families? Ø Before making any recommendations or providing any services, explicitly offer the EBP framework (glue and goal), giving everyone an important role. Ø Explain your values and how you would like to work with them. Ø Find out what they value. Ø As you offer recommendations, overtly state how you ve considered their values. Ø Bring the evidence! 8

9 What does this look like with families? Ø If your recommendation is not accepted or the family chooses to do something different, remind them of the EBP framework and together map out the sources influencing the decision. Ø Offer to help evaluate the effectiveness of the alternative intervention. ü Establish a monitoring system Ø Ensure it is not detrimental to client. What does that look like with other professionals? Ø Learn a foreign language! 2.10 ü Read their journals, understand their theories, and go to their conferences. Ø (c) Behavior analysts are responsible for review and appraisal of likely effects of all alternative treatments including those provided by other disciplines and no intervention. BACB, 2010 What does this look like with other professionals? Find a passionate, patient, and honest teacher (with a password). Commit to something together e.g. data. Ø Have many reciprocating discussions. ü Seek to understand ü Not just to be understood Ø Explain the assumptions and bias behind your recommendations and interventions. Ø Make it OBVIOUS that your disagreements are philosophical and NOT personal. 9

10 Ø Process by which the best available evidence is applied to individual cases in specific contexts. Ø An evidence-based practice framework provides an opportunity for human service fields to better understand the role of professional judgment. ü Should not be neglected!!! Evidence-based Practice Framework Client and Context! Spencer, Detrich, & Slocum,

11 Client and Context! Ø Hudson engages dangerous behaviors such as climbing houses to get attention and because climbing is fun. What treatment should I use? Client and Context! Ø Search for a systematic review or studies on treatments for dangerous behaviors that involve attention and automatic reinforcement functions. Client and Context! Ø Differential reinforcement of other behaviors and a regular routine of appropriate replacement activities. 11

12 Client and Context! Ø For every day that Hudson does not engage in dangerous behaviors, he earns 30 minutes of a special activity. In addition, he participates in a rock climbing club for children every week. Client and Context! Ø Hudson can monitor his dangerous behaviors and record their absence on a calendar. He can report to his mother every evening and request 30 minutes of a special activity. Spencer, Detrich, & Slocum,

13 Ø Practitioners use professional judgment every day and all day. Ø These judgments are a necessary part of effective practice. Ø The pervasiveness of professional judgment suggests that professionals should be very skilled at it and that we should know a lot about it. Ø Where does PJ come from? ü Art of practice ü Professional wisdom developed purely through experience (Kamhi, 1994; Whitehurst, 2002). ü Clinical opinion Ø Vulnerable to bias and errors Ø Too simplistic Ø Should not override evidence Ø APA Task Force (2005) on Evidencebased Practice described eight professional judgment competencies. 1. Formulating the problem so that treatment is possible 2. Making clinical decisions, implementing treatments, & monitoring progress 3. Interpersonal expertise 4. Continuous development of professional skills 5. Evaluating research evidence 6. Understanding the influence of context on treatment 7. Utilizing available resources 8. Having a cogent rationale for treatment 13

14 RECOMMENDATION Establish a scientific basis for professional judgment and commit to teaching/learning professional judgment skills. Bases of Ø Professionals can draw from the available evidence to strengthen professional judgments (scientific basis). Principles of learning and behavior Ongoing progress monitoring Ø Problem behavior = personal opinion PJ Ø Replacement behavior = data based PJ Bases of Ø Principles of behavior, grounded in decades of experimental and applied research, serve as a foundation for judgments made by professionals. Ø Professionals can identify the principles that contribute to specific decisions (e.g., active student responding, rapid pace, differential reinforcement, contingent and corrective feedback, systematic scaffolding and prompt fading). 14

15 Bases of Ø Most of our decisions can and should be informed by close and continual contact with relevant outcome data. ü This does not mean that we do not use professional judgment. ü It means our judgments are data-based. Ø Ongoing progress monitoring is THE best evidence of what works with specific individuals in specific contexts regardless of the strength of the empirical research. Ø Given professional judgment is ubiquitous, necessary and it can have a scientific basis, human service disciplines should embrace it, develop it, and promote it. Ø Doing so, has implications for training and professional development. Professionals vs. Technicians Ø Professionals ü Masters or doctoral degrees ü Supervised field experiences ü Licensing requirements Ø Technicians ü Bachelors or less education ü No supervised experiences ü No license 15

16 Research to practice gap is largest in education! Ø Fundamental disagreements with science Ø Minimal training in empirically supported interventions Ø Minimal training in research methods Ø Minimal training in principles of effective instruction Ø Walsh, Glaser, & Wilcox (2006) Ø Klinger, Vaughn, Hughes, & Arguelles (1999) Ø Baker, Gersten, Dimino, & Griffiths (2004) Ø Fuchs, Fuchs, & Stecker (2010) Ø Levine, (2006) Ø Reschley, Holdheide, Behrstock, & Weber (2009) PJ training and professional development are needed Ø Professional judgment can be developed and enhanced via quality preservice training and supervised field experiences. ü Teachers are only as effective as they know how to be. Train in effective methods, principles of effective instruction, and how to consume research. ü Thinking for others engenders resentment. ü Data rich and decision poor. Train problem solving, evidence-based decision making, and analytical skills. Spencer, Petersen, & Gillam (2008) Ø 3 Case Examples Ø Sign or PECS? Ø Decision making process 1. Develop parameters to guide search 2. Search for evidence 3. Evaluate the certainty and relevance of evidence 4. Consider student/family variables 5. Consider teacher/school variables 6. Integrate evidence 7. Make a selection 16

17 Summary of Evidence Ø Both PECS and Sign Language had sufficient research to support them as choices for young children w/ autism. Ø Sign Language had more studies suggesting that its use leads to vocal speech. Ø Learner characteristics such as imitation and fine motor skills may be critical to the acquisition of signs. Juan Ø Imitation of fine motor movements Ø Limited speech, some vocalizations Ø Difficulty with matching and discriminating between pictures Ø Parents are willing to try anything team recommends Ø Specialized preschool class with aide Ø No barriers to either PECS or sign Stephanie Ø No vocal or motor imitation Ø Tantrums and hits to communicate Ø Low intellectual abilities Ø Does not match basic objects/pictures Ø Parents apprehensive about learning another language Ø PECS is used by some of her peers in the special education class Ø School district has a specialized AAC trained SLP 17

18 Ø Low intellectual abilities David Ø No vocal or motor imitation Ø Does not use speech to communicate Ø Minimal social interaction Ø Follows simple 1-step directions and matches picture to picture Ø David s cousin uses sign language Ø Parents requested that the classroom use sign language Ø SLP and OT believe David is a candidate for a picture based assistive device Selections Ø Juan sign language Ø Stephanie PECS Ø David Disagreed ü Family wanted sign language ü Staff believed PECS would be better ü Agreed to try PECS for 3 months and reconvene to evaluate progress ü At 3 month mark, switched to sign language Ethics and EBP Ø The philosophy of EBP emphasizes that ethical and evidence considerations are intertwined. ü Informed consent ü Right to choose ü Right to effective treatment ü Do not harm Ø Transmitted through professionals 18

19 1.01 Responsible Conduct for Behavior Analysts Ø Behavior analysts rely on scientifically and professionally derived knowledge when making scientific or professional judgments. BACB, Responsible Conduct for Behavior Analysts Ø (a) The behavior analyst always has the responsibility to recommend scientifically supported most effective treatment procedures. Ø (b) Clients have a right to effective treatment (i.e., based on the research literature and adapted to the individual client). BACB, Responsible Conduct for Behavior Analysts Ø The behavior analyst (a) designs programs based on behavior analytic principles, (b) involves the client or the client-surrogate in the planning of such programs BACB,

20 4.08 Responsible Conduct for Behavior Analysts Ø The behavior analyst modifies the program on the basis of data. BACB, Responsible Conduct for Behavior Analysts Ø (d) In those instances where more than one scientifically supported treatment has been established, additional factors may be considered in selecting interventions, including, but not limited to, efficiency and cost-effectiveness, risks and side-effects of the interventions, client preference, and practitioner experience and training. BACB, Caution! Ø (d) In those instances where more than one scientifically supported treatment has been established... ü There is no definition of scientifically supported in the BACB guidelines. ü What amount and type of evidence is sufficient? ü For many practical questions, there is not even one treatment with documented (through a systematic review) high certainty and high relevance. BACB,

21 Ethical EBP TABLE 2: Evidence-Based in Substance or Name Only? Evidence-Based In Substance Evidence-Based In Name Only Questions researchers focus on come from clients or direct line staff. The evidentiary status of services or programs is clearly described. Rigorous criteria are used to evaluate the evidentiary status of services. Direct line staff and supervisors are provided the training and tools needed for evidence-informed decisions. Evidentiary status is a key factor in purchasing services. Clients are fully informed regarding the risks and benefits of recommended services and of alternatives. Training programs offered use formats that maximize learning and focus on information needs and skills directly related to decisions made by line staff. Questions focused on are selected by researchers or administrators. The evidentiary status of service programs is hidden or misrepresented. Nonrigorous criteria are used to evaluate services. Neither line staff nor supervisors have skills or resources required to make evidence-informed decisions. Services are purchased based on availability and popularity. Clients are involved as uninformed or misinformed participants. Training programs are selected based on entertainment value and popularity. Gambrill,

22 Role of and Client Values, Preferences, and Context References Albin, R. W. Lucyshyn, J. M., Horner, R. H., & Flannery, K. B. (1996). Contextual fit for behavior support plans: A model for goodness of fit. In L. Koegel, R. Koegel, & G. Dunlap (Eds.), Positive behavior support: Including people with difficult behavior in the community (pp ). Baltimore, MD: Paul H. Brookes. APA Presidential Task Force of Evidence-Based Practice. (2006). Evidence-based practice in psychology. American Psychologist, 61, Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimension of applied behavior analysis. Journal of Applied Behavior Analysis, 1, Baker, S., Gersten, R., Dimino, J., & Griffiths, R. (2004). The sustained use of research-based instructional practice: A case study of peer-assisted learning strategies in mathematics. Remedial and Special Education, 25, Carnine, D. (1995). Trustworthiness, Useability, and Accessibility of Educational Research. Journal of Behavioral Education, 5(3), Detrich, R., & Lewis, T. (2012). A decade of evidence-based education: Where are we and where do we need to go? Journal of Positive Behavior Interventions. Online First, 1-7. Dunlap, G. Iovannone, R. Kincaid, D., Wilson, K., Christiansen, K., Strain, P., & English, C. (2010). Prevent-Teach-Reinforce: A school based model of positive behavior. Baltimore, MD: Paul H. Brookes. Fuchs, D., Fuchs, L. S., & Stecker, P. M. (2010). The blurring of special education in a new continuum of general education placements and services. Exceptional Children, 76(3), Gambrill, E. (2006). Evidence-based practice and policy: Choices ahead. Research on Social Work Practice, 16(3), Harry, B., Rueda, R., & Kalyanpur, M. (1999). Cultural reciprocity in sociocultural perspective: Adapting the normalization principle for family collaboration. Exceptional Children, 66(1), Klingner, J.K., Vaughn, S., Hughes, M.T., & Arguelles, M.E. (1999). Sustaining research-based practices in reading: A 3-year follow-up. Remedial and Special Education, 20, Levine, Arthur. (2006). Educating School Teachers. The Education Schools Project. Princeton, NJ.

23 Reschly, D. J., Holdheide, L. R., Behrstock, E., & Weber, G. (2009). Enhancing teacher preparation, development and support. In L. R. Goe (Ed.), America s opportunity: Teacher effectiveness and equity in K-12 classrooms (pp ). Washington DC: Learning Point Associates, National Comprehensive Center on Teacher Quality. Spencer, T. D., Detrich, R., & Slocum, T. (2012). Evidence-based practice: A framework for making effective decisions. Education and Treatment of Children, 35, Spencer, T. D., Petersen, D. B., & Gillam, S. L. (2008). Picture exchange communication system (PECS) or sign language: An evidence-based decision making example. Teaching Exceptional Children, 41(2), Strain, P. S., Barton, E. E., & Dunlap, G. (2012) Lessons learned about the utility of social validity. Education and Treatment of Children, 35, Walsh, K., Glaser, D., & Wilcox, D. D. (2006). What education schools aren t teaching about reading, and what elementary teachers aren t learning. Washington, DC: National Council on Teacher Quality. Webster-Stratton, C. (2008). The Incredible Years: Parents and children series. Leader s guide: Preschool version of BASIC (ages 3-6 years). Seattle, WA: The Incredible Years. Wolf, M. (1978). Social validity: The case for subjective measurement, or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11,