Abstract
Background and Purpose Our nation's suboptimal health care quality and unsustainable costs can be linked to the failure to implement evidence-based interventions. Implementation is the bridge between the decision to adopt a strategy and its sustained use in practice. The purpose of this case report is threefold: (1) to outline the historical implementation of an evidence-based quality improvement project, (2) to describe the program's future direction using a systems perspective to identify implementation barriers, and (3) to provide implications for the profession as it works toward closing the evidence-to-practice gap.
Case Description The University of Pittsburgh Medical Center (UPMC) Centers for Rehab Services is a large, multicenter physical therapy organization. In 2005, they implemented a Low Back Initiative utilizing evidence-based protocols to guide clinical decision making.
Outcomes The initial implementation strategy used a multifaceted approach. Formative evaluations were used repeatedly to identify barriers to implementation. Barriers may exist outside the organization, they can be created internally, they may result from personnel, or they may be a direct function of the research evidence. Since the program launch, 3 distinct improvement cycles have been utilized to address identified implementation barriers.
Discussion Implementation is an iterative process requiring evaluation, measurement, and refinement. During this period, behavior change is actualized as clinicians become increasingly proficient and committed to their use of new evidence. Successfully incorporating evidence into routine practice requires a systems perspective to account for the complexity of the clinical setting. The value the profession provides can be enhanced by improving the implementation of evidence-based strategies. Achieving this outcome will require a concerted effort in all areas of the profession. New skills will be needed by leaders, researchers, managers, and clinicians.
Improved health care value is a central theme of current reform efforts. The focus on value stems from our nation's well-chronicled problems of unwarranted clinical variation, substandard quality of care, and unsustainable health care costs. The failure of clinicians and health care organizations to assimilate evidence-based care strategies into the routine care of patients contributes significantly to maintenance of the status quo. These failures are not new and have been a barrier to high-quality care for close to 40 years.1 The scientific literature is replete with examples of evidence-based practice (EBP) guidelines, more effective evaluation and treatment methods, and redesigned care processes that have not been systematically incorporated into everyday clinical practice. This troubling obstacle to high value care pervades every corner of health care delivery, including physical therapy. Recently, the physical therapy profession has been “called to action” to demonstrate the value it creates or risk professional “irrelevance.”2 A proposed strategy to meet this challenge is to shrink the evidence-to-practice gap.
Increased attention to the “bench-to-practice” gap has resulted in proliferation of implementation research. Implementation is the bridge between a decision to adopt an evidence-based strategy and the sustained use of that strategy in everyday clinical practice. Implementation is an iterative activity requiring process evaluation, performance measurement, and cyclical refinement (Fig. 1). It is during this period that behavior change is actualized as clinicians and supporting staff become increasingly proficient, consistent, and committed to their use of new evidence.3 Implementation research, therefore, is aimed at identifying interventions that facilitate behavior change and speed the use of evidence-based strategies in clinical practice.
Iterative cycle of implementation. Adapted with permission of John Wiley & Sons Inc from: Graham ID, Logan J, Harrison MB, et al. Lost in knowledge translation: time for a map? J Contin Educ Health Prof. 2006;26:13–24.
The purpose of this case report is threefold. First, we will describe the 9-year evolution of a low back pain (LBP) quality improvement project in a multicenter physical therapy organization. We will specifically highlight our use of evidence-based implementation interventions and a continuous cycle of performance measurement and program refinement. Second, we will discuss the future directions of the initiative as we use a broader systems perspective to identify additional barriers to implementation. Finally, using insights gained through our organization's implementation experience, we will provide practical implications for the physical therapy profession as it seeks to demonstrate value by closing the evidence to practice gap.
Setting
The University of Pittsburgh Medical Center (UPMC) is an integrated health care finance and delivery system headquartered in Pittsburgh, Pennsylvania. It operates more than 20 hospitals, with more than 400 outpatient locations. The UPMC Centers for Rehab Services (CRS), a component of the Community Provider Services division of UPMC, provides all inpatient, long-term care and outpatient physical therapist services. The CRS has more than 60 outpatient sites throughout western Pennsylvania, with clinics located in urban, suburban, and rural environments.
The CRS employs more than 170 full-time and part-time physical therapists in their outpatient clinics. The average clinic is staffed by 3.3 full-time equivalents (FTE), but staffing ranges from 1.0 to 26.5 FTE per clinic. More than 80% of their clinics staff 3 or fewer FTE. Their physical therapy workforce is predominantly female (64%) and under 40 years of age (70%). Fifty-six percent of the therapists have achieved a doctor of physical therapy (DPT) degree, and 40% received their training at the University of Pittsburgh. They average 12.1 years in clinical practice and 7.0 years of employment with CRS.
Low Back Quality Improvement (LBQI) Initiative
The UPMC and its insurance division, UPMC Health Plan, identified LBP as a high-impact condition based on the prevalence and cost of care within their membership. As a result, an enterprise-wide, multidisciplinary low back initiative involving physicians, physical therapists, and the health plan was launched in 2005. The initiative had 3 main components: (1) a 72-hour algorithm to encourage early referral for physical therapy for patients with LBP, (2) dissemination of evidence-based management protocols at all levels of care, and (3) collection of detailed data regarding processes of care, clinical decision making, and physical therapist adherence to LBP management guidelines.
As part of the initiative, CRS executive leadership made the decision to adopt evidence-based management protocols and developed a comprehensive strategy for implementation. Goals were to standardize care, reduce unwarranted clinical variability, and improve outcomes for patients with LBP. The recommended management protocols reflected best available evidence and were consistent with current medical and physical therapy LBP clinical practice guidelines.4–6 The protocols were designed to assist therapists' decision making at the point of care using both patient-reported and provider-generated findings. Importantly, previous work has demonstrated that when care is aligned with evidence, patients achieve better functional outcomes, have lower physical therapy costs, and accrue fewer downstream costs (eg, medication, imaging, surgery).7–9
Initial Implementation—A Multifaceted Evidence-Based Strategy
The CRS' initial strategy included a multifaceted set of evidence-based implementation interventions. A combined approach is important in light of research suggesting multifaceted strategies, those including 2 or more implementation interventions, are more effective in producing improvement than isolated interventions.10 It should be noted that more interventions are not necessarily better, as effectiveness is not a linear function of the number of interventions used.11 Instead, it is more important that each component of a multifaceted strategy target different barriers to implementation. The synergy created by this approach has been identified as an important factor for success.12 In this case, the initial implementation strategy included 4 interventions: (1) local consensus, (2) implementation champions, (3) educational meetings, and (4) printed educational material.
Local Consensus Process
Local consensus processes are designed to create a sense of ownership within the clinical setting. The approach includes local providers in the decisions made regarding management of the targeted clinical condition. Barriers to committed use of a clinical innovation can arise when outside standards are imposed without customizing the program to meet the unique needs of the organization.3,13 Prior to rollout of the program, CRS convened a series of meetings to build consensus within the organization. These meetings included LBP experts from the University of Pittsburgh, clinicians, and quality improvement representatives from CRS. The primary objectives of these meetings were to discuss low back initiative management protocols, to agree on specific clinical decision-making algorithms, and to ensure the protocols could be feasibly deployed within the CRS clinics.
An electronic monitoring tool that would capture patient and process-of-care data was deemed critical to the success of the program. Therefore, once agreement on the clinical aspects was reached, the consensus development team was expanded to include representatives from UPMC's information technology group. The monitoring tool, or minimum data set (MDS), captures patient demographics, LBP history, and summary scores from the Fear-Avoidance Beliefs Questionnaire (FABQ) and modified Oswestry Disability Index (mODI). The MDS also collects key examination findings, process-of-care information, and the physical therapist's plan of care (Appendix 1). The MDS data are collected at the point of care and then uploaded by the therapist or rehabilitation aide into a centralized, secure database where the data can then be analyzed using a proprietary computerized algorithm to determine the physical therapist's patient-specific and overall adherence to the LBP management protocols.
Implementation Champions
Implementation champions are individuals who advocate advancement of the new clinical change.13 Champions are particularly helpful overcoming indifference or resistance encountered from clinicians or staff. The CRS' Spine Program Director (SPD) was the company clinical champion. The SPD is the recognized spine care expert within the organization. He serves as the primary resource for therapists' diagnostic and treatment questions and provides individualized clinical skills training when needed.
Educational Meetings
Eight regional educational meetings were held prior to the LBQI program launch. These meetings were led by CRS' Vice President of Research and Education and supported by the SPD. The didactic sessions provided an overview of the initiative, a detailed discussion of the management protocols, and a description of the operational procedures. Educational meetings are commonly used to improve professional practice. An important distinction must be made between didactic meetings and interactive workshops. Didactic educational sessions target knowledge barriers, whereas interactive workshops target knowledge, attitudes, and skill barriers at the individual and group levels.14 Current research shows a mix of interactive and didactic education appears to be more effective than either alone.14
Printed Educational Materials
The educational sessions were augmented with printed educational materials documenting the management protocols and operational procedures. Printed educational materials are meant to improve awareness, knowledge, attitudes, and skills and ultimately improve clinical practice.15 Printed materials may be an important component of a multifaceted implementation strategy; however, when used in isolation, they have a minimal effect on professional practice.15
Evaluation of the Initial LBQI Implementation
Implementation of new intervention strategies is a process, not an event (Fig. 1). Therefore, managing the implementation requires strong leadership, robust monitoring systems, and the willingness to adapt and tailor the program to overcome barriers that inevitably surface.16 Measurement of implementation fidelity, or the degree to which a strategy is delivered as intended, is critical in understanding how the evidence-based strategy is actually being used.17 By comparing actual versus planned implementation activities, it is possible to identify and resolve actionable barriers and maximize facilitators of behavior change.18
Formative evaluations are an effective mechanism to identify factors that positively or negative influence the progress of implementation efforts.18 Formative data may include quantitative measures collected from structured survey instruments or administrative data. As part of formative evaluations, it is also common to gather qualitative information through direct observation, chart review, or formal and informal interviews. The information obtained is used to make midcourse refinements to the processes in an effort to increase the likelihood of success.19 Throughout the LBQI implementation, CRS has used quantitative and qualitative inputs from formative evaluations to monitor and improve the program. Using these evaluations, the LBQI has progressed through 3 distinct improvement cycles.
Improvement Cycle 1
The CRS' first structured formative evaluation revealed a lack of compliance with data submission procedures. As stated earlier, a key component of the program was the development of a monitoring tool (ie, MDS) designed to capture clinical data, process-of-care information, and the physical therapist's plan of care. These data elements are considered mission critical, as they are used to determine physical therapists' adherence to the LBP management protocols. However, this process required therapists to upload the data collected at the point of care into the centralized database. Initial compliance with to this additional administrative task (ie, data submission) was found to be extremely low (∼18%).20 This low level of data submission prevented analysis of process of care and algorithm adherence metrics, thereby creating a significant barrier to further program evaluation. To specifically address this barrier, greater oversight of the data submission process was accomplished by implementing an audit and feedback system.
Audit and Feedback
Audit and feedback can be integral to the implementation process. Previous research has shown that health professionals substantially overestimate their actual performance when their self-rating is compared with empirically derived performance measures.21 Audit and feedback interventions objectively measure process, clinical, or health care outcomes and compare a clinician's performance with professional standards or predefined targets.12,22 Results of the comparisons are then provided to the clinician in an effort to stimulate behavior change when performance does not meet standards.12,22 This technique has the potential to produce significant change in professional behavior and is most effective for low performing providers.22
For the next implementation cycle, the Director of Quality and Risk Management was given oversight of the audit and feedback intervention. Her role as the initial point of contact for personnel, processes, and systems questions allowed her full access to the data and the ability to send individual, bimonthly compliance reports to each clinician. These reports provided 2 measures of data submission compliance: (1) the proportion of each therapist's LBP patient population for whom an MDS was submitted and (2) the proportion of MDS submissions in which all required data fields were completed. By bringing attention to data compliance through audit and feedback, the data submission and completion rate was successfully raised from 18% to over 90% within less than 2 months20 (Fig. 2).
Minimum data set (MDS) submission compliance. Blue bars indicate submitted MDS; orange bars indicate complete MDS.
Improvement Cycle 2
Improved data submission resulting from the audit and feedback intervention allowed more rigorous exploration of therapists' adherence to the LBP management protocols as well as the cost consequences of adherent versus nonadherent care. To do this, we studied a cohort of 750 consecutive adult patients with LBP treated from November 2007 through October 2008. A validated computer algorithm was developed to analyze each patient's clinical status using data from the MDS and to determine whether therapists treated their patients according to the LBP management protocols. Definitions for patient classification and the criteria used for determination of adherent and nonadherent status are presented in Appendix 2.20 In addition to therapists' adherence to the treatment protocols, we also investigated total and component treatment costs (ie, medical, therapy, prescription) for these patients using claims data from the UPMC Health Plan.
Our analysis revealed that, overall, therapists' initial plans of care adhered to the LBP management protocols approximately 50% of the time.20 This adherence rate was well below the desired 80% compliance target. There was also substantial variation in the therapists' adherence rates according to the 3 clinical classifications: (1) directional preference exercise (81%), (2) stabilization exercise (49%), and (3) application of thrust mobilization (manual therapy) (14%)20 (Fig. 3).
Low back management protocol adherence—October 2007–July 2009. Blue bars represent percentage protocol adherent (n); orange bars represent percentage protocol nonadherent (n).
Despite suboptimal and inconsistent adherence rates, our analysis suggests that protocol adherent treatment was cost saving for both patients and the UPMC Health Plan. For example, the mean per episode cost was 47% higher, and the overall costs were 43% more for those patients receiving nonadherent care ($941,897.55) than for those receiving adherent care ($658,477.94).20 Additionally, physical therapy costs were 14% higher and patients spent 35% more out-of-pocket when care did not adhere to the management protocols20 (Table). Perhaps even more importantly, nonadherent care was predictive for incurring total treatment costs in the highest quarter of the sample after adjusting for baseline characteristics (adjusted odds ratio=1.51; 95% confidence interval=1.07, 2.15).20 In light of the apparent value produced for patients and the payer, further program improvements were justified. In the next improvement cycle, CRS targeted barriers preventing proper execution of the protocols using a comprehensive educational strategy.
Mean Total Health Care, Physical Therapy, and Patient Out-of-Pocket Costs per Episode of Carea Based on Low Back Quality Initiative Management Protocol Adherence
Based on the highly variable adherence patterns identified in our analysis, it appeared that skills training (eg, manual therapy techniques) was required, especially for new employees. As a result, an educational intervention combining interactive clinical skills laboratory sessions and didactic educational components was implemented. During this implementation cycle, a combined approach was chosen because, as stated previously, mixed educational approaches target barriers in knowledge, attitudes, and skills and are more effective than either approach alone.14
The centerpiece of the educational intervention was an 8-module, self-directed online curriculum. The course used multimedia presentations to provide comprehensive educational content on all aspects of the LBP management protocols. Learning was reinforced with pretesting and posttesting, which also allowed continuing education credits to be granted upon successful completion. Completion of this course is now a requirement for all new employees. To address motor competencies, new employees also were required to attend a spine care skills training course led by the SPD. An electronic LBQI library was established on the corporate Intranet to provide easy access to program resources. Unfortunately, further analysis has revealed that this educational intervention did not demonstrably improve adherence to the management protocols.
Improvement Cycle 3—Moving to a Systems View
Despite a multifaceted implementation strategy and 2 distinct improvement cycles, adherence to the LBP management protocols has not yet been fully optimized. The inability of the organization to achieve targeted adherence rates indicates that other levers of change are needed to improve performance. Rather than abandoning the process, the UPMC Health Plan and CRS continue to partner because of the potential savings for the plan and its members. Barriers to the sustained use of evidence-based strategies can be multifactorial and context specific. However, the interventions previously deployed have primarily targeted one domain within the system: the physical therapists' behaviors. Thus, CRS' next improvement cycle is expanding the investigative window by conducting a systems-level formative evaluation. The goal is to develop a tailored implementation prescription that can account for barriers in multiple domains.
The formative evaluation is guided by the Consolidated Framework for Implementation Research (CFIR), a meta-theoretical framework ideally suited for multilevel systems evaluations.13 The CFIR defines 5 domains that are all potential targets for tailored implementation strategies23:
Outer setting (eg, patient needs, peer pressure, external policies, and reimbursement);
Inner setting (eg, leadership, culture, networks and communication, goals and feedback);
Characteristics of individuals (eg, knowledge and beliefs, personal attributes);
Characteristics of the clinical innovation (eg, evidentiary strength, complexity); and
Implementation process (eg, planning, opinion leaders, champions).
This broader diagnostic evaluation will use sequential quantitative and qualitative methods. In the current improvement cycle, we are using structured surveys to gather information from the therapists specific to CFIR domains 1 through 4 (see above). We are assessing inner setting factors (eg, leadership, culture, resources) with the Alberta Context Tool.24–26 The LBP management protocols are being assessed using the Ottawa Acceptability of Decision Rules Instrument.27 Physical therapists' attitudes and beliefs toward LBP will be investigated using the Pain Attitudes and Beliefs Scale for Physical Therapists.28,29 These findings, when combined with data from the monitoring system, will allow the quantification of factors associated with variation in adherence. These data also may point to potential barriers and facilitators to implementation within and between system domains. The second step in the evaluation will use semistructured interviews with both high- and low-adhering therapists to thoroughly examine their views regarding potential barriers and facilitating factors identified in the first phase of the evaluation. The goal is to use the findings from these studies to design a tailored implementation prescription targeting the important mutable barriers to adherence at multiple levels within the system.30
Bundled Payments
In addition to the systems-level diagnostic evaluation, there has been a recent change external to CRS (ie, CFIR domain 1) that may facilitate greater adherence to the management protocols. The UPMC Health Plan has launched a pilot program to institute a bundled payment for patients with LBP. The UPMC's interest in this reimbursement mechanism stems from barriers created by the traditional fee-for-service (FFS) payment structure, which rewards volume over quality.31 Bundled payments pay a provider a single price for all of the services needed by a patient for a specific episode of care or over a fixed time period. Bundled payments are often described as a hybrid between FFS reimbursement (in which providers are paid for each service rendered to a patient) and capitation (in which providers are paid a “lump sum” per patient regardless of how many services the patient receives).32 Fee-for-service payments leave all of the risk for utilization and costs on the payer. Conversely, capitation payments shift almost all of the risk to the provider.33 Bundled payments are viewed as a “middle ground” in which risk is shared. These programs have shown promising results for enhancing guideline adherence and improving outcomes for certain conditions, but models in physical therapy are still untested.34
A test of a bundled payment model for select LBP diagnoses is currently under way. The bundled payment covers patients with nonspecific LBP for up to 8 weeks of care. As part of the bundled payment, the patient is only required to pay a single co-payment for the entire 8-week episode of care. This single co-payment is designed to significantly reduce patients' initial cost burden and has the potential to enhance compliance, thereby improving clinical outcomes. It is too early to judge the effectiveness of the third improvement cycle, but early indications point to increased compliance with LBP management protocols.
Implications for the Physical Therapy Profession
The move toward value-based health care is upon us and accelerating rapidly. This massive paradigm shift will require unprecedented transformation by providers. Providers will be held accountable for the quality of the service delivered as payers initiate broad payment reforms, including performance-based reimbursements. In response, physical therapists' use of evidence-based treatment strategies will need to become the norm rather than a novelty.
The inescapable conclusion is that the profession, as a whole, must improve implementation and dissemination of evidence-based care strategies. Adherence to evidence-based care can improve quality, but compliance with evidence-based guidelines by physical therapists, like members of all other professions, is suboptimal. The increased attention to quality improvement and implementation can create an atmosphere that identifies the active ingredients of implementation interventions and provides insight into factors that mediate or moderate its success. The insights gained also can allow generalization across the different contextual settings of physical therapist practice.
In the clinics, new skills will be needed by clinicians and managers. Successful incorporation of evidence into routine practice will require an understanding of the complexity of the clinical setting from a systems perspective. As demonstrated in our organization, educational interventions alone will not be enough to gain the sustained and committed use of evidence-based care by clinicians. Instead, education should be a component of a diverse aggregation of implementation strategies tailored to the specific “systems” of each organization. Additionally, we must scale our expectations, remembering that implementation is an iterative process rather than a single event. These process changes will require vision, leadership, commitment, and perseverance from all members of an organization.
Finally, measurement, reassessment, and refinement will be imperative components of the implementation process. The link between outcomes and value will require a robust data monitoring program, ideally linking evidence-based care processes with patient outcomes. Electronic medical records must be easily accessible and provide meaningful reports to clinicians and their managers. This transformation to a data-driven profession can improve the use of evidence-based strategies and potentially provide quantifiable evidence highlighting the importance of consumer access to physical therapy in a high-value health care system.35
Appendix 1.
University of Pittsburgh Medical Center (UPMC) Centers for Rehab Services Low Back Pain Minimum Data Set (MDS) Forma
a PT=physical therapist, DOB=date of birth, LBP=low back pain, N/A=not applicable, FABQ=Fear-Avoidance Beliefs Questionnaire, FABQ–PA=Fear-Avoidance Beliefs Questionnaire–physical activity subscale, FABQ–W=Fear-Avoidance Beliefs Questionnaire–work subscale, Avg SLR=average straight leg raise, ROM=range of motion, NMES=neuromuscular electrical stimulation. This document is property of UPMC Centers for Rehab Services and may not be reproduced without expressed written consent (05/12).
Appendix 2.
Definitions for Patient Classification and Criteria Used for Determination of Adherent and Nonadherent Statusa
a MDS=minimum data set, LBP=low back pain, N/A=not applicable, FABQ=Fear-Avoidance Beliefs Questionnaire, FABQ–PA=Fear-Avoidance Beliefs Questionnaire–physical activity subscale, SLR=straight leg raise. This document is property of UPMC Centers for Rehab Services and may not be reproduced without expressed written consent (05/12).
Footnotes
All authors provided concept/idea/project design. Dr Stevans, Dr Bise, Dr McGee, Ms Miller, and Dr Delitto provided writing. Dr Stevans, Dr McGee, and Ms Miller provided data collection and analysis. Dr Stevans, Ms Miller, Dr Rockar, and Dr Delitto provided project management. Dr Rockar provided patients. Dr Rockar and Dr Delitto provided facilities/equipment, institutional liaisons, and administrative support. Dr McGee, Ms Miller, Dr Rockar, and Dr Delitto provided consultation (including review of manuscript before submission).
The authors would like to acknowledge the late Michael Culyba, MD, Vice President of Medical Affairs, UPMC Health Plan, who shared the vision and was instrumental in launching the Low Back Quality Improvement Initiative (LBQI). They also would like to acknowledge Nuket Curran, Director of Quality and Risk Management, UPMC Centers for Rehab Services, for her ongoing oversight and tireless efforts to continuously improve the LBQI.
- Received November 12, 2013.
- Accepted January 2, 2015.
- © 2015 American Physical Therapy Association
References
Issue highlights
- Article
- Abstract
- Setting
- Low Back Quality Improvement (LBQI) Initiative
- Initial Implementation—A Multifaceted Evidence-Based Strategy
- Evaluation of the Initial LBQI Implementation
- Improvement Cycle 1
- Improvement Cycle 2
- Improvement Cycle 3—Moving to a Systems View
- Bundled Payments
- Implications for the Physical Therapy Profession
- Appendix 1.
- Appendix 2.
- Footnotes
- References
- Figures & Data
- Info & Metrics