Abstract
Background and Purpose Pediatric physical therapists face many challenges related to the application of research evidence to clinical practice. A multicomponent knowledge translation (KT) program may be an effective strategy to support practice change. The purpose of this case report is to describe the use of a KT program to improve the knowledge and frequency of use of standardized outcome measures by pediatric physical therapists practicing in an outpatient clinic.
Case Description This program occurred at a pediatric outpatient facility with 1 primary clinic and 3 additional satellite clinics, and a total of 17 physical therapists. The initial underlying problem was inconsistency across staff recommendations for frequency and duration of physical therapist services. Formal and informal discussion with the department administrator and staff identified a need for increased use of standardized outcome measures to inform these decisions. The KT program to address this need spanned 6 months and included identification of barriers, the use of a knowledge broker, multiple workshop and practice sessions, online and hard-copy resources, and ongoing evaluation of the KT program with dissemination of results to staff. Outcome measures included pre- and post-knowledge assessment and self-report surveys and chart review data on use of outcome measures.
Outcomes Participants (N=17) gained knowledge and increased the frequency of use of standardized outcome measures based on data from self-report surveys, a knowledge assessment, and chart reviews.
Discussion Administrators and others interested in supporting practice change in physical therapy may consider implementing a systematic KT program that includes a knowledge broker, ongoing engagement with staff, and a variety of accessible resources.
The definition of evidence-based practice suggests that clinical decision making should be guided, in part, by the best available scientific evidence.1 However, an extensive body of literature indicates that physical therapists have difficulty accessing, analyzing, and applying research evidence.2–9 Knowledge translation (KT) strategies have emerged as a potential solution to the challenges of consistently using research evidence to guide clinical practice and to support optimal change in clinical practice behaviors. Knowledge translation is defined as the exchange, synthesis, and ethically sound application of knowledge within a complex system of interactions among researchers and users.10,11 The emphasis is on application and on consideration of the multiple factors that may influence whether the application is successful and sustained.
The knowledge-to-action framework developed by Graham and colleagues12 provides a template for consideration of a number of key factors. Successful KT is more likely if the relevance of the new knowledge is clear to the user.13 Increasing accessibility also enhances the success of KT by ensuring timely access to the relevant research evidence in the form of actionable messages, policy recommendations, or suggested implementation strategies. The format and method in which new knowledge is shared with new users can have a direct impact on its perceived value and the likelihood it will be used in practice. Ensuring that knowledge is clearly and concisely presented increases probability of use. Effective presentation strategies include educational outreach visits, reminders of research findings, and multifaceted interventions, including combinations of audit and feedback, marketing, and local consensus processes.13,14 Limiting knowledge-sharing methods to the provision of educational materials or didactic educational methods has minimal effects.15
Some evidence suggests that supporting KT can lead to practice change for pediatric physical therapists. Participation in active, multicomponent KT activities leads to improved evidence-based knowledge and practice behaviors compared with passive dissemination strategies.16,17 Schreiber et al8 reported modest gains in self-reported evidence-based practice behaviors of pediatric physical therapists following collaborative development and implementation of strategies aimed at making improvements in these areas. Some preliminary evidence suggests that adding some elements of KT to traditional continuing education programs has led to increased implementation of course-related information.6,18,19 Russell et al20 used a multifaceted KT intervention in a publicly funded community children's rehabilitation program in Canada. In this program, physical therapists acted as “knowledge brokers” (KBs) to facilitate the use of 4 evidence-based measurement tools in clinical practice. This KB model was designed to overcome many of the barriers to research transfer identified in the literature. The results suggest that physical therapists were able to increase knowledge and application of the tests and measures after a 6-month program and at a 12-month follow-up.20
An underlying rationale for the study on effective use of KBs was that pediatric physical therapists are not confident with the selection, administration, and interpretation of standardized outcome measures.17,20,21 The barriers include a lack of time, limited knowledge about measures and measurement principles, and inadequate training.22,23 Logistical and organizational barriers such as documentation requirements or lack of adequate space or materials may hinder use of standardized outcome measures. Efforts to increase knowledge about standardized outcome measures have been successful, but this increased knowledge has not led to an increase in use.17,18 Therefore, the evidence to date suggests that a more comprehensive KT approach is needed to change practice behaviors related to the use of standardized outcome measures. The purpose of this administrative case report is to describe the use of a multicomponent KT program to increase the use of standardized outcome measures in an outpatient pediatric physical therapy clinic setting.
Case Description: Target Setting
This program was conducted at a pediatric outpatient facility with 1 primary clinic and 3 additional satellite clinics, and a total of 18 physical therapists. Patients from birth to 18 years of age and with a wide variety of diagnoses received services at each clinic site. The 18 staff physical therapists represented a broad spectrum of practitioners, including new graduates and those with more than 20 years of experience (median=7 years). Five individuals were board-certified pediatric clinical specialists, 3 had an entry-level bachelor's degree, 8 had an entry-level master's degree, and 7 had a doctorate in physical therapy. All staff members were required to participate in the KT project, and all agreed to allow their data to be analyzed anonymously to assess the effectiveness of the project and, therefore, signed an informed consent form.
A critical first step in the KT process is identification of the clinical problem.12 The initial underlying problem identified by the outpatient department administrator was inconsistency of recommendations for frequency and duration of physical therapist services among the staff members. Subsequent formal and informal discussions with the department administrator and staff identified a need for increased use of standardized outcome measures to provide patient outcome data to support frequency and duration recommendations. The limited use of standardized outcome measures was further supported by baseline data gathered from a review of all physical therapy documentation in the electronic medical record at this facility (Tab. 1). Therefore, improving knowledge and skills related to use of these measures was considered to be an intermediate outcome to address the initial problem.
Frequency of Administration of Standardized Pediatric Outcome Measuresa
This baseline information and previous evidence suggested the need for a systematic effort aimed at increasing the use of standardized outcome measures at this outpatient department. Figure 1 provides a flowchart to describe the implementation of the KT program. The standardized pediatric outcome measures were selected through a collaborative effort between the primary investigator and the staff and department administrator (Appendix 1). These measures were chosen based on feasibility, relevance to patient population and frequency and duration decisions, and support in the research literature for validity and reliability. The consensus among staff members and the department administrator was that more frequent use of outcome measures would lead to clinical decisions that are based on quantitative data and, therefore, more defendable and consistent.
Flowchart of study design: identification of practice setting knowledge translation (KT) needs, implementation of training strategies, and baseline and 8-month outcome measurements. GMFM-66=66-item Gross Motor Function Measure, GMFM-88=88-item Gross Motor Function Measure, GMFCS=Gross Motor Function Classification System, PEDI=Pediatric Evaluation of Disability Inventory, TUG=Timed “Up & Go” Test, TUDS=Timed Up and Down Stairs Test.
Development and Application of the KT Program
A number of elements were included in the KT program, based in part on the knowledge-to-action framework.12 First, all staff completed a baseline knowledge assessment and self-report survey on knowledge and frequency of use of standardized outcome measures (Appendixes 2 and 3). This testing established baseline measures for each therapist to allow for program assessment and influenced adaptation of the knowledge to the local context. Individual therapist scores were available to the department administrator and were shared privately with each therapist.
A 2-hour practice-based workshop was offered at each satellite clinic, with additional 1-hour follow-up sessions approximately 2 months after the first workshop. All 18 therapists participated in the workshops and follow-up sessions. These workshops and follow-up sessions were led by the first author (J.S.), who was employed as a part-time staff member at one of the clinic sites and, therefore, was well known to all staff members. The use of local opinion leaders—colleagues who are seen as influential and trustworthy—shows promise as an effective way to improve KT.24,25 These individuals (ie, KBs) act as a link between clinicians and the research evidence,26 helping them interpret and use evidence during clinical decisions on an ongoing basis.
The workshop included presession reading materials, brief lectures on content, opportunities to practice standardized testing procedures, and identification of site-specific barriers and suggested solutions.12 Some examples of barriers included limited space available for testing, a need for visible and accessible distance measurements for gait speed and the Timed “Up & Go” Test, access to stairs for the Timed Up and Down Stairs Test, and suboptimal access to testing materials, electronic resources, and online scoring for documentation. There is some evidence to suggest that participation in active, multicomponent education strategies is more effective than passive dissemination for enhancing evidence-based knowledge and practice behaviors.16,27 In addition, some evidence for adult and professional education suggests that learning meant to improve practice must take into account the contingencies of the workplace and use the challenges presented by day-to-day practice to solidify the learning.28
During the workshop, each staff member received a hard-copy binder with all workshop materials, including test score sheets, normative values where available, and instructions for implementing and interpreting each standardized outcome measure. A decision-making algorithm was included to highlight appropriate use of each of the measures as a routine aspect of clinical decision making. Each of the project materials also was posted on a team website, along with video demonstrations for the outcome measures. This was an effort to provide easy accessibility to this information during the daily routine and to sustain the practice change.29 Providing ongoing ease of access to training materials to support decision making within clinician daily work flow has been shown to increase the likelihood of implementation.30 The use of these hard-copy and online materials also afforded all stakeholders access to update the materials as needed and to share the information with new staff members.
The KT program also included ongoing interaction among all staff members and the first author. A staff online discussion board was created, and each staff member was required to post an example where one of the standardized measures was used, along with the impact on clinical decision making and any barriers that occurred. This activity engaged all stakeholders in ongoing reflection and interaction. As noted above, each satellite clinic also participated in a 1-hour follow-up discussion with the first author, where successes and challenges with ongoing implementation were described. Finally, all staff members received periodic newsletter updates and reminders about the project and ongoing implementation.
Outcome Measures to Assess the Effectiveness of the KT Program
Several different measures were used to assess the effectiveness of the KT program. These measures included a knowledge assessment, developed for this project by the first author and administered at baseline and at 8-month follow-up (Appendix 2). The validity and reliability of this knowledge assessment were not established. However, the items of the assessment were based on the objectives and purpose of the KT program and provided an additional marker for each therapist, both to compare individual progress and to compare scores to the group mean. Ideally, this approach led to enhanced self-reflection and individual efforts to improve knowledge and skills related to pediatric outcome measures and to improved scores on this assessment at the 8-month follow-up.
The Self-Assessment of Knowledge and Frequency of Performance (SAKFP) was administered at baseline and 8-month follow-up. This tool was based on previously published work and was modified slightly to reflect the content and focus of this project. The previous version of the SAKFP was reviewed for face validity.18,19 The first author and the department administrator collaborated in an effort to ensure that the modifications of this tool reflected the objectives and purpose of the KT program (Appendix 3). However, the validity and reliability of the SAKFP for this program were not established. The SAKFP was divided into 4 subsections for administration and analysis: selection, administration, interpretation, and sharing information from the standardized pediatric outcome measures. The items within each subsection were added together to provide a score for each area. For example, in the selection subsection, there were 5 items, with a maximum potential score of 25. We hypothesized that scores in each subsection would increase for both frequency and knowledge at the 8-month follow-up.
Finally, the electronic medical records were queried at 2-month intervals prior to and following the workshops to determine the number of documented instances when the standardized pediatric tests and measures in this project were administered. All physical therapy outpatient documentation was queried. We hypothesized that the frequency of all measures would increase if the KT program was effective. One exception was frequency of performance of the 88-item Gross Motor Function Measure (GMFM-88), as we anticipated that an increase in the frequency of use of the 66-item Gross Motor Function Measure (GMFM-66) would lead to a decrease in use of the lengthier GMFM-88.
Data Management and Analysis
Participant responses were reduced to 4 subsection scores for knowledge and performance. Descriptive statistics (mean±SD) were estimated for all outcome measure scores at baseline and at 8 months after the KT intervention. Clinician changes in outcome measures resulting from the KT intervention between baseline and 8-month follow-up were tested using the nonparametric Wilcoxon signed ranks test for paired data. Preintervention to postintervention changes were judged significant at P<.05.
Outcomes
Follow-up assessments on all outcome measures were obtained from 17 clinicians. One clinician participated in the workshop and follow-up session but left employment at this clinic prior to the 8-month follow-up testing. Knowledge assessment scores significantly increased between baseline (54.1±13.5) and 8-month follow-up (81.8±12.7) (P<.001).
Self-reported knowledge of testing and measurement significantly improved across the 4 subdomains of test selection, administration, interpretation, and sharing of results between baseline and 8-month follow-up (Fig. 2). The greatest improvement was observed in the administration subdomain (mean change=8.1±5.3 points, P=.001). The smallest magnitude of change was observed in the knowledge of sharing of results subdomain (mean change=1.8±2.8 points, P=.022).
Knowledge subsection scores (mean±SD) for clinicians (N=17) at baseline and 8 months after knowledge translation intervention in an outpatient pediatric practice setting: test selection, administration, interpretation, and sharing of results. Error bars=±1 SD.
Self-reported performance of testing and measurement significantly improved across the 3 subdomains of test selection, administration, and interpretation between baseline and 8-month follow-up (Fig. 3). Among the performance subdomains with significant change, the greatest improvement was observed in the administration subdomain (mean change=11.6±5.9 points, P<.001). The smallest magnitude of change was observed in the performance of interpretation of results subdomain (mean change=4.2±4.6 points, P=.006). The performance of results sharing did not show a significant change between baseline and 8-month follow-up (mean change=2.1±4.6 points, P<.091).
Performance subsection scores (mean±SD) for clinicians (N=17) at baseline and 8 months after knowledge translation intervention in an outpatient pediatric practice setting: test selection, administration, interpretation, and sharing of results. Error bars=±1 SD.
Data from the electronic medical record query are displayed in Table 2. Documented frequency of administration increased for all pediatric outcome measures after the initiation of the KT program, and this increase was sustained over the 8-month period.
Frequency of Administration of Tests and Measures Over 2-Month Intervalsa
Discussion
The outcomes of the KT program indicate that the outpatient physical therapy staff did improve knowledge and increase frequency of usage of standardized pediatric outcome measures. The scores on the knowledge assessment tool, along with self-reported knowledge, all improved. In addition, frequency of documented use of the standardized pediatric outcome measures increased, and self-reported frequency increased in all subsections with the exception of sharing information. There was some fluctuation in the frequency of documentation, as noted in Table 2, which may have been due to seasonal fluctuations in patient census, intermittent challenges with the electronic medical record system, and an initial excess of tests and measures followed by a more realistic and appropriate frequency as the clinicians became more knowledgeable about each of the tests and as the need for gathering data on all patients stabilized.
Similar to previous research,8,13,16,17,20,24,25,31 the results of this administrative case report support the efficacy of a multicomponent KT program in this pediatric outpatient clinic. The KT program included a variety of activities to facilitate increased use of standardized pediatric outcome measures. The critical first step was meeting with the department administrator and staff members to identify the clinical problem.12,24,32 In this project, all constituencies agreed that inconsistent clinical decision making related to frequency and duration of outpatient physical therapist services was a significant clinical problem and that improved selection, administration, interpretation, and sharing of results of pediatric outcome measures was critical to addressing that problem. Commitment of the various stakeholders within the organization is critical to success, as is a clear understanding of the underlying rationale for the practice change effort.33,34 Effective KT requires a change in behavior among administrators, policy makers, and practitioners.
Subsequently, strategies were developed collaboratively to address the clinical problem. The first author functioned as a KB in this project. Some evidence suggests that use of a KB can support the KT process.20,25,33 Critical characteristics for an effective KB include a strong understanding of the clinical and organizational contexts, strong research skills, enthusiasm, and accessibility.33 The KB worked part-time (4 hours per week) at one of the satellite clinics for this facility and was readily accessible to all staff either in person or via e-mail throughout the course of this project. In addition, the KB had collaborated with the department administration and various staff members on several previous projects. This collaboration included lunch-time 1-hour in-services on a variety of topics, including use of standardized outcome measures. The shared perception between staff, administrators, and the first author that these in-services were ineffective was one of the initial factors that led to the development of this project. The administration supported an additional 4 hours per week for the KB for this project, over a 3-month period.
Additional strategies to support KT included the structure of the workshops and follow-up sessions. Evidence suggests that active and multifaceted learning strategies are more likely to lead to behavior change than passive strategies such as lectures and provision of research articles.16,31,35 The workshops were a combination of information sharing between the KB and the clinic staff, identification of site-specific barriers and supports related to application of the workshop content, and opportunities to practice the outcome measures in the clinic. In addition, these 2-hour workshops occurred during the workday, which required considerable administrative efforts to organize and clear schedules so that all staff at each satellite clinic could participate in the workshops. It is likely that this extensive administrative support for this project conveyed an implicit message regarding the importance of the project.
A hard-copy binder that included reference materials and instructions on administration and interpretation of the outcome measures was provided for each participant. The provision of this hard-copy binder was based on recommendations from the department administrator and reflected the importance of easily accessible information to support decision making during the daily routine.29 Online materials also were provided as a resource for current and future staff members. Providing ease of access to materials to support decision making within clinician daily work flow has been shown to increase the likelihood of implementation.30 Clinical decision support systems, which provide practitioners with patient-specific assessments or recommendations to aid clinical decision making, are defined as any electronic or nonelectronic system designed to aid directly in clinical decision making.30,36 A common theme is that these systems increase ease of use, suggesting that an effective system must minimize the effort required of the clinician to receive and act on the system recommendations.30,36 The binders were stored on each staff member's desk and, therefore, were easily available during documentation and session planning activities.
The primary goal for the department administrator was to increase the consistency of clinical decisions regarding frequency and duration of physical therapist services. We did not collect data to determine whether this goal was achieved. However, the department administrator, staff, and KB agreed that a critical intermediary step to achieving this goal was increased use of standardized outcome measures to guide collaborative decision making. There is limited evidence for the effectiveness of interventions to improve the ability of clinicians to adopt a shared decision-making approach with patients.37 However, informal discussion and online case-based interaction appeared to suggest that clinicians who participated in this project were more readily doing so. Some staff members also reported difficulty sharing outcomes data with caregivers and expressed a need for more training in this area and for additional written educational materials to assist with this process. Therefore, it is likely that other factors in addition to knowledge and performance of standardized outcome measures affect the capability of practitioners to make appropriate and consistent clinical practice recommendations. These factors, which may include parent/caregiver readiness, appropriate goal setting, and the impact of child age and diagnosis, also must be addressed to aid pediatric physical therapists in achieving more consistency in clinical decision making in this area.
Limitations
No effort was made to control for other extraneous variables; therefore, any changes that did occur cannot be solely attributed to the KT program. In addition, the validity of the outcome measures has not been well established. The knowledge assessment tool was developed by the first author exclusively for this project and has not been specifically validated to measure knowledge in this content area. Follow-up scores may have been affected by familiarity with the items in the assessment. A similar version of the SAKFP has been used previously, but this tool has not undergone evaluation of the psychometric properties. However, administrators and others interested in evaluating the impact of a KT program may consider this framework to evaluate the success of the KT effort. Finally, an effective KB is often necessary for effective KT. In this program, the first author was able to fulfill this role. Alternatively, it may be appropriate to identify a full-time staff member as a KB. Critical factors for the success of the KB include knowledge and expertise in the content area, along with knowledge about the unique environmental constraints and supports in a particular setting.
Conclusions
The KT program described in this case report led to improved knowledge and increased frequency of performance of standardized outcome measures in a pediatric outpatient clinic. The changes were sustained at 8 months following the initiation of the program. Key factors included identification of a clinical problem, administrative support, multifaceted intervention, ready access to knowledge materials, and sustained interaction with a KB. Physical therapists should consider integrating each of these factors into initiatives designed to improve clinical practice. Future research should investigate the impact of each of these factors and should identify valid and reliable outcome measures to assess clinical practice behavior change.
Appendix 1.
Standardized Pediatric Outcome Measuresa
a GMFM-66=66-item Gross Motor Function Measure, GMFM-88=88-item Gross Motor Function Measure, GMFCS=Gross Motor Function Classification System, PEDI=Pediatric Evaluation of Disability Inventory, TUG=Timed “Up & Go” Test, TUDS=Timed Up and Down Stairs Test.
Appendix 2.
Baseline Assessment of Knowledge Related to Standardized Pediatric Outcome Measuresa
a The Baseline Assessment of Knowledge Related to Standardize Pediatric Outcome Measures may not be used reproduced without written permission from the authors.
Appendix 3.
Self-Assessment of Knowledge and Frequency of Performancea
a The Self-Assessment of Knowledge and Frequency of Performance may not be used or reproduced without written permission from the authors.
Footnotes
Dr Schreiber, Ms Racicot, and Ms Kaminski provided concept/idea/project design and data collection. All authors provided writing. Dr Schreiber, Dr Marchetti, and Ms Racicot provided data analysis. Dr Schreiber and Ms Racicot provided project management. Ms Racicot provided facilities/equipment.
This project was reviewed and approved by the Chatham University Institutional Review Board and the research committee of the participating facility.
Data from this project were presented at the Combined Sections Meeting of the American Physical Therapy Association; January 21–24, 2013; San Diego, California.
- Received September 11, 2013.
- Accepted July 13, 2014.
- © 2015 American Physical Therapy Association
References
Issue highlights
