Abstract
Background Physical therapists and occupational therapists practicing in acute care hospitals play a crucial role in discharge planning. A standardized assessment of patients' function could be useful for discharge recommendations.
Objectives The study objective was to determine the accuracy of “6-Clicks” basic mobility and daily activity measures for predicting discharge from an acute care hospital to a home or institutional setting.
Design The study was retrospective and observational.
Methods “6-Clicks” scores obtained at initial visits by physical therapists or occupational therapists and patients' discharge destinations were used to develop and validate receiver operating characteristic curves for predicting discharge destination. Positive predictive values (PPV), negative predictive values (NPV), and likelihood ratios were calculated.
Results Areas under the receiver operating characteristic curves for basic mobility scores were 0.857 (95% confidence interval [CI]=0.852, 0.862) and 0.855 (95% CI=0.850, 0.860) in development and validation samples, respectively. Areas under the curves for daily activity scores were 0.846 (95% CI=0.841, 0.851) and 0.845 (95% CI=0.840, 0.850) in development and validation samples, respectively. Cutoff scores providing the best accuracy for determining discharge destination were 42.9 for basic mobility and 39.4 for daily activity. For basic mobility, the PPV was 0.748 and the NPV was 0.801 in both development and validation samples. For daily activity, the PPVs were 0.787 and 0.784 and the NPVs were 0.748 and 0.746 in development and validation samples, respectively.
Limitations Limitations included lack of information on the rater reliability of “6-Clicks” instruments, use of surrogate data for some discharge designations, and use of a clinical database for research purposes.
Conclusions This study provides evidence of the accuracy of “6-Clicks” scores for predicting destination after discharge from an acute care hospital.
In acute care hospitals, many patients with neurological, musculoskeletal, and cardiopulmonary problems have limitations in walking and moving, self-care, changing and maintaining body positions, or several of these tasks.1 A primary focus for hospital-based physical therapists and occupational therapists is evaluating patients' mobility and self-care abilities to determine the need for skilled care, which drives discharge planning.2–4 These professionals are uniquely qualified to assess the congruency between patients' living situations and functional abilities, particularly in terms of understanding the level of assistance or environmental supports that patients may require to safely perform essential mobility and daily activities. The aim of assessment and planning for patients' discharge is to enhance quality of care by ensuring that patients are discharged to post–acute care settings that match their health and social needs. Equally important, adequate discharge planning may improve the efficiency of care and reduce costs5 by transitioning patients from the expensive acute care setting to the next appropriate level of care in a timely manner. One of the reported reasons for inefficient, delayed discharge processes is inadequate assessment of patients and their life situations.5 Another possible reason for delayed discharge is the time required for post–acute care placement processes to reach completion.6,7 For these reasons, complete and relevant assessments of patients for the purposes of discharge planning should be performed early in the course of hospitalization.
To reduce delays in discharge from acute care settings through early planning, physical therapists and occupational therapists attempt to accurately project discharge needs using information available at hospital admission and their first visit with a patient. An algorithm for selecting a post–acute care setting for patients with stroke, published as part of a clinical practice guideline in 1995, included determining functional ability and adequate home support as a factor in directing recommendations.8 In several studies, models for predicting discharge setting included measures of patients' function during an acute care hospital stay.9–17 Limitations of those studies included prediction models based on measures of function after several rehabilitation sessions or close to discharge, rather than early in an episode of care9–12; a focus on patients with only 1 type of condition9,10,13–15; and the use of complicated instruments that may not be practical for everyday use in a fast-paced acute care hospital setting.12,16,17 Existing standardized measures of patient function seem not to be widely used by therapists in the treatment of patients in acute care settings.18 The reported low level of use of existing measures by clinicians may be due to their length, ambiguous interpretation of findings, or their ineffectiveness in facilitating the prediction of an appropriate discharge destination.14
Efficient and effective discharge planning processes and decisions may also be critical in reducing hospital readmissions.12,19,20 One recent study showed that the incidence of hospital readmission for patients discharged to settings recommended by physical therapists was lower than that for patients discharged to settings not in concert with physical therapists' recommendations.21 Poor communication between providers in acute care and discharge settings may also contribute to readmissions.22 A systematic review examining communication between acute care facilities and primary care physicians at patient discharge resulted in recommendations for thorough discharge summaries that follow a standardized format, use information technology, and contain information about patients' functional status.20
Recently, physical therapists and occupational therapists at Cleveland Clinic Health System hospitals pilot tested the use in an acute care setting of new standardized functional assessment instruments that may meet these requirements. These tools, called “6-Clicks,” are short forms created from the Activity Measure for Post-Acute Care (AM-PAC) instrument, developed by researchers at Boston University.23 The AM-PAC measures 3 functional domains: basic mobility, daily activities, and applied cognition. It may be used for assessment in adults with a wide range of diagnoses and levels of performance in the 3 domains. One “6-Clicks” instrument assesses basic mobility, such as walking and moving from 1 position to another; a second instrument assesses daily activities, such as dressing and toileting. The applied cognitive domain is not measured by “6-Clicks” instruments. The advantages of these instruments include the fact that they are completed quickly; they provide discrete data that can be entered into an electronic medical record as part of the documentation of therapist visits; they can be completed through direct observation or estimation of patients' capabilities on the basis of clinical judgment; and they are derived from and scored on the same standardized metric as the AM-PAC instrument, which can be used in any post–acute care setting. These advantages can promote discharge-related communication. A previous study demonstrated evidence for the validity of “6-Clicks” instruments.24
The purpose of the present study was to determine the accuracy of “6-Clicks” measures for predicting discharge destination when applied at the first visit of a physical therapist or an occupational therapist with a patient in a hospital setting.
Method
Data Source
A clinical database derived from the MediLinks electronic medical record system used by Cleveland Clinic Health System Physical Therapy Department and Occupational Therapy Department was the source of data for analyses. It contained more than 200,000 entries, with data on 92,899 individual patients seen in acute care settings by physical therapists, occupational therapists, or both types of therapists from April 2011 to December 2012. This time frame represented the time from the initial use of the “6-Clicks” forms to the beginning of data analysis for this project. For our study sample, we selected patients for whom “6-Clicks” scores from initial physical therapist or occupational therapist visits and discharge destination were available. Discharge destination was missing for 17% of patients who had an initial-visit daily activity score and for 24% of patients who had an initial-visit basic mobility score (Fig. 1). From the dataset, we extracted data for first-visit basic mobility and daily activity “6-Clicks” scores; patient background information, including age, sex, primary medical or surgical diagnosis, and preadmission living situation; number of therapy visits; length of hospital stay; days from admission to first therapy visits; and discharge destination.
Case selection flow chart. Asterisks indicate that 28,553 patients had both basic mobility and daily activity “6-Clicks” scores.
Setting and Procedures
The Cleveland Clinic Health System is a nonprofit system that includes 3,700 beds in the main campus hospital and 8 regional hospitals. The Rehabilitation and Sports Therapy enterprise includes more than 700 therapy professionals, with 90 physical therapists and 45 occupational therapists primarily assigned to acute hospital care. These therapists manage more than 1,900 patient evaluations per week. In April 2011, therapists began using “6-Clicks” electronic data entry forms to document patients' function. Physical therapists completed the basic mobility form, and occupational therapists completed the daily activity form. Therapists determined the score for each item either by observing patients' performance or by using their clinical judgment about patients' capabilities. Therapists entered patients' “6-Clicks” scores into the electronic medical record system as part of their visit documentation. The dataset included only de-identified data.
Instrument
The “6-Clicks” short forms include items selected from the AM-PAC, a validated measure designed to be used for patients receiving post–acute care services regardless of type of condition or setting.23 One “6-Clicks” form assesses basic mobility function, and another form assesses daily activity function.25 Each item is scored from 1 to 4 on the basis of the amount of difficulty a patient has or how much help is needed from another person in completing the task. Lower scores equate to lower levels of function. A previous study provided evidence for convergent and divergent validity as well as the responsiveness of the “6-Clicks” forms.24
Data Analysis
Descriptive statistics were used to describe the characteristics of the patients in the sample and to summarize “6-Clicks” basic mobility and daily activity scores at the first physical therapist or occupational therapist visit. Our dataset included actual discharge destination only for patients who were discharged to home with services or to an institutional setting; the data field was left empty for patients who were discharged to home without services. However, we were unable to determine whether an empty data field represented discharge to home or whether the data were simply missing. The dataset also included the discharge recommendations made by physical therapists and occupational therapists after their last visit with a patient; we determined that in 83% of patients for whom complete data were available, the physical therapists' and occupational therapists' recommendations for discharge setting agreed with the actual discharge setting. Therefore, when the actual discharge destination field was empty, we coded it as “home” if the recommendations of both physical therapists and occupational therapists agreed on home discharge; if they did not agree, we coded the field as missing data.
To determine the usefulness of “6-Clicks” scores for early discharge decision making and potential cutoff scores for recommending discharge to home versus a post–acute care institutional setting, we selected a random sample of 50% of the patients in our dataset to develop receiver operating characteristic (ROC) curves for first-visit “6-Clicks” scores for basic mobility and daily activity. In an ROC analysis, the sensitivity and 1 − specificity are calculated for each possible cutoff point in a scale. These values are then plotted with sensitivity on the y-axis and 1 − specificity on the x-axis. The ROC analysis allowed us to define the best cutoff score for determining discharge to home on the basis of the highest sensitivity and specificity associated with the various scores. Using a standardized prevalence for discharge to home of 50%, we calculated positive and negative predictive values. To test the accuracy of our findings, we repeated the analysis for the remaining 50% of the sample. Model calibration for each “6-Clicks” measure was examined with the Hosmer-Lemeshow goodness-of-fit test. Because the Hosmer-Lemeshow test result is affected by sample size and our sample size was large, we were concerned that small differences between observed and predicted values would be statistically significant, thereby suggesting a lack of model fit. For this reason, we also examined goodness of fit by plotting observed and predicted frequencies of discharge to home by decile of predicted probability of discharge to home for the entire sample. Predictions were based on logistic regression analysis with discharge setting as the dependent measure and initial “6-Clicks” score as the independent measure. All analyses were conducted with IBM-SPSS Statistics version 20 (IBM Corp, Armonk, New York).
Results
Approximately 50% of the patients were 65 to 85 years of age, and the largest proportion (25.4%) had cardiovascular or pulmonary conditions. Approximately 87% of the patients were admitted from home and had a median hospital length of stay of 4.7 days (interquartile range=9.6). Approximately 24% of the patients were visited by a physical therapist or an occupational therapist within 1 day of admission. Participant characteristics are shown in Table 1.
Participant Characteristicsa
The areas under the curve for the first-visit basic mobility score were 0.857 (95% confidence interval=0.852, 0.862) and 0.855 (95% confidence interval=0.850, 0.860) in the development and validation samples, respectively. The areas under the curve for the first-visit daily activity score were 0.846 (95% confidence interval=0.841, 0.851) and 0.845 (95% confidence interval=0.840, 0.850) in the development and validation samples, respectively. The cutoff scores yielding the most accurate predictions of discharge destination in both the development and the validation samples were 42.9 for basic mobility and 39.4 for daily activity.
Figure 2 shows the degree of correct prediction on the basis of these cutoff scores. Using the cutoff scores, we calculated positive and negative predictive values (with a standardized prevalence of 50%) of 0.748 and 0.801, respectively, for basic mobility and 0.787 and 0.748, respectively, for daily activity in the development sample (Tab. 2). Using the cutoff scores and the related values for sensitivity and specificity in the development sample, we calculated positive likelihood ratios for basic mobility and daily activity of 2.963 and 3.705, respectively; negative likelihood ratios were 0.247 and 0.336, respectively. The Hosmer-Lemeshow test demonstrated lack of fit for each model (P<.001). Model calibration is shown in Figure 3.
Stacked scatterplots of “6-Clicks” scores by discharge destination, indicating optimal cutoff scores for basic mobility (A) and daily activity (B).
Accuracy of “6-Clicks” Scoresa
Model calibration: observed versus expected frequencies of discharge to home by decile of predicted probability on the basis of basic mobility (A) and daily activity (B) scores.
Discussion
The present study demonstrated the accuracy of the “6-Clicks” basic mobility and daily activity forms for predicting destination after discharge from an acute care hospital. To our knowledge, the present study is the first to examine the usefulness for discharge planning of simple, standardized assessments of patients' function completed by physical therapists and occupational therapists at their first visit with a patient in an acute care setting. Although discharge recommendations are made on the basis of a holistic assessment, in which a patient's function is only part of the consideration,4 physical therapists and occupational therapists might find quick, easy-to-use measures of function beneficial for facilitating their recommendations. Given the short lengths of stay in acute care hospitals, measurement tools that objectify discharge recommendations at an early stage could prove to be indispensable. Clinicians are often asked to make discharge recommendations on the basis of their assessment from the first encounter with a patient, and patients may have no more than 1 visit with a physical therapist or an occupational therapist during their hospitalization. In the sample used for the present study, 61% of patients had only 1 occupational therapist visit, and 65% of patients had only 1 physical therapist visit (data not shown).
Our findings demonstrated that “6-Clicks” scores at the first visit could be useful objective measures for enhancing the accuracy of acute care hospital discharge recommendations. The areas under the ROC curves showed fair-to-good accuracy of “6-Clicks” scores derived from the first physical therapist and occupational therapist visits in determining discharge destination. In a previous study, in which a score for predicting patients at risk of discharge to a post–acute care facility was derived from 5 patient attributes (including bathing and transfer abilities on day 3 of hospitalization), the area under the ROC curve was 0.82.7 We found similar areas under the curves for measures derived from first visits, approximately 24% of which were determined within 1 day of hospital admission. In 2010, in the United States, the average length of hospital stay was 4.8 days.26 This fact suggests the need to assess patients for potential discharge destination early in their admission.
The ROC curves in the present study showed that the use of a cutoff score of 42.9 on the basic mobility form and a cutoff score of 39.4 on the daily activity form provided the highest sensitivity, specificity, and positive and negative predictive values. Given that predictive values are affected by the sample prevalence of the condition being investigated, we used a standardized prevalence of discharge to home of 50% to calculate these values. The positive predictive values represented the proportion of patients who had scores higher than the cutoff score (more functional) and were discharged to home with services. The negative predictive values represented the proportion of patients who had scores lower than the cutoff score (less functional) and were discharged to institutional settings. Generally, the goal is to maximize correct decisions; therefore, for making decisions about which cutoff scores are the best, the consequences of being incorrect must be determined. Therapists would need to determine whether recommending discharge to home for a patient who should be discharged to an institutional setting would have worse consequences (eg, failure to initiate an early referral to a post–acute care facility) than recommending discharge to an institutional setting for a patient who should be discharged to home (eg, additional costs of care in a post–acute care facility).
The following example demonstrates how a therapist collecting “6-Clicks” data from a patient could use the scores for recommending a discharge setting. The patient's basic mobility and daily activity scores are 47.7 and 42.0, respectively. The pretest probability of the patient being discharged to the home setting, on the basis of the data generated by our sample, is 0.43. The pretest odds of discharge to home are 0.43/(1 − 0.43), or 0.75. The posttest odds of discharge to home are calculated by multiplying the pretest odds by the positive likelihood ratio (0.75 × 2.96), or 2.22. On the basis of these calculations, the probability of discharge to home for this patient increases to 2.22/(2.22 + 1), or 0.69. Similarly, on the basis of the positive likelihood ratio for the daily activity score and the fact that the score is above the cutoff score for daily activity, the probability of this patient being discharged to home increases to 0.73. For each test, positive results (scores above the cutoff score) improve the accuracy of the prediction of discharge to home.
Limitations
One limitation of the present study is the fact that discharge destination likely is influenced by many factors. For example, other studies have shown the influence of better cognitive function, in addition to physical function, on discharge to home,15,17 and lack of health insurance has been shown to be related to a greater likelihood of discharge to home rather than to an inpatient post–acute care setting for patients with burns.27 Social support and family support are also critical factors in determining patients' discharge destinations.17,27 The true “gold standard” for determining the accuracy of “6-Clicks” scores likely is the “appropriate” discharge setting for patients in terms of their function, safety, and ongoing health care needs rather than their actual discharge setting. However, measuring the appropriateness of a discharge setting is challenging.
The present study also lacked data on discharge destination for patients discharged to home without services; surrogate data points were used to replace the missing data for those patients. The fact that the patients for whom the discharge destination field was coded as missing data were younger and more likely to be living at home than the patients for whom we identified the actual discharge destination suggests some misclassification. The aforementioned possibilities may partially explain the lack of model fit determined with the Hosmer-Lemeshow test. However, the plotted frequencies of predicted and observed discharge to home did not reveal large differences (Fig. 3).
Another limitation is that although the reliability of the AM-PAC instrument has been reported,28 the rater reliability of the “6-Clicks” forms has not been tested. Determining rater reliability may be particularly important given that “6-Clicks” scores can be based on clinical judgment or on observation of actual performance. Clinicians' judgments of patients' functional capabilities may be more variable than their ability to score actual performance. However, in a previous study, good internal consistency reliability of each instrument was demonstrated (0.96 for basic mobility and 0.91 for daily activity).24
An additional limitation was the use of a clinical database to provide data for analyses. The limitations of using clinical databases for research have been well documented.29 The database that we used was developed to enhance clinical operations and allow quality assessments, not for research purposes. Clinical databases also are designed to be helpful to clinicians in documenting patients' status. Therefore, they may not include the type of data necessary to address research questions or the form of data useful for statistical analysis. Clinical databases that include narrative information, such as that used in the present study, can provide rich descriptions that may be helpful for the care of patients; however, such information may be difficult to code accurately when it is used for research purposes, and the result may be misclassifications and missing data points.
Another limitation is that the present study took place in only 1 health care system; it is unclear how the “6-Clicks” instruments may perform in other settings. On the other hand, the data used in the present study were collected by a large and diverse population of clinicians practicing in a heterogeneous health care system comprising an academic medical center and community hospitals in both urban and suburban locations.
In conclusion, the present study demonstrated the accuracy of “6-Clicks” basic mobility and daily activity scores obtained at the first physical therapist or occupational therapist visit in an acute care hospital for predicting discharge destination. Cutoff scores of 42.9 for basic mobility and 39.4 for daily activity at the first visit provided fair to good accuracy for predicting discharge destination. Given the short lengths of stay in acute care hospitals and the focus of rehabilitation professionals on assessing patients to make discharge recommendations, measurement tools that enhance the accuracy of discharge recommendations at an early stage could prove to be indispensable.
Footnotes
All authors provided concept/idea/research design. Dr D. Jette, Dr Frost, and Dr A. Jette provided writing. Dr Stilphen, Mr Ranganathan, Dr Passek, and Dr Frost provided data collection. Dr D. Jette, Mr Ranganathan, and Dr Passek provided data analysis. Mr Ranganathan and Dr Frost provided project management. Dr Stilphen and Dr Passek provided study participants. Dr Stilphen, Dr Passek, and Dr Frost provided facilities/equipment. Dr Stilphen, Mr Ranganathan, Dr Frost, and Dr A. Jette provided institutional liaisons. Dr Passek and Dr Frost provided clerical/secretarial support. Dr Stilphen, Mr Ranganathan, Dr Passek, Dr Frost, and Dr A. Jette provided consultation (including review of manuscript before submission).
The institutional review boards at the Cleveland Clinic Health System and the University of Vermont classified this project as “nonhuman subjects” research.
- Received September 6, 2013.
- Accepted April 16, 2014.
- © 2014 American Physical Therapy Association
References
Issue highlights
