Abstract
Background Early, accurate prediction of discharge destination from the acute hospital assists individual patients and the wider hospital system. The Trauma Rehabilitation and Prediction Tool (TRaPT), developed using registry data, determines probability of inpatient rehabilitation discharge for patients with isolated lower limb fractures.
Objective The aims of this study were: (1) to prospectively validatate the TRaPT, (2) to assess whether its performance could be improved by adding additional demographic data, and (3) to simplify it for use as a bedside tool.
Design This was a cohort, measurement-focused study.
Methods Patients with isolated lower limb fractures (N=114) who were admitted to a major trauma center in Melbourne, Australia, were included. The participants' TRaPT scores were calculated from admission data. Performance of the TRaPT score alone, and in combination with frailty, weight-bearing status, and home supports, was assessed using measures of discrimination and calibration. A simplified TRaPT was developed by rounding the coefficients of variables in the original model and grouping age into 8 categories. Simplified TRaPT performance measures, including specificity, sensitivity, and positive and negative predictive values, were evaluated.
Results Prospective validation of the TRaPT showed excellent discrimination (C-statistic=0.90 [95% confidence interval=0.82, 0.97]), a sensitivity of 80%, and specificity of 94%. All participants able to weight bear were discharged directly home. Simplified TRaPT scores had a sensitivity of 80% and a specificity of 88%.
Limitations Generalizability may be limited given the compensation system that exists in Australia, but the methods used will assist in designing a similar tool in any population.
Conclusions The TRaPT accurately predicted discharge destination for 80% of patients and may form a useful aid for discharge decision making, with the simplified version facilitating its use as a bedside tool.
Early determination of which patients require inpatient rehabilitation and who may go directly home following an acute hospital admission is advantageous to the patient and the hospital system. A consistent reason for reported delays in discharge from the acute hospital, across health care jurisdictions, is the lack of rehabilitation beds.1,2 Earlier rehabilitation review and listing of patients for inpatient rehabilitation should assist with the timeliness of patient discharge and decrease the patient's uncertainty regarding his or her short-term plans. Initiating early referral to relevant health care professionals and utilizing this input for patients who can go directly home also may facilitate earlier patient discharge.
To optimize discharge planning following trauma, the Trauma Rehabilitation Prediction Tool (TRaPT) was developed, using registry data3 from 1,429 patients with lower limb fractures,4 and is a measure of who goes to inpatient rehabilitation, rather than who may actually need rehabilitation. Internal validation of the TRaPT showed outstanding discrimination (C-statistic=0.92) and good predictive qualities, with a specificity of 94% and a sensitivity of 70%.4 However, there were factors not available from the registry, such as frailty5 and patient expectation,6 that have shown an association with discharge to inpatient rehabilitation and that potentially could improve the predictive reliability of the TraPT.7
The TRaPT is scored for a patient on admission following lower limb fracture to determine the likelihood of discharge to inpatient rehabilitation, and this score can be used, in conjunction with the patient's wishes, to refer early to inpatient rehabilitation or begin facilitating discharge directly home once medically stable. If the patient scores greater than 50%, discussion is undertaken with the patient and family about requirements for inpatient rehabilitation, the referral is made, and the entire team is aware of the plan. If the score is less than 50%, discussion is focused on discharge home, and the health care team is unified in their approach to facilitate the patient's discharge. A simplified version that could be scored at the bedside may allow the patient to feel increased involvement in decision making and planning. The TRaPT further expands on the concept published by Neimeijer et al8 in 2010 that the most crucial improvement in decreasing length of stay is the collective attention of the physicians, nurses, and patient to the discharge and that this decision should be made within the first 24 hours. The advantage of the TRaPT over other discharge tools is that it can be used on admission based on patient and injury factors, hence allowing for very early decision making, which assists both the patient and the wider hospital system.
For discharge prediction algorithms to be clinically useful, they should be easy to administer at the bedside, as previous literature suggests that simplicity is an important criterion in the development of clinical tools.7,9 The TRaPT was based on a complex mathematical formula, which limits its clinical utility. Converting the TRaPT into a bedside tool would enhance its clinical relevance. The Risk Assessment and Predictor Tool (RAPT) is an example of a simple tool used internationally to assist with discharge planning following total joint arthroplasty.6,10
The primary aim of this study was to prospectively validate the TRaPT and assess whether the inclusion of additional factors could improve its predictive capacity. A secondary aim was to determine whether the TRaPT could be modified for use as a simple bedside instrument.
Method
Setting and Participants
Patients were included in this study if they met the following criteria: (1) admitted to The Alfred Hospital from June 2011 to January 2012, (2) 18 years of age or older at the time of injury, and (3) sustained an isolated lower limb fracture. Patients were excluded if they had additional injuries, other than minor soft tissue injuries or brief loss of consciousness with no neurological sequelae. Patients also were excluded if they were admitted from a nursing home. As with the original TRaPT development, patients with femoral neck fractures were excluded, as they were shown to require increased resources and have poorer outcomes than the rest of the orthopedic trauma population.11
Procedure
Data were collected on admission to the ward, including for factors required for calculation of the TRaPT score (Figure). These were age, compensation/insurance status, injury type, mechanism of injury, work status prior to injury, and self-reported disability (on a 5-point scale from “none” to “severe”). Additional information was collected on admission, including frailty, as calculated by the Reported Edmonton Frailty Scale (REFS).12,13 This scale has the following domains: cognition, general health, functional independence, social support, medication use, nutrition, mood, continence, and self-reported performance. The REFS is scored out of 18 points, with scores of less than 5 considered “not frail”13 and scores of 3 or less being associated with a higher chance of discharge directly home.5 Body mass index (BMI, <30 kg/m2, ≥30 kg/ m2), stairs at home (0/1, >1 step), open/closed fracture, patient preference of home or inpatient rehabilitation, and whether the patient had support at home on discharge also were recorded, as this information is considered potentially important when determining discharge destination. Number of operations and weight-bearing status (full/partial/touch/none) were recorded postoperatively.
Scoring for Trauma Rehabilitation and Predictor Tool (TRaPT). Scoring of the original TRaPT involves placing a zero (no) or 1 (yes) in each bracket (and multiplied by the preceding number) according to whether the patient has this factor. In the age bracket, the patient's age should be included, and the 0.043 is multiplied by this number.
The outcome of interest was discharge to inpatient rehabilitation. Discharge destination (home or inpatient rehabilitation) was collected by review of the patient's medical record.
Data Analysis
Descriptive analyses were used to describe the participants. The TRaPT score was calculated for each participant and compared with the actual discharge destination to allow for prospective validation of the score.14
Univariate logistic regression analysis was undertaken to determine the association between discharge destination and potential predictors not included in the TRaPT. Variables demonstrating a P value of <.2 were entered into a multivariable model that included all variables required for the TRaPT tool as well as other variables considered relevant, with discharge destination to inpatient rehabilitation as the outcome of interest.
The performance measures for the TRaPT alone were calculated, including sensitivity (proportion of patients who went to inpatient rehabilitation and were identified by the model as requiring this treatment intervention), specificity (proportion of patients who were not discharged to inpatient rehabilitation and were identified by the model as not requiring this treatment intervention), positive predictive value (PPV) (the proportion of patients who are predicted to be discharged to inpatient rehabilitation and do so), and negative predictive value (NPV) (the proportion of patients who are predicted not to require discharge to inpatient rehabilitation and do not do so).15 Positive likelihood ratio (LR+) and negative likelihood ratio (LR−) also were calculated to quantify how much a positive TRaPT result changes the odds of being discharged to inpatient rehabilitation and how much a negative result changes the odds of going home.16 A score above 10 for the LR+ or a score below 0.1 for the LR− is considered strong evidence that the test rules in or out going to inpatient rehabilitation, respectively, whereas an LR+ score less than 5 or an LR− score greater than 0.2 is considered relatively weak evidence.16 The area under the receiver operating characteristic curves, or C-statistic, which measures the discrimination of the tool, also was calculated. The C-statistic describes the ability of the model to discriminate between patients who are discharged to inpatient rehabilitation and those who go directly home.7 It is widely accepted that a C-statistic >0.90 is described as outstanding discrimination, and values of 0.80 to 0.90 are considered excellent.7 Calibration measures how accurately the tool predicts over the entire range and was assessed using the Hosmer-Lemeshow (H-L) statistic. The H-L statistic is a “goodness of fit” statistic and looks at replication within subpopulations.17 An associated P value >.05 indicates that the model's predictions are not different from the observed values and that the model has adequate fit.
Variables associated with discharge destination on univariate analysis were added to the TRaPT individually, and then in combination, and the calibration and discrimination calculated. The equality of the area under the curves was compared using the algorithm suggested by DeLong et al18 to obtain a chi-square statistic. A difference in C-statistic of >0.05 was used as justification to include new variables into the TRaPT.
In order to produce a simplified TRaPT (sTRaPT), the coefficients of the variables in the original model were rounded to the nearest 0.5. Age was categorized into 8 groups: 18–29, 30–39, 40–49, 50–59, 60–69, 70–79, 80–89, and >90, and the median age for each group was used for the score calculation. The resultant sTRaPT score was then compared with the original TraPT score for our patient population in terms of the performance measures of the 2 tools.
From hospital records, it was estimated that within 6 months, approximately 100 patients would sustain an isolated lower limb fracture. It has been shown previously that approximately 25% of patients would be discharged to inpatient rehabilitation and the sensitivity of the prediction tool would be approximately 70%.4 Based on these figures, the 95% confidence interval around the sensitivity would be ±18%. However, by doubling the time period, the sensitivity would have a precision of ±14%. Given that this calculation provided only minimal improvement, data collection over a 6-month period was chosen. Analysis was performed using Stata version 11.2 (StataCorp, College Station, Texas) with a P value of <.05 considered significant.
Role of the Funding Source
This study was supported by an Alfred Hospital Allied Health Research Grant.
Results
One hundred fourteen participants were enrolled in the study. Twenty participants (17.5%) were discharged to rehabilitation. The demographic profile of participants discharged to inpatient rehabilitation and directly home is shown in Table 1. A TRaPT score cutoff of 50% provided the best balance of sensitivity and specificity in this population, with a score of <50% predicting discharge home. The performance measures of the TRaPT are shown in Table 2. Assuming a cutoff for the TRaPT score of 50%, 10 participants (8.8%) were incorrectly classified by the TRaPT score. Six participants were incorrectly classified as requiring inpatient rehabilitation (actually discharged home), and 4 were incorrectly classified as suitable for discharge home but went to inpatient rehabilitation.
Demographic Characteristics and Preinjury and Injury Event Factors (N=114)a
Performance Measures Comparing the TRaPT and sTRaPTa
Variables relating to open-closed fracture, number of operations, and stairs at home were excluded from further analysis because they were not associated (P<.20) with discharge destination on univariate testing. All other variables, including the REFS, BMI, patient preference, and support at home, were added separately to the TRaPT using logistic regression.
The performance measures of the original TRaPT and the TRaPT with the addition of these variables are shown in Tables 2 and 3. The sTRaPT measures are shown in Tables 4 and 5. In order to further investigate the utility of the TRaPT and sTRaPT, we calculated the LR+ and LR−. The LR− for the TRaPT and sTRaPT showed a moderate shift in the posttest probability of being discharged directly home, and the LR+ for the TRaPT was strong, as it provides an important shift in the posttest probability of discharge to inpatient rehabilitation. The LR+ of the sTRaPT was only moderate. There was no significant improvement in the C-statistic when additional variables were added to the original TRaPT model.
Performance of TRaPT With Additional Variables Added for Prospective Validationa
Simplified Trauma Rehabilitation Prediction Tool (sTRaPT) Scoring System
Calculation of Simplified Trauma Rehabilitation Prediction Tool (sTRaPT) Score: Probability of Discharge to Inpatient Rehabilitation Following Lower Limb Fracturea
Weight-bearing status had perfect association with discharge destination, with all participants able to bear weight discharged directly home. Therefore, this variable could not be included in the logistic regression model. Thirty participants (26%) were allowed to bear weight in some form following fracture fixation. Of the 6 participants who were predicted to require inpatient rehabilitation but were subsequently discharged home, 4 were allowed to either partially or fully bear weight.
Five additional participants were incorrectly classified using the sTRaPT, all of whom went home. They all scored between 45% and 49% on the original TRaPT and between 50% and 62% using the simplified version.
Discussion
This prospective study has confirmed the statistical and clinical validity of the TRaPT in patients with isolated lower limb trauma. Initial tool development relied solely on registry data; therefore, some factors thought to be potentially relevant to discharge planning were not available. This study explored the impact of adding variables to the original TRaPT and reviewed the addition of these factors on the performance measures of the tool. The sTRaPT allows for rapid calculation at the bedside and provides the patient with the opportunity to be included in the discharge planning process. The performance measures of the TRaPT and the sTRaPT were similar, although 5 additional participants would have been incorrectly classified based on this simplified scoring system.
During initial development of the TRaPT, a split-file approach was used with a training dataset to develop the TRaPT and a validation dataset to internally validate this tool. The PPV was 80% in the training dataset but reduced to 68% when tested initially in the validation dataset.4 This finding indicated a reduced capacity to predict who would be discharged to rehabilitation. The calibration curves showed a divergence of the predicted and observed outcomes in those people in whom a rehabilitation discharge was likely, suggesting that additional factors may improve the predictive capabilities of the tool.4. This current prospective validation study collected data for additional variables considered relevant in discharge planning.
Our findings suggest that weight-bearing status is highly influential for determining discharge destination for patients with isolated lower limb fractures. All participants allowed weight bearing (26%) were discharged to home, including 4 of those with TRaPT scores greater than 80%. Weight-bearing status is determined by the surgeon managing the fracture. The decision to limit weight bearing is determined primarily by the stability and type of fracture fixation.19 It is likely that people who are allowed weight bearing have more stable and perhaps less severe fractures than those who require a non–weight-bearing period. Additionally, patients may be more confident and have improved balance when allowed weight bearing, thus making discharge home more likely. Other factors that were considered potentially associated with discharge destination were BMI, number of operations, stairs at home, and whether the fracture type was open or closed. None of these factors significantly improved prediction of the TRaPT model either alone or in combination.
In a previous study, frailty was independently associated with failure to be discharged directly home in a group of patients, most of whom underwent orthopedic procedures.5 The proportion of individuals classified as frail in our study was low, with only 13 participants being deemed frail. Twelve of the 13 patients classified as frail were discharged to inpatient rehabilitation, with 11 having a TRaPT score of higher than 60. Given this finding, the extra time required to perform the REFS may not be justified, although further analysis in a larger population or an older cohort would be recommended to further assess the value of the REFS. We hypothesise that the TRaPT may already account for frailty, as 10 of the 13 participants deemed frail were aged 65 years or older, and most reported preinjury disability of mild or above. Both of these factors are already accounted for in the original TRaPT and increase their probability of discharge to inpatient rehabilitation based on the TRaPT score. Support at home also was examined, as it has been shown to contribute significantly to prediction of discharge destination in the elective arthroplasty population.6 In our study, the addition of support at home did not significantly improve the discrimination but did improve the calibration of the TRaPT (Tab. 3).
Oldmeadow et al,6 in their study of patients who had undergone elective arthroplasty, identified patient expectation as the most heavily weighted of all factors predicting discharge. In our study, only 10 participants reported a preference for discharge to inpatient rehabilitation, and these individuals were older and had a higher TRaPT score than those preferring a discharge directly home. Seven of these participants were discharged to inpatient rehabilitation, including 1 participant who had a BMI of 45 kg/m2 yet scored less than 10% probability of discharge to rehabilitation on the TRaPT. This was thought to be an important factor and improved the calibration of the TRaPT when added to the original score (Tab. 3).
The findings of this study confirm that the TRaPT score has strong predictive capacity and could be a useful adjunct to clinical decision making in terms of discharge destination following lower limb fracture. The TRaPT can assist with early referral of appropriate patients to inpatient rehabilitation, as well as appropriate utilization of acute hospital staff to facilitate discharge directly home for patients predicted to be able to achieve this outcome. The original TRaPT would require a computer for scoring. The usefulness of available tools has been limited by their complexities,8 and as a result, a simplified discharge predictor tool (sTRaPT) was evaluated (Tabs. 2 and 3). The sTRaPT has the advantage of being a visual tool that can be shown to the patient when discussing discharge destination. A score less than zero is representative of home discharge, and a score greater than 2.5 predicts discharge to rehabilitation, regardless of age (Tabs. 4 and 5). Although there are advantages to simplified prediction tools, the trade-off may be a reduction in accuracy. In our study population, 5 additional patients would have been misclassified based on this simplified version, although overall the sTRaPT was found to have little additional error compared with the original TRaPT in the study population.
One option to further reduce the error is considering the sTRaPT as part of a 3-tiered approach to decision making, which would allow for further consideration of patients scoring around 50% on the sTRaPT. The sTRaPT score could be divided into 3 categories: >65% represents probable discharge to inpatient rehabilitation, <40% represents probable discharge directly home, and 40% to 65% represents a gray area in which further factors may be required to assist with discharge planning (Tabs. 4 and 5). All 5 participants who were misclassified using the sTRaPT would fit into this area. These factors may include frailty, support at home, and patient preference, which have been shown to be important predictors of discharge destination in other studies5,6 and improved the discrimination in our study population. Patients who score in this middle region also could be targeted for increased therapy in the acute hospital or increased home support with the aim of achieving discharge directly home. Targeting therapy to a group predicted to be at medium risk of discharge to inpatient rehabilitation has been undertaken previously in patients with elective lower limb arthroplasty (based on a RAPT score), resulting in 82% of patients being discharged directly home, without increase in length of stay.20
Strengths of this study are the real-world, prospective testing and evaluation of the TRaPT. This study has shown the predictive capacity of the TRaPT in a prospective fashion, as it is accurate for patients other than those from whom the data were derived.14 Missing data were limited due to the prospective nature of the research, and a variety of performance measures were used to analyze the models. The limitations of this study revolve mainly around its generalizability. Compensable status is heavily weighted in both the TRaPT and sTRaPT. The TRaPT was designed using data from 4 hospitals in Victoria, Australia, which contribute data to the Victorian Orthopaedic Trauma Outcomes Registry (VOTOR) database. Victoria has a no-fault motor vehicle compensation scheme, the Transport Accident Commission (TAC), which allows for good access to inpatient rehabilitation and may influence the high weighting of compensation status in the TRaPT and sTRaPT. In Australia, Tasmania and Northern Territory also have a no-fault compensation system, and both South Australia and New South Wales are considering this change as well,21 thereby making this tool applicable in most Australian states. There may be concern regarding the heavy weighting of compensation or payment as a driver of discharge to rehabilitation, but it is important to note that the TRaPT is a measure of who goes to inpatient rehabilitation rather than who may actually need rehabilitation; therefore, the ability to pay for this service is a significant factor in discharge decision making.22
The RAPT,6 developed in Victoria, is now used in many countries to assist with discharge planning for patients with elective arthroplasty, giving confidence in terms of the applicability of this tool as well. Small numbers in some groups (ie, support at home, patient preference, and frailty) also may have influenced our results; these factors may be relevant in older or medically unwell populations and, therefore, could be important in the development of tools for predicting discharge in populations such as people with hip fracture, which were specifically excluded from this study. Nevertheless, for patients scoring in the middle (or more uncertain category) of the sTRaPT, it is recommended that these factors be considered. Additionally, our study was underpowered, as we calculated our sample size based on 25% of patients being discharged to inpatient rehabilitation, but this outcome occurred in only 18% in our study cohort. This limitation resulted in wide confidence intervals for the sensitivity and positive predictive values of our tool (with the null hypothesis included in the 95% confidence interval of the positive predictive value), although the specificity and negative predictive value have tighter confidence intervals. The LR+ and LR− for the TRaPT suggest it may be clinically useful in deciding who is most likely to go to inpatient rehabilitation. Caution should be taken, however, when using these scoring systems, as the lower bound of the 95% confidence interval of the LR+ is relatively low and the upper bound of the 95% confidence interval for the LR− is high, probably influenced by the small numbers of participants in this study who were discharged to inpatient rehabilitation.
In conclusion, this study showed that the TRaPT has reasonable predictive capacity in a group other than the patient group from whom the tool was derived, with sensitivity and specificity higher than 80%. The performance of the TRaPT was improved by considering the weight-bearing status of the patient, as all participants able to weight bear were discharged directly home. The sTRaPT also had sensitivity and specificity greater than 80%, with no difference in calibration, compared with the TRaPT and can be used as a quick and easy bedside assessment of discharge destination. For patients in whom discharge destination remains uncertain after using the TRaPT, consideration of additional factors such as frailty, patient preference, and support at home appear to improve the calibration of the tool and may assist in determining the optimal discharge destination. Given the uncertainty in the likelihood ratios with wide 95% confidence intervals, further study is warranted with larger sample sizes to improve our confidence in the TRaPT and sTRaPT scores. We also recommend further evaluation of the tool in other jurisdictions.
Footnotes
Ms Kimmel, Associate Professor Holland, Associate Professor Edwards, and Professor Gabbe provided concept/idea/research design and writing. Ms Kimmel provided data collection and project management. Ms Kimmel, Associate Professor Holland, Ms Simpson, and Professor Gabbe provided data analysis. Professor Gabbe provided facilities/equipment and clerical support. Associate Professor Holland and Professor Gabbe provided institutional liaisons. Associate Professor Holland, Ms Simpson, Associate Professor Edwards, and Professor Gabbe provided consultation (including review of manuscript before submission). The Alfred Hospital provided study participants. The authors acknowledge The Alfred Hospital Physiotherapy Department for assisting with data collection.
Ethics approval from the Alfred Health Human Research Ethics Committee (HREC) and the Monash University HREC was obtained.
This study was supported by an Alfred Hospital Allied Health Research Grant.
- Received September 11, 2013.
- Accepted April 2, 2014.
- © 2014 American Physical Therapy Association
References
Issue highlights
