Abstract
Background Intensive care unit (ICU) stays often lead to reduced physical functioning. Change in physical functioning in patients in the ICU is inadequately assessed through available instruments. The de Morton Mobility Index (DEMMI), developed to assess mobility in elderly hospitalized patients, is promising for use in patients who are critically ill.
Objective The aim of this study was to evaluate the clinimetric properties of the DEMMI for patients in the ICU.
Design A prospective, observational reliability and validity study was conducted.
Methods To evaluate interrater and intrarater reliability (intraclass correlation coefficients), patients admitted to the ICU were assessed with the DEMMI during and after ICU stay. Validity was evaluated by correlating the DEMMI with the Barthel Index (BI), the Katz Index of Independence in Activities of Daily Living (Katz ADL), and manual muscle testing (MMT). Feasibility was evaluated based on the percentage of participants in which the DEMMI could be assessed, the floor and ceiling effects, and the number of adverse events.
Results One hundred fifteen participants were included (Acute Physiology and Chronic Health Evaluation II [APACHE II] mean score=15.2 and Sepsis-related Organ Failure Assessment [SOFA] mean score=7). Interrater reliability was .93 in the ICU and .97 on the wards, whereas intrarater reliability during the ICU stay was .68. Validity (Spearman rho coefficient) during the ICU stay was .56, −.45, and .57 for the BI, Katz ADL, and MMT, respectively. The DEMMI showed low floor and ceiling effects (2.6%) during and after ICU discharge. There were no major adverse events.
Limitations Rapid changes in participants' health status may have led to underestimation of intrarater reliability.
Conclusion The DEMMI was found to be clinically feasible, reliable, and valid for measuring mobility in an ICU population. Therefore, the DEMMI should be considered a preferred instrument for measuring mobility in patients during and after their ICU stay.
Due to critical illness and prolonged inactivity, survivors of the intensive care unit (ICU) often show decreased physical functioning and mobility, limiting their daily activities and quality of life.1–3 Previous research has shown the effectiveness of physical therapy interventions in patients who are critically ill. Early stimulation of physical activity and mobilization decreased the duration of mechanical ventilation, delirium, and length of stay in the ICU and hospital and improved physical functioning and quality of life.4–7
To facilitate physical therapy goal setting and the evaluation of the recovery process in patients in the ICU, an accurate assessment of physical activities of patients in the ICU based on the domains of the International Classification of Functioning, Disability and Health (ICF) framework is required.8,9 The ICF framework consists of 3 core domains to describe the level of functioning: (1) body functions and structures, (2) activities, and (3) participation.8,9 It also has been proposed as a model in which measurements can be organized.8,9 Measuring physical activities in patients in the ICU is complex due to the different stages of the disease, which influences task completion, coordination, processing of visual information, and central and peripheral motor drive.8 The instruments frequently used for physical measuring activities within the ICF domain in the ICU are the Barthel Index (BI), the Katz Index of Independence in Activities of Daily Living (Katz ADL), the Physical Function ICU Test (PFIT), and the Functional Status Score for the Intensive Care Unit (FSS-ICU).
The BI and the Katz ADL are frequently used to measure activities in clinical settings and are reliable and valid tools, measuring physical performance capacity of 10 basic activities of daily living (ADL).10,11 These are multidimensional instruments for measuring ADL tasks (eg, transfers, mobility, stair climbing), but some activities are not applicable for all patients in the ICU (eg, bladder and bowel function, bathing, clothing, stair climbing). The PFIT is also a reliable and valid instrument (intraclass correlation coefficient [ICC]=1, minimal clinically important difference=1.5/10 points) for measuring physical activities (eg, muscle strength, coming to a standing position, walking) in patients in the ICU.12 Patients entering the ICU who are not capable of performing transfers or coming to a standing position may not be able to perform the activities tested by the PFIT. The PFIT contains items that are too difficult for many patients who are critically ill, such as performing out-of-bed tasks, resulting in notable floor effects within the ICU and ceiling effects from 20% at discharge.12 The FSS-ICU is primarily designed to measure the physical performance of low-level activities (eg, rolling, supine-to-sit transfers, unsupported sitting, sit-to-stand transfers, ambulation).13 Clinimetric properties of this instrument have not yet been reported. In summary, the currently available instruments are not fully appropriate for measuring the level of activities of patients who are critically ill during the various stages of recovery because they include items that may be too difficult (eg, walking, dressing) or lack relevant ICU activities such as bed mobility skills (eg, rolling, transfer from supine to sitting position, sitting balance).8 These limitations result in “floor effects” (the inability of a test to measure below a certain point because its items are too difficult) in scores observed in the ICU and “ceiling effects” (the inability of a test to measure above a certain point because its items are too easy) after discharge.8,14
The de Morton Mobility Index (DEMMI) was originally developed to measure the full range of mobility within the ICF activity domain in elderly patients admitted to the hospital.15 It consists of 15 hierarchical mobility items (3 bed, 3 chair, 4 static balance, 2 walking, and 3 dynamic balance items). The total score is converted with Rasch analysis to an interval score ranging from 0 to 100, where 0 represents poor mobility and 100 indicates high levels of independent mobility. The DEMMI is a freely available at http://www.demmi.org.au. However, it includes items that also appear to be relevant and feasible for patients who are critically ill in the ICU (eg, bed mobility and transfers) and after ICU discharge (eg, ambulation and jumping). It seems feasible, therefore, to evaluate recovery throughout the rehabilitation trajectory during and after an ICU stay.8 The unidimensionality of the DEMMI was confirmed by Rasch modelling, and the instrument showed strong clinimetric properties within a diverse range of elderly people with acute and chronic illnesses in different settings (clinical ward, rehabilitation, and community).15–20 A Dutch translation of the DEMMI was validated and found to be reliable.18
Due to the lack of a suitable instrument for measuring activities in patients in the ICU in detail, and in view of the promising measurement properties of the DEMMI in various hospital populations and rehabilitation and community settings, the aim of this study was to evaluate the feasibility, reliability, and validity, focusing especially on floor and ceiling effects of the DEMMI in a population in the ICU.
Method
Study Design and Setting
This prospective, observational reliability and validity study was performed in the Academic Medical Center (AMC) in Amsterdam, a 1,000 bed university hospital with 34-bed mixed medical and surgical format ICUs and medium intensive care units (M-ICUs). Patients in the M-ICU of the AMC characteristically are off mechanical ventilation but still require a high level of care and full-time monitoring. Measurements were performed within the ICU and M-ICU and on the regular wards until hospital discharge.
Participants
In a 3-month period (November 2013–January 2014), we included all consecutive adult patients (age 18 years or older) admitted to the ICU and M-ICU who were referred for physical therapy. Patients admitted for 24 hours at the ICU or M-ICU and for whom it was safe to perform physical exercises according to the safety criteria as described in the evidence statement by Sommers et al21 (Appendix) were included in the study. Exclusion criteria were neurological or neurosurgical admission diagnosis, imminent death, and insufficient comprehension of the Dutch language. Sample size calculations indicated that 101 participants were needed to estimate the ICC statistics with 95% confidence (±0.2%) around the point estimate, assumed to be .70.22
Procedure
Participant characteristics (sex, age, medical category, and Sepsis-related Organ Failure Assessment [SOFA] score) were recorded from the medical charts. Participants were measured at 2 time points: at admission to the ICU or M-ICU (T0ICU and T0M-ICU) and after discharge from the ICU or M-ICU at the regular hospital ward (T1). The interrater reliability of the DEMMI was evaluated by simultaneous and independent measurements by 2 physical therapists at T0 and T1. The DEMMI was executed according to the standard procedure as described by de Morton at al.15 One physical therapist provided the instructions and guided the patient, while the other therapist observed without interference. Therapists were blinded from each other's scoring forms. To examine intrarater reliability, each measurement was repeated by both assessors within a 1-hour assessment at T0 to reduce the influence of fatigue and the occurrence of rapid changes in medical conditions. Data from all of the assessments were entered into a database by a researcher who was not involved in the assessments. To evaluate validity, the BI, Katz ADL, and manual muscle testing (MMT) using the Medical Research Council sum-score (MRC-SS) were administered by a physical therapist at all assessment occasions. The MRC-SS has a total score ranging from 0 to 60 points obtained from bilateral testing of 6 muscle groups, where the minimal score is 0 (indicating no muscular contraction) and the maximum score is 5 (indicating normal muscle strength). In the sequence of performing measures, we started with MMT, followed by questioning items of the Katz ADL and the BI and, after a rest period of 30 minutes, measuring the items of the DEMMI. All assessments were performed by 6 trained and experienced ICU physical therapists involved in the treatment of patients in the ICU. The feasibility of the DEMMI was measured by recording the number of patients who were referred for physical therapy for whom the DEMMI could be administered and the adverse events that occurred during the measurements (eg, fall to knees or ground, loss of consciousness, cardiac arrest, dislodgement of medical equipment) and by analysis of floor and ceiling effects at the ICU and the regular hospital ward.
Assessments
The raw DEMMI (a 15-item unidimensional measure of mobility, ranging from 0 to 19 points) was Rasch converted to a 0 to 100 interval scale. A score of 0 indicates no mobility, and a score of 100 points represents full mobility.15 The Dutch-translated version of the original DEMMI was used, which has shown validity and reliability coefficients similar to those of the original version.15,18 The BI and the Katz ADL were used to assess ADL.23 The BI consists of 10 items, ranging from 0 to 20 points, with a higher score indicating better functioning.24 This index includes 2 mobility-related items that are also part of the DEMMI, as well as additional items on ADL and bladder- and bowel incontinence. The Katz ADL consists of 6 items on ADL and incontinence. A score of 0 indicates that the patient is independent in performing ADL, and a score of 6 points indicates full dependence for these activities.25,26 Manual muscle testing was used to assess muscle weakness and was performed according to the scoring system of the Medical Research Council scale for muscle strength (MRC), in which a score of 0 indicates no contraction and the maximum score of 5 indicates contraction against strong resistance.27,28 For measuring the MRC-SS, 6 muscle groups were assessed bilaterally: abduction of the arm, flexion of the elbow, dorsiflexion of the wrist, flexion of the hip, extension of the knee, and dorsiflexion of the ankle. The total score ranges from 0 to 60 points; a cutoff point of <48 points was used to indicate significant muscular weakness, and a score of <36 points indicates severe muscular weakness.28
The BI, Katz ADL, and MMT served as convergent validity measures of the DEMMI. To classify the severity of the disease at ICU admission, the Acute Physiology and Chronic Health Evaluation II (APACHE II) score was used. The APACHE II is a severity-of-disease classification system based on physiological measurements, such as body temperature, respiratory rate, and white blood cell count.29 The total score ranges from 0 to 71 points, where higher scores correspond to more severe disease.29 The APACHE II does not include items on mobility; therefore, it was used to assess the divergent validity of the DEMMI.
Data Analysis
The interrater and intrarater reliability of the DEMMI total score were calculated using the one-way random ICC model. Kappa values were calculated to evaluate the reproducibility of items with a binary response scale, and weighted kappa values with quadratic weights were calculated for items 3, 5, 11, and 12, having a polytomous response scale.
Convergent validity was calculated by using Spearman rho correlation coefficients for the DEMMI with the BI, Katz ADL, and MMT. Divergent validity was evaluated likewise for the DEMMI with the APACHE II score. A high correlation was defined as having a rho correlation coefficient of 1, a strong correlation was defined as having a rho correlation coefficient of .7 to .9, and a moderate correlation was defined as having a rho correlation coefficient between .4 and .6.30 Known-groups validity (that is, validity for groups that would be expected to differ in their mobility) of the DEMMI was assessed using the following ICU-acquired weakness (ICU-AW) categories: MRC-SS cutoff point <36 (severe weakness), MRC-SS cutoff point <48 (significant weakness), and MRC-SS values ≥48 (no weakness).28,31 The Kruskal-Wallis test and eta-squared effect sizes (ES; 0.02=small, 0.13=medium, and >0.26=large) were used to assess the ability of the DEMMI, BI, and Katz ADL to detect differences in ICU-AW score groups.32 Post hoc Mann-Whitney U tests with Bonferroni corrections were performed to analyze possible statistical differences between the ICU-AW subgroups detected by the DEMMI.
Feasibility of the instruments in the ICU sample (ie, their content validity or the extent to which the measures are targeted to the sample) was evaluated by examining score distributions and by plotting histograms with normal curves and calculating the percentage of participants with a minimum or maximum score.
Sensitivity to change was analyzed by calculating the minimal detectable change at the 90% level of confidence (MDC90). The MDC90 should suffice in patients who are critically ill, where safety criteria were already applied, and to track improvement in rehabilitation. By calculating the MDC90, comparison with earlier reported sensitivity to change is possible (eg, MDC90=8.9 in study by de Morton et al15).33
All statistics were interpreted as significant with P values of .05 or lower and associated 95% confidence intervals (95% CIs). Statistical analyses were performed using IBM SPSS Statistics for Windows version 20.0 (IBM Corp, Armonk, New York).
Results
During the 3-month inclusion period, 361 patients were admitted to the ICU or M-ICU; 246 of these patients were excluded because they did not meet the inclusion criteria (Fig. 1). In total, 115 eligible patients participated in this study, of whom 77 (67%) were admitted to the ICU and 38 (33%) to the M-ICU (Tab. 1). Fifty-three percent of the included patients were admitted due to elective surgery (scheduled thoracic and abdominal surgery), 33% due to medical indications (sepsis, respiratory insufficiency due to pneumonia, and infectious diseases), and 14% due to nonelective surgery. The baseline scores and interquartile ranges of the DEMMI, BI, and Katz ADL are presented in Table 1. Of the 115 patients at T0, 86 were assessed after a mean of 6 days (SD=9) at the regular hospital wards at T1. A flowchart of the study sample is provided in Figure 1. During the measurements at T0 and T1, no major adverse events were reported. The administration of the DEMMI could be performed in all patients in the ICU and in patients at the regular hospital ward.
Flowchart of study population and measurements. T0=time of inclusion in either the intensive care unit (ICU) or the medium ICU (M-ICU), T1=time after discharge from the ICU or M-ICU at the regular hospital ward.
Characteristics of the Study Sample at Admission to the ICU and M-ICU (T0) and Regular Hospital Wards (T1)a
The interrater and intrarater reliability of the DEMMI sum-scores and item scores during the assessments are presented in Table 2. The interrater reliability was .93 at T0 and .97 at T1. The intrarater reliability was .68 at T0. Reproducibility of the DEMMI items ranged from .52 for item 4 (sit unsupported in chair) to 1.00 for items 9, 10, 13, and 15 (Tab. 2). Similar kappa values were found at T1.
Clinimetric Properties of the DEMMI and Reproducibility of the DEMMI at Each Assessmenta
Convergent, divergent, and known-groups validity coefficients are presented in Table 3. For the BI, Katz ADL, and MMT with the DEMMI, the convergent validity coefficient was 0.56, −0.45, and 0.57, respectively, at T0 and 0.75, −0.76, and 0.63, respectively, at T1. Divergent correlation with the APACHE II score was −0.18.
Clinimetric Properties of the DEMMI and Validity of the DEMMI at Each Assessmenta
For known-groups validity, significant differences (Kruskal-Wallis test, P<.001) in DEMMI scores were observed among the ICU-AW categories (Tab. 3). The eta-squared ES for the relationship with ICU-AW groups was 0.22 for the DEMMI, 0.14 for the BI, and 0.17 for the Katz ADL at T0. These eta-squared ES values were similar to those found at T1, as shown in Table 3. Post hoc analysis showed that the DEMMI was able to differentiate among the 3 ICU-AW categories at T0 and T1 (Tab. 3). However, the DEMMI showed no significant ability to differentiate between severe and significant weakness (P=1.00) at T1. Also, no significant differences were observed between the severe weakness group and the no weakness group at T1.
The score distributions of the DEMMI, Katz ADL, and BI for the different assessment occasions are presented in Fig. 2. At T0, a floor effect was shown in 30 participants (26.1%) in the Katz-ADL results and in 10 participants (8.7%) for the BI, whereas the DEMMI showed the lowest proportion of the floor effect in 3 participants (2.6%). Ceiling effects were observed only in the Katz ADL scores in 3 participants (2.6%) at T0 (Fig. 2A). At T1, the DEMMI scores did not show a floor effect, whereas the BI scores showed one participant (0.9%) with the lowest score. The Katz ADL had a floor effect in 4 participants (3.5%). At T1, the Katz ADL scores showed 22 participants (19.1%) with a ceiling effect, whereas a low ceiling effect was shown by the DEMMI in 3 participants (2.6%), and the BI did not show any ceiling effects (Fig. 2B).
Floor and ceiling effects of the de Morton Mobility Index (DEMMI), Barthel Index, and Katz Index of Independence in Activities of Daily Living (Katz ADL) (A) at time of inclusion in either the intensive care unit (ICU) or the medium ICU (M-ICU) (T0) and (B) at time after discharge from the ICU or M-ICU at the regular hospital ward (T1).
The MDC90 score for sensitivity to change was 6.73 points at T0 (Tab. 4). This value was similar to that at T1 at the regular hospital ward, where 8.23 points was found for the MDC90 score.
Clinimetric Properties of the DEMMI and Sensitivity to Change of the DEMMI at Each Assessmenta
Discussion
In the present study, we showed that the DEMMI is a feasible, reliable, and moderately valid instrument with minimal floor or ceiling effects to measure mobility within the ICF activity domain in patients who are critically ill in the ICU, M-ICU, and regular hospital ward. The intrarater and interrater reliability were high on all assessment occasions, and the validity of the DEMMI was shown by moderately convergent correlations with the Katz ADL, BI, and MMT and a low divergent correlation with the APACHE II. As a result, the DEMMI has shown good clinimetric properties to assess activity levels in patients who are critically ill.
Within the literature, other instruments for measuring ADL or mobility, such as the BI and the Katz ADL, have been shown to be reliable and valid.2,14 However, they have the disadvantage of being multidimensional, including several items (eg, bladder, bowel) nonrelevant to patients within the ICU. These items are scored negatively during ICU stay, leading to a floor effect in the ICU. The Physical Function ICU Test short version (PFIT) and the FSS-ICU are recently developed instruments measuring unidimensional mobility in patients in the ICU.12,13 The clinimetric properties of the PFIT have been studied, and the applicability in a population in the ICU was confirmed by Rasch modeling.12,34 Nonetheless, it contains items that are too difficult for many patients in the ICU, resulting in notable floor effects in the ICU. The ceiling effects at discharge (20%) were probably caused by the highest-order item of the PFIT (ie, marching), whereas other higher-order tasks (eg, walking away from the bed) were possibly needed to realize a lower ceiling effect.12 The FSS-ICU seems appropriate for the use in patients who are critically ill, as it includes relevant ICU functional tasks, such as rolling, supine-to-sit transfers, sitting at the edge of bed, and sit-to-stand transfers. However, the reliability and validity of the FSS-ICU in an ICU setting and other settings have not been reported.8,35
Our results illustrate that the DEMMI does not share these limitations due to the range of items in the ICU or regular hospital wards. The DEMMI could be administered in all patients in the ICU and in patients at the regular hospital ward. Moreover, in a recent systematic review by Parry et al,8 a schematic guide for the use of outcome measures for patients who are critically ill within the ICF framework was recommended. Based on the clinimetric properties in various populations outside the ICU, the DEMMI was proposed for the measurement of mobility after ICU discharge at the regular ward and after return to the community.8 Our study showed good clinimetric properties and feasibility of the DEMMI when used in patients who are critically ill in the ICU and, therefore, should be considered as a standard clinimetric tool.
The clinimetric properties of the DEMMI found in the present study are similar to those found in previous studies. De Morton et al17 found a reliability coefficient of .87 in a general medical population. Other research groups confirmed these results in a different general elderly population with knee or hip osteoarthritis.18 The research group of de Morton et al15 showed comparable results regarding the validity of the DEMMI, with a correlation of .68 with the BI, as well as regarding the divergent validity, showing a correlation of .07 with the APACHE II. The MDC90 in our study was lower than that reported by de Morton et al.16
Our study had some intrinsic limitations. Our results were based on a mixed patient group (n=115) with a mean APACHE II score of 15.2 at T0. This relatively low severity of illness score at ICU admission could be attributed to the large group of patients with elective surgery. In our sample, some of these patients developed severe complications, such as sepsis, acute respiratory distress syndrome, and multiple organ failure, regardless of the low APACHE II score at admission. These findings are illustrated by the calculated SOFA score, representing the severity of illness score for hospital mortality and morbidity (mean=7, SD=3.6). In calculating the interrater and intrarater reliability, the study sample size (n=115) is comparable to that of other reliability studies of the DEMMI.15–17 To improve the generalizability of our study, increasing the sample size and inclusion of multiple ICU hospitals and other clinical wards would have been preferable.
Another consideration in this study was that physical assessment in patients in the ICU is complicated due to critical pulmonary and hemodynamic conditions necessitating medication and invasive equipment. In addition, due to critical illness, this medical situation might change rapidly.36 This factor might have biased the results with respect to the intrarater reliability. We found that the intrarater reliability was lower than the interrater reliability. This finding also might have been the cause of a relatively low ICC of the intrarater reliability below .9 at T0, as these measurements were performed within 1 hour due to the fact that the DEMMI should be administered in total at the same time. In this critically ill population, it is expected that the ability to perform mobility activities will change within an hour due to fatigue and exertion. Another limitation is that we did not determine the intrarater reliability at T1 because we anticipated practical feasibility issues for the repeated measurements in patients on the regular ward. Based on a previous study by de Morton et al17 showing high intrarater reliability (Pearson r=.86) in elderly patients with frailty at a hospital ward, we assumed high intrarater reliability on the regular ward because of the stabilized pulmonary and hemodynamic conditions of the patients compared with the ICU situation. Finally, this study could be criticized for the loss to follow-up, primarily due to early hospital discharge or mortality (Fig. 1). We do not think that the loss to follow-up biased our results because the remaining sample size retained sufficient power to perform the analysis.
The DEMMI provides the physical therapist with accurate and reliable information on the level of mobility in patients in the ICU. Its clinimetric properties and the range of items indicate that the DEMMI is a clinically feasible instrument. Due to minimal floor or ceiling effects, it has great potential to be used throughout the rehabilitation process of patients who are critically ill.
The ICU health care team, therefore, should recommend measuring the functional status of patients in the ICU at each stage of critical illness within the ICU and hospital regular wards until discharge. Further research is indicated to evaluate the hierarchical structure of the mobility items in the DEMMI in an ICU population.
Appendix.
Summary of Contraindications for Physical Therapy21,a
a MAP=mean arterial pressure, Spo2=oxygen saturation, Fio2=fractional concentration of inspired oxygen, PEEP=positive end expiratory pressure, RASS=Richmond Agitation Sedation Scale, ICP=intracranial pressure.
Footnotes
Mrs Sommers, Mr Vredeveld, Dr Engelbert, and Dr van der Schaaf provided concept/idea/research design. All authors provided writing. Mrs Sommers and Mr Vredeveld provided data collection. Mrs Sommers, Mr Vredeveld, Mr Lindeboom, Dr Engelbert, and Dr van der Schaaf provided data analysis. Dr Engelbert and Dr van der Schaaf provided project management. Dr van der Schaaf provided fund procurement and institutional liaisons. Mr Lindeboom, Dr Engelbert, and Dr van der Schaaf provided consultation (including review of manuscript before submission). The authors thank physical therapists Robin Kwakman, Denise Wieferink, Daniela Dettling, Dennis Gommers, and Tineke van Heuveln for their comprehensive guidance of patients during the data collection of this study.
The local medical ethics committee provided a waiver for this study.
- Received June 11, 2015.
- Accepted March 31, 2016.
- © 2016 American Physical Therapy Association
References
Issue highlights
