Abstract
Background Advances in sensor technologies and signal processing techniques provide a method to accurately measure walking activity in the home and community. Activity monitors geared toward consumer or patient use may be an alternative to more expensive monitors designed for research to measure stepping activity.
Objective The objective of this study was to examine the accuracy of 2 consumer/patient activity monitors, the Fitbit Ultra and the Nike+ Fuelband, in identifying stepping activity in people with stroke and traumatic brain injury (TBI). Secondarily, the study sought to compare the accuracy of these 2 activity monitors with that of the StepWatch Activity Monitor (SAM) and a pedometer, the Yamax Digi-Walker SW-701 pedometer (YDWP).
Design A cross-sectional design was used for this study.
Method People with chronic stroke and TBI wore the 4 activity monitors while they performed the Two-Minute Walk Test (2MWT), during which they were videotaped. Activity monitor estimated steps taken were compared with actual steps taken counted from videotape. Accuracy and agreement between activity monitor estimated steps and actual steps were examined using intraclass correlation coefficients (ICC [2,1]) and the Bland-Altman method.
Results The SAM demonstrated the greatest accuracy (ICC [2,1]=.97, mean difference between actual steps and SAM estimated steps=4.7 steps) followed by the Fitbit Ultra (ICC [2,1]=.73, mean difference between actual steps and Fitbit Ultra estimated steps=−9.7 steps), the YDWP (ICC [2,1]=.42, mean difference between actual steps and YDWP estimated steps=−28.8 steps), and the Nike+ Fuelband (ICC [2,1]=.20, mean difference between actual steps and Nike+ Fuelband estimated steps=−66.2 steps).
Limitations Walking activity was measured over a short distance in a closed environment, and participants were high functioning ambulators, with a mean gait speed of 0.93 m/s.
Conclusions The Fitbit Ultra may be a low-cost alternative to measure the stepping activity in level, predictable environments of people with stroke and TBI who can walk at speeds ≥0.58 m/s.
The recovery of walking ability is a primary goal for many people with stroke, traumatic brain injury (TBI), and other neurological disorders.1–4 The clinical gold standard for measuring walking ability is gait speed.5,6 However, gait speed may not accurately reflect the actual walking activity that people with neurological disorders engage in while at home and in the community.7–9 Advances in sensor technologies and signal processing techniques may provide a method to accurately and precisely measure walking activity in the home and community in an unobtrusive manner.10–13 Accelerometers, gyroscopes, and force-sensitive resistors are among the different types of sensors that have been used either alone or in combination as wearable, mobile health activity monitors.12,14,15 The sensors generate different signal patterns as patients move about their home and community wearing the activity monitor. These patterns then are analyzed using machine-learning techniques13,14,16 to identify the number of steps the wearer has taken.
Examples of activity monitors that are currently available commercially include the StepWatch Activity Monitor (SAM) (Orthocare Innovations, Oklahoma City, Oklahoma),17–19 Intelligent Device for Energy Expenditure and Physical Activity (IDEEA) (MiniSun LLC, Fresno, California),20 activPAL (PAL Technologies Ltd, Glasgow, United Kingdom),21 and ActiGraph (ActiGraph LLC, Pensacola, Florida).22 These activity monitors gather a variety of different types of data such as steps taken, activity counts, time in sedentary versus upright postures, and energy expenditure. These activity monitors appear to be primarily designed for use by researchers and may not be easily usable in a clinical environment due to cost and time constraints in setting up the device and examining the data. Also, some of these activity monitors may not be accurate in identifying stepping activity in people with neurological disorders and older adults who walk at slower speeds.21,23,24
More recently, activity monitors that are geared toward consumer/patient use have been developed. Devices such as the Nike+ Fuelband (Nike Inc, Beaverton, Oregon), Fitbit Ultra (Fitbit Inc, San Francisco, California), and BodyMedia (BodyMedia Inc, Pittsburgh, Pennsylvania) track steps taken, calories burned, and other health-related variables. These activity monitors also have interactive websites on which the user can view and interact with their data. These body-worn sensors appear to be geared primarily toward promoting weight loss.25 eTable 1 provides an overview and comparison of some of the different research-grade and consumer/patient-grade body-worn activity monitors.
These consumer/patient activity monitors might be a useful alternative to the more research-geared activity monitors for physical therapists to track walking activity in people with stroke and TBI. However, the algorithms used to analyze the sensor data and identify stepping activity in these types of activity monitors often are developed from people who are healthy. Because people with stroke and TBI may walk at slower speeds and may have different movement patterns than people without stroke and TBI, these activity monitors may not be accurate or reliable in these populations. To our knowledge, the accuracy of these consumer/patient-geared activity monitors to identify stepping activity has not been established in people with stroke and TBI. Therefore, the purpose of this study was to examine the accuracy of 2 such activity monitors, the Fitbit Ultra and Nike+, in identifying stepping activity in people with stroke and TBI. Secondarily, we sought to compare the accuracy of these activity monitors with that of the SAM, which has been found to provide reliable and valid measurements of daily step activity in people with stroke,17–19 and a pedometer.
Method
Potential participants were recruited from 4 different sites (1 from the Southwest, 2 from the East Coast, and 1 from the Midwest). Inclusion criteria were: diagnosis of stroke or TBI; able to walk with at most minimal assistance, assistive devices and orthotic devices allowed; and able to follow commands to participate in the study and provide informed consent. Potential participants with communication impairments were excluded. All participants provided informed consent, and the study was approved by all participating institutional review boards.
The following demographic information and clinical tests were recorded and performed to characterize the sample: age, date of injury or diagnosis, location of injury or diagnosis, Fugl-Meyer lower extremity motor score on the most affected lower extremity (FM-MALE), Berg Balance Scale (BBS),26–28 and comfortable gait speed (CGS).29–32
Instruments
Four different activity monitors were used: Fitbit Ultra, Nike+ Fuelband, Yamax Digi-Walker SW-701 pedometer (YAMAX Health & Sports Inc, San Antonio, Texas) (YDWP), and SAM. The Fitbit Ultra activity monitor is a small, commercially available device that is clipped to the user's belt or waistband or can be worn in a pants pocket. The device contains a 3-dimensional accelerometer and altimeter, and based on proprietary algorithms, the device is able to estimate number of steps taken, flights of stairs climbed, distance walked, and calories expended. The Nike+ Fuelband activity monitor is a wristband that contains a 3-dimensional accelerometer and uses proprietary algorithms to estimate number of steps taken, calories burned, and Nike Fuel Points. The manufacturer states that Nike Fuel Points are a measure of overall movement and activity. The YDWP is a pedometer that is clipped on the belt or waistband of the user and is able to estimate number of steps taken, calories burned, and distance walked. The SAM is an accelerometer-based step activity monitor that is worn on the less affected lower extremity. It consists of an accelerometer and electronic filter that detect leg movements from which step counts are determined. The SAM is 75 × 50 × 20 mm and weighs approximately 38 g. Data are downloaded through a docking station to a personal computer. The SAM is a valid and reliable measure of steps taken per day in people with stroke.17–19
Procedure
Participants performed the Two-Minute Walk Test (2MWT)33 while wearing all of the activity monitors. During the 2MWT, participants walked at their normal, comfortable pace. All participants were supervised, as necessary, for safety by one of the researchers. The 2MWT was selected so that all participants would likely be able to continuously step for the entire time of data collection. The Fitbit Ultra and YDWP were both worn on the belt or waistband on the side of the less affected lower extremity between the anterior superior iliac spine and umbilicus. The Nike+ Fuelband was worn on the wrist of the less affected upper extremity, and the SAM was strapped above the lateral malleolus of the less affected lower extremity, as recommended by the manufacturers. The SAM was calibrated for each participant's height and weight according to the manufacturer's instructions prior to performing the walking tests. We placed the activity monitors on the less affected side in order to potentially minimize any abnormal movement patterns that may be detected by the different monitors. Participants wore all 4 activity monitors simultaneously, which did not interfere with their walking.
At the beginning of the 2MWT, the participants started in a standing position and began walking when instructed by one of the researchers. The Fitbit Ultra and Nike+ Fuelband step counts cannot be reset to 0 within the course of a 24-hour period. Accordingly, for these monitors, the number of steps displayed prior to and upon completion of the 2MWT was recorded by the researcher while the participant was in a quiet standing position immediately prior to starting and upon completion of the 2MWT. The number of steps recorded by the Fitbit Ultra and Nike+ Fuelband was calculated and recorded as the difference between step counts from the beginning and end of each walk test. The YDWP was set to 0 prior to the 2MWT, and the step count at the end of the 2MWT was recorded. Because the number of steps taken is not displayed on the SAM, it was removed after the 2MWT was completed and connected to a computer where the number of steps estimated by the SAM was downloaded and recorded.
Written step-by-step instructions for the use and application of each device and for the 2MWT were developed and sent to each site for review. Investigators at each site then participated in a conference call to review procedures as a group and to discuss any questions or concerns.
Criterion Standard
A research assistant videotaped the participants as they performed the 2MWT. The gold standard of actual steps taken was determined by having a researcher count steps taken by the participants from the video record. Researchers counted the number of steps taken from the video on 2 occasions separated by at least 1 week. The ICC [3,1] for agreement between these 2 counts was .99. We used the data from the first count of the video record in all the analyses. We set a target to recruit at least 50 participants, which is the necessary sample size for the statistical analysis described below.34
Data Analysis
The accuracy of the different activity monitors for estimating steps was analyzed using intraclass correlation coefficient (ICC [2,1]) and the Bland-Altman method. The ICC [2,1] was used to examine the relative reliability between the different activity monitor identified steps and actual steps taken during the 2MWT determined from the video record.35 We used the Bland-Altman method34,36–38 to examine the absolute reliability between the different activity monitor estimated steps and actual steps taken, as follows: the differences between the activity monitor estimated steps and actual steps taken were plotted against the average measures obtained by the 2 methods. Horizontal lines were drawn at the mean difference and at the 95% confidence interval (95% CI) to identify any outliers or systematic trends.39
We also examined the accuracy of the different activity monitors for 3 subgroups: participants with stroke, participants with TBI, and all participants who walked ≥0.58 m/s using the same methods described above. This speed was chosen as it represents people with stroke who are classified as least limited-community and full-community ambulators.40 This subgroup analysis was performed because there is some indication in the literature that activity monitors are more accurate in detecting steps in people who walk at closer to age norm walking speeds and not as accurate in people who walk at slower speeds.21,23,24,41,42
Pearson correlation coefficients were used to explore the relationship between the error of the different activity monitors (difference between actual steps and different activity monitor estimated steps) and participant characteristics (CGS, FM-MALE, BBS). This analysis was performed to investigate the potential impact of walking speed, balance, and motor function on the accuracy of the different activity monitors. Two-sided correlations were performed, with a Bonferroni correction factor for 12 correlations, resulting in P=.004.
Results
A total of 50 people across the 4 sites participated in the study: 30 with stroke and 20 with TBI (Tab. 1). The mean CGS was 0.93 m/s. Nine participants used an assistive device (5 used a cane, 3 used a small-base quad cane, and 1 used a hemiwalker) and 8 used an ankle-foot orthosis. During the 2MWT, participants took a mean (SD) of 195.4 (32.7) steps and covered a mean distance of 107.2 (35.5) m. One site did not have access to a SAM (20 participants with TBI and 4 participants with stroke), and 1 site did not have access to a Nike+ Fuelband activity monitor (10 participants with stroke).
Participant Characteristics
Of the different activity monitors, the SAM demonstrated the greatest accuracy (ICC [2,1]=.97, mean difference between actual steps and SAM estimated steps=4.7 steps). The next most accurate activity monitor was the Fitbit Ultra, followed by the YDWP and the Nike+ Fuelband (Tab. 2 and Figure). The Bland-Altman plot and 95% CI revealed that the SAM systematically overestimated steps taken, while the Fitbit Ultra, YDWP, and Nike+ Fuelband all systematically underestimated steps. The distribution of the points in the Bland Altman plot for the SAM (Figure, graph A) suggests that the SAM was accurate across all participants, with the likely exception of 2 outliers in whom it overcounted steps. The distribution of the points in the Bland-Altman plot for the Fitbit Ultra (Figure, graph B), with clustering of points along the bias line in participants who took more steps during the 2MWT and a slightly wider 95% CI, suggests the Fitbit Ultra was generally accurate in participants who took more steps. The points on the Bland-Altman plots for the YDWP and Nike+ Fuelband (Figure, graphs C and D) are more widely scattered, with a wider 95% CI, which suggests that these 2 monitors were not accurate in estimating steps in this sample. The number of steps taken during the 2MWT, however, appears to play a small role in the accuracy of both of these activity monitors. A similar pattern was seen in the subanalysis for participants with stroke (Tab. 3), those with TBI (Tab. 4), and those who walked at a speed of ≥0.58 m/s (eTab. 2).
Accuracy of Different Activity Monitors in All Participants (N=50)a
Bland-Altman plots: (A) Bland-Altman plot demonstrating step count agreement between actual steps taken counted from video and StepWatch Activity Monitor estimated steps. Solid bold line is the mean difference between video counted steps and StepWatch Activity Monitor estimated steps, dashed lines are the 95% confidence interval. (B) Bland-Altman plot demonstrating step count agreement between actual steps taken counted from video and Fitbit Ultra activity monitor estimated steps. Solid bold line is the mean difference between video counted steps and Fitbit Ultra estimated steps, dashed lines are the 95% confidence interval. (C) Bland-Altman plot demonstrating step count agreement between actual steps taken counted from video and Yamax Digi-Walker SW-701 pedometer estimated steps. Solid bold line is the mean difference between video counted steps and Yamax Digi-Walker SW-701 pedometer estimated steps, dashed lines are the 95% confidence interval. (D) Bland-Altman plot demonstrating step count agreement between actual steps taken counted from video and Nike+ Fuelband activity monitor estimated steps. Solid bold line is the mean difference between video counted steps and Nike+ Fuelband estimated steps, dashed lines are the 95% confidence interval. Note that the scale on the y-axis is not the same between the graphs.
Accuracy of the Different Activity Monitors in Participants With Stroke (n=30)a
Accuracy of the Different Activity Monitors in Participants With TBI (n=20)a
Pearson correlation coefficient analysis revealed no significant relationship between the error of the SAM and gait speed, BBS, or FM-MALE (P>.05). There was a moderate relationship between the error of the Fitbit Ultra and gait speed and BBS. There was a moderate relationship between the error of the YDWP and gait speed and BBS. There also was a moderate relationship between the error of the Nike+ Fuelband and gait speed (Tab. 5).
Pearson Correlation Coefficients (r) for Relationship Between Activity Monitors' Accuracy and Clinical Variablesa
Discussion
The Fitbit Ultra activity monitor may be a less costly alternative to research-based activity monitors for identifying steps taken in people with stroke and TBI. Our results indicate that the Fitbit Ultra has fair to good accuracy in identifying stepping activity in people with stroke and TBI who are relatively high functioning, particularly in people who are classified as least limited-community and full-community ambulators.40 The accuracy of the Fitbit Ultra was close to that of the SAM, and better than that of the YDWP and Nike+ Fuelband. In this study, the YDWP and the Nike+ Fuelband were not accurate in identifying steps. Over the course of the 2MWT, the Fitbit Ultra, on average, underestimated by 9.7 steps (5.0% of the actual steps taken); the pedometer, on average, underestimated by 28.8 steps (14.7% of the actual steps taken); and the Nike+ Fuelband, on average, underestimated by 66.2 steps (33.9% of the actual steps taken).
Although it is difficult to directly compare the findings of this study with those of other studies due to differences in participants and study design, the reliability of the Fitbit Ultra in identifying stepping activity appears comparable to that of other accelerometer-based activity monitors that are used primarily for research purposes. In a group of 36 hospitalized older adults (of whom 14 had a diagnosis of stroke), Taraldsen and colleagues21 found that the activPAL activity monitor had an absolute error between 40.3% and 29.1% when identifying steps. In a study by Ryan and colleagues,43 the activPAL had an absolute error of less than 1% when identifying steps in individuals without disability. Macko and colleagues17 found that the SAM was 98.7% accurate in identifying stride count frequency in people with chronic stroke.
Walking speed and balance appear to affect the accuracy of the Fitbit Ultra activity monitor. The distribution of the Bland-Altman plots indicates that the majority of participants in whom the Fitbit Ultra was not accurate (≥18 steps difference between Fitbit Ultra and actual steps taken determined from video count) ambulated at speeds of <0.58 m/s and had BBS scores of <40. The Fitbit Ultra was accurate in the participants with TBI, all of whom walked at speeds of >0.70 m/s and had BBS scores of >40. Participants who walked at relatively faster speeds and who had less balance impairment may have moved in a walking pattern, as detected by the Fitbit Ultra, that is more similar to that of individuals who were healthy on whom the algorithms for detecting steps from the accelerometer signal were developed.
Our findings that a pedometer is not accurate and undercounts steps are similar to the findings of other studies. In people who were healthy and walking at slow speeds, Le Masurier et al44 found that a Yamax pedometer undercounted steps by approximately 30%, and by 12.8% at normal walking speed. Both Carroll and colleagues45 and Ellsworth and colleagues41 found that a pedometer undercounts steps in people with stroke and other neurological disorders by 9% to 17%.
People with stroke and TBI are less active and often do not participate in regular exercise.46,47 Pedometer-based walking programs are effective in reducing weight, increasing activity, and improving health.48 This type of program has not been implemented in people with chronic neurological health conditions. The Fitbit Ultra activity monitor may be used by people with stroke and TBI to increase their walking activity, as it provides an accurate way to self-monitor, set goals, and receive behavior-enhancing feedback. Other advantages of the Fitbit Ultra are that it is commercially available, easy to use, and not cost prohibitive.
The study results should be viewed in light of its limitations. The ability of the devices to estimate steps was tested over 2 minutes of continuous indoor walking. Although it is common practice to measure the accuracy and reliability of activity monitoring devices during short bouts of indoor walking,17,23,24,45 it is nevertheless important that their accuracy be further tested in open and ecologically valid environments.
The participants in the study were relatively high functioning, with an average gait speed of 0.93 m/s and an average BBS score of 48.4. These findings may have contributed to the accuracy of the Fitbit Ultra device because our data also showed that accuracy was best in those with gait speeds of ≥0.58 m/s, and there was a moderate relationship between Fitbit Ultra accuracy and gait speed. Future studies should examine the accuracy and reliability of the Fitbit Ultra in people whose walking ability is more severely affected by their stroke or TBI.
It also should be emphasized that this study examined the ability of the devices to detect steps only. Future studies will need to examine whether the devices are accurate for measuring other activities, such as stair climbing. Our analyses of the subgroups also should be viewed with caution due to the smaller sample sizes.
Conclusion
The Fitbit Ultra activity monitor could be used as a low-cost alternative by physical therapists working in the clinic to measure stepping activity in level, predictable environments of people with stroke and TBI who can walk at speeds of ≥0.58 m/s. The Fitbit Ultra potentially could be used by consumers with stroke and TBI as part of a wellness program to self-monitor walking activity. Compared with the SAM, the Fitbit Ultra has the advantage of being less expensive and easier to use and provides real-time feedback. Unlike the SAM, however, walking speed and balance negatively affected the ability of the Fitbit Ultra to detect steps.
Footnotes
Dr Fulk, Dr Combs, Dr Danks, Dr Nirider, and Dr Reisman provided concept/idea/research design. Dr Fulk, Dr Combs, Dr Danks, and Dr Reisman provided writing. Dr Fulk, Dr Combs, Dr Danks, and Dr Nirider provided data collection. Dr Fulk, Dr Combs, Dr Raja, and Dr Reisman provided data analysis. Dr Fulk, Dr Danks, and Dr Nirider provided project management. Dr Fulk, Dr Combs, and Dr Nirider provided study participants. Dr Fulk, Dr Combs, Dr Nirider, and Dr Reisman provided facilities/equipment. Dr Combs, Dr Danks, and Dr Nirider provided consultation (including review of manuscript before submission).
Preliminary data from this study were presented in poster format at the Combined Sections Meeting of the American Physical Therapy Association; January 21–24, 2013; San Diego, California, and the Texas Physical Therapy Association Annual Conference; October 25–28, 2012; San Antonio, Texas.
- Received December 18, 2012.
- Accepted September 11, 2013.
- © 2014 American Physical Therapy Association
References
Issue highlights
