Abstract
Background There remains a need for a quickly administered, stroke-specific, bedside measure of active wrist and finger movement for the expanding stroke population. The wrist stability and hand mobility scales of the upper extremity Fugl-Meyer Assessment (w/h UE FM) constitute a valid, reliable measure of paretic UE impairment in patients with active wrist and finger movement.
Objective The aim of this study was to determine performance on the w/h UE FM in a stable cohort of survivors of stroke with only palpable movement in their paretic wrist flexors.
Design A single-center cohort study was conducted.
Method Thirty-two individuals exhibiting stable, moderate upper extremity hemiparesis (15 male, 17 female; mean age=56.6 years, SD=10.1; mean time since stroke=4.6 years, SD=5.8) participated in the study, which was conducted at an outpatient rehabilitation clinic in the midwestern United States. The w/h UE FM and Action Research Arm Test (ARAT) were administered twice. Intraclass correlation coefficients (ICCs), Cronbach alpha, and ordinal alpha were computed to determine reliability, and Spearman rank correlation coefficients and Bland-Altman plots were computed to establish validity.
Results Intraclass correlation coefficients for the w/h UE FM and ARAT were .95 and .99, respectively. The w/h UE FM intrarater reliability and internal consistency were greater than .80, and concurrent validity was greater than .70. This also was the first stroke rehabilitative study to apply ordinal alpha to examine internal consistency values, revealing w/h UE FM levels greater than .85. Concurrent validity findings were corroborated by Bland-Altman plots.
Conclusions It appears that the w/h UE FM is a promising tool to measure distal upper extremity movement in patients with little active paretic wrist and finger movement. This finding widens the segment of patients on whom the w/h UE FM can be effectively used and addresses a gap, as commonly used measures necessitate active distal upper extremity movement.
Stroke remains a leading cause of adult disability. Yet, commonly used clinical and research measures can be uninformative regarding the extent of motor recovery and treatment response. For example, on the National Institutes of Health Stroke Scale,1 a patient with stroke may be considered “recovered” by having a score of 0 yet may retain upper extremity (UE) motor deficits that compromise function.2 Similarly, assessments such as the Functional Independence Measure3 and modified Rankin Scale4 determine the level of assistance required to carry out daily activities but do not indicate the active level of UE movement that patients exhibit. These levels of assistance may reflect adaptive equipment use, available care partner support, or other factors that are not indicative of the overall efficacy of rehabilitative treatments.
Among measures available to measure poststroke motor status, the UE Fugl-Meyer Assessment5 (UE FM) is a well-established stroke motor measure, iteratively determining active movement at each joint of the paretic extremity. Moreover, it is recommended for use in stroke rehabilitative trials6 and, unlike other measures of paretic extremity dysfunction,7–9 requires few household items to administer and little setup. Recently, the wrist stability and hand mobility scales of the UE FM (w/h UE FM) were shown to constitute a valid, reliable measure of paretic UE impairment in patients with a high degree of active, distal UE movement.10 This finding importantly addressed the need for a quickly administered, rigorous bedside measure of active UE movement in this segment of patients. However, w/h UE FM measurement properties remain unknown in individuals exhibiting moderate UE wrist and hand impairment levels. Such information would widen the segment of patients on whom this easily administered, iterative measure could be effectively used and would address the paucity of measures examining active UE movement in this group. The genesis of a measurement approach for the moderately impaired stroke population also was justified by the fact that proximal to distal UE recovery patterns are not always exhibited by people with stroke11 (meaning that people with moderate hemiparesis may sometimes exhibit distal movement in some areas and not others), thus creating a need for quantification of these deficits. This information also would address a gap, as the above-mentioned measures6–8 necessitate active distal UE movement.
As a next step in this line of research, the purpose of this study was to determine the reliability and concurrent validity of the w/h UE FM with an established, stroke-specific measure of distal paretic UE movement (ie, the Action Research Arm Test [ARAT]6) in a well-defined cohort of patients with chronic stroke and with stable UE impairments and only palpable movement in their paretic wrist flexors. As was the case with individuals with less impairment,10 we hypothesized that the w/h UE FM would display high internal consistency and reliability and high concurrent validity with the ARAT (chosen for its focus on distal UE movement and common use in stroke rehabilitative trials, as discussed below).
A secondary study objective was the novel application of ordinal alpha as a secondary measure of reliability in our measures. To our knowledge, this was the first time that this statistic had been applied to rehabilitative measures, despite the fact that the alpha coefficient may be underestimated when fewer than 5 scale points are used, as is common with Likert-based response scales used to measure function and clinical outcomes (eg, FM, ARAT).
Method
Study Design and Participants
Participants had been recruited from outpatient rehabilitation clinics and stroke support groups in the midwestern United States for several outpatient randomized controlled trials approved by the local ethics board. The current study was a secondary analysis of preintervention scores from these trials. To be eligible, patients met the following inclusion criteria: (1) palpable contraction in the wrist flexors of the paretic UE (a manual muscle test score of 1/5 at the wrist flexors), (2) stroke experienced more than 12 months prior to study enrollment, (3) score of greater than 70 on the Modified Mini-Mental Status Examination,12 (4) age greater than 18 years and less than 85 years, (5) had experienced only one stroke, and (6) discharged from all forms of physical rehabilitation. Exclusion criteria were: (1) excessive spasticity in the paretic UE, as defined as a score of ≥2 in the paretic elbow, wrist, or fingers using the Modified Ashworth Spasticity Scale13; (2) excessive pain in the paretic UE, as measured by a score of ≥5 on a 10-point visual analog scale; and (3) participating in any experimental rehabilitation or drug studies.
Instruments
The UE FM items are organized into scales that discern isolated movements at increasingly distal UE regions. Data arise from 33 items that are scored using a 3-point ordinal scale (0=cannot perform, 1=partially performed, and 2=can perform fully), for a total score of 66 points. All 33 items of the UE FM were administered, although this study focused only on the items comprising the w/h UE FM. The w/h UE FM comprises the 12 most distal UE FM items, which collectively make up the UE FM wrist stability and mobility and hand scales. Greater description of items comprising this scale, including a table depicting scoring and administration, is provided elsewhere.10 The w/h UE FM requires no more than 10 minutes to administer.
To discern concurrent validity of the w/h UE FM, we also administered the ARAT,7 a 19-item test of UE functional limitation divided into 4 categories (grasp, grip, pinch, and gross movement), with 16 of the 19 ARAT items primarily focusing on distal regions of the arm (eg, pinching a ball bearing or marble between the thumb and each finger of the affected hand). The ARAT's distal UE focus makes it an ideal comparator to the w/h UE FM, particularly because it has been used in a number of major stroke rehabilitative trials but requires equipment and a rubric to administer. Thus, the w/h UE FM, in some cases, may constitute a more easily administered alternative to the ARAT. Each ARAT item is graded on a 4-point ordinal scale (0=can perform no part of the test, 1=performs test partially, 2=completes test but takes abnormally long time or has great difficulty, and 3=performs test normally), for a total possible score of 57 points. For this test, participants were seated in a comfortable chair with a straight back while the ARAT items that they had to grasp were placed on an adjustable table in front of them. Table height was adjusted according to the needs of each individual. The ARAT has high intrarater (r=.99) and retest (r=.98) reliability and validity,14,15 all in stroke-induced UE hemiparesis.
Each of the above measures was administered twice to each participant by a single rater with more than 8 years of experience with the measures. The rater was blinded in that he had no knowledge of the intent of the current study. The tests were administered twice, approximately 1 week and 2 weeks before the study intervention phase (pretests 1 and 2, respectively). As noted in the study criteria, participants had chronic stroke (ie, past the point of spontaneous neurological recovery) and were neurologically stable as verified by medical records and clinicians.
Data Analysis
Participant demographics and clinical characteristics were summarized by their descriptive statistics, including the count, mean, standard deviation, and range. We assessed reliability of the w/h UE FM and ARAT across the 2 pretreatment measurements by computing the intraclass correlation coefficient (ICC) using a one-way analysis of variance model. Satisfactory reliability was considered to be indicated by an ICC of ≥.80.16 Internal consistency of items comprising the w/h UE FM at each administration was measured by Cronbach alpha, along with the associated bootstrap confidence intervals. A Cronbach alpha of ≥.70 and ≤.95 was considered satisfactory.17 Internal consistency also was assessed by ordinal alpha because Cronbach alpha can sometimes underestimate these estimates, as noted previously.18,19 Concurrent validity was established between the w/h UE FM and ARAT using the Spearman rank correlation coefficient. A value of ≤.70 represented satisfactory association between the measures. Finally, limits of agreement using the Bland-Altman method20 for adjusting trend were calculated and plotted. Analyses were performed using Stata Statistical Software, Release 10 (StataCorp LP, College Station, Texas) and R software (R Foundation for Statistical Computing, Vienna, Austria).21
Role of the Funding Source
This study was funded by the National Center for Complementary and Alternative Medicine.
Results
Data were collected from 32 participants (15 male, 17 female; mean age=56.6 years, SD=10.1, range=38–75), with a total of 64 test administrations collected (2 per participant). Mean time since stroke onset to first pretest was 4.6 years (SD=5.8), and the range of time since stroke was 0.6 to 27.0 years (17 participants with ischemic stroke and 18 participants with hemiparesis affecting their right arms). Average w/h UE FM and ARAT scores were 4.1 (SD=3.8) and 12.5 (SD=12.9), respectively.
The ICCs for the w/h UE FM and ARAT were .95 and .99, respectively (Table). The w/h UE FM intrarater reliability and internal consistency (measured by Cronbach alpha) were greater than .80, and mean concurrent validity was greater than .70 across the testing periods (Table). Ordinal alpha was greater than .85 for both pretest 1 and pretest 2 (Table). In both cases, ordinal alpha was greater than Cronbach alpha.
Psychometric Values Obtained on the w/h UE FM and ARATa
To compare and describe the level of agreement between the 2 measures even further, Bland-Altman plots were created (Figure). These plots regressed the difference between the measures and the average to compute the limits of agreement due to the varying scales of these measures. Three percent and 6% of observations fell outside the limits of agreement in pretest measures 1 and 2, respectively. The mean difference comparing measures for pretest 1 was (−1.07 + 1.19 × average), with limits of agreement estimated to be (±2.46 × [1.37 + 0.12 × average]). Similarly, the mean difference comparing measures for pretest 2 was (−1.52 + 1.13 × average), with limits of agreement estimated to be (±2.46 × [1.53 + 0.15 × average]). Points fell both above and below the mean difference, indicating no consistent bias.
Level of agreement between the Action Research Arm Test (ARAT) and the wrist stability and hand mobility scales of the Upper Extremity Fugl-Meyer Assessment (w/h UE FM) at the 2 administration periods: Bland-Altman plot with regression adjustment for trend. Gray lines indicate the regression-based 95% limits of agreement. The mean difference for pretest 1 was (−1.07 + 1.19 × average), with limits of agreement estimated to be (±2.46 × [1.37 + 0.12 × average]). The mean difference comparing measures for pretest 2 was (−1.52 + 1.13 × average), with limits of agreement estimated to be (±2.46 × [1.53 + 0.15 × average]).
Discussion
The genesis of promising regimens targeting the paretic UE necessitates scales to measure efficacy of such approaches. Some stroke measures1,2 do not capture the motor gains derived from these treatments, and others are validated in people with active distal UE movement7–9—an ability exhibited by only a minority of survivors of stroke.22,23 The current study examined w/h UE FM psychometrics in patients with only palpable wrist flexion, hoping to extend the number of patients with stroke who could be rigorously tested with a bedside measure. This was the first study assessing the w/h UE FM's psychometric characteristics in this segment of the rapidly expanding stroke survivor population.
Results suggest high w/h UE FM intrarater reliability and item consistency in this group. Because there was considerable time and many other test administrations for this and other studies between a particular participant's first and second UE FM administrations, the possibility of memory bias (ie, the rater recalling a particular participant's first rating and allowing it to influence his or her second rating) was low. Additionally, the reliability and concurrent validity values obtained in the current study compare favorably with UE FM and ARAT reliability and validity values in previous work,14,24 although individuals in previous studies exhibited more UE movement than those in the current study. Thus, our study is a “first” in the literature not only for its analyses of the w/h UE FM but also in that it is the first to examine ARAT psychometrics in patients with only palpable wrist flexor activity.
Although psychometric values met the criteria that were set forth a priori, the second pretesting session's concurrent validity values were slightly less than the .70 criterion (P=.67). However, regression-adjusted Bland-Altman plots, used to graphically explore agreement, revealed that few observations (1 and 2 of 32 observations, respectively, between pretest measurements) resided outside the 95% limits of agreement, with these points lying close to the boundary of the limits. Furthermore, points in the Bland-Altman plots were scattered both above and below the mean value, indicating that bias was not overwhelmingly in one direction or the other. These findings confirmed that the w/h UE FM exhibited adequate concurrent validity with the ARAT and further support its use as a “stand-alone” bedside scale to measure distal paretic UE movement.
Cronbach alpha is an established25 measure of internal consistency. It is commonly used to assess reliability during the development of new measures.26 However, recent investigations19,20,27 demonstrated that the alpha coefficient may be underestimated when fewer than 5 scale points are used. This finding is particularly concerning within rehabilitation contexts, where Likert-type response scales are often used to measure function and clinical outcomes (eg, FM, ARAT). Specifically, Cronbach alpha assumes that item responses will be normally distributed and measured using interval or ratio scales. This assumption preents 2 problems. First, item responses in clinical settings may not be normally distributed. For example, the distributions of responses of survivors of severe stroke to an assessment of motor impairment are unlikely to be normally distributed. Rather, such responses are likely to be positively skewed, with the majority of cases indicating significant impairment of motor function. Distributional analyses of the FM data (pretest 1, pretest 2) collected as part of this study confirm this hypothesis. Second, Cronbach alpha is inappropriate because Likert-type responses are never continuous.28 The violation of this assumption is confirmed based on visual inspection of distributional analyses.
Our use of reliability analyses of the w/h UE FM using ordinal alpha are significant for a number of reasons. First, the estimates of ordinal alpha exceed those of Cronbach alpha (Table). These findings confirm the expectation presented in previous work on ordinal alpha.18,19 Second, to our knowledge, this article presents the first application of ordinal alpha within the field of physical rehabilitation. This reason is significant because of the high prevalence of Likert-type measurements used to measure health, function, and clinical outcomes in the rehabilitation professions. Based on our findings, the use of ordinal alpha is highly recommended for therapists, physicians, and other rehabilitation scientists conducting clinical research that uses Likert-type response scales. This approach allows researchers to account for the discrete nature of data generated by these types of instruments. Third, because Cronbach alpha may sometimes underestimate the internal consistency of an instrument (although it is not believed to have done so in the current study), clinicians may inaccurately determine that a measure is unreliable when it actually is reliable. Finally, analyses conducted using ordinal alpha further confirm the high reliability of the w/h UE FM.
When coupled with the administration instructions provided elsewhere,10 it is hoped that the above findings will encourage clinicians to use the w/h UE FM to quantify UE movement in patients exhibiting at least palpable paretic wrist flexor activity. Clinicians can use w/h UE FM scores to rigorously assess current UE movement, to plan treatments, and to gauge efficacy of an intervention by administering the test longitudinally. For example, deficiencies seen in items measuring mass flexion and mass extension may give the clinician reason to target grasping and releasing a coffee cup as part of therapy sessions. Moreover, motor performance constitutes a significant predictor of eventual motor recovery in the months poststroke.29 As next steps, it would be prudent to examine predictive validity and responsiveness of the measure as well as its interrater reliability, as only one rater was used in this study. Examining sensitivity to change, such as when an intervention is provided, also would be a desirable next step.
Like other measures of distal UE movement,8,9 the ARAT requires patients to be seated in a chair and uses a specific setup for patient and object positioning. Although important for research purposes, these conditions may not always be plausible in clinical situations. These shortfalls also make the w/h UE FM well situated as a quick screen or outcome measure in research situations. Yet, it should be noted that the w/h UE FM is a measure of UE impairment, whereas other measures (eg, ARAT, Arm Motor Ability Test) examine UE functional limitation. Although scores on these measures are highly associated, they do not measure entirely equivalent constructs (ie, motor impairment with the w/h UE FM versus functional limitation using the latter measures). Care should be taken by researchers to consider the construct in which they expect and hope to measure changes when using these measures. It should be noted that a short form of the UE FM, consisting of 12 items, was recently validated.30 However, inclusion of items on this particular version were predicated on the premise of proximal to distal motor return, which has been shown to not always be valid.11 Moreover, one must still administer the more proximal items on this and other revised UE FM versions31 that have been forwarded. The current study remains valuable because it examines the administration of the w/h UE FM as a stand-alone measure, without the more proximally focused items. Clinicians will want to choose the best UE FM version depending on the impairment level of the patient, the time, and the tools that they have available.
Footnotes
Dr Page and Dr Persch provided concept/idea/research design. All authors provided writing. Dr Page provided data collection, project management, fund procurement, participants, facilities/equipment, and institutional liaisons. Dr Page and Dr Hade provided data analysis. Dr Persch provided consultation (including review of manuscript before submission).
This study was funded by the National Center for Complementary and Alternative Medicine.
- Received June 13, 2013.
- Accepted August 23, 2014.
- © 2015 American Physical Therapy Association