Abstract
Background The “gold standard” for measuring gross motor function in children with cerebral palsy is the 66-item Gross Motor Function Measure (GMFM-66).
Objective The purpose of this study was to estimate the validity and reliability of 2 abbreviated versions of the GMFM-66; one version involves an item set approach, and the other version involves a basal and ceiling approach.
Design This was a measurement study comprising concurrent validity, comparability, and test-retest reliability components.
Methods The study participants were 26 children who were 2 to 6 years of age and had cerebral palsy across all Gross Motor Function Classification System levels. In the first session, both abbreviated versions were administered by 2 independent raters; next, the full GMFM-66 was administered. In the second session, only the abbreviated versions were administered by the same raters. Concurrent validity, comparability of versions, and test-retest reliability were determined with intraclass correlation coefficients [ICC (2,1)].
Results Both versions demonstrated high levels of validity, with an ICC of .99 (95% confidence interval=0.972–0.997), reflecting associations with the GMFM-66. Both versions also were shown to be highly reliable, with ICCs of greater than .98 (95% confidence interval=0.965–0.994).
Limitations A smaller-than-expected sample was recruited for this study and may be a potential limitation of the study.
Conclusion Both versions of the GMFM-66 can be used in clinical practice or research. However, the GMFM-66 with the basal and ceiling approach is recommended as the preferred abbreviated version.
The current “gold standard” for obtaining an estimate of gross motor function in children with cerebral palsy is the Gross Motor Function Measure (GMFM).1 The GMFM is an evaluative instrument designed to measure change over time or change in response to an intervention. The GMFM initially comprised 85 items; the measure later consisted of 88 items to enable the evaluation of some items bilaterally and was subsequently referred to as the GMFM-88.1 Russell et al1 demonstrated intraclass correlation coefficients (ICCs) of .99 for the intrarater and interrater reliability of the GMFM-88 and .99 for the test-retest reliability of this measure. Drouin et al2 supported the construct validity of the GMFM by reporting significant linear relationships between gait speed and dimensions D (standing) (r=.91) and E (walking, running, and jumping) (r=.93) of the GMFM-88. Furthermore, Damiano and Abel3 found a strong correlation between computerized gait analysis parameters and GMFM-88 scores. These findings independently confirmed the construct validity of the GMFM-88.
After extensive use of the GMFM-88 in both clinical practice and research, efforts were made to improve the scaling of the measure.4,5 Rasch analysis is a statistical technique that can be used to create interval-level scores from ordinal-level measures.4 Rasch analysis is based on a probabilistic model that uses maximum-likelihood estimation to order items along a difficulty continuum. Rasch analysis was applied to the GMFM-88 to create the 66-item GMFM (GMFM-66) and to allow the hierarchical structure of the items to be revealed.4 A computer program (the Gross Motor Ability Estimator [GMAE]) was created for clinicians and researchers to compute GMFM-66 scores.4 The GMFM-66 was shown to be highly reliable (ICCs=.97–.99)5,6 and sensitive to change.5,6 One research study subsequently supported the greater sensitivity to change of the GMFM-66 compared with the GMFM-88, as determined through receiver operating characteristic curve analysis.7
Avery et al4 demonstrated that as few as 13 items would be needed to provide an accurate estimation of a child's gross motor abilities, and although clinicians and researchers were eager to have a shorter version available, no guidelines for choosing appropriate subsets of items existed in the public domain at the onset of this project. As a result, 2 independent shortened versions of the GMFM-66 were created concurrently. The rationale for both shortened versions was the same: shorter tests allow for the elimination of items not considered clinically relevant to a given child; therefore, only items relevant to the child's current ability are tested. The hierarchical ordering of items obtained from the Rasch analysis to create the GMFM-66 informed the creation of both abbreviated versions.
One of the shortened versions consists of item sets (GMFM-66-IS) and was created through the use of an algorithm. This version is administered by assessing the child with 3 GMFM-66 “decision” items; performance on those items ultimately dictates which 1 of 4 item sets should be administered and scored.8 The creation and validation of the item sets and the application of the algorithm are described elsewhere.8 The GMFM-66-IS was applied to an existing data set to estimate the validity of this shortened version. An ICC was used to confirm that agreement between scores on the GMFM-66-IS and scores on the GMFM-66 was high (ICC=.994) at a single time point.8
Before the completion of the algorithm approach, the second author (D.J.B.) required an abbreviated instrument for a nationally funded project using methods of comprehensive rehabilitation outcomes research9,10; as a result, a second shortened version was created. This alternate shortened version uses a basal and ceiling approach (GMFM-66-B&C), in which the user tests only items relevant to a given child's ability. Accordingly, a new data collection sheet was developed; items were placed in order of difficulty, as established by the Rasch analysis,4 and age and the Gross Motor Function Classification System (GMFCS) were used as guidelines for when to commence administering items (Appendix 1). The guidelines for age and GMFCS level11 were extrapolated from gross motor function curves12 based on the average GMFM-66 score, as related to the item map by difficulty.1
The basal and ceiling approach is commonly used in developmental testing (eg, Peabody Developmental Motor Scales13). A basal score of 3 consecutive “3s” (“completes”) must be obtained at the start of the test. A ceiling score is reached when the child scores 3 consecutive “0s” (“does not initiate”). For the test to be complete, at least 15 items must be tested between (or outside of) the basal and ceiling scores. In this approach, the items are ordered by difficulty, thereby removing them from the dimensional approach previously used in the GMFM-88 and GMFM-66. Pilot testing with existing data (n=50) has demonstrated strong agreement between scores on the GMFM-66-B&C and scores on the full GMFM-66 (ICC=.99, 95% confidence interval=0.98–0.99) (Doreen J. Bartlett, unpublished data). A limitation of the previously reported results for both abbreviated versions is that all of the analyses were conducted retrospectively on existing data sets. Whether the abbreviated versions perform well in a real practice setting has not yet been determined.
The purposes of this study were to estimate the concurrent validity of the 2 abbreviated versions with the criterion standard (the GMFM-66) and to estimate both their comparability and their test-retest reliability.
Method
This was a measurement study of the validity, comparability, and reliability of the 2 shortened versions of the GMFM-66.
Participants
The children who participated in this study were recruited from 4 sites located in southwestern Ontario, Canada. Children who were between the ages of 2 and 6 years and who had a primary diagnosis of cerebral palsy were included in this study. The lower limit of the age range was chosen on the basis of the previously established reliability of the GMFCS,11 which was greater after 2 years of age than before this age cutoff. The upper limit of the age range was chosen because motor development plateaus in children with cerebral palsy after 7 or 8 years of age.11 Recent data also suggested that a decline in motor abilities may be exhibited as early as 6.9 years of age.14 An attempt was made to achieve representation across all levels of the GMFCS to allow findings to be generalized to the population of children with cerebral palsy as a whole. Additional information gathered about the children included age, sex, and distribution of involvement; the type of motor disorder was not included in this study because of issues about lack of reliability.15 This convenience sample comprised 26 participants. The ideal target sample size, as determined from the expected ICCs, was 40 participants16; however, challenges in recruitment limited the sample size. Table 1 shows the characteristics of the participants.
Measures
Three measures were used in this study: the full GMFM-661 (score sheet available at the CanChild Web site [www.canchild.ca]), the GMFM-66-IS8 (score sheet available as an online appendix8), and the GMFM-66-B&C (score sheet available in Appendix 1). All measures were scored by the first author (L.K.B.) using GMAE software.
Each child was tested on 2 occasions. The first session contained assessments with both shortened versions and then administration of the full GMFM-66 (administered by L.K.B.) to establish concurrent validity. One of the shortened versions was randomly allocated to a therapist assessor, and the other was allocated to the first author. The first author always began the assessment; thus, the shortened versions were applied in a random order between participants. The full GMFM-66 was administered last because it was the longest assessment. Before administration of the full GMFM-66, the children received a rest break, if needed. Additionally, this protocol required a minimal amount of time from the participating physical therapists and still ensured random ordering of the 2 abbreviated versions.
The second session was conducted 2 weeks after the first session (a time frame during which no change was expected) and included assessments with both abbreviated versions (administered by the same raters and in the same order as in the first session) to establish comparability and test-retest reliability.
All therapist assessors received a training booklet and participated in a 1-hour teleconference in which they were introduced to the study protocol. Before commencement of the study, the investigator and each physical therapist passed a criterion test in scoring selected videotaped items of the GMFM-66. Each rater obtained greater than or equal to 80% item agreement on this test before data collection. The 9 therapists had 2 to 36 years of experience in pediatric physical therapist practice (X̄=15 years), and all of them were familiar with the GMFM-66, using it at least occasionally in their practice before participating in this study. None were familiar with either abbreviated version. All 9 therapists referred clients from their case loads. As a result, the number of children tested by the therapists varied; 1 therapist assessed only 1 child, and the other therapists assessed between 2 and 5 children each.
Concurrent validity, comparability, and test-retest reliability were examined with ICC (2,1). Time to completion was recorded for each shortened version. In addition, the physical therapists were asked to indicate their preference for either of the abbreviated versions after their final assessment; this preference was measured on a 5-point Likert scale, with the ends representing a strong preference for the GMFM-66-IS (a score of 1) or the GMFM-66-B&C (a score of 5) and the middle representing a neutral opinion. Seven of the 9 therapists had exposure to both versions (1 therapist tested only 1 child; for another therapist, who tested 2 children, the same version was randomly allocated in both sessions), and only their preferences were recorded.
Role of the Funding Source
This research was supported through the Canadian Institutes of Health Research (MOP-81107).
Results
Score means and standard deviations, number of items administered, and time to completion in the first and second sessions are shown in Table 2. On average, the GMFM-66-B&C involved testing 15 and 16 fewer items than the GMFM-66-IS in the first and second sessions, respectively. This result was expected because the item sets have a predetermined number of items (ranging from 15 to 39 items). Time to completion was not significantly different in the 2 abbreviated versions at either time point (first or second session). A 2-way analysis of variance revealed no significant effect of time (F=1.00, df=1, P=.32) or version (F=2.99, df=1, P=.09) and no significant time × version interaction (F=0.26, df=1, P=.61).
Gross Motor Function Measure (GMFM) Scores, Number of Items Tested, and Time to Completiona
Table 3 shows the concurrent validity and test-retest reliability of both shortened versions as well as the comparability of the 2 abbreviated versions at the 2 time points. In addition, the standard error of measurement and the minimal detectable change (at the 95% level) are shown.
Psychometric Properties of Abbreviated Versions of the Gross Motor Function Measure (GMFM)a
Figures 1 and 2 visually demonstrate the concurrent validity of the GMFM-66 and both abbreviated versions, including confidence and prediction bands. Six of the 7 therapists who had exposure to both measures preferred the GMFM-66-B&C over the GMFM-66-IS, with a median value on the Likert scale of 5.
Concurrent validity of the 66-item Gross Motor Function Measure with the item set approach (GMFM-66-IS) and the 66-item Gross Motor Function Measure (GMFM-66).
Concurrent validity of the 66-item Gross Motor Function Measure with the basal and ceiling approach (GMFM-66-B&C) and the 66-item Gross Motor Function Measure (GMFM-66).
Discussion
The results of the present study indicate that both abbreviated versions of the GMFM-66 had high levels of validity and reliability and can be used in clinical practice or research endeavors. According to Portney and Watkins,17 ICCs greater than .75 indicate good reliability. The present study revealed excellent validity and reliability of both shortened versions of the GMFM-66, with ICCs greater than .98 for concurrent validity and all indexes of reliability.
All but 1 of the physical therapists in the present study strongly preferred the GMFM-66-B&C, with most anecdotally citing the second decision item of the GMFM-66-IS as a problem. It appeared that item 67 (stand, 2 arms held: walks forward 10 steps) was problematic as a decision item because most children at GMFCS levels I to IV, regardless of age, completed this item. This finding was partially due to the manner in which the therapists facilitated walking in these children in the context of the present study. They were observed to facilitate either from in front of the child or from behind the child, despite the guidelines for administration, which clearly state that the assessor should be in front of the child “to reduce the inclination to facilitate walking.”1(p108) From a clinical perspective, even though training emphasizes adherence to administration and scoring guidelines, therapists may continue to facilitate children in this manner; as a result, some children are evaluated with item set 3, although they probably should be assessed with item set 2. The choice of another item at approximately the same difficulty level may prevent this misclassification.
Additionally, the misclassification of children in the present study could explain why no significant difference in time to completion was seen in the 2 abbreviated versions. Many therapists indicated frustration with using an item set because there were few items that the children could complete successfully (score of 3), few items that the children could complete partially (score of 1 or 2), and many items that the children could not perform or even attain the starting position for (score of 0). The results for time to completion could have been affected because, even though 39 items were “tested,” many of these items were not attempted as a result of the children being unable to attain the starting position (eg, children at GMFCS level IV could not maintain their body weight in a standing position; therefore, items 53–58 were not attempted and were given a score of 0).
Therapists who preferred the GMFM-66-B&C also indicated that this version usually contained items that were more clinically relevant to the children that they were assessing than those of the GMFM-66-IS as a result of the misclassification. Many other developmental scales use the basal and ceiling approach, and clinicians are familiar with and skilled in using this approach. The 1 therapist who preferred the GMFM-66-IS cited its ease of administration because items are still ordered by number, as in the full GMFM-66, and items in similar starting positions remain grouped together.
The first author also preferred the GMFM-66-B&C. The various item sets in the GMFM-66-IS often contained items representing milestones that a child had already attained and subsequently surpassed to reach a more advanced but related milestone, and it became difficult to score the easier item because of a lack of cooperation or interest from the child. For example, item set 3 contained items 65 and 66, related to cruising. When a child can walk independently, performance of these items is difficult to elicit because of a lack of interest.
The results of the present study indicated that the GMFM-66-B&C is the preferred shortened version of the GMFM-66. This version requires, on average, 20 to 25 minutes to complete; in contrast, the GMFM-66 is estimated to require 45 to 60 minutes.1 The GMFM-66-B&C can therefore reduce the burden of assessment time for both children and therapists but still provide a valid and reliable estimate of gross motor function in both clinical and research settings.
The smaller-than-expected sample size is a limitation of the present study; however, the high estimates of concurrent validity and reliability, along with the tight 95% confidence intervals, suggest that the sample size was adequate. Additionally, there was an uneven sex distribution in the present study. However, there is no evidence that sex influences GMFM-66 scores; therefore, this limitation was not significant.
A relative disadvantage of the GMFM-66-B&C is that the process of entering scores into the GMAE software (conducted by the first author in the present study) is more cumbersome because of the nature of the score sheet (ie, scores are ordered by difficulty and not in the order that the GMAE software presents for scores to be entered). Although there are columns indicating which dimension the items are from (potentially assisting with the process of entering scores if one is using the GMAE), the process is slower than the process of entering the original GMFM-66 scores. A syntax that we developed is available for use by researchers to simplify scoring. This syntax converts the items so that they can be scored using the GMAE and converted for subsequent use in SPSS* (Appendix 2; this syntax also is available at the CanChild Web site [www.canchild.ca]). Entering scores for the GMFM-66-IS also should be studied because the item sets comprise items from various dimensions and, although they are arranged in numerical order, not all items are present in the item sets.
In the present study, both versions were shown to have high levels of validity and reliability for use in clinical or research settings. However, the physical therapists in the present study preferred the GMFM-66-B&C, perceiving it as more clinically relevant to their clients.
Appendix 1.
Alternate Shortened Version of the Gross Motor Function Measure (GMFM) Score Sheet1 Using a Basal and Ceiling Approach (GMFM-66-B&C)a
a GMFCS=Gross Motor Function Classification System, PR=prone, SUP=supine, STD=standing, KN=kneeling, R=right, L=left. The GMFM-66-B&C Score Sheet is adapted and used with permission from: Russell DJ, Rosenbaum PL, Avery LM, Lane M. Gross Motor Function Measure (GMFM-66 and GMFM-88) User's Manual. London, United Kingdom: Mac Keith Press; 2002. The GMFM-66-B&C Score Sheet may not be reproduced without written permission of the authors.
Appendix 2.
Instructions for Preparing an SPSS Data File of 66-Item Gross Motor Function Measure (GMFM-66) Data for Scoring With the Gross Motor Ability Estimator (GMAE) Program (All File Names Are Given as Examples and to Illustrate the Steps)
Footnotes
-
Dr Bartlett provided concept/idea/research design, project management, fund procurement, facilities/equipment, institutional liaisons, and consultation (including review of manuscript before submission). Both authors provided writing and data analysis. Ms Brunton provided data collection. The authors thank Barbara Stoskopf for her efforts related to this project and Dianne Russell for her input at the design stage of this project and for a review of an earlier version of the manuscript.
-
This study was approved by the ethics boards at The University of Western Ontario, McMaster University, and the Thames Valley Children's Centre before data collection.
-
The results of this study were presented orally at the 63rd annual conference of the American Academy of Cerebral Palsy and Developmental Medicine; September 23–26,2009; Scottsdale, Arizona. The same results were presented in a poster format at the Ontario Association of Children's Rehabilitation Services conference; November 8–10, 2009; Toronto, Ontario, Canada.
-
This research was supported through the Canadian Institutes of Health Research (MOP-81107).
-
↵* SPSS Inc, 233 S Wacker Dr, Chicago, IL 60606.
- Received August 23, 2010.
- Accepted November 22, 2010.
- © 2011 American Physical Therapy Association
References
Issue highlights
