A Comparison Study of KOOS-PS and KOOS Function and Sport Scores

Discussion

Our study examined 2 questions: (1) Does the KOOS-PS perform equivalently to the sum of KOOS Function and Sport items? and (2) Do KOOS-PS_Rasch scores perform better than KOOS-PS_raw scores? Within the context of our sample, the hypotheses probing these questions were not supported. The KOOS FunSport_sum scores displayed significantly greater correlations with both reference standards' score, and there was no difference in KOOS_Rasch and KOOS_raw correlations with the performance reference standard.

One goal of the KOOS-PS was to include items with a greater range of difficulty levels than those of the KOOS Function. We attempted to examine the potential benefits of this goal in 2 ways. First, we investigated floor and ceiling effects by counting the number of responses at the extremes of the scale range. Although the KOOS-PS did not display a strong ceiling effect, it appeared susceptible to a floor effect. Second, we chose a patient-reported outcome measures reference standard (ie, LEFS) that includes a spectrum of activity difficulty levels ranging from rolling over in bed to running over uneven ground and making sharp turns while running fast.³ However, because our sampling frame focused on patients seeking care rather than those recovering from the effects of treatment, it is possible that our sample did not provide a good test of the KOOS-PS for patients with higher functioning.

To gain an insight into how the KOOS-PS performs with patients with higher functioning, we performed a post hoc correlation analysis of patients with LEFS scores ≥50. The result of this analysis, reported in Table 3, is similar in format to that presented in Table 2. Because this analysis represents post hoc inquiry, we avoided formal between-measure comparisons. That the point estimates of coefficients presented in Table 3 are smaller than those reported in Table 2 is to be expected owing to the restricted range in function levels of patients with LEFS scores ≥50. When correlated with the performance reference standard, the differences among the various KOOS and KOOS-PS correlation coefficients were unremarkable. In contrast, KOOS Sport scores appeared to correlate much higher with LEFS scores than did KOOS-PS scores. This finding is not entirely unexpected given KOOS Sports items assess higher-level functional activities.

When examining construct validity of the KOOS-PS, some investigations have applied the Spearman rank order correlation coefficient.¹² Because this coefficient is based on rank ordering, the benefit of an interval-scale measure may be lost: Spearman correlation of raw scores will equal Spearman correlation of Rasch scores. To allow a comparison of raw and Rasch scores, we applied the Pearson correlation coefficient. The coefficients reported in Tables 2 and 3 were nearly identical for raw and Rasch scores. This finding suggests that the added complexity of converting raw scores to Rasch-equivalent scores using the reported third-degree polynomial may not be necessary.

The KOOS-PS was intended to provide an efficient measure of physical function from early to late disease for people with OA.⁷ We interpret the phrase “early to late disease” to represent mild to severe functional status limitations. Although the KOOS-PS is one-third the length of the KOOS FunSport_sum items, we found its ability to assess lower extremity functional status was inferior to that of the KOOS FunSport_sum. One explanation for the performance of the KOOS-PS may relate to its content validity: none of the activities require patients to take one step. If the ability to move around is considered to be an important component of lower extremity functional status, the KOOS-PS performance may well be due to its absence of items that capture this dimension. An explanation for this finding likely rests on the method used in the item reduction process. Although Rasch analysis is an excellent method for identifying unidimensional items, it does not assess the extent to which the items capture the characteristic of interest. A second explanation may be that regardless of the item-specific content, a fixed 7-item measure cannot capture the breadth of activities necessary to adequately evaluate lower extremity functional status.

It is expected that measurement properties of a well-developed measure will exceed those of a shortened version of the measure. An important question when judging the value of a shortened measure is, to what extent can a reduction in measurement properties be tolerated without compromising the validity of inferences drawn from a measured value? In part, the answer is dependent on the context in which the measure will be applied. Measures intended for group decisions can withstand greater reductions in reliability than measures proposed for individual patient use. The KOOS-PS, along with pain and structure measures, were conceived to shape a dichotomous decision that would label a patient as a candidate for total joint replacement.⁷ Applied in this context, measurement properties consistent with individual patient application would seem relevant. Another consideration when judging reported KOOS-PS measurement properties is that they apply to continuous rather than binary scores. Collapsing KOOS-PS scores to form a dichotomous decision (ie, surgical candidate: yes/no) will decrease the magnitude of its reported reliability and validity coefficients. When considering the adequacy of a shortened measure, it may be of greater value to focus on the confidence in inferences drawn from the measure's scores rather than on often-cited minimal acceptable reliability values.³⁵

We have raised 2 concerns that we believe extend beyond the KOOS-PS: one addresses content validity, and the other acknowledges the inevitable reduction in validity of a shortened measure. If the goals are to maximize validity and minimize patient response burden, perhaps a better direction would be to develop a computerized adaptive test (CAT) of lower extremity functional status. A CAT contains an item bank capturing the range of ability/difficulty levels for the characteristic of interest. Items are administered sequentially, usually starting with an item of median difficulty (ie, the most informative item). Based on a patient's response to the initial item, a second item is selected approximating the difficulty level conveyed by the patient's response to the first item. This process is repeated until criteria for the stopping rule are met. For example, Hart et al³⁶ developed a CAT based on 18 LEFS items. The first item administered to a patient was “Walking 2 blocks.” Subsequent items were selected based on responses to previous items. Item administration terminated when either the standard error for the provisional level of functional status was less than 4 out of 100 points or the change in the provisional level of functional status estimates for the last 3 items was less than 1 in 100. The CAT reported by Hart et al³⁶ required the administration of an average of 7 items to obtain a score consistent with individual patient application. Development of a CAT would require a clear definition of lower extremity functional status—something that we believe is missing—and the identification of the anticipated activity spectrum to be assessed. We also believe that such discussions would benefit by considering whether the concept of lower extremity functional status is independent of the condition being assessed. For example, is the concept of lower extremity functional status the same for people with OA of the hip or knee, anterior cruciate ligament reconstruction, and neurological problems?

Our study was not without limitations. One limitation was that our sample did not contain many patients with severe or mild lower extremity functional status limitations. Owing to the sigmoidal relationship between raw and Rasch scores, the full benefit of a Rasch-scored measure may not be fully apparent. One direction for further inquiry would be to compare the KOOS-PS with existing measures on samples that contain a more uniform distribution of functional status limitations across the scale range.

A second potential limitation was the application of the sum of the KOOS Function and Sport subscales as a comparison. We did this comparison to gain an impression of what the maximum possible correlation to the reference standards would be. With the exception of the post hoc comparisons, the differences in correlations between the summed score and KOOS_Rasch were near identical to the differences between the KOOS Function and the KOOS_Rasch. A third potential limitation was that patients contributing performance data tended to be younger than those not providing performance data. We cannot anticipate the extent to which this limitation may have affected the relative magnitudes of the measures' validity coefficients. Another potential limitation was the arbitrary choice of a significant difference in correlation coefficients of .07. We chose this value based on the belief than .05 was too stringent and .10 was too large. Inspection of the CI on the differences may assist readers who would prefer to apply another significant difference value.

This study explored the construct validity of the KOOS-PS in the context of patients seeking care for knee OA. The construct validity of the KOOS-PS was significantly less than that of KOOS FunSport_sum. It may be that a fixed 7-item measure is too restricted in range to provide a precise and valid estimate of a patient's lower extremity functional status level. Rather than attempting to generate a shorter measure of lower extremity functional status, it may be more productive to pursue the development of a CAT.