Author Response

We would like to thank Fitzgerald for his thoughtful and generous commentary¹ on our work.² We believe the recent dialogue on this topic in the physical therapy community is very valuable and a sign of the maturity of the profession. We have only a few points that we would like to add. We agree with Fitzgerald on the key point that a randomized controlled trial (RCT) with appropriate analysis “must be performed to qualify that a CPR [clinical prediction rule] has indeed been developed.” In selecting the potential predictors for investigation in the RCT, we agree that a well-conducted single-arm study may be useful, along with selecting predictors based on a strong theoretical rationale or clinical experience. One note of caution relates to the suggestion by Fitzgerald that using a single-arm trial rather than an underpowered RCT is less likely to miss important predictors. We would argue that single-arm trials may be even more likely to miss true predictors of response to the intervention. As single-arm studies are designed to identify predictors of good prognosis regardless of treatment, a factor associated with a poor prognosis but a good response to treatment will likely not be identified in a single-arm trial. Therefore, single-arm trials can easily miss important predictors of response to treatment. We believe secondary analysis of the thousands of existing RCTs may provide as good, if not better, information than single-arm trials to help in selecting the best predictors for analysis in a prospective, adequately powered RCT. Secondary analyses of existing RCTs also are cheaper and easier to do than performing single-arm trials, which are still expensive to perform.

The language regarding stages of development of CPRs is well defined in the literature.³ This language, however, was developed for studies of diagnosis and prognosis. In the physical therapy literature investigating responders to treatment, this language has been modified, resulting in much confusion. Over recent years in the physical therapy literature, a single-arm study has been considered a derivation study, with the following RCT considered a validation study. As mentioned above, we agree strongly with Fitzgerald's emphasis that an RCT must be performed “to qualify that a CPR has indeed been developed [our emphasis].” Therefore, we do not consider that the first RCT is, in any way, a “validation” study, as referred to in the literature on development of CPRs. The single-arm study is simply a method to generate hypotheses of potential predictors in the same way as a theoretical rationale or clinical experience may be used. Because we do not know what the potential predictors arising from a single-arm study actually are (nonspecific predictors of outcome, treatment effect modifier, or error?), a CPR for a specific intervention cannot exist until we have confirmed that these potential predictors are, in fact, treatment effect modifiers, which is accomplished by performing an RCT and analyzing the predictor variable × treatment group interaction. Therefore, we argue that the initial RCT following a single-arm study confirms only that a CPR exists; it does not validate a CPR, and it should be considered a derivation study according to the original literature on CPR development. A second RCT would be the true validation study. This point has been recommended in recent articles.^4,5

We have previously responded to comments about the study by Hancock et al⁶ mentioned by Fitzgerald and will only briefly comment here. If, as mentioned above, we consider the study by Childs et al⁷ a derivation study (first RCT on this CPR⁸), then Hancock and colleagues' study⁶ is the first attempt at validation. We were interested in whether the rule would generalize to a different setting including a modified treatment. This is not an unreasonable question, as evidenced by the fact that the same researchers also recently published an RCT to investigate the CPR in patients receiving different treatments (alternative manipulation and mobilization).⁹ To refer to the study by Hancock et al⁶ as flawed is, in our opinion, a misunderstanding of the aim and interpretation of the study. The study demonstrated the CPR does not generalize to this new setting including a modified treatment but says nothing about whether it will generalize in a new sample of patients if the identical manipulation is used. We await a second appropriately designed RCT to answer that question.

Last, we would like to extend Fitzgerald's call to researchers to test CPRs using RCTs. Beyond derivation and validation of CPRs, it also is important to perform an impact analysis.³ It is essential to know whether the rule changes clinician behavior, whether it improves patient outcomes, and whether it reduces costs. An accurate CPR may not change clinician behavior or improve patient outcomes for many reasons. It may be too difficult and time-consuming to apply to patients, clinicians may not be adequately trained in the treatment specified by the CPR (or may not feel comfortable performing the treatment), or for practical reasons (eg, liability) the CPR may not be used.

• © 2010 American Physical Therapy Association