Skip to main content
  • Other Publications
  • Subscribe
  • Contact Us
Advertisement
JCORE Reference
this is the JCORE Reference site slogan
  • Home
  • Most Read
  • About Us
    • About Us
    • Editorial Board
  • More
    • Advertising
    • Alerts
    • Feedback
    • Folders
    • Help
  • Patients
  • Reference Site Links
    • View Regions
  • Archive

Author Response

Susan Armijo-Olivo, Greta G. Cummings, Jorge Fuentes, Humam Saltaji, Bruno R. da Costa, Christine Ha, Annabritt Chisholm, Dion Pasichnyk, Todd Rogers
DOI: 10.2522/ptj.2014.94.12.1826.2 Published 1 December 2014
Susan Armijo-Olivo
S. Armijo-Olivo, BScPT, MScPT, PhD, CLEAR (Connecting Leadership and Research) Outcomes Research Program, and Faculty of Rehabilitation Medicine, 3-48 Corbett Hall, Department of Physical Therapy, University of Alberta, Edmonton, Alberta, Canada T6G 2G4.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Greta G. Cummings
G.G. Cummings, RN, PhD, FCAHS, CLEAR Outcomes Research Program, Edmonton Clinic Health Academy (ECHA), Faculty of Nursing, University of Alberta.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jorge Fuentes
J. Fuentes, BPT, MSc, PhD, Faculty of Rehabilitation Medicine, University of Alberta, and Department of Physical Therapy, Catholic University of Maule, Talca, Chile.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Humam Saltaji
H. Saltaji, DDS, MSc(Ortho), Orthodontic Graduate Program, School of Dentistry, University of Alberta.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Bruno R. da Costa
B.R. da Costa, PT, BScPT, MSc, PhD, Department of Physical Therapy, Florida International University, Miami, Florida.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Christine Ha
C. Ha, BSc, Rehabilitation Research Center, Faculty of Rehabilitation Medicine, University of Alberta.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Annabritt Chisholm
A. Chisholm, BSc, Alberta Research Centre for Health Evidence, Department of Pediatrics, Faculty of Medicine and Dentistry, University of Alberta.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Dion Pasichnyk
D. Pasichnyk, MSc(Public Health), Institute of Health Economics, Edmonton, Alberta, Canada.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Todd Rogers
T. Rogers, PhD, Centre for Research in Applied Measurement and Evaluation (CRAME), University of Alberta.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Info & Metrics
  • PDF
Loading

In response to Maher and colleagues' letter,1 we would like to point out that in our report,2 we made a general statement regarding psychometric properties of the quality assessment tools used in physical therapy, and we did not specifically state that the Physiotherapy Evidence Database (PEDro) scale has not been appropriately tested for validity or reliability. This statement was based on our previous review, which investigated the psychometric properties of tools to evaluate the quality of randomized controlled trials in health research and especially in physical therapy.3 The above-mentioned review highlighted that most of the existing scales to determine trial quality were not adequately developed, including the PEDro scale. According to the developers of the PEDro scale, the scale was developed based on the Delphi list.4 Based on our results,3 the Delphi list lacked internal consistency and construct validity. These psychometric properties are of importance in any scale because they indicate that the construct—in this case, “methodological quality” (or “risk of bias,” the more current term for this concept)—is fully represented by the items of the scale (internal consistency) and that the scores of a scale are based on hypothetical grounds and should behave based on predefined hypotheses. In other words, the PEDro scale was developed based on another tool that is not completely validated. In addition, the PEDro scale added 2 new items (items 8 and 10), which, to our knowledge, were not obtained from a thorough validation process. Therefore, the set of items included in the PEDro scale might not be entirely valid to represent the construct of “risk of bias.”

In addition, in a recent study,5 we recognized that some items used by quality assessment tools were related more to reporting quality and not to trial conduct or potential bias (eg, in PEDro scale, reporting confidence intervals or other measures of variability). Moreover, we noticed that there was a lack of agreement regarding item relevance to trial quality or risk of bias. Thus, these results called for an in-depth analysis of the items used to determine trial quality and risk of bias of clinical trials to provide evidence of validity for these items. This also applies to the PEDro scale. Although it has been claimed that this is a valid tool to determine study quality using the overall score, the validation studies mentioned failed to demonstrate that the items used by this tool are supported by empirical evidence and are related to bias in clinical trials. This step is of crucial importance to determine the validity of the tool.

Furthermore, previous and recent evidence has questioned the use of summary scores to determine risk of bias of RCTs.6–10 The effects of relevant criteria, such as concealment of allocation, may be diluted or confounded by the summary quality score because they include other items not related to the risk of bias or not important in a given context. Indeed, items that are important in some situations may not be relevant in others, yet they receive the same weight in a quality scale.6,7 The Cochrane Bias Methods Group and Cochrane Statistical Methods Group, therefore, recommend that summary scores obtained from quality scales should not be used.11 Rather, the relevant biases should be assessed one by one, including: selection bias, performance bias, detection bias, attrition bias, reporting bias, and other context-specific biases.9 Thus, the use of total score of the PEDro scale has been criticized.

A case study was presented8 as an example to highlight the fact that biases are introduced into systematic reviews and meta-analyses when summary quality scores are used as an eligibility criterion or somehow introduced into the analysis8 as proposed by others.6,7,10 A Cochrane review on knee osteoarthritis using a physical therapist intervention was revisited for this exercise.8 The PEDro scale summary scores were compared with the Cochrane domain approach where only trials appropriately concealed and blinded were used for analysis. The results of this study indicated that although “low risk of bias” trials according to the PEDro score showed highly clinically and statistically significant results, trials with proper blinding and concealment of allocation based on the Cochrane domain analysis indicated a treatment effect estimate at least 3 times lower, with confidence intervals including the null effect. This example showed that, depending on which scale is used to assess quality and which score of that scale is used as a threshold to define “low risk of bias,” point estimates of treatment effects may be completely different. This finding, in turn, may have serious implications on conclusions drawn by investigators who incorporate summary quality scores into their reviews.

Finally, we acknowledge the efforts performed by the developers of the PEDro scale to validate this scale; however, we feel that the roots of this scale and some of its items, as well as most of the methodological quality assessment tools, are not completely valid to assess the risk of bias of physical therapy trials. Also, the use of the summary score to determine “risk of bias” or “trial quality” might not be appropriate based on the existing evidence.

Footnotes

  • This letter was posted as a Rapid Response on September 22, 2014. at ptjournal.apta.org

  • © 2014 American Physical Therapy Association

References

  1. ↵
    1. Maher CG,
    2. Elkins MR,
    3. Herbert RD,
    4. et al
    . Letter to the editor on “Identifying items to assess methodological quality in physical therapy trials: a factor analysis.” Phys Ther. 2014;94:1826.
    OpenUrlFREE Full Text
  2. ↵
    1. Armijo-Olivo S,
    2. Cummings GG,
    3. Fuentes CJ,
    4. et al
    . Identifying items to assess methodological quality in physical therapy trials: a factor analysis. Phys Ther. 2014;94:1272–1284.
    OpenUrlAbstract/FREE Full Text
  3. ↵
    1. Armijo-Olivo S,
    2. Macedo LG,
    3. Gadotti IC,
    4. et al
    . Scales to assess the quality of randomized controlled trials: a systematic review. Phys Ther. 2008;88:156–175.
    OpenUrlAbstract/FREE Full Text
  4. ↵
    1. Verhagen AP,
    2. de Vet HC,
    3. de Bie RA,
    4. et al
    . The Delphi list: a criteria list for quality assessment of randomized clinical trials for conducting systematic reviews developed by Delphi consensus. J Clin Epidemiol. 1998;51:1235–1241.
    OpenUrlCrossRefPubMedWeb of Science
  5. ↵
    1. Armijo-Olivo S,
    2. Fuentes J,
    3. Ospina M,
    4. et al
    . Inconsistency in the items included in tools used in general health research and physical therapy to evaluate the methodological quality of randomized controlled trials: a descriptive analysis. BMC Med Res Methodol. 2013;13:116.
    OpenUrlCrossRefPubMed
  6. ↵
    1. Greenland S
    . Quality scores are useless and potentially misleading: Reply to “Re: A critical look at some popular analytic methods.” Am J Epidemiol. 1994;140:300–301.
    OpenUrlFREE Full Text
  7. ↵
    1. Jüni P,
    2. Witschi A,
    3. Bloch R,
    4. Egger M
    . The hazards of scoring the quality of clinical trials for meta-analysis. JAMA. 1999;282:1054–1060.
    OpenUrlCrossRefPubMedWeb of Science
  8. ↵
    1. da Costa BR,
    2. Hilfiker R,
    3. Egger M
    . PEDro's bias: summary quality scores should not be used in meta-analysis. J Clin Epidemiol. 2013;66:75–77.
    OpenUrlCrossRefPubMed
  9. ↵
    1. Higgins JPT,
    2. Altman DG,
    3. Gøetzsche PC,
    4. et al
    ; for the Cochrane Bias Methods Group and Cochrane Statistical Methods Group. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928.
    OpenUrlFREE Full Text
  10. ↵
    1. Herbison P,
    2. Hay-Smith J,
    3. Gillespie WJ
    . Adjustment of meta-analyses on the basis of quality scores should be abandoned. J Clin Epidemiol. 2006;59:1249–1256.
    OpenUrlPubMedWeb of Science
  11. ↵
    1. Higgins J,
    2. Green S
    1. Higgins J,
    2. Altman D
    . Assessing risk of bias in included studies. In: Higgins J, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions, Version 5.0. Chichester, United Kingdom: John Wiley & Sons Ltd; 2008.
View Abstract
PreviousNext
Back to top
Vol 94 Issue 12 Table of Contents
Physical Therapy: 94 (12)

Issue highlights

  • Exercise Therapy for Patients With Patellofemoral Pain
  • Resistive Inspiratory Muscle Training
  • Moderate- Versus High-Intensity Exercise Training in People With Type 2 Diabetes
  • Pelvic-Floor Muscle Rehabilitation
  • Developmental Outcomes and Mastery Motivation in Toddlers
  • Impaired Reactive Stepping in Patients After Stroke
  • Exercise Participation in Australian Women With Breast Cancer
  • Influence of Fear-Avoidance Beliefs on Disability in Subacromial Shoulder Pain
  • Patient Self-Referral for Physical Therapy in the Netherlands
  • Aerobic Training Within Stroke Rehabilitation
  • Spanish Language Training for Doctor of Physical Therapy Students
  • Chronic Stress, Cortisol, and Pain
Email

Thank you for your interest in spreading the word on JCORE Reference.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Author Response
(Your Name) has sent you a message from JCORE Reference
(Your Name) thought you would like to see the JCORE Reference web site.
Print
Author Response
Susan Armijo-Olivo, Greta G. Cummings, Jorge Fuentes, Humam Saltaji, Bruno R. da Costa, Christine Ha, Annabritt Chisholm, Dion Pasichnyk, Todd Rogers
Physical Therapy Dec 2014, 94 (12) 1826-1828; DOI: 10.2522/ptj.2014.94.12.1826.2

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Save to my folders

Share
Author Response
Susan Armijo-Olivo, Greta G. Cummings, Jorge Fuentes, Humam Saltaji, Bruno R. da Costa, Christine Ha, Annabritt Chisholm, Dion Pasichnyk, Todd Rogers
Physical Therapy Dec 2014, 94 (12) 1826-1828; DOI: 10.2522/ptj.2014.94.12.1826.2
del.icio.us logo Digg logo Reddit logo Technorati logo Twitter logo CiteULike logo Connotea logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One
  • Article
    • Footnotes
    • References
  • Info & Metrics
  • PDF

Related Articles

Cited By...

More in this TOC Section

  • On “Benka Wallén M, Sorjonen K, Löfgren N, Franzén E. Structural validity of the Mini-Balance Evaluation Systems Test (Mini-BESTest) in people with mild to moderate Parkinson disease.” Phys Ther. 2016;96:1799–1806.
  • Author Response
  • Author Response
Show more Letters and Responses

Subjects

Footer Menu 1

  • menu 1 item 1
  • menu 1 item 2
  • menu 1 item 3
  • menu 1 item 4

Footer Menu 2

  • menu 2 item 1
  • menu 2 item 2
  • menu 2 item 3
  • menu 2 item 4

Footer Menu 3

  • menu 3 item 1
  • menu 3 item 2
  • menu 3 item 3
  • menu 3 item 4

Footer Menu 4

  • menu 4 item 1
  • menu 4 item 2
  • menu 4 item 3
  • menu 4 item 4
footer second
footer first
Copyright © 2013 The HighWire JCore Reference Site | Print ISSN: 0123-4567 | Online ISSN: 1123-4567
advertisement bottom
Advertisement Top