Invited Commentary

A decade ago, the Institute of Medicine (IOM) published the first of 3 reports detailing the significant problems with the quality of health care in the United States. The first report, titled “The Urgent Need to Improve Health Care Quality,”¹ discussed 3 major problems: overuse, underuse, and misuse of health care. This report was followed, in 2000, by To Err is Human: Building a Safer Health System,² which documented serious, widespread errors in our health care delivery system that resulted in frequent and avoidable patient injuries. In 2001, the IOM published Crossing the Quality Chasm: A New Health System for the 21st Century,³ which concluded that the current health care system needed major restructuring to achieve the goal of consistent, high-quality care to all Americans.

Although many in the health services and health policy arenas responded to the IOM reports^4,5 and some modest improvements have been made in the quality of our nation's health care,⁶ little progress has been made in understanding variation in and factors associated with the quality of physical therapy. Resnik and colleagues’ study⁷ is one of few that have begun to tackle the issue of quality of physical therapy. Their study focused on 2 quality measures: effectiveness, as measured by change in health status, and efficiency, which we believe was best captured by the overall performance measure, a composite of their effectiveness and utilization measures.

Resnik and colleagues’ overriding objective was to determine whether staffing and organizational characteristics of physical therapy clinics were associated with clinic effectiveness, utilization, and overall performance in the treatment of patients with low back pain syndromes. Building on work in other areas of health care, they hypothesized that 5 main clinic-level characteristics would be positively associated with better outcomes: (1) higher physical therapist versus physical therapist assistant utilization, (2) therapist experience, (3) size of the physical therapy staff, (4) volume of patients, and (5) being a hospital-based physical therapist practice. Several studies in nursing^8–11 suggest higher ratios of registered nurses to nonlicensed personnel, higher nurse education levels, and increased registered nurse staffing are associated with better quality of patient care. In the surgical literature, numerous studies have indicated that high surgeon volume, surgeon specialization, and high hospital surgery volume are associated with better quality of care.¹²

Unlike the nursing literature, Resnik and colleagues found that clinics with a higher ratio of physical therapists to physical therapist assistants did not deliver higher-quality care. Clinics classified as low physical therapist assistant utilizers, however, did. We were curious about the discrepancy in these findings because we might expect the 2 measures (ie, physical therapist/physical therapist assistant ratio and physical therapist assistant utilization) to be correlated, as Resnik and colleagues demonstrated in earlier work.¹³ What is being captured by their physical therapist assistant utilization variable? It could be could argued that physical therapist assistant utilization is a more valid measure than staffing ratio because physical therapists at a given clinic may not necessarily utilize assistants in a consistent manner. More information on the relationship between the staffing and utilization variables and on the distribution of physical therapist assistant use both within and across clinics would be helpful in sorting this out. How did staffing ratios at the high- and low-utilization clinics compare? What percentage of therapists within a clinic used physical therapist assistants? For clinics classified as high physical therapist assistant utilizers, what were the mean and range for the percentage of patients who had higher physical therapist assistant utilization?

We also wondered about the characteristics of patients who received a majority of their treatments from physical therapist assistants. Were these patients more or less impaired relative to patients who did not receive most of their treatment from physical therapist assistants? Considering the findings, we might question whether patients with high physical therapist assistant utilization possessed certain clinical characteristics that decreased their likelihood of improvement relative to those with low physical therapist assistant utilization.

Contrary to the nursing and surgical literature, physical therapist years of experience, number of full-time–equivalent physical therapists, clinic volume of patients with low back pain, and volume of new patients per physical therapist per month were not associated with quality. The surgical literature suggests that high individual surgeon volume and specialization, more so than high hospital surgery volume, are associated with better outcomes.¹² Perhaps a more therapist-specific and diagnosis-specific measure of volume (eg, number of new patients with low back pain per physical therapist per month) would be associated with quality. Likewise, a more specific measure of therapist experience, such as orthopedic certification (ie, Orthopaedic Certified Specialist, American Academy of Orthopaedic Manual Physical Therapists certification, or Manual Therapy Certified) or years with certification, might be associated with quality. In earlier work with FOTO data from 1999–2000, Resnik and Hart¹⁴ found no clear relationship between advanced certification and patient outcomes but noted that few physical therapists in the database had advanced certification. Because the data for this study are from 2000–2001, representation of certified specialists is likely still low.

Resnik and colleagues’ measures of effectiveness and utilization also may not have been sensitive enough to pick up some of the associations with their independent variables. As they noted, the mean improvement in high-effectiveness clinics was a 19.2-point increase in health status, and the mean improvement in low- effectiveness clinics was a 16.4-point increase. The difference in the mean number of visits between low- versus high-utilization clinics also was small (7.7 and 9.3 visits, respectively). Perhaps different outcome measures would demonstrate greater variation. Two commonly used low back pain–specific functional disability measures, the Roland-Morris Disability Questionnaire and the Oswestry Disability Index, may be more sensitive to change than a more global measure such as the FOTO overall health status measure (OHS). Cost of the episode of care also may be a more specific measure of utilization because physical therapist visits could potentially range in time from approximately 15 to 60 minutes.

Resnik and colleagues’ finding of hospital-based clinics having lower utilization is interesting considering such facilities are exempt from the Medicare therapy caps. The therapy caps, however, were only in effect from 1999 to 2000,¹⁵ prior to their data period. As the authors note, the for-profit concerns of non–hospital-based physical therapy clinics may have driven the increased utilization. Most of the clinics in their sample, however, were hospital-based, and some could have potentially been for-profit. Perhaps some other underlying characteristic of hospital-based physical therapy clinics is related to lower utilization. Although hospital-based physical therapy clinics had lower utilization, the overall cost of care in these settings may have been higher. Based on some of our work with the Medical Expenditure Panel Survey data, we found that in 2004 the average cost of a hospital-based physical therapy visit for low back pain was $179, whereas the average cost of an office-based physical therapy visit was $95.¹⁶

The strength of Resnik and colleagues’ findings rests on the validity of their method of classifying clinics into high, middle, or low effectiveness and high, middle, or low utilization. They did this by first using a multilevel modeling approach to risk adjust outcomes of care at the patient level by controlling for several patient-level and 2 clinic-level characteristics that would likely have an impact on outcomes. They then calculated residual scores based on the results of these models. The residual scores then were aggregated to the clinic level and used to classify clinics. Their multilevel modeling approach where patients are nested within therapists and therapists are nested within clinics is appropriate. Their approach of creating residual scores based on these models also seems reasonable. As the authors noted, the residual scores represent unexplained variance in outcomes (ie, OHS and visits) that could potentially be explained by factors other than patient characteristics. These residual scores also may be explained by patient characteristics that were unobserved (ie, not measured). As with any observational study, the inability to account for differences in unobserved patient characteristics is always a threat to internal validity.

Resnik and colleagues’ study will no doubt serve as a point of departure for other studies examining the quality of physical therapy. Future research on the association between physical therapist assistant utilization and quality of care, in particular, should consider how patient characteristics might differ for those seen by physical therapists and physical therapist assistants versus physical therapists only. Characteristics of the physical therapist/physical therapist assistant dyad (eg, degree of supervision, collaboration styles) also need elaboration. Exploring quality of care in other patient populations and other physical therapy settings also is essential. Finally, in addition to effectiveness and efficiency, we need to explore other aspects of health care quality, including whether care is safe, patient-centered, timely, and equitable.¹⁷

We were extremely pleased to see such a study conducted and published in PTJ. The authors used a novel approach to identify organizational and service delivery characteristics associated with variation in the quality and utilization of physical therapy for the treatment of people with low back pain syndromes. As health care costs continue to rise, Americans are becoming increasingly concerned about access to affordable, high-quality health care. Research on factors associated with the quality of physical therapy is necessary to begin to understand ways in which we can improve the care we deliver.

• American Physical Therapy Association