User login
Take-Home Points
- Systematic use of PROMs allows physicians to review data on pain, physical function, and psychological status to aid in clinical decision-making and best practices.
- PROMs should include both general outcome measures (VAS, SF-36, or EQ-5D) and reliable, valid, and responsive disease specific measures.
- PROM questionnaires should collect pertinent information while limiting the length to maximize patient compliance and reliability.
- PROMIS has been developed to standardize questionnaires, but generality for specific orthopedic procedures may result in less effective measures.
- PROMs can also be used for predictive modeling, which has the potential to help develop more cost-effective care and predict expected outcomes and recovery trajectories for individual patients.
Owing to their unique ability to recognize patients as stakeholders in their own healthcare, patient-reported outcome measures (PROMs) are becoming increasingly popular in the assessment of medical and surgical outcomes.1 PROMs are an outcome measures subset in which patients complete questionnaires about their perceptions of their overall health status and specific health limitations. By systematically using PROMs before and after a clearly defined episode of care, clinicians can collect data on perceived pain level, physical function, and psychological status and use the data to validate use of surgical procedures and shape clinical decisions about best practices.2-4 Although mortality and morbidity rates and other traditional measures are valuable in assessing outcomes, they do not represent or communicate the larger impact of an episode of care. As many orthopedic procedures are elective, and some are low-risk, the evaluation of changes in quality of life and self-reported functional improvement is an important addition to morbidity and mortality rates in capturing the true impact of a surgical procedure and recovery. The patient’s preoperative and postoperative perspectives on his or her health status have become important as well; our healthcare system has been placing more emphasis on patient-centered quality care.2,5
Although PROMs have many benefits, implementation in an orthopedic surgery practice has its challenges. With so many PROMs available, selecting those that fit the patient population for a specialized orthopedic surgery practice can be difficult. In addition, although PROM data are essential for research and for measuring individual or institutional recovery trajectories for surgical procedures, in a busy practice getting patients to provide these data can be difficult.
PROMs are heavily used for outcomes assessment in the orthopedics literature, but there are few resources for orthopedic surgeons who want to implement PROMs in their practices. In this article, we review the literature on the challenges of effectively implementing PROMs in an orthopedic surgery practice.
PROM Selection Considerations
PROMs can be categorized as either generic or disease-specific,4 but together they are used to adequately capture the impact, both broad and local, of an orthopedic condition.
Generic Outcome Measures
Generic outcome measures apply to a range of subspecialties or anatomical regions, allowing for evaluation of a patient’s overall health or quality of life. The most widely accepted measure of pain is the visual analog scale (VAS). The VAS for pain quantifies the level of pain a patient experiences at a given time on a graphic sliding scale from 0 (no pain) to 10 (worst possible pain). The VAS is used in clinical evaluation of pain and in reported outcomes literature.6,7
Many generic PROMs assess mental health status in addition to physical limitations. Poor preoperative mental health status has been recognized as a predictor of worse outcomes across a variety of orthopedic procedures.8,9 Therefore, to assess the overall influence of an orthopedic condition, it is important to include at least 1 generic PROM that assesses mental health status before and after an episode of care. Generic PROMs commonly used in orthopedic surgery include the 36-Item Short Form Health Survey (SF-36), the shorter SF-12, the Veterans RAND 12-Item Health Survey (VR-12), the World Health Organization Disability Assessment Schedule (WHODAS), the European Quality of Life-5 Dimensions (EQ-5D) index, and the 10-item Patient-Reported Outcomes Measurement Information System Global Health (PROMIS-10) scale.10-14
Some generic outcome measures (eg, the EQ-5D index) offer the “utility” calculation, which represents a preference for a patient’s desired health status. Such utilities allow for a measurement of quality of life, represented by quality-adjusted life years (QALY), which is a standardized measure of disease burden. Calculated QALY from measures such as the EQ-5D can be used in cost-effectiveness analyses of surgical interventions and have been used to validate use of procedures, particularly in arthroplasty.15-17
Disease-Specific Outcome Measures
Likewise, there is a range of disease-specific PROMs validated for use in orthopedic surgery, and providers select PROMs that fit their scope of practice. In anatomical regions such as the knee, hip, and shoulder, disease-specific outcome measures vary significantly by subspecialty and patient population. When selecting disease-specific PROMs, providers must consider tools such as reliability, validity, responsiveness, and available population norms. One study used Evaluating Measures of Patient-Reported Outcomes (EMPRO) to assess the quality of a PROM in shoulders and concluded that the American Shoulder and Elbow Surgeons (ASES) index, the Simple Shoulder Test (SST), and the Oxford Shoulder Score (OSS) were all supported for use in practice.18 It is important to note that reliability, validity, and responsiveness of a PROM may vary with the diagnosis or the patient population studied. For example, the SST was found to be responsive in assessing rotator cuff injury but not as useful in assessing shoulder instability or arthritis.19 Variable responsiveness highlights the need for a diagnosis-based level of PROM customization. For example, patients who undergo a surgical intervention for shoulder instability are given a customized survey, which includes PROMs specific to their condition, such as the Western Ontario Shoulder Instability (WOSI) index.20 For patients with knee instability, similar considerations apply; specific measures such as the Lysholm score and the Tenger Activity Scale capture the impact of injury in physically demanding activities.21 When selecting disease-specific PROMs, providers should consult articles like those by Davidson and Keating22 and Bent and colleagues,23 who present provider-friendly tools that can be used to examine the effectiveness of a PROM, and provide additional background information on selecting disease-specific PROMs. For hip and knee arthroplasty subspecialties, the International Society of Arthroplasty Registries (ISAR) created a working group that determines best practices for PROM collection and identifies PROMs most commonly reported in arthroplasty.24
Questionnaire Length Considerations
When PROMs are used in a practice, a balance must be struck between gathering enough information to determine functionality and limiting the patient burden of questionnaire length. A decision to use several PROMs all at once, at a single data collection point, can lengthen the questionnaire significantly. One study found that, with use of longer questionnaires, patients may lose interest, resulting in decreased reliability and compliance.25 For example, providers who use the long (42-item) Knee Injury and Osteoarthritis Outcome Score (KOOS) questionnaire to assess knee function are often limited in what other PROMs they may administer at the same time. Efforts to shorten this questionnaire while still capturing necessary information led to the development of the 7-item KOOS Jr, which was validated for use in knee arthroplasty and had its 7 items drawn from the original 42.26 Similarly, the 40-item Hip Disability and Osteoarthritis Outcome Score (HOOS) questionnaire was shortened to the 6-item HOOS Jr, which was validated for use in hip arthroplasty,27 and the generic SF-36 was shortened to the SF-12.11 Providers trying to build an outcomes database while minimizing patient burden should consider using the shorter versions of these questionnaires but should also consider their validity, as KOOS Jr and HOOS Jr have been validated for use only in knee and hip arthroplasty and not in other knee and hip conditions.
PROM Data Collection Considerations
Comprehensive collection of longitudinal PROM data poses many challenges for providers and patients. For providers, the greatest challenges are infrastructure, technology, and the personnel needed to administer and store paper or electronic surveys. For patients, the most common survey completion barriers are questionnaire length, confusing or irrelevant content, and, in the case of some older adults, inability to complete surveys electronically.25
Identifying a nonresponsive or noncompliant patient population is an important issue in collecting PROM data for research or other purposes. A study of factors associated with higher nonresponse rates in elective surgery patients (N = 135,474) found that noncompliance was higher for males, patients under age 55 years, nonwhites, patients in the lowest socioeconomic quintile, patients living alone, patients needing assistance in completing questionnaires, and patients who previously underwent surgery for their condition.28 In a systematic review of methods that increased the response rates of postal and electronic surveys, Edwards and colleagues29 found significantly higher odds of response for patients who were prenotified of the survey, given shorter questionnaires, or given a deadline for survey completion. Of note, response rates were lower when the word survey was used in the subject line of an email.
PROM distribution has evolved with the rise of technological advances that allow for electronic survey distribution and data capture. Several studies have found that electronically administered PROMs have high response rates.3,30,31 In a study of patients who underwent total hip arthroplasty, Rolfson and colleagues32 found that response rates were significantly higher for those who were surveyed on paper than for those surveyed over the internet. A randomized controlled study found that, compared with paper surveys, digital tablet surveys effectively and reliably collected PROM data; in addition, digital tablets provided instant data storage, and improved survey completion by requiring that all questions be answered before the survey could be submitted.33 However, age, race/ethnicity, and income disparities in technology use must be considered when administering internet-based follow-up surveys and analyzing data collected with web-based methods.34 A study of total joint arthroplasty candidates found that several groups were less likely to complete electronic PROM questionnaires: patients over age 75 years, Hispanic or black patients, patients with Medicare or Medicaid, patients who previously underwent orthopedic surgery, patients undergoing revision total joint arthroplasty, patients with other comorbidities, and patients whose primary language was not English.35 Providers interested in implementing PROMs must consider their patient population when selecting a method for survey distribution and follow-up. A study found that a majority of PROMs were written at a level many patients may not have understood, because of their literacy level or age; this lack of understanding created a barrier to compliance in many patient populations.36
PROM Limitations and PROMIS Use
Use of PROMs has its limitations. The large variety of PROMs available for use in orthopedic surgery has led to several standardization initiatives. The National Institutes of Health funded the development of PROMIS, a person-centered measures database that evaluates and monitors the physical, social, and emotional health of adults and children.37 The goal of PROMIS is to develop a standardized method of selecting PROMs, so that all medical disciplines and subspecialties can choose an applicable set of questions from the PROMIS question bank and use it in practice. Orthopedic surgery can use questions pertaining to physical functioning of the lower and upper extremities as well as quality of life and mental health. PROMIS physical function questions have been validated for use in several areas of orthopedic surgery.38-40 A disadvantage of PROMIS is the overgenerality of its questions, which may not be as effective in capturing the implications of specific diagnoses. For example, it is difficult to use generalized questions to determine the implications of a diagnosis such as shoulder instability, which may affect only higher functioning activities or sports. More research on best PROM selection practices is needed in order to either standardize PROMs or move toward use of a single database such as PROMIS.
Future Directions in PROM Applications
PROMs are being used for research and patient engagement, but there are many other applications on the horizon. As already mentioned, predictive modeling is of particular interest. The existence of vast collaborative PROM databases that capture a diverse patient population introduces the possibility of creating models capable of predicting a patient outcome and enhancing shared decision-making.3 Predicting good or excellent patient outcomes for specific patient populations may allow elimination of certain postoperative visits, thereby creating more cost-effective care and reducing the burden of unnecessary clinic visits for both patients and physicians.
As with other healthcare areas, PROM data collection technology is rapidly advancing. Not only has electronic technology almost entirely replaced paper-and-pencil collection methods, but a new method of outcome data collection has been developed: computerized adaptive testing (CAT). CAT uses item-response theory to minimize the number of questions patients must answer in order for validated and reliable outcome scores to be calculated. According to multiple studies, CAT used across several questionnaires has reliably assessed PROMs while minimizing floor and ceiling effects, eliminating irrelevant questions, and shortening survey completion time.41-43
Besides becoming more patient-friendly and accessible across multiple interfaces (mobile devices and computers), PROMs are also beginning to be integrated into the electronic medical record, allowing easier access to information during chart reviews. Use of statistical and predictive modeling, as described by Chang,3 could give PROMs a role in clinical decision-making. Informing patients of their expected outcome and recovery trajectory—based on demographics, comorbidities, preoperative functional status, and other factors—could influence their decision to undergo surgical intervention. As Halawi and colleagues44 pointed out, it is important to discuss patient expectations before surgery, as unrealistic ones can negatively affect outcomes and lead to dissatisfaction. With clinicians having ready access to statistics and models in patient charts, we may see a transformation in clinical practices and surgical decision-making.
Conclusion
PROMs offer many ways to improve research and clinical care in orthopedic surgery. However, implementing PROMs in practice is not without challenges. Interested orthopedic surgeons should select the PROMs that are most appropriate—reliable, validated, and responsive to their patient population. Electronic distribution of PROM questionnaires is effective and allows data to be stored on entry, but orthopedic surgeons must consider their patient population to ensure accurate data capture and compliance in longitudinal surveys. Proper implementation of PROMs in a practice can allow clinicians to formulate expectations for postoperative recovery and set reasonable postoperative goals while engaging patients in improving quality of care.
1. Howie L, Hirsch B, Locklear T, Abernethy AP. Assessing the value of patient-generated data to comparative effectiveness research. Health Aff (Millwood). 2014;33(7):1220-1228.
2. Haywood KL. Patient-reported outcome I: measuring what matters in musculoskeletal care. Musculoskeletal Care. 2006;4(4):187-203.
3. Chang CH. Patient-reported outcomes measurement and management with innovative methodologies and technologies. Qual Life Res. 2007;16(suppl 1):157-166.
4. Black N. Patient reported outcome measures could help transform healthcare. BMJ. 2013;346:f167.
5. Porter ME. A strategy for health care reform—toward a value-based system. N Engl J Med. 2009;361(2):109-112.
6. Scott J, Huskisson EC. Graphic representation of pain. Pain. 1976;2(2):175-184.
7. de Nies F, Fidler MW. Visual analog scale for the assessment of total hip arthroplasty. J Arthroplasty. 1997;12(4):416-419.
8. Ayers DC, Franklin PD, Ring DC. The role of emotional health in functional outcomes after orthopaedic surgery: extending the biopsychosocial model to orthopaedics: AOA critical issues. J Bone Joint Surg Am. 2013;95(21):e165.
9. Edwards RR, Haythornthwaite JA, Smith MT, Klick B, Katz JN. Catastrophizing and depressive symptoms as prospective predictors of outcomes following total knee replacement. Pain Res Manag. 2009;14(4):307-311.
10. Patel AA, Donegan D, Albert T. The 36-Item Short Form. J Am Acad Orthop Surg. 2007;15(2):126-134.
11. Ware J Jr, Kosinski M, Keller SD. A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34(3):220-233.
12. About the VR-36, VR-12 and VR-6D. Boston University School of Public Health website. http://www.bu.edu/sph/research/research-landing-page/vr-36-vr-12-and-vr-6d/. Accessed October 4, 2017.
13. Jansson KA, Granath F. Health-related quality of life (EQ-5D) before and after orthopedic surgery. Acta Orthop. 2011;82(1):82-89.
14. Oak SR, Strnad GJ, Bena J, et al. Responsiveness comparison of the EQ-5D, PROMIS Global Health, and VR-12 questionnaires in knee arthroscopy. Orthop J Sports Med. 2016;4(12):2325967116674714.
15. Lavernia CJ, Iacobelli DA, Brooks L, Villa JM. The cost-utility of total hip arthroplasty: earlier intervention, improved economics. J Arthroplasty. 2015;30(6):945-949.
16. Mather RC 3rd, Watters TS, Orlando LA, Bolognesi MP, Moorman CT 3rd. Cost effectiveness analysis of hemiarthroplasty and total shoulder arthroplasty. J Shoulder Elbow Surg. 2010;19(3):325-334.
17. Brauer CA, Rosen AB, Olchanski NV, Neumann PJ. Cost-utility analyses in orthopaedic surgery. J Bone Joint Surg Am. 2005;87(6):1253-1259.
18. Schmidt S, Ferrer M, González M, et al; EMPRO Group. Evaluation of shoulder-specific patient-reported outcome measures: a systematic and standardized comparison of available evidence. J Shoulder Elbow Surg. 2014;23(3):434-444.
19. Godfrey J, Hamman R, Lowenstein S, Briggs K, Kocher M. Reliability, validity, and responsiveness of the Simple Shoulder Test: psychometric properties by age and injury type. J Shoulder Elbow Surg. 2007;16(3):260-267.
20. Kirkley A, Griffin S, McLintock H, Ng L. The development and evaluation of a disease-specific quality of life measurement tool for shoulder instability. The Western Ontario Shoulder Instability Index (WOSI). Am J Sports Med. 1998;26(6):764-772.
21. Briggs KK, Lysholm J, Tegner Y, Rodkey WG, Kocher MS, Steadman JR. The reliability, validity, and responsiveness of the Lysholm score and Tegner Activity Scale for anterior cruciate ligament injuries of the knee: 25 years later. Am J Sports Med. 2009;37(5):890-897.
22. Davidson M, Keating J. Patient-reported outcome measures (PROMs): how should I interpret reports of measurement properties? A practical guide for clinicians and researchers who are not biostatisticians. Br J Sports Med. 2014;48(9):792-796.
23. Bent NP, Wright CC, Rushton AB, Batt ME. Selecting outcome measures in sports medicine: a guide for practitioners using the example of anterior cruciate ligament rehabilitation. Br J Sports Med. 2009;43(13):1006-1012.
24. Rolfson O, Eresian Chenok K, Bohm E, et al; Patient-Reported Outcome Measures Working Group of the International Society of Arthroplasty Registries. Patient-reported outcome measures in arthroplasty registries. Acta Orthop. 2016;87(suppl 1):3-8.
25. Franklin PD, Lewallen D, Bozic K, Hallstrom B, Jiranek W, Ayers DC. Implementation of patient-reported outcome measures in U.S. total joint replacement registries: rationale, status, and plans. J Bone Joint Surg Am. 2014;96(suppl 1):104-109.
26. Lyman S, Lee YY, Franklin PD, Li W, Cross MB, Padgett DE. Validation of the KOOS, JR: a short-form knee arthroplasty outcomes survey. Clin Orthop Relat Res. 2016;474(6):1461-1471.
27. Lyman S, Lee YY, Franklin PD, Li W, Mayman DJ, Padgett DE. Validation of the HOOS, JR: a short-form hip replacement survey. Clin Orthop Relat Res. 2016;474(6):1472-1482.
28. Hutchings A, Neuburger J, Grosse Frie K, Black N, van der Meulen J. Factors associated with non-response in routine use of patient reported outcome measures after elective surgery in England. Health Qual Life Outcomes. 2012;10:34.
29. Edwards PJ, Roberts I, Clarke MJ, et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev. 2009;(3):MR000008.
30. Gakhar H, McConnell B, Apostolopoulos AP, Lewis P. A pilot study investigating the use of at-home, web-based questionnaires compiling patient-reported outcome measures following total hip and knee replacement surgeries. J Long Term Eff Med Implants. 2013;23(1):39-43.
31. Bojcic JL, Sue VM, Huon TS, Maletis GB, Inacio MC. Comparison of paper and electronic surveys for measuring patient-reported outcomes after anterior cruciate ligament reconstruction. Perm J. 2014;18(3):22-26.
32. Rolfson O, Salomonsson R, Dahlberg LE, Garellick G. Internet-based follow-up questionnaire for measuring patient-reported outcome after total hip replacement surgery—reliability and response rate. Value Health. 2011;14(2):316-321.
33. Shah KN, Hofmann MR, Schwarzkopf R, et al. Patient-reported outcome measures: how do digital tablets stack up to paper forms? A randomized, controlled study. Am J Orthop. 2016;45(7):E451-E457.
34. Kaiser Family Foundation. The Digital Divide and Access to Health Information Online. http://kff.org/disparities-policy/poll-finding/the-digital-divide-and-access-to-health/. Published April 1, 2011. Accessed October 4, 2017.
35. Schamber EM, Takemoto SK, Chenok KE, Bozic KJ. Barriers to completion of patient reported outcome measures. J Arthroplasty. 2013;28(9):1449-1453.
36. El-Daly I, Ibraheim H, Rajakulendran K, Culpan P, Bates P. Are patient-reported outcome measures in orthopaedics easily read by patients? Clin Orthop Relat Res. 2016;474(1):246-255.
37. Intro to PROMIS. 2016. Health Measures website. http://www.healthmeasures.net/explore-measurement-systems/promis/intro-to-promis. Accessed October 4, 2017.
38. Hung M, Baumhauer JF, Latt LD, Saltzman CL, SooHoo NF, Hunt KJ; National Orthopaedic Foot & Ankle Outcomes Research Network. Validation of PROMIS ® Physical Function computerized adaptive tests for orthopaedic foot and ankle outcome research. Clin Orthop Relat Res. 2013;471(11):3466-3474.
39. Hung M, Clegg DO, Greene T, Saltzman CL. Evaluation of the PROMIS Physical Function item bank in orthopaedic patients. J Orthop Res. 2011;29(6):947-953.
40. Tyser AR, Beckmann J, Franklin JD, et al. Evaluation of the PROMIS Physical Function computer adaptive test in the upper extremity. J Hand Surg Am. 2014;39(10):2047-2051.e4.
41. Hung M, Stuart AR, Higgins TF, Saltzman CL, Kubiak EN. Computerized adaptive testing using the PROMIS Physical Function item bank reduces test burden with less ceiling effects compared with the Short Musculoskeletal Function Assessment in orthopaedic trauma patients. J Orthop Trauma. 2014;28(8):439-443.
42. Hung M, Clegg DO, Greene T, Weir C, Saltzman CL. A lower extremity physical function computerized adaptive testing instrument for orthopaedic patients. Foot Ankle Int. 2012;33(4):326-335.
43. Döring AC, Nota SP, Hageman MG, Ring DC. Measurement of upper extremity disability using the Patient-Reported Outcomes Measurement Information System. J Hand Surg Am. 2014;39(6):1160-1165.
44. Halawi MJ, Greene K, Barsoum WK. Optimizing outcomes of total joint arthroplasty under the comprehensive care for joint replacement model. Am J Orthop. 2016;45(3):E112-E113.
Take-Home Points
- Systematic use of PROMs allows physicians to review data on pain, physical function, and psychological status to aid in clinical decision-making and best practices.
- PROMs should include both general outcome measures (VAS, SF-36, or EQ-5D) and reliable, valid, and responsive disease specific measures.
- PROM questionnaires should collect pertinent information while limiting the length to maximize patient compliance and reliability.
- PROMIS has been developed to standardize questionnaires, but generality for specific orthopedic procedures may result in less effective measures.
- PROMs can also be used for predictive modeling, which has the potential to help develop more cost-effective care and predict expected outcomes and recovery trajectories for individual patients.
Owing to their unique ability to recognize patients as stakeholders in their own healthcare, patient-reported outcome measures (PROMs) are becoming increasingly popular in the assessment of medical and surgical outcomes.1 PROMs are an outcome measures subset in which patients complete questionnaires about their perceptions of their overall health status and specific health limitations. By systematically using PROMs before and after a clearly defined episode of care, clinicians can collect data on perceived pain level, physical function, and psychological status and use the data to validate use of surgical procedures and shape clinical decisions about best practices.2-4 Although mortality and morbidity rates and other traditional measures are valuable in assessing outcomes, they do not represent or communicate the larger impact of an episode of care. As many orthopedic procedures are elective, and some are low-risk, the evaluation of changes in quality of life and self-reported functional improvement is an important addition to morbidity and mortality rates in capturing the true impact of a surgical procedure and recovery. The patient’s preoperative and postoperative perspectives on his or her health status have become important as well; our healthcare system has been placing more emphasis on patient-centered quality care.2,5
Although PROMs have many benefits, implementation in an orthopedic surgery practice has its challenges. With so many PROMs available, selecting those that fit the patient population for a specialized orthopedic surgery practice can be difficult. In addition, although PROM data are essential for research and for measuring individual or institutional recovery trajectories for surgical procedures, in a busy practice getting patients to provide these data can be difficult.
PROMs are heavily used for outcomes assessment in the orthopedics literature, but there are few resources for orthopedic surgeons who want to implement PROMs in their practices. In this article, we review the literature on the challenges of effectively implementing PROMs in an orthopedic surgery practice.
PROM Selection Considerations
PROMs can be categorized as either generic or disease-specific,4 but together they are used to adequately capture the impact, both broad and local, of an orthopedic condition.
Generic Outcome Measures
Generic outcome measures apply to a range of subspecialties or anatomical regions, allowing for evaluation of a patient’s overall health or quality of life. The most widely accepted measure of pain is the visual analog scale (VAS). The VAS for pain quantifies the level of pain a patient experiences at a given time on a graphic sliding scale from 0 (no pain) to 10 (worst possible pain). The VAS is used in clinical evaluation of pain and in reported outcomes literature.6,7
Many generic PROMs assess mental health status in addition to physical limitations. Poor preoperative mental health status has been recognized as a predictor of worse outcomes across a variety of orthopedic procedures.8,9 Therefore, to assess the overall influence of an orthopedic condition, it is important to include at least 1 generic PROM that assesses mental health status before and after an episode of care. Generic PROMs commonly used in orthopedic surgery include the 36-Item Short Form Health Survey (SF-36), the shorter SF-12, the Veterans RAND 12-Item Health Survey (VR-12), the World Health Organization Disability Assessment Schedule (WHODAS), the European Quality of Life-5 Dimensions (EQ-5D) index, and the 10-item Patient-Reported Outcomes Measurement Information System Global Health (PROMIS-10) scale.10-14
Some generic outcome measures (eg, the EQ-5D index) offer the “utility” calculation, which represents a preference for a patient’s desired health status. Such utilities allow for a measurement of quality of life, represented by quality-adjusted life years (QALY), which is a standardized measure of disease burden. Calculated QALY from measures such as the EQ-5D can be used in cost-effectiveness analyses of surgical interventions and have been used to validate use of procedures, particularly in arthroplasty.15-17
Disease-Specific Outcome Measures
Likewise, there is a range of disease-specific PROMs validated for use in orthopedic surgery, and providers select PROMs that fit their scope of practice. In anatomical regions such as the knee, hip, and shoulder, disease-specific outcome measures vary significantly by subspecialty and patient population. When selecting disease-specific PROMs, providers must consider tools such as reliability, validity, responsiveness, and available population norms. One study used Evaluating Measures of Patient-Reported Outcomes (EMPRO) to assess the quality of a PROM in shoulders and concluded that the American Shoulder and Elbow Surgeons (ASES) index, the Simple Shoulder Test (SST), and the Oxford Shoulder Score (OSS) were all supported for use in practice.18 It is important to note that reliability, validity, and responsiveness of a PROM may vary with the diagnosis or the patient population studied. For example, the SST was found to be responsive in assessing rotator cuff injury but not as useful in assessing shoulder instability or arthritis.19 Variable responsiveness highlights the need for a diagnosis-based level of PROM customization. For example, patients who undergo a surgical intervention for shoulder instability are given a customized survey, which includes PROMs specific to their condition, such as the Western Ontario Shoulder Instability (WOSI) index.20 For patients with knee instability, similar considerations apply; specific measures such as the Lysholm score and the Tenger Activity Scale capture the impact of injury in physically demanding activities.21 When selecting disease-specific PROMs, providers should consult articles like those by Davidson and Keating22 and Bent and colleagues,23 who present provider-friendly tools that can be used to examine the effectiveness of a PROM, and provide additional background information on selecting disease-specific PROMs. For hip and knee arthroplasty subspecialties, the International Society of Arthroplasty Registries (ISAR) created a working group that determines best practices for PROM collection and identifies PROMs most commonly reported in arthroplasty.24
Questionnaire Length Considerations
When PROMs are used in a practice, a balance must be struck between gathering enough information to determine functionality and limiting the patient burden of questionnaire length. A decision to use several PROMs all at once, at a single data collection point, can lengthen the questionnaire significantly. One study found that, with use of longer questionnaires, patients may lose interest, resulting in decreased reliability and compliance.25 For example, providers who use the long (42-item) Knee Injury and Osteoarthritis Outcome Score (KOOS) questionnaire to assess knee function are often limited in what other PROMs they may administer at the same time. Efforts to shorten this questionnaire while still capturing necessary information led to the development of the 7-item KOOS Jr, which was validated for use in knee arthroplasty and had its 7 items drawn from the original 42.26 Similarly, the 40-item Hip Disability and Osteoarthritis Outcome Score (HOOS) questionnaire was shortened to the 6-item HOOS Jr, which was validated for use in hip arthroplasty,27 and the generic SF-36 was shortened to the SF-12.11 Providers trying to build an outcomes database while minimizing patient burden should consider using the shorter versions of these questionnaires but should also consider their validity, as KOOS Jr and HOOS Jr have been validated for use only in knee and hip arthroplasty and not in other knee and hip conditions.
PROM Data Collection Considerations
Comprehensive collection of longitudinal PROM data poses many challenges for providers and patients. For providers, the greatest challenges are infrastructure, technology, and the personnel needed to administer and store paper or electronic surveys. For patients, the most common survey completion barriers are questionnaire length, confusing or irrelevant content, and, in the case of some older adults, inability to complete surveys electronically.25
Identifying a nonresponsive or noncompliant patient population is an important issue in collecting PROM data for research or other purposes. A study of factors associated with higher nonresponse rates in elective surgery patients (N = 135,474) found that noncompliance was higher for males, patients under age 55 years, nonwhites, patients in the lowest socioeconomic quintile, patients living alone, patients needing assistance in completing questionnaires, and patients who previously underwent surgery for their condition.28 In a systematic review of methods that increased the response rates of postal and electronic surveys, Edwards and colleagues29 found significantly higher odds of response for patients who were prenotified of the survey, given shorter questionnaires, or given a deadline for survey completion. Of note, response rates were lower when the word survey was used in the subject line of an email.
PROM distribution has evolved with the rise of technological advances that allow for electronic survey distribution and data capture. Several studies have found that electronically administered PROMs have high response rates.3,30,31 In a study of patients who underwent total hip arthroplasty, Rolfson and colleagues32 found that response rates were significantly higher for those who were surveyed on paper than for those surveyed over the internet. A randomized controlled study found that, compared with paper surveys, digital tablet surveys effectively and reliably collected PROM data; in addition, digital tablets provided instant data storage, and improved survey completion by requiring that all questions be answered before the survey could be submitted.33 However, age, race/ethnicity, and income disparities in technology use must be considered when administering internet-based follow-up surveys and analyzing data collected with web-based methods.34 A study of total joint arthroplasty candidates found that several groups were less likely to complete electronic PROM questionnaires: patients over age 75 years, Hispanic or black patients, patients with Medicare or Medicaid, patients who previously underwent orthopedic surgery, patients undergoing revision total joint arthroplasty, patients with other comorbidities, and patients whose primary language was not English.35 Providers interested in implementing PROMs must consider their patient population when selecting a method for survey distribution and follow-up. A study found that a majority of PROMs were written at a level many patients may not have understood, because of their literacy level or age; this lack of understanding created a barrier to compliance in many patient populations.36
PROM Limitations and PROMIS Use
Use of PROMs has its limitations. The large variety of PROMs available for use in orthopedic surgery has led to several standardization initiatives. The National Institutes of Health funded the development of PROMIS, a person-centered measures database that evaluates and monitors the physical, social, and emotional health of adults and children.37 The goal of PROMIS is to develop a standardized method of selecting PROMs, so that all medical disciplines and subspecialties can choose an applicable set of questions from the PROMIS question bank and use it in practice. Orthopedic surgery can use questions pertaining to physical functioning of the lower and upper extremities as well as quality of life and mental health. PROMIS physical function questions have been validated for use in several areas of orthopedic surgery.38-40 A disadvantage of PROMIS is the overgenerality of its questions, which may not be as effective in capturing the implications of specific diagnoses. For example, it is difficult to use generalized questions to determine the implications of a diagnosis such as shoulder instability, which may affect only higher functioning activities or sports. More research on best PROM selection practices is needed in order to either standardize PROMs or move toward use of a single database such as PROMIS.
Future Directions in PROM Applications
PROMs are being used for research and patient engagement, but there are many other applications on the horizon. As already mentioned, predictive modeling is of particular interest. The existence of vast collaborative PROM databases that capture a diverse patient population introduces the possibility of creating models capable of predicting a patient outcome and enhancing shared decision-making.3 Predicting good or excellent patient outcomes for specific patient populations may allow elimination of certain postoperative visits, thereby creating more cost-effective care and reducing the burden of unnecessary clinic visits for both patients and physicians.
As with other healthcare areas, PROM data collection technology is rapidly advancing. Not only has electronic technology almost entirely replaced paper-and-pencil collection methods, but a new method of outcome data collection has been developed: computerized adaptive testing (CAT). CAT uses item-response theory to minimize the number of questions patients must answer in order for validated and reliable outcome scores to be calculated. According to multiple studies, CAT used across several questionnaires has reliably assessed PROMs while minimizing floor and ceiling effects, eliminating irrelevant questions, and shortening survey completion time.41-43
Besides becoming more patient-friendly and accessible across multiple interfaces (mobile devices and computers), PROMs are also beginning to be integrated into the electronic medical record, allowing easier access to information during chart reviews. Use of statistical and predictive modeling, as described by Chang,3 could give PROMs a role in clinical decision-making. Informing patients of their expected outcome and recovery trajectory—based on demographics, comorbidities, preoperative functional status, and other factors—could influence their decision to undergo surgical intervention. As Halawi and colleagues44 pointed out, it is important to discuss patient expectations before surgery, as unrealistic ones can negatively affect outcomes and lead to dissatisfaction. With clinicians having ready access to statistics and models in patient charts, we may see a transformation in clinical practices and surgical decision-making.
Conclusion
PROMs offer many ways to improve research and clinical care in orthopedic surgery. However, implementing PROMs in practice is not without challenges. Interested orthopedic surgeons should select the PROMs that are most appropriate—reliable, validated, and responsive to their patient population. Electronic distribution of PROM questionnaires is effective and allows data to be stored on entry, but orthopedic surgeons must consider their patient population to ensure accurate data capture and compliance in longitudinal surveys. Proper implementation of PROMs in a practice can allow clinicians to formulate expectations for postoperative recovery and set reasonable postoperative goals while engaging patients in improving quality of care.
Take-Home Points
- Systematic use of PROMs allows physicians to review data on pain, physical function, and psychological status to aid in clinical decision-making and best practices.
- PROMs should include both general outcome measures (VAS, SF-36, or EQ-5D) and reliable, valid, and responsive disease specific measures.
- PROM questionnaires should collect pertinent information while limiting the length to maximize patient compliance and reliability.
- PROMIS has been developed to standardize questionnaires, but generality for specific orthopedic procedures may result in less effective measures.
- PROMs can also be used for predictive modeling, which has the potential to help develop more cost-effective care and predict expected outcomes and recovery trajectories for individual patients.
Owing to their unique ability to recognize patients as stakeholders in their own healthcare, patient-reported outcome measures (PROMs) are becoming increasingly popular in the assessment of medical and surgical outcomes.1 PROMs are an outcome measures subset in which patients complete questionnaires about their perceptions of their overall health status and specific health limitations. By systematically using PROMs before and after a clearly defined episode of care, clinicians can collect data on perceived pain level, physical function, and psychological status and use the data to validate use of surgical procedures and shape clinical decisions about best practices.2-4 Although mortality and morbidity rates and other traditional measures are valuable in assessing outcomes, they do not represent or communicate the larger impact of an episode of care. As many orthopedic procedures are elective, and some are low-risk, the evaluation of changes in quality of life and self-reported functional improvement is an important addition to morbidity and mortality rates in capturing the true impact of a surgical procedure and recovery. The patient’s preoperative and postoperative perspectives on his or her health status have become important as well; our healthcare system has been placing more emphasis on patient-centered quality care.2,5
Although PROMs have many benefits, implementation in an orthopedic surgery practice has its challenges. With so many PROMs available, selecting those that fit the patient population for a specialized orthopedic surgery practice can be difficult. In addition, although PROM data are essential for research and for measuring individual or institutional recovery trajectories for surgical procedures, in a busy practice getting patients to provide these data can be difficult.
PROMs are heavily used for outcomes assessment in the orthopedics literature, but there are few resources for orthopedic surgeons who want to implement PROMs in their practices. In this article, we review the literature on the challenges of effectively implementing PROMs in an orthopedic surgery practice.
PROM Selection Considerations
PROMs can be categorized as either generic or disease-specific,4 but together they are used to adequately capture the impact, both broad and local, of an orthopedic condition.
Generic Outcome Measures
Generic outcome measures apply to a range of subspecialties or anatomical regions, allowing for evaluation of a patient’s overall health or quality of life. The most widely accepted measure of pain is the visual analog scale (VAS). The VAS for pain quantifies the level of pain a patient experiences at a given time on a graphic sliding scale from 0 (no pain) to 10 (worst possible pain). The VAS is used in clinical evaluation of pain and in reported outcomes literature.6,7
Many generic PROMs assess mental health status in addition to physical limitations. Poor preoperative mental health status has been recognized as a predictor of worse outcomes across a variety of orthopedic procedures.8,9 Therefore, to assess the overall influence of an orthopedic condition, it is important to include at least 1 generic PROM that assesses mental health status before and after an episode of care. Generic PROMs commonly used in orthopedic surgery include the 36-Item Short Form Health Survey (SF-36), the shorter SF-12, the Veterans RAND 12-Item Health Survey (VR-12), the World Health Organization Disability Assessment Schedule (WHODAS), the European Quality of Life-5 Dimensions (EQ-5D) index, and the 10-item Patient-Reported Outcomes Measurement Information System Global Health (PROMIS-10) scale.10-14
Some generic outcome measures (eg, the EQ-5D index) offer the “utility” calculation, which represents a preference for a patient’s desired health status. Such utilities allow for a measurement of quality of life, represented by quality-adjusted life years (QALY), which is a standardized measure of disease burden. Calculated QALY from measures such as the EQ-5D can be used in cost-effectiveness analyses of surgical interventions and have been used to validate use of procedures, particularly in arthroplasty.15-17
Disease-Specific Outcome Measures
Likewise, there is a range of disease-specific PROMs validated for use in orthopedic surgery, and providers select PROMs that fit their scope of practice. In anatomical regions such as the knee, hip, and shoulder, disease-specific outcome measures vary significantly by subspecialty and patient population. When selecting disease-specific PROMs, providers must consider tools such as reliability, validity, responsiveness, and available population norms. One study used Evaluating Measures of Patient-Reported Outcomes (EMPRO) to assess the quality of a PROM in shoulders and concluded that the American Shoulder and Elbow Surgeons (ASES) index, the Simple Shoulder Test (SST), and the Oxford Shoulder Score (OSS) were all supported for use in practice.18 It is important to note that reliability, validity, and responsiveness of a PROM may vary with the diagnosis or the patient population studied. For example, the SST was found to be responsive in assessing rotator cuff injury but not as useful in assessing shoulder instability or arthritis.19 Variable responsiveness highlights the need for a diagnosis-based level of PROM customization. For example, patients who undergo a surgical intervention for shoulder instability are given a customized survey, which includes PROMs specific to their condition, such as the Western Ontario Shoulder Instability (WOSI) index.20 For patients with knee instability, similar considerations apply; specific measures such as the Lysholm score and the Tenger Activity Scale capture the impact of injury in physically demanding activities.21 When selecting disease-specific PROMs, providers should consult articles like those by Davidson and Keating22 and Bent and colleagues,23 who present provider-friendly tools that can be used to examine the effectiveness of a PROM, and provide additional background information on selecting disease-specific PROMs. For hip and knee arthroplasty subspecialties, the International Society of Arthroplasty Registries (ISAR) created a working group that determines best practices for PROM collection and identifies PROMs most commonly reported in arthroplasty.24
Questionnaire Length Considerations
When PROMs are used in a practice, a balance must be struck between gathering enough information to determine functionality and limiting the patient burden of questionnaire length. A decision to use several PROMs all at once, at a single data collection point, can lengthen the questionnaire significantly. One study found that, with use of longer questionnaires, patients may lose interest, resulting in decreased reliability and compliance.25 For example, providers who use the long (42-item) Knee Injury and Osteoarthritis Outcome Score (KOOS) questionnaire to assess knee function are often limited in what other PROMs they may administer at the same time. Efforts to shorten this questionnaire while still capturing necessary information led to the development of the 7-item KOOS Jr, which was validated for use in knee arthroplasty and had its 7 items drawn from the original 42.26 Similarly, the 40-item Hip Disability and Osteoarthritis Outcome Score (HOOS) questionnaire was shortened to the 6-item HOOS Jr, which was validated for use in hip arthroplasty,27 and the generic SF-36 was shortened to the SF-12.11 Providers trying to build an outcomes database while minimizing patient burden should consider using the shorter versions of these questionnaires but should also consider their validity, as KOOS Jr and HOOS Jr have been validated for use only in knee and hip arthroplasty and not in other knee and hip conditions.
PROM Data Collection Considerations
Comprehensive collection of longitudinal PROM data poses many challenges for providers and patients. For providers, the greatest challenges are infrastructure, technology, and the personnel needed to administer and store paper or electronic surveys. For patients, the most common survey completion barriers are questionnaire length, confusing or irrelevant content, and, in the case of some older adults, inability to complete surveys electronically.25
Identifying a nonresponsive or noncompliant patient population is an important issue in collecting PROM data for research or other purposes. A study of factors associated with higher nonresponse rates in elective surgery patients (N = 135,474) found that noncompliance was higher for males, patients under age 55 years, nonwhites, patients in the lowest socioeconomic quintile, patients living alone, patients needing assistance in completing questionnaires, and patients who previously underwent surgery for their condition.28 In a systematic review of methods that increased the response rates of postal and electronic surveys, Edwards and colleagues29 found significantly higher odds of response for patients who were prenotified of the survey, given shorter questionnaires, or given a deadline for survey completion. Of note, response rates were lower when the word survey was used in the subject line of an email.
PROM distribution has evolved with the rise of technological advances that allow for electronic survey distribution and data capture. Several studies have found that electronically administered PROMs have high response rates.3,30,31 In a study of patients who underwent total hip arthroplasty, Rolfson and colleagues32 found that response rates were significantly higher for those who were surveyed on paper than for those surveyed over the internet. A randomized controlled study found that, compared with paper surveys, digital tablet surveys effectively and reliably collected PROM data; in addition, digital tablets provided instant data storage, and improved survey completion by requiring that all questions be answered before the survey could be submitted.33 However, age, race/ethnicity, and income disparities in technology use must be considered when administering internet-based follow-up surveys and analyzing data collected with web-based methods.34 A study of total joint arthroplasty candidates found that several groups were less likely to complete electronic PROM questionnaires: patients over age 75 years, Hispanic or black patients, patients with Medicare or Medicaid, patients who previously underwent orthopedic surgery, patients undergoing revision total joint arthroplasty, patients with other comorbidities, and patients whose primary language was not English.35 Providers interested in implementing PROMs must consider their patient population when selecting a method for survey distribution and follow-up. A study found that a majority of PROMs were written at a level many patients may not have understood, because of their literacy level or age; this lack of understanding created a barrier to compliance in many patient populations.36
PROM Limitations and PROMIS Use
Use of PROMs has its limitations. The large variety of PROMs available for use in orthopedic surgery has led to several standardization initiatives. The National Institutes of Health funded the development of PROMIS, a person-centered measures database that evaluates and monitors the physical, social, and emotional health of adults and children.37 The goal of PROMIS is to develop a standardized method of selecting PROMs, so that all medical disciplines and subspecialties can choose an applicable set of questions from the PROMIS question bank and use it in practice. Orthopedic surgery can use questions pertaining to physical functioning of the lower and upper extremities as well as quality of life and mental health. PROMIS physical function questions have been validated for use in several areas of orthopedic surgery.38-40 A disadvantage of PROMIS is the overgenerality of its questions, which may not be as effective in capturing the implications of specific diagnoses. For example, it is difficult to use generalized questions to determine the implications of a diagnosis such as shoulder instability, which may affect only higher functioning activities or sports. More research on best PROM selection practices is needed in order to either standardize PROMs or move toward use of a single database such as PROMIS.
Future Directions in PROM Applications
PROMs are being used for research and patient engagement, but there are many other applications on the horizon. As already mentioned, predictive modeling is of particular interest. The existence of vast collaborative PROM databases that capture a diverse patient population introduces the possibility of creating models capable of predicting a patient outcome and enhancing shared decision-making.3 Predicting good or excellent patient outcomes for specific patient populations may allow elimination of certain postoperative visits, thereby creating more cost-effective care and reducing the burden of unnecessary clinic visits for both patients and physicians.
As with other healthcare areas, PROM data collection technology is rapidly advancing. Not only has electronic technology almost entirely replaced paper-and-pencil collection methods, but a new method of outcome data collection has been developed: computerized adaptive testing (CAT). CAT uses item-response theory to minimize the number of questions patients must answer in order for validated and reliable outcome scores to be calculated. According to multiple studies, CAT used across several questionnaires has reliably assessed PROMs while minimizing floor and ceiling effects, eliminating irrelevant questions, and shortening survey completion time.41-43
Besides becoming more patient-friendly and accessible across multiple interfaces (mobile devices and computers), PROMs are also beginning to be integrated into the electronic medical record, allowing easier access to information during chart reviews. Use of statistical and predictive modeling, as described by Chang,3 could give PROMs a role in clinical decision-making. Informing patients of their expected outcome and recovery trajectory—based on demographics, comorbidities, preoperative functional status, and other factors—could influence their decision to undergo surgical intervention. As Halawi and colleagues44 pointed out, it is important to discuss patient expectations before surgery, as unrealistic ones can negatively affect outcomes and lead to dissatisfaction. With clinicians having ready access to statistics and models in patient charts, we may see a transformation in clinical practices and surgical decision-making.
Conclusion
PROMs offer many ways to improve research and clinical care in orthopedic surgery. However, implementing PROMs in practice is not without challenges. Interested orthopedic surgeons should select the PROMs that are most appropriate—reliable, validated, and responsive to their patient population. Electronic distribution of PROM questionnaires is effective and allows data to be stored on entry, but orthopedic surgeons must consider their patient population to ensure accurate data capture and compliance in longitudinal surveys. Proper implementation of PROMs in a practice can allow clinicians to formulate expectations for postoperative recovery and set reasonable postoperative goals while engaging patients in improving quality of care.
1. Howie L, Hirsch B, Locklear T, Abernethy AP. Assessing the value of patient-generated data to comparative effectiveness research. Health Aff (Millwood). 2014;33(7):1220-1228.
2. Haywood KL. Patient-reported outcome I: measuring what matters in musculoskeletal care. Musculoskeletal Care. 2006;4(4):187-203.
3. Chang CH. Patient-reported outcomes measurement and management with innovative methodologies and technologies. Qual Life Res. 2007;16(suppl 1):157-166.
4. Black N. Patient reported outcome measures could help transform healthcare. BMJ. 2013;346:f167.
5. Porter ME. A strategy for health care reform—toward a value-based system. N Engl J Med. 2009;361(2):109-112.
6. Scott J, Huskisson EC. Graphic representation of pain. Pain. 1976;2(2):175-184.
7. de Nies F, Fidler MW. Visual analog scale for the assessment of total hip arthroplasty. J Arthroplasty. 1997;12(4):416-419.
8. Ayers DC, Franklin PD, Ring DC. The role of emotional health in functional outcomes after orthopaedic surgery: extending the biopsychosocial model to orthopaedics: AOA critical issues. J Bone Joint Surg Am. 2013;95(21):e165.
9. Edwards RR, Haythornthwaite JA, Smith MT, Klick B, Katz JN. Catastrophizing and depressive symptoms as prospective predictors of outcomes following total knee replacement. Pain Res Manag. 2009;14(4):307-311.
10. Patel AA, Donegan D, Albert T. The 36-Item Short Form. J Am Acad Orthop Surg. 2007;15(2):126-134.
11. Ware J Jr, Kosinski M, Keller SD. A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34(3):220-233.
12. About the VR-36, VR-12 and VR-6D. Boston University School of Public Health website. http://www.bu.edu/sph/research/research-landing-page/vr-36-vr-12-and-vr-6d/. Accessed October 4, 2017.
13. Jansson KA, Granath F. Health-related quality of life (EQ-5D) before and after orthopedic surgery. Acta Orthop. 2011;82(1):82-89.
14. Oak SR, Strnad GJ, Bena J, et al. Responsiveness comparison of the EQ-5D, PROMIS Global Health, and VR-12 questionnaires in knee arthroscopy. Orthop J Sports Med. 2016;4(12):2325967116674714.
15. Lavernia CJ, Iacobelli DA, Brooks L, Villa JM. The cost-utility of total hip arthroplasty: earlier intervention, improved economics. J Arthroplasty. 2015;30(6):945-949.
16. Mather RC 3rd, Watters TS, Orlando LA, Bolognesi MP, Moorman CT 3rd. Cost effectiveness analysis of hemiarthroplasty and total shoulder arthroplasty. J Shoulder Elbow Surg. 2010;19(3):325-334.
17. Brauer CA, Rosen AB, Olchanski NV, Neumann PJ. Cost-utility analyses in orthopaedic surgery. J Bone Joint Surg Am. 2005;87(6):1253-1259.
18. Schmidt S, Ferrer M, González M, et al; EMPRO Group. Evaluation of shoulder-specific patient-reported outcome measures: a systematic and standardized comparison of available evidence. J Shoulder Elbow Surg. 2014;23(3):434-444.
19. Godfrey J, Hamman R, Lowenstein S, Briggs K, Kocher M. Reliability, validity, and responsiveness of the Simple Shoulder Test: psychometric properties by age and injury type. J Shoulder Elbow Surg. 2007;16(3):260-267.
20. Kirkley A, Griffin S, McLintock H, Ng L. The development and evaluation of a disease-specific quality of life measurement tool for shoulder instability. The Western Ontario Shoulder Instability Index (WOSI). Am J Sports Med. 1998;26(6):764-772.
21. Briggs KK, Lysholm J, Tegner Y, Rodkey WG, Kocher MS, Steadman JR. The reliability, validity, and responsiveness of the Lysholm score and Tegner Activity Scale for anterior cruciate ligament injuries of the knee: 25 years later. Am J Sports Med. 2009;37(5):890-897.
22. Davidson M, Keating J. Patient-reported outcome measures (PROMs): how should I interpret reports of measurement properties? A practical guide for clinicians and researchers who are not biostatisticians. Br J Sports Med. 2014;48(9):792-796.
23. Bent NP, Wright CC, Rushton AB, Batt ME. Selecting outcome measures in sports medicine: a guide for practitioners using the example of anterior cruciate ligament rehabilitation. Br J Sports Med. 2009;43(13):1006-1012.
24. Rolfson O, Eresian Chenok K, Bohm E, et al; Patient-Reported Outcome Measures Working Group of the International Society of Arthroplasty Registries. Patient-reported outcome measures in arthroplasty registries. Acta Orthop. 2016;87(suppl 1):3-8.
25. Franklin PD, Lewallen D, Bozic K, Hallstrom B, Jiranek W, Ayers DC. Implementation of patient-reported outcome measures in U.S. total joint replacement registries: rationale, status, and plans. J Bone Joint Surg Am. 2014;96(suppl 1):104-109.
26. Lyman S, Lee YY, Franklin PD, Li W, Cross MB, Padgett DE. Validation of the KOOS, JR: a short-form knee arthroplasty outcomes survey. Clin Orthop Relat Res. 2016;474(6):1461-1471.
27. Lyman S, Lee YY, Franklin PD, Li W, Mayman DJ, Padgett DE. Validation of the HOOS, JR: a short-form hip replacement survey. Clin Orthop Relat Res. 2016;474(6):1472-1482.
28. Hutchings A, Neuburger J, Grosse Frie K, Black N, van der Meulen J. Factors associated with non-response in routine use of patient reported outcome measures after elective surgery in England. Health Qual Life Outcomes. 2012;10:34.
29. Edwards PJ, Roberts I, Clarke MJ, et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev. 2009;(3):MR000008.
30. Gakhar H, McConnell B, Apostolopoulos AP, Lewis P. A pilot study investigating the use of at-home, web-based questionnaires compiling patient-reported outcome measures following total hip and knee replacement surgeries. J Long Term Eff Med Implants. 2013;23(1):39-43.
31. Bojcic JL, Sue VM, Huon TS, Maletis GB, Inacio MC. Comparison of paper and electronic surveys for measuring patient-reported outcomes after anterior cruciate ligament reconstruction. Perm J. 2014;18(3):22-26.
32. Rolfson O, Salomonsson R, Dahlberg LE, Garellick G. Internet-based follow-up questionnaire for measuring patient-reported outcome after total hip replacement surgery—reliability and response rate. Value Health. 2011;14(2):316-321.
33. Shah KN, Hofmann MR, Schwarzkopf R, et al. Patient-reported outcome measures: how do digital tablets stack up to paper forms? A randomized, controlled study. Am J Orthop. 2016;45(7):E451-E457.
34. Kaiser Family Foundation. The Digital Divide and Access to Health Information Online. http://kff.org/disparities-policy/poll-finding/the-digital-divide-and-access-to-health/. Published April 1, 2011. Accessed October 4, 2017.
35. Schamber EM, Takemoto SK, Chenok KE, Bozic KJ. Barriers to completion of patient reported outcome measures. J Arthroplasty. 2013;28(9):1449-1453.
36. El-Daly I, Ibraheim H, Rajakulendran K, Culpan P, Bates P. Are patient-reported outcome measures in orthopaedics easily read by patients? Clin Orthop Relat Res. 2016;474(1):246-255.
37. Intro to PROMIS. 2016. Health Measures website. http://www.healthmeasures.net/explore-measurement-systems/promis/intro-to-promis. Accessed October 4, 2017.
38. Hung M, Baumhauer JF, Latt LD, Saltzman CL, SooHoo NF, Hunt KJ; National Orthopaedic Foot & Ankle Outcomes Research Network. Validation of PROMIS ® Physical Function computerized adaptive tests for orthopaedic foot and ankle outcome research. Clin Orthop Relat Res. 2013;471(11):3466-3474.
39. Hung M, Clegg DO, Greene T, Saltzman CL. Evaluation of the PROMIS Physical Function item bank in orthopaedic patients. J Orthop Res. 2011;29(6):947-953.
40. Tyser AR, Beckmann J, Franklin JD, et al. Evaluation of the PROMIS Physical Function computer adaptive test in the upper extremity. J Hand Surg Am. 2014;39(10):2047-2051.e4.
41. Hung M, Stuart AR, Higgins TF, Saltzman CL, Kubiak EN. Computerized adaptive testing using the PROMIS Physical Function item bank reduces test burden with less ceiling effects compared with the Short Musculoskeletal Function Assessment in orthopaedic trauma patients. J Orthop Trauma. 2014;28(8):439-443.
42. Hung M, Clegg DO, Greene T, Weir C, Saltzman CL. A lower extremity physical function computerized adaptive testing instrument for orthopaedic patients. Foot Ankle Int. 2012;33(4):326-335.
43. Döring AC, Nota SP, Hageman MG, Ring DC. Measurement of upper extremity disability using the Patient-Reported Outcomes Measurement Information System. J Hand Surg Am. 2014;39(6):1160-1165.
44. Halawi MJ, Greene K, Barsoum WK. Optimizing outcomes of total joint arthroplasty under the comprehensive care for joint replacement model. Am J Orthop. 2016;45(3):E112-E113.
1. Howie L, Hirsch B, Locklear T, Abernethy AP. Assessing the value of patient-generated data to comparative effectiveness research. Health Aff (Millwood). 2014;33(7):1220-1228.
2. Haywood KL. Patient-reported outcome I: measuring what matters in musculoskeletal care. Musculoskeletal Care. 2006;4(4):187-203.
3. Chang CH. Patient-reported outcomes measurement and management with innovative methodologies and technologies. Qual Life Res. 2007;16(suppl 1):157-166.
4. Black N. Patient reported outcome measures could help transform healthcare. BMJ. 2013;346:f167.
5. Porter ME. A strategy for health care reform—toward a value-based system. N Engl J Med. 2009;361(2):109-112.
6. Scott J, Huskisson EC. Graphic representation of pain. Pain. 1976;2(2):175-184.
7. de Nies F, Fidler MW. Visual analog scale for the assessment of total hip arthroplasty. J Arthroplasty. 1997;12(4):416-419.
8. Ayers DC, Franklin PD, Ring DC. The role of emotional health in functional outcomes after orthopaedic surgery: extending the biopsychosocial model to orthopaedics: AOA critical issues. J Bone Joint Surg Am. 2013;95(21):e165.
9. Edwards RR, Haythornthwaite JA, Smith MT, Klick B, Katz JN. Catastrophizing and depressive symptoms as prospective predictors of outcomes following total knee replacement. Pain Res Manag. 2009;14(4):307-311.
10. Patel AA, Donegan D, Albert T. The 36-Item Short Form. J Am Acad Orthop Surg. 2007;15(2):126-134.
11. Ware J Jr, Kosinski M, Keller SD. A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34(3):220-233.
12. About the VR-36, VR-12 and VR-6D. Boston University School of Public Health website. http://www.bu.edu/sph/research/research-landing-page/vr-36-vr-12-and-vr-6d/. Accessed October 4, 2017.
13. Jansson KA, Granath F. Health-related quality of life (EQ-5D) before and after orthopedic surgery. Acta Orthop. 2011;82(1):82-89.
14. Oak SR, Strnad GJ, Bena J, et al. Responsiveness comparison of the EQ-5D, PROMIS Global Health, and VR-12 questionnaires in knee arthroscopy. Orthop J Sports Med. 2016;4(12):2325967116674714.
15. Lavernia CJ, Iacobelli DA, Brooks L, Villa JM. The cost-utility of total hip arthroplasty: earlier intervention, improved economics. J Arthroplasty. 2015;30(6):945-949.
16. Mather RC 3rd, Watters TS, Orlando LA, Bolognesi MP, Moorman CT 3rd. Cost effectiveness analysis of hemiarthroplasty and total shoulder arthroplasty. J Shoulder Elbow Surg. 2010;19(3):325-334.
17. Brauer CA, Rosen AB, Olchanski NV, Neumann PJ. Cost-utility analyses in orthopaedic surgery. J Bone Joint Surg Am. 2005;87(6):1253-1259.
18. Schmidt S, Ferrer M, González M, et al; EMPRO Group. Evaluation of shoulder-specific patient-reported outcome measures: a systematic and standardized comparison of available evidence. J Shoulder Elbow Surg. 2014;23(3):434-444.
19. Godfrey J, Hamman R, Lowenstein S, Briggs K, Kocher M. Reliability, validity, and responsiveness of the Simple Shoulder Test: psychometric properties by age and injury type. J Shoulder Elbow Surg. 2007;16(3):260-267.
20. Kirkley A, Griffin S, McLintock H, Ng L. The development and evaluation of a disease-specific quality of life measurement tool for shoulder instability. The Western Ontario Shoulder Instability Index (WOSI). Am J Sports Med. 1998;26(6):764-772.
21. Briggs KK, Lysholm J, Tegner Y, Rodkey WG, Kocher MS, Steadman JR. The reliability, validity, and responsiveness of the Lysholm score and Tegner Activity Scale for anterior cruciate ligament injuries of the knee: 25 years later. Am J Sports Med. 2009;37(5):890-897.
22. Davidson M, Keating J. Patient-reported outcome measures (PROMs): how should I interpret reports of measurement properties? A practical guide for clinicians and researchers who are not biostatisticians. Br J Sports Med. 2014;48(9):792-796.
23. Bent NP, Wright CC, Rushton AB, Batt ME. Selecting outcome measures in sports medicine: a guide for practitioners using the example of anterior cruciate ligament rehabilitation. Br J Sports Med. 2009;43(13):1006-1012.
24. Rolfson O, Eresian Chenok K, Bohm E, et al; Patient-Reported Outcome Measures Working Group of the International Society of Arthroplasty Registries. Patient-reported outcome measures in arthroplasty registries. Acta Orthop. 2016;87(suppl 1):3-8.
25. Franklin PD, Lewallen D, Bozic K, Hallstrom B, Jiranek W, Ayers DC. Implementation of patient-reported outcome measures in U.S. total joint replacement registries: rationale, status, and plans. J Bone Joint Surg Am. 2014;96(suppl 1):104-109.
26. Lyman S, Lee YY, Franklin PD, Li W, Cross MB, Padgett DE. Validation of the KOOS, JR: a short-form knee arthroplasty outcomes survey. Clin Orthop Relat Res. 2016;474(6):1461-1471.
27. Lyman S, Lee YY, Franklin PD, Li W, Mayman DJ, Padgett DE. Validation of the HOOS, JR: a short-form hip replacement survey. Clin Orthop Relat Res. 2016;474(6):1472-1482.
28. Hutchings A, Neuburger J, Grosse Frie K, Black N, van der Meulen J. Factors associated with non-response in routine use of patient reported outcome measures after elective surgery in England. Health Qual Life Outcomes. 2012;10:34.
29. Edwards PJ, Roberts I, Clarke MJ, et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev. 2009;(3):MR000008.
30. Gakhar H, McConnell B, Apostolopoulos AP, Lewis P. A pilot study investigating the use of at-home, web-based questionnaires compiling patient-reported outcome measures following total hip and knee replacement surgeries. J Long Term Eff Med Implants. 2013;23(1):39-43.
31. Bojcic JL, Sue VM, Huon TS, Maletis GB, Inacio MC. Comparison of paper and electronic surveys for measuring patient-reported outcomes after anterior cruciate ligament reconstruction. Perm J. 2014;18(3):22-26.
32. Rolfson O, Salomonsson R, Dahlberg LE, Garellick G. Internet-based follow-up questionnaire for measuring patient-reported outcome after total hip replacement surgery—reliability and response rate. Value Health. 2011;14(2):316-321.
33. Shah KN, Hofmann MR, Schwarzkopf R, et al. Patient-reported outcome measures: how do digital tablets stack up to paper forms? A randomized, controlled study. Am J Orthop. 2016;45(7):E451-E457.
34. Kaiser Family Foundation. The Digital Divide and Access to Health Information Online. http://kff.org/disparities-policy/poll-finding/the-digital-divide-and-access-to-health/. Published April 1, 2011. Accessed October 4, 2017.
35. Schamber EM, Takemoto SK, Chenok KE, Bozic KJ. Barriers to completion of patient reported outcome measures. J Arthroplasty. 2013;28(9):1449-1453.
36. El-Daly I, Ibraheim H, Rajakulendran K, Culpan P, Bates P. Are patient-reported outcome measures in orthopaedics easily read by patients? Clin Orthop Relat Res. 2016;474(1):246-255.
37. Intro to PROMIS. 2016. Health Measures website. http://www.healthmeasures.net/explore-measurement-systems/promis/intro-to-promis. Accessed October 4, 2017.
38. Hung M, Baumhauer JF, Latt LD, Saltzman CL, SooHoo NF, Hunt KJ; National Orthopaedic Foot & Ankle Outcomes Research Network. Validation of PROMIS ® Physical Function computerized adaptive tests for orthopaedic foot and ankle outcome research. Clin Orthop Relat Res. 2013;471(11):3466-3474.
39. Hung M, Clegg DO, Greene T, Saltzman CL. Evaluation of the PROMIS Physical Function item bank in orthopaedic patients. J Orthop Res. 2011;29(6):947-953.
40. Tyser AR, Beckmann J, Franklin JD, et al. Evaluation of the PROMIS Physical Function computer adaptive test in the upper extremity. J Hand Surg Am. 2014;39(10):2047-2051.e4.
41. Hung M, Stuart AR, Higgins TF, Saltzman CL, Kubiak EN. Computerized adaptive testing using the PROMIS Physical Function item bank reduces test burden with less ceiling effects compared with the Short Musculoskeletal Function Assessment in orthopaedic trauma patients. J Orthop Trauma. 2014;28(8):439-443.
42. Hung M, Clegg DO, Greene T, Weir C, Saltzman CL. A lower extremity physical function computerized adaptive testing instrument for orthopaedic patients. Foot Ankle Int. 2012;33(4):326-335.
43. Döring AC, Nota SP, Hageman MG, Ring DC. Measurement of upper extremity disability using the Patient-Reported Outcomes Measurement Information System. J Hand Surg Am. 2014;39(6):1160-1165.
44. Halawi MJ, Greene K, Barsoum WK. Optimizing outcomes of total joint arthroplasty under the comprehensive care for joint replacement model. Am J Orthop. 2016;45(3):E112-E113.