Gender and racial biases in Press Ganey patient satisfaction surveys

Article Type
Article
Changed
Wed, 07/19/2023 - 11:39
Author(s)
Laura N. Homewood, MD
Deirdre A. Lum, MD
Lisa J. Rogo-Gupta, MD

ILLUSTRATION © IRYNA INSHYNA/SHUTTERSTOCK

Patient satisfaction questionnaires were developed in the 1980s as part of the movement to better understand the patient’s experience and their perspective of the quality of care. In 1985, the Press Ganey survey—now the most widely used method to assess patient satisfaction—was developed by 2 professors in anthropology and sociology-statistics at Notre Dame. Initially intended for inpatient admissions, the survey was validated based on a few thousand survey results.1 Given the strong interest in improving patient satisfaction at the time, it became widely adopted and quickly expanded into outpatient encounters and ambulatory surgery settings.

Although other surveys have been developed,2 the Press Ganey survey is the most commonly used assessment tool for patient satisfaction metrics in the United States, with approximately 50% of all hospitals and more than 41,000 health care organizations using its services.3,4 The survey consists of 6 domains related to satisfaction with:

1. the care provider

2. the nurse or assistant

3. personal issues

4. overall assessment

5. access

6. moving through the visit. 

Survey items are scored using a 5-point Likert scale, with scores ranging from “very poor” (a score of 1) to “very good” (a score of 5). According to the company, because this format is balanced and parallel (unlike a “poor” to “excellent” format), responses can be quantified and used statistically without violating methodologic assumptions. Also, variability in patients’ responses with this format allows for the identification of opportunities to improve, unlike “yes/no” response formats.1 There are limitations to this design, however, which can impact data quality,5 as we will see.

While the distribution process varies by institution, there is an algorithm laid out by Press Ganey for administering surveys to patients in their preferred language after outpatient visits. Based on recent research into Press Ganey response rates, the typical response rate is estimated to be 16% to 19%.6 Although this low response rate is typical of survey data, it inherently introduces the risk of responder bias—meaning results may be skewed by patients who represent the extremes of satisfaction or dissatisfaction.

Adoption of the survey as we move toward value-based care

More recently, patients’ satisfaction with their health care has received increased attention as we move to a patient-centered care model and as health care reimbursement models shift toward value-based care. Current trends in health care policy statements include the importance of raising the standard of care and shifting from a “fee-for-service” to a “pay-for-performance” reimbursement model.7,8 As a result, hospitals are establishing systems to measure “performance” that are not nationally standardized or extensively studied with objective measures. The changing standard of health care expectations in the United States is a topic of much public debate.9 And as expectations and new standards are defined, the impact of implementing novel measures of performance should be evaluated prior to widespread adoption and utilization.

Patient satisfaction also has been identified as a driver for hospital finances through loyalty, described as the “likelihood to return to that system for future medical services.”10,11 This measure has contributed to policy changes that reinforce prioritization of patient satisfaction. For example, the Affordable Care Act tied Medicare reimbursement and patient satisfaction together in the Hospital Value-Based Purchasing Program. This program uses measures of clinical processes, efficiency, outcomes, and patient experiences to calculate a total score that results in hospital reimbursement and incentives,12 which creates a direct pathway from patient experience to reimbursement—underscoring hospitals’ desire for ongoing assessment of patient satisfaction.

Another commonly used patient satisfaction survey

In 2005, the Centers for Medicare and Medicaid Services and the Agency for Health care Research and Quality developed the Hospital Consumer Assessment of Health care Providers and Systems (HCAHPS) survey in response to criticisms of the Press Ganey survey. The HCAHPS survey consists of 27 questions with 3 broad goals19:

  • to produce data about patients’ perspectives of care that allow for objective and meaningful comparisons of hospitals
  •  to publicly report survey results and create new incentives for hospitals to improve quality of care
  •  to produce public reports that enhance accountability by increasing transparency.

One difference with the HCAHPS is that it measures frequency, or how often a service was performed (“never”, “sometimes”, “usually”, “always”), whereas Press Ganey measures satisfaction. It also only surveys inpatients and does not address outpatient encounters. Despite the differences, it is a widely used patient satisfaction survey and is subject to similar issues and biases as the Press Ganey survey.

Continue to: Gender, race, and age bias...

 

 

Gender, race, and age bias

Although the rationale behind gathering patient input is important, recent data suggest that patient satisfaction surveys are subject to inherent biases.6,13,14 These biases tend to negatively impact women and non-White physicians, adding to the systemic discrimination against women and physicians of color that already exists in health care.

In a single-site retrospective study performed in 2018 by Rogo-Gupta et al, female gynecologists were found to be 47% less likely to receive top patient satisfaction scores than their male counterparts owing to their gender alone, suggesting that gender bias may impact the results of patient satisfaction questionnaires.13 The authors encouraged that the results of patient satisfaction surveys be interpreted with great caution until the impact on female physicians is better understood.

A multi-center study by the same group (Rogo-Gupta et al) assessed the same construct across 5 different geographically diverse institutions.15 This study confirmed that female gynecologists were less likely to receive a top satisfaction score from their patients (19% lower odds when compared with male gynecologists). They also studied the effects of other patient demographics, including age, race/ethnicity, and race concordance. Older patients (aged ≥63 years) had an over-3-fold increase in odds of providing a top satisfaction score than younger patients. Additionally, Asian physicians had significantly lower odds of receiving a top satisfaction score when compared with White physicians, while Asian patients had significantly lower odds of providing a top satisfaction score when compared with White patients. Lastly, in most cases, when underrepresented-in-medicine patients saw an underrepresented-in-medicine physician (race concordance), there was a significant increase in odds of receiving a top satisfaction score. Asian race concordance, however, actually resulted in a lower likelihood of receiving a top satisfaction score.15

Literature from other specialties supports these findings. These results are consistent with emerging data from other medical specialties that also suggest that Press Ganey survey data are subject to inherent biases. For example, data from emergency medicine literature have shown discrepancies between patient satisfaction for providers at tertiary inner-city institutions versus those in affluent suburban populations,16 and that worse mortality is actually correlated with better patient satisfaction scores, and vice versa.17

Another study by Sotto-Santiago in 2019 assessed patient satisfaction scores in multiple specialties at a single institution where quality-related financial incentives were offered based on this metric. They found a significant difference in patient satisfaction scores between underrepresented and White physicians, which suggests a potential bias among patients and institutional practices—ultimately leading to pay inequities through differences in financial incentives.18

Percentile differences reveal small gaps in satisfaction ratings

When examining the difference between raw Press Ganey patient satisfaction data and the percentiles associated with these scores, an interesting finding arises. Looking at the 2023 multicenter study by Rogo-Gupta et al, the difference in the top raw scores between male and female gynecologists appears to be small (3.3%).15 However, in 2020, the difference in top scores separating the top (75th) and bottom (25th) percentile quartiles of physicians was also small, at only 6.9%.

Considering the percentiles, if a provider who scores in the 25th percentile is compared with a colleague who scores in the 75th percentile, they may think the reported satisfaction score differences were quite large. This may potentially invoke feelings of decreased self-worth, negatively impact their professional identity or overall well-being, and they may seek (or be told to seek) improvement opportunities. Now imagine the provider in question realizes the difference between the 25th percentile and 75th percentile is actually only 6.9%. This information may completely change how the results are interpreted and acted upon by administrators. This is further changed with the understanding that 3.3% of the difference may be due to gender alone, narrowing the gap even further. Providers would become understandably frustrated if measures of success such as reimbursement, financial bonus or incentives, promotion, or advancement are linked to these results. It violates the value of fairness and does not offer an equitable starting point.

Evolution of the data distribution. Another consideration, as noted by Robert C. Lloyd, PhD, one of the statisticians who helped develop the percentile statistical analysis mapping in 1985, is that it was based on a classic bell-shaped distribution of patient satisfaction survey scores.19 Because hospitals, medical groups, and physicians have been working these past 20 years to achieve higher Press Ganey scores, the data no longer have a bell-shaped distribution. Rather, there are significant clusters of raw scores at the high end with a very narrow response range. When these data are mapped to the percentile spectrum, they are highly inaccurate.19

Impact of sample size. According to Press Ganey, a minimum of 30 survey responses collected over the designated time period is necessary to draw meaningful conclusions of the data for a specific individual, program, or hospital. Despite this requirement to achieve statistical significance, Sullivan and DeLucia found that the firm often provides comparative data about hospital departments and individual physicians based on a smaller sample size that may create an unacceptably large margin of error.20 Sullivan, for example, said his department may only have 8 to 10 Press Ganey survey responses per month and yet still receives monthly reports from the company analyzing the data. Because of the small sample size, 1 month his department ranked in the 1st percentile and 2 months later it ranked in the 99th percentile.20

The effect of a high ceiling rate. A psychometrics report for the Press Ganey survey is available from the vendor that provides vague assessments of reliability and validity based on 2,762 surveys from 12 practices across 10 states. This report describes a 12-question version of the survey with “no problems encountered” with missingness and response variability. The report further states that the Press Ganey survey demonstrates construct, convergent, divergent, and predictive validities, and high reliability; however, these data are not made available.1

In response to this report, Presson et al analyzed more than 34,000 surveys from one institution to evaluate the reliability and validity of the Press Ganey survey.21 Overall, the survey demonstrated suitable psychometric properties for most metrics. However, Presson et al noted a significantly high ceiling rate of 29.3% for the total score, which ranged from 55.4% to 84.1% across items.21 (Ceiling rates are considered substantial if they occur more than 20% of the time.) Ultimately, a high ceiling rate reduces the power to discriminate between patients who have high satisfaction (everyone is “happy”) with those who are just slightly less than happy, but not dissatisfied. This data quality metric can impact the reliability and validity of a survey—and substantial ceiling rates can notably impact percentile rankings of scores within an institution, offering a possible explanation for the small percentage change between the top and bottom percentiles.

Continue to: Other issues with surveys...

 

 

Other issues with surveys

In addition to the limitations associated with percentile groupings, survey data are always subject to nonresponse bias, and small sample size can lead to nonsignificant results. Skewed responses also can make it difficult to identify true outlying providers who may need remediation or may be offering a superior patient experience. Satisfaction surveys also lack an assessment of objective data and instead assess how patients perceive and feel, which introduces subjectivity to the results.

Additionally, focusing on improving patient experience ratings can incentivize unnecessary or inappropriate care (ie, overprescribing of narcotics, prescribing antibiotics when not indicated, or ordering testing that may not change management). Some physicians even state that they are not getting the type of feedback that they are asking for and that the survey is not asking the right questions to elicit patient input that is meaningful to their practice. Lastly, the incorporation of trainees and advanced practice providers in the patient care experience leads to the assessment of an alternative provider being included in the ultimate score and may not be representative of that physician.

Patients’ perception and survey results. In some circumstances, the patient’s understanding of their medical situation may affect their responses. Some may argue that patients may mistake a physician’s confidence for competence, when in reality these two entities are mutually exclusive. In a randomized controlled trial, researchers from Mount Sinai School of Medicine and Columbia University Medical Center surveyed inner-city women with newly diagnosed and surgically treated early-stage breast cancer for their perceived quality of care and the process of getting care, including referrals, test results, and treatments. They compared the responses with patient records to determine the actual quality of care. Of the 374 women who received treatment for early-stage breast cancer, 55% said they received “excellent care,” but most—88%—actually got care that was in line with the best current treatment guidelines. Interestingly, the study found African American women were less likely to report excellent care than White or Hispanic women, less likely to trust their doctor, and more likely to say they experienced bias during the process. However, there was no difference in actual quality of care received in any group.22

You can’t improve what you can’t control. Ultimately, while many providers think patient satisfaction survey results may help inform some aspects of their practice, they cannot improve what they cannot control. For example, the multicenter study by Rogo-Gupta et al found that older patients (≥63 years) have more than a 3-fold increase in odds of giving a top satisfaction score than younger patients (≤33 years), independent of other aspects of the care experience.15 Additionally, they found that older physicians (≥56 years) had a significant increase in odds of receiving a top satisfaction score when compared with physicians who were younger than 55 years old.15 Given that physicians clearly cannot control their own age or the age of their patients, the negative impacts of these biases need to be addressed and remedied at a systems level.

Why might these biases exist?

While we cannot completely understand all of the possible explanations for these biases, it is important to emphasize the long-standing prejudice and discrimination against women and people of color in our society and how this has impacted our behavior. While strides have been made, there clearly still seems to be a difference between what we say and how our biases impact our behavior. Women are still tougher on women in professional evaluations in other fields as well23; it is not unique to medicine. While workplace improvements are slowly changing, women still face inequities. The more research we publish to describe it, the more we hope the conversation continues, allowing us to reduce the impact of bias on our sense of self-worth and identity related to our careers, narrow the pay gap, and see women advance at the same rate as male counterparts. Considerable transformation is crucial to prevent further workforce attrition.

With regard to the lower scores provided by Asian patients, studies suggest that cultural response bias, rather than true differences in quality of care, may account for these discrepancies. Previous literature has found that Asian patients are more likely to select midpoints, rather than extremes, when completing Likert-type studies24 and are not more likely to change medical providers than other race/ethnicities, indicating that lower ratings may not necessarily imply greater dissatisfaction with care.25

Far-reaching effects on finances, income, well-being, job satisfaction, etc.

Depending on how the results are distributed and used, the effects of patient satisfaction surveys can extend well beyond the original intentions. At some institutions, income for physicians is directly tied to their Press Ganey satisfaction scores, which could have profound implications for female and Asian physicians,13,15 who would be paid less—resulting in a wider pay gap than already exists.18

When negative and not constructive, patient evaluations can contribute to physician burnout and a loss of productive members of the workforce.26 This is especially important in obstetrics and gynecology, where physicians are most likely to experience burnout due to multiple factors such as high-risk medical conditions, pressures of the electronic medical record (EMR), the medicolegal environment, and difficulty balancing patient expectations for autonomy with professional judgement.27 Burnout also disproportionately affects women and younger physicians, which is especially concerning given that, in 2017, approximately one-third of practicing obstetrician/gynecologists were women, while that same year more than 80% of trainees matching into the field were women.28 In one survey sent to members of a prominent medical society, 20% of the medical professionals who responded said they have had their employment threatened by low patient satisfaction scores, 78% reported that patient satisfaction surveys moderately or severely affected their job satisfaction, and 28% stated they had considered quitting their job or leaving the medical profession.29Another related effect is the association between malpractice proceedings and a lack of satisfaction with perceived quality of physician-patient communication.30 This may be an important determinant of malpractice lawsuits, and ensuring high patient satisfaction may be a form of defensive medicine.

Continue to: Controlling the narrative for the future: Proposed strategies...

 

 

Controlling the narrative for the future: Proposed strategies

The rapid, widespread adoption of the Press Ganey survey across specialties, clinical care settings, and geographic areas may have been largely due to the ease and operational benefits for hospitals rather than after rigorous study and validation. For example, repeated use of a specific measurement tool may facilitate comparison across areas within a hospital but also across institutions, which can help assess performance at a national level. Hospitals also may have a financial incentive to work with a single third-party or vendor instead of using multiple options across multiple vendors. However, the impact of adoption of novel measures of performance should be evaluated prior to widespread adoption and utilization.

A similar example of an emergence of a technological advancement that has changed the field of medicine and how we provide care is the EMR. Epic is now the most commonly used medical record system and holds the market share of the industry, covering 78% of patients in the United States.31 While there are certainly many potential benefits of a common EMR, such as ease of information sharing and standardization of formatting, opportunities are identified in real time and require product adjustment. For example, modifications have been made to accurately represent gender outside of the previously used dichotomous options. Diagnoses such as cervical cancer screening can now be used even if the patient gender is listed as male.

Similarly, the Press Ganey and other patient satisfaction questionnaires should be evaluated and modified to address existing societal biases. The World Health Organization estimates that it will take 300 years to fix gender inequality,32 but we have an opportunity now to control the narrative and improve patient feedback.

Future research avenues

Ultimately, there is a need to further explore currently available methods of evaluating clinical encounters to better understand the inherent biases and limitations. We hope this review will encourage other physicians to examine their specialties and hospitals and require similar analyses from vendors of such satisfaction rating products prior to using them. At the very least, health systems should be willing to partner with vendors and physicians on an ongoing basis to better understand the biases involved in these survey results and make modifications as needed. Patients also obtain information from and contribute to self-reported, publicly available websites; therefore, additional exploration into a nationalized standard for assessing patient satisfaction also may serve as an opportunity to standardize the information patients evaluate.33 Further assessment of the potential financial and emotional impact of using the currently available patient-reported surveys on female physicians, Asian physicians, young physicians, and physicians who see young patients is needed. It is time to put pressure on a broken patient satisfaction system and improve on a national level to avoid undue negative consequences on our physicians. Use of patient satisfaction survey data should be limited until we all understand and account for the biases that are evident. ●

Proposed strategies to address bias in patient satisfaction surveys
  • Appeal to the Press Ganey corporation with the results of recent data and other studies to ensure they are aware of the biases that exist in their product
  • Appeal to hospital-level administration to refrain from using Press Ganey scores as a tool to dictate reimbursement; instead rely on other more objective measures of performance (such as publications, presentations, research accomplishments, patient and surgical outcomes, promotion, committees, national leadership roles, etc)
  • Apply a “corrective factor” or “adjustment factor” to eliminate the baseline discrepancy between scores for men and women
  • Consider moving to an alternative survey methodology
  • Provide patient education to define “performance” (ie, frame what a patient can expect from a provider such as being on time, courteous, and empathetic; caution against asking patients to assess competence and knowledge)
References
  1. Outpatient Services (OU) Survey Psychometrics Report. Published online 2019.
  2. Zusman EE. HCAHPS replaces Press Ganey Survey as quality  measure for patient hospital experience. Neurosurgery. 2012;71:N21-N24. doi: 10.1227/01.neu.0000417536.07871.ed
  3. Press Ganey. Company. Accessed April 20, 2023. www.pressganey. com/company/
  4.  Press, Ganey--first year of patient satisfaction measurement. Hosp Guest Relations Rep. 1986;1:4-5.
  5. DeCastellarnau A. A classification of response scale characteristics that affect data quality: a literature review. Qual Quant. 2018;52:15231559. doi: 10.1007/s11135-017-0533-4
  6. Tyser AR, Abtahi AM, McFadden M, et al. Evidence of non-response bias in the Press-Ganey patient satisfaction survey. BMC Health Serv Res. 2016;16:350. doi: 10.1186/s12913-016-1595-z
  7. Duseja R, Durham M, Schreiber M. CMS quality measure development. JAMA. 2020;324:1213-1214. doi: 10.1001/jama.2020.12070
  8. Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. National Academies Press; 2001. doi: 10.17226/10027
  9. Parmet WE. Health: policy or law? A population-based analysis of the Supreme Court’s ACA cases. J Health Polit Policy Law. 2016;41:10611081. doi: 10.1215/03616878-3665949
  10. Richter JP, Muhlestein DB. Patient experience and hospital profitability: is there a link? Health Care Manage Rev. 2017;42:247-257. doi: 10.1097/HMR.0000000000000105
  11. Huang C-H, Wu H-H, Lee Y-C, et al. What role does patient gratitude play in the relationship between relationship quality and patient loyalty? Inquiry. 2019;56:46958019868324. doi: 10.1177/0046958019868324
  12. Centers for Medicare & Medicaid Services (CMS), HHS. Medicare program; hospital inpatient value-based purchasing program. Final rule. Fed Regist. 2011;76:26490-26547.
  13. Rogo-Gupta LJ, Haunschild C, Altamirano J, et al. Physician gender is associated with Press Ganey patient satisfaction scores in outpatient gynecology. Womens Health Issues. 2018;28:281-285. doi: 10.1016 /j.whi.2018.01.001
  14. DeLoughery EP. Physician race and specialty influence Press Ganey survey results. Neth J Med. 2019;77:366-369.
  15. Homewood L, Altamirano J, Fassiotto M, et al. Women gynecologists receive lower Press Ganey patient satisfaction scores in a multicenter cross-sectional study. Am J Obstet Gynecol. 2023;228:S801. doi: 10.1016/j.ajog.2022.12.025
  16. Sharp B, Johnson J, Hamedani AG, et al. What are we measuring? Evaluating physician-specific satisfaction scores between emergency departments. West J Emerg Med. 2019;20:454-459. doi: 10.5811 /westjem.2019.4.41040
  17. Mosley M. Viewpoint: Press Ganey is a worthless tool for the ED. Emerg Med News. 2019;41:3-4. doi: 10.1097/01.EEM.0000616512.68475.69
  18. Sotto-Santiago S, Slaven JE, Rohr-Kirchgraber T. (Dis)Incentivizing patient satisfaction metrics: the unintended consequences of institutional bias. Health Equity. 2019;3:13-18. doi: 10.1089/heq.2018.0065
  19. Lloyd RC. Quality Health Care: A Guide to Developing and Using Indicators. 2nd ed. Jones & Bartlett Learning; 2019. Accessed April 23, 2023. www.jblearning.com/catalog/productdetails /9781284023077
  20. 2+2=7? Seven things you may not know about Press Ganey statistics. Emergency Physicians Monthly. Accessed April 23, 2023. epmonthly. com/article/227-seven-things-you-may-not-know-about-pressgainey-statistics/
  21. Presson AP, Zhang C, Abtahi AM, et al. Psychometric properties of the Press Ganey® Outpatient Medical Practice Survey. Health Qual Life Outcomes. 2017;15:32. doi: 10.1186/s12955-017-0610-3
  22. Bickell NA, Neuman J, Fei K, et al. Quality of breast cancer care: perception versus practice. J Clin Oncol. 2012;30:1791-1795. doi: 10.1200 /JCO.2011.38.7605
  23. Strauss K. Women in the workplace: are women tougher on other women? Forbes. July 18, 2016. Accessed April 27, 2023. www.forbes. com/sites/karstenstrauss/2016/07/18/women-in-the-workplace -are-women-tougher-on-other-women/
  24. Lee JW, Jones PS, Mineyama Y, et al. Cultural differences in responses to a Likert scale. Res Nurs Health. 2002;25:295-306. doi: 10.1002 /nur.10041
  25. Saha S, Hickam DH. Explaining low ratings of patient satisfaction among Asian-Americans. Am J Med Qual. 2003;18:256-264. doi: 10.1177/106286060301800606
  26. Halbesleben JRB, Rathert C. Linking physician burnout and patient outcomes: exploring the dyadic relationship between physicians and patients. Health Care Manage Rev. 2008;33:29-39. doi: 10.1097/01. HMR.0000304493.87898.72
  27. Bradford L, Glaser G. Addressing physician burnout and ensuring high-quality care of the physician workforce. Obstet Gynecol. 2021;137:3-11. doi: 10.1097/AOG.0000000000004197
  28. Boyle P. Nation’s physician workforce evolves: more women, a bit older, and toward different specialties. AAMCNEWS. February 2, 2021. Accessed April 20, 2023. www.aamc.org/news-insights/nations-physician-workforce-evolves-more-women-bit-older-and-towarddifferent-specialties
  29. Zgierska A, Rabago D, Miller MM. Impact of patient satisfaction ratings on physicians and clinical care. Patient Prefer Adherence. 2014;8:437-446. doi: 10.2147/PPA.S59077
  30. Yeh J, Nagel EE. Patient satisfaction in obstetrics and gynecology: individualized patient-centered communication. Clin Med Insights  Womens Health. 2010;3:23. doi: 10.4137/CMWH.S5870
  31. Epic. About us. Accessed April 19, 2023. www.epic.com/about
  32. United Nations. Without investment, gender equality will take nearly 300 years: UN report. September 7, 2022. Accessed April 19, 2023. news.un.org/en/story/2022/09/1126171
  33. Ryan T, Specht J, Smith S, et al. Does the Press Ganey Survey correlate to online health grades for a major academic otolaryngology department? Otolaryngol Head Neck Surg. 2016;155:411-415. doi: 10.1177/0194599816652386
Article PDF
obgm_sgs_0723.pdf
Author and Disclosure Information

Laura N. Homewood, MD 

Assistant Professor 
University of Virginia Health 
Charlottesville, Virginia 

Deirdre A. Lum, MD 

Clinical Associate Professor 
Obstetrics and Gynecology 
Stanford Medicine 
Palo Alto, California 

Lisa J. Rogo-Gupta, MD 

Clinical Associate Professor 
Obstetrics and Gynecology 
Clinical Associate Professor (by courtesy) 
Urology 
Stanford Medicine 
Palo Alto, California

The authors report no financial disclosures related to this editorial. 

Issue
OBG Management - 35(7)
Publications
MDedge ObGyn
OBG Management
Topics
Practice Management
Page Number
SS13-SS19
Sections
Clinical Review
Author(s)
Laura N. Homewood, MD
Deirdre A. Lum, MD
Lisa J. Rogo-Gupta, MD
Author(s)
Laura N. Homewood, MD
Deirdre A. Lum, MD
Lisa J. Rogo-Gupta, MD
Author and Disclosure Information

Laura N. Homewood, MD 

Assistant Professor 
University of Virginia Health 
Charlottesville, Virginia 

Deirdre A. Lum, MD 

Clinical Associate Professor 
Obstetrics and Gynecology 
Stanford Medicine 
Palo Alto, California 

Lisa J. Rogo-Gupta, MD 

Clinical Associate Professor 
Obstetrics and Gynecology 
Clinical Associate Professor (by courtesy) 
Urology 
Stanford Medicine 
Palo Alto, California

The authors report no financial disclosures related to this editorial. 

Author and Disclosure Information

Laura N. Homewood, MD 

Assistant Professor 
University of Virginia Health 
Charlottesville, Virginia 

Deirdre A. Lum, MD 

Clinical Associate Professor 
Obstetrics and Gynecology 
Stanford Medicine 
Palo Alto, California 

Lisa J. Rogo-Gupta, MD 

Clinical Associate Professor 
Obstetrics and Gynecology 
Clinical Associate Professor (by courtesy) 
Urology 
Stanford Medicine 
Palo Alto, California

The authors report no financial disclosures related to this editorial. 

Article PDF
obgm_sgs_0723.pdf
Article PDF
obgm_sgs_0723.pdf

ILLUSTRATION © IRYNA INSHYNA/SHUTTERSTOCK

Patient satisfaction questionnaires were developed in the 1980s as part of the movement to better understand the patient’s experience and their perspective of the quality of care. In 1985, the Press Ganey survey—now the most widely used method to assess patient satisfaction—was developed by 2 professors in anthropology and sociology-statistics at Notre Dame. Initially intended for inpatient admissions, the survey was validated based on a few thousand survey results.1 Given the strong interest in improving patient satisfaction at the time, it became widely adopted and quickly expanded into outpatient encounters and ambulatory surgery settings.

Although other surveys have been developed,2 the Press Ganey survey is the most commonly used assessment tool for patient satisfaction metrics in the United States, with approximately 50% of all hospitals and more than 41,000 health care organizations using its services.3,4 The survey consists of 6 domains related to satisfaction with:

1. the care provider

2. the nurse or assistant

3. personal issues

4. overall assessment

5. access

6. moving through the visit. 

Survey items are scored using a 5-point Likert scale, with scores ranging from “very poor” (a score of 1) to “very good” (a score of 5). According to the company, because this format is balanced and parallel (unlike a “poor” to “excellent” format), responses can be quantified and used statistically without violating methodologic assumptions. Also, variability in patients’ responses with this format allows for the identification of opportunities to improve, unlike “yes/no” response formats.1 There are limitations to this design, however, which can impact data quality,5 as we will see.

While the distribution process varies by institution, there is an algorithm laid out by Press Ganey for administering surveys to patients in their preferred language after outpatient visits. Based on recent research into Press Ganey response rates, the typical response rate is estimated to be 16% to 19%.6 Although this low response rate is typical of survey data, it inherently introduces the risk of responder bias—meaning results may be skewed by patients who represent the extremes of satisfaction or dissatisfaction.

Adoption of the survey as we move toward value-based care

More recently, patients’ satisfaction with their health care has received increased attention as we move to a patient-centered care model and as health care reimbursement models shift toward value-based care. Current trends in health care policy statements include the importance of raising the standard of care and shifting from a “fee-for-service” to a “pay-for-performance” reimbursement model.7,8 As a result, hospitals are establishing systems to measure “performance” that are not nationally standardized or extensively studied with objective measures. The changing standard of health care expectations in the United States is a topic of much public debate.9 And as expectations and new standards are defined, the impact of implementing novel measures of performance should be evaluated prior to widespread adoption and utilization.

Patient satisfaction also has been identified as a driver for hospital finances through loyalty, described as the “likelihood to return to that system for future medical services.”10,11 This measure has contributed to policy changes that reinforce prioritization of patient satisfaction. For example, the Affordable Care Act tied Medicare reimbursement and patient satisfaction together in the Hospital Value-Based Purchasing Program. This program uses measures of clinical processes, efficiency, outcomes, and patient experiences to calculate a total score that results in hospital reimbursement and incentives,12 which creates a direct pathway from patient experience to reimbursement—underscoring hospitals’ desire for ongoing assessment of patient satisfaction.

Another commonly used patient satisfaction survey

In 2005, the Centers for Medicare and Medicaid Services and the Agency for Health care Research and Quality developed the Hospital Consumer Assessment of Health care Providers and Systems (HCAHPS) survey in response to criticisms of the Press Ganey survey. The HCAHPS survey consists of 27 questions with 3 broad goals19:

  • to produce data about patients’ perspectives of care that allow for objective and meaningful comparisons of hospitals
  •  to publicly report survey results and create new incentives for hospitals to improve quality of care
  •  to produce public reports that enhance accountability by increasing transparency.

One difference with the HCAHPS is that it measures frequency, or how often a service was performed (“never”, “sometimes”, “usually”, “always”), whereas Press Ganey measures satisfaction. It also only surveys inpatients and does not address outpatient encounters. Despite the differences, it is a widely used patient satisfaction survey and is subject to similar issues and biases as the Press Ganey survey.

Continue to: Gender, race, and age bias...

 

 

Gender, race, and age bias

Although the rationale behind gathering patient input is important, recent data suggest that patient satisfaction surveys are subject to inherent biases.6,13,14 These biases tend to negatively impact women and non-White physicians, adding to the systemic discrimination against women and physicians of color that already exists in health care.

In a single-site retrospective study performed in 2018 by Rogo-Gupta et al, female gynecologists were found to be 47% less likely to receive top patient satisfaction scores than their male counterparts owing to their gender alone, suggesting that gender bias may impact the results of patient satisfaction questionnaires.13 The authors encouraged that the results of patient satisfaction surveys be interpreted with great caution until the impact on female physicians is better understood.

A multi-center study by the same group (Rogo-Gupta et al) assessed the same construct across 5 different geographically diverse institutions.15 This study confirmed that female gynecologists were less likely to receive a top satisfaction score from their patients (19% lower odds when compared with male gynecologists). They also studied the effects of other patient demographics, including age, race/ethnicity, and race concordance. Older patients (aged ≥63 years) had an over-3-fold increase in odds of providing a top satisfaction score than younger patients. Additionally, Asian physicians had significantly lower odds of receiving a top satisfaction score when compared with White physicians, while Asian patients had significantly lower odds of providing a top satisfaction score when compared with White patients. Lastly, in most cases, when underrepresented-in-medicine patients saw an underrepresented-in-medicine physician (race concordance), there was a significant increase in odds of receiving a top satisfaction score. Asian race concordance, however, actually resulted in a lower likelihood of receiving a top satisfaction score.15

Literature from other specialties supports these findings. These results are consistent with emerging data from other medical specialties that also suggest that Press Ganey survey data are subject to inherent biases. For example, data from emergency medicine literature have shown discrepancies between patient satisfaction for providers at tertiary inner-city institutions versus those in affluent suburban populations,16 and that worse mortality is actually correlated with better patient satisfaction scores, and vice versa.17

Another study by Sotto-Santiago in 2019 assessed patient satisfaction scores in multiple specialties at a single institution where quality-related financial incentives were offered based on this metric. They found a significant difference in patient satisfaction scores between underrepresented and White physicians, which suggests a potential bias among patients and institutional practices—ultimately leading to pay inequities through differences in financial incentives.18

Percentile differences reveal small gaps in satisfaction ratings

When examining the difference between raw Press Ganey patient satisfaction data and the percentiles associated with these scores, an interesting finding arises. Looking at the 2023 multicenter study by Rogo-Gupta et al, the difference in the top raw scores between male and female gynecologists appears to be small (3.3%).15 However, in 2020, the difference in top scores separating the top (75th) and bottom (25th) percentile quartiles of physicians was also small, at only 6.9%.

Considering the percentiles, if a provider who scores in the 25th percentile is compared with a colleague who scores in the 75th percentile, they may think the reported satisfaction score differences were quite large. This may potentially invoke feelings of decreased self-worth, negatively impact their professional identity or overall well-being, and they may seek (or be told to seek) improvement opportunities. Now imagine the provider in question realizes the difference between the 25th percentile and 75th percentile is actually only 6.9%. This information may completely change how the results are interpreted and acted upon by administrators. This is further changed with the understanding that 3.3% of the difference may be due to gender alone, narrowing the gap even further. Providers would become understandably frustrated if measures of success such as reimbursement, financial bonus or incentives, promotion, or advancement are linked to these results. It violates the value of fairness and does not offer an equitable starting point.

Evolution of the data distribution. Another consideration, as noted by Robert C. Lloyd, PhD, one of the statisticians who helped develop the percentile statistical analysis mapping in 1985, is that it was based on a classic bell-shaped distribution of patient satisfaction survey scores.19 Because hospitals, medical groups, and physicians have been working these past 20 years to achieve higher Press Ganey scores, the data no longer have a bell-shaped distribution. Rather, there are significant clusters of raw scores at the high end with a very narrow response range. When these data are mapped to the percentile spectrum, they are highly inaccurate.19

Impact of sample size. According to Press Ganey, a minimum of 30 survey responses collected over the designated time period is necessary to draw meaningful conclusions of the data for a specific individual, program, or hospital. Despite this requirement to achieve statistical significance, Sullivan and DeLucia found that the firm often provides comparative data about hospital departments and individual physicians based on a smaller sample size that may create an unacceptably large margin of error.20 Sullivan, for example, said his department may only have 8 to 10 Press Ganey survey responses per month and yet still receives monthly reports from the company analyzing the data. Because of the small sample size, 1 month his department ranked in the 1st percentile and 2 months later it ranked in the 99th percentile.20

The effect of a high ceiling rate. A psychometrics report for the Press Ganey survey is available from the vendor that provides vague assessments of reliability and validity based on 2,762 surveys from 12 practices across 10 states. This report describes a 12-question version of the survey with “no problems encountered” with missingness and response variability. The report further states that the Press Ganey survey demonstrates construct, convergent, divergent, and predictive validities, and high reliability; however, these data are not made available.1

In response to this report, Presson et al analyzed more than 34,000 surveys from one institution to evaluate the reliability and validity of the Press Ganey survey.21 Overall, the survey demonstrated suitable psychometric properties for most metrics. However, Presson et al noted a significantly high ceiling rate of 29.3% for the total score, which ranged from 55.4% to 84.1% across items.21 (Ceiling rates are considered substantial if they occur more than 20% of the time.) Ultimately, a high ceiling rate reduces the power to discriminate between patients who have high satisfaction (everyone is “happy”) with those who are just slightly less than happy, but not dissatisfied. This data quality metric can impact the reliability and validity of a survey—and substantial ceiling rates can notably impact percentile rankings of scores within an institution, offering a possible explanation for the small percentage change between the top and bottom percentiles.

Continue to: Other issues with surveys...

 

 

Other issues with surveys

In addition to the limitations associated with percentile groupings, survey data are always subject to nonresponse bias, and small sample size can lead to nonsignificant results. Skewed responses also can make it difficult to identify true outlying providers who may need remediation or may be offering a superior patient experience. Satisfaction surveys also lack an assessment of objective data and instead assess how patients perceive and feel, which introduces subjectivity to the results.

Additionally, focusing on improving patient experience ratings can incentivize unnecessary or inappropriate care (ie, overprescribing of narcotics, prescribing antibiotics when not indicated, or ordering testing that may not change management). Some physicians even state that they are not getting the type of feedback that they are asking for and that the survey is not asking the right questions to elicit patient input that is meaningful to their practice. Lastly, the incorporation of trainees and advanced practice providers in the patient care experience leads to the assessment of an alternative provider being included in the ultimate score and may not be representative of that physician.

Patients’ perception and survey results. In some circumstances, the patient’s understanding of their medical situation may affect their responses. Some may argue that patients may mistake a physician’s confidence for competence, when in reality these two entities are mutually exclusive. In a randomized controlled trial, researchers from Mount Sinai School of Medicine and Columbia University Medical Center surveyed inner-city women with newly diagnosed and surgically treated early-stage breast cancer for their perceived quality of care and the process of getting care, including referrals, test results, and treatments. They compared the responses with patient records to determine the actual quality of care. Of the 374 women who received treatment for early-stage breast cancer, 55% said they received “excellent care,” but most—88%—actually got care that was in line with the best current treatment guidelines. Interestingly, the study found African American women were less likely to report excellent care than White or Hispanic women, less likely to trust their doctor, and more likely to say they experienced bias during the process. However, there was no difference in actual quality of care received in any group.22

You can’t improve what you can’t control. Ultimately, while many providers think patient satisfaction survey results may help inform some aspects of their practice, they cannot improve what they cannot control. For example, the multicenter study by Rogo-Gupta et al found that older patients (≥63 years) have more than a 3-fold increase in odds of giving a top satisfaction score than younger patients (≤33 years), independent of other aspects of the care experience.15 Additionally, they found that older physicians (≥56 years) had a significant increase in odds of receiving a top satisfaction score when compared with physicians who were younger than 55 years old.15 Given that physicians clearly cannot control their own age or the age of their patients, the negative impacts of these biases need to be addressed and remedied at a systems level.

Why might these biases exist?

While we cannot completely understand all of the possible explanations for these biases, it is important to emphasize the long-standing prejudice and discrimination against women and people of color in our society and how this has impacted our behavior. While strides have been made, there clearly still seems to be a difference between what we say and how our biases impact our behavior. Women are still tougher on women in professional evaluations in other fields as well23; it is not unique to medicine. While workplace improvements are slowly changing, women still face inequities. The more research we publish to describe it, the more we hope the conversation continues, allowing us to reduce the impact of bias on our sense of self-worth and identity related to our careers, narrow the pay gap, and see women advance at the same rate as male counterparts. Considerable transformation is crucial to prevent further workforce attrition.

With regard to the lower scores provided by Asian patients, studies suggest that cultural response bias, rather than true differences in quality of care, may account for these discrepancies. Previous literature has found that Asian patients are more likely to select midpoints, rather than extremes, when completing Likert-type studies24 and are not more likely to change medical providers than other race/ethnicities, indicating that lower ratings may not necessarily imply greater dissatisfaction with care.25

Far-reaching effects on finances, income, well-being, job satisfaction, etc.

Depending on how the results are distributed and used, the effects of patient satisfaction surveys can extend well beyond the original intentions. At some institutions, income for physicians is directly tied to their Press Ganey satisfaction scores, which could have profound implications for female and Asian physicians,13,15 who would be paid less—resulting in a wider pay gap than already exists.18

When negative and not constructive, patient evaluations can contribute to physician burnout and a loss of productive members of the workforce.26 This is especially important in obstetrics and gynecology, where physicians are most likely to experience burnout due to multiple factors such as high-risk medical conditions, pressures of the electronic medical record (EMR), the medicolegal environment, and difficulty balancing patient expectations for autonomy with professional judgement.27 Burnout also disproportionately affects women and younger physicians, which is especially concerning given that, in 2017, approximately one-third of practicing obstetrician/gynecologists were women, while that same year more than 80% of trainees matching into the field were women.28 In one survey sent to members of a prominent medical society, 20% of the medical professionals who responded said they have had their employment threatened by low patient satisfaction scores, 78% reported that patient satisfaction surveys moderately or severely affected their job satisfaction, and 28% stated they had considered quitting their job or leaving the medical profession.29Another related effect is the association between malpractice proceedings and a lack of satisfaction with perceived quality of physician-patient communication.30 This may be an important determinant of malpractice lawsuits, and ensuring high patient satisfaction may be a form of defensive medicine.

Continue to: Controlling the narrative for the future: Proposed strategies...

 

 

Controlling the narrative for the future: Proposed strategies

The rapid, widespread adoption of the Press Ganey survey across specialties, clinical care settings, and geographic areas may have been largely due to the ease and operational benefits for hospitals rather than after rigorous study and validation. For example, repeated use of a specific measurement tool may facilitate comparison across areas within a hospital but also across institutions, which can help assess performance at a national level. Hospitals also may have a financial incentive to work with a single third-party or vendor instead of using multiple options across multiple vendors. However, the impact of adoption of novel measures of performance should be evaluated prior to widespread adoption and utilization.

A similar example of an emergence of a technological advancement that has changed the field of medicine and how we provide care is the EMR. Epic is now the most commonly used medical record system and holds the market share of the industry, covering 78% of patients in the United States.31 While there are certainly many potential benefits of a common EMR, such as ease of information sharing and standardization of formatting, opportunities are identified in real time and require product adjustment. For example, modifications have been made to accurately represent gender outside of the previously used dichotomous options. Diagnoses such as cervical cancer screening can now be used even if the patient gender is listed as male.

Similarly, the Press Ganey and other patient satisfaction questionnaires should be evaluated and modified to address existing societal biases. The World Health Organization estimates that it will take 300 years to fix gender inequality,32 but we have an opportunity now to control the narrative and improve patient feedback.

Future research avenues

Ultimately, there is a need to further explore currently available methods of evaluating clinical encounters to better understand the inherent biases and limitations. We hope this review will encourage other physicians to examine their specialties and hospitals and require similar analyses from vendors of such satisfaction rating products prior to using them. At the very least, health systems should be willing to partner with vendors and physicians on an ongoing basis to better understand the biases involved in these survey results and make modifications as needed. Patients also obtain information from and contribute to self-reported, publicly available websites; therefore, additional exploration into a nationalized standard for assessing patient satisfaction also may serve as an opportunity to standardize the information patients evaluate.33 Further assessment of the potential financial and emotional impact of using the currently available patient-reported surveys on female physicians, Asian physicians, young physicians, and physicians who see young patients is needed. It is time to put pressure on a broken patient satisfaction system and improve on a national level to avoid undue negative consequences on our physicians. Use of patient satisfaction survey data should be limited until we all understand and account for the biases that are evident. ●

Proposed strategies to address bias in patient satisfaction surveys
  • Appeal to the Press Ganey corporation with the results of recent data and other studies to ensure they are aware of the biases that exist in their product
  • Appeal to hospital-level administration to refrain from using Press Ganey scores as a tool to dictate reimbursement; instead rely on other more objective measures of performance (such as publications, presentations, research accomplishments, patient and surgical outcomes, promotion, committees, national leadership roles, etc)
  • Apply a “corrective factor” or “adjustment factor” to eliminate the baseline discrepancy between scores for men and women
  • Consider moving to an alternative survey methodology
  • Provide patient education to define “performance” (ie, frame what a patient can expect from a provider such as being on time, courteous, and empathetic; caution against asking patients to assess competence and knowledge)

ILLUSTRATION © IRYNA INSHYNA/SHUTTERSTOCK

Patient satisfaction questionnaires were developed in the 1980s as part of the movement to better understand the patient’s experience and their perspective of the quality of care. In 1985, the Press Ganey survey—now the most widely used method to assess patient satisfaction—was developed by 2 professors in anthropology and sociology-statistics at Notre Dame. Initially intended for inpatient admissions, the survey was validated based on a few thousand survey results.1 Given the strong interest in improving patient satisfaction at the time, it became widely adopted and quickly expanded into outpatient encounters and ambulatory surgery settings.

Although other surveys have been developed,2 the Press Ganey survey is the most commonly used assessment tool for patient satisfaction metrics in the United States, with approximately 50% of all hospitals and more than 41,000 health care organizations using its services.3,4 The survey consists of 6 domains related to satisfaction with:

1. the care provider

2. the nurse or assistant

3. personal issues

4. overall assessment

5. access

6. moving through the visit. 

Survey items are scored using a 5-point Likert scale, with scores ranging from “very poor” (a score of 1) to “very good” (a score of 5). According to the company, because this format is balanced and parallel (unlike a “poor” to “excellent” format), responses can be quantified and used statistically without violating methodologic assumptions. Also, variability in patients’ responses with this format allows for the identification of opportunities to improve, unlike “yes/no” response formats.1 There are limitations to this design, however, which can impact data quality,5 as we will see.

While the distribution process varies by institution, there is an algorithm laid out by Press Ganey for administering surveys to patients in their preferred language after outpatient visits. Based on recent research into Press Ganey response rates, the typical response rate is estimated to be 16% to 19%.6 Although this low response rate is typical of survey data, it inherently introduces the risk of responder bias—meaning results may be skewed by patients who represent the extremes of satisfaction or dissatisfaction.

Adoption of the survey as we move toward value-based care

More recently, patients’ satisfaction with their health care has received increased attention as we move to a patient-centered care model and as health care reimbursement models shift toward value-based care. Current trends in health care policy statements include the importance of raising the standard of care and shifting from a “fee-for-service” to a “pay-for-performance” reimbursement model.7,8 As a result, hospitals are establishing systems to measure “performance” that are not nationally standardized or extensively studied with objective measures. The changing standard of health care expectations in the United States is a topic of much public debate.9 And as expectations and new standards are defined, the impact of implementing novel measures of performance should be evaluated prior to widespread adoption and utilization.

Patient satisfaction also has been identified as a driver for hospital finances through loyalty, described as the “likelihood to return to that system for future medical services.”10,11 This measure has contributed to policy changes that reinforce prioritization of patient satisfaction. For example, the Affordable Care Act tied Medicare reimbursement and patient satisfaction together in the Hospital Value-Based Purchasing Program. This program uses measures of clinical processes, efficiency, outcomes, and patient experiences to calculate a total score that results in hospital reimbursement and incentives,12 which creates a direct pathway from patient experience to reimbursement—underscoring hospitals’ desire for ongoing assessment of patient satisfaction.

Another commonly used patient satisfaction survey

In 2005, the Centers for Medicare and Medicaid Services and the Agency for Health care Research and Quality developed the Hospital Consumer Assessment of Health care Providers and Systems (HCAHPS) survey in response to criticisms of the Press Ganey survey. The HCAHPS survey consists of 27 questions with 3 broad goals19:

  • to produce data about patients’ perspectives of care that allow for objective and meaningful comparisons of hospitals
  •  to publicly report survey results and create new incentives for hospitals to improve quality of care
  •  to produce public reports that enhance accountability by increasing transparency.

One difference with the HCAHPS is that it measures frequency, or how often a service was performed (“never”, “sometimes”, “usually”, “always”), whereas Press Ganey measures satisfaction. It also only surveys inpatients and does not address outpatient encounters. Despite the differences, it is a widely used patient satisfaction survey and is subject to similar issues and biases as the Press Ganey survey.

Continue to: Gender, race, and age bias...

 

 

Gender, race, and age bias

Although the rationale behind gathering patient input is important, recent data suggest that patient satisfaction surveys are subject to inherent biases.6,13,14 These biases tend to negatively impact women and non-White physicians, adding to the systemic discrimination against women and physicians of color that already exists in health care.

In a single-site retrospective study performed in 2018 by Rogo-Gupta et al, female gynecologists were found to be 47% less likely to receive top patient satisfaction scores than their male counterparts owing to their gender alone, suggesting that gender bias may impact the results of patient satisfaction questionnaires.13 The authors encouraged that the results of patient satisfaction surveys be interpreted with great caution until the impact on female physicians is better understood.

A multi-center study by the same group (Rogo-Gupta et al) assessed the same construct across 5 different geographically diverse institutions.15 This study confirmed that female gynecologists were less likely to receive a top satisfaction score from their patients (19% lower odds when compared with male gynecologists). They also studied the effects of other patient demographics, including age, race/ethnicity, and race concordance. Older patients (aged ≥63 years) had an over-3-fold increase in odds of providing a top satisfaction score than younger patients. Additionally, Asian physicians had significantly lower odds of receiving a top satisfaction score when compared with White physicians, while Asian patients had significantly lower odds of providing a top satisfaction score when compared with White patients. Lastly, in most cases, when underrepresented-in-medicine patients saw an underrepresented-in-medicine physician (race concordance), there was a significant increase in odds of receiving a top satisfaction score. Asian race concordance, however, actually resulted in a lower likelihood of receiving a top satisfaction score.15

Literature from other specialties supports these findings. These results are consistent with emerging data from other medical specialties that also suggest that Press Ganey survey data are subject to inherent biases. For example, data from emergency medicine literature have shown discrepancies between patient satisfaction for providers at tertiary inner-city institutions versus those in affluent suburban populations,16 and that worse mortality is actually correlated with better patient satisfaction scores, and vice versa.17

Another study by Sotto-Santiago in 2019 assessed patient satisfaction scores in multiple specialties at a single institution where quality-related financial incentives were offered based on this metric. They found a significant difference in patient satisfaction scores between underrepresented and White physicians, which suggests a potential bias among patients and institutional practices—ultimately leading to pay inequities through differences in financial incentives.18

Percentile differences reveal small gaps in satisfaction ratings

When examining the difference between raw Press Ganey patient satisfaction data and the percentiles associated with these scores, an interesting finding arises. Looking at the 2023 multicenter study by Rogo-Gupta et al, the difference in the top raw scores between male and female gynecologists appears to be small (3.3%).15 However, in 2020, the difference in top scores separating the top (75th) and bottom (25th) percentile quartiles of physicians was also small, at only 6.9%.

Considering the percentiles, if a provider who scores in the 25th percentile is compared with a colleague who scores in the 75th percentile, they may think the reported satisfaction score differences were quite large. This may potentially invoke feelings of decreased self-worth, negatively impact their professional identity or overall well-being, and they may seek (or be told to seek) improvement opportunities. Now imagine the provider in question realizes the difference between the 25th percentile and 75th percentile is actually only 6.9%. This information may completely change how the results are interpreted and acted upon by administrators. This is further changed with the understanding that 3.3% of the difference may be due to gender alone, narrowing the gap even further. Providers would become understandably frustrated if measures of success such as reimbursement, financial bonus or incentives, promotion, or advancement are linked to these results. It violates the value of fairness and does not offer an equitable starting point.

Evolution of the data distribution. Another consideration, as noted by Robert C. Lloyd, PhD, one of the statisticians who helped develop the percentile statistical analysis mapping in 1985, is that it was based on a classic bell-shaped distribution of patient satisfaction survey scores.19 Because hospitals, medical groups, and physicians have been working these past 20 years to achieve higher Press Ganey scores, the data no longer have a bell-shaped distribution. Rather, there are significant clusters of raw scores at the high end with a very narrow response range. When these data are mapped to the percentile spectrum, they are highly inaccurate.19

Impact of sample size. According to Press Ganey, a minimum of 30 survey responses collected over the designated time period is necessary to draw meaningful conclusions of the data for a specific individual, program, or hospital. Despite this requirement to achieve statistical significance, Sullivan and DeLucia found that the firm often provides comparative data about hospital departments and individual physicians based on a smaller sample size that may create an unacceptably large margin of error.20 Sullivan, for example, said his department may only have 8 to 10 Press Ganey survey responses per month and yet still receives monthly reports from the company analyzing the data. Because of the small sample size, 1 month his department ranked in the 1st percentile and 2 months later it ranked in the 99th percentile.20

The effect of a high ceiling rate. A psychometrics report for the Press Ganey survey is available from the vendor that provides vague assessments of reliability and validity based on 2,762 surveys from 12 practices across 10 states. This report describes a 12-question version of the survey with “no problems encountered” with missingness and response variability. The report further states that the Press Ganey survey demonstrates construct, convergent, divergent, and predictive validities, and high reliability; however, these data are not made available.1

In response to this report, Presson et al analyzed more than 34,000 surveys from one institution to evaluate the reliability and validity of the Press Ganey survey.21 Overall, the survey demonstrated suitable psychometric properties for most metrics. However, Presson et al noted a significantly high ceiling rate of 29.3% for the total score, which ranged from 55.4% to 84.1% across items.21 (Ceiling rates are considered substantial if they occur more than 20% of the time.) Ultimately, a high ceiling rate reduces the power to discriminate between patients who have high satisfaction (everyone is “happy”) with those who are just slightly less than happy, but not dissatisfied. This data quality metric can impact the reliability and validity of a survey—and substantial ceiling rates can notably impact percentile rankings of scores within an institution, offering a possible explanation for the small percentage change between the top and bottom percentiles.

Continue to: Other issues with surveys...

 

 

Other issues with surveys

In addition to the limitations associated with percentile groupings, survey data are always subject to nonresponse bias, and small sample size can lead to nonsignificant results. Skewed responses also can make it difficult to identify true outlying providers who may need remediation or may be offering a superior patient experience. Satisfaction surveys also lack an assessment of objective data and instead assess how patients perceive and feel, which introduces subjectivity to the results.

Additionally, focusing on improving patient experience ratings can incentivize unnecessary or inappropriate care (ie, overprescribing of narcotics, prescribing antibiotics when not indicated, or ordering testing that may not change management). Some physicians even state that they are not getting the type of feedback that they are asking for and that the survey is not asking the right questions to elicit patient input that is meaningful to their practice. Lastly, the incorporation of trainees and advanced practice providers in the patient care experience leads to the assessment of an alternative provider being included in the ultimate score and may not be representative of that physician.

Patients’ perception and survey results. In some circumstances, the patient’s understanding of their medical situation may affect their responses. Some may argue that patients may mistake a physician’s confidence for competence, when in reality these two entities are mutually exclusive. In a randomized controlled trial, researchers from Mount Sinai School of Medicine and Columbia University Medical Center surveyed inner-city women with newly diagnosed and surgically treated early-stage breast cancer for their perceived quality of care and the process of getting care, including referrals, test results, and treatments. They compared the responses with patient records to determine the actual quality of care. Of the 374 women who received treatment for early-stage breast cancer, 55% said they received “excellent care,” but most—88%—actually got care that was in line with the best current treatment guidelines. Interestingly, the study found African American women were less likely to report excellent care than White or Hispanic women, less likely to trust their doctor, and more likely to say they experienced bias during the process. However, there was no difference in actual quality of care received in any group.22

You can’t improve what you can’t control. Ultimately, while many providers think patient satisfaction survey results may help inform some aspects of their practice, they cannot improve what they cannot control. For example, the multicenter study by Rogo-Gupta et al found that older patients (≥63 years) have more than a 3-fold increase in odds of giving a top satisfaction score than younger patients (≤33 years), independent of other aspects of the care experience.15 Additionally, they found that older physicians (≥56 years) had a significant increase in odds of receiving a top satisfaction score when compared with physicians who were younger than 55 years old.15 Given that physicians clearly cannot control their own age or the age of their patients, the negative impacts of these biases need to be addressed and remedied at a systems level.

Why might these biases exist?

While we cannot completely understand all of the possible explanations for these biases, it is important to emphasize the long-standing prejudice and discrimination against women and people of color in our society and how this has impacted our behavior. While strides have been made, there clearly still seems to be a difference between what we say and how our biases impact our behavior. Women are still tougher on women in professional evaluations in other fields as well23; it is not unique to medicine. While workplace improvements are slowly changing, women still face inequities. The more research we publish to describe it, the more we hope the conversation continues, allowing us to reduce the impact of bias on our sense of self-worth and identity related to our careers, narrow the pay gap, and see women advance at the same rate as male counterparts. Considerable transformation is crucial to prevent further workforce attrition.

With regard to the lower scores provided by Asian patients, studies suggest that cultural response bias, rather than true differences in quality of care, may account for these discrepancies. Previous literature has found that Asian patients are more likely to select midpoints, rather than extremes, when completing Likert-type studies24 and are not more likely to change medical providers than other race/ethnicities, indicating that lower ratings may not necessarily imply greater dissatisfaction with care.25

Far-reaching effects on finances, income, well-being, job satisfaction, etc.

Depending on how the results are distributed and used, the effects of patient satisfaction surveys can extend well beyond the original intentions. At some institutions, income for physicians is directly tied to their Press Ganey satisfaction scores, which could have profound implications for female and Asian physicians,13,15 who would be paid less—resulting in a wider pay gap than already exists.18

When negative and not constructive, patient evaluations can contribute to physician burnout and a loss of productive members of the workforce.26 This is especially important in obstetrics and gynecology, where physicians are most likely to experience burnout due to multiple factors such as high-risk medical conditions, pressures of the electronic medical record (EMR), the medicolegal environment, and difficulty balancing patient expectations for autonomy with professional judgement.27 Burnout also disproportionately affects women and younger physicians, which is especially concerning given that, in 2017, approximately one-third of practicing obstetrician/gynecologists were women, while that same year more than 80% of trainees matching into the field were women.28 In one survey sent to members of a prominent medical society, 20% of the medical professionals who responded said they have had their employment threatened by low patient satisfaction scores, 78% reported that patient satisfaction surveys moderately or severely affected their job satisfaction, and 28% stated they had considered quitting their job or leaving the medical profession.29Another related effect is the association between malpractice proceedings and a lack of satisfaction with perceived quality of physician-patient communication.30 This may be an important determinant of malpractice lawsuits, and ensuring high patient satisfaction may be a form of defensive medicine.

Continue to: Controlling the narrative for the future: Proposed strategies...

 

 

Controlling the narrative for the future: Proposed strategies

The rapid, widespread adoption of the Press Ganey survey across specialties, clinical care settings, and geographic areas may have been largely due to the ease and operational benefits for hospitals rather than after rigorous study and validation. For example, repeated use of a specific measurement tool may facilitate comparison across areas within a hospital but also across institutions, which can help assess performance at a national level. Hospitals also may have a financial incentive to work with a single third-party or vendor instead of using multiple options across multiple vendors. However, the impact of adoption of novel measures of performance should be evaluated prior to widespread adoption and utilization.

A similar example of an emergence of a technological advancement that has changed the field of medicine and how we provide care is the EMR. Epic is now the most commonly used medical record system and holds the market share of the industry, covering 78% of patients in the United States.31 While there are certainly many potential benefits of a common EMR, such as ease of information sharing and standardization of formatting, opportunities are identified in real time and require product adjustment. For example, modifications have been made to accurately represent gender outside of the previously used dichotomous options. Diagnoses such as cervical cancer screening can now be used even if the patient gender is listed as male.

Similarly, the Press Ganey and other patient satisfaction questionnaires should be evaluated and modified to address existing societal biases. The World Health Organization estimates that it will take 300 years to fix gender inequality,32 but we have an opportunity now to control the narrative and improve patient feedback.

Future research avenues

Ultimately, there is a need to further explore currently available methods of evaluating clinical encounters to better understand the inherent biases and limitations. We hope this review will encourage other physicians to examine their specialties and hospitals and require similar analyses from vendors of such satisfaction rating products prior to using them. At the very least, health systems should be willing to partner with vendors and physicians on an ongoing basis to better understand the biases involved in these survey results and make modifications as needed. Patients also obtain information from and contribute to self-reported, publicly available websites; therefore, additional exploration into a nationalized standard for assessing patient satisfaction also may serve as an opportunity to standardize the information patients evaluate.33 Further assessment of the potential financial and emotional impact of using the currently available patient-reported surveys on female physicians, Asian physicians, young physicians, and physicians who see young patients is needed. It is time to put pressure on a broken patient satisfaction system and improve on a national level to avoid undue negative consequences on our physicians. Use of patient satisfaction survey data should be limited until we all understand and account for the biases that are evident. ●

Proposed strategies to address bias in patient satisfaction surveys
  • Appeal to the Press Ganey corporation with the results of recent data and other studies to ensure they are aware of the biases that exist in their product
  • Appeal to hospital-level administration to refrain from using Press Ganey scores as a tool to dictate reimbursement; instead rely on other more objective measures of performance (such as publications, presentations, research accomplishments, patient and surgical outcomes, promotion, committees, national leadership roles, etc)
  • Apply a “corrective factor” or “adjustment factor” to eliminate the baseline discrepancy between scores for men and women
  • Consider moving to an alternative survey methodology
  • Provide patient education to define “performance” (ie, frame what a patient can expect from a provider such as being on time, courteous, and empathetic; caution against asking patients to assess competence and knowledge)
References
  1. Outpatient Services (OU) Survey Psychometrics Report. Published online 2019.
  2. Zusman EE. HCAHPS replaces Press Ganey Survey as quality  measure for patient hospital experience. Neurosurgery. 2012;71:N21-N24. doi: 10.1227/01.neu.0000417536.07871.ed
  3. Press Ganey. Company. Accessed April 20, 2023. www.pressganey. com/company/
  4.  Press, Ganey--first year of patient satisfaction measurement. Hosp Guest Relations Rep. 1986;1:4-5.
  5. DeCastellarnau A. A classification of response scale characteristics that affect data quality: a literature review. Qual Quant. 2018;52:15231559. doi: 10.1007/s11135-017-0533-4
  6. Tyser AR, Abtahi AM, McFadden M, et al. Evidence of non-response bias in the Press-Ganey patient satisfaction survey. BMC Health Serv Res. 2016;16:350. doi: 10.1186/s12913-016-1595-z
  7. Duseja R, Durham M, Schreiber M. CMS quality measure development. JAMA. 2020;324:1213-1214. doi: 10.1001/jama.2020.12070
  8. Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. National Academies Press; 2001. doi: 10.17226/10027
  9. Parmet WE. Health: policy or law? A population-based analysis of the Supreme Court’s ACA cases. J Health Polit Policy Law. 2016;41:10611081. doi: 10.1215/03616878-3665949
  10. Richter JP, Muhlestein DB. Patient experience and hospital profitability: is there a link? Health Care Manage Rev. 2017;42:247-257. doi: 10.1097/HMR.0000000000000105
  11. Huang C-H, Wu H-H, Lee Y-C, et al. What role does patient gratitude play in the relationship between relationship quality and patient loyalty? Inquiry. 2019;56:46958019868324. doi: 10.1177/0046958019868324
  12. Centers for Medicare & Medicaid Services (CMS), HHS. Medicare program; hospital inpatient value-based purchasing program. Final rule. Fed Regist. 2011;76:26490-26547.
  13. Rogo-Gupta LJ, Haunschild C, Altamirano J, et al. Physician gender is associated with Press Ganey patient satisfaction scores in outpatient gynecology. Womens Health Issues. 2018;28:281-285. doi: 10.1016 /j.whi.2018.01.001
  14. DeLoughery EP. Physician race and specialty influence Press Ganey survey results. Neth J Med. 2019;77:366-369.
  15. Homewood L, Altamirano J, Fassiotto M, et al. Women gynecologists receive lower Press Ganey patient satisfaction scores in a multicenter cross-sectional study. Am J Obstet Gynecol. 2023;228:S801. doi: 10.1016/j.ajog.2022.12.025
  16. Sharp B, Johnson J, Hamedani AG, et al. What are we measuring? Evaluating physician-specific satisfaction scores between emergency departments. West J Emerg Med. 2019;20:454-459. doi: 10.5811 /westjem.2019.4.41040
  17. Mosley M. Viewpoint: Press Ganey is a worthless tool for the ED. Emerg Med News. 2019;41:3-4. doi: 10.1097/01.EEM.0000616512.68475.69
  18. Sotto-Santiago S, Slaven JE, Rohr-Kirchgraber T. (Dis)Incentivizing patient satisfaction metrics: the unintended consequences of institutional bias. Health Equity. 2019;3:13-18. doi: 10.1089/heq.2018.0065
  19. Lloyd RC. Quality Health Care: A Guide to Developing and Using Indicators. 2nd ed. Jones & Bartlett Learning; 2019. Accessed April 23, 2023. www.jblearning.com/catalog/productdetails /9781284023077
  20. 2+2=7? Seven things you may not know about Press Ganey statistics. Emergency Physicians Monthly. Accessed April 23, 2023. epmonthly. com/article/227-seven-things-you-may-not-know-about-pressgainey-statistics/
  21. Presson AP, Zhang C, Abtahi AM, et al. Psychometric properties of the Press Ganey® Outpatient Medical Practice Survey. Health Qual Life Outcomes. 2017;15:32. doi: 10.1186/s12955-017-0610-3
  22. Bickell NA, Neuman J, Fei K, et al. Quality of breast cancer care: perception versus practice. J Clin Oncol. 2012;30:1791-1795. doi: 10.1200 /JCO.2011.38.7605
  23. Strauss K. Women in the workplace: are women tougher on other women? Forbes. July 18, 2016. Accessed April 27, 2023. www.forbes. com/sites/karstenstrauss/2016/07/18/women-in-the-workplace -are-women-tougher-on-other-women/
  24. Lee JW, Jones PS, Mineyama Y, et al. Cultural differences in responses to a Likert scale. Res Nurs Health. 2002;25:295-306. doi: 10.1002 /nur.10041
  25. Saha S, Hickam DH. Explaining low ratings of patient satisfaction among Asian-Americans. Am J Med Qual. 2003;18:256-264. doi: 10.1177/106286060301800606
  26. Halbesleben JRB, Rathert C. Linking physician burnout and patient outcomes: exploring the dyadic relationship between physicians and patients. Health Care Manage Rev. 2008;33:29-39. doi: 10.1097/01. HMR.0000304493.87898.72
  27. Bradford L, Glaser G. Addressing physician burnout and ensuring high-quality care of the physician workforce. Obstet Gynecol. 2021;137:3-11. doi: 10.1097/AOG.0000000000004197
  28. Boyle P. Nation’s physician workforce evolves: more women, a bit older, and toward different specialties. AAMCNEWS. February 2, 2021. Accessed April 20, 2023. www.aamc.org/news-insights/nations-physician-workforce-evolves-more-women-bit-older-and-towarddifferent-specialties
  29. Zgierska A, Rabago D, Miller MM. Impact of patient satisfaction ratings on physicians and clinical care. Patient Prefer Adherence. 2014;8:437-446. doi: 10.2147/PPA.S59077
  30. Yeh J, Nagel EE. Patient satisfaction in obstetrics and gynecology: individualized patient-centered communication. Clin Med Insights  Womens Health. 2010;3:23. doi: 10.4137/CMWH.S5870
  31. Epic. About us. Accessed April 19, 2023. www.epic.com/about
  32. United Nations. Without investment, gender equality will take nearly 300 years: UN report. September 7, 2022. Accessed April 19, 2023. news.un.org/en/story/2022/09/1126171
  33. Ryan T, Specht J, Smith S, et al. Does the Press Ganey Survey correlate to online health grades for a major academic otolaryngology department? Otolaryngol Head Neck Surg. 2016;155:411-415. doi: 10.1177/0194599816652386
References
  1. Outpatient Services (OU) Survey Psychometrics Report. Published online 2019.
  2. Zusman EE. HCAHPS replaces Press Ganey Survey as quality  measure for patient hospital experience. Neurosurgery. 2012;71:N21-N24. doi: 10.1227/01.neu.0000417536.07871.ed
  3. Press Ganey. Company. Accessed April 20, 2023. www.pressganey. com/company/
  4.  Press, Ganey--first year of patient satisfaction measurement. Hosp Guest Relations Rep. 1986;1:4-5.
  5. DeCastellarnau A. A classification of response scale characteristics that affect data quality: a literature review. Qual Quant. 2018;52:15231559. doi: 10.1007/s11135-017-0533-4
  6. Tyser AR, Abtahi AM, McFadden M, et al. Evidence of non-response bias in the Press-Ganey patient satisfaction survey. BMC Health Serv Res. 2016;16:350. doi: 10.1186/s12913-016-1595-z
  7. Duseja R, Durham M, Schreiber M. CMS quality measure development. JAMA. 2020;324:1213-1214. doi: 10.1001/jama.2020.12070
  8. Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the Quality Chasm: A New Health System for the 21st Century. National Academies Press; 2001. doi: 10.17226/10027
  9. Parmet WE. Health: policy or law? A population-based analysis of the Supreme Court’s ACA cases. J Health Polit Policy Law. 2016;41:10611081. doi: 10.1215/03616878-3665949
  10. Richter JP, Muhlestein DB. Patient experience and hospital profitability: is there a link? Health Care Manage Rev. 2017;42:247-257. doi: 10.1097/HMR.0000000000000105
  11. Huang C-H, Wu H-H, Lee Y-C, et al. What role does patient gratitude play in the relationship between relationship quality and patient loyalty? Inquiry. 2019;56:46958019868324. doi: 10.1177/0046958019868324
  12. Centers for Medicare & Medicaid Services (CMS), HHS. Medicare program; hospital inpatient value-based purchasing program. Final rule. Fed Regist. 2011;76:26490-26547.
  13. Rogo-Gupta LJ, Haunschild C, Altamirano J, et al. Physician gender is associated with Press Ganey patient satisfaction scores in outpatient gynecology. Womens Health Issues. 2018;28:281-285. doi: 10.1016 /j.whi.2018.01.001
  14. DeLoughery EP. Physician race and specialty influence Press Ganey survey results. Neth J Med. 2019;77:366-369.
  15. Homewood L, Altamirano J, Fassiotto M, et al. Women gynecologists receive lower Press Ganey patient satisfaction scores in a multicenter cross-sectional study. Am J Obstet Gynecol. 2023;228:S801. doi: 10.1016/j.ajog.2022.12.025
  16. Sharp B, Johnson J, Hamedani AG, et al. What are we measuring? Evaluating physician-specific satisfaction scores between emergency departments. West J Emerg Med. 2019;20:454-459. doi: 10.5811 /westjem.2019.4.41040
  17. Mosley M. Viewpoint: Press Ganey is a worthless tool for the ED. Emerg Med News. 2019;41:3-4. doi: 10.1097/01.EEM.0000616512.68475.69
  18. Sotto-Santiago S, Slaven JE, Rohr-Kirchgraber T. (Dis)Incentivizing patient satisfaction metrics: the unintended consequences of institutional bias. Health Equity. 2019;3:13-18. doi: 10.1089/heq.2018.0065
  19. Lloyd RC. Quality Health Care: A Guide to Developing and Using Indicators. 2nd ed. Jones & Bartlett Learning; 2019. Accessed April 23, 2023. www.jblearning.com/catalog/productdetails /9781284023077
  20. 2+2=7? Seven things you may not know about Press Ganey statistics. Emergency Physicians Monthly. Accessed April 23, 2023. epmonthly. com/article/227-seven-things-you-may-not-know-about-pressgainey-statistics/
  21. Presson AP, Zhang C, Abtahi AM, et al. Psychometric properties of the Press Ganey® Outpatient Medical Practice Survey. Health Qual Life Outcomes. 2017;15:32. doi: 10.1186/s12955-017-0610-3
  22. Bickell NA, Neuman J, Fei K, et al. Quality of breast cancer care: perception versus practice. J Clin Oncol. 2012;30:1791-1795. doi: 10.1200 /JCO.2011.38.7605
  23. Strauss K. Women in the workplace: are women tougher on other women? Forbes. July 18, 2016. Accessed April 27, 2023. www.forbes. com/sites/karstenstrauss/2016/07/18/women-in-the-workplace -are-women-tougher-on-other-women/
  24. Lee JW, Jones PS, Mineyama Y, et al. Cultural differences in responses to a Likert scale. Res Nurs Health. 2002;25:295-306. doi: 10.1002 /nur.10041
  25. Saha S, Hickam DH. Explaining low ratings of patient satisfaction among Asian-Americans. Am J Med Qual. 2003;18:256-264. doi: 10.1177/106286060301800606
  26. Halbesleben JRB, Rathert C. Linking physician burnout and patient outcomes: exploring the dyadic relationship between physicians and patients. Health Care Manage Rev. 2008;33:29-39. doi: 10.1097/01. HMR.0000304493.87898.72
  27. Bradford L, Glaser G. Addressing physician burnout and ensuring high-quality care of the physician workforce. Obstet Gynecol. 2021;137:3-11. doi: 10.1097/AOG.0000000000004197
  28. Boyle P. Nation’s physician workforce evolves: more women, a bit older, and toward different specialties. AAMCNEWS. February 2, 2021. Accessed April 20, 2023. www.aamc.org/news-insights/nations-physician-workforce-evolves-more-women-bit-older-and-towarddifferent-specialties
  29. Zgierska A, Rabago D, Miller MM. Impact of patient satisfaction ratings on physicians and clinical care. Patient Prefer Adherence. 2014;8:437-446. doi: 10.2147/PPA.S59077
  30. Yeh J, Nagel EE. Patient satisfaction in obstetrics and gynecology: individualized patient-centered communication. Clin Med Insights  Womens Health. 2010;3:23. doi: 10.4137/CMWH.S5870
  31. Epic. About us. Accessed April 19, 2023. www.epic.com/about
  32. United Nations. Without investment, gender equality will take nearly 300 years: UN report. September 7, 2022. Accessed April 19, 2023. news.un.org/en/story/2022/09/1126171
  33. Ryan T, Specht J, Smith S, et al. Does the Press Ganey Survey correlate to online health grades for a major academic otolaryngology department? Otolaryngol Head Neck Surg. 2016;155:411-415. doi: 10.1177/0194599816652386
Issue
OBG Management - 35(7)
Issue
OBG Management - 35(7)
Page Number
SS13-SS19
Page Number
SS13-SS19
Publications
MDedge ObGyn
OBG Management
Publications
MDedge ObGyn
OBG Management
Topics
Practice Management
Article Type
Article
Sections
Clinical Review
Disallow All Ads
Content Gating
No Gating (article Unlocked/Free)
Alternative CME
Disqus Comments
Default
Use ProPublica
Hide sidebar & use full width
render the right sidebar.
Conference Recap Checkbox
Not Conference Recap
Clinical Edge
Display the Slideshow in this Article
Medscape Article
Display survey writer
Reuters content
Disable Inline Native ads
WebMD Article
Teaser Media
Article PDF Media
Subscribe to Deirdre A. Lum, MD