Public reporting needs reform!

Like many of my colleagues, I support President Obama’s call to demonstrate value as part of health care reform. One way to do that is through public reporting. The rationale is that public scrutiny of outcomes will motivate the health care “industry” to improve the “product” (outcomes), rather than accelerating valueless economic activity (process) that often benefits providers more than patients.

Fair enough. But does the existing system of quality indicators support the goals of reform identified by the Institute of Medicine (IOM)?¹ That is, does it make the system safer, more effective and efficient, timely, equitable, and patient-centered?

Not necessarily.

The reason is 2-fold. First, the best quality indicators are patient-oriented outcomes (eg, quality of life, morbidity, mortality), but that’s not what’s being reported. Second, many publicly reported surrogate measures are more harmful than helpful, and in need of serious reform themselves.

My experience

The Wisconsin Collaborative for Healthcare Quality (WCHQ), which I’ve been involved with for nearly 10 years, is composed of health care organizations, mine included, committed to voluntary reporting of quality metrics. The TABLE features a list of the metrics, chosen by the WCHQ, that are reported.

I’ve rated each metric on 2 criteria:

1. How good is the evidence for the screening tool or intervention? (There is good evidence for colorectal cancer screening, for example, but evidence for low-density lipoprotein [LDL] testing is poor.)
2. How good is the quality indicator itself, including the frequency? (There’s good evidence for Pap testing within 3 years, whereas twice-yearly HbA1c testing is opinion-based.)

TABLE 1
How do the “quality indicators” rate?

Some worrisome examples

While the ratings are partly subjective, they’re meant to illustrate that not all publicly reported metrics are supported by good evidence.

This is particularly troubling, given the fact that acting on fair or poor evidence may cause more harm than good. Consider these worrisome examples:

LDL control <100 mg/dL. I’ve known patients who had their first myocardial infarction when their LDL cholesterol was <100 mg/dL. After the event, these patients weren’t given a statin because they were already “at goal”; they subsequently had a reinfarction.

LDL should not be used as a quality indicator in secondary prevention for (at least) 2 reasons: First, some LDL-lowering drugs are harmful or have no net benefit (eg, estrogen in women, fibrates).² Second, statin benefit may or may not be related to lipid lowering, and the magnitude of benefit is not related to any arbitrary LDL goal.³

There is clear, compelling evidence supporting near-universal statin therapy for patients at high cardiovascular risk regardless of their LDL cholesterol values—but a lack of evidence that titrating lipid therapy to achieve proposed low LDL levels is beneficial or safe.⁴ Receiving the maximum tolerated dose of statin is therefore the appropriate evidence-based surrogate quality indicator, not LDL.

Mammography. The Cochrane collaboration has concluded that “for every 2000 women invited for [mammography] screening throughout 10 years, 1 will have her life prolonged,” and 10 healthy women who would not have been diagnosed without the screening will be treated unnecessarily.⁵ The Cochrane review thus concluded that it’s not clear whether mammography screening does more good than harm.⁵

The US Preventive Services Task Force (USPSTF) recently downgraded mammography screening for women over age 50 from an A- to a B-rated recommendation. The fine balance between benefit and harm in this and other USPSTF B-rated preventive measures requires that clinicians educate patients and elicit their preferences. But this doesn’t occur when health plans strive to outdo one another in achieving higher publicly reported screening goals.⁶ Documentation of valid shared decision-making, not screening rates, is the appropriate quality indicator.

The evidence vs the “business” of medicine

I have no illusions that my recommendations will be adopted easily—or soon. After all, we practice in an environment in which evidence-based practice recommendations can conflict with financial and operational goals perceived as necessary to survive. However, I believe that evidence trumps business in achieving the IOM goals.

It remains to be seen whether we can simultaneously move toward valid evidence-based public reporting and health-care financial reform. But one thing is clear: To insist that evidence-based patient-oriented quality indicators are too difficult to measure, or to ignore or deny the evidence, puts the lie to claims of patient-centered care and, ultimately, to long-needed health care reform looming on the horizon.

Like many of my colleagues, I support President Obama’s call to demonstrate value as part of health care reform. One way to do that is through public reporting. The rationale is that public scrutiny of outcomes will motivate the health care “industry” to improve the “product” (outcomes), rather than accelerating valueless economic activity (process) that often benefits providers more than patients.

Fair enough. But does the existing system of quality indicators support the goals of reform identified by the Institute of Medicine (IOM)?¹ That is, does it make the system safer, more effective and efficient, timely, equitable, and patient-centered?

Not necessarily.

The reason is 2-fold. First, the best quality indicators are patient-oriented outcomes (eg, quality of life, morbidity, mortality), but that’s not what’s being reported. Second, many publicly reported surrogate measures are more harmful than helpful, and in need of serious reform themselves.

My experience

The Wisconsin Collaborative for Healthcare Quality (WCHQ), which I’ve been involved with for nearly 10 years, is composed of health care organizations, mine included, committed to voluntary reporting of quality metrics. The TABLE features a list of the metrics, chosen by the WCHQ, that are reported.

I’ve rated each metric on 2 criteria:

1. How good is the evidence for the screening tool or intervention? (There is good evidence for colorectal cancer screening, for example, but evidence for low-density lipoprotein [LDL] testing is poor.)
2. How good is the quality indicator itself, including the frequency? (There’s good evidence for Pap testing within 3 years, whereas twice-yearly HbA1c testing is opinion-based.)

TABLE 1
How do the “quality indicators” rate?

Some worrisome examples

While the ratings are partly subjective, they’re meant to illustrate that not all publicly reported metrics are supported by good evidence.

This is particularly troubling, given the fact that acting on fair or poor evidence may cause more harm than good. Consider these worrisome examples:

LDL control <100 mg/dL. I’ve known patients who had their first myocardial infarction when their LDL cholesterol was <100 mg/dL. After the event, these patients weren’t given a statin because they were already “at goal”; they subsequently had a reinfarction.

LDL should not be used as a quality indicator in secondary prevention for (at least) 2 reasons: First, some LDL-lowering drugs are harmful or have no net benefit (eg, estrogen in women, fibrates).² Second, statin benefit may or may not be related to lipid lowering, and the magnitude of benefit is not related to any arbitrary LDL goal.³

There is clear, compelling evidence supporting near-universal statin therapy for patients at high cardiovascular risk regardless of their LDL cholesterol values—but a lack of evidence that titrating lipid therapy to achieve proposed low LDL levels is beneficial or safe.⁴ Receiving the maximum tolerated dose of statin is therefore the appropriate evidence-based surrogate quality indicator, not LDL.

Mammography. The Cochrane collaboration has concluded that “for every 2000 women invited for [mammography] screening throughout 10 years, 1 will have her life prolonged,” and 10 healthy women who would not have been diagnosed without the screening will be treated unnecessarily.⁵ The Cochrane review thus concluded that it’s not clear whether mammography screening does more good than harm.⁵

The US Preventive Services Task Force (USPSTF) recently downgraded mammography screening for women over age 50 from an A- to a B-rated recommendation. The fine balance between benefit and harm in this and other USPSTF B-rated preventive measures requires that clinicians educate patients and elicit their preferences. But this doesn’t occur when health plans strive to outdo one another in achieving higher publicly reported screening goals.⁶ Documentation of valid shared decision-making, not screening rates, is the appropriate quality indicator.

The evidence vs the “business” of medicine

I have no illusions that my recommendations will be adopted easily—or soon. After all, we practice in an environment in which evidence-based practice recommendations can conflict with financial and operational goals perceived as necessary to survive. However, I believe that evidence trumps business in achieving the IOM goals.

It remains to be seen whether we can simultaneously move toward valid evidence-based public reporting and health-care financial reform. But one thing is clear: To insist that evidence-based patient-oriented quality indicators are too difficult to measure, or to ignore or deny the evidence, puts the lie to claims of patient-centered care and, ultimately, to long-needed health care reform looming on the horizon.

Like many of my colleagues, I support President Obama’s call to demonstrate value as part of health care reform. One way to do that is through public reporting. The rationale is that public scrutiny of outcomes will motivate the health care “industry” to improve the “product” (outcomes), rather than accelerating valueless economic activity (process) that often benefits providers more than patients.

Fair enough. But does the existing system of quality indicators support the goals of reform identified by the Institute of Medicine (IOM)?¹ That is, does it make the system safer, more effective and efficient, timely, equitable, and patient-centered?

Not necessarily.

The reason is 2-fold. First, the best quality indicators are patient-oriented outcomes (eg, quality of life, morbidity, mortality), but that’s not what’s being reported. Second, many publicly reported surrogate measures are more harmful than helpful, and in need of serious reform themselves.

My experience

The Wisconsin Collaborative for Healthcare Quality (WCHQ), which I’ve been involved with for nearly 10 years, is composed of health care organizations, mine included, committed to voluntary reporting of quality metrics. The TABLE features a list of the metrics, chosen by the WCHQ, that are reported.

I’ve rated each metric on 2 criteria:

1. How good is the evidence for the screening tool or intervention? (There is good evidence for colorectal cancer screening, for example, but evidence for low-density lipoprotein [LDL] testing is poor.)
2. How good is the quality indicator itself, including the frequency? (There’s good evidence for Pap testing within 3 years, whereas twice-yearly HbA1c testing is opinion-based.)

TABLE 1
How do the “quality indicators” rate?

Some worrisome examples

While the ratings are partly subjective, they’re meant to illustrate that not all publicly reported metrics are supported by good evidence.

This is particularly troubling, given the fact that acting on fair or poor evidence may cause more harm than good. Consider these worrisome examples:

LDL control <100 mg/dL. I’ve known patients who had their first myocardial infarction when their LDL cholesterol was <100 mg/dL. After the event, these patients weren’t given a statin because they were already “at goal”; they subsequently had a reinfarction.

LDL should not be used as a quality indicator in secondary prevention for (at least) 2 reasons: First, some LDL-lowering drugs are harmful or have no net benefit (eg, estrogen in women, fibrates).² Second, statin benefit may or may not be related to lipid lowering, and the magnitude of benefit is not related to any arbitrary LDL goal.³

There is clear, compelling evidence supporting near-universal statin therapy for patients at high cardiovascular risk regardless of their LDL cholesterol values—but a lack of evidence that titrating lipid therapy to achieve proposed low LDL levels is beneficial or safe.⁴ Receiving the maximum tolerated dose of statin is therefore the appropriate evidence-based surrogate quality indicator, not LDL.

Mammography. The Cochrane collaboration has concluded that “for every 2000 women invited for [mammography] screening throughout 10 years, 1 will have her life prolonged,” and 10 healthy women who would not have been diagnosed without the screening will be treated unnecessarily.⁵ The Cochrane review thus concluded that it’s not clear whether mammography screening does more good than harm.⁵

The US Preventive Services Task Force (USPSTF) recently downgraded mammography screening for women over age 50 from an A- to a B-rated recommendation. The fine balance between benefit and harm in this and other USPSTF B-rated preventive measures requires that clinicians educate patients and elicit their preferences. But this doesn’t occur when health plans strive to outdo one another in achieving higher publicly reported screening goals.⁶ Documentation of valid shared decision-making, not screening rates, is the appropriate quality indicator.

The evidence vs the “business” of medicine

I have no illusions that my recommendations will be adopted easily—or soon. After all, we practice in an environment in which evidence-based practice recommendations can conflict with financial and operational goals perceived as necessary to survive. However, I believe that evidence trumps business in achieving the IOM goals.

It remains to be seen whether we can simultaneously move toward valid evidence-based public reporting and health-care financial reform. But one thing is clear: To insist that evidence-based patient-oriented quality indicators are too difficult to measure, or to ignore or deny the evidence, puts the lie to claims of patient-centered care and, ultimately, to long-needed health care reform looming on the horizon.