SCIP Compliance Does Not Predict Outcomes

Major Finding: Hospitals ranged from 53.7% to 91.4% in their compliance with Centers for Medicare and Medicaid surgical processes of care measures, but a hospital's score didn't correspond to its rates of risk-adjusted mortality, venous thromboembolism, or surgical site infection.

Data Source: Medicare inpatient claims data Jan. 1, 2005, through Dec. 31, 2006, and Surgical Care Improvement Project scores reported on the Hospital Compare Web site.

Disclosures: The study's authors were supported by several federal grants and funds from the Robert Wood Johnson Foundation.

Risk-adjusted patient outcomes don't vary between hospitals, regardless of how well they scored on the measures of quality-process compliance behind Medicare's Hospital Compare Web site, researchers have found.

“Despite the intentions of the CMS [Centers for Medicare and Medicaid Services] to provide patients with information that will facilitate patient choice of high-quality hospitals, currently available information on the Hospital Compare Web site will not help patients identify hospitals with better outcomes for high-risk surgery,” wrote the authors, from the University of Michigan and the Michigan Surgical Collaborative for Outcomes Research and Evaluation, both in Ann Arbor.

The fault may lie in the Surgical Care Improvement Project (SCIP) measures used to generate the Hospital Compare data, which mainly track very rare complications, such as deep venous thrombosis, and less important events, such as superficial surgical site infections, they said.

The study looked at data from 2,000 hospitals on three SCIP outcomes measures: 30-day postoperative mortality, venous thromboembolisms, and surgical site infections (Arch. Surg. 2010;145:999-1004).

The CMS mandates reporting of two sets of SCIP data – one on infection and one on venous thromboembolism – for hospitals to receive annual payment increases. Hospitals submit their data quarterly. The data are then posted at www.hospitalcompare.hhs.gov

They looked at patient outcomes after six high-risk surgical procedures: abdominal aortic aneurysm repair, aortic valve repair, coronary artery bypass graft, esophageal resection, mitral valve repair, and pancreatic resection.

Compliance rates with the SCIP measures ranged considerably from 53.7% to 91.4%, but the study found little evidence of a consistent relationship between a hospital's score and its rates of risk-adjusted mortality, venous thromboembolism, or surgical site infection.

The authors also looked at data on extended lengths of stay, which can result from numerous postoperative complications. Patients at hospitals that most often complied with SCIP were 12% less likely to experience an extended stay relative to middle-compliance hospitals, but there was no difference between the middle- and lowest-compliance hospitals.

The lack of correlation between “process compliance,” as measured by SCIP numbers, will be important as quality measures have increasing impact on reimbursements for care, said the authors. “If there is a weak link between process compliance and surgical outcomes, CMS public reporting and pay-for-performance efforts will be unlikely to stimulate important improvements or to help patients find the safest hospitals,” they wrote.

They advised the CMS to “devote greater attention to profiling hospitals based on outcomes for improved public reporting and pay-for-performance programs.”

Major Finding: Hospitals ranged from 53.7% to 91.4% in their compliance with Centers for Medicare and Medicaid surgical processes of care measures, but a hospital's score didn't correspond to its rates of risk-adjusted mortality, venous thromboembolism, or surgical site infection.

Data Source: Medicare inpatient claims data Jan. 1, 2005, through Dec. 31, 2006, and Surgical Care Improvement Project scores reported on the Hospital Compare Web site.

Disclosures: The study's authors were supported by several federal grants and funds from the Robert Wood Johnson Foundation.

Risk-adjusted patient outcomes don't vary between hospitals, regardless of how well they scored on the measures of quality-process compliance behind Medicare's Hospital Compare Web site, researchers have found.

“Despite the intentions of the CMS [Centers for Medicare and Medicaid Services] to provide patients with information that will facilitate patient choice of high-quality hospitals, currently available information on the Hospital Compare Web site will not help patients identify hospitals with better outcomes for high-risk surgery,” wrote the authors, from the University of Michigan and the Michigan Surgical Collaborative for Outcomes Research and Evaluation, both in Ann Arbor.

The fault may lie in the Surgical Care Improvement Project (SCIP) measures used to generate the Hospital Compare data, which mainly track very rare complications, such as deep venous thrombosis, and less important events, such as superficial surgical site infections, they said.

The study looked at data from 2,000 hospitals on three SCIP outcomes measures: 30-day postoperative mortality, venous thromboembolisms, and surgical site infections (Arch. Surg. 2010;145:999-1004).

The CMS mandates reporting of two sets of SCIP data – one on infection and one on venous thromboembolism – for hospitals to receive annual payment increases. Hospitals submit their data quarterly. The data are then posted at www.hospitalcompare.hhs.gov

They looked at patient outcomes after six high-risk surgical procedures: abdominal aortic aneurysm repair, aortic valve repair, coronary artery bypass graft, esophageal resection, mitral valve repair, and pancreatic resection.

Compliance rates with the SCIP measures ranged considerably from 53.7% to 91.4%, but the study found little evidence of a consistent relationship between a hospital's score and its rates of risk-adjusted mortality, venous thromboembolism, or surgical site infection.

The authors also looked at data on extended lengths of stay, which can result from numerous postoperative complications. Patients at hospitals that most often complied with SCIP were 12% less likely to experience an extended stay relative to middle-compliance hospitals, but there was no difference between the middle- and lowest-compliance hospitals.

The lack of correlation between “process compliance,” as measured by SCIP numbers, will be important as quality measures have increasing impact on reimbursements for care, said the authors. “If there is a weak link between process compliance and surgical outcomes, CMS public reporting and pay-for-performance efforts will be unlikely to stimulate important improvements or to help patients find the safest hospitals,” they wrote.

They advised the CMS to “devote greater attention to profiling hospitals based on outcomes for improved public reporting and pay-for-performance programs.”

Major Finding: Hospitals ranged from 53.7% to 91.4% in their compliance with Centers for Medicare and Medicaid surgical processes of care measures, but a hospital's score didn't correspond to its rates of risk-adjusted mortality, venous thromboembolism, or surgical site infection.

Data Source: Medicare inpatient claims data Jan. 1, 2005, through Dec. 31, 2006, and Surgical Care Improvement Project scores reported on the Hospital Compare Web site.

Disclosures: The study's authors were supported by several federal grants and funds from the Robert Wood Johnson Foundation.

Risk-adjusted patient outcomes don't vary between hospitals, regardless of how well they scored on the measures of quality-process compliance behind Medicare's Hospital Compare Web site, researchers have found.

“Despite the intentions of the CMS [Centers for Medicare and Medicaid Services] to provide patients with information that will facilitate patient choice of high-quality hospitals, currently available information on the Hospital Compare Web site will not help patients identify hospitals with better outcomes for high-risk surgery,” wrote the authors, from the University of Michigan and the Michigan Surgical Collaborative for Outcomes Research and Evaluation, both in Ann Arbor.

The fault may lie in the Surgical Care Improvement Project (SCIP) measures used to generate the Hospital Compare data, which mainly track very rare complications, such as deep venous thrombosis, and less important events, such as superficial surgical site infections, they said.

The study looked at data from 2,000 hospitals on three SCIP outcomes measures: 30-day postoperative mortality, venous thromboembolisms, and surgical site infections (Arch. Surg. 2010;145:999-1004).

The CMS mandates reporting of two sets of SCIP data – one on infection and one on venous thromboembolism – for hospitals to receive annual payment increases. Hospitals submit their data quarterly. The data are then posted at www.hospitalcompare.hhs.gov

They looked at patient outcomes after six high-risk surgical procedures: abdominal aortic aneurysm repair, aortic valve repair, coronary artery bypass graft, esophageal resection, mitral valve repair, and pancreatic resection.

Compliance rates with the SCIP measures ranged considerably from 53.7% to 91.4%, but the study found little evidence of a consistent relationship between a hospital's score and its rates of risk-adjusted mortality, venous thromboembolism, or surgical site infection.

The authors also looked at data on extended lengths of stay, which can result from numerous postoperative complications. Patients at hospitals that most often complied with SCIP were 12% less likely to experience an extended stay relative to middle-compliance hospitals, but there was no difference between the middle- and lowest-compliance hospitals.

The lack of correlation between “process compliance,” as measured by SCIP numbers, will be important as quality measures have increasing impact on reimbursements for care, said the authors. “If there is a weak link between process compliance and surgical outcomes, CMS public reporting and pay-for-performance efforts will be unlikely to stimulate important improvements or to help patients find the safest hospitals,” they wrote.

They advised the CMS to “devote greater attention to profiling hospitals based on outcomes for improved public reporting and pay-for-performance programs.”