ACS NSQIP hospitals steadily improve surgical outcomes

Surgical performance data gathered within the American College of Surgeons, National Surgical Quality Improvement Program (ACS NSQIP) from the period of 2006-2013 indicate improved performance by most participating hospitals over time. In the categories of mortality, morbidity, and surgical site infections (SSI), the ratio of observed to expected surgery-related adverse events declined over time, showing that with increasing years of participation in ACS NSQIP, hospitals saw cumulative reductions in observed/expected ratios.

Dr. Clifford Y. Ko, coauthor of the report, an ACS Fellow and professor of surgery at David Geffen School of Medicine, University of California Los Angeles, spoke in an interview about the results, which were published in Annals of Surgery.

“An important finding from our study is that hospitals get better when they participate in ACS NSQIP, and the longer they are in the program, the better they get. It means that if hospitals commit to going on the journey of quality improvement, the longer they are on that journey, the better they get, the more experience they get in quality improvement, and the more their outcomes improve.

“Another important point that became evident from the analysis was that the biggest area for improvement is in the rate of complications, more so than mortality. The mortality rate is already low, and the number of preventable deaths is low. There are many more preventable complications than preventable deaths. The data show that risk-adjusted complications decrease over time because many opportunities lie in decreasing rates of infection, blood clot, pneumonia, urinary tract infection, and others. Whatever rate the hospital starts in ACS NSQIP, there is more opportunity to realize improvements in the number of complications than in mortality.”

The researchers used clinical data collected from 2006 to 2013 from 515 hospitals participating in ACS NSQIP.

Hospitals participating in the program receive semiannual reports that benchmark performance compared to other similar institutions. But the performance benchmarks are continuously updated to reflect current practices across groups of similar institutions, so individual hospitals cannot gauge whether their performance improves over time relative to fixed standard.

“The aim of the study was to determine if over several years hospitals who participated in ACS NSQIP got better. Benchmarking data provides feedback, but does not indicate whether over time hospitals improve. Our analysis looked at hospital performance longitudinally to evaluate if they got better from year to year,” said Dr. Ko.

To make meaningful comparisons of surgical outcomes, expected outcomes were calculated based on risk adjustments to compensate for differences in patient characteristics and the risk profile of procedures performed. Dr. Ko explained, “Large, tertiary institutions that care for very sick patients, or very complex cases, are expected to have worse outcomes than hospitals that perform more common procedures with lower expected event rates. In this analysis, the results of interest are the trends in observed/expected (O/E) event rates, which adjust for these risks.”

For all hospitals, 62%, 70%, and 65% had negative slopes for mortality, morbidity, and any SSI, respectively. The O/E ratios for mortality, morbidity, and SSI declined with each year of participation in the program, and the slope of the decline was steepest for morbidity and SSI. SSI is the most common complication and largely mirrors morbidity. For hospitals in the program at least 3 years, annual reductions in mortality, morbidity, and SSI were estimated to be 0.8%, 3.1%, and 2.6%, respectively.

One of the hallmarks of ACS NSQIP is the accuracy of the data, which come directly from medical records. In a large study comparing clinical with billing (or administrative) data, Dr. Ko and colleagues found the accuracy of billing data to be low.

“In a 2-year study we merged clinical and administrative data from over 100,000 patients and found the agreement to be poor. Billing data had a detection rate for SSI of 25% and a false-positive rate of 70%,” he said.

“In developing programs to improve patient care, we want to base our work on the most accurate data possible. Our analysis shows that hospitals committed to using ACS NSQIP improve over time.”

The authors reported having no financial disclosures.

Surgical performance data gathered within the American College of Surgeons, National Surgical Quality Improvement Program (ACS NSQIP) from the period of 2006-2013 indicate improved performance by most participating hospitals over time. In the categories of mortality, morbidity, and surgical site infections (SSI), the ratio of observed to expected surgery-related adverse events declined over time, showing that with increasing years of participation in ACS NSQIP, hospitals saw cumulative reductions in observed/expected ratios.

Dr. Clifford Y. Ko, coauthor of the report, an ACS Fellow and professor of surgery at David Geffen School of Medicine, University of California Los Angeles, spoke in an interview about the results, which were published in Annals of Surgery.

“An important finding from our study is that hospitals get better when they participate in ACS NSQIP, and the longer they are in the program, the better they get. It means that if hospitals commit to going on the journey of quality improvement, the longer they are on that journey, the better they get, the more experience they get in quality improvement, and the more their outcomes improve.

“Another important point that became evident from the analysis was that the biggest area for improvement is in the rate of complications, more so than mortality. The mortality rate is already low, and the number of preventable deaths is low. There are many more preventable complications than preventable deaths. The data show that risk-adjusted complications decrease over time because many opportunities lie in decreasing rates of infection, blood clot, pneumonia, urinary tract infection, and others. Whatever rate the hospital starts in ACS NSQIP, there is more opportunity to realize improvements in the number of complications than in mortality.”

The researchers used clinical data collected from 2006 to 2013 from 515 hospitals participating in ACS NSQIP.

Hospitals participating in the program receive semiannual reports that benchmark performance compared to other similar institutions. But the performance benchmarks are continuously updated to reflect current practices across groups of similar institutions, so individual hospitals cannot gauge whether their performance improves over time relative to fixed standard.

“The aim of the study was to determine if over several years hospitals who participated in ACS NSQIP got better. Benchmarking data provides feedback, but does not indicate whether over time hospitals improve. Our analysis looked at hospital performance longitudinally to evaluate if they got better from year to year,” said Dr. Ko.

To make meaningful comparisons of surgical outcomes, expected outcomes were calculated based on risk adjustments to compensate for differences in patient characteristics and the risk profile of procedures performed. Dr. Ko explained, “Large, tertiary institutions that care for very sick patients, or very complex cases, are expected to have worse outcomes than hospitals that perform more common procedures with lower expected event rates. In this analysis, the results of interest are the trends in observed/expected (O/E) event rates, which adjust for these risks.”

For all hospitals, 62%, 70%, and 65% had negative slopes for mortality, morbidity, and any SSI, respectively. The O/E ratios for mortality, morbidity, and SSI declined with each year of participation in the program, and the slope of the decline was steepest for morbidity and SSI. SSI is the most common complication and largely mirrors morbidity. For hospitals in the program at least 3 years, annual reductions in mortality, morbidity, and SSI were estimated to be 0.8%, 3.1%, and 2.6%, respectively.

One of the hallmarks of ACS NSQIP is the accuracy of the data, which come directly from medical records. In a large study comparing clinical with billing (or administrative) data, Dr. Ko and colleagues found the accuracy of billing data to be low.

“In a 2-year study we merged clinical and administrative data from over 100,000 patients and found the agreement to be poor. Billing data had a detection rate for SSI of 25% and a false-positive rate of 70%,” he said.

“In developing programs to improve patient care, we want to base our work on the most accurate data possible. Our analysis shows that hospitals committed to using ACS NSQIP improve over time.”

The authors reported having no financial disclosures.

Surgical performance data gathered within the American College of Surgeons, National Surgical Quality Improvement Program (ACS NSQIP) from the period of 2006-2013 indicate improved performance by most participating hospitals over time. In the categories of mortality, morbidity, and surgical site infections (SSI), the ratio of observed to expected surgery-related adverse events declined over time, showing that with increasing years of participation in ACS NSQIP, hospitals saw cumulative reductions in observed/expected ratios.

Dr. Clifford Y. Ko, coauthor of the report, an ACS Fellow and professor of surgery at David Geffen School of Medicine, University of California Los Angeles, spoke in an interview about the results, which were published in Annals of Surgery.

“An important finding from our study is that hospitals get better when they participate in ACS NSQIP, and the longer they are in the program, the better they get. It means that if hospitals commit to going on the journey of quality improvement, the longer they are on that journey, the better they get, the more experience they get in quality improvement, and the more their outcomes improve.

“Another important point that became evident from the analysis was that the biggest area for improvement is in the rate of complications, more so than mortality. The mortality rate is already low, and the number of preventable deaths is low. There are many more preventable complications than preventable deaths. The data show that risk-adjusted complications decrease over time because many opportunities lie in decreasing rates of infection, blood clot, pneumonia, urinary tract infection, and others. Whatever rate the hospital starts in ACS NSQIP, there is more opportunity to realize improvements in the number of complications than in mortality.”

The researchers used clinical data collected from 2006 to 2013 from 515 hospitals participating in ACS NSQIP.

Hospitals participating in the program receive semiannual reports that benchmark performance compared to other similar institutions. But the performance benchmarks are continuously updated to reflect current practices across groups of similar institutions, so individual hospitals cannot gauge whether their performance improves over time relative to fixed standard.

“The aim of the study was to determine if over several years hospitals who participated in ACS NSQIP got better. Benchmarking data provides feedback, but does not indicate whether over time hospitals improve. Our analysis looked at hospital performance longitudinally to evaluate if they got better from year to year,” said Dr. Ko.

To make meaningful comparisons of surgical outcomes, expected outcomes were calculated based on risk adjustments to compensate for differences in patient characteristics and the risk profile of procedures performed. Dr. Ko explained, “Large, tertiary institutions that care for very sick patients, or very complex cases, are expected to have worse outcomes than hospitals that perform more common procedures with lower expected event rates. In this analysis, the results of interest are the trends in observed/expected (O/E) event rates, which adjust for these risks.”

For all hospitals, 62%, 70%, and 65% had negative slopes for mortality, morbidity, and any SSI, respectively. The O/E ratios for mortality, morbidity, and SSI declined with each year of participation in the program, and the slope of the decline was steepest for morbidity and SSI. SSI is the most common complication and largely mirrors morbidity. For hospitals in the program at least 3 years, annual reductions in mortality, morbidity, and SSI were estimated to be 0.8%, 3.1%, and 2.6%, respectively.

One of the hallmarks of ACS NSQIP is the accuracy of the data, which come directly from medical records. In a large study comparing clinical with billing (or administrative) data, Dr. Ko and colleagues found the accuracy of billing data to be low.

“In a 2-year study we merged clinical and administrative data from over 100,000 patients and found the agreement to be poor. Billing data had a detection rate for SSI of 25% and a false-positive rate of 70%,” he said.

“In developing programs to improve patient care, we want to base our work on the most accurate data possible. Our analysis shows that hospitals committed to using ACS NSQIP improve over time.”

The authors reported having no financial disclosures.