Study finds inconsistencies in MBSAQIP database

BALTIMORE – A study that examined the quality of data from the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (MBASQIP) registry found inconsistencies that may undermine the quality of the registry, which could be misleading for clinicians and investigators who used the data, according to an analysis presented at the annual meeting of the Society of American Gastrointestinal and Endoscopic Surgeons. The researchers recommended ways to improve data-gathering techniques to better identify the nature of these inconsistencies.

“Our original enthusiasm about the availability of data from MBASQIP turned into a cautious optimism about potential usefulness of these data,” Katia Noyes, PhD, of the State University of New York at Buffalo, said in presenting the study. She noted her research team’s analysis of 168,093 cases in the 2015 MBASQIP Participant Use Data File found that 20% of the cases (n = 33,868) had missing or unusable information for at least one key variable, such as age, race, ethnicity, body mass index (BMI) before and after surgery, and American Society of Anesthesiologists classification. Specifically, preoperative and postoperative BMI data were missing or zero in 6.7% of cases (n = 11,211).

The researchers developed a single flat file for patient-level outcomes evaluation using five files (main, BMI, readmission, intervention, and reoperation). They used logic and validity tests that included individual profiles of patient BMI changes over time, individual patient care pathways (chronologic record of patient admission, discharge and procedure history), and correlation tests between pairs of variables associated with the same clinical encounters (emergency intervention vs. procedure type; related admission with intervention vs. planned intervention).

“Weight reduction at the first postoperative visit ranged from –71% to a gain of 132% of preoperative weight,” she said. “We also found inconsistency in the sequence of events. Seven percent of readmissions and 12.5% of postoperative interventions were categorized as planned, which is not a problem, but when you look at the reported reasons for planned procedures, they could not all have been possibly planned before discharge.”

Based on 2015 MBASQIP data, “planned” readmissions and postoperative procedures included admissions for nonspecific abdominal pain, band erosion, slippage or prolapse, bleeding, gastrogastric fistula, incisional hernia, infection and/or fever, pneumonia, and wound infection, among other reasons.

“Our analysis found inconsistent quality of data for key parameters, missing and miscoded values and lack of clarity for coding and definitions,” Dr. Noyes said.

The study made four recommendations to improve the quality of data submitted to MASQIP.

• Use health IT applications to provide automated data checks to validate completeness of submitted data – by utilizing a no-skip pattern for core variables – and accuracy of data– by flagging values outside predefined acceptable ranges.
• Perform data audits for consistency, using multiple variables to conduct logic checks, such as by not allowing “readmission” before discharge for the index admission.
• Give data auditors specific recommendations for definitions of registry variables, standardization of algorithms for abstracting values based on commonly used clinical data systems, such as Allscripts and Epic, and standardized use of diagnostic and procedure codes to link with payers’ reimbursement schedules.
• Provide ongoing education to stakeholders such as researchers and hospital administrators on best data management practices and how to best use the data for quality improvement.

Dr. Noyes had no financial relationships to disclose. Coauthor Steven Schwaitzberg, MD, disclosed consulting arrangements with New View Surgical, AcuityBio, Activ Surgical, Human Extensions, Levita Magnetics, and Arch Therapeutics. Aaron Hoffman, MD, disclosed a consulting arrangement with Ethicon.

SOURCE: Noyes K et al. SAGES 2019, Abstract 21

BALTIMORE – A study that examined the quality of data from the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (MBASQIP) registry found inconsistencies that may undermine the quality of the registry, which could be misleading for clinicians and investigators who used the data, according to an analysis presented at the annual meeting of the Society of American Gastrointestinal and Endoscopic Surgeons. The researchers recommended ways to improve data-gathering techniques to better identify the nature of these inconsistencies.

“Our original enthusiasm about the availability of data from MBASQIP turned into a cautious optimism about potential usefulness of these data,” Katia Noyes, PhD, of the State University of New York at Buffalo, said in presenting the study. She noted her research team’s analysis of 168,093 cases in the 2015 MBASQIP Participant Use Data File found that 20% of the cases (n = 33,868) had missing or unusable information for at least one key variable, such as age, race, ethnicity, body mass index (BMI) before and after surgery, and American Society of Anesthesiologists classification. Specifically, preoperative and postoperative BMI data were missing or zero in 6.7% of cases (n = 11,211).

The researchers developed a single flat file for patient-level outcomes evaluation using five files (main, BMI, readmission, intervention, and reoperation). They used logic and validity tests that included individual profiles of patient BMI changes over time, individual patient care pathways (chronologic record of patient admission, discharge and procedure history), and correlation tests between pairs of variables associated with the same clinical encounters (emergency intervention vs. procedure type; related admission with intervention vs. planned intervention).

“Weight reduction at the first postoperative visit ranged from –71% to a gain of 132% of preoperative weight,” she said. “We also found inconsistency in the sequence of events. Seven percent of readmissions and 12.5% of postoperative interventions were categorized as planned, which is not a problem, but when you look at the reported reasons for planned procedures, they could not all have been possibly planned before discharge.”

Based on 2015 MBASQIP data, “planned” readmissions and postoperative procedures included admissions for nonspecific abdominal pain, band erosion, slippage or prolapse, bleeding, gastrogastric fistula, incisional hernia, infection and/or fever, pneumonia, and wound infection, among other reasons.

“Our analysis found inconsistent quality of data for key parameters, missing and miscoded values and lack of clarity for coding and definitions,” Dr. Noyes said.

The study made four recommendations to improve the quality of data submitted to MASQIP.

• Use health IT applications to provide automated data checks to validate completeness of submitted data – by utilizing a no-skip pattern for core variables – and accuracy of data– by flagging values outside predefined acceptable ranges.
• Perform data audits for consistency, using multiple variables to conduct logic checks, such as by not allowing “readmission” before discharge for the index admission.
• Give data auditors specific recommendations for definitions of registry variables, standardization of algorithms for abstracting values based on commonly used clinical data systems, such as Allscripts and Epic, and standardized use of diagnostic and procedure codes to link with payers’ reimbursement schedules.
• Provide ongoing education to stakeholders such as researchers and hospital administrators on best data management practices and how to best use the data for quality improvement.

Dr. Noyes had no financial relationships to disclose. Coauthor Steven Schwaitzberg, MD, disclosed consulting arrangements with New View Surgical, AcuityBio, Activ Surgical, Human Extensions, Levita Magnetics, and Arch Therapeutics. Aaron Hoffman, MD, disclosed a consulting arrangement with Ethicon.

SOURCE: Noyes K et al. SAGES 2019, Abstract 21

BALTIMORE – A study that examined the quality of data from the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (MBASQIP) registry found inconsistencies that may undermine the quality of the registry, which could be misleading for clinicians and investigators who used the data, according to an analysis presented at the annual meeting of the Society of American Gastrointestinal and Endoscopic Surgeons. The researchers recommended ways to improve data-gathering techniques to better identify the nature of these inconsistencies.

“Our original enthusiasm about the availability of data from MBASQIP turned into a cautious optimism about potential usefulness of these data,” Katia Noyes, PhD, of the State University of New York at Buffalo, said in presenting the study. She noted her research team’s analysis of 168,093 cases in the 2015 MBASQIP Participant Use Data File found that 20% of the cases (n = 33,868) had missing or unusable information for at least one key variable, such as age, race, ethnicity, body mass index (BMI) before and after surgery, and American Society of Anesthesiologists classification. Specifically, preoperative and postoperative BMI data were missing or zero in 6.7% of cases (n = 11,211).

The researchers developed a single flat file for patient-level outcomes evaluation using five files (main, BMI, readmission, intervention, and reoperation). They used logic and validity tests that included individual profiles of patient BMI changes over time, individual patient care pathways (chronologic record of patient admission, discharge and procedure history), and correlation tests between pairs of variables associated with the same clinical encounters (emergency intervention vs. procedure type; related admission with intervention vs. planned intervention).

“Weight reduction at the first postoperative visit ranged from –71% to a gain of 132% of preoperative weight,” she said. “We also found inconsistency in the sequence of events. Seven percent of readmissions and 12.5% of postoperative interventions were categorized as planned, which is not a problem, but when you look at the reported reasons for planned procedures, they could not all have been possibly planned before discharge.”

Based on 2015 MBASQIP data, “planned” readmissions and postoperative procedures included admissions for nonspecific abdominal pain, band erosion, slippage or prolapse, bleeding, gastrogastric fistula, incisional hernia, infection and/or fever, pneumonia, and wound infection, among other reasons.

“Our analysis found inconsistent quality of data for key parameters, missing and miscoded values and lack of clarity for coding and definitions,” Dr. Noyes said.

The study made four recommendations to improve the quality of data submitted to MASQIP.

• Use health IT applications to provide automated data checks to validate completeness of submitted data – by utilizing a no-skip pattern for core variables – and accuracy of data– by flagging values outside predefined acceptable ranges.
• Perform data audits for consistency, using multiple variables to conduct logic checks, such as by not allowing “readmission” before discharge for the index admission.
• Give data auditors specific recommendations for definitions of registry variables, standardization of algorithms for abstracting values based on commonly used clinical data systems, such as Allscripts and Epic, and standardized use of diagnostic and procedure codes to link with payers’ reimbursement schedules.
• Provide ongoing education to stakeholders such as researchers and hospital administrators on best data management practices and how to best use the data for quality improvement.

Dr. Noyes had no financial relationships to disclose. Coauthor Steven Schwaitzberg, MD, disclosed consulting arrangements with New View Surgical, AcuityBio, Activ Surgical, Human Extensions, Levita Magnetics, and Arch Therapeutics. Aaron Hoffman, MD, disclosed a consulting arrangement with Ethicon.

SOURCE: Noyes K et al. SAGES 2019, Abstract 21