Joshua L. Woelk, MD, MS

In 2007, we and our colleagues began assessing the experience necessary to gain proficiency with robotic hysterectomy, and we published our findings early this year.¹ We concluded that the number of cases needed to reach this threshold is about 91—many more than the 20 to 50 cases previously reported.^2-4 Earlier studies defined proficiency in relation to the stabilization of operative times, which is subjective, somewhat arbitrary, and ignores patient outcomes.

To better elucidate the learning curve of robotic hysterectomy, we focused on a more objective, patient-centered analysis that utilized cumulative summation, or CUSUM, analysis and operative complications. This approach mitigates many of the problems encountered in earlier studies and reveals broader implications for the adoption of new surgical techniques and surgical quality control.

How CUSUM analysis works

E. S. Page introduced CUSUM analysis in 1954 for use in industrial quality control.⁵ This approach has been applied more recently to the construction of learning curves in cardiac surgery, general surgery, and anesthesiology.^6-9 Standard CUSUM methodology defines each event—in our study, each robotic hysterectomy case—as a success or failure and tracks the sequence of events between two predefined parameters—the acceptable control limit and the unacceptable control limit. For each success, the CUSUM score decreases toward the acceptable control limit; for each failure, it increases toward the unacceptable limit.

In our study, a procedure was considered a success if no complication occurred; it was a failure if a complication did occur. The acceptable control limit was based on published complication rates of abdominal hysterectomy, and the unacceptable limit was set at twice that rate. A surgeon would be considered proficient when his or her CUSUM chart crosses the lower control limit, signifying that the surgeon’s complication rate is lower than the rate associated with abdominal hysterectomy. We used abdominal complication rates rather than those of laparoscopic hysterectomy because only abdominal and vaginal hysterectomy were performed at our institution, and the robotic system was introduced as a minimally invasive alternative to the abdominal approach.

We also conducted a risk-adjusted CUSUM analysis that was weighted for identified risk factors for complications. As in the standard CUSUM analysis, each score decreases for a successful attempt and increases for an unsuccessful attempt, but the scores are variable, depending on patient risk factors. That is, the score increases more for a complication in a low-risk patient than in a high-risk patient, and vice versa.

Instead of tracking between acceptable and unacceptable limits, the CUSUM scores were plotted around a line representing a predicted complication rate to determine whether complications for a particular surgeon were occurring more often, less often, or as predicted, based on patient risk factors.

Results based on intraoperative complications. With the score based only on intraoperative complications, we observed one surgeon to cross the acceptable control limit after 96 cases and a second surgeon to be trending toward a similar crossing point, although this surgeon had completed only 76 procedures. We calculated the average number of cases needed to develop proficiency to be 91 to cross the acceptable control limit.

Results based on intraoperative and postoperative complications. We also conducted a second analysis that was based on intraoperative and postoperative complications within 6 weeks of surgery. Our two surgeons crossed the acceptable control limit after 21 and 14 cases, respectively, using these parameters. We calculated the average number of cases needed to cross the acceptable control limit to be 44. We considered intraoperative complications to be most indicative of surgical skill; therefore, we concluded that 91 cases are needed to become proficient.

Any learning curve is an individual process

Our findings should not be used as a blanket mark of proficiency. Our conclusion is at first striking, but must be viewed within the context of CUSUM methodology. Ninety-one hysterectomy cases is an average number based on acceptable and unacceptable complication rates; we found it to be consistent with our observations of two active robotic surgeons.

However, any learning curve—not just in robotic hysterectomy—is an individual process dependent on many variables. An experienced, high-volume laparoscopic surgeon may reach proficiency with robotic hysterectomy in many fewer cases than our ballpark number of 91, just as an inexperienced, low-volume surgeon may take many more than 91 procedures to become proficient. Some surgeons may never become proficient. For these reasons, it is inappropriate to assign any single number as a mark of proficiency. Because of its original intent, CUSUM analysis assesses each surgeon on an individual basis and compares that surgeon to an objective benchmark, enabling it to take individual variances in surgeon attributes into account.

CUSUM analysis is a useful tool for surgical quality monitoring

Because it was designed for quality control, this methodology is most suitable when it is applied to assess a surgeon’s progress toward (or away from) proficiency, rather than to assign a representative number to classify a surgeon as proficient. By tracking a surgeon’s particular successes or failures with a procedure, CUSUM analysis can identify problems in an individual’s surgical quality.

If complication rates are tracking near, or cross, the unacceptable control limit using the standard method, or if they trend upward, away from the predicted complication rate with the risk-adjusted method, this fact should arouse concern so that the problem can be identified before patient safety is compromised.

Potential problems contributing to increased complications

Identifiable contributors to an increased complication rate could be intrinsic to the surgeon, such as:

• inadequate training
• low surgical volume
• sleep deprivation
• other personal issues.

Problems extrinsic to the surgeon also could be identified, such as:

• new policy changes in the surgical suite
• new staff assistance during cases
• excessive trainee involvement in surgery.

Ideally, both the standard and risk-adjusted CUSUM methods would be based on institution-specific complication rates and patient risk factors to make them internally valid. In this scenario, CUSUM analysis provides an opportunity for intervention to improve surgical quality and patient outcomes not only in robotic hysterectomy but also in any surgical procedure.

A surgeon’s proficiency waxes and wanes

At its most fundamental level, a learning curve for robotic surgery should be considered an individual continuum. A surgeon’s proficiency will wax and wane throughout his or her career, depending on any number of variables, including surgical volume, case complexity, practice setting, and personal attributes.

Although our findings suggest that a gynecologist, on average, will require 91 cases to become proficient in robotic hysterectomy, an overall benefit of robotic hysterectomy over abdominal hysterectomy was observed after completion of 21 and 14 cases by our two surgeons. We do not believe that credentialing bodies should mandate that 91 robotic hysterectomies be required of a surgeon. That approach would be too simplistic and obfuscates many of the true implications of our study—most importantly, that learning a new procedure is an individual process that must be compared with an acceptable outcome to determine proficiency and maintain patient safety.

In 2007, we and our colleagues began assessing the experience necessary to gain proficiency with robotic hysterectomy, and we published our findings early this year.¹ We concluded that the number of cases needed to reach this threshold is about 91—many more than the 20 to 50 cases previously reported.^2-4 Earlier studies defined proficiency in relation to the stabilization of operative times, which is subjective, somewhat arbitrary, and ignores patient outcomes.

To better elucidate the learning curve of robotic hysterectomy, we focused on a more objective, patient-centered analysis that utilized cumulative summation, or CUSUM, analysis and operative complications. This approach mitigates many of the problems encountered in earlier studies and reveals broader implications for the adoption of new surgical techniques and surgical quality control.

How CUSUM analysis works

E. S. Page introduced CUSUM analysis in 1954 for use in industrial quality control.⁵ This approach has been applied more recently to the construction of learning curves in cardiac surgery, general surgery, and anesthesiology.^6-9 Standard CUSUM methodology defines each event—in our study, each robotic hysterectomy case—as a success or failure and tracks the sequence of events between two predefined parameters—the acceptable control limit and the unacceptable control limit. For each success, the CUSUM score decreases toward the acceptable control limit; for each failure, it increases toward the unacceptable limit.

In our study, a procedure was considered a success if no complication occurred; it was a failure if a complication did occur. The acceptable control limit was based on published complication rates of abdominal hysterectomy, and the unacceptable limit was set at twice that rate. A surgeon would be considered proficient when his or her CUSUM chart crosses the lower control limit, signifying that the surgeon’s complication rate is lower than the rate associated with abdominal hysterectomy. We used abdominal complication rates rather than those of laparoscopic hysterectomy because only abdominal and vaginal hysterectomy were performed at our institution, and the robotic system was introduced as a minimally invasive alternative to the abdominal approach.

We also conducted a risk-adjusted CUSUM analysis that was weighted for identified risk factors for complications. As in the standard CUSUM analysis, each score decreases for a successful attempt and increases for an unsuccessful attempt, but the scores are variable, depending on patient risk factors. That is, the score increases more for a complication in a low-risk patient than in a high-risk patient, and vice versa.

Instead of tracking between acceptable and unacceptable limits, the CUSUM scores were plotted around a line representing a predicted complication rate to determine whether complications for a particular surgeon were occurring more often, less often, or as predicted, based on patient risk factors.

Results based on intraoperative complications. With the score based only on intraoperative complications, we observed one surgeon to cross the acceptable control limit after 96 cases and a second surgeon to be trending toward a similar crossing point, although this surgeon had completed only 76 procedures. We calculated the average number of cases needed to develop proficiency to be 91 to cross the acceptable control limit.

Results based on intraoperative and postoperative complications. We also conducted a second analysis that was based on intraoperative and postoperative complications within 6 weeks of surgery. Our two surgeons crossed the acceptable control limit after 21 and 14 cases, respectively, using these parameters. We calculated the average number of cases needed to cross the acceptable control limit to be 44. We considered intraoperative complications to be most indicative of surgical skill; therefore, we concluded that 91 cases are needed to become proficient.

Any learning curve is an individual process

Our findings should not be used as a blanket mark of proficiency. Our conclusion is at first striking, but must be viewed within the context of CUSUM methodology. Ninety-one hysterectomy cases is an average number based on acceptable and unacceptable complication rates; we found it to be consistent with our observations of two active robotic surgeons.

However, any learning curve—not just in robotic hysterectomy—is an individual process dependent on many variables. An experienced, high-volume laparoscopic surgeon may reach proficiency with robotic hysterectomy in many fewer cases than our ballpark number of 91, just as an inexperienced, low-volume surgeon may take many more than 91 procedures to become proficient. Some surgeons may never become proficient. For these reasons, it is inappropriate to assign any single number as a mark of proficiency. Because of its original intent, CUSUM analysis assesses each surgeon on an individual basis and compares that surgeon to an objective benchmark, enabling it to take individual variances in surgeon attributes into account.

CUSUM analysis is a useful tool for surgical quality monitoring

Because it was designed for quality control, this methodology is most suitable when it is applied to assess a surgeon’s progress toward (or away from) proficiency, rather than to assign a representative number to classify a surgeon as proficient. By tracking a surgeon’s particular successes or failures with a procedure, CUSUM analysis can identify problems in an individual’s surgical quality.

If complication rates are tracking near, or cross, the unacceptable control limit using the standard method, or if they trend upward, away from the predicted complication rate with the risk-adjusted method, this fact should arouse concern so that the problem can be identified before patient safety is compromised.

Potential problems contributing to increased complications

Identifiable contributors to an increased complication rate could be intrinsic to the surgeon, such as:

• inadequate training
• low surgical volume
• sleep deprivation
• other personal issues.

Problems extrinsic to the surgeon also could be identified, such as:

• new policy changes in the surgical suite
• new staff assistance during cases
• excessive trainee involvement in surgery.

Ideally, both the standard and risk-adjusted CUSUM methods would be based on institution-specific complication rates and patient risk factors to make them internally valid. In this scenario, CUSUM analysis provides an opportunity for intervention to improve surgical quality and patient outcomes not only in robotic hysterectomy but also in any surgical procedure.

A surgeon’s proficiency waxes and wanes

At its most fundamental level, a learning curve for robotic surgery should be considered an individual continuum. A surgeon’s proficiency will wax and wane throughout his or her career, depending on any number of variables, including surgical volume, case complexity, practice setting, and personal attributes.

Although our findings suggest that a gynecologist, on average, will require 91 cases to become proficient in robotic hysterectomy, an overall benefit of robotic hysterectomy over abdominal hysterectomy was observed after completion of 21 and 14 cases by our two surgeons. We do not believe that credentialing bodies should mandate that 91 robotic hysterectomies be required of a surgeon. That approach would be too simplistic and obfuscates many of the true implications of our study—most importantly, that learning a new procedure is an individual process that must be compared with an acceptable outcome to determine proficiency and maintain patient safety.

In 2007, we and our colleagues began assessing the experience necessary to gain proficiency with robotic hysterectomy, and we published our findings early this year.¹ We concluded that the number of cases needed to reach this threshold is about 91—many more than the 20 to 50 cases previously reported.^2-4 Earlier studies defined proficiency in relation to the stabilization of operative times, which is subjective, somewhat arbitrary, and ignores patient outcomes.

To better elucidate the learning curve of robotic hysterectomy, we focused on a more objective, patient-centered analysis that utilized cumulative summation, or CUSUM, analysis and operative complications. This approach mitigates many of the problems encountered in earlier studies and reveals broader implications for the adoption of new surgical techniques and surgical quality control.

How CUSUM analysis works

E. S. Page introduced CUSUM analysis in 1954 for use in industrial quality control.⁵ This approach has been applied more recently to the construction of learning curves in cardiac surgery, general surgery, and anesthesiology.^6-9 Standard CUSUM methodology defines each event—in our study, each robotic hysterectomy case—as a success or failure and tracks the sequence of events between two predefined parameters—the acceptable control limit and the unacceptable control limit. For each success, the CUSUM score decreases toward the acceptable control limit; for each failure, it increases toward the unacceptable limit.

In our study, a procedure was considered a success if no complication occurred; it was a failure if a complication did occur. The acceptable control limit was based on published complication rates of abdominal hysterectomy, and the unacceptable limit was set at twice that rate. A surgeon would be considered proficient when his or her CUSUM chart crosses the lower control limit, signifying that the surgeon’s complication rate is lower than the rate associated with abdominal hysterectomy. We used abdominal complication rates rather than those of laparoscopic hysterectomy because only abdominal and vaginal hysterectomy were performed at our institution, and the robotic system was introduced as a minimally invasive alternative to the abdominal approach.

We also conducted a risk-adjusted CUSUM analysis that was weighted for identified risk factors for complications. As in the standard CUSUM analysis, each score decreases for a successful attempt and increases for an unsuccessful attempt, but the scores are variable, depending on patient risk factors. That is, the score increases more for a complication in a low-risk patient than in a high-risk patient, and vice versa.

Instead of tracking between acceptable and unacceptable limits, the CUSUM scores were plotted around a line representing a predicted complication rate to determine whether complications for a particular surgeon were occurring more often, less often, or as predicted, based on patient risk factors.

Results based on intraoperative complications. With the score based only on intraoperative complications, we observed one surgeon to cross the acceptable control limit after 96 cases and a second surgeon to be trending toward a similar crossing point, although this surgeon had completed only 76 procedures. We calculated the average number of cases needed to develop proficiency to be 91 to cross the acceptable control limit.

Results based on intraoperative and postoperative complications. We also conducted a second analysis that was based on intraoperative and postoperative complications within 6 weeks of surgery. Our two surgeons crossed the acceptable control limit after 21 and 14 cases, respectively, using these parameters. We calculated the average number of cases needed to cross the acceptable control limit to be 44. We considered intraoperative complications to be most indicative of surgical skill; therefore, we concluded that 91 cases are needed to become proficient.

Any learning curve is an individual process

Our findings should not be used as a blanket mark of proficiency. Our conclusion is at first striking, but must be viewed within the context of CUSUM methodology. Ninety-one hysterectomy cases is an average number based on acceptable and unacceptable complication rates; we found it to be consistent with our observations of two active robotic surgeons.

However, any learning curve—not just in robotic hysterectomy—is an individual process dependent on many variables. An experienced, high-volume laparoscopic surgeon may reach proficiency with robotic hysterectomy in many fewer cases than our ballpark number of 91, just as an inexperienced, low-volume surgeon may take many more than 91 procedures to become proficient. Some surgeons may never become proficient. For these reasons, it is inappropriate to assign any single number as a mark of proficiency. Because of its original intent, CUSUM analysis assesses each surgeon on an individual basis and compares that surgeon to an objective benchmark, enabling it to take individual variances in surgeon attributes into account.

CUSUM analysis is a useful tool for surgical quality monitoring

Because it was designed for quality control, this methodology is most suitable when it is applied to assess a surgeon’s progress toward (or away from) proficiency, rather than to assign a representative number to classify a surgeon as proficient. By tracking a surgeon’s particular successes or failures with a procedure, CUSUM analysis can identify problems in an individual’s surgical quality.

If complication rates are tracking near, or cross, the unacceptable control limit using the standard method, or if they trend upward, away from the predicted complication rate with the risk-adjusted method, this fact should arouse concern so that the problem can be identified before patient safety is compromised.

Potential problems contributing to increased complications

Identifiable contributors to an increased complication rate could be intrinsic to the surgeon, such as:

• inadequate training
• low surgical volume
• sleep deprivation
• other personal issues.

Problems extrinsic to the surgeon also could be identified, such as:

• new policy changes in the surgical suite
• new staff assistance during cases
• excessive trainee involvement in surgery.

Ideally, both the standard and risk-adjusted CUSUM methods would be based on institution-specific complication rates and patient risk factors to make them internally valid. In this scenario, CUSUM analysis provides an opportunity for intervention to improve surgical quality and patient outcomes not only in robotic hysterectomy but also in any surgical procedure.

A surgeon’s proficiency waxes and wanes

At its most fundamental level, a learning curve for robotic surgery should be considered an individual continuum. A surgeon’s proficiency will wax and wane throughout his or her career, depending on any number of variables, including surgical volume, case complexity, practice setting, and personal attributes.

Although our findings suggest that a gynecologist, on average, will require 91 cases to become proficient in robotic hysterectomy, an overall benefit of robotic hysterectomy over abdominal hysterectomy was observed after completion of 21 and 14 cases by our two surgeons. We do not believe that credentialing bodies should mandate that 91 robotic hysterectomies be required of a surgeon. That approach would be too simplistic and obfuscates many of the true implications of our study—most importantly, that learning a new procedure is an individual process that must be compared with an acceptable outcome to determine proficiency and maintain patient safety.