Eleven on a scale of 1 to 10

 

I literally rode into the sunset recently as I finished my tour of duty as one of the Director examiners for the American Board of Surgery. I was heading out of St. Louis westward toward my home in Kansas. It was a 9-hour drive, which gave me plenty of time to reflect on the 6 years I shared the responsibility of administering the certifying exam known by most surgeons as “the oral exam.”

As most of our readers know, the exam consists of three 28-minute sessions presided over by a team of six certified surgeons. Each session is given by two surgeons, one a current ABS Director and one an associate examiner. A session covers four clinical scenarios, and each scenario is graded separately as “pass,” “equivocal,” or “fail.” A statistical adjustment based on the grading pattern of each senior examiner is then applied. Some of us consistently grade harder than others, and so an adjustment is made to mitigate the possibility of three senior examiners who are either extreme hawks or extreme doves in grading.

Over the last dozen years, Directors V. Suzanne Klimberg and Karen J. Brasel, along with former Executive Director Frank Lewis, a team of psychometricians at the Board, and members of the certification committee of the Board, worked tirelessly to create a testing instrument as fair and statistically sound as possible given the inherently qualitative exam. I believe they did a magnificent job. Gone are the legends of yesteryear where candidates were subjected to the whims of whatever crossed the mind of the examiners, including their prejudices about the “right” answer. The oral exam now represents a well-constructed survey of surgical judgment issues that have been thoroughly vetted.

Being an examiner for the orals means you arrive Sunday afternoon before the exams that are given over the next two and a half days. Each examiner undergoes an overview briefing on Sunday afternoon and then studies “the book” for that test’s content usually until late into the night. This book is an impressive document.

We arise around 0530 to attend a breakfast meeting, which includes breaking into our six-person teams and going over each question that will be given that day. At 0800, the first candidates for the first session walk into a room and meet the two surgeons who will make some of the most important decisions affecting that candidate’s career. If you rate the intensity of this moment on a scale of 1 to 10, this is an 11 for both candidates and examiners. No one in the room knows how it will turn out because every session has its own twists and turns. Everyone there wants to see a passing score, but the two examiners know that they must make a decision that is safe for the public and fair for the candidate.

Twelve exams are given per team per day except for the final day which has only six. So, each team examines 30 candidates over 3 days. I’ve opened a door and shaken the moist hand of 438 candidates. I’ve seen every sort of emotion during those sessions. I’ve had moments of great joy and times of profound sadness as candidates respond to the questions. I’ve always tried to be friendly, but just like surgery, it is a serious business and decisions have to be made. That means ignoring one’s hopes and acting on the best facts available at the moment. Most surgeons remember their oral examiners and what they were asked for a lifetime. I know I do.

I could write a book on this experience (I won’t, though). But as I reflect on my time as a Director, what stands out in my mind are the associate examiners with whom I’ve worked. These surgeons are invited to participate and receive no compensation. It’s 3 days out of their lives, and because they don’t give the exam as frequently as the Directors do, the amount of study and effort is greater for them. Each is selected because he or she is considered to be a thoughtful surgeon with high standards. These surgeons do this job because they care about quality in our profession.

 

Most of the associates I have worked with are far more accomplished than I. I was once paired with a renowned breast surgeon (okay, it was Kelly K. Hunt). My ego was at great risk because I knew how accomplished she was. But like all the other associates, she was gracious and hard working. We rarely work with another Director, but Anne G. Rizzo, who later became a Director, and I did an exam together. She was a dazzling questioner with very high standards. In other words, she was typical of the people I met. My first associate (also later a Director) was Reid Adams. He was great; I was nervous. My last associate was Marc L. Melcher, a transplant surgeon who asked penetrating questions in a calm manner. I wish I could name each of my associates and thank them for making my work so much better, for teaching me things I didn’t know, for deepening my own knowledge, and serving in a hard job with grace. This column can’t be that long, but you all know who you are. Thank you.


At the end of the day, I believe the oral exam to be a great thing for our profession. When you think about the number of patients potentially affected throughout a surgeon’s career, the impact of decisions made on the day of the exam can be enormous. Given that, over a 20-year career, a surgeon may operate on 25,000 patients, a summative check on a surgeon’s judgment and knowledge is important. Each year, the ABS adjudicates on some 1,100 surgeons. A single year’s set of surgeons over the following 20-year period translates into 27.5 million patients. I hope we never stop doing the orals because of cost, time, or convenience. The exam is just too important to our profession to risk forgoing this last, big step before a surgeon is presented to the world as “certified.”

 

I literally rode into the sunset recently as I finished my tour of duty as one of the Director examiners for the American Board of Surgery. I was heading out of St. Louis westward toward my home in Kansas. It was a 9-hour drive, which gave me plenty of time to reflect on the 6 years I shared the responsibility of administering the certifying exam known by most surgeons as “the oral exam.”

As most of our readers know, the exam consists of three 28-minute sessions presided over by a team of six certified surgeons. Each session is given by two surgeons, one a current ABS Director and one an associate examiner. A session covers four clinical scenarios, and each scenario is graded separately as “pass,” “equivocal,” or “fail.” A statistical adjustment based on the grading pattern of each senior examiner is then applied. Some of us consistently grade harder than others, and so an adjustment is made to mitigate the possibility of three senior examiners who are either extreme hawks or extreme doves in grading.

Over the last dozen years, Directors V. Suzanne Klimberg and Karen J. Brasel, along with former Executive Director Frank Lewis, a team of psychometricians at the Board, and members of the certification committee of the Board, worked tirelessly to create a testing instrument as fair and statistically sound as possible given the inherently qualitative exam. I believe they did a magnificent job. Gone are the legends of yesteryear where candidates were subjected to the whims of whatever crossed the mind of the examiners, including their prejudices about the “right” answer. The oral exam now represents a well-constructed survey of surgical judgment issues that have been thoroughly vetted.

Being an examiner for the orals means you arrive Sunday afternoon before the exams that are given over the next two and a half days. Each examiner undergoes an overview briefing on Sunday afternoon and then studies “the book” for that test’s content usually until late into the night. This book is an impressive document.

We arise around 0530 to attend a breakfast meeting, which includes breaking into our six-person teams and going over each question that will be given that day. At 0800, the first candidates for the first session walk into a room and meet the two surgeons who will make some of the most important decisions affecting that candidate’s career. If you rate the intensity of this moment on a scale of 1 to 10, this is an 11 for both candidates and examiners. No one in the room knows how it will turn out because every session has its own twists and turns. Everyone there wants to see a passing score, but the two examiners know that they must make a decision that is safe for the public and fair for the candidate.

Twelve exams are given per team per day except for the final day which has only six. So, each team examines 30 candidates over 3 days. I’ve opened a door and shaken the moist hand of 438 candidates. I’ve seen every sort of emotion during those sessions. I’ve had moments of great joy and times of profound sadness as candidates respond to the questions. I’ve always tried to be friendly, but just like surgery, it is a serious business and decisions have to be made. That means ignoring one’s hopes and acting on the best facts available at the moment. Most surgeons remember their oral examiners and what they were asked for a lifetime. I know I do.

I could write a book on this experience (I won’t, though). But as I reflect on my time as a Director, what stands out in my mind are the associate examiners with whom I’ve worked. These surgeons are invited to participate and receive no compensation. It’s 3 days out of their lives, and because they don’t give the exam as frequently as the Directors do, the amount of study and effort is greater for them. Each is selected because he or she is considered to be a thoughtful surgeon with high standards. These surgeons do this job because they care about quality in our profession.

 

Most of the associates I have worked with are far more accomplished than I. I was once paired with a renowned breast surgeon (okay, it was Kelly K. Hunt). My ego was at great risk because I knew how accomplished she was. But like all the other associates, she was gracious and hard working. We rarely work with another Director, but Anne G. Rizzo, who later became a Director, and I did an exam together. She was a dazzling questioner with very high standards. In other words, she was typical of the people I met. My first associate (also later a Director) was Reid Adams. He was great; I was nervous. My last associate was Marc L. Melcher, a transplant surgeon who asked penetrating questions in a calm manner. I wish I could name each of my associates and thank them for making my work so much better, for teaching me things I didn’t know, for deepening my own knowledge, and serving in a hard job with grace. This column can’t be that long, but you all know who you are. Thank you.


At the end of the day, I believe the oral exam to be a great thing for our profession. When you think about the number of patients potentially affected throughout a surgeon’s career, the impact of decisions made on the day of the exam can be enormous. Given that, over a 20-year career, a surgeon may operate on 25,000 patients, a summative check on a surgeon’s judgment and knowledge is important. Each year, the ABS adjudicates on some 1,100 surgeons. A single year’s set of surgeons over the following 20-year period translates into 27.5 million patients. I hope we never stop doing the orals because of cost, time, or convenience. The exam is just too important to our profession to risk forgoing this last, big step before a surgeon is presented to the world as “certified.”

 

I literally rode into the sunset recently as I finished my tour of duty as one of the Director examiners for the American Board of Surgery. I was heading out of St. Louis westward toward my home in Kansas. It was a 9-hour drive, which gave me plenty of time to reflect on the 6 years I shared the responsibility of administering the certifying exam known by most surgeons as “the oral exam.”

As most of our readers know, the exam consists of three 28-minute sessions presided over by a team of six certified surgeons. Each session is given by two surgeons, one a current ABS Director and one an associate examiner. A session covers four clinical scenarios, and each scenario is graded separately as “pass,” “equivocal,” or “fail.” A statistical adjustment based on the grading pattern of each senior examiner is then applied. Some of us consistently grade harder than others, and so an adjustment is made to mitigate the possibility of three senior examiners who are either extreme hawks or extreme doves in grading.

Over the last dozen years, Directors V. Suzanne Klimberg and Karen J. Brasel, along with former Executive Director Frank Lewis, a team of psychometricians at the Board, and members of the certification committee of the Board, worked tirelessly to create a testing instrument as fair and statistically sound as possible given the inherently qualitative exam. I believe they did a magnificent job. Gone are the legends of yesteryear where candidates were subjected to the whims of whatever crossed the mind of the examiners, including their prejudices about the “right” answer. The oral exam now represents a well-constructed survey of surgical judgment issues that have been thoroughly vetted.

Being an examiner for the orals means you arrive Sunday afternoon before the exams that are given over the next two and a half days. Each examiner undergoes an overview briefing on Sunday afternoon and then studies “the book” for that test’s content usually until late into the night. This book is an impressive document.

We arise around 0530 to attend a breakfast meeting, which includes breaking into our six-person teams and going over each question that will be given that day. At 0800, the first candidates for the first session walk into a room and meet the two surgeons who will make some of the most important decisions affecting that candidate’s career. If you rate the intensity of this moment on a scale of 1 to 10, this is an 11 for both candidates and examiners. No one in the room knows how it will turn out because every session has its own twists and turns. Everyone there wants to see a passing score, but the two examiners know that they must make a decision that is safe for the public and fair for the candidate.

Twelve exams are given per team per day except for the final day which has only six. So, each team examines 30 candidates over 3 days. I’ve opened a door and shaken the moist hand of 438 candidates. I’ve seen every sort of emotion during those sessions. I’ve had moments of great joy and times of profound sadness as candidates respond to the questions. I’ve always tried to be friendly, but just like surgery, it is a serious business and decisions have to be made. That means ignoring one’s hopes and acting on the best facts available at the moment. Most surgeons remember their oral examiners and what they were asked for a lifetime. I know I do.

I could write a book on this experience (I won’t, though). But as I reflect on my time as a Director, what stands out in my mind are the associate examiners with whom I’ve worked. These surgeons are invited to participate and receive no compensation. It’s 3 days out of their lives, and because they don’t give the exam as frequently as the Directors do, the amount of study and effort is greater for them. Each is selected because he or she is considered to be a thoughtful surgeon with high standards. These surgeons do this job because they care about quality in our profession.

 

Most of the associates I have worked with are far more accomplished than I. I was once paired with a renowned breast surgeon (okay, it was Kelly K. Hunt). My ego was at great risk because I knew how accomplished she was. But like all the other associates, she was gracious and hard working. We rarely work with another Director, but Anne G. Rizzo, who later became a Director, and I did an exam together. She was a dazzling questioner with very high standards. In other words, she was typical of the people I met. My first associate (also later a Director) was Reid Adams. He was great; I was nervous. My last associate was Marc L. Melcher, a transplant surgeon who asked penetrating questions in a calm manner. I wish I could name each of my associates and thank them for making my work so much better, for teaching me things I didn’t know, for deepening my own knowledge, and serving in a hard job with grace. This column can’t be that long, but you all know who you are. Thank you.


At the end of the day, I believe the oral exam to be a great thing for our profession. When you think about the number of patients potentially affected throughout a surgeon’s career, the impact of decisions made on the day of the exam can be enormous. Given that, over a 20-year career, a surgeon may operate on 25,000 patients, a summative check on a surgeon’s judgment and knowledge is important. Each year, the ABS adjudicates on some 1,100 surgeons. A single year’s set of surgeons over the following 20-year period translates into 27.5 million patients. I hope we never stop doing the orals because of cost, time, or convenience. The exam is just too important to our profession to risk forgoing this last, big step before a surgeon is presented to the world as “certified.”