User login
Better late than later: Lessons learned from an investigator-led clinical trial
As a second-year gastroenterology fellow, I designed a prospective, double-blind randomized, controlled trial for vitamin D repletion in patients with Crohn’s disease at a referral inflammatory bowel disease (IBD) center. I had the support of a dedicated research team, several mentors, and a 2-year time frame in which to complete this study. Intellectually curious and academically eager, I labored over a grant application that I did not receive. Under generous financial support from my department, I forged on and opened the trial for enrollment at the start of my advanced IBD fellowship year. However, we experienced recruitment challenges that ultimately led to the study’s premature termination. Through this journey, I gained invaluable experience that will continue to serve me – and, I hope, the reader – as I progress in my career. Below are some important insights gleaned from this experience that may benefit others interested in clinical trial design.
Know your “why” (personally and clinically)
Asked to reflect on their career path, experts tend to recount their “being in the right place at the right time” and having “good mentors.” While luck and good mentorship are necessary, I propose that doing your homework is equally as important. Before ambling down the path of an investigator-led trial, I urge a hard pause to reflect on your “why.” The personal “why” of “getting into fellowship,” “advancement in the department,” or “learning more about the principles of research,” are all valid. But I suggest a deeper dive is in order. Successful clinical trials require resources, a substantial time commitment, lots of sweat and maybe a few tears. In the ideal setting, your trial experience will serve as the foundation for a compelling personal narrative and might help launch a productive clinical research career. With stakes that high, asking the tough questions is critical.
The clinical “why” is just as critical. We press our attendings on why they used this drug or that clip, so don’t be afraid to ask whether this is a space in which others have succeeded. Or conversely, why is there such a large gap in the literature? I remember emailing the world’s expert about my topic because the published data were so murky. He had more questions than answers which, in retrospect, should have raised red flags about the ability to design a sound study. It is equally important to determine if patients are vested in the research question. A successful clinical trial hinges on subject participation, often outside their clinic visit. Patients with complex chronic diseases spend a lot of time navigating the health care system. Participating in a clinical trial needs to be meaningful to them if you want your patients to fully engage. Thoughtfully answering these questions on the front end – for yourself and the study in question – will improve both your experience and the ultimate outcome exponentially.
Identify your village (and listen to them)
Designing a clinical trial truly takes a village, and you need to identify the villagers early. As trainees, the value of mentorship is frequently underscored. But the importance of, and the nuance involved in managing, collaborating, and support cannot be overstated. Meet with a biostatistician to ensure your sample size calculations are correct. Work closely with the research pharmacist to ensure the medication formulation is available; decide on the manner of distribution so as not to inconvenience subjects; create a budget with an experienced manager. Seek out research coordinators often for assistance in creating case report forms, learning appropriate documentation, and crafting responses to Institutional Review Board concerns. Ask clinic personnel about arranging consent or follow-up visits around subjects’ clinic appointments. Present a draft of the protocol to your colleagues, as you will ultimately need them on board to recruit patients. And most importantly, listen to them ... all of them. Get your biases in check and write down all constructive criticism. Thoughtfully address each concern encountered to your satisfaction (and your mentor’s) and present to your village again. Rinse and repeat. Throw nothing under the rug, because if you do, it will eventually rear its head while you are in the throes of the study.
In retrospect, there were concerns raised by faculty and grant reviewers that I did not adequately address. First was the feasibility of screening, recruiting, and enrolling 80 patients during a busy clinical fellowship. While I took this criticism as a reflection of my personal commitment to the project, it was actually a call to consider the impact on clinical (and familial) responsibilities. But as I started enrolling patients, I realized there were logistic issues implied in this suggestion. I could not recruit subjects in Clinic A if I was assigned to see patients at the same time in Clinic B. Patients were not likely to come back another day for study-related discussions. Second, I designed eligibility criteria to make the data as clean as possible. Limiting the study to subjects with Crohn’s disease, in clinical and biochemical remission, without complications of their disease, who also have vitamin deficiency, may be an unrealistic recipe to recruit 80 people in a limited period of time. Finally, I designed laboratory follow-up schedules based on the pharmacokinetics of the drug alone, failing to consider the clinical milieu from which study subjects were recruited. Neglecting the fact that many patients obtain labs with their infusions, my study increased lab draw burden, heaped more patient reminders onto my plate – and more concerning – decreased overall study compliance. In short, trial design cannot be done in a vacuum or by just poring over published data. There are logistical and patient-related considerations that require early input from physicians and clinical staff in order for all the moving parts of a clinical trial to successfully work in harmony.
Create a timetable and follow it
Make a recruitment timetable very early on in the enrollment period. Set up biweekly meetings with mentors to discuss enrollment numbers and reflect on any unforeseen challenges. And be sure to celebrate the wins as well – not matter how small. In our study, falling behind on recruitment goals forced me to amend the stringent eligibility criteria and add additional manpower to help with reminders for laboratory follow-up and patient screening. These pivots caused study delays and cost resources. Ultimately, having a timetable forced me to take pause when it became clear I could not finish the study in the allotted time.
Know when to fold ’em
Knowing when to close a study is far easier said than done. The sunk cost fallacy says it is much harder to abandon a project after investing so many resources into it. For us, it was the recruitment timetable that gave us pause. Finishing trial accrual by the end of my advanced fellowship year was wholly unfeasible. When it became clear that nobody in the department could see it through to completion, I was propelled to terminate the study. If there is concern about termination, I suggest sending the protocol, recruitment numbers, and timeline to an outside colleague for a second, unbiased opinion. Review the already compiled data for any notable findings worthy of a smaller publication. It is said, we often learn more from our failures than our successes. The experience described herein – largely in part to my mentors, collaborators, and the patients who put their faith in me – translates to a lasting, invaluable win.
Mentor’s note
Clinical research is hard. Many trainees meet with me to “get involved” in clinical research, and the challenge as a mentor is to identify a project appropriate to the level of training and provide the infrastructure and resources to facilitate success for the motivated trainee. Trainees have various goals of their involvement in research – to foster a relationship in the hopes of receiving a strong letter of support, to facilitate getting into a competitive training program, and/or to publish. My goal as a mentor is to help my trainees reach their goals, but as a clinical researcher, I look for the trainee’s desire to engage with and learn the research process, with the ancillary potential for a letter, for acceptance to a program, or for publication.
This particular study, a randomized, controlled trial of vitamin D in patients with Crohn’s disease, involved an enormous undertaking by a very motivated trainee who took the project from its inception; to putting a thoughtful grant proposal together; to developing a full clinical trial protocol with its ancillary regulatory documents; and obtaining institutional review board approval, statistician input, pharmacy support, and buy-in from faculty and ancillary staff stakeholders. The study ultimately failed because of low enrollment – patients did not want to participate (for reasons elucidated above) – not because of poor design or execution of the myriad components of a prospective clinical trial. Low enrollment has led to the failure of many otherwise excellent studies, including several in our field of IBD.1,2 As a mentor, it is rational to accept blame for the failure of a trainee project; how could I have better foreseen the outcome of this study? Could this have been prevented with more support, more oversight, or more “micromanagement,” to the potential detriment of fostering independence?
Ultimately, the value of clinical research to trainees is multifaceted. If the goal was a first-author publication with high clinical impact, this trial failed. But if the goal was to learn about the clinical trial process, this study was a resounding success. Ultimately, it behooves trainees and their mentors to engage in early, upfront conversations about research. What are the goals? What does success look like? What if the trial fails? By shifting the focus from the success of the project to the success of the mentorship and educational process, even failed projects are resounding successes, upon which future careers can be further developed.
References
1. Kan S et al. When subjects violate the research covenant: Lessons learned from a failed clinical trial of fecal microbiota transplantation. Am J Gastroenterol 2016;111:1508-10.
2. Dassopoulos T et al. Randomised clinical trial: Individualised vs.weight-based dosing of azathioprine in Crohn’s disease. Aliment Pharmacol Ther. 2014 Jan;39(2):163-75.
Dr. Cohen is an inflammatory bowel disease fellow, inflammatory bowel disease center, division of gastroenterology and hepatology, department of medicine, Cedars-Sinai Medical Center, Los Angeles. Dr. Melmed is director, inflammatory bowel disease clinical research and codirector, clinical inflammatory bowel disease, inflammatory bowel disease center, division of gastroenterology and hepatology, department of medicine, Cedars-Sinai Medical Center, Los Angeles.
As a second-year gastroenterology fellow, I designed a prospective, double-blind randomized, controlled trial for vitamin D repletion in patients with Crohn’s disease at a referral inflammatory bowel disease (IBD) center. I had the support of a dedicated research team, several mentors, and a 2-year time frame in which to complete this study. Intellectually curious and academically eager, I labored over a grant application that I did not receive. Under generous financial support from my department, I forged on and opened the trial for enrollment at the start of my advanced IBD fellowship year. However, we experienced recruitment challenges that ultimately led to the study’s premature termination. Through this journey, I gained invaluable experience that will continue to serve me – and, I hope, the reader – as I progress in my career. Below are some important insights gleaned from this experience that may benefit others interested in clinical trial design.
Know your “why” (personally and clinically)
Asked to reflect on their career path, experts tend to recount their “being in the right place at the right time” and having “good mentors.” While luck and good mentorship are necessary, I propose that doing your homework is equally as important. Before ambling down the path of an investigator-led trial, I urge a hard pause to reflect on your “why.” The personal “why” of “getting into fellowship,” “advancement in the department,” or “learning more about the principles of research,” are all valid. But I suggest a deeper dive is in order. Successful clinical trials require resources, a substantial time commitment, lots of sweat and maybe a few tears. In the ideal setting, your trial experience will serve as the foundation for a compelling personal narrative and might help launch a productive clinical research career. With stakes that high, asking the tough questions is critical.
The clinical “why” is just as critical. We press our attendings on why they used this drug or that clip, so don’t be afraid to ask whether this is a space in which others have succeeded. Or conversely, why is there such a large gap in the literature? I remember emailing the world’s expert about my topic because the published data were so murky. He had more questions than answers which, in retrospect, should have raised red flags about the ability to design a sound study. It is equally important to determine if patients are vested in the research question. A successful clinical trial hinges on subject participation, often outside their clinic visit. Patients with complex chronic diseases spend a lot of time navigating the health care system. Participating in a clinical trial needs to be meaningful to them if you want your patients to fully engage. Thoughtfully answering these questions on the front end – for yourself and the study in question – will improve both your experience and the ultimate outcome exponentially.
Identify your village (and listen to them)
Designing a clinical trial truly takes a village, and you need to identify the villagers early. As trainees, the value of mentorship is frequently underscored. But the importance of, and the nuance involved in managing, collaborating, and support cannot be overstated. Meet with a biostatistician to ensure your sample size calculations are correct. Work closely with the research pharmacist to ensure the medication formulation is available; decide on the manner of distribution so as not to inconvenience subjects; create a budget with an experienced manager. Seek out research coordinators often for assistance in creating case report forms, learning appropriate documentation, and crafting responses to Institutional Review Board concerns. Ask clinic personnel about arranging consent or follow-up visits around subjects’ clinic appointments. Present a draft of the protocol to your colleagues, as you will ultimately need them on board to recruit patients. And most importantly, listen to them ... all of them. Get your biases in check and write down all constructive criticism. Thoughtfully address each concern encountered to your satisfaction (and your mentor’s) and present to your village again. Rinse and repeat. Throw nothing under the rug, because if you do, it will eventually rear its head while you are in the throes of the study.
In retrospect, there were concerns raised by faculty and grant reviewers that I did not adequately address. First was the feasibility of screening, recruiting, and enrolling 80 patients during a busy clinical fellowship. While I took this criticism as a reflection of my personal commitment to the project, it was actually a call to consider the impact on clinical (and familial) responsibilities. But as I started enrolling patients, I realized there were logistic issues implied in this suggestion. I could not recruit subjects in Clinic A if I was assigned to see patients at the same time in Clinic B. Patients were not likely to come back another day for study-related discussions. Second, I designed eligibility criteria to make the data as clean as possible. Limiting the study to subjects with Crohn’s disease, in clinical and biochemical remission, without complications of their disease, who also have vitamin deficiency, may be an unrealistic recipe to recruit 80 people in a limited period of time. Finally, I designed laboratory follow-up schedules based on the pharmacokinetics of the drug alone, failing to consider the clinical milieu from which study subjects were recruited. Neglecting the fact that many patients obtain labs with their infusions, my study increased lab draw burden, heaped more patient reminders onto my plate – and more concerning – decreased overall study compliance. In short, trial design cannot be done in a vacuum or by just poring over published data. There are logistical and patient-related considerations that require early input from physicians and clinical staff in order for all the moving parts of a clinical trial to successfully work in harmony.
Create a timetable and follow it
Make a recruitment timetable very early on in the enrollment period. Set up biweekly meetings with mentors to discuss enrollment numbers and reflect on any unforeseen challenges. And be sure to celebrate the wins as well – not matter how small. In our study, falling behind on recruitment goals forced me to amend the stringent eligibility criteria and add additional manpower to help with reminders for laboratory follow-up and patient screening. These pivots caused study delays and cost resources. Ultimately, having a timetable forced me to take pause when it became clear I could not finish the study in the allotted time.
Know when to fold ’em
Knowing when to close a study is far easier said than done. The sunk cost fallacy says it is much harder to abandon a project after investing so many resources into it. For us, it was the recruitment timetable that gave us pause. Finishing trial accrual by the end of my advanced fellowship year was wholly unfeasible. When it became clear that nobody in the department could see it through to completion, I was propelled to terminate the study. If there is concern about termination, I suggest sending the protocol, recruitment numbers, and timeline to an outside colleague for a second, unbiased opinion. Review the already compiled data for any notable findings worthy of a smaller publication. It is said, we often learn more from our failures than our successes. The experience described herein – largely in part to my mentors, collaborators, and the patients who put their faith in me – translates to a lasting, invaluable win.
Mentor’s note
Clinical research is hard. Many trainees meet with me to “get involved” in clinical research, and the challenge as a mentor is to identify a project appropriate to the level of training and provide the infrastructure and resources to facilitate success for the motivated trainee. Trainees have various goals of their involvement in research – to foster a relationship in the hopes of receiving a strong letter of support, to facilitate getting into a competitive training program, and/or to publish. My goal as a mentor is to help my trainees reach their goals, but as a clinical researcher, I look for the trainee’s desire to engage with and learn the research process, with the ancillary potential for a letter, for acceptance to a program, or for publication.
This particular study, a randomized, controlled trial of vitamin D in patients with Crohn’s disease, involved an enormous undertaking by a very motivated trainee who took the project from its inception; to putting a thoughtful grant proposal together; to developing a full clinical trial protocol with its ancillary regulatory documents; and obtaining institutional review board approval, statistician input, pharmacy support, and buy-in from faculty and ancillary staff stakeholders. The study ultimately failed because of low enrollment – patients did not want to participate (for reasons elucidated above) – not because of poor design or execution of the myriad components of a prospective clinical trial. Low enrollment has led to the failure of many otherwise excellent studies, including several in our field of IBD.1,2 As a mentor, it is rational to accept blame for the failure of a trainee project; how could I have better foreseen the outcome of this study? Could this have been prevented with more support, more oversight, or more “micromanagement,” to the potential detriment of fostering independence?
Ultimately, the value of clinical research to trainees is multifaceted. If the goal was a first-author publication with high clinical impact, this trial failed. But if the goal was to learn about the clinical trial process, this study was a resounding success. Ultimately, it behooves trainees and their mentors to engage in early, upfront conversations about research. What are the goals? What does success look like? What if the trial fails? By shifting the focus from the success of the project to the success of the mentorship and educational process, even failed projects are resounding successes, upon which future careers can be further developed.
References
1. Kan S et al. When subjects violate the research covenant: Lessons learned from a failed clinical trial of fecal microbiota transplantation. Am J Gastroenterol 2016;111:1508-10.
2. Dassopoulos T et al. Randomised clinical trial: Individualised vs.weight-based dosing of azathioprine in Crohn’s disease. Aliment Pharmacol Ther. 2014 Jan;39(2):163-75.
Dr. Cohen is an inflammatory bowel disease fellow, inflammatory bowel disease center, division of gastroenterology and hepatology, department of medicine, Cedars-Sinai Medical Center, Los Angeles. Dr. Melmed is director, inflammatory bowel disease clinical research and codirector, clinical inflammatory bowel disease, inflammatory bowel disease center, division of gastroenterology and hepatology, department of medicine, Cedars-Sinai Medical Center, Los Angeles.
As a second-year gastroenterology fellow, I designed a prospective, double-blind randomized, controlled trial for vitamin D repletion in patients with Crohn’s disease at a referral inflammatory bowel disease (IBD) center. I had the support of a dedicated research team, several mentors, and a 2-year time frame in which to complete this study. Intellectually curious and academically eager, I labored over a grant application that I did not receive. Under generous financial support from my department, I forged on and opened the trial for enrollment at the start of my advanced IBD fellowship year. However, we experienced recruitment challenges that ultimately led to the study’s premature termination. Through this journey, I gained invaluable experience that will continue to serve me – and, I hope, the reader – as I progress in my career. Below are some important insights gleaned from this experience that may benefit others interested in clinical trial design.
Know your “why” (personally and clinically)
Asked to reflect on their career path, experts tend to recount their “being in the right place at the right time” and having “good mentors.” While luck and good mentorship are necessary, I propose that doing your homework is equally as important. Before ambling down the path of an investigator-led trial, I urge a hard pause to reflect on your “why.” The personal “why” of “getting into fellowship,” “advancement in the department,” or “learning more about the principles of research,” are all valid. But I suggest a deeper dive is in order. Successful clinical trials require resources, a substantial time commitment, lots of sweat and maybe a few tears. In the ideal setting, your trial experience will serve as the foundation for a compelling personal narrative and might help launch a productive clinical research career. With stakes that high, asking the tough questions is critical.
The clinical “why” is just as critical. We press our attendings on why they used this drug or that clip, so don’t be afraid to ask whether this is a space in which others have succeeded. Or conversely, why is there such a large gap in the literature? I remember emailing the world’s expert about my topic because the published data were so murky. He had more questions than answers which, in retrospect, should have raised red flags about the ability to design a sound study. It is equally important to determine if patients are vested in the research question. A successful clinical trial hinges on subject participation, often outside their clinic visit. Patients with complex chronic diseases spend a lot of time navigating the health care system. Participating in a clinical trial needs to be meaningful to them if you want your patients to fully engage. Thoughtfully answering these questions on the front end – for yourself and the study in question – will improve both your experience and the ultimate outcome exponentially.
Identify your village (and listen to them)
Designing a clinical trial truly takes a village, and you need to identify the villagers early. As trainees, the value of mentorship is frequently underscored. But the importance of, and the nuance involved in managing, collaborating, and support cannot be overstated. Meet with a biostatistician to ensure your sample size calculations are correct. Work closely with the research pharmacist to ensure the medication formulation is available; decide on the manner of distribution so as not to inconvenience subjects; create a budget with an experienced manager. Seek out research coordinators often for assistance in creating case report forms, learning appropriate documentation, and crafting responses to Institutional Review Board concerns. Ask clinic personnel about arranging consent or follow-up visits around subjects’ clinic appointments. Present a draft of the protocol to your colleagues, as you will ultimately need them on board to recruit patients. And most importantly, listen to them ... all of them. Get your biases in check and write down all constructive criticism. Thoughtfully address each concern encountered to your satisfaction (and your mentor’s) and present to your village again. Rinse and repeat. Throw nothing under the rug, because if you do, it will eventually rear its head while you are in the throes of the study.
In retrospect, there were concerns raised by faculty and grant reviewers that I did not adequately address. First was the feasibility of screening, recruiting, and enrolling 80 patients during a busy clinical fellowship. While I took this criticism as a reflection of my personal commitment to the project, it was actually a call to consider the impact on clinical (and familial) responsibilities. But as I started enrolling patients, I realized there were logistic issues implied in this suggestion. I could not recruit subjects in Clinic A if I was assigned to see patients at the same time in Clinic B. Patients were not likely to come back another day for study-related discussions. Second, I designed eligibility criteria to make the data as clean as possible. Limiting the study to subjects with Crohn’s disease, in clinical and biochemical remission, without complications of their disease, who also have vitamin deficiency, may be an unrealistic recipe to recruit 80 people in a limited period of time. Finally, I designed laboratory follow-up schedules based on the pharmacokinetics of the drug alone, failing to consider the clinical milieu from which study subjects were recruited. Neglecting the fact that many patients obtain labs with their infusions, my study increased lab draw burden, heaped more patient reminders onto my plate – and more concerning – decreased overall study compliance. In short, trial design cannot be done in a vacuum or by just poring over published data. There are logistical and patient-related considerations that require early input from physicians and clinical staff in order for all the moving parts of a clinical trial to successfully work in harmony.
Create a timetable and follow it
Make a recruitment timetable very early on in the enrollment period. Set up biweekly meetings with mentors to discuss enrollment numbers and reflect on any unforeseen challenges. And be sure to celebrate the wins as well – not matter how small. In our study, falling behind on recruitment goals forced me to amend the stringent eligibility criteria and add additional manpower to help with reminders for laboratory follow-up and patient screening. These pivots caused study delays and cost resources. Ultimately, having a timetable forced me to take pause when it became clear I could not finish the study in the allotted time.
Know when to fold ’em
Knowing when to close a study is far easier said than done. The sunk cost fallacy says it is much harder to abandon a project after investing so many resources into it. For us, it was the recruitment timetable that gave us pause. Finishing trial accrual by the end of my advanced fellowship year was wholly unfeasible. When it became clear that nobody in the department could see it through to completion, I was propelled to terminate the study. If there is concern about termination, I suggest sending the protocol, recruitment numbers, and timeline to an outside colleague for a second, unbiased opinion. Review the already compiled data for any notable findings worthy of a smaller publication. It is said, we often learn more from our failures than our successes. The experience described herein – largely in part to my mentors, collaborators, and the patients who put their faith in me – translates to a lasting, invaluable win.
Mentor’s note
Clinical research is hard. Many trainees meet with me to “get involved” in clinical research, and the challenge as a mentor is to identify a project appropriate to the level of training and provide the infrastructure and resources to facilitate success for the motivated trainee. Trainees have various goals of their involvement in research – to foster a relationship in the hopes of receiving a strong letter of support, to facilitate getting into a competitive training program, and/or to publish. My goal as a mentor is to help my trainees reach their goals, but as a clinical researcher, I look for the trainee’s desire to engage with and learn the research process, with the ancillary potential for a letter, for acceptance to a program, or for publication.
This particular study, a randomized, controlled trial of vitamin D in patients with Crohn’s disease, involved an enormous undertaking by a very motivated trainee who took the project from its inception; to putting a thoughtful grant proposal together; to developing a full clinical trial protocol with its ancillary regulatory documents; and obtaining institutional review board approval, statistician input, pharmacy support, and buy-in from faculty and ancillary staff stakeholders. The study ultimately failed because of low enrollment – patients did not want to participate (for reasons elucidated above) – not because of poor design or execution of the myriad components of a prospective clinical trial. Low enrollment has led to the failure of many otherwise excellent studies, including several in our field of IBD.1,2 As a mentor, it is rational to accept blame for the failure of a trainee project; how could I have better foreseen the outcome of this study? Could this have been prevented with more support, more oversight, or more “micromanagement,” to the potential detriment of fostering independence?
Ultimately, the value of clinical research to trainees is multifaceted. If the goal was a first-author publication with high clinical impact, this trial failed. But if the goal was to learn about the clinical trial process, this study was a resounding success. Ultimately, it behooves trainees and their mentors to engage in early, upfront conversations about research. What are the goals? What does success look like? What if the trial fails? By shifting the focus from the success of the project to the success of the mentorship and educational process, even failed projects are resounding successes, upon which future careers can be further developed.
References
1. Kan S et al. When subjects violate the research covenant: Lessons learned from a failed clinical trial of fecal microbiota transplantation. Am J Gastroenterol 2016;111:1508-10.
2. Dassopoulos T et al. Randomised clinical trial: Individualised vs.weight-based dosing of azathioprine in Crohn’s disease. Aliment Pharmacol Ther. 2014 Jan;39(2):163-75.
Dr. Cohen is an inflammatory bowel disease fellow, inflammatory bowel disease center, division of gastroenterology and hepatology, department of medicine, Cedars-Sinai Medical Center, Los Angeles. Dr. Melmed is director, inflammatory bowel disease clinical research and codirector, clinical inflammatory bowel disease, inflammatory bowel disease center, division of gastroenterology and hepatology, department of medicine, Cedars-Sinai Medical Center, Los Angeles.