User login
Preliminary Evaluation of an Order Template to Improve Diagnosis and Testosterone Therapy of Hypogonadism in Veterans
Testosterone treatment is clinically indicated when a patient presents with symptoms and signs and biochemical evidence of testosterone deficiency, ie, male hypogonadism. Laboratory confirmation of hypogonadism requires repeatedly low serum testosterone concentrations; between 8
Recent studies have reported an increase in testosterone prescriptions and raised concerns regarding health care provider (HCP) prescribing practices despite current clinical practice guidelines from major societies, such as the Endocrine Society. In the US from 2001 to 2011, testosterone use among men aged ≥ 40 years increased more than 3-fold in all age groups.3 Subsequently in the years from 2013 to 2016, prescription rates declined perhaps due to the cardiovascular and stroke concerns.4
In the US Department of Veterans Affairs (VA), new testosterone prescriptions across VA medical centers increased from 20,437 in fiscal year (FY) 2009 to 36,394 in FY 2012. Yet only 3.1% of men who received testosterone therapy had 2 or more low morning total or free testosterone concentrations measured; LH and/or FSH levels assessed; and presence of contraindications to therapy documented. Remarkably, 16.5% of these veterans did not have a testosterone level tested prior to being prescribed testosterone. Among veterans who were prescribed testosterone, 1.4% had a diagnosis of prostate cancer, 7.6% had a diagnosis of obstructive sleep apnea (OSA), and 3.5% had elevated hematocrit at baseline.5 These findings raised concerns of whether the diagnosis and etiology of hypogonadism were appropriately established and risks were considered before testosterone treatment was initiated.5,6
To further understand VA prescribing practices of testosterone therapy, a 2018 VA Office of the Inspector General (OIG) report evaluated the initiation and follow-up of testosterone replacement therapy. The OIG randomly sampled and reviewed 1,091 male patients who filled at least 1 outpatient testosterone prescription from VA in FY 2014 and who did not have a prior testosterone prescription in FY 2013. Patients were followed through September 30, 2015. Within 1 year prior to initiating testosterone, only 1.5% had clinical signs and symptoms of testosterone deficiency documented prior to testosterone testing (76% within 18 months of starting testosterone); only 9.1% of veterans had the recommended measurements of 2 low morning testosterone levels; and only 12% had LH and FSH levels measured. Within 3 to 6 months after starting testosterone therapy, only 24% of veterans were assessed for symptom improvement, and 29% to 33% were evaluated for adverse effects, hematocrit levels and adherence to the therapy. The OIG report concluded that VA HCPs were not adhering to guidelines (referencing the Endocrine Society guidelines) when evaluating and treating veterans with testosterone deficiency.7
Considering the OIG recommendations and need to improve current practices among providers, VA Puget Sound Health Care System (VAPSHCS) in Washington established a multidisciplinary workgroup consisting of an endocrinologist, geriatrician, primary care provider (PCP), pharmacists, VA information technology (IT) specialist, and health products support (HPS) clinical team in the spring of 2019 to assess and improve testosterone prescribing practices.
Methods
A testosterone order template was developed, approved by VAPSHCS Pharmacy and Therapeutics Committee, and implemented on July 1, 2019, at VAPSHCS, a 1a medical facility caring for more than 112,000 veterans. Given its potential risks and the propensity for varied prescribing practices, testosterone was designated as a restricted drug requiring a prior authorization drug request (PADR) and required completion of the testosterone order template in the Computerized Patient Record System (CPRS).
Testosterone Order Template
The testosterone order template had 2 components. Completion of the template for new testosterone orders was required to initiate treatment unless the patient had known organic hypogonadism or was a transgender male. The template ensured documentation of defined symptoms and signs of testosterone deficiency; low serum testosterone levels on at least 2 occasions and LH and FSH concentrations; no contraindications to testosterone treatment; discussion of risks and benefits of therapy; and baseline hematocrit (Figure 1). Relevant educational content (eg, risks and benefits of testosterone) was incorporated in the template. The second template was required for the first renewal of testosterone to document adherence to or reason for discontinuation of testosterone; improvement of symptoms and signs; and confirm monitoring hematocrit and testosterone levels during treatment.
Prior to implementation, the PADR template was introduced to HCPs at 2 chief-of-medicine rounds on the diagnosis and evaluation of hypogonadism by a pharmacist and endocrinologist. These educational sessions used case examples and discussions to teach the appropriate use of testosterone therapy in men with hypogonadism. The target audience was PCPs, residents, and other specialists who might prescribe testosterone.
Retrospective Chart Review
To assess the impact of the new testosterone order template on adherence to OIG recommendations, a retrospective chart review was completed comparing the appropriateness of initiating testosterone replacement therapy pretemplate period (July 1 to December 31, 2018) vs posttemplate period (July 1 to December 31, 2019). Inclusion and exclusion criteria were modeled after the 2018 OIG report to allow for comparison with the OIG study population. Eligible veterans in each time period included males who received a new testosterone prescription without having been prescribed testosterone in the previous 12 months. Exclusion criteria included community care network prescriptions (CCNRx); current testosterone prescription from a different VA site; clinic administration of testosterone in the previous 12 months; an organic hypogonadism (ie, Klinefelter syndrome) or gender dysphoria diagnosis; and whether the testosterone prescription was never dispensed (PADR was denied or veteran never had the prescription filled). Veterans who met the inclusion criteria in CPRS were identified by an algorithm developed by the VAPSHCS pharmacoeconomist.
Determining the appropriateness of testosterone prescribing, such as symptoms and laboratory measurements to confirm the diagnosis of hypogonadism, was based on the OIG report and Endocrine Society guidelines. A chart review of the 12 months before testosterone prescribing was completed for each veteran, assessing for documentation of symptoms of testosterone deficiency and laboratory measurements of serum testosterone, LH, and FSH. Also, documentation of a discussion of risks and benefits of testosterone therapy in the 3 months before prescribing was assessed, which matched the timeframe in the VA OIG report.
Interim Analysis
After initial template implementation, the multidisciplinary workgroup reconvened for a preplanned interim analysis in November 2019. The evaluation at this meeting revealed multiple order pathways in CPRS that were not linked to the PADR testosterone order template. Testosterone could be ordered in the generic order dialog, medications by drug class, and medications by alphabet, and endocrinology specialty menus without prompting to complete the testosterone order template or redirection to the restricted drug menu (Figure 2). These alternative testosterone ordering pathways were removed in early December 2019 and additional data collection was conducted for 3 months after discontinuation of alternative order pathways, the posttemplate/no alternative ordering pathways period, from December 7, 2019 to February 29, 2020.
Exclusion of Previous Testosterone Prescriptions Predating Chart Review Period, Subgroup Analysis
In the OIG report and the initial retrospective chart review, only veterans without a testosterone prescription in the previous 12 months were evaluated. To assess whether a previous testosterone prescription influenced completion of the PADR and order template, a further subgroup analysis was conducted that excluded veterans who had a previous testosterone prescription at any time before the chart review periods. Therefore, “new testosterone prescription” refers to a veteran who never had a history of being on testosterone vs “former testosterone prescription,” meaning a patient could have had a previous testosterone prescription > 1 year prior to a new VA testosterone prescription.
Results
One hundred seventy-five veterans with a new testosterone prescription were identified in the pretemplate period; of these 80 (46%) met eligibility criteria; only 20 eligible veterans (25%) had a completed PADR (Figure 3). Ninety-one veterans with a new testosterone prescription were identified in the posttemplate period of which 41 (46%) veterans were eligible; 18 eligible veterans (44%) had a completed PADR, but only 7 (17%) had a completed testosterone order template.
After excluding veterans who had alternative ordering pathways for testosterone, 46 veterans were identified in the posttemplate/no alternative ordering pathways period of which 19 (41%) veterans were eligible. Compared with the posttemplate period, a higher proportion of eligible veterans, 68% (13) had a completed PADR, and 58% (11) had a testosterone order template during the posttemplate/no alternative ordering pathways period.
Compared with the OIG report findings, a similar percentage of veterans at VAPSHCS in the pretemplate period had documented clinical symptoms of testosterone deficiency and documented discussion of risks and benefits of testosterone therapy (Figure 4). However, a higher percentage of veterans had biochemical confirmation of testosterone deficiency with ≥ 2 low testosterone levels and evaluation of LH and FSH levels in the pretemplate period (23%) vs that in the OIG report (2%).
Compared with the pretemplate period, activation of the testosterone ordering template in the posttemplate period (Figure 4) had little effect on documented clinical symptoms and discussion of risks and benefits of testosterone treatment. However, the percentage of veterans who had ≥ 2 low testosterone levels and gonadotropins tested was higher in the posttemplate period (41%) vs both the pretemplate period and OIG report.
After removing alternative ordering pathways of testosterone, the percentages of veterans who had documented clinical symptoms, discussion of risks and benefits of testosterone, and ≥ 2 low testosterone levels and gonadotropin tests performed were similar in the posttemplate/no alternative ordering pathways vs posttemplate period.
Excluding veterans who had previously received a former testosterone prescription at any time prior to chart review periods, this subgroup analysis resulted in greater adherence to Endocrine Society guidelines for testosterone treatment with introduction of the testosterone order template, particularly after removal of alternative ordering pathway (Figure 5). With the exclusion of veterans who formerly received testosterone prescriptions, the percentages of veterans who had documented clinical symptoms, discussion of risks and benefits, and ≥ 2 low testosterone levels with gonadotropin tests were higher (100%, 57%, and 71%, respectively) in the posttemplate/no alternative ordering pathways period, compared with the pretemplate period (86%, 30%, and 32%, respectively).
Discussion
The 2018 OIG report found that VA practitioners demonstrated poor adherence to evidence-based clinical practice guidelines for testosterone treatment in men with hypogonadism. Based on OIG recommendations, we developed a PADR testosterone ordering template to help HCPs improve practice by better adherence to guidelines for the diagnosis and treatment of hypogonadism in veterans. Before implementation of the PADR template, the percentage of veterans at VAPSHCS who had biochemical confirmation of hypogonadism was higher than that in the OIG report. Activation of the PADR testosterone ordering template (with or without removal of options for alternative ordering pathways of testosterone) resulted only in an improvement of laboratory confirmation and evaluation of etiology of hypogonadism. This is when we reasoned that clinicians may have access to prior records and laboratory testing beyond just the past year, and this information may have influenced their use of the PADR template. Subsequently, with exclusion of veterans who were previously prescribed testosterone, implementation of the PADR testosterone order template improved documentation of symptoms of testosterone deficiency, discussion of risks and benefits of testosterone therapy, and biochemical diagnosis and evaluation of hypogonadism relative to the period before implementation.
The lack of effects of implementing the testosterone order template on documentation of symptoms of testosterone deficiency and discussion of risks and benefits of testosterone therapy may be due to local expertise resulting in the relatively high adherence to these guideline recommendations at VAPSHCS before activation of the template vs that in the OIG report. The template improved documentation of the diagnosis and evaluation of hypogonadism for genuinely new testosterone prescriptions in veterans without a history of testosterone prescriptions; while those with a previous prescription had limited improvement. It is possible that in veterans who had testosterone prescribed previously, HCPs may have assumed or had bias that the diagnosis and evaluation of hypogonadism originally made was adequate. This finding underscores the need to develop strategies for reviewing PADR requests where there is historical testosterone use. Perhaps a clinical team member, such as a clinical pharmacist, with the background and training in guidelines for the evaluation of hypogonadism could review PADR requests in veterans with previous testosterone use.
Removal of alternative ordering pathways for testosterone increased the completion rate of PADR requests and the testosterone ordering template, although the latter was not completed in one-third of veterans. Possible reasons for HCPs’ suboptimal completion of the testosterone template despite the PADR initiation include clinicians’ lack of willingness to read the PADR completely and familiarize themselves with the clinical guidelines due to workload demands of PCPs. In addition there maybe pressure from patients to receive testosterone for age-related symptoms due to heavy marketing. In addition, there may have been pharmacists who reviewed the PADR and approved the incomplete testosterone template. At VAPSHCS there were up to 40 pharmacists during different periods reviewing the testosterone PADRs. Likely, not everyone was completely familiar with this implementation process, and a possible future consideration would be further education to staff pharmacists who are verifying these prescriptions. There were several advantages to using this new testosterone order template when HCPs attempted to order a prescription. First, they were prompted to complete the PADR. Subsequently, a pharmacist reviewed the template and approved or rejected the prescription if the template was incomplete. The completed template served as documentation in the electronic health record for the prescribing HCP. The template was constructed to populate the required laboratory tests for ease of use and documentation. In addition, educational information regarding the symptoms and signs of testosterone deficiency, laboratory tests needed to confirm and evaluate hypogonadism, contraindications to testosterone treatment, and risks and benefits of therapy were incorporated into the template to assist HCPs in understanding the requirements for a complete diagnosis and evaluation. Finally, on completion of the template, HCPs were able to order testosterone via link to various testosterone formulations.
Before its implementation, the PADR testosterone order template was introduced to PCPs and internal medicine residents at 2 case-based conferences aimed at the diagnosis and treatment of male hypogonadism. These conferences were well received and helped launch the testosterone PADR template at VAPSHCS. Similar outreach to HCPs who prescribe testosterone is highly recommended in other VA facilities before implementation of the testosterone ordering template. It is possible that more targeted education to other HCPs would have resulted in greater use of the testosterone ordering template and adherence to clinical practice guidelines.
The VAPSHCS multidisciplinary workgroup was essential for the development, implementation, evaluation, and revision of the PADR and testosterone ordering template. The workgroup met routinely to follow up on the ease of installation in CPRS and discuss technical corrections that were needed. This was an essential for quality improvement, as loopholes in CPRS were identified where the HCP could order testosterone without being prompted to use the new PADR testosterone order template (alternative ordering pathways). The workgroup swiftly informed the IT specialist and HPS team to remove alternative ordering pathways of testosterone. Continuous quality improvement evaluations are highly recommended during implementation of the template in other facilities to accommodate specific local modifications that might be needed.
After February 2020 due to the COVID-19 pandemic, the National VA Pharmacy and Medication Board halted PADR requirements. As a result, further evaluation of the New Testosterone Order template and planned initial assessment of First Renewal Testosterone Order template could not be performed. In addition, due to the COVID-19 pandemic, there was restricted in-person outpatient visits and reduced adjustments to prescribing practices. To address recommendations made in the OIG report, the VAPSHCS testosterone order template was modified into a clinical reminder dialog format by a VA National IT Specialist and HPS team, tested for usability at several VA test sites and approved by the National Clinical Template Workgroup for implementation nationally across all VAs. The National Endocrinology Ambulatory Council Workgroup will ensure that this template is adopted in a similar format when the new electronic health record system Cerner is introduced to the VA.
Conclusions
The creation and implementation of a PADR testosterone order template may be a beneficial approach to improve the diagnosis of hypogonadism and facilitate appropriate use of testosterone therapy in veterans in accordance with established clinical practice guidelines, particularly in veterans without any prior testosterone use. Key future strategies to improve testosterone prescribing should focus on identifying clinical team members, such as a local clinical pharmacist, to review and steward PADR requests to ensure that testosterone is indicated, and treatment is appropriately monitored.
1. Bhasin S, Cunningham GR, Hayes FJ, Matsumoto AM, Snyder PJ, Swerdloff RS, Montori VM; Task Force, Endocrine Society. Testosterone therapy in men with androgen deficiency syndromes: an Endocrine Society clinical practice guideline. J Clin Endocrinol Metab. 2010;95(6):2536-2559. doi:10.1210/jc.2009-2354
2. Grossmann M, Matsumoto AM. A perspective on middle-aged and older men with functional hypogonadism: focus on holistic management. J Clin Endocrinol Metab. 2017;102(3):1067-1075. doi:10.1210/jc.2016-3580
3. Baillargeon J, Urban RJ, Kuo YF, et al. Screening and monitoring in men prescribed testosterone therapy in the US, 2001-2010. Public Health Rep. 2015;130(2):143-152. doi:10.1177/003335491513000207
4. Baillargeon J, Kuo Y, Westra JR, Urban RJ, Goodwin JS. Testosterone prescribing in the United States, 2002-2016. JAMA. 2018;320(2):200-202. doi:10.1001/jama.2018.7999
5. Jasuja GK, Bhasin S, Reisman JI, Berlowitz DR, Rose AJ. Ascertainment of testosterone prescribing practices in the VA. Med Care. 2015;53(9):746-52. doi:10.1097/MLR.0000000000000398
6. Jasuja GK, Bhasin S, Rose AJ. Patterns of testosterone prescription overuse. Curr Opin Endocrinol Diabetes Obes. 2017;24(3):240-245. doi:10.1097/MED.0000000000000336
7. US Department of Veterans Affairs, Office of Inspector General. Office of Healthcare Inspections. Report No. 15-03215-154. Published April 11, 2018. Accessed February 24, 2021. https://www.va.gov/oig/pubs/VAOIG-15-03215-154.pdf
Testosterone treatment is clinically indicated when a patient presents with symptoms and signs and biochemical evidence of testosterone deficiency, ie, male hypogonadism. Laboratory confirmation of hypogonadism requires repeatedly low serum testosterone concentrations; between 8
Recent studies have reported an increase in testosterone prescriptions and raised concerns regarding health care provider (HCP) prescribing practices despite current clinical practice guidelines from major societies, such as the Endocrine Society. In the US from 2001 to 2011, testosterone use among men aged ≥ 40 years increased more than 3-fold in all age groups.3 Subsequently in the years from 2013 to 2016, prescription rates declined perhaps due to the cardiovascular and stroke concerns.4
In the US Department of Veterans Affairs (VA), new testosterone prescriptions across VA medical centers increased from 20,437 in fiscal year (FY) 2009 to 36,394 in FY 2012. Yet only 3.1% of men who received testosterone therapy had 2 or more low morning total or free testosterone concentrations measured; LH and/or FSH levels assessed; and presence of contraindications to therapy documented. Remarkably, 16.5% of these veterans did not have a testosterone level tested prior to being prescribed testosterone. Among veterans who were prescribed testosterone, 1.4% had a diagnosis of prostate cancer, 7.6% had a diagnosis of obstructive sleep apnea (OSA), and 3.5% had elevated hematocrit at baseline.5 These findings raised concerns of whether the diagnosis and etiology of hypogonadism were appropriately established and risks were considered before testosterone treatment was initiated.5,6
To further understand VA prescribing practices of testosterone therapy, a 2018 VA Office of the Inspector General (OIG) report evaluated the initiation and follow-up of testosterone replacement therapy. The OIG randomly sampled and reviewed 1,091 male patients who filled at least 1 outpatient testosterone prescription from VA in FY 2014 and who did not have a prior testosterone prescription in FY 2013. Patients were followed through September 30, 2015. Within 1 year prior to initiating testosterone, only 1.5% had clinical signs and symptoms of testosterone deficiency documented prior to testosterone testing (76% within 18 months of starting testosterone); only 9.1% of veterans had the recommended measurements of 2 low morning testosterone levels; and only 12% had LH and FSH levels measured. Within 3 to 6 months after starting testosterone therapy, only 24% of veterans were assessed for symptom improvement, and 29% to 33% were evaluated for adverse effects, hematocrit levels and adherence to the therapy. The OIG report concluded that VA HCPs were not adhering to guidelines (referencing the Endocrine Society guidelines) when evaluating and treating veterans with testosterone deficiency.7
Considering the OIG recommendations and need to improve current practices among providers, VA Puget Sound Health Care System (VAPSHCS) in Washington established a multidisciplinary workgroup consisting of an endocrinologist, geriatrician, primary care provider (PCP), pharmacists, VA information technology (IT) specialist, and health products support (HPS) clinical team in the spring of 2019 to assess and improve testosterone prescribing practices.
Methods
A testosterone order template was developed, approved by VAPSHCS Pharmacy and Therapeutics Committee, and implemented on July 1, 2019, at VAPSHCS, a 1a medical facility caring for more than 112,000 veterans. Given its potential risks and the propensity for varied prescribing practices, testosterone was designated as a restricted drug requiring a prior authorization drug request (PADR) and required completion of the testosterone order template in the Computerized Patient Record System (CPRS).
Testosterone Order Template
The testosterone order template had 2 components. Completion of the template for new testosterone orders was required to initiate treatment unless the patient had known organic hypogonadism or was a transgender male. The template ensured documentation of defined symptoms and signs of testosterone deficiency; low serum testosterone levels on at least 2 occasions and LH and FSH concentrations; no contraindications to testosterone treatment; discussion of risks and benefits of therapy; and baseline hematocrit (Figure 1). Relevant educational content (eg, risks and benefits of testosterone) was incorporated in the template. The second template was required for the first renewal of testosterone to document adherence to or reason for discontinuation of testosterone; improvement of symptoms and signs; and confirm monitoring hematocrit and testosterone levels during treatment.
Prior to implementation, the PADR template was introduced to HCPs at 2 chief-of-medicine rounds on the diagnosis and evaluation of hypogonadism by a pharmacist and endocrinologist. These educational sessions used case examples and discussions to teach the appropriate use of testosterone therapy in men with hypogonadism. The target audience was PCPs, residents, and other specialists who might prescribe testosterone.
Retrospective Chart Review
To assess the impact of the new testosterone order template on adherence to OIG recommendations, a retrospective chart review was completed comparing the appropriateness of initiating testosterone replacement therapy pretemplate period (July 1 to December 31, 2018) vs posttemplate period (July 1 to December 31, 2019). Inclusion and exclusion criteria were modeled after the 2018 OIG report to allow for comparison with the OIG study population. Eligible veterans in each time period included males who received a new testosterone prescription without having been prescribed testosterone in the previous 12 months. Exclusion criteria included community care network prescriptions (CCNRx); current testosterone prescription from a different VA site; clinic administration of testosterone in the previous 12 months; an organic hypogonadism (ie, Klinefelter syndrome) or gender dysphoria diagnosis; and whether the testosterone prescription was never dispensed (PADR was denied or veteran never had the prescription filled). Veterans who met the inclusion criteria in CPRS were identified by an algorithm developed by the VAPSHCS pharmacoeconomist.
Determining the appropriateness of testosterone prescribing, such as symptoms and laboratory measurements to confirm the diagnosis of hypogonadism, was based on the OIG report and Endocrine Society guidelines. A chart review of the 12 months before testosterone prescribing was completed for each veteran, assessing for documentation of symptoms of testosterone deficiency and laboratory measurements of serum testosterone, LH, and FSH. Also, documentation of a discussion of risks and benefits of testosterone therapy in the 3 months before prescribing was assessed, which matched the timeframe in the VA OIG report.
Interim Analysis
After initial template implementation, the multidisciplinary workgroup reconvened for a preplanned interim analysis in November 2019. The evaluation at this meeting revealed multiple order pathways in CPRS that were not linked to the PADR testosterone order template. Testosterone could be ordered in the generic order dialog, medications by drug class, and medications by alphabet, and endocrinology specialty menus without prompting to complete the testosterone order template or redirection to the restricted drug menu (Figure 2). These alternative testosterone ordering pathways were removed in early December 2019 and additional data collection was conducted for 3 months after discontinuation of alternative order pathways, the posttemplate/no alternative ordering pathways period, from December 7, 2019 to February 29, 2020.
Exclusion of Previous Testosterone Prescriptions Predating Chart Review Period, Subgroup Analysis
In the OIG report and the initial retrospective chart review, only veterans without a testosterone prescription in the previous 12 months were evaluated. To assess whether a previous testosterone prescription influenced completion of the PADR and order template, a further subgroup analysis was conducted that excluded veterans who had a previous testosterone prescription at any time before the chart review periods. Therefore, “new testosterone prescription” refers to a veteran who never had a history of being on testosterone vs “former testosterone prescription,” meaning a patient could have had a previous testosterone prescription > 1 year prior to a new VA testosterone prescription.
Results
One hundred seventy-five veterans with a new testosterone prescription were identified in the pretemplate period; of these 80 (46%) met eligibility criteria; only 20 eligible veterans (25%) had a completed PADR (Figure 3). Ninety-one veterans with a new testosterone prescription were identified in the posttemplate period of which 41 (46%) veterans were eligible; 18 eligible veterans (44%) had a completed PADR, but only 7 (17%) had a completed testosterone order template.
After excluding veterans who had alternative ordering pathways for testosterone, 46 veterans were identified in the posttemplate/no alternative ordering pathways period of which 19 (41%) veterans were eligible. Compared with the posttemplate period, a higher proportion of eligible veterans, 68% (13) had a completed PADR, and 58% (11) had a testosterone order template during the posttemplate/no alternative ordering pathways period.
Compared with the OIG report findings, a similar percentage of veterans at VAPSHCS in the pretemplate period had documented clinical symptoms of testosterone deficiency and documented discussion of risks and benefits of testosterone therapy (Figure 4). However, a higher percentage of veterans had biochemical confirmation of testosterone deficiency with ≥ 2 low testosterone levels and evaluation of LH and FSH levels in the pretemplate period (23%) vs that in the OIG report (2%).
Compared with the pretemplate period, activation of the testosterone ordering template in the posttemplate period (Figure 4) had little effect on documented clinical symptoms and discussion of risks and benefits of testosterone treatment. However, the percentage of veterans who had ≥ 2 low testosterone levels and gonadotropins tested was higher in the posttemplate period (41%) vs both the pretemplate period and OIG report.
After removing alternative ordering pathways of testosterone, the percentages of veterans who had documented clinical symptoms, discussion of risks and benefits of testosterone, and ≥ 2 low testosterone levels and gonadotropin tests performed were similar in the posttemplate/no alternative ordering pathways vs posttemplate period.
Excluding veterans who had previously received a former testosterone prescription at any time prior to chart review periods, this subgroup analysis resulted in greater adherence to Endocrine Society guidelines for testosterone treatment with introduction of the testosterone order template, particularly after removal of alternative ordering pathway (Figure 5). With the exclusion of veterans who formerly received testosterone prescriptions, the percentages of veterans who had documented clinical symptoms, discussion of risks and benefits, and ≥ 2 low testosterone levels with gonadotropin tests were higher (100%, 57%, and 71%, respectively) in the posttemplate/no alternative ordering pathways period, compared with the pretemplate period (86%, 30%, and 32%, respectively).
Discussion
The 2018 OIG report found that VA practitioners demonstrated poor adherence to evidence-based clinical practice guidelines for testosterone treatment in men with hypogonadism. Based on OIG recommendations, we developed a PADR testosterone ordering template to help HCPs improve practice by better adherence to guidelines for the diagnosis and treatment of hypogonadism in veterans. Before implementation of the PADR template, the percentage of veterans at VAPSHCS who had biochemical confirmation of hypogonadism was higher than that in the OIG report. Activation of the PADR testosterone ordering template (with or without removal of options for alternative ordering pathways of testosterone) resulted only in an improvement of laboratory confirmation and evaluation of etiology of hypogonadism. This is when we reasoned that clinicians may have access to prior records and laboratory testing beyond just the past year, and this information may have influenced their use of the PADR template. Subsequently, with exclusion of veterans who were previously prescribed testosterone, implementation of the PADR testosterone order template improved documentation of symptoms of testosterone deficiency, discussion of risks and benefits of testosterone therapy, and biochemical diagnosis and evaluation of hypogonadism relative to the period before implementation.
The lack of effects of implementing the testosterone order template on documentation of symptoms of testosterone deficiency and discussion of risks and benefits of testosterone therapy may be due to local expertise resulting in the relatively high adherence to these guideline recommendations at VAPSHCS before activation of the template vs that in the OIG report. The template improved documentation of the diagnosis and evaluation of hypogonadism for genuinely new testosterone prescriptions in veterans without a history of testosterone prescriptions; while those with a previous prescription had limited improvement. It is possible that in veterans who had testosterone prescribed previously, HCPs may have assumed or had bias that the diagnosis and evaluation of hypogonadism originally made was adequate. This finding underscores the need to develop strategies for reviewing PADR requests where there is historical testosterone use. Perhaps a clinical team member, such as a clinical pharmacist, with the background and training in guidelines for the evaluation of hypogonadism could review PADR requests in veterans with previous testosterone use.
Removal of alternative ordering pathways for testosterone increased the completion rate of PADR requests and the testosterone ordering template, although the latter was not completed in one-third of veterans. Possible reasons for HCPs’ suboptimal completion of the testosterone template despite the PADR initiation include clinicians’ lack of willingness to read the PADR completely and familiarize themselves with the clinical guidelines due to workload demands of PCPs. In addition there maybe pressure from patients to receive testosterone for age-related symptoms due to heavy marketing. In addition, there may have been pharmacists who reviewed the PADR and approved the incomplete testosterone template. At VAPSHCS there were up to 40 pharmacists during different periods reviewing the testosterone PADRs. Likely, not everyone was completely familiar with this implementation process, and a possible future consideration would be further education to staff pharmacists who are verifying these prescriptions. There were several advantages to using this new testosterone order template when HCPs attempted to order a prescription. First, they were prompted to complete the PADR. Subsequently, a pharmacist reviewed the template and approved or rejected the prescription if the template was incomplete. The completed template served as documentation in the electronic health record for the prescribing HCP. The template was constructed to populate the required laboratory tests for ease of use and documentation. In addition, educational information regarding the symptoms and signs of testosterone deficiency, laboratory tests needed to confirm and evaluate hypogonadism, contraindications to testosterone treatment, and risks and benefits of therapy were incorporated into the template to assist HCPs in understanding the requirements for a complete diagnosis and evaluation. Finally, on completion of the template, HCPs were able to order testosterone via link to various testosterone formulations.
Before its implementation, the PADR testosterone order template was introduced to PCPs and internal medicine residents at 2 case-based conferences aimed at the diagnosis and treatment of male hypogonadism. These conferences were well received and helped launch the testosterone PADR template at VAPSHCS. Similar outreach to HCPs who prescribe testosterone is highly recommended in other VA facilities before implementation of the testosterone ordering template. It is possible that more targeted education to other HCPs would have resulted in greater use of the testosterone ordering template and adherence to clinical practice guidelines.
The VAPSHCS multidisciplinary workgroup was essential for the development, implementation, evaluation, and revision of the PADR and testosterone ordering template. The workgroup met routinely to follow up on the ease of installation in CPRS and discuss technical corrections that were needed. This was an essential for quality improvement, as loopholes in CPRS were identified where the HCP could order testosterone without being prompted to use the new PADR testosterone order template (alternative ordering pathways). The workgroup swiftly informed the IT specialist and HPS team to remove alternative ordering pathways of testosterone. Continuous quality improvement evaluations are highly recommended during implementation of the template in other facilities to accommodate specific local modifications that might be needed.
After February 2020 due to the COVID-19 pandemic, the National VA Pharmacy and Medication Board halted PADR requirements. As a result, further evaluation of the New Testosterone Order template and planned initial assessment of First Renewal Testosterone Order template could not be performed. In addition, due to the COVID-19 pandemic, there was restricted in-person outpatient visits and reduced adjustments to prescribing practices. To address recommendations made in the OIG report, the VAPSHCS testosterone order template was modified into a clinical reminder dialog format by a VA National IT Specialist and HPS team, tested for usability at several VA test sites and approved by the National Clinical Template Workgroup for implementation nationally across all VAs. The National Endocrinology Ambulatory Council Workgroup will ensure that this template is adopted in a similar format when the new electronic health record system Cerner is introduced to the VA.
Conclusions
The creation and implementation of a PADR testosterone order template may be a beneficial approach to improve the diagnosis of hypogonadism and facilitate appropriate use of testosterone therapy in veterans in accordance with established clinical practice guidelines, particularly in veterans without any prior testosterone use. Key future strategies to improve testosterone prescribing should focus on identifying clinical team members, such as a local clinical pharmacist, to review and steward PADR requests to ensure that testosterone is indicated, and treatment is appropriately monitored.
Testosterone treatment is clinically indicated when a patient presents with symptoms and signs and biochemical evidence of testosterone deficiency, ie, male hypogonadism. Laboratory confirmation of hypogonadism requires repeatedly low serum testosterone concentrations; between 8
Recent studies have reported an increase in testosterone prescriptions and raised concerns regarding health care provider (HCP) prescribing practices despite current clinical practice guidelines from major societies, such as the Endocrine Society. In the US from 2001 to 2011, testosterone use among men aged ≥ 40 years increased more than 3-fold in all age groups.3 Subsequently in the years from 2013 to 2016, prescription rates declined perhaps due to the cardiovascular and stroke concerns.4
In the US Department of Veterans Affairs (VA), new testosterone prescriptions across VA medical centers increased from 20,437 in fiscal year (FY) 2009 to 36,394 in FY 2012. Yet only 3.1% of men who received testosterone therapy had 2 or more low morning total or free testosterone concentrations measured; LH and/or FSH levels assessed; and presence of contraindications to therapy documented. Remarkably, 16.5% of these veterans did not have a testosterone level tested prior to being prescribed testosterone. Among veterans who were prescribed testosterone, 1.4% had a diagnosis of prostate cancer, 7.6% had a diagnosis of obstructive sleep apnea (OSA), and 3.5% had elevated hematocrit at baseline.5 These findings raised concerns of whether the diagnosis and etiology of hypogonadism were appropriately established and risks were considered before testosterone treatment was initiated.5,6
To further understand VA prescribing practices of testosterone therapy, a 2018 VA Office of the Inspector General (OIG) report evaluated the initiation and follow-up of testosterone replacement therapy. The OIG randomly sampled and reviewed 1,091 male patients who filled at least 1 outpatient testosterone prescription from VA in FY 2014 and who did not have a prior testosterone prescription in FY 2013. Patients were followed through September 30, 2015. Within 1 year prior to initiating testosterone, only 1.5% had clinical signs and symptoms of testosterone deficiency documented prior to testosterone testing (76% within 18 months of starting testosterone); only 9.1% of veterans had the recommended measurements of 2 low morning testosterone levels; and only 12% had LH and FSH levels measured. Within 3 to 6 months after starting testosterone therapy, only 24% of veterans were assessed for symptom improvement, and 29% to 33% were evaluated for adverse effects, hematocrit levels and adherence to the therapy. The OIG report concluded that VA HCPs were not adhering to guidelines (referencing the Endocrine Society guidelines) when evaluating and treating veterans with testosterone deficiency.7
Considering the OIG recommendations and need to improve current practices among providers, VA Puget Sound Health Care System (VAPSHCS) in Washington established a multidisciplinary workgroup consisting of an endocrinologist, geriatrician, primary care provider (PCP), pharmacists, VA information technology (IT) specialist, and health products support (HPS) clinical team in the spring of 2019 to assess and improve testosterone prescribing practices.
Methods
A testosterone order template was developed, approved by VAPSHCS Pharmacy and Therapeutics Committee, and implemented on July 1, 2019, at VAPSHCS, a 1a medical facility caring for more than 112,000 veterans. Given its potential risks and the propensity for varied prescribing practices, testosterone was designated as a restricted drug requiring a prior authorization drug request (PADR) and required completion of the testosterone order template in the Computerized Patient Record System (CPRS).
Testosterone Order Template
The testosterone order template had 2 components. Completion of the template for new testosterone orders was required to initiate treatment unless the patient had known organic hypogonadism or was a transgender male. The template ensured documentation of defined symptoms and signs of testosterone deficiency; low serum testosterone levels on at least 2 occasions and LH and FSH concentrations; no contraindications to testosterone treatment; discussion of risks and benefits of therapy; and baseline hematocrit (Figure 1). Relevant educational content (eg, risks and benefits of testosterone) was incorporated in the template. The second template was required for the first renewal of testosterone to document adherence to or reason for discontinuation of testosterone; improvement of symptoms and signs; and confirm monitoring hematocrit and testosterone levels during treatment.
Prior to implementation, the PADR template was introduced to HCPs at 2 chief-of-medicine rounds on the diagnosis and evaluation of hypogonadism by a pharmacist and endocrinologist. These educational sessions used case examples and discussions to teach the appropriate use of testosterone therapy in men with hypogonadism. The target audience was PCPs, residents, and other specialists who might prescribe testosterone.
Retrospective Chart Review
To assess the impact of the new testosterone order template on adherence to OIG recommendations, a retrospective chart review was completed comparing the appropriateness of initiating testosterone replacement therapy pretemplate period (July 1 to December 31, 2018) vs posttemplate period (July 1 to December 31, 2019). Inclusion and exclusion criteria were modeled after the 2018 OIG report to allow for comparison with the OIG study population. Eligible veterans in each time period included males who received a new testosterone prescription without having been prescribed testosterone in the previous 12 months. Exclusion criteria included community care network prescriptions (CCNRx); current testosterone prescription from a different VA site; clinic administration of testosterone in the previous 12 months; an organic hypogonadism (ie, Klinefelter syndrome) or gender dysphoria diagnosis; and whether the testosterone prescription was never dispensed (PADR was denied or veteran never had the prescription filled). Veterans who met the inclusion criteria in CPRS were identified by an algorithm developed by the VAPSHCS pharmacoeconomist.
Determining the appropriateness of testosterone prescribing, such as symptoms and laboratory measurements to confirm the diagnosis of hypogonadism, was based on the OIG report and Endocrine Society guidelines. A chart review of the 12 months before testosterone prescribing was completed for each veteran, assessing for documentation of symptoms of testosterone deficiency and laboratory measurements of serum testosterone, LH, and FSH. Also, documentation of a discussion of risks and benefits of testosterone therapy in the 3 months before prescribing was assessed, which matched the timeframe in the VA OIG report.
Interim Analysis
After initial template implementation, the multidisciplinary workgroup reconvened for a preplanned interim analysis in November 2019. The evaluation at this meeting revealed multiple order pathways in CPRS that were not linked to the PADR testosterone order template. Testosterone could be ordered in the generic order dialog, medications by drug class, and medications by alphabet, and endocrinology specialty menus without prompting to complete the testosterone order template or redirection to the restricted drug menu (Figure 2). These alternative testosterone ordering pathways were removed in early December 2019 and additional data collection was conducted for 3 months after discontinuation of alternative order pathways, the posttemplate/no alternative ordering pathways period, from December 7, 2019 to February 29, 2020.
Exclusion of Previous Testosterone Prescriptions Predating Chart Review Period, Subgroup Analysis
In the OIG report and the initial retrospective chart review, only veterans without a testosterone prescription in the previous 12 months were evaluated. To assess whether a previous testosterone prescription influenced completion of the PADR and order template, a further subgroup analysis was conducted that excluded veterans who had a previous testosterone prescription at any time before the chart review periods. Therefore, “new testosterone prescription” refers to a veteran who never had a history of being on testosterone vs “former testosterone prescription,” meaning a patient could have had a previous testosterone prescription > 1 year prior to a new VA testosterone prescription.
Results
One hundred seventy-five veterans with a new testosterone prescription were identified in the pretemplate period; of these 80 (46%) met eligibility criteria; only 20 eligible veterans (25%) had a completed PADR (Figure 3). Ninety-one veterans with a new testosterone prescription were identified in the posttemplate period of which 41 (46%) veterans were eligible; 18 eligible veterans (44%) had a completed PADR, but only 7 (17%) had a completed testosterone order template.
After excluding veterans who had alternative ordering pathways for testosterone, 46 veterans were identified in the posttemplate/no alternative ordering pathways period of which 19 (41%) veterans were eligible. Compared with the posttemplate period, a higher proportion of eligible veterans, 68% (13) had a completed PADR, and 58% (11) had a testosterone order template during the posttemplate/no alternative ordering pathways period.
Compared with the OIG report findings, a similar percentage of veterans at VAPSHCS in the pretemplate period had documented clinical symptoms of testosterone deficiency and documented discussion of risks and benefits of testosterone therapy (Figure 4). However, a higher percentage of veterans had biochemical confirmation of testosterone deficiency with ≥ 2 low testosterone levels and evaluation of LH and FSH levels in the pretemplate period (23%) vs that in the OIG report (2%).
Compared with the pretemplate period, activation of the testosterone ordering template in the posttemplate period (Figure 4) had little effect on documented clinical symptoms and discussion of risks and benefits of testosterone treatment. However, the percentage of veterans who had ≥ 2 low testosterone levels and gonadotropins tested was higher in the posttemplate period (41%) vs both the pretemplate period and OIG report.
After removing alternative ordering pathways of testosterone, the percentages of veterans who had documented clinical symptoms, discussion of risks and benefits of testosterone, and ≥ 2 low testosterone levels and gonadotropin tests performed were similar in the posttemplate/no alternative ordering pathways vs posttemplate period.
Excluding veterans who had previously received a former testosterone prescription at any time prior to chart review periods, this subgroup analysis resulted in greater adherence to Endocrine Society guidelines for testosterone treatment with introduction of the testosterone order template, particularly after removal of alternative ordering pathway (Figure 5). With the exclusion of veterans who formerly received testosterone prescriptions, the percentages of veterans who had documented clinical symptoms, discussion of risks and benefits, and ≥ 2 low testosterone levels with gonadotropin tests were higher (100%, 57%, and 71%, respectively) in the posttemplate/no alternative ordering pathways period, compared with the pretemplate period (86%, 30%, and 32%, respectively).
Discussion
The 2018 OIG report found that VA practitioners demonstrated poor adherence to evidence-based clinical practice guidelines for testosterone treatment in men with hypogonadism. Based on OIG recommendations, we developed a PADR testosterone ordering template to help HCPs improve practice by better adherence to guidelines for the diagnosis and treatment of hypogonadism in veterans. Before implementation of the PADR template, the percentage of veterans at VAPSHCS who had biochemical confirmation of hypogonadism was higher than that in the OIG report. Activation of the PADR testosterone ordering template (with or without removal of options for alternative ordering pathways of testosterone) resulted only in an improvement of laboratory confirmation and evaluation of etiology of hypogonadism. This is when we reasoned that clinicians may have access to prior records and laboratory testing beyond just the past year, and this information may have influenced their use of the PADR template. Subsequently, with exclusion of veterans who were previously prescribed testosterone, implementation of the PADR testosterone order template improved documentation of symptoms of testosterone deficiency, discussion of risks and benefits of testosterone therapy, and biochemical diagnosis and evaluation of hypogonadism relative to the period before implementation.
The lack of effects of implementing the testosterone order template on documentation of symptoms of testosterone deficiency and discussion of risks and benefits of testosterone therapy may be due to local expertise resulting in the relatively high adherence to these guideline recommendations at VAPSHCS before activation of the template vs that in the OIG report. The template improved documentation of the diagnosis and evaluation of hypogonadism for genuinely new testosterone prescriptions in veterans without a history of testosterone prescriptions; while those with a previous prescription had limited improvement. It is possible that in veterans who had testosterone prescribed previously, HCPs may have assumed or had bias that the diagnosis and evaluation of hypogonadism originally made was adequate. This finding underscores the need to develop strategies for reviewing PADR requests where there is historical testosterone use. Perhaps a clinical team member, such as a clinical pharmacist, with the background and training in guidelines for the evaluation of hypogonadism could review PADR requests in veterans with previous testosterone use.
Removal of alternative ordering pathways for testosterone increased the completion rate of PADR requests and the testosterone ordering template, although the latter was not completed in one-third of veterans. Possible reasons for HCPs’ suboptimal completion of the testosterone template despite the PADR initiation include clinicians’ lack of willingness to read the PADR completely and familiarize themselves with the clinical guidelines due to workload demands of PCPs. In addition there maybe pressure from patients to receive testosterone for age-related symptoms due to heavy marketing. In addition, there may have been pharmacists who reviewed the PADR and approved the incomplete testosterone template. At VAPSHCS there were up to 40 pharmacists during different periods reviewing the testosterone PADRs. Likely, not everyone was completely familiar with this implementation process, and a possible future consideration would be further education to staff pharmacists who are verifying these prescriptions. There were several advantages to using this new testosterone order template when HCPs attempted to order a prescription. First, they were prompted to complete the PADR. Subsequently, a pharmacist reviewed the template and approved or rejected the prescription if the template was incomplete. The completed template served as documentation in the electronic health record for the prescribing HCP. The template was constructed to populate the required laboratory tests for ease of use and documentation. In addition, educational information regarding the symptoms and signs of testosterone deficiency, laboratory tests needed to confirm and evaluate hypogonadism, contraindications to testosterone treatment, and risks and benefits of therapy were incorporated into the template to assist HCPs in understanding the requirements for a complete diagnosis and evaluation. Finally, on completion of the template, HCPs were able to order testosterone via link to various testosterone formulations.
Before its implementation, the PADR testosterone order template was introduced to PCPs and internal medicine residents at 2 case-based conferences aimed at the diagnosis and treatment of male hypogonadism. These conferences were well received and helped launch the testosterone PADR template at VAPSHCS. Similar outreach to HCPs who prescribe testosterone is highly recommended in other VA facilities before implementation of the testosterone ordering template. It is possible that more targeted education to other HCPs would have resulted in greater use of the testosterone ordering template and adherence to clinical practice guidelines.
The VAPSHCS multidisciplinary workgroup was essential for the development, implementation, evaluation, and revision of the PADR and testosterone ordering template. The workgroup met routinely to follow up on the ease of installation in CPRS and discuss technical corrections that were needed. This was an essential for quality improvement, as loopholes in CPRS were identified where the HCP could order testosterone without being prompted to use the new PADR testosterone order template (alternative ordering pathways). The workgroup swiftly informed the IT specialist and HPS team to remove alternative ordering pathways of testosterone. Continuous quality improvement evaluations are highly recommended during implementation of the template in other facilities to accommodate specific local modifications that might be needed.
After February 2020 due to the COVID-19 pandemic, the National VA Pharmacy and Medication Board halted PADR requirements. As a result, further evaluation of the New Testosterone Order template and planned initial assessment of First Renewal Testosterone Order template could not be performed. In addition, due to the COVID-19 pandemic, there was restricted in-person outpatient visits and reduced adjustments to prescribing practices. To address recommendations made in the OIG report, the VAPSHCS testosterone order template was modified into a clinical reminder dialog format by a VA National IT Specialist and HPS team, tested for usability at several VA test sites and approved by the National Clinical Template Workgroup for implementation nationally across all VAs. The National Endocrinology Ambulatory Council Workgroup will ensure that this template is adopted in a similar format when the new electronic health record system Cerner is introduced to the VA.
Conclusions
The creation and implementation of a PADR testosterone order template may be a beneficial approach to improve the diagnosis of hypogonadism and facilitate appropriate use of testosterone therapy in veterans in accordance with established clinical practice guidelines, particularly in veterans without any prior testosterone use. Key future strategies to improve testosterone prescribing should focus on identifying clinical team members, such as a local clinical pharmacist, to review and steward PADR requests to ensure that testosterone is indicated, and treatment is appropriately monitored.
1. Bhasin S, Cunningham GR, Hayes FJ, Matsumoto AM, Snyder PJ, Swerdloff RS, Montori VM; Task Force, Endocrine Society. Testosterone therapy in men with androgen deficiency syndromes: an Endocrine Society clinical practice guideline. J Clin Endocrinol Metab. 2010;95(6):2536-2559. doi:10.1210/jc.2009-2354
2. Grossmann M, Matsumoto AM. A perspective on middle-aged and older men with functional hypogonadism: focus on holistic management. J Clin Endocrinol Metab. 2017;102(3):1067-1075. doi:10.1210/jc.2016-3580
3. Baillargeon J, Urban RJ, Kuo YF, et al. Screening and monitoring in men prescribed testosterone therapy in the US, 2001-2010. Public Health Rep. 2015;130(2):143-152. doi:10.1177/003335491513000207
4. Baillargeon J, Kuo Y, Westra JR, Urban RJ, Goodwin JS. Testosterone prescribing in the United States, 2002-2016. JAMA. 2018;320(2):200-202. doi:10.1001/jama.2018.7999
5. Jasuja GK, Bhasin S, Reisman JI, Berlowitz DR, Rose AJ. Ascertainment of testosterone prescribing practices in the VA. Med Care. 2015;53(9):746-52. doi:10.1097/MLR.0000000000000398
6. Jasuja GK, Bhasin S, Rose AJ. Patterns of testosterone prescription overuse. Curr Opin Endocrinol Diabetes Obes. 2017;24(3):240-245. doi:10.1097/MED.0000000000000336
7. US Department of Veterans Affairs, Office of Inspector General. Office of Healthcare Inspections. Report No. 15-03215-154. Published April 11, 2018. Accessed February 24, 2021. https://www.va.gov/oig/pubs/VAOIG-15-03215-154.pdf
1. Bhasin S, Cunningham GR, Hayes FJ, Matsumoto AM, Snyder PJ, Swerdloff RS, Montori VM; Task Force, Endocrine Society. Testosterone therapy in men with androgen deficiency syndromes: an Endocrine Society clinical practice guideline. J Clin Endocrinol Metab. 2010;95(6):2536-2559. doi:10.1210/jc.2009-2354
2. Grossmann M, Matsumoto AM. A perspective on middle-aged and older men with functional hypogonadism: focus on holistic management. J Clin Endocrinol Metab. 2017;102(3):1067-1075. doi:10.1210/jc.2016-3580
3. Baillargeon J, Urban RJ, Kuo YF, et al. Screening and monitoring in men prescribed testosterone therapy in the US, 2001-2010. Public Health Rep. 2015;130(2):143-152. doi:10.1177/003335491513000207
4. Baillargeon J, Kuo Y, Westra JR, Urban RJ, Goodwin JS. Testosterone prescribing in the United States, 2002-2016. JAMA. 2018;320(2):200-202. doi:10.1001/jama.2018.7999
5. Jasuja GK, Bhasin S, Reisman JI, Berlowitz DR, Rose AJ. Ascertainment of testosterone prescribing practices in the VA. Med Care. 2015;53(9):746-52. doi:10.1097/MLR.0000000000000398
6. Jasuja GK, Bhasin S, Rose AJ. Patterns of testosterone prescription overuse. Curr Opin Endocrinol Diabetes Obes. 2017;24(3):240-245. doi:10.1097/MED.0000000000000336
7. US Department of Veterans Affairs, Office of Inspector General. Office of Healthcare Inspections. Report No. 15-03215-154. Published April 11, 2018. Accessed February 24, 2021. https://www.va.gov/oig/pubs/VAOIG-15-03215-154.pdf