User login
Updated June 17, 2022
The US Department of Veterans Affairs (VA) provides health care for > 9 million military veterans, nearly half of all former service members.1 Over the past 15 years, there has been a steady and substantial increase in the frequency of disability awards for veterans with post-9/11 military service. Recent data from the Bureau of Labor Statistics indicate that 41% of veterans who served after 9/11 receive service-connected disability benefits compared with 28% of veterans overall.2 More than 5 million veterans receive VA service-related disability benefits.2,3 More than half of the VA $243 billion budget for fiscal year (FY) 2021 ($135.5 billion) was allocated to the Veterans Benefits Administration (VBA), of which $115.7 billion (85%) was allocated specifically for service-related compensation claims payments.4
The VA predicted that VBA will have completed 1.4 million ratings for disability claims in 2021.5 A substantial percentage of these claims will be for mental disorders, specifically posttraumatic stress disorder (PTSD). VA officials testifying before Congress in 2017 noted that the number of PTSD claims had nearly tripled in the previous 10 years.6 As far back as 2013, McNally and Frueh analyzed “the skyrocketing of disability claims,” particularly for PTSD, among veterans who served in Iraq and Afghanistan.7
This large increase has placed an unprecedented burden on the VBA to expand its capacity to conduct initial PTSD disability evaluations that by regulations are completed by psychologists or psychiatrists. This need has led the VBA to make significant changes in the compensation and pension (C&P) process, including a reduced role for Veterans Health Administration (VHA) examiners and increased reliance on non-VA (contract) examiners through the Contract Medical Disability Examination (MDE) program. In 2019, the MDE budget was $1.23 billion; in 2020, it was increased to $1.79 billion, and for 2021, it was $2.23 billion, reflecting the increasing investment of resources in non-VA examiners, ostensibly to both increase capacity and save costs.5
Anecdotally, concerns have been raised regarding inadequate training of contract examiners as well as inadequate reports by these examiners. A 2018 Government Accountability Office (GAO) report concluded that VA lacked the data to determine whether contract examiners were meeting standards for quality, timeliness, and accuracy.8 The GAO report noted that VA required 92% of contractor reports contained no obvious errors, a relatively low target; however, in the first half of 2017, only 1 contractor group met that target. The report noted further that “VBA does not verify if examiners have completed training nor does it collect information to assess training effectiveness in preparing examiners.”8 A subsequent analysis of contract examinations completed by the VA Office of the Inspector General (OIG) in 2019 concluded that the MDE program was “hampered in their ability to provide oversight because of limitations with VBA’s electronic examination management systems, the lack of reliable data, and inadequate staffing of the program.”9
These reports have focused almost exclusively on simple performance metrics, such as timeliness of examination completion. However, the 2018 GAO report referenced isolated “focused reviews” of complaints about the quality of examinations by contract examiners and gave as an example an isolated “review of one contracted examiner who had high rates of diagnosing severe posttraumatic stress disorder.”8 After review indicated the examiner’s reports were of poor quality, the VBA discontinued the examiner’s contract.
Unfortunately, despite such anecdotal reports and isolated actions, to date there are no published reports examining and comparing the quality of PTSD examination reports completed by VHA and contract examiners or the subsequent disability determinations made by the VBA as a result of these evaluations. In a November 2020 letter to the VA Secretary, 11 US Senators expressed “grave concerns” regarding the VA decision to privatize C&P programs noting, among other concerns, that there were “no clinical quality measurement for, or evaluation of, contractor examinations.”10 The letter cited anecdotal evidence of contract examiners not reviewing veteran’s medical records and diagnosing conditions “without supporting evidence.”10
The purpose of the present evaluation was to provide a systematic comparison of the content and quality of initial PTSD disability examinations conducted by VHA and non-VA contract examiners. In addition, this study compared the disability rating decisions resulting from VHA and contract examinations.
Methods
A random sample of 100 Initial PTSD Disability Benefits Questionnaires (DBQs)—structured forms completed by all examiners—were obtained from a list supplied by the VA Office of Performance Analysis and Integrity. All examinations were from the Veterans Integrated Service Network (VISN) 1, encompassing the New England region and were conducted in 2019 and 2020. Two of the 100 cases were excluded for technical reasons, resulting in 98 examination reports. However, the final pool yielded 62 contract examinations and only 36 VHA examinations. To make the sample sizes more comparable, an additional 15 examinations were randomly selected from the local examination database (also VISN 1) to complement the original examination pool.
Once DBQs were retrieved, all identifying information was deleted, and cases were analyzed using assigned record numbers. All coding was completed by the 2 principal investigators, both VA psychologists with extensive training and experience in C&P evaluation and treatment of veterans with PTSD. Due to inherent structural differences between the forms used for VA and contract examinations, raters could not be masked/blinded to the source of the report.
A number of measures were taken to reduce bias and enhance objectivity of rating. First, objectively coded variables (eg, age and sex of veteran, period of service, trauma type, diagnoses rendered by the examiner, impairment category endorsed, number and type of symptoms) were transcribed directly from the DBQ as recorded by the examiner. Second, to rate report quality, an initial categorical rating scale was developed based on predetermined elements of examination quality that were considered essential. After refinement and preliminary analysis of interrater reliabilities, 3 quality-related indices were identified: (1) level of detail in description of key content areas (history before service, service trauma, after service social and vocational history, mental health history, substance use); (2) synthesis of history and findings in explaining opinion rendered; and (3) clarity of opinion regarding causation required “at least as likely as not” degree of confidence. The first 2 quality ratings were based on a 3-point scale (poor, fair, good), and the third variable was coded as yes or no. (eAppendix available at doi:10.12788/fp.0225). Interrater reliabilities calculated based on a subsample of 18 cases, randomly selected and rated by both raters, yielded Cohen κ in the acceptable range (.61, .72, and .89 for detail, synthesis, and clarity, respectively). Finally, for information regarding VBA decision making, rating decision documents contained in the Veterans Benefit Management System database were reviewed to determine whether the veteran was granted service connection for PTSD or another mental disorder based on the examination report in question and, if so, the disability rating percentage awarded. These were recorded independently after all other coding had been completed.
Results
Comparison of VHA and contract examinations revealed no significant differences between groups on relevant sociodemographic and other measures (Table). Missing data were not obtained from other records or sources, and for this study, reflect only what is recorded in the examination reports except for age, which was calculated using veteran’s date of birth and the date of examination.
To examine differences between VHA and contract examinations, the groups were first compared on a set of predetermined objectively coded variables taken directly from the DBQ. The frequency of PTSD diagnoses by VHA (57%) and contract (71%) examiners was not significantly different nor were rates of non-PTSD diagnoses by VHA (51%) and contract (73%) examiners. There also was no difference in the mean number of PTSD symptoms endorsed across PTSD diagnostic criteria B, C, D, and E (maximum of 20) recorded by VHA (9.4) and contract (10.9) examiners.
Contract examiners recorded a significantly greater mean number of “other symptoms” on a checklist of 31 possible symptoms as compared to VHA examiners: 7.3 vs 5.8, respectively (t[104] = 2.27, P < .05). An initial analysis of overall social/vocational impairment ratings coded by examiners did not reveal significant differences between examiner groups. However, when the 2 most severe impairment categories were combined to create a pooled “severe” category, 31% of contract examiners rated veterans as severely impaired compared with only 12% of VHA examiners (χ2 = 5.79, 1 df, P < .05) (Figure 1).
VHA and contract examinations were compared on 3 measures of report quality. Significant differences were found for both level of detail (χ2 = 16.44, 2 df, P < .01) and synthesis (χ2 = 6.68, 2 df, P < .05). Contract examinations were more likely to be rated as poor and less likely to be rated good, with a similar proportion of fair ratings for the 2 examination types (Figures 2 and 3). There was no significant difference in the proportion of VHA and contract examinations providing clear statement of opinion regarding causation (ie, whether or not the diagnosed condition was service related), with the majority rendering an adequate opinion in both examiner groups (VHA, 78%; contract, 79%).
Qualitative review revealed examples of markedly deficient examinations among contract examinations, including several reports that contained no review of records, no report of relevant background, and no mention or assessment of social and vocational function needed to inform opinions about diagnosis and impairment.
Finally, the VBA database was used to compare the resulting disability award decisions made by VBA based on the examination reports in question. Examination by contractors resulted in significantly higher mean service-connected disability ratings for examinees compared with VHA examiners (46.8 vs 33.5, respectively; t[108] = 2.3, P < .05).
Discussion
The present study provides the first reported systematic comparison of VA disability examinations for PTSD completed by examiners employed by the VHA and those hired as contract examiners through the MDE program. Although the frequency of PTSD diagnoses by contract examiners was higher than that of VHA examiners (71% vs 57%, respectively), the difference was not statistically significant. However, contract examiners recorded significantly more symptoms for examinees and rated them as severely impaired more frequently than did their VHA counterparts. In keeping with rating guidelines used by the VBA, these differences in examination content resulted in higher disability ratings for veterans seen by contract examiners.
Along with these elevated reports in symptom and severity ratings, contract examiners were less likely to provide adequate detail in the narrative sections of their reports and less frequently provided a satisfactory explanation and synthesis of relevant history and findings in support of their conclusions. Although not reflected in the statistical analysis, case-by-case review revealed some startlingly inadequate examination reports by contract examiners, several of which contained no review of records, no report or discussion of relevant background, and no discussion or analysis of social and vocational function to inform and support their opinion about level of impairment. None of the VHA examination reports reviewed lacked information to that degree.
Such deficiencies in detail and synthesis run counter to accepted guidelines for the adequate assessment of psychological injury in general and in VA disability claims specifically.11,12 For example, Watson and colleagues proposed that a minimum of 3 hours was required to conduct an initial PTSD examination, with more complex cases possibly taking longer.11 There is no information available about how long contract examiners take to complete their examinations and how that compares with the time taken by VA examiners. The VBA failure to monitor whether or not examiners follow accepted guidelines for PTSD examination has not previously been evaluated. Historically, a large number of clinicians, researchers, and policy critics have raised concerns about the potential for exaggeration or malingering among VA PTSD disability claimants and have urged the need to adequately assess for unreliable reporting and presentation.13,14 However, the possibility of systematic examiner deficiency and/or bias increasing the frequency of false or inflated claims being approved has received little empirical attention.
Although contract examiners did not diagnose PTSD significantly more frequently than VHA examiners (71% vs 57%, respectively), the overall frequency of PTSD diagnosis across both groups (65%) was substantially higher than previous figures that have, on average, estimated the lifetime prevalence of PTSD in trauma-exposed veterans to be about 31%.15 A re-analysis of the same National Vietnam Veterans Readjustment Survey data, but applying more conservative diagnostic criteria, reduced the lifetime prevalence to just under 19%, with point prevalence estimates even lower.15,16
In a study of concordance rates between service connection for PTSD and both current and lifetime diagnosis by independent, structured assessment, Marx and colleagues found that a “significant minority” of veterans who were already receiving service-connected disability for PTSD did not meet lifetime and/or current diagnostic criteria.17 Although it is possible that the group of veterans who were applying for disability benefits in our study had a higher rate of PTSD, it also is possible if not likely that the PTSD examination process overall yields inflated rates of diagnosis and levels of impairment. This speaks to the concern raised by Marx and colleagues who found that veterans with service connection for PTSD who received related benefits “may not have the disorder.”17
Limitations
A methodological limitation of the present study was that, due to structural differences in the DBQ forms used for VHA and contract examinations, the reports could not be de-identified as to examiner type and thus raters could not be masked/blinded. To mitigate bias, a predetermined, piloted, and refined coding and rating plan for report quality metrics was adhered to strictly, and interrater reliabilities were acceptable. Future study is suggested in which all report content is standardized for coding using the same format, which at present would require a complete rewriting of the entire report; this problem could be resolved by having the VBA adopt a more coherent system in which all reports, regardless of examiner type, use a single, standardized template. Further study using larger data sets and expanding to other VA regions also is needed.
Conclusions
The present study suggests that poor examination and report quality—by contract examiners and to a lesser degree VHA examiners—are not uncommon. The findings confirm and extend previous anecdotal reports of deficiencies in PTSD examinations performed by contract examiners and provide empirical support for concerns raised of global deficiencies in the VBA oversight of the MDE program. Such deficiencies have significant implications for the quality and integrity of the VA disability determination process for veterans claiming PTSD related to military service.
The current findings support and strengthen the call for development and management of a structured and enforced training and quality assurance/improvement program for VA PTSD disability examinations. Such training and oversight will be critical to improve the quality and integrity of these examinations, reduce error and waste in VBA’s Compensation and Pension process, and in doing so optimize VA financial resources to best serve veterans’ benefits and health care needs.
1. US Department of Veterans Affairs, Veterans Health Adminstration. About VHA. Updated April 23, 2021. Accessed January 6, 2022. www.va.gov/health/aboutvha.asp
2. US Department of Labor, Bureau of Labor Statistics. News release. Employment situation of veterans—2020. Published March 18, 2020. Accessed January 6, 2022. https://www.bls.gov/news.release/pdf/vet.pdf
3. US Department of Veterans Affairs, National Center for Veterans Analysis and Statistics. Department of Veterans Affairs statistics at a glance. Updated December 31, 2020. Accessed January 6, 2022. https://www.va.gov/vetdata/docs/Quickfacts/Stats_at_a_glance_12_31_20.PDF
4. US Department of Veterans Affairs. FY 2021 Budget submission: budget in brief. Published February 2020. Accessed January 6, 2022. https://www.va.gov/budget/docs/summary/archive/FY-2021-VA-BudgetSubmission.zip
5. US Department of Veterans Affairs. FY 2021 budget submission: benefits and burial programs and Departmental Administration volume 3 of 4:178. Published February 2020. Accessed January 6, 2022. https://www.va.gov/budget/docs/summary/archive/FY-2021-VA-BudgetSubmission.zip
6. Statement of Ronald Burke, assistant deputy under secretary, office of field operations Veterans Benefits Administration before the Subcommittee on Disability And Memorial Affairs of the House Committee on Veterans’ Affairs. Published July 25, 2017. Accessed January 6, 2022. https://www.congress.gov/115/meeting/house/106322/witnesses/HHRG-115-VR09-Wstate-BurkeR-20170725.pdf
7. McNally RJ, Frueh BC. Why are Iraq and Afghanistan War veterans seeking PTSD disability compensation at unprecedented rates? J Anxiety Disord. 2013;27(5):520-526. doi:10.1016/j.janxdis.2013.07.002
8. US Government Accountability Office. VA disability exams: improved performance analysis and training oversight needed for contracted exams. GAO-19-13. Published October 2018. Accessed January 6, 2022. https://www.gao.gov/assets/gao-19-13.pdf
9. US Department of Veterans Affairs, Office of Inspector General. Inadequate oversight of contracted disability exam cancellations. Report #18-04266-115. Published June 10, 2019. Accessed January 6, 2022. https://www.va.gov/oig/pubs/VAOIG-18-04266-115.pdf
10. Letter to VA Secretary Wilkie. Published November 11, 2020. Accessed January 6, 2022. https://www.veterans.senate.gov/download/candp-exam-va-letter
11. Watson PW, McFall M, McBrine C, Schnurr PP, Friedman MJ, Keane TM, Hamblen JL (2005). Best practice manual for posttraumatic stress disorder (PTSD) compensation and pension examinations. Portland, OR: Northwest Network Mental Illness Research, Education, and Clinical Center, VA Puget Sound Healthcare System.
12. Worthen MD, Moering RG. A practical guide to conducting VA compensation and pension exams for PTSD and other mental disorders. Psychol Inj and Law. 2011;4:187-216. doi:10.1007/s12207-011-9115-2
13. DeViva JC, Bloem WD. Symptom exaggeration and compensation seeking among combat veterans with posttraumatic stress disorder. J Trauma Stress. 2003;16(5):503-507. doi:10.1023/A:1025766713188
14. Ray CL. Feigning screeners in VA PTSD compensation and pension examinations. Psychol Inj and Law. 2014;7:370-387. doi:10.1007/s12207-014-9210-2
15. Kulka RA, Schlenger WE, Fairbank JA, et al. Trauma and the Vietnam War Generation: Report of Findings From the National Vietnam Veterans Readjustment Study. Brunner Mazel Publishers; 1990.
16. Dohrenwend BP, Turner JB, Turse NA, Adams BG, Koenen KC, Marshall R. The psychological risks of Vietnam for U.S. veterans: a revisit with new data and methods. Science. 2006;313(5789):979-982. doi:10.1126/science.1128944
17. Marx BP, Bovin MJ, Szafranski DD, et al. Validity of posttraumatic stress disorder service connection status in Veterans Affairs electronic records of Iraq and Afghanistan Veterans. J Clin Psychiatry. 2016;77(4):517-522. doi:10.4088/JCP.14m09666
Updated June 17, 2022
The US Department of Veterans Affairs (VA) provides health care for > 9 million military veterans, nearly half of all former service members.1 Over the past 15 years, there has been a steady and substantial increase in the frequency of disability awards for veterans with post-9/11 military service. Recent data from the Bureau of Labor Statistics indicate that 41% of veterans who served after 9/11 receive service-connected disability benefits compared with 28% of veterans overall.2 More than 5 million veterans receive VA service-related disability benefits.2,3 More than half of the VA $243 billion budget for fiscal year (FY) 2021 ($135.5 billion) was allocated to the Veterans Benefits Administration (VBA), of which $115.7 billion (85%) was allocated specifically for service-related compensation claims payments.4
The VA predicted that VBA will have completed 1.4 million ratings for disability claims in 2021.5 A substantial percentage of these claims will be for mental disorders, specifically posttraumatic stress disorder (PTSD). VA officials testifying before Congress in 2017 noted that the number of PTSD claims had nearly tripled in the previous 10 years.6 As far back as 2013, McNally and Frueh analyzed “the skyrocketing of disability claims,” particularly for PTSD, among veterans who served in Iraq and Afghanistan.7
This large increase has placed an unprecedented burden on the VBA to expand its capacity to conduct initial PTSD disability evaluations that by regulations are completed by psychologists or psychiatrists. This need has led the VBA to make significant changes in the compensation and pension (C&P) process, including a reduced role for Veterans Health Administration (VHA) examiners and increased reliance on non-VA (contract) examiners through the Contract Medical Disability Examination (MDE) program. In 2019, the MDE budget was $1.23 billion; in 2020, it was increased to $1.79 billion, and for 2021, it was $2.23 billion, reflecting the increasing investment of resources in non-VA examiners, ostensibly to both increase capacity and save costs.5
Anecdotally, concerns have been raised regarding inadequate training of contract examiners as well as inadequate reports by these examiners. A 2018 Government Accountability Office (GAO) report concluded that VA lacked the data to determine whether contract examiners were meeting standards for quality, timeliness, and accuracy.8 The GAO report noted that VA required 92% of contractor reports contained no obvious errors, a relatively low target; however, in the first half of 2017, only 1 contractor group met that target. The report noted further that “VBA does not verify if examiners have completed training nor does it collect information to assess training effectiveness in preparing examiners.”8 A subsequent analysis of contract examinations completed by the VA Office of the Inspector General (OIG) in 2019 concluded that the MDE program was “hampered in their ability to provide oversight because of limitations with VBA’s electronic examination management systems, the lack of reliable data, and inadequate staffing of the program.”9
These reports have focused almost exclusively on simple performance metrics, such as timeliness of examination completion. However, the 2018 GAO report referenced isolated “focused reviews” of complaints about the quality of examinations by contract examiners and gave as an example an isolated “review of one contracted examiner who had high rates of diagnosing severe posttraumatic stress disorder.”8 After review indicated the examiner’s reports were of poor quality, the VBA discontinued the examiner’s contract.
Unfortunately, despite such anecdotal reports and isolated actions, to date there are no published reports examining and comparing the quality of PTSD examination reports completed by VHA and contract examiners or the subsequent disability determinations made by the VBA as a result of these evaluations. In a November 2020 letter to the VA Secretary, 11 US Senators expressed “grave concerns” regarding the VA decision to privatize C&P programs noting, among other concerns, that there were “no clinical quality measurement for, or evaluation of, contractor examinations.”10 The letter cited anecdotal evidence of contract examiners not reviewing veteran’s medical records and diagnosing conditions “without supporting evidence.”10
The purpose of the present evaluation was to provide a systematic comparison of the content and quality of initial PTSD disability examinations conducted by VHA and non-VA contract examiners. In addition, this study compared the disability rating decisions resulting from VHA and contract examinations.
Methods
A random sample of 100 Initial PTSD Disability Benefits Questionnaires (DBQs)—structured forms completed by all examiners—were obtained from a list supplied by the VA Office of Performance Analysis and Integrity. All examinations were from the Veterans Integrated Service Network (VISN) 1, encompassing the New England region and were conducted in 2019 and 2020. Two of the 100 cases were excluded for technical reasons, resulting in 98 examination reports. However, the final pool yielded 62 contract examinations and only 36 VHA examinations. To make the sample sizes more comparable, an additional 15 examinations were randomly selected from the local examination database (also VISN 1) to complement the original examination pool.
Once DBQs were retrieved, all identifying information was deleted, and cases were analyzed using assigned record numbers. All coding was completed by the 2 principal investigators, both VA psychologists with extensive training and experience in C&P evaluation and treatment of veterans with PTSD. Due to inherent structural differences between the forms used for VA and contract examinations, raters could not be masked/blinded to the source of the report.
A number of measures were taken to reduce bias and enhance objectivity of rating. First, objectively coded variables (eg, age and sex of veteran, period of service, trauma type, diagnoses rendered by the examiner, impairment category endorsed, number and type of symptoms) were transcribed directly from the DBQ as recorded by the examiner. Second, to rate report quality, an initial categorical rating scale was developed based on predetermined elements of examination quality that were considered essential. After refinement and preliminary analysis of interrater reliabilities, 3 quality-related indices were identified: (1) level of detail in description of key content areas (history before service, service trauma, after service social and vocational history, mental health history, substance use); (2) synthesis of history and findings in explaining opinion rendered; and (3) clarity of opinion regarding causation required “at least as likely as not” degree of confidence. The first 2 quality ratings were based on a 3-point scale (poor, fair, good), and the third variable was coded as yes or no. (eAppendix available at doi:10.12788/fp.0225). Interrater reliabilities calculated based on a subsample of 18 cases, randomly selected and rated by both raters, yielded Cohen κ in the acceptable range (.61, .72, and .89 for detail, synthesis, and clarity, respectively). Finally, for information regarding VBA decision making, rating decision documents contained in the Veterans Benefit Management System database were reviewed to determine whether the veteran was granted service connection for PTSD or another mental disorder based on the examination report in question and, if so, the disability rating percentage awarded. These were recorded independently after all other coding had been completed.
Results
Comparison of VHA and contract examinations revealed no significant differences between groups on relevant sociodemographic and other measures (Table). Missing data were not obtained from other records or sources, and for this study, reflect only what is recorded in the examination reports except for age, which was calculated using veteran’s date of birth and the date of examination.
To examine differences between VHA and contract examinations, the groups were first compared on a set of predetermined objectively coded variables taken directly from the DBQ. The frequency of PTSD diagnoses by VHA (57%) and contract (71%) examiners was not significantly different nor were rates of non-PTSD diagnoses by VHA (51%) and contract (73%) examiners. There also was no difference in the mean number of PTSD symptoms endorsed across PTSD diagnostic criteria B, C, D, and E (maximum of 20) recorded by VHA (9.4) and contract (10.9) examiners.
Contract examiners recorded a significantly greater mean number of “other symptoms” on a checklist of 31 possible symptoms as compared to VHA examiners: 7.3 vs 5.8, respectively (t[104] = 2.27, P < .05). An initial analysis of overall social/vocational impairment ratings coded by examiners did not reveal significant differences between examiner groups. However, when the 2 most severe impairment categories were combined to create a pooled “severe” category, 31% of contract examiners rated veterans as severely impaired compared with only 12% of VHA examiners (χ2 = 5.79, 1 df, P < .05) (Figure 1).
VHA and contract examinations were compared on 3 measures of report quality. Significant differences were found for both level of detail (χ2 = 16.44, 2 df, P < .01) and synthesis (χ2 = 6.68, 2 df, P < .05). Contract examinations were more likely to be rated as poor and less likely to be rated good, with a similar proportion of fair ratings for the 2 examination types (Figures 2 and 3). There was no significant difference in the proportion of VHA and contract examinations providing clear statement of opinion regarding causation (ie, whether or not the diagnosed condition was service related), with the majority rendering an adequate opinion in both examiner groups (VHA, 78%; contract, 79%).
Qualitative review revealed examples of markedly deficient examinations among contract examinations, including several reports that contained no review of records, no report of relevant background, and no mention or assessment of social and vocational function needed to inform opinions about diagnosis and impairment.
Finally, the VBA database was used to compare the resulting disability award decisions made by VBA based on the examination reports in question. Examination by contractors resulted in significantly higher mean service-connected disability ratings for examinees compared with VHA examiners (46.8 vs 33.5, respectively; t[108] = 2.3, P < .05).
Discussion
The present study provides the first reported systematic comparison of VA disability examinations for PTSD completed by examiners employed by the VHA and those hired as contract examiners through the MDE program. Although the frequency of PTSD diagnoses by contract examiners was higher than that of VHA examiners (71% vs 57%, respectively), the difference was not statistically significant. However, contract examiners recorded significantly more symptoms for examinees and rated them as severely impaired more frequently than did their VHA counterparts. In keeping with rating guidelines used by the VBA, these differences in examination content resulted in higher disability ratings for veterans seen by contract examiners.
Along with these elevated reports in symptom and severity ratings, contract examiners were less likely to provide adequate detail in the narrative sections of their reports and less frequently provided a satisfactory explanation and synthesis of relevant history and findings in support of their conclusions. Although not reflected in the statistical analysis, case-by-case review revealed some startlingly inadequate examination reports by contract examiners, several of which contained no review of records, no report or discussion of relevant background, and no discussion or analysis of social and vocational function to inform and support their opinion about level of impairment. None of the VHA examination reports reviewed lacked information to that degree.
Such deficiencies in detail and synthesis run counter to accepted guidelines for the adequate assessment of psychological injury in general and in VA disability claims specifically.11,12 For example, Watson and colleagues proposed that a minimum of 3 hours was required to conduct an initial PTSD examination, with more complex cases possibly taking longer.11 There is no information available about how long contract examiners take to complete their examinations and how that compares with the time taken by VA examiners. The VBA failure to monitor whether or not examiners follow accepted guidelines for PTSD examination has not previously been evaluated. Historically, a large number of clinicians, researchers, and policy critics have raised concerns about the potential for exaggeration or malingering among VA PTSD disability claimants and have urged the need to adequately assess for unreliable reporting and presentation.13,14 However, the possibility of systematic examiner deficiency and/or bias increasing the frequency of false or inflated claims being approved has received little empirical attention.
Although contract examiners did not diagnose PTSD significantly more frequently than VHA examiners (71% vs 57%, respectively), the overall frequency of PTSD diagnosis across both groups (65%) was substantially higher than previous figures that have, on average, estimated the lifetime prevalence of PTSD in trauma-exposed veterans to be about 31%.15 A re-analysis of the same National Vietnam Veterans Readjustment Survey data, but applying more conservative diagnostic criteria, reduced the lifetime prevalence to just under 19%, with point prevalence estimates even lower.15,16
In a study of concordance rates between service connection for PTSD and both current and lifetime diagnosis by independent, structured assessment, Marx and colleagues found that a “significant minority” of veterans who were already receiving service-connected disability for PTSD did not meet lifetime and/or current diagnostic criteria.17 Although it is possible that the group of veterans who were applying for disability benefits in our study had a higher rate of PTSD, it also is possible if not likely that the PTSD examination process overall yields inflated rates of diagnosis and levels of impairment. This speaks to the concern raised by Marx and colleagues who found that veterans with service connection for PTSD who received related benefits “may not have the disorder.”17
Limitations
A methodological limitation of the present study was that, due to structural differences in the DBQ forms used for VHA and contract examinations, the reports could not be de-identified as to examiner type and thus raters could not be masked/blinded. To mitigate bias, a predetermined, piloted, and refined coding and rating plan for report quality metrics was adhered to strictly, and interrater reliabilities were acceptable. Future study is suggested in which all report content is standardized for coding using the same format, which at present would require a complete rewriting of the entire report; this problem could be resolved by having the VBA adopt a more coherent system in which all reports, regardless of examiner type, use a single, standardized template. Further study using larger data sets and expanding to other VA regions also is needed.
Conclusions
The present study suggests that poor examination and report quality—by contract examiners and to a lesser degree VHA examiners—are not uncommon. The findings confirm and extend previous anecdotal reports of deficiencies in PTSD examinations performed by contract examiners and provide empirical support for concerns raised of global deficiencies in the VBA oversight of the MDE program. Such deficiencies have significant implications for the quality and integrity of the VA disability determination process for veterans claiming PTSD related to military service.
The current findings support and strengthen the call for development and management of a structured and enforced training and quality assurance/improvement program for VA PTSD disability examinations. Such training and oversight will be critical to improve the quality and integrity of these examinations, reduce error and waste in VBA’s Compensation and Pension process, and in doing so optimize VA financial resources to best serve veterans’ benefits and health care needs.
Updated June 17, 2022
The US Department of Veterans Affairs (VA) provides health care for > 9 million military veterans, nearly half of all former service members.1 Over the past 15 years, there has been a steady and substantial increase in the frequency of disability awards for veterans with post-9/11 military service. Recent data from the Bureau of Labor Statistics indicate that 41% of veterans who served after 9/11 receive service-connected disability benefits compared with 28% of veterans overall.2 More than 5 million veterans receive VA service-related disability benefits.2,3 More than half of the VA $243 billion budget for fiscal year (FY) 2021 ($135.5 billion) was allocated to the Veterans Benefits Administration (VBA), of which $115.7 billion (85%) was allocated specifically for service-related compensation claims payments.4
The VA predicted that VBA will have completed 1.4 million ratings for disability claims in 2021.5 A substantial percentage of these claims will be for mental disorders, specifically posttraumatic stress disorder (PTSD). VA officials testifying before Congress in 2017 noted that the number of PTSD claims had nearly tripled in the previous 10 years.6 As far back as 2013, McNally and Frueh analyzed “the skyrocketing of disability claims,” particularly for PTSD, among veterans who served in Iraq and Afghanistan.7
This large increase has placed an unprecedented burden on the VBA to expand its capacity to conduct initial PTSD disability evaluations that by regulations are completed by psychologists or psychiatrists. This need has led the VBA to make significant changes in the compensation and pension (C&P) process, including a reduced role for Veterans Health Administration (VHA) examiners and increased reliance on non-VA (contract) examiners through the Contract Medical Disability Examination (MDE) program. In 2019, the MDE budget was $1.23 billion; in 2020, it was increased to $1.79 billion, and for 2021, it was $2.23 billion, reflecting the increasing investment of resources in non-VA examiners, ostensibly to both increase capacity and save costs.5
Anecdotally, concerns have been raised regarding inadequate training of contract examiners as well as inadequate reports by these examiners. A 2018 Government Accountability Office (GAO) report concluded that VA lacked the data to determine whether contract examiners were meeting standards for quality, timeliness, and accuracy.8 The GAO report noted that VA required 92% of contractor reports contained no obvious errors, a relatively low target; however, in the first half of 2017, only 1 contractor group met that target. The report noted further that “VBA does not verify if examiners have completed training nor does it collect information to assess training effectiveness in preparing examiners.”8 A subsequent analysis of contract examinations completed by the VA Office of the Inspector General (OIG) in 2019 concluded that the MDE program was “hampered in their ability to provide oversight because of limitations with VBA’s electronic examination management systems, the lack of reliable data, and inadequate staffing of the program.”9
These reports have focused almost exclusively on simple performance metrics, such as timeliness of examination completion. However, the 2018 GAO report referenced isolated “focused reviews” of complaints about the quality of examinations by contract examiners and gave as an example an isolated “review of one contracted examiner who had high rates of diagnosing severe posttraumatic stress disorder.”8 After review indicated the examiner’s reports were of poor quality, the VBA discontinued the examiner’s contract.
Unfortunately, despite such anecdotal reports and isolated actions, to date there are no published reports examining and comparing the quality of PTSD examination reports completed by VHA and contract examiners or the subsequent disability determinations made by the VBA as a result of these evaluations. In a November 2020 letter to the VA Secretary, 11 US Senators expressed “grave concerns” regarding the VA decision to privatize C&P programs noting, among other concerns, that there were “no clinical quality measurement for, or evaluation of, contractor examinations.”10 The letter cited anecdotal evidence of contract examiners not reviewing veteran’s medical records and diagnosing conditions “without supporting evidence.”10
The purpose of the present evaluation was to provide a systematic comparison of the content and quality of initial PTSD disability examinations conducted by VHA and non-VA contract examiners. In addition, this study compared the disability rating decisions resulting from VHA and contract examinations.
Methods
A random sample of 100 Initial PTSD Disability Benefits Questionnaires (DBQs)—structured forms completed by all examiners—were obtained from a list supplied by the VA Office of Performance Analysis and Integrity. All examinations were from the Veterans Integrated Service Network (VISN) 1, encompassing the New England region and were conducted in 2019 and 2020. Two of the 100 cases were excluded for technical reasons, resulting in 98 examination reports. However, the final pool yielded 62 contract examinations and only 36 VHA examinations. To make the sample sizes more comparable, an additional 15 examinations were randomly selected from the local examination database (also VISN 1) to complement the original examination pool.
Once DBQs were retrieved, all identifying information was deleted, and cases were analyzed using assigned record numbers. All coding was completed by the 2 principal investigators, both VA psychologists with extensive training and experience in C&P evaluation and treatment of veterans with PTSD. Due to inherent structural differences between the forms used for VA and contract examinations, raters could not be masked/blinded to the source of the report.
A number of measures were taken to reduce bias and enhance objectivity of rating. First, objectively coded variables (eg, age and sex of veteran, period of service, trauma type, diagnoses rendered by the examiner, impairment category endorsed, number and type of symptoms) were transcribed directly from the DBQ as recorded by the examiner. Second, to rate report quality, an initial categorical rating scale was developed based on predetermined elements of examination quality that were considered essential. After refinement and preliminary analysis of interrater reliabilities, 3 quality-related indices were identified: (1) level of detail in description of key content areas (history before service, service trauma, after service social and vocational history, mental health history, substance use); (2) synthesis of history and findings in explaining opinion rendered; and (3) clarity of opinion regarding causation required “at least as likely as not” degree of confidence. The first 2 quality ratings were based on a 3-point scale (poor, fair, good), and the third variable was coded as yes or no. (eAppendix available at doi:10.12788/fp.0225). Interrater reliabilities calculated based on a subsample of 18 cases, randomly selected and rated by both raters, yielded Cohen κ in the acceptable range (.61, .72, and .89 for detail, synthesis, and clarity, respectively). Finally, for information regarding VBA decision making, rating decision documents contained in the Veterans Benefit Management System database were reviewed to determine whether the veteran was granted service connection for PTSD or another mental disorder based on the examination report in question and, if so, the disability rating percentage awarded. These were recorded independently after all other coding had been completed.
Results
Comparison of VHA and contract examinations revealed no significant differences between groups on relevant sociodemographic and other measures (Table). Missing data were not obtained from other records or sources, and for this study, reflect only what is recorded in the examination reports except for age, which was calculated using veteran’s date of birth and the date of examination.
To examine differences between VHA and contract examinations, the groups were first compared on a set of predetermined objectively coded variables taken directly from the DBQ. The frequency of PTSD diagnoses by VHA (57%) and contract (71%) examiners was not significantly different nor were rates of non-PTSD diagnoses by VHA (51%) and contract (73%) examiners. There also was no difference in the mean number of PTSD symptoms endorsed across PTSD diagnostic criteria B, C, D, and E (maximum of 20) recorded by VHA (9.4) and contract (10.9) examiners.
Contract examiners recorded a significantly greater mean number of “other symptoms” on a checklist of 31 possible symptoms as compared to VHA examiners: 7.3 vs 5.8, respectively (t[104] = 2.27, P < .05). An initial analysis of overall social/vocational impairment ratings coded by examiners did not reveal significant differences between examiner groups. However, when the 2 most severe impairment categories were combined to create a pooled “severe” category, 31% of contract examiners rated veterans as severely impaired compared with only 12% of VHA examiners (χ2 = 5.79, 1 df, P < .05) (Figure 1).
VHA and contract examinations were compared on 3 measures of report quality. Significant differences were found for both level of detail (χ2 = 16.44, 2 df, P < .01) and synthesis (χ2 = 6.68, 2 df, P < .05). Contract examinations were more likely to be rated as poor and less likely to be rated good, with a similar proportion of fair ratings for the 2 examination types (Figures 2 and 3). There was no significant difference in the proportion of VHA and contract examinations providing clear statement of opinion regarding causation (ie, whether or not the diagnosed condition was service related), with the majority rendering an adequate opinion in both examiner groups (VHA, 78%; contract, 79%).
Qualitative review revealed examples of markedly deficient examinations among contract examinations, including several reports that contained no review of records, no report of relevant background, and no mention or assessment of social and vocational function needed to inform opinions about diagnosis and impairment.
Finally, the VBA database was used to compare the resulting disability award decisions made by VBA based on the examination reports in question. Examination by contractors resulted in significantly higher mean service-connected disability ratings for examinees compared with VHA examiners (46.8 vs 33.5, respectively; t[108] = 2.3, P < .05).
Discussion
The present study provides the first reported systematic comparison of VA disability examinations for PTSD completed by examiners employed by the VHA and those hired as contract examiners through the MDE program. Although the frequency of PTSD diagnoses by contract examiners was higher than that of VHA examiners (71% vs 57%, respectively), the difference was not statistically significant. However, contract examiners recorded significantly more symptoms for examinees and rated them as severely impaired more frequently than did their VHA counterparts. In keeping with rating guidelines used by the VBA, these differences in examination content resulted in higher disability ratings for veterans seen by contract examiners.
Along with these elevated reports in symptom and severity ratings, contract examiners were less likely to provide adequate detail in the narrative sections of their reports and less frequently provided a satisfactory explanation and synthesis of relevant history and findings in support of their conclusions. Although not reflected in the statistical analysis, case-by-case review revealed some startlingly inadequate examination reports by contract examiners, several of which contained no review of records, no report or discussion of relevant background, and no discussion or analysis of social and vocational function to inform and support their opinion about level of impairment. None of the VHA examination reports reviewed lacked information to that degree.
Such deficiencies in detail and synthesis run counter to accepted guidelines for the adequate assessment of psychological injury in general and in VA disability claims specifically.11,12 For example, Watson and colleagues proposed that a minimum of 3 hours was required to conduct an initial PTSD examination, with more complex cases possibly taking longer.11 There is no information available about how long contract examiners take to complete their examinations and how that compares with the time taken by VA examiners. The VBA failure to monitor whether or not examiners follow accepted guidelines for PTSD examination has not previously been evaluated. Historically, a large number of clinicians, researchers, and policy critics have raised concerns about the potential for exaggeration or malingering among VA PTSD disability claimants and have urged the need to adequately assess for unreliable reporting and presentation.13,14 However, the possibility of systematic examiner deficiency and/or bias increasing the frequency of false or inflated claims being approved has received little empirical attention.
Although contract examiners did not diagnose PTSD significantly more frequently than VHA examiners (71% vs 57%, respectively), the overall frequency of PTSD diagnosis across both groups (65%) was substantially higher than previous figures that have, on average, estimated the lifetime prevalence of PTSD in trauma-exposed veterans to be about 31%.15 A re-analysis of the same National Vietnam Veterans Readjustment Survey data, but applying more conservative diagnostic criteria, reduced the lifetime prevalence to just under 19%, with point prevalence estimates even lower.15,16
In a study of concordance rates between service connection for PTSD and both current and lifetime diagnosis by independent, structured assessment, Marx and colleagues found that a “significant minority” of veterans who were already receiving service-connected disability for PTSD did not meet lifetime and/or current diagnostic criteria.17 Although it is possible that the group of veterans who were applying for disability benefits in our study had a higher rate of PTSD, it also is possible if not likely that the PTSD examination process overall yields inflated rates of diagnosis and levels of impairment. This speaks to the concern raised by Marx and colleagues who found that veterans with service connection for PTSD who received related benefits “may not have the disorder.”17
Limitations
A methodological limitation of the present study was that, due to structural differences in the DBQ forms used for VHA and contract examinations, the reports could not be de-identified as to examiner type and thus raters could not be masked/blinded. To mitigate bias, a predetermined, piloted, and refined coding and rating plan for report quality metrics was adhered to strictly, and interrater reliabilities were acceptable. Future study is suggested in which all report content is standardized for coding using the same format, which at present would require a complete rewriting of the entire report; this problem could be resolved by having the VBA adopt a more coherent system in which all reports, regardless of examiner type, use a single, standardized template. Further study using larger data sets and expanding to other VA regions also is needed.
Conclusions
The present study suggests that poor examination and report quality—by contract examiners and to a lesser degree VHA examiners—are not uncommon. The findings confirm and extend previous anecdotal reports of deficiencies in PTSD examinations performed by contract examiners and provide empirical support for concerns raised of global deficiencies in the VBA oversight of the MDE program. Such deficiencies have significant implications for the quality and integrity of the VA disability determination process for veterans claiming PTSD related to military service.
The current findings support and strengthen the call for development and management of a structured and enforced training and quality assurance/improvement program for VA PTSD disability examinations. Such training and oversight will be critical to improve the quality and integrity of these examinations, reduce error and waste in VBA’s Compensation and Pension process, and in doing so optimize VA financial resources to best serve veterans’ benefits and health care needs.
1. US Department of Veterans Affairs, Veterans Health Adminstration. About VHA. Updated April 23, 2021. Accessed January 6, 2022. www.va.gov/health/aboutvha.asp
2. US Department of Labor, Bureau of Labor Statistics. News release. Employment situation of veterans—2020. Published March 18, 2020. Accessed January 6, 2022. https://www.bls.gov/news.release/pdf/vet.pdf
3. US Department of Veterans Affairs, National Center for Veterans Analysis and Statistics. Department of Veterans Affairs statistics at a glance. Updated December 31, 2020. Accessed January 6, 2022. https://www.va.gov/vetdata/docs/Quickfacts/Stats_at_a_glance_12_31_20.PDF
4. US Department of Veterans Affairs. FY 2021 Budget submission: budget in brief. Published February 2020. Accessed January 6, 2022. https://www.va.gov/budget/docs/summary/archive/FY-2021-VA-BudgetSubmission.zip
5. US Department of Veterans Affairs. FY 2021 budget submission: benefits and burial programs and Departmental Administration volume 3 of 4:178. Published February 2020. Accessed January 6, 2022. https://www.va.gov/budget/docs/summary/archive/FY-2021-VA-BudgetSubmission.zip
6. Statement of Ronald Burke, assistant deputy under secretary, office of field operations Veterans Benefits Administration before the Subcommittee on Disability And Memorial Affairs of the House Committee on Veterans’ Affairs. Published July 25, 2017. Accessed January 6, 2022. https://www.congress.gov/115/meeting/house/106322/witnesses/HHRG-115-VR09-Wstate-BurkeR-20170725.pdf
7. McNally RJ, Frueh BC. Why are Iraq and Afghanistan War veterans seeking PTSD disability compensation at unprecedented rates? J Anxiety Disord. 2013;27(5):520-526. doi:10.1016/j.janxdis.2013.07.002
8. US Government Accountability Office. VA disability exams: improved performance analysis and training oversight needed for contracted exams. GAO-19-13. Published October 2018. Accessed January 6, 2022. https://www.gao.gov/assets/gao-19-13.pdf
9. US Department of Veterans Affairs, Office of Inspector General. Inadequate oversight of contracted disability exam cancellations. Report #18-04266-115. Published June 10, 2019. Accessed January 6, 2022. https://www.va.gov/oig/pubs/VAOIG-18-04266-115.pdf
10. Letter to VA Secretary Wilkie. Published November 11, 2020. Accessed January 6, 2022. https://www.veterans.senate.gov/download/candp-exam-va-letter
11. Watson PW, McFall M, McBrine C, Schnurr PP, Friedman MJ, Keane TM, Hamblen JL (2005). Best practice manual for posttraumatic stress disorder (PTSD) compensation and pension examinations. Portland, OR: Northwest Network Mental Illness Research, Education, and Clinical Center, VA Puget Sound Healthcare System.
12. Worthen MD, Moering RG. A practical guide to conducting VA compensation and pension exams for PTSD and other mental disorders. Psychol Inj and Law. 2011;4:187-216. doi:10.1007/s12207-011-9115-2
13. DeViva JC, Bloem WD. Symptom exaggeration and compensation seeking among combat veterans with posttraumatic stress disorder. J Trauma Stress. 2003;16(5):503-507. doi:10.1023/A:1025766713188
14. Ray CL. Feigning screeners in VA PTSD compensation and pension examinations. Psychol Inj and Law. 2014;7:370-387. doi:10.1007/s12207-014-9210-2
15. Kulka RA, Schlenger WE, Fairbank JA, et al. Trauma and the Vietnam War Generation: Report of Findings From the National Vietnam Veterans Readjustment Study. Brunner Mazel Publishers; 1990.
16. Dohrenwend BP, Turner JB, Turse NA, Adams BG, Koenen KC, Marshall R. The psychological risks of Vietnam for U.S. veterans: a revisit with new data and methods. Science. 2006;313(5789):979-982. doi:10.1126/science.1128944
17. Marx BP, Bovin MJ, Szafranski DD, et al. Validity of posttraumatic stress disorder service connection status in Veterans Affairs electronic records of Iraq and Afghanistan Veterans. J Clin Psychiatry. 2016;77(4):517-522. doi:10.4088/JCP.14m09666
1. US Department of Veterans Affairs, Veterans Health Adminstration. About VHA. Updated April 23, 2021. Accessed January 6, 2022. www.va.gov/health/aboutvha.asp
2. US Department of Labor, Bureau of Labor Statistics. News release. Employment situation of veterans—2020. Published March 18, 2020. Accessed January 6, 2022. https://www.bls.gov/news.release/pdf/vet.pdf
3. US Department of Veterans Affairs, National Center for Veterans Analysis and Statistics. Department of Veterans Affairs statistics at a glance. Updated December 31, 2020. Accessed January 6, 2022. https://www.va.gov/vetdata/docs/Quickfacts/Stats_at_a_glance_12_31_20.PDF
4. US Department of Veterans Affairs. FY 2021 Budget submission: budget in brief. Published February 2020. Accessed January 6, 2022. https://www.va.gov/budget/docs/summary/archive/FY-2021-VA-BudgetSubmission.zip
5. US Department of Veterans Affairs. FY 2021 budget submission: benefits and burial programs and Departmental Administration volume 3 of 4:178. Published February 2020. Accessed January 6, 2022. https://www.va.gov/budget/docs/summary/archive/FY-2021-VA-BudgetSubmission.zip
6. Statement of Ronald Burke, assistant deputy under secretary, office of field operations Veterans Benefits Administration before the Subcommittee on Disability And Memorial Affairs of the House Committee on Veterans’ Affairs. Published July 25, 2017. Accessed January 6, 2022. https://www.congress.gov/115/meeting/house/106322/witnesses/HHRG-115-VR09-Wstate-BurkeR-20170725.pdf
7. McNally RJ, Frueh BC. Why are Iraq and Afghanistan War veterans seeking PTSD disability compensation at unprecedented rates? J Anxiety Disord. 2013;27(5):520-526. doi:10.1016/j.janxdis.2013.07.002
8. US Government Accountability Office. VA disability exams: improved performance analysis and training oversight needed for contracted exams. GAO-19-13. Published October 2018. Accessed January 6, 2022. https://www.gao.gov/assets/gao-19-13.pdf
9. US Department of Veterans Affairs, Office of Inspector General. Inadequate oversight of contracted disability exam cancellations. Report #18-04266-115. Published June 10, 2019. Accessed January 6, 2022. https://www.va.gov/oig/pubs/VAOIG-18-04266-115.pdf
10. Letter to VA Secretary Wilkie. Published November 11, 2020. Accessed January 6, 2022. https://www.veterans.senate.gov/download/candp-exam-va-letter
11. Watson PW, McFall M, McBrine C, Schnurr PP, Friedman MJ, Keane TM, Hamblen JL (2005). Best practice manual for posttraumatic stress disorder (PTSD) compensation and pension examinations. Portland, OR: Northwest Network Mental Illness Research, Education, and Clinical Center, VA Puget Sound Healthcare System.
12. Worthen MD, Moering RG. A practical guide to conducting VA compensation and pension exams for PTSD and other mental disorders. Psychol Inj and Law. 2011;4:187-216. doi:10.1007/s12207-011-9115-2
13. DeViva JC, Bloem WD. Symptom exaggeration and compensation seeking among combat veterans with posttraumatic stress disorder. J Trauma Stress. 2003;16(5):503-507. doi:10.1023/A:1025766713188
14. Ray CL. Feigning screeners in VA PTSD compensation and pension examinations. Psychol Inj and Law. 2014;7:370-387. doi:10.1007/s12207-014-9210-2
15. Kulka RA, Schlenger WE, Fairbank JA, et al. Trauma and the Vietnam War Generation: Report of Findings From the National Vietnam Veterans Readjustment Study. Brunner Mazel Publishers; 1990.
16. Dohrenwend BP, Turner JB, Turse NA, Adams BG, Koenen KC, Marshall R. The psychological risks of Vietnam for U.S. veterans: a revisit with new data and methods. Science. 2006;313(5789):979-982. doi:10.1126/science.1128944
17. Marx BP, Bovin MJ, Szafranski DD, et al. Validity of posttraumatic stress disorder service connection status in Veterans Affairs electronic records of Iraq and Afghanistan Veterans. J Clin Psychiatry. 2016;77(4):517-522. doi:10.4088/JCP.14m09666