Updated June 17, 2022
The US Department of Veterans Affairs (VA) provides health care for > 9 million military veterans, nearly half of all former service members.1 Over the past 15 years, there has been a steady and substantial increase in the frequency of disability awards for veterans with post-9/11 military service. Recent data from the Bureau of Labor Statistics indicate that 41% of veterans who served after 9/11 receive service-connected disability benefits compared with 28% of veterans overall.2 More than 5 million veterans receive VA service-related disability benefits.2,3 More than half of the VA $243 billion budget for fiscal year (FY) 2021 ($135.5 billion) was allocated to the Veterans Benefits Administration (VBA), of which $115.7 billion (85%) was allocated specifically for service-related compensation claims payments.4
The VA predicted that VBA will have completed 1.4 million ratings for disability claims in 2021.5 A substantial percentage of these claims will be for mental disorders, specifically posttraumatic stress disorder (PTSD). VA officials testifying before Congress in 2017 noted that the number of PTSD claims had nearly tripled in the previous 10 years.6 As far back as 2013, McNally and Frueh analyzed “the skyrocketing of disability claims,” particularly for PTSD, among veterans who served in Iraq and Afghanistan.7
This large increase has placed an unprecedented burden on the VBA to expand its capacity to conduct initial PTSD disability evaluations that by regulations are completed by psychologists or psychiatrists. This need has led the VBA to make significant changes in the compensation and pension (C&P) process, including a reduced role for Veterans Health Administration (VHA) examiners and increased reliance on non-VA (contract) examiners through the Contract Medical Disability Examination (MDE) program. In 2019, the MDE budget was $1.23 billion; in 2020, it was increased to $1.79 billion, and for 2021, it was $2.23 billion, reflecting the increasing investment of resources in non-VA examiners, ostensibly to both increase capacity and save costs.5
Anecdotally, concerns have been raised regarding inadequate training of contract examiners as well as inadequate reports by these examiners. A 2018 Government Accountability Office (GAO) report concluded that VA lacked the data to determine whether contract examiners were meeting standards for quality, timeliness, and accuracy.8 The GAO report noted that VA required 92% of contractor reports contained no obvious errors, a relatively low target; however, in the first half of 2017, only 1 contractor group met that target. The report noted further that “VBA does not verify if examiners have completed training nor does it collect information to assess training effectiveness in preparing examiners.”8 A subsequent analysis of contract examinations completed by the VA Office of the Inspector General (OIG) in 2019 concluded that the MDE program was “hampered in their ability to provide oversight because of limitations with VBA’s electronic examination management systems, the lack of reliable data, and inadequate staffing of the program.”9
These reports have focused almost exclusively on simple performance metrics, such as timeliness of examination completion. However, the 2018 GAO report referenced isolated “focused reviews” of complaints about the quality of examinations by contract examiners and gave as an example an isolated “review of one contracted examiner who had high rates of diagnosing severe posttraumatic stress disorder.”8 After review indicated the examiner’s reports were of poor quality, the VBA discontinued the examiner’s contract.
Unfortunately, despite such anecdotal reports and isolated actions, to date there are no published reports examining and comparing the quality of PTSD examination reports completed by VHA and contract examiners or the subsequent disability determinations made by the VBA as a result of these evaluations. In a November 2020 letter to the VA Secretary, 11 US Senators expressed “grave concerns” regarding the VA decision to privatize C&P programs noting, among other concerns, that there were “no clinical quality measurement for, or evaluation of, contractor examinations.”10 The letter cited anecdotal evidence of contract examiners not reviewing veteran’s medical records and diagnosing conditions “without supporting evidence.”10
The purpose of the present evaluation was to provide a systematic comparison of the content and quality of initial PTSD disability examinations conducted by VHA and non-VA contract examiners. In addition, this study compared the disability rating decisions resulting from VHA and contract examinations.
Methods
A random sample of 100 Initial PTSD Disability Benefits Questionnaires (DBQs)—structured forms completed by all examiners—were obtained from a list supplied by the VA Office of Performance Analysis and Integrity. All examinations were from the Veterans Integrated Service Network (VISN) 1, encompassing the New England region and were conducted in 2019 and 2020. Two of the 100 cases were excluded for technical reasons, resulting in 98 examination reports. However, the final pool yielded 62 contract examinations and only 36 VHA examinations. To make the sample sizes more comparable, an additional 15 examinations were randomly selected from the local examination database (also VISN 1) to complement the original examination pool.
Once DBQs were retrieved, all identifying information was deleted, and cases were analyzed using assigned record numbers. All coding was completed by the 2 principal investigators, both VA psychologists with extensive training and experience in C&P evaluation and treatment of veterans with PTSD. Due to inherent structural differences between the forms used for VA and contract examinations, raters could not be masked/blinded to the source of the report.
A number of measures were taken to reduce bias and enhance objectivity of rating. First, objectively coded variables (eg, age and sex of veteran, period of service, trauma type, diagnoses rendered by the examiner, impairment category endorsed, number and type of symptoms) were transcribed directly from the DBQ as recorded by the examiner. Second, to rate report quality, an initial categorical rating scale was developed based on predetermined elements of examination quality that were considered essential. After refinement and preliminary analysis of interrater reliabilities, 3 quality-related indices were identified: (1) level of detail in description of key content areas (history before service, service trauma, after service social and vocational history, mental health history, substance use); (2) synthesis of history and findings in explaining opinion rendered; and (3) clarity of opinion regarding causation required “at least as likely as not” degree of confidence. The first 2 quality ratings were based on a 3-point scale (poor, fair, good), and the third variable was coded as yes or no. (eAppendix available at doi:10.12788/fp.0225). Interrater reliabilities calculated based on a subsample of 18 cases, randomly selected and rated by both raters, yielded Cohen κ in the acceptable range (.61, .72, and .89 for detail, synthesis, and clarity, respectively). Finally, for information regarding VBA decision making, rating decision documents contained in the Veterans Benefit Management System database were reviewed to determine whether the veteran was granted service connection for PTSD or another mental disorder based on the examination report in question and, if so, the disability rating percentage awarded. These were recorded independently after all other coding had been completed.
