Targeting Quality Improvement Activities for Depression

Depressive disorders affect 5% to 20% of primary care patients and are associated with significant morbidity, functional impairment, and increased medical costs.^1-5 Yet depression often remains undiagnosed, and even when diagnosed, patients may experience continued morbidity.^6-10 Because patients with depression incur high costs and may have disappointing outcomes, researchers and quality improvement (QI) specialists have become interested in monitoring and improving their care.

The resources that can be devoted to improving the care of depressed patients, however, are limited. The large health care organizations that conduct the majority of QI activities in the United States operate in highly competitive markets. They must conduct all components of their QI activities (identification, treatment monitoring, and patient/provider intervention) with an eye on attendant costs.

Many organizations use administrative data to identify patients with depression for QI programs. Reviewing these data for pertinent encounter diagnoses or coding is less expensive and faster than identifying patients through other means, such as querying primary care physicians (PCPs) or reviewing medical charts. For off-site or “carved out” managed behavioral health care organizations, administrative data are often the only information available on the mental health care delivered in affiliated primary care clinics.

However, administrative data will identify only a small subset of the primary care patients with depression. Studies that use direct physician inquiry, nonspecific notation of distress, or depressive diagnoses in the medical chart identify progressively smaller proportions of depressed patients.¹¹ Because many PCPs deliberately miscode depression on encounter or billing forms,¹² claims and billing data will likely identify the smallest proportion. This select group of patients may have more severe depressive disorders or receive different treatments than patients identified through other means, such as screening, chart review, or PCP inquiry. They may also require different QI approaches.

Several studies have reported that patients whose disorders are detected by their PCPs are more likely to have a past history of depression, more severe depressive symptoms, more anxiety, and more occupational disability than those with undetected disorders.⁷ These characteristics may be even more pronounced among patients who are given an administrative diagnosis of depression. If these patients have relatively severe depressive disorders and are not receiving standard treatments, they might comprise an ideal population for common process-oriented QI programs, such as those that monitor adherence to guidelines or accepted standards of care. However, if they have relatively refractory disorders and are already receiving appropriate treatment, they might comprise a relatively poor target group for these types of activities.

We need to know more about the patients who would be identified through administrative data as having depression: their clinical characteristics, functional status, treatments, or the degree to which they might show change over time on instruments used to monitor clinical progress. Such information would be helpful for determining whether these patients comprise a suitable target group for common QI activities or for determining the types of activities that might be more appropriate.

Methods

Study Sites and Participants

There were 2 study sites: 1 family practice clinic located in a midsize midwestern city and another located in a small town 17 miles from that city. Together they served 28,000 unique patients (56,000 visits per year); approximately 70% of these patients were aged between 18 and 65 years.

Patients were eligible for participation in our study if they met the age criteria for the study (18 to 65 years) and had a written diagnosis of depressive disorder on a clinic encounter form between March 10, 1995, and July 15, 1996. Most patients were diagnosed with “depression, not otherwise specified.” Patients with encounter diagnoses or a PCP report of bipolar I disorder, psychotic depression, mental retardation, or dementia were excluded.

Encounter forms were completed by either a clinic PCP or a psychiatrist who evaluated a small number of patients referred by clinic PCPs. The encounter forms listed the diagnoses given and procedures performed during the visit, and formed the basis of the clinics’ administrative data sets.

Patient Enrollment and Data Collection

A research associate contacted the PCPs of eligible patients identified through encounter forms and requested permission to recruit their patients. If the PCPs gave permission, their patients received a letter followed by a telephone call requesting participation. Patients identified through the psychiatrist were referred directly to research staff for recruitment.

Participants were interviewed by the research associate using the Primary Care Evaluation of Mental Disorders questionnaire (PRIME-MD), a 2-step screening and diagnostic instrument. The patients also completed the baseline assessment from the Depression Outcomes Modules (DOM) 8.1,^13,14 the Symptom Check List-25 (SCL-25),^15-17 the Alcohol Use Disorders Identification Test (AUDIT),^18,19 and answered questions about their use of mental health services. They were mailed follow-up assessments approximately 5 months after study entry, including the follow-up module from the DOM 8.3. Nonrespondents received up to 3 follow-up mailings and 1 telephone call from a study staff member. Only assessments completed between 5 and 12 months postenrollment were included in the study.

We conducted a focused chart review to determine the dates when patients were first seen in the study clinics, when they first had chart notations of depression, and whether antidepressant medication was prescribed.

Measures

The DOM 8.1 and 8.3 include questions about the presence, severity, and duration of depressive symptoms, prognostic factors, the presence or absence of 20 concurrent medical conditions, mental health services use, and health-related functional status.^13,14 The DOM also includes the Medical Outcomes Study (MOS) Short Form-36 (SF-36) which assesses health status in 8 domains: physical functioning, role limitations due to physical conditions, bodily pain, general health perceptions, vitality, role limitations due to emotional conditions, social functioning, and mental health.

The AUDIT consists of 10 items and screens for alcohol-related problems,^18,19 and the SCL-25 collects information about the presence and severity of anxiety and depressive symptoms.^15-17 The PRIME-MD screens for 5 common categories of mental health disorders in primary care (mood disorders, anxiety disorders, somatoform disorders, eating disorders, and alcohol abuse) and then uses a semistructured interview to make a Diagnostic and Statistical Manual of Mental Disorders (DSM) diagnosis.²⁰

We excluded somatoform disorders from consideration, since the PRIME-MD diagnostic instrument requires the interviewer to use clinical judgment to decide if somatic symptoms have a physical explanation “sufficient to explain their severity and associated disability.” The research associate completing the PRIME-MD was not in a position to make these judgments.

All of the study instruments have satisfactory reliability and validity in primary care as demonstrated by test-retest reliability,^14,17 internal consistency of subscales,^14,19 concurrent validity,^14,18-21 or construct validity.^14,18,20,22

Data Analysis

We conducted descriptive analyses for the 103 eligible patients who enrolled in the study and 83 patients who returned follow-up assessments.

Bivariate analyses were done to evaluate differences between eligible patients who enrolled or did not enroll and between enrolled patients who did or did not return follow-up data. We used Student t tests for continuous variables and chi-squares for dichotomous or categorical variables.

We examined differences between baseline and follow-up SCL and SF-36 scores using paired t tests. The effect sizes of changes in SF-36 scores were calculated as suggested by Kazis and colleagues²³ by dividing the change score for each scale by the standard deviation of the scale at baseline. The percentages of patients who improved, remained the same, or worsened on each scale were calculated using the criterion for change of at least 1 standard error of measurement, as described by Wolinsky and coworkers.²⁴

Using multivariate analyses, we explored factors that might be associated with patient improvement. Outcome variables were change scores for the SCL depression scale and SF-36 Mental Health scale, and predictors were age, sex, presence of chronic depression, number of concurrent medical conditions, concurrent mental health care at enrollment, antidepressant medication during follow-up, and baseline SCL depression or SF-36 mental health scores. The number of concurrent medical conditions and the presence of a chronic depression were ascertained from responses to DOM questions.

All statistical analyses were completed using the SAS software, version 6.12 (SAS Inc, Cary, NC) or STATA, version 6.0 (Stata Corp, College Station, Tex).

Results

Enrollment

A total of 418 patients were identified as eligible for study participation; 397 were identified through PCP encounter forms and 21 through the psychiatrist’s encounter diagnoses. All patients identified by the psychiatrist had been referred by clinic PCPs.

Research staff requested permission from PCPs to contact the 389 patients identified as depressed on encounter forms. PCPs responded to 245 (63%) of these requests, with responding physicians giving permission for research staff to contact 200 patients (82%). Of those, 135 (68%) were contacted by telephone. Eighty-four (62%) agreed to participate. Eight patients who had been identified through PCP encounter forms were referred to the psychiatrist shortly after the PCP encounter. These 8 patients and the 21 patients identified through the psychiatrist were directly referred to the research staff for recruitment. Nineteen (66%) agreed to participate. Thus, a total of 103 patients (25% of all patients identified in the administrative data) were enrolled in our study ([Figure 1]).

Demographic characteristics of enrolled patients are summarized in [Table 1]. The majority were white (92%) and women (72%), with a mean age of 42 (±10) years.

Differences in Enrolled and Nonenrolled Patients

There were no significant differences between eligible patients who enrolled or did not enroll in sex, clinic site, or encounter diagnosis. Patients who enrolled in the study were slightly older than patients who did not enroll (mean age = 41.8 years and 39.4 years, respectively; t=-1.9; P=.054). Not surprisingly, because of the shorter recruitment pathway, patients seen by the psychiatrist were more likely to enroll than patients recruited only through PCP encounter forms (c2=27.3; P <.001).

Patient Diagnosis at Enrollment

Most patients identified as depressed by their PCPs in administrative data had a current depressive diagnosis on structured psychiatric interview (the PRIME-MD). Fifty-one percent had a primary diagnosis of current major depressive disorder (MDD); 31% had MDD in partial remission or recurrence; 3% had a dysthymic disorder; and 1% had a minor depression. Many patients (43%) had “double depression” with a primary diagnosis of MDD and a secondary diagnosis of dysthymia. Ten percent of the patients did not meet criteria for any PRIME-MD psychiatric diagnosis; however, these patients usually had significant depressive or anxiety symptoms on the SCL-25. Only 2 patients did not have a PRIME-MD psychiatric diagnosis or significant symptoms on the SCL-25 (scores Ž1.75).

Clinical Characteristics

Eighty-five percent of the patients had significant depressive symptoms and 69% had severe depressive symptoms on the SCL-25 (scores Ž1.75 or Ž2.10, respectively) at enrollment; 61% had significant anxiety symptoms. Almost all patients reported having at least 1 previous episode of a depressed or sad mood lasting 2 or more weeks; 77% reported 3 or more such episodes; and most patients (55%) indicated having such an episode by the age of 18. The majority of the patients (60%) reported being depressed most days during the previous 2 years. Although enrolled patients were fairly young, they reported having an average of 3 concurrent medical conditions at the time of enrollment. The majority (68%) also had more than one psychiatric diagnosis on the PRIME-MD.

Not surprisingly, patients had significant functional limitations on the SF-36. Their scores were significantly below United States norms on all 8 functional domains (P <.001 for all domains) and were comparable with the scores of MOS patients with depression ([Table 2]). The MOS sample included depressed patients seen by mental health providers (MHPs), in addition to those seen in primary care.

Previous Treatment

Enrolled patients reported high rates of previous treatment for psychiatric conditions; 83%, previous counseling or psychotherapy; 91%, previous medication for depression; and 16%, previous psychiatric hospitalization. Thirty-three percent reported that they were currently in treatment with a MHP in addition to their PCP. When patients seen by the psychiatrist in proximity to enrollment were excluded these figures did not greatly change: 86% reported previous counseling or psychotherapy; 94%, previous medication; and 28%, concurrent MHP treatment.

Patient Follow-Up

Eighty-three patients (81%) returned follow-up questionnaires between 5 and 12 months (median=7 months) after enrollment. Patients who returned questionnaires did not differ from nonreturners in age, sex, education, presence of a PRIME-MD diagnosis, baseline SCL depression scores, AUDIT scores, or most SF-36 domains. They did have lower physical functioning (t=2.15; P=.03) and lower vitality scores (t=2.62; P=.01) at baseline than nonreturners.

Patients continued to report high rates of treatment during the months following their identification in administrative data. Ninety percent had a medication prescribed; 35% were in concurrent treatment with a MHP at the time of follow-up; and 3% were hospitalized for depression during the follow-up period. When patients identified by the psychiatrist were excluded, 92% received medications, and 30% were in concurrent MHP treatment at follow-up.

Changes in depressive symptoms and functioning are summarized in [Table 3]. As a group, patients improved significantly on the SCL-25 depression scale and in the SF-36 functional domains thought to be most affected by mental disorders: vitality, social functioning, role functioning-emotional, and mental health. There were no comparable changes in SF-36 domains related to physical functioning, despite significant limitations in these domains at enrollment. Aggregate changes appeared to be the result of moderate improvements in many patients rather than dramatic changes in a minority.

However, despite significant improvements, patients continued to demonstrate deficits in all SF-36 domains compared with United States norms at follow-up (P <.001 for all domains). The majority (60%) also continued to have clinically significant depressive symptoms on the SCL-25.

Predictors of Improvement

Exploratory analyses of factors associated with improvement indicated that patients with more symptomatic or functional impairment at baseline showed the most improvement at follow-up. Having chronic depressive symptoms (patient report of feeling depressed most days during the last 2 years) was associated with less improvement in depression scores. Sex, age, concurrent mental health treatment at the time of enrollment, treatment with medication during follow-up, and number of concurrent medical conditions did not predict improvement.

Discussion

Only a small percentage of primary care patients are identified as depressed in encounter or administrative data, and these patients have relatively severe, recurrent DSM depressive disorders and high levels of symptoms at the time of the notations. Thus, organizations using administrative data to target patients for QI efforts will likely select a population with clinically significant and currently symptomatic disorders.

Most patients identified in this manner will have received standard treatments in the past, including medication and psychotherapy, and will continue to receive antidepressant medication during follow-up, even without QI intervention. A substantial minority will also receive concurrent treatment by a MHP.

Although QI programs will find that most patients identified in encounter data improve during the next 5 to 12 months, these patients are likely to continue to experience significant depressive symptoms and functional limitations in SF-36 domains. Patients improve more in functional domains related to mental health than physical health, suggesting that some improvements could be due to treatment; however, some improvements may also be due to regression to the mean.

These findings have implications for organizations that are considering using administrative data to select patients for QI programs. Since patients identified in this manner are likely to be highly symptomatic, meet full criteria for DSM depressive disorders, and continue to experience symptoms over time-despite receiving antidepressants and having intermittent contact with MHPs-limited QI programs may not be sufficient. QI programs that simply assure the provision of antidepressants or encourage referral to mental health professionals, while undoubtedly helpful for some populations of patients, are unlikely to improve the outcomes of this target group. More intensive QI programs may be required, such as programs that monitor clinical status or offer intensive alternatives, such as a disease management or stepped care.

Limitations

Several limitations should be considered when interpreting our findings. First, our study was conducted in only 2 sites. Physicians in other settings may be more or less likely to note depressive diagnoses in administrative data. Many settings will not have MHPs available on a consultative basis; having this availability may affect the frequency of depression diagnosis or treatments prescribed. A PCP’s willingness to make an encounter diagnosis of depression is likely influenced by a number of factors, including the characteristics of the clinic population, clinic norms, team composition, and reimbursement policies of third-party payers.

Only 25% of the patients identified as depressed in administrative data were enrolled in our study, and the possibility of systematic selection must be carefully considered. Biases may have been introduced by many factors: variation in PCP willingness to have patients recruited into the study; which patients were approved for recruitment; who could be contacted by telephone; who agreed to participate; and the shorter recruitment route for patients seen by the psychiatrist. Most potential enrollees were lost simply because clinicians failed to respond to written requests for permission to contact their patients. Yet, more than 60% of patients were retained at each step in the enrollment process, and no systematic differences were found between enrolled and nonenrolled patients in sex, clinic site, or encounter diagnosis. Although there was considerable variation in the time to follow-up assessment, with mailed assessments being returned between 5 and 12 months, time frames of 6 to 12 months are commonly used in QI studies of depression outcomes.

Since QI programs will often need to secure PCP permission and patient cooperation if they are to monitor patients’ clinical status, they are likely to face many of the same selection pressures that potentially affected our results. They are also likely to experience similar variation in the timing of patients’ completion and return of any mailed assessments. The enrollment rate in our study is similar to that of many QI projects in primary care. Thus, our data remain germane to QI programs that are considering targeting this population.

Conclusions

Health care organizations using administrative data to identify primary care patients with depression for QI activities are likely to select a cohort that is significantly ill, has had previous trials of psychotherapy, and is currently receiving antidepressants. Most patients will improve, yet many will continue to have significant symptoms. For QI efforts to have a significant impact on this population, they may need to offer intensive alternatives, such as disease management programs, intensive monitoring of patient adherence, or stepped care, rather than simply monitoring the provision of standard treatments.

Acknowledgments

Funding for this research was provided by the Office of Clinical Affairs, University of Michigan, and the Department of Veterans Affairs, HSR&D Research Career Development Award, RC-98350-1. We would like to thank the primary care providers in the BWO and CFC clinics for their assistance in this study. They would also like to thank Sharon Blanchard for data support. The study protocol was approved by the University of Michigan Institutional Review Board and by research committees at the study clinics.

Depressive disorders affect 5% to 20% of primary care patients and are associated with significant morbidity, functional impairment, and increased medical costs.^1-5 Yet depression often remains undiagnosed, and even when diagnosed, patients may experience continued morbidity.^6-10 Because patients with depression incur high costs and may have disappointing outcomes, researchers and quality improvement (QI) specialists have become interested in monitoring and improving their care.

The resources that can be devoted to improving the care of depressed patients, however, are limited. The large health care organizations that conduct the majority of QI activities in the United States operate in highly competitive markets. They must conduct all components of their QI activities (identification, treatment monitoring, and patient/provider intervention) with an eye on attendant costs.

Many organizations use administrative data to identify patients with depression for QI programs. Reviewing these data for pertinent encounter diagnoses or coding is less expensive and faster than identifying patients through other means, such as querying primary care physicians (PCPs) or reviewing medical charts. For off-site or “carved out” managed behavioral health care organizations, administrative data are often the only information available on the mental health care delivered in affiliated primary care clinics.

However, administrative data will identify only a small subset of the primary care patients with depression. Studies that use direct physician inquiry, nonspecific notation of distress, or depressive diagnoses in the medical chart identify progressively smaller proportions of depressed patients.¹¹ Because many PCPs deliberately miscode depression on encounter or billing forms,¹² claims and billing data will likely identify the smallest proportion. This select group of patients may have more severe depressive disorders or receive different treatments than patients identified through other means, such as screening, chart review, or PCP inquiry. They may also require different QI approaches.

Several studies have reported that patients whose disorders are detected by their PCPs are more likely to have a past history of depression, more severe depressive symptoms, more anxiety, and more occupational disability than those with undetected disorders.⁷ These characteristics may be even more pronounced among patients who are given an administrative diagnosis of depression. If these patients have relatively severe depressive disorders and are not receiving standard treatments, they might comprise an ideal population for common process-oriented QI programs, such as those that monitor adherence to guidelines or accepted standards of care. However, if they have relatively refractory disorders and are already receiving appropriate treatment, they might comprise a relatively poor target group for these types of activities.

We need to know more about the patients who would be identified through administrative data as having depression: their clinical characteristics, functional status, treatments, or the degree to which they might show change over time on instruments used to monitor clinical progress. Such information would be helpful for determining whether these patients comprise a suitable target group for common QI activities or for determining the types of activities that might be more appropriate.

Methods

Study Sites and Participants

There were 2 study sites: 1 family practice clinic located in a midsize midwestern city and another located in a small town 17 miles from that city. Together they served 28,000 unique patients (56,000 visits per year); approximately 70% of these patients were aged between 18 and 65 years.

Patients were eligible for participation in our study if they met the age criteria for the study (18 to 65 years) and had a written diagnosis of depressive disorder on a clinic encounter form between March 10, 1995, and July 15, 1996. Most patients were diagnosed with “depression, not otherwise specified.” Patients with encounter diagnoses or a PCP report of bipolar I disorder, psychotic depression, mental retardation, or dementia were excluded.

Encounter forms were completed by either a clinic PCP or a psychiatrist who evaluated a small number of patients referred by clinic PCPs. The encounter forms listed the diagnoses given and procedures performed during the visit, and formed the basis of the clinics’ administrative data sets.

Patient Enrollment and Data Collection

A research associate contacted the PCPs of eligible patients identified through encounter forms and requested permission to recruit their patients. If the PCPs gave permission, their patients received a letter followed by a telephone call requesting participation. Patients identified through the psychiatrist were referred directly to research staff for recruitment.

Participants were interviewed by the research associate using the Primary Care Evaluation of Mental Disorders questionnaire (PRIME-MD), a 2-step screening and diagnostic instrument. The patients also completed the baseline assessment from the Depression Outcomes Modules (DOM) 8.1,^13,14 the Symptom Check List-25 (SCL-25),^15-17 the Alcohol Use Disorders Identification Test (AUDIT),^18,19 and answered questions about their use of mental health services. They were mailed follow-up assessments approximately 5 months after study entry, including the follow-up module from the DOM 8.3. Nonrespondents received up to 3 follow-up mailings and 1 telephone call from a study staff member. Only assessments completed between 5 and 12 months postenrollment were included in the study.

We conducted a focused chart review to determine the dates when patients were first seen in the study clinics, when they first had chart notations of depression, and whether antidepressant medication was prescribed.

Measures

The DOM 8.1 and 8.3 include questions about the presence, severity, and duration of depressive symptoms, prognostic factors, the presence or absence of 20 concurrent medical conditions, mental health services use, and health-related functional status.^13,14 The DOM also includes the Medical Outcomes Study (MOS) Short Form-36 (SF-36) which assesses health status in 8 domains: physical functioning, role limitations due to physical conditions, bodily pain, general health perceptions, vitality, role limitations due to emotional conditions, social functioning, and mental health.

The AUDIT consists of 10 items and screens for alcohol-related problems,^18,19 and the SCL-25 collects information about the presence and severity of anxiety and depressive symptoms.^15-17 The PRIME-MD screens for 5 common categories of mental health disorders in primary care (mood disorders, anxiety disorders, somatoform disorders, eating disorders, and alcohol abuse) and then uses a semistructured interview to make a Diagnostic and Statistical Manual of Mental Disorders (DSM) diagnosis.²⁰

We excluded somatoform disorders from consideration, since the PRIME-MD diagnostic instrument requires the interviewer to use clinical judgment to decide if somatic symptoms have a physical explanation “sufficient to explain their severity and associated disability.” The research associate completing the PRIME-MD was not in a position to make these judgments.

All of the study instruments have satisfactory reliability and validity in primary care as demonstrated by test-retest reliability,^14,17 internal consistency of subscales,^14,19 concurrent validity,^14,18-21 or construct validity.^14,18,20,22

Data Analysis

We conducted descriptive analyses for the 103 eligible patients who enrolled in the study and 83 patients who returned follow-up assessments.

Bivariate analyses were done to evaluate differences between eligible patients who enrolled or did not enroll and between enrolled patients who did or did not return follow-up data. We used Student t tests for continuous variables and chi-squares for dichotomous or categorical variables.

We examined differences between baseline and follow-up SCL and SF-36 scores using paired t tests. The effect sizes of changes in SF-36 scores were calculated as suggested by Kazis and colleagues²³ by dividing the change score for each scale by the standard deviation of the scale at baseline. The percentages of patients who improved, remained the same, or worsened on each scale were calculated using the criterion for change of at least 1 standard error of measurement, as described by Wolinsky and coworkers.²⁴

Using multivariate analyses, we explored factors that might be associated with patient improvement. Outcome variables were change scores for the SCL depression scale and SF-36 Mental Health scale, and predictors were age, sex, presence of chronic depression, number of concurrent medical conditions, concurrent mental health care at enrollment, antidepressant medication during follow-up, and baseline SCL depression or SF-36 mental health scores. The number of concurrent medical conditions and the presence of a chronic depression were ascertained from responses to DOM questions.

All statistical analyses were completed using the SAS software, version 6.12 (SAS Inc, Cary, NC) or STATA, version 6.0 (Stata Corp, College Station, Tex).

Results

Enrollment

A total of 418 patients were identified as eligible for study participation; 397 were identified through PCP encounter forms and 21 through the psychiatrist’s encounter diagnoses. All patients identified by the psychiatrist had been referred by clinic PCPs.

Research staff requested permission from PCPs to contact the 389 patients identified as depressed on encounter forms. PCPs responded to 245 (63%) of these requests, with responding physicians giving permission for research staff to contact 200 patients (82%). Of those, 135 (68%) were contacted by telephone. Eighty-four (62%) agreed to participate. Eight patients who had been identified through PCP encounter forms were referred to the psychiatrist shortly after the PCP encounter. These 8 patients and the 21 patients identified through the psychiatrist were directly referred to the research staff for recruitment. Nineteen (66%) agreed to participate. Thus, a total of 103 patients (25% of all patients identified in the administrative data) were enrolled in our study ([Figure 1]).

Demographic characteristics of enrolled patients are summarized in [Table 1]. The majority were white (92%) and women (72%), with a mean age of 42 (±10) years.

Differences in Enrolled and Nonenrolled Patients

There were no significant differences between eligible patients who enrolled or did not enroll in sex, clinic site, or encounter diagnosis. Patients who enrolled in the study were slightly older than patients who did not enroll (mean age = 41.8 years and 39.4 years, respectively; t=-1.9; P=.054). Not surprisingly, because of the shorter recruitment pathway, patients seen by the psychiatrist were more likely to enroll than patients recruited only through PCP encounter forms (c2=27.3; P <.001).

Patient Diagnosis at Enrollment

Most patients identified as depressed by their PCPs in administrative data had a current depressive diagnosis on structured psychiatric interview (the PRIME-MD). Fifty-one percent had a primary diagnosis of current major depressive disorder (MDD); 31% had MDD in partial remission or recurrence; 3% had a dysthymic disorder; and 1% had a minor depression. Many patients (43%) had “double depression” with a primary diagnosis of MDD and a secondary diagnosis of dysthymia. Ten percent of the patients did not meet criteria for any PRIME-MD psychiatric diagnosis; however, these patients usually had significant depressive or anxiety symptoms on the SCL-25. Only 2 patients did not have a PRIME-MD psychiatric diagnosis or significant symptoms on the SCL-25 (scores Ž1.75).

Clinical Characteristics

Eighty-five percent of the patients had significant depressive symptoms and 69% had severe depressive symptoms on the SCL-25 (scores Ž1.75 or Ž2.10, respectively) at enrollment; 61% had significant anxiety symptoms. Almost all patients reported having at least 1 previous episode of a depressed or sad mood lasting 2 or more weeks; 77% reported 3 or more such episodes; and most patients (55%) indicated having such an episode by the age of 18. The majority of the patients (60%) reported being depressed most days during the previous 2 years. Although enrolled patients were fairly young, they reported having an average of 3 concurrent medical conditions at the time of enrollment. The majority (68%) also had more than one psychiatric diagnosis on the PRIME-MD.

Not surprisingly, patients had significant functional limitations on the SF-36. Their scores were significantly below United States norms on all 8 functional domains (P <.001 for all domains) and were comparable with the scores of MOS patients with depression ([Table 2]). The MOS sample included depressed patients seen by mental health providers (MHPs), in addition to those seen in primary care.

Previous Treatment

Enrolled patients reported high rates of previous treatment for psychiatric conditions; 83%, previous counseling or psychotherapy; 91%, previous medication for depression; and 16%, previous psychiatric hospitalization. Thirty-three percent reported that they were currently in treatment with a MHP in addition to their PCP. When patients seen by the psychiatrist in proximity to enrollment were excluded these figures did not greatly change: 86% reported previous counseling or psychotherapy; 94%, previous medication; and 28%, concurrent MHP treatment.

Patient Follow-Up

Eighty-three patients (81%) returned follow-up questionnaires between 5 and 12 months (median=7 months) after enrollment. Patients who returned questionnaires did not differ from nonreturners in age, sex, education, presence of a PRIME-MD diagnosis, baseline SCL depression scores, AUDIT scores, or most SF-36 domains. They did have lower physical functioning (t=2.15; P=.03) and lower vitality scores (t=2.62; P=.01) at baseline than nonreturners.

Patients continued to report high rates of treatment during the months following their identification in administrative data. Ninety percent had a medication prescribed; 35% were in concurrent treatment with a MHP at the time of follow-up; and 3% were hospitalized for depression during the follow-up period. When patients identified by the psychiatrist were excluded, 92% received medications, and 30% were in concurrent MHP treatment at follow-up.

Changes in depressive symptoms and functioning are summarized in [Table 3]. As a group, patients improved significantly on the SCL-25 depression scale and in the SF-36 functional domains thought to be most affected by mental disorders: vitality, social functioning, role functioning-emotional, and mental health. There were no comparable changes in SF-36 domains related to physical functioning, despite significant limitations in these domains at enrollment. Aggregate changes appeared to be the result of moderate improvements in many patients rather than dramatic changes in a minority.

However, despite significant improvements, patients continued to demonstrate deficits in all SF-36 domains compared with United States norms at follow-up (P <.001 for all domains). The majority (60%) also continued to have clinically significant depressive symptoms on the SCL-25.

Predictors of Improvement

Exploratory analyses of factors associated with improvement indicated that patients with more symptomatic or functional impairment at baseline showed the most improvement at follow-up. Having chronic depressive symptoms (patient report of feeling depressed most days during the last 2 years) was associated with less improvement in depression scores. Sex, age, concurrent mental health treatment at the time of enrollment, treatment with medication during follow-up, and number of concurrent medical conditions did not predict improvement.

Discussion

Only a small percentage of primary care patients are identified as depressed in encounter or administrative data, and these patients have relatively severe, recurrent DSM depressive disorders and high levels of symptoms at the time of the notations. Thus, organizations using administrative data to target patients for QI efforts will likely select a population with clinically significant and currently symptomatic disorders.

Most patients identified in this manner will have received standard treatments in the past, including medication and psychotherapy, and will continue to receive antidepressant medication during follow-up, even without QI intervention. A substantial minority will also receive concurrent treatment by a MHP.

Although QI programs will find that most patients identified in encounter data improve during the next 5 to 12 months, these patients are likely to continue to experience significant depressive symptoms and functional limitations in SF-36 domains. Patients improve more in functional domains related to mental health than physical health, suggesting that some improvements could be due to treatment; however, some improvements may also be due to regression to the mean.

These findings have implications for organizations that are considering using administrative data to select patients for QI programs. Since patients identified in this manner are likely to be highly symptomatic, meet full criteria for DSM depressive disorders, and continue to experience symptoms over time-despite receiving antidepressants and having intermittent contact with MHPs-limited QI programs may not be sufficient. QI programs that simply assure the provision of antidepressants or encourage referral to mental health professionals, while undoubtedly helpful for some populations of patients, are unlikely to improve the outcomes of this target group. More intensive QI programs may be required, such as programs that monitor clinical status or offer intensive alternatives, such as a disease management or stepped care.

Limitations

Several limitations should be considered when interpreting our findings. First, our study was conducted in only 2 sites. Physicians in other settings may be more or less likely to note depressive diagnoses in administrative data. Many settings will not have MHPs available on a consultative basis; having this availability may affect the frequency of depression diagnosis or treatments prescribed. A PCP’s willingness to make an encounter diagnosis of depression is likely influenced by a number of factors, including the characteristics of the clinic population, clinic norms, team composition, and reimbursement policies of third-party payers.

Only 25% of the patients identified as depressed in administrative data were enrolled in our study, and the possibility of systematic selection must be carefully considered. Biases may have been introduced by many factors: variation in PCP willingness to have patients recruited into the study; which patients were approved for recruitment; who could be contacted by telephone; who agreed to participate; and the shorter recruitment route for patients seen by the psychiatrist. Most potential enrollees were lost simply because clinicians failed to respond to written requests for permission to contact their patients. Yet, more than 60% of patients were retained at each step in the enrollment process, and no systematic differences were found between enrolled and nonenrolled patients in sex, clinic site, or encounter diagnosis. Although there was considerable variation in the time to follow-up assessment, with mailed assessments being returned between 5 and 12 months, time frames of 6 to 12 months are commonly used in QI studies of depression outcomes.

Since QI programs will often need to secure PCP permission and patient cooperation if they are to monitor patients’ clinical status, they are likely to face many of the same selection pressures that potentially affected our results. They are also likely to experience similar variation in the timing of patients’ completion and return of any mailed assessments. The enrollment rate in our study is similar to that of many QI projects in primary care. Thus, our data remain germane to QI programs that are considering targeting this population.

Conclusions

Health care organizations using administrative data to identify primary care patients with depression for QI activities are likely to select a cohort that is significantly ill, has had previous trials of psychotherapy, and is currently receiving antidepressants. Most patients will improve, yet many will continue to have significant symptoms. For QI efforts to have a significant impact on this population, they may need to offer intensive alternatives, such as disease management programs, intensive monitoring of patient adherence, or stepped care, rather than simply monitoring the provision of standard treatments.

Acknowledgments

Funding for this research was provided by the Office of Clinical Affairs, University of Michigan, and the Department of Veterans Affairs, HSR&D Research Career Development Award, RC-98350-1. We would like to thank the primary care providers in the BWO and CFC clinics for their assistance in this study. They would also like to thank Sharon Blanchard for data support. The study protocol was approved by the University of Michigan Institutional Review Board and by research committees at the study clinics.

Depressive disorders affect 5% to 20% of primary care patients and are associated with significant morbidity, functional impairment, and increased medical costs.^1-5 Yet depression often remains undiagnosed, and even when diagnosed, patients may experience continued morbidity.^6-10 Because patients with depression incur high costs and may have disappointing outcomes, researchers and quality improvement (QI) specialists have become interested in monitoring and improving their care.

The resources that can be devoted to improving the care of depressed patients, however, are limited. The large health care organizations that conduct the majority of QI activities in the United States operate in highly competitive markets. They must conduct all components of their QI activities (identification, treatment monitoring, and patient/provider intervention) with an eye on attendant costs.

Many organizations use administrative data to identify patients with depression for QI programs. Reviewing these data for pertinent encounter diagnoses or coding is less expensive and faster than identifying patients through other means, such as querying primary care physicians (PCPs) or reviewing medical charts. For off-site or “carved out” managed behavioral health care organizations, administrative data are often the only information available on the mental health care delivered in affiliated primary care clinics.

However, administrative data will identify only a small subset of the primary care patients with depression. Studies that use direct physician inquiry, nonspecific notation of distress, or depressive diagnoses in the medical chart identify progressively smaller proportions of depressed patients.¹¹ Because many PCPs deliberately miscode depression on encounter or billing forms,¹² claims and billing data will likely identify the smallest proportion. This select group of patients may have more severe depressive disorders or receive different treatments than patients identified through other means, such as screening, chart review, or PCP inquiry. They may also require different QI approaches.

Several studies have reported that patients whose disorders are detected by their PCPs are more likely to have a past history of depression, more severe depressive symptoms, more anxiety, and more occupational disability than those with undetected disorders.⁷ These characteristics may be even more pronounced among patients who are given an administrative diagnosis of depression. If these patients have relatively severe depressive disorders and are not receiving standard treatments, they might comprise an ideal population for common process-oriented QI programs, such as those that monitor adherence to guidelines or accepted standards of care. However, if they have relatively refractory disorders and are already receiving appropriate treatment, they might comprise a relatively poor target group for these types of activities.

We need to know more about the patients who would be identified through administrative data as having depression: their clinical characteristics, functional status, treatments, or the degree to which they might show change over time on instruments used to monitor clinical progress. Such information would be helpful for determining whether these patients comprise a suitable target group for common QI activities or for determining the types of activities that might be more appropriate.

Methods

Study Sites and Participants

There were 2 study sites: 1 family practice clinic located in a midsize midwestern city and another located in a small town 17 miles from that city. Together they served 28,000 unique patients (56,000 visits per year); approximately 70% of these patients were aged between 18 and 65 years.

Patients were eligible for participation in our study if they met the age criteria for the study (18 to 65 years) and had a written diagnosis of depressive disorder on a clinic encounter form between March 10, 1995, and July 15, 1996. Most patients were diagnosed with “depression, not otherwise specified.” Patients with encounter diagnoses or a PCP report of bipolar I disorder, psychotic depression, mental retardation, or dementia were excluded.

Encounter forms were completed by either a clinic PCP or a psychiatrist who evaluated a small number of patients referred by clinic PCPs. The encounter forms listed the diagnoses given and procedures performed during the visit, and formed the basis of the clinics’ administrative data sets.

Patient Enrollment and Data Collection

A research associate contacted the PCPs of eligible patients identified through encounter forms and requested permission to recruit their patients. If the PCPs gave permission, their patients received a letter followed by a telephone call requesting participation. Patients identified through the psychiatrist were referred directly to research staff for recruitment.

Participants were interviewed by the research associate using the Primary Care Evaluation of Mental Disorders questionnaire (PRIME-MD), a 2-step screening and diagnostic instrument. The patients also completed the baseline assessment from the Depression Outcomes Modules (DOM) 8.1,^13,14 the Symptom Check List-25 (SCL-25),^15-17 the Alcohol Use Disorders Identification Test (AUDIT),^18,19 and answered questions about their use of mental health services. They were mailed follow-up assessments approximately 5 months after study entry, including the follow-up module from the DOM 8.3. Nonrespondents received up to 3 follow-up mailings and 1 telephone call from a study staff member. Only assessments completed between 5 and 12 months postenrollment were included in the study.

We conducted a focused chart review to determine the dates when patients were first seen in the study clinics, when they first had chart notations of depression, and whether antidepressant medication was prescribed.

Measures

The DOM 8.1 and 8.3 include questions about the presence, severity, and duration of depressive symptoms, prognostic factors, the presence or absence of 20 concurrent medical conditions, mental health services use, and health-related functional status.^13,14 The DOM also includes the Medical Outcomes Study (MOS) Short Form-36 (SF-36) which assesses health status in 8 domains: physical functioning, role limitations due to physical conditions, bodily pain, general health perceptions, vitality, role limitations due to emotional conditions, social functioning, and mental health.

The AUDIT consists of 10 items and screens for alcohol-related problems,^18,19 and the SCL-25 collects information about the presence and severity of anxiety and depressive symptoms.^15-17 The PRIME-MD screens for 5 common categories of mental health disorders in primary care (mood disorders, anxiety disorders, somatoform disorders, eating disorders, and alcohol abuse) and then uses a semistructured interview to make a Diagnostic and Statistical Manual of Mental Disorders (DSM) diagnosis.²⁰

We excluded somatoform disorders from consideration, since the PRIME-MD diagnostic instrument requires the interviewer to use clinical judgment to decide if somatic symptoms have a physical explanation “sufficient to explain their severity and associated disability.” The research associate completing the PRIME-MD was not in a position to make these judgments.

All of the study instruments have satisfactory reliability and validity in primary care as demonstrated by test-retest reliability,^14,17 internal consistency of subscales,^14,19 concurrent validity,^14,18-21 or construct validity.^14,18,20,22

Data Analysis

We conducted descriptive analyses for the 103 eligible patients who enrolled in the study and 83 patients who returned follow-up assessments.

Bivariate analyses were done to evaluate differences between eligible patients who enrolled or did not enroll and between enrolled patients who did or did not return follow-up data. We used Student t tests for continuous variables and chi-squares for dichotomous or categorical variables.

We examined differences between baseline and follow-up SCL and SF-36 scores using paired t tests. The effect sizes of changes in SF-36 scores were calculated as suggested by Kazis and colleagues²³ by dividing the change score for each scale by the standard deviation of the scale at baseline. The percentages of patients who improved, remained the same, or worsened on each scale were calculated using the criterion for change of at least 1 standard error of measurement, as described by Wolinsky and coworkers.²⁴

Using multivariate analyses, we explored factors that might be associated with patient improvement. Outcome variables were change scores for the SCL depression scale and SF-36 Mental Health scale, and predictors were age, sex, presence of chronic depression, number of concurrent medical conditions, concurrent mental health care at enrollment, antidepressant medication during follow-up, and baseline SCL depression or SF-36 mental health scores. The number of concurrent medical conditions and the presence of a chronic depression were ascertained from responses to DOM questions.

All statistical analyses were completed using the SAS software, version 6.12 (SAS Inc, Cary, NC) or STATA, version 6.0 (Stata Corp, College Station, Tex).

Results

Enrollment

A total of 418 patients were identified as eligible for study participation; 397 were identified through PCP encounter forms and 21 through the psychiatrist’s encounter diagnoses. All patients identified by the psychiatrist had been referred by clinic PCPs.

Research staff requested permission from PCPs to contact the 389 patients identified as depressed on encounter forms. PCPs responded to 245 (63%) of these requests, with responding physicians giving permission for research staff to contact 200 patients (82%). Of those, 135 (68%) were contacted by telephone. Eighty-four (62%) agreed to participate. Eight patients who had been identified through PCP encounter forms were referred to the psychiatrist shortly after the PCP encounter. These 8 patients and the 21 patients identified through the psychiatrist were directly referred to the research staff for recruitment. Nineteen (66%) agreed to participate. Thus, a total of 103 patients (25% of all patients identified in the administrative data) were enrolled in our study ([Figure 1]).

Demographic characteristics of enrolled patients are summarized in [Table 1]. The majority were white (92%) and women (72%), with a mean age of 42 (±10) years.

Differences in Enrolled and Nonenrolled Patients

There were no significant differences between eligible patients who enrolled or did not enroll in sex, clinic site, or encounter diagnosis. Patients who enrolled in the study were slightly older than patients who did not enroll (mean age = 41.8 years and 39.4 years, respectively; t=-1.9; P=.054). Not surprisingly, because of the shorter recruitment pathway, patients seen by the psychiatrist were more likely to enroll than patients recruited only through PCP encounter forms (c2=27.3; P <.001).

Patient Diagnosis at Enrollment

Most patients identified as depressed by their PCPs in administrative data had a current depressive diagnosis on structured psychiatric interview (the PRIME-MD). Fifty-one percent had a primary diagnosis of current major depressive disorder (MDD); 31% had MDD in partial remission or recurrence; 3% had a dysthymic disorder; and 1% had a minor depression. Many patients (43%) had “double depression” with a primary diagnosis of MDD and a secondary diagnosis of dysthymia. Ten percent of the patients did not meet criteria for any PRIME-MD psychiatric diagnosis; however, these patients usually had significant depressive or anxiety symptoms on the SCL-25. Only 2 patients did not have a PRIME-MD psychiatric diagnosis or significant symptoms on the SCL-25 (scores Ž1.75).

Clinical Characteristics

Eighty-five percent of the patients had significant depressive symptoms and 69% had severe depressive symptoms on the SCL-25 (scores Ž1.75 or Ž2.10, respectively) at enrollment; 61% had significant anxiety symptoms. Almost all patients reported having at least 1 previous episode of a depressed or sad mood lasting 2 or more weeks; 77% reported 3 or more such episodes; and most patients (55%) indicated having such an episode by the age of 18. The majority of the patients (60%) reported being depressed most days during the previous 2 years. Although enrolled patients were fairly young, they reported having an average of 3 concurrent medical conditions at the time of enrollment. The majority (68%) also had more than one psychiatric diagnosis on the PRIME-MD.

Not surprisingly, patients had significant functional limitations on the SF-36. Their scores were significantly below United States norms on all 8 functional domains (P <.001 for all domains) and were comparable with the scores of MOS patients with depression ([Table 2]). The MOS sample included depressed patients seen by mental health providers (MHPs), in addition to those seen in primary care.

Previous Treatment

Enrolled patients reported high rates of previous treatment for psychiatric conditions; 83%, previous counseling or psychotherapy; 91%, previous medication for depression; and 16%, previous psychiatric hospitalization. Thirty-three percent reported that they were currently in treatment with a MHP in addition to their PCP. When patients seen by the psychiatrist in proximity to enrollment were excluded these figures did not greatly change: 86% reported previous counseling or psychotherapy; 94%, previous medication; and 28%, concurrent MHP treatment.

Patient Follow-Up

Eighty-three patients (81%) returned follow-up questionnaires between 5 and 12 months (median=7 months) after enrollment. Patients who returned questionnaires did not differ from nonreturners in age, sex, education, presence of a PRIME-MD diagnosis, baseline SCL depression scores, AUDIT scores, or most SF-36 domains. They did have lower physical functioning (t=2.15; P=.03) and lower vitality scores (t=2.62; P=.01) at baseline than nonreturners.

Patients continued to report high rates of treatment during the months following their identification in administrative data. Ninety percent had a medication prescribed; 35% were in concurrent treatment with a MHP at the time of follow-up; and 3% were hospitalized for depression during the follow-up period. When patients identified by the psychiatrist were excluded, 92% received medications, and 30% were in concurrent MHP treatment at follow-up.

Changes in depressive symptoms and functioning are summarized in [Table 3]. As a group, patients improved significantly on the SCL-25 depression scale and in the SF-36 functional domains thought to be most affected by mental disorders: vitality, social functioning, role functioning-emotional, and mental health. There were no comparable changes in SF-36 domains related to physical functioning, despite significant limitations in these domains at enrollment. Aggregate changes appeared to be the result of moderate improvements in many patients rather than dramatic changes in a minority.

However, despite significant improvements, patients continued to demonstrate deficits in all SF-36 domains compared with United States norms at follow-up (P <.001 for all domains). The majority (60%) also continued to have clinically significant depressive symptoms on the SCL-25.

Predictors of Improvement

Exploratory analyses of factors associated with improvement indicated that patients with more symptomatic or functional impairment at baseline showed the most improvement at follow-up. Having chronic depressive symptoms (patient report of feeling depressed most days during the last 2 years) was associated with less improvement in depression scores. Sex, age, concurrent mental health treatment at the time of enrollment, treatment with medication during follow-up, and number of concurrent medical conditions did not predict improvement.

Discussion

Only a small percentage of primary care patients are identified as depressed in encounter or administrative data, and these patients have relatively severe, recurrent DSM depressive disorders and high levels of symptoms at the time of the notations. Thus, organizations using administrative data to target patients for QI efforts will likely select a population with clinically significant and currently symptomatic disorders.

Most patients identified in this manner will have received standard treatments in the past, including medication and psychotherapy, and will continue to receive antidepressant medication during follow-up, even without QI intervention. A substantial minority will also receive concurrent treatment by a MHP.

Although QI programs will find that most patients identified in encounter data improve during the next 5 to 12 months, these patients are likely to continue to experience significant depressive symptoms and functional limitations in SF-36 domains. Patients improve more in functional domains related to mental health than physical health, suggesting that some improvements could be due to treatment; however, some improvements may also be due to regression to the mean.

These findings have implications for organizations that are considering using administrative data to select patients for QI programs. Since patients identified in this manner are likely to be highly symptomatic, meet full criteria for DSM depressive disorders, and continue to experience symptoms over time-despite receiving antidepressants and having intermittent contact with MHPs-limited QI programs may not be sufficient. QI programs that simply assure the provision of antidepressants or encourage referral to mental health professionals, while undoubtedly helpful for some populations of patients, are unlikely to improve the outcomes of this target group. More intensive QI programs may be required, such as programs that monitor clinical status or offer intensive alternatives, such as a disease management or stepped care.

Limitations

Several limitations should be considered when interpreting our findings. First, our study was conducted in only 2 sites. Physicians in other settings may be more or less likely to note depressive diagnoses in administrative data. Many settings will not have MHPs available on a consultative basis; having this availability may affect the frequency of depression diagnosis or treatments prescribed. A PCP’s willingness to make an encounter diagnosis of depression is likely influenced by a number of factors, including the characteristics of the clinic population, clinic norms, team composition, and reimbursement policies of third-party payers.

Only 25% of the patients identified as depressed in administrative data were enrolled in our study, and the possibility of systematic selection must be carefully considered. Biases may have been introduced by many factors: variation in PCP willingness to have patients recruited into the study; which patients were approved for recruitment; who could be contacted by telephone; who agreed to participate; and the shorter recruitment route for patients seen by the psychiatrist. Most potential enrollees were lost simply because clinicians failed to respond to written requests for permission to contact their patients. Yet, more than 60% of patients were retained at each step in the enrollment process, and no systematic differences were found between enrolled and nonenrolled patients in sex, clinic site, or encounter diagnosis. Although there was considerable variation in the time to follow-up assessment, with mailed assessments being returned between 5 and 12 months, time frames of 6 to 12 months are commonly used in QI studies of depression outcomes.

Since QI programs will often need to secure PCP permission and patient cooperation if they are to monitor patients’ clinical status, they are likely to face many of the same selection pressures that potentially affected our results. They are also likely to experience similar variation in the timing of patients’ completion and return of any mailed assessments. The enrollment rate in our study is similar to that of many QI projects in primary care. Thus, our data remain germane to QI programs that are considering targeting this population.

Conclusions

Health care organizations using administrative data to identify primary care patients with depression for QI activities are likely to select a cohort that is significantly ill, has had previous trials of psychotherapy, and is currently receiving antidepressants. Most patients will improve, yet many will continue to have significant symptoms. For QI efforts to have a significant impact on this population, they may need to offer intensive alternatives, such as disease management programs, intensive monitoring of patient adherence, or stepped care, rather than simply monitoring the provision of standard treatments.

Acknowledgments

Funding for this research was provided by the Office of Clinical Affairs, University of Michigan, and the Department of Veterans Affairs, HSR&D Research Career Development Award, RC-98350-1. We would like to thank the primary care providers in the BWO and CFC clinics for their assistance in this study. They would also like to thank Sharon Blanchard for data support. The study protocol was approved by the University of Michigan Institutional Review Board and by research committees at the study clinics.