User login
From the Milwaukee County Behavioral Health Division, Milwaukee, WI.
Abstract
Objective: Implementation of the Quadruple Aim of health care must begin with a clearly articulated set of concepts, or core domains (CDs), that comprise each aim. These CDs can then be operationalized with existing or new measures. If aligned to the organization’s mission and strategic goals, these CDs have the potential to focus quality improvement activities and reduce measurement burden. This article represents the efforts of a publicly funded behavioral health system to operationalize the Quadruple Aim through the development of CDs.
Methods: Various stakeholders across the organization were consulted on their perceptions of the Quadruple Aim and the CDs they believed should support it. Then, a review of existing literature on core metrics for health care and population health was completed, summarized, and integrated with the stakeholder feedback.
Results: These efforts led to the development and adoption of 15 CDs, with an accompanying literature review and set of recommendations of new and existing measures for each domain.
Conclusions: It is possible to create a comprehensive yet economical set of CDs and attendant measures that can be implemented in a staged, scalable, enterprise manner. It is hoped that the process articulated here, and the accompanying literature review, may be of some benefit to other public or government-run health systems in their own quality improvement journey to operationalize the Quadruple Aim by developing a set of CDs.
Keywords: quality measures; quality improvement; adult behavioral health.
First articulated in 2008, the Triple Aim proposes that health care systems should simultaneously seek to improve the patient’s experience of care, improve the health of populations, and reduce the per capita costs of care for populations.1 More recently, some have argued that health care provider burnout can deleteriously impact the attainment of the Triple Aim and have therefore advocated for an expanded focus to include a fourth Aim, the work life quality of the staff.2 Milwaukee County Behavioral Health Division (BHD), a publicly funded, county-based behavioral health care system in Milwaukee, Wisconsin, recently adopted the Quadruple Aim as the framework by which it will organize its quality activities.
Although originally developed for medical organizations, BHD believes that the Quadruple Aim has strong applicability to county-level behavioral health services. Many county-based behavioral health divisions provide a variety of programs to large segments of the county based on financial eligibility and/or clinical need, and thus often have responsibilities to populations or subpopulations, rather than programs. County health divisions, such as Milwaukee County’s Department of Health and Human Services, are often asked to improve outcomes and client experience of care with neutral growth budgets and less reliance on taxes to fund programs, while simultaneously attracting and retaining competent staff.
Crucial to the effective implementation of the Quadruple Aim, however, is a clear set of population- level measures that help organizations assess their progress.3 Unfortunately, as some authors have noted, evaluation of the Quadruple Aim remains a challenge because the “concepts of (population) health, quality of care and costs are not unanimously defined and measures for these concepts are under construction.”4 Several authors have provided some guidance to assist in the development of a set of measures that effectively capture the elements of the Quadruple Aim.5,6 However, the recent rapid proliferation of quality measures in health care7,8 has been both burdensome and costly for providers.9,10 Any measures adopted should not only be as meaningful as possible with regards to assessing progress towards the basic aims of health care, but should also be parsimonious, to limit measurement burden for providers (and patients) and focus attention on important issues.11,12
To select the most effective, parsimonious set of measures possible, one must first select a set of key foci from among the many possible areas of focus that the core measure is intended to represent. The core domains (CDs), if appropriately consistent with the strategic goals of the organization, provide a mechanism to orient the efforts of the organization at every level and help every staff member of the organization understand how his or her work impacts the progress towards these goals.11 The CDs, therefore, represent the opportunity to affect a greater integration of efforts across the organization toward these shared aims, creating uniformity of purpose at every level. Further, increasing organizational attention on the CDs can also help to reduce measurement burden by streamlining and focusing the data capture processes on the most valuable elements of quality and health, and discarding other extraneous measures (albeit not at the expense of other reporting requirements).11 The remainder of this article describes the CDs selected by BHD to assess its progress toward implementation of the Quadruple Aim and are organized by the Aim which they best represent.
Methods
To effectively implement the Quadruple Aim at BHD, it was necessary to clearly define the subpopulation of focus for our efforts.6 In this case, the subpopulation of interest was defined as all adult clients (18 years and older) who received at least 1 service encounter within a specified time frame from a program that BHD either operated or contracted with to provide care. Services provided by the BHD network include everything from psychiatric inpatient services to mental health and addiction treatment and care management. A limited array of social services, including housing and employment services, is also available to eligible consumers. BHD is the county-run behavioral health provider for individuals who are uninsured or underinsured in Milwaukee County, a demographically diverse, primarily urban county of approximately 950,000 people located in Wisconsin. Approximately 15,000 adults receive services at BHD each year.
This work began by obtaining executive sponsorship for the project, in this case from the Chief Operations Officer and Executive Medical Director of BHD. With their backing, an initial review of the literature produced a preliminary set of possible domains, for which we created working definitions. We then made a list of key stakeholders throughout BHD to whom we needed to present the idea of the Quadruple Aim, and the CDs under each Aim, to secure their support. These stakeholders, which included individuals involved in quality activities, program managers, and executive leadership, were strategically selected based on their relative influence within the organization. A set of brief presentations and handouts explaining the project were then developed and shared at different focus groups with these stakeholders over the course of 6 months. These focus groups served to not only educate the organization about the Quadruple Aim and the CDs but afforded participants an opportunity to provide feedback as well.
During the focus groups, we asked participants which domains they believed were most important (were “core”) when operationalizing the Quadruple Aim. The focus groups provided feedback on the domain definitions, feedback that was used to develop uniform, mutually agreed upon definitions for the CDs that were generalizable to all departments at BHD, regardless of the focus of their services within the continuum of care or the continuum of age. This was a crucial step, as it will eventually enable BHD to aggregate data across departments, even if there are minor discrepancies in the specific items they use to assess the CDs. Comments from the focus groups ultimately resulted in a truncated list of domains and definitions, which, coupled with the literature review, resulted in our final set of CDs.
During our review of the literature, we also looked for items that we felt could best represent each CD in the briefest, most meaningful way. (These items were not meant to supersede existing data, but to provide examples that could be implemented with existing data or recommendations that could be utilized in the absence of existing data.) During this process, we made every effort to make use of existing data-reporting requirements. For example, if we had a state mandate to collect data on housing status, we attempted to leverage this required data point to represent the CD related to housing. In other cases, we attempted to utilize claims or other administrative data to operationalize the CD, such as in the cost-of-care metric articulated in the section the Third Aim. For CDs for which no data existed or were insufficient, we emphasized the use of single- versus multi-item scales. For example, if we found a single-item global assessment of quality of life that had good psychometric properties relative to its longer parent scale, we selected the single item. This approach to item selection allowed us to create the most efficient, parsimonious set of measures possible, which we believed would enable us to comprehensively assess all the CDs with the least amount of burden to staff and clients. These items were presented at stakeholder focus groups, during which we asked for comments on the existing measures in their program or department and gave them the opportunity to comment on the new recommended measures.
A working definition is provided for each CD, followed by a brief review of the research base supporting its inclusion in the final list. The item(s) selected by BHD to represent each CD and the source of the item(s) are then supplied. These items were based either on measures currently collected because of existing reporting mandates or, in the case where extant measures were not available, on new items that demonstrated acceptable psychometric properties in the research literature. The CDs and items are organized by the Aim they best represent. A full list of the CDs by Quadruple Aim and items by CD is provided in the Appendix of the online version of this article. This article concludes with a brief summary of this effort and a discussion of how staff will utilize these items at different levels throughout the BHD system.
The First Aim: Population Health
Health Outcomes
Deaths. This can be defined as the cause of death, as determined by the medical examiner’s office (where appropriate) or as the age at time of death. This CD can also be reported as proportion of deaths considered premature (eg, before age 75) or calculated as total years of potential life lost.
Brief review and suggested item(s). Rates and causes of premature mortality are critical foci for the County Health Rankings & Roadmaps,13 the Institute for Healthcare Improvement’s “Guide to Measuring the Triple Aim,”6 the Centers for Disease Control and Prevention’s “Community Health Assessment for Population Health Improvement,”14 and the Institute of Medicine’s (IOM) “Vital Signs: Core Metrics for Health and Health Care Progress.”11 There is ample evidence that individuals with serious mental illness are at increased risk of early mortality relative to the general population,15-18 and this risk applies to those with substance use disorders as well.15,19-20 BHD tracks all deaths that occur while patients are receiving BHD-funded, community-based services.
Self-Reported Health and Well-Being. This CD asks patients to rate their current physical and mental health status, as well as their overall quality of life.
Brief review and suggested item(s): Self-rated physical health. Premature mortality among individuals with behavioral health issues appears to be due, in large part, to their increased vulnerability to the development of medical comorbidities.16,21 A single self-rating question has demonstrated considerable sensitivity to premature mortality,22,23 with predictive properties up to a decade prior to death.24,25 Further, self-rated health has been associated with subsequent functional decline,26,27 acute service utilization,28,29 and overall health care costs.28
Brief review and suggested item(s): Self-rated mental health. Mental health disorders are associated with significant disability worldwide,30 and comorbid mental health issues can exacerbate the course of other medical problems. For example, depression is associated with increased rates of mortality among individuals with diabetes and31 cardiovascular disease,32 as well as with rates of overall mortality,33 and psychiatric comorbidity is associated with longer lengths of stay and higher costs among patients hospitalized for medical problems.34 Research has found that a single-item measure of self-rated mental health is associated with the presence of psychiatric diagnoses, psychiatric symptoms, and subsequent depression and serious mental illness up to 1 year post-assessment.35,36 There is even evidence that self-rated mental health may be more strongly associated with self-ratings of overall health than self-ratings of physical health.37
Brief review and suggested item(s): Self-rated quality of life. Quality of life is a critical component of the recovery journey and overall health.38 For example, the County Health Rankings & Roadmaps lists “quality of life” as 1 of its key “health outcomes” in its County Health Rankings.13 As some authors have noted, quality of life is often inferred from other “objective” recovery domains, such as employment, health status, or housing status. However, there is evidence that these objective domains are functionally distinct from the inherently subjective construct of quality of life.39 This has led other authors to conclude that these domains should be assessed separately when evaluating outcomes.40 Single-item quality of life assessments have been used in research with individuals with cancer,41 adults with disabilities,42 patients with cystic fibrosis,43 and children with epilepsy.44 For this effort, BHD selected the first global quality of life item from the World Health Organization’s WHOQOL-BREF quality of life assessment,45 an item used in other quality of life research.46
Health Factors
Substance Use. This CD is a composite of 4 different types of substance use, any recent heavy alcohol use (defined as 5 or more drinks in one sitting), any recent drug use, any recent prescription drug abuse, and any recent tobacco use.
Brief review and suggested item(s). As noted, substance use disorders confer an increased risk for early mortality15,19 and are significantly implicated in disease disability burden worldwide.30 Substance use has also been associated with both the onset47,48 and exacerbation of mental health diagnoses.49-51 Further, substance use appears to heighten the risk of violence in the general population52 and especially among those with a co-occurring mental illness.53,54 The County Health Rankings & Roadmaps list alcohol and drug use as key behaviors to address to improve the overall health of a given county,13 and the Centers for Medicare & Medicaid Services (CMS) has endorsed initiation and engagement in addiction treatment as one of the measures in its Adult Core Set.55
Tobacco use continues to be one of the most significant risk factors for early mortality worldwide, and evidence indicates that it is associated with a lower life expectancy of nearly 10 years.56 Unfortunately, rates of tobacco use are even higher among those with severe mental illness relative to the general population, and their rates of smoking cessation are lower.57,58 Tobacco use is a significant risk factor for the high rates of early mortality in individuals with severe mental illness.18 Further, a recent meta-analysis noted that, relative to those who continued to smoke, those who ceased smoking had reduced rates of psychological distress and increased quality of life rankings.59 Reducing tobacco use is one of the key components of the County Health Rankings & Roadmaps, and medication assistance with smoking and tobacco use cessation is also listed in the CMS Adult Core Set.13,55
An accumulating body of evidence suggests that single-item measures can adequately detect alcohol60-62 and drug use disorders.60-64 McNeely and colleagues recently developed and tested a brief 4-item screen, the Tobacco, Alcohol, Prescription medication, and other Substance use (TAPS) tool.65,66 Preliminary evidence suggests that the TAPS tool can effectively identify the presence of problematic and disordered use of tobacco, alcohol, prescription medications, and other drugs.65-67 BHD will use the 4 items from the TAPS tool to represent its substance use CD.
Education/Employment Status. This CD assesses the proportion of BHD members who have completed high school, who are in some type of educational or training program, or who are engaged in some type of employment activity (defined as full-time, part-time, supported, sheltered workshop, or as a full-time homemaker).
Brief review and suggested item(s). Research indicates that unemployment is a risk factor for mortality, even after controlling for other risk factors (eg, age, sex, socioeconomic status [SES], health).68 Unemployment is associated with poorer physical and mental health in the general population and among those with disabilities.69-71 Promisingly, evidence suggests that gaining employment or re-employment is associated with better health,72 even for individuals with substance use disorders73 or moderate74 to severe mental health disorders.75-78 Some authors have even proposed that, above and beyond the associated health benefits, employment may also help to realize a modest cost savings due to reduced service utilization and disability.79,80 Employment is a core tenet in the Substance Abuse and Mental Health Services Administration’s (SAMHSA’s) model of recovery,81 and is also listed as an important recovery goal for individuals with behavioral health issues.82 BHD collects data on employment status on all the patients it serves as part of its state-mandated reporting requirements and will use this item in the CD data set.83
Living Situation. This is measured as the proportion of people who live in permanent, supportive, stable housing; it may also be measured as the percentage of the population living with severe housing problems or who are homeless.
Brief review and suggested item(s). Housing problems can be conceptualized as 3 inter-related components: conditions within the home, neighborhood conditions, and housing affordability, each of which can contribute uniquely to poorer physical and mental health of individuals and families84 and to educational outcomes for children.85,86 Further, individuals who are homeless have a standardized mortality ratio 2 to 5 times that of the general population,87-89 even after controlling for low income status,90 and some evidence suggests these rates are even higher among unsheltered versus sheltered homeless individuals.91 Interventions to improve the condition of housing have demonstrated positive impacts on both physical and mental health,92 and a recent study found that individuals receiving housing assistance in the form of public housing or multifamily housing from the Department of Housing and Urban Development had better self-rated physical and mental health relative to individuals on the wait list for housing assistance.93 Moreover, the provision of housing has been shown to promote reductions in substance use and health service utilization among homeless individuals with substance use disorders.94 Rog and colleagues reviewed the literature on permanent supportive housing for individuals with substance use or mental health disorders who were homeless or disabled, and found that provision of housing led to reduced rates of homelessness, emergency department (ED) and inpatient utilization and increased consumer satisfaction.95
Importantly, evidence suggests that housing is viewed as facilitative of recovery. For example, in a recent qualitative study of homeless individuals with mental illness, housing was seen as a critical first step in recovery, providing a sense of security, increasing feelings of personal independence and autonomy, improving perceptions of health and well-being, and affording a stable environment to rebuild relationships with important others.96 BHD collects data on housing status on all the patients it serves as part of its state-mandated reporting requirements and will utilize this item in the CD data set.83
Social Relationships. This is defined as recent interactions with family, supportive networks (formal and informal), and other recovery services.
Brief review and suggested item(s). Research has long established that social relationships have a significant impact on health, including rates of mortality as well as physical and mental health morbidity.97-99 Social connectedness is another of the pillars supporting an individual’s recovery in SAMHSA’s formulation. Several reviews of the recovery literature38,82 support its importance to the recovery process and inclusion in any assessment of holistic recovery. Social support has been shown to promote recovery among individuals with severe mental illness100-102 and substance use disorders,103 and may mitigate the progression of chronic, life-threatening physical illnesses.97 For the purposes of BHD’s CD data set, the social support question from the “100 Million Healthier Lives Common Questionnaire for Adults” will be used to assess individuals’ perceived adequacy of social support.104
Legal Involvement. Defined as involvement with the civil or criminal justice system, including arrests, imprisonment, or detainment.
Brief review and suggested item(s). Involvement in the criminal justice system is both disruptive for the individual in recovery and expensive to the larger health care system.105 Individuals with substance use106 and severe mental health disorders107 are over-represented in the prison system, and evidence suggests that general physical and mental health declines while individuals are in prison.108,109 Perhaps even more concerning, numerous studies have demonstrated an increase in mortality rates for individuals recently released from prison relative to the general population, particularly during the period immediately following release.108-110 This relationship may even persist long term.111 Further, research indicates that individuals recently released from prison have increased emergency care and hospital utilization.112,113
Incarceration can have significant impacts on the health of the broader community as well. For example, research has found an association between parental incarceration to rates of infant mortality,114 increased behavioral and developmental problems of children of incarcerated parents,115,116 lower rates of child support payments,117 and poorer cardiovascular health of female partners of incarcerated individuals.118 Formerly incarcerated individuals experience slower wage growth as well.119 However, evidence also indicates that engagement in mental health120 and substance abuse121 treatment can reduce the likelihood of subsequent recidivism. As part of its state-mandated reporting, BHD is required to provide information on the criminal justice system involvement of its clients in the previous 6 months, including whether they have been jailed or imprisoned,83 and this will function as its measure of legal involvement in its CD data set.
Socioeconomic Status. Socioeconomic status is the social standing or class of an individual or group. It is often measured as a combination of education, income, and occupation. It can also be defined subjectively, such as one’s evaluation of status relative to similar others or based on an individual’s interpretation of her or his financial needs.
Brief review and suggested item(s). A large body of evidence supports the existence of a robust relationship between lower SES and poor health, including mortality and chronic medical diseases,122-124 as well as mental illness.125-127 Although previous research has examined this relationship using objective indicators of SES (eg, income, education level, occupation), there has recently been an increased interest in exploring the relationship of subjective SES with health indices. Subjective SES is generally assessed by asking individuals to rate themselves relative to others in the society in which they live, in terms of wealth, occupation, educational level, or other indicators of social status. Evidence suggests that subjective SES is associated with objective measures of SES,128-130 and relates to measures of physical and mental health as well, even after controlling for objective SES.130-135 BHD will be using a modified version of the Subject SES Scale,131,135 which is deployed in the “100 Million Healthier Lives Common Questionnaire for Adults.”104
Acute Service Use. This is defined as an admission to a medical or psychiatric emergency room or to a medical or psychiatric hospital or to a detoxification facility.
Brief review and suggested item(s). The CMS Adult Core Set includes “plan all cause readmissions” as a key quality metric.55 Hospital readmissions are also endorsed by the National Committee on Quality Assurance as one of its Health Effectiveness Data and Information Set (HEDIS) measures and by the National Quality Forum. Readmissions, despite their widespread endorsement, are a somewhat controversial measure. Although readmissions are costly to the health care system,136 the relationship between readmissions and quality is inconsistent. For example, Krumholz and colleagues137 found differential rates of readmission for the same patient discharged from 2 different hospitals, which were categorized based on previous readmission rates, suggesting that hospitals do have different levels of performance even when treating the same patient. However, other data indicate that 30-day, all-cause, risk-standardized readmission rates are not associated with hospital 30-day, all-cause, risk-standardized mortality rates.138
Chin and colleague found that readmissions to the hospital that occurred more than 7 days post-discharge were likely due to community- and household-related factors, rather than hospital-related quality factors.139 Transitional care interventions that have successfully reduced 30-day readmission rates are most often multicomponent and focus not just on hospital-based interventions (eg, discharge planning, education) but on follow-up care in the community by formal supports (eg, in-home visits, telephone calls, outpatient clinic appointments, case management) and informal supports (eg, family and friends).140-143 Further, qualitative evidence suggests that some individuals perceive psychiatric hospitalizations to be the result of insufficient resources or unsuccessful attempts to maintain their stability in the community.144 Thus, unplanned or avoidable hospital readmissions may represent a failure of the continuum of care not only from the perspective of the health care system, but from the patient perspective as well.
Frequent or nonurgent use of EDs is conceptually similar to excessive or avoidable inpatient utilization in several ways. For example, overuse of EDs is costly, with some estimates suggesting that it is responsible for up to $38 billion in wasteful spending each year.145 Individuals with frequent ED visits have a greater disease burden146 and an increased risk of mortality compared to nonfrequent users.147 Research suggests that individuals who visit the ED for non-urgent issues do so because of perceived difficulties associated with accessing primary care, and the convenience of EDs relative to primary care.148-150 Moreover, similar to the hospital readmission literature discussed earlier, successful strategies to reduce high rates of ED utilization generally focus on continuum of care interventions, such as provision of case management services.151-155
This evidence implies that frequent ED utilization and hospital readmissions may not be a fundamental issue of quality (or lack thereof) in hospitals or EDs but rather a lack of, or ineffectual, transitional and continuum of care strategies and services. To underscore this point, some authors have argued that a system that is excessively crisis-oriented hinders recovery because it is reactive rather than proactive, predicated on the notion that one’s condition must deteriorate to receive care.156
Although some organizations may have access to claims data or may function as self-contained health systems (eg, the Veterans Health Administration [VHA] ), others may not have access to such data. In the absence of claims data, patient self-report of service utilization has been used as a proxy for actual agency records.157 Although concordance between medical and/or agency records and patient self-report has been variable,157 evidence generally suggests that rates of agreement are higher the shorter the recall time interval.158,159 BHD does not have access to comprehensive claims data and has therefore chosen to use 5 dichotomously scored (yes/no) questions—related to medical inpatient, medical ED, psychiatric inpatient, psychiatric ED, and detoxification use in the last 30 days—to represent the CD of acute service utilization.
The Second Aim: Quality of Care
Safety
Safety is defined as avoiding injuries to patients from the care that is intended to help them.
Brief review and suggested item(s). As noted in “Crossing the Quality Chasm,” the IOM’s seminal document, “the health care environment should be safe for all patients, in all of its processes, all the time.”160 The landmark Harvard Medical Practice Study in 1991 found that adverse events occurred in nearly 4% of all hospital admissions and, among these, over a quarter were due to negligence.161 Other estimates of adverse events range as high as 17%.162 Indeed, a recent article by Makary and Daniel estimated that medical errors may be the third leading cause of death in the United States.163 Unfortunately, research on safety in the mental health field has lagged behind that of physical health,164 with evidence indicating that research in nonhospital settings in mental health care may be particularly scarce.165 In a study of adverse events that occurred in psychiatric inpatient units in the VHA system between 2015 and 2016, Mills and colleagues found that of the 87 root cause analysis reports, suicide attempts were the most frequent, and, among safety events, falls were the most frequently reported, followed by medication events.166 Another report on data collected from psychiatric inpatient units in the VHA revealed that nearly one-fifth of patients experienced a safety event, over half of which were deemed preventable.167 These numbers likely represent an underestimation of the true volume of safety events, as another study by the same research group found that less than 40% of safety events described in patient medical records were documented in the incident reporting system.168 BHD will utilize the total number of complaints and incident reports submitted within a given time frame as its “safety” metric in the CD data set.
Wait Time for Service
The CD is defined as the length of time between the date a patient first contacted BHD for services and the date of their first clinical service.
Brief review and suggested item(s). “Timeliness” was listed among the 6 aims for improvement in “Crossing the Quality Chasm” in 2001, and it remains no less relevant today.160 For example, evidence indicates that access to primary care is inversely related to avoidable hospitalizations.169 One study found that, of patients hospitalized for cardiovascular problems, those who had difficulty accessing routine care post discharge had higher 30-day readmission rates.170 Among VHA patients, longer wait times are associated with more avoidable hospitalizations and higher rates of mortality.171 Longer wait times appear to decrease the likelihood of attending a first appointment for individuals with substance use172,173 and mental health disorders.174 Importantly, longer wait times are associated with lower ratings of the patient experience of care, including perceptions of the quality of and satisfaction with care,175 and may be associated with worse outcomes for individuals in early intervention for psychosis treatment.176 For the purposes of the CD data set, BHD will monitor the length of time between the date a patient first contacted BHD for services and the date of their first clinical service.
Patient Satisfaction
Patient satisfaction is defined as the degree of patients’ satisfaction with the care they have received.
Brief review and suggested item(s). Research has consistently demonstrated the relationship of the patient’s experience of care to a variety of safety and clinical effectiveness measures in medical health care,177 and the therapeutic alliance is one of the most consistent predictors of outcomes in behavioral health, regardless of therapeutic modality.178 Patient satisfaction is a commonly assessed aspect of the patient experience of care. Patient satisfaction scores have been correlated with patient adherence to recommended treatment regimens, care quality, and health outcomes.179 For example, Aiken et al found that patient satisfaction with hospital care was associated with higher ratings of the quality and safety of nursing care in these hospitals.180 Increased satisfaction with inpatient care has been associated with lower 30-day readmission rates for patients with acute myocardial infarction, heart failure, and pneumonia,181 and patients with schizophrenia who reported higher treatment satisfaction also reported better quality of life.182,183 Many satisfaction survey options exist to evaluate this CD, including the Consumer Assessment of Healthcare Providers and Systems and the Client Satisfaction Questionnaire; BHD will utilize an outpatient behavioral health survey from a third-party vendor.
The Third Aim: Cost of Care
Cost of Care
This can be defined as the average cost to provide care per patient per month.
Brief review and suggested item(s). Per capita cost, or rather, the total cost of providing care to a circumscribed population divided by the total population, has been espoused as an important metric for the Triple Aim and the County Health Rankings.6,13 Indeed, between 1960 and 2016, per capita expenditures for health care have grown 70-fold, and the percent of the national gross domestic product accounted for by health expenditures has more than tripled (5.0% to 17.9%).184 One of the more common metrics deployed for assessing health care cost is the per capita per month cost, or rather, the per member per month cost of the predefined population for a given health care system.6,185,186 In fact, some authors have proposed that cost of care can be used not only to track efficient resource allocation, but can also be a proxy for a healthier population as well (ie, as health improves, individuals use fewer and less-expensive services, thus costing the system less).187 To assess this metric, BHD will calculate the total amount billed for patient care provided within BHD’s health network each month (irrespective of funding source) and then divide this sum by the number of members served each month. Although this measure does not account for care received at other health care facilities outside BHD’s provider network, nor does it include all the overhead costs associated with the care provided by BHD itself, it is consistent with the claims-based approach used or recommended by other authors.6,188
The Fourth Aim: Staff Well-being
Staff Quality of Work Life
This can be defined as the quality of the work life of health care clinicians and staff.
Brief review and suggested item(s). Some authors have suggested that the Triple Aim framework is incomplete and have proffered compelling arguments that provider well-being and the quality of work life constitutes a fourth aim.2 Provider burnout is prevalent in both medical2,189 and behavioral health care.190,191 Burnout among health care professionals has been associated with higher rates of perceived medical errors,192 lower patient satisfaction scores,189,193 lower rates of provider empathy,194 more negative attitudes towards patients,195 and poorer staff mental and physical health.191
Burnout is also associated with higher rates of absenteeism, turnover intentions, and turnover.190,191,196,197 However, burnout is not the only predictor of staff turnover; for example, turnover rates are a useful proxy for staff quality of work life for several reasons.198 First, turnover is associated with substantial direct and indirect costs, including lost productivity, increased errors, and lost revenue and recruitment costs, with some turnover cost estimates as high as $17 billion for physicians and $14 billion for nurses annually.199-201 Second, research indicates that staff turnover can have a deleterious impact on implementation of evidence-based interventions.202-205 Finally, consistent with the philosophy of utilizing existing data sources for the CD measures, turnover can be relatively easily extracted from administrative data for operated or contracted programs, and its collection does not place any additional burden on staff. As a large behavioral health system that is both a provider and payer of care, BHD will therefore examine the turnover rates of its internal administrative and clinical staff as well as the turnover of staff in its contracted provider network as its measures for the Staff Quality of Work Life CD.
Clinical Implications
These metrics can be deployed at any level of the organization. Clinicians may use 1 or more of the measures to track the recovery of individual clients, or in aggregate for their entire caseload. Similarly, managers can use these measures to assess the overall effectiveness of the programs for which they are responsible. Executive leaders can evaluate the impact of several programs or the system of care on the health of a subpopulation of clients with a specific condition, or for all their enrolled members. Further, not all measures need be utilized for every dashboard or evaluative effort. The benefit of a comprehensive set of measures lies in their flexibility—1 or more of the measures may be selected depending on the project being implemented or the interests of the stakeholder.
It is important to note that many of the CDs (and their accompanying measures) are aligned to/consistent with social determinants of health.206,207 Evidence suggests that social determinants make substantial contributions to the overall health of individuals and populations and may even account for a greater proportion of variance in health outcomes than health care itself.208 The measures articulated here, therefore, can be used to assess whether and how effectively care provision has addressed these social determinants, as well as the relative impact their resolution may have on other health outcomes (eg, mortality, self-rated health).
These measures can also be used to stratify clients by clinical severity or degree of socioeconomic deprivation. The ability to adjust for risk has many applications in health care, particularly when organizations are attempting to implement value-based purchasing models, such as pay-for-performance contracts or other alternative payment models (population health-based payment models).209 Indeed, once fully implemented, the CDs and measures will enable BHD to more effectively build and execute different conceptual models of “value” (see references 210 and 211 for examples). We will be able to assess the progress our clients have made in care, the cost associated with that degree of improvement, the experience of those clients receiving that care, and the clinical and social variables that may influence the relative degree of improvement (or lack thereof). Thus, the CDs provide a conceptual and data-driven foundation for the Quadruple Aim and any quality initiatives that either catalyze or augment its implementation.
Conclusion
This article provides an overview of the CDs selected by BHD to help organize, focus, advance, and track its quality efforts within the framework of the Quadruple Aim. Although items aligned to each of these CDs are offered, the CDs themselves have been broadly conceptualized such that they can flexibly admit a variety of possible items and/or assessments to operationalize each CD and thus have potential applicability to other behavioral health systems, particularly public systems that have state-mandated and other data reporting requirements.
Bearing in mind the burden that growing data collection requirements can have on the provision of quality care and staff work satisfaction and burnout,10,212 the CDs (and the items selected to represent each) are designed with “strategic parsimony” in mind. Although the CDs are inclusive in that they cover care quality, cost of care, staff quality of life, and general population health, only CDs and items undergirded by a solid evidence base and high value with regards to BHD’s mission and values, as determined by key stakeholders, were selected. Moreover, BHD attempted to make use of existing data collection and reporting mandates when selecting the final pool of items to reduce the measurement burden on staff and clients. Thus, the final set of CDs and items are designed to be comprehensive yet economical.
The CDs are deeply interrelated. Although each CD may be individually viewed as a valuable metric, improvements in any 1 CD will impact the others (eg, increasing care quality should impact population health, increasing staff quality of life should impact the quality of care). Moreover, this idea of interrelatedness acknowledges the need to view health systems and the populations they serve holistically, in that improvement is not simply the degree of change in any given metric (whether individually or collectively), but rather something more entirely. The concepts of value, quality, and health are complex, multidimensional, and dynamic, and the CDs that comprise these concepts should not be considered independently from one another. The CDs (and items) offered in this article are scalable in that they can be used at different levels of an organization depending on the question or stakeholder, and can be used individually or in combination with one another. Moreover, they are adaptable to a variety of risk-adjusted program, population health, and value-based evaluation models. It is hoped that the process articulated here, and the accompanying literature review, may benefit other public or government-run health systems in their own quality journey to operationalize the Quadruple Aim by developing a set of CDs.
Corresponding author: Walter Matthew Drymalski, PhD; walter.drymalski@milwaukeecountywi.gov.
Financial disclosures: None.
1. Berwick DM, Nolan TW, Whittington J. The Triple Aim: Care, health, and cost. Health Aff (Millwood). 2008;27(3):759-769.
2. Bodenheimer T, Sinsky C. From Triple to Quadruple Aim: Care of the patient requires care of the provider. Ann Fam Med. 2014;12(6):573-576.
3. Whittington JW, Nolan K, Lewis N, Torres T. Pursuing the Triple Aim: The first 7 years. Milbank Q. 2015;93(2):263-300.
4. Hendrikx RJP, Drewes HW, Spreeuwenberg M, et al. Which Triple Aim related measures are being used to evaluate population management initiatives? An international comparative analysis. Health Policy. 2016;120(5):471-485.
5. Kassler WJ, Howerton M, Thompson A, et al. Population Health Measurement at Centers for Medicare & Medicaid Services: Bridging the gap between public health and clinical quality. Popul Health Manag. 2017;20(3):173-180.
6. Stiefel MC, Nolan K. A Guide to Measuring the Triple Aim: Population health, experience of care, and per capita cost. IHI Innovation Series white paper. Institute for Healthcare Improvement; 2012.
7. Panzer RJ, Gitomer RS, Greene WH, et al. Increasing demands for quality measurement. JAMA. 2013;310(18):1971-1980.
8. Schuster MA, Onorato SE, Meltzer DO. Measuring the cost of quality measurement: a missing link in quality strategy. JAMA. 2017;318(13):1219-1220.
9. Casalino LP, Gans D, Weber R, et al. US physician practices spend more than $15.4 billion annually to report quality measures. Health Aff (Millwood). 2016;35(3):401-406.
10. Rao SK, Kimball AB, Lehrhoff SR, et al. The impact of administrative burden on academic physicians: results of a hospital-wide physician survey. Acad Med. 2017;92(2):237-243.
11. Institute of Medicine. Vital signs: Core metrics for health and health care progress. National Academies Press; 2015.
12. Meyer GS, Nelson EC, Pryor DB, et al. More quality measures versus measuring what matters: a call for balance and parsimony: Table 1. BMJ Qual Saf. 2012;21(11):964-968.
13. County Health Rankings. Measures & data sources. County Health Rankings & Roadmaps. Accessed January 11, 2021. https://www.countyhealthrankings.org/explore-health-rankings/measures-data-sources
14. U.S. Centers for Disease Control and Prevention. Community Health Assessment for population health improvement: Resource of most frequently recommended health outcomes and determinants. Office of Surveillance, Epidemiology, and Laboratory Services; 2013.
15. Chang C-K, Hayes RD, Perera G, et al. Life expectancy at birth for people with serious mental illness and other major disorders from a secondary mental health care case register in London. PLoS One. 2011;6(5):e19590.
16. De Hert M, Correll CU, Bobes J, et al. Physical illness in patients with severe mental disorders. I. Prevalence, impact of medications and disparities in health care. World Psychiatry. 2011;10(1):52-77.
17. Druss BG, Zhao L, Von Esenwein S, Morrato EH, Marcus SC. Understanding excess mortality in persons with mental illness: 17-year follow up of a nationally representative US survey. Med Care. 2011;49(6):599-604.
18. National Association of State Mental Health Program Directors, (NASMHPD) Medical Directors Council. Morbidity and mortality in people with serious mental illness. National Association of State Mental Health Program Directors, (NASMHPD) Medical Directors Council; 2006.
19. Nordentoft M, Wahlbeck K, Hällgren J, et al. Excess mortality, causes of death and life expectancy in 270,770 patients with recent onset of mental disorders in Denmark, Finland and Sweden. PLoS One. 2013;8(1):e55176.
20. Griswold MG, Fullman N, Hawley C, et al. Alcohol use and burden for 195 countries and territories, 1990–2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet. 2018;392(10152):1015-1035.
21. Walker ER, Druss BG. A public health perspective on mental and medical comorbidity. JAMA. 2016;316(10):1104-1105.
22. DeSalvo KB, Bloser N, Reynolds K, et al. Mortality prediction with a single general self-rated health question: a meta-analysis. J Gen Intern Med. 2005;21(3):267-275.
23. Mavaddat N, Parker RA, Sanderson S, et al. Relationship of self-rated health with fatal and non-fatal outcomes in cardiovascular disease: a systematic review and meta-analysis. PLoS One. 2014;9(7):e103509.
24. Lima-Costa MF, Cesar CC, Chor D, Proietti FA. Self-rated health compared with objectively measured health status as a tool for mortality risk screening in older adults: 10-year follow-up of the Bambui Cohort Study of Aging. Am J Epidemiol. 2012;175(3):228-235.
25. Stenholm S, Pentti J, Kawachi I, et al. Self-rated health in the last 12 years of life compared to matched surviving controls: the Health and Retirement Study. PLoS One. 2014;9(9):e107879.
26. Lee Y. The predictive value of self-assessed general, physical, and mental health on functional decline and mortality in older adults. J Epidemiol Community Health. 2000;54(2):123-129.
27. Tomioka K, Kurumatani N, Hosoi H. Self-rated health predicts decline in instrumental activities of daily living among high-functioning community-dwelling older people. Age Ageing. 2017;46(2):265-270.
28. DeSalvo KB, Jones TM, Peabody J, et al. Health care expenditure prediction with a single item, self-rated health measure. Med Care. 2009;47(4):440-447.
29. Farkas J, Kosnik M, Flezar M, Suskovic S, Lainscak M. Self-rated health predicts acute exacerbations and hospitalizations in patients with COPD. Chest. 2010;138(2):323-330.
30. Whiteford HA, Degenhardt L, Rehm J, et al. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet. 2013;382(9904):1575-1586.
31. Park M, Katon WJ, Wolf FM. Depression and risk of mortality in individuals with diabetes: a meta-analysis and systematic review. Gen Hosp Psychiatry. 2013;35(3):217-225.
32. Hare DL, Toukhsati SR, Johansson P, Jaarsma T. Depression and cardiovascular disease: a clinical review. Eur Heart J. 2014;35(21):1365-1372.
33. Cuijpers P, Schoevers RA. Increased mortality in depressive disorders: a review. Curr Psychiatry Rep. 2004;6(6):430-437.
34. Jansen L, van Schijndel M, van Waarde J, van Busschbach J. Health-economic outcomes in hospital patients with medical-psychiatric comorbidity: a systematic review and meta-analysis. PLoS One. 2018;13(3):e0194029.
35. Ahmad F, Jhajj AK, Stewart DE, et al. Single item measures of self-rated mental health: a scoping review. BMC Health Serv Res. 2014;14:398.
36. McAlpine DD, McCreedy E, Alang S. The meaning and predictive value of self-rated mental health among persons with a mental health problem. J Health Soc Behav. 2018;59(2):200-214.
37. Levinson D, Kaplan G. What does self-rated mental health represent? J Public Health Res. 2014;3(3):287.
38. Leamy M, Bird V, Boutillier CL, et al. Conceptual framework for personal recovery in mental health: systematic review and narrative synthesis. Br J Psychiatry. 2011;199(6):445-452.
39. Smith KW, Avis NE, Assmann SF. Distinguishing between quality of life and health status in quality of life research: a meta-analysis. Qual Life Res. 1999;8(5):447-459.
40. Hamming JF, De Vries J. Measuring quality of life. Br J Surg. 2007;94(8):923-924.
41. Singh JA, Satele D, Pattabasavaiah S, et al. Normative data and clinically significant effect sizes for single-item numerical linear analogue self-assessment (LASA) scales. Health Qual Life Outcomes. 2014;12(1):187.
42. Siebens HC, Tsukerman D, Adkins RH, et al. Correlates of a single-item quality-of-life measure in people aging with disabilities. Am J Phys Med Rehabil. 2015;94(12):1065-1074.
43. Yohannes AM, Dodd M, Morris J, Webb K. Reliability and validity of a single item measure of quality of life scale for adult patients with cystic fibrosis. Health Qual Life Outcomes. 2011;9(1):105.
44. Conway L, Widjaja E, Smith ML. Single-item measure for assessing quality of life in children with drug-resistant epilepsy. Epilepsia Open. 2018;3(1):46-54.
45. Skevington SM, Lofty M, O’Connell KA. The World Health Organization’s WHOQOL-BREF quality of life assessment: psychometric properties and results of the international field trial. Qual Life Res. 2004;13:299-310.
46. Atroszko PA, Baginska P, Mokosinska M, et al. Validity and reliability of single-item self-report measures of general quality of life, general health and sleep quality. In: CER Comparative European Research Conference: Research Track. Vol 2. Sciemcee Publishing; 2015:207-211.
47. Beaulieu S, Saury S, Sareen J, et al.
48. Marconi A, Di Forti M, Lewis CM, et al. Meta-analysis of the association between the level of cannabis use and risk of psychosis. Schizophr Bull. 2016;42(5):1262-1269.
49. Baker AL, Hiles SA, Thornton LK, et al. A systematic review of psychological interventions for excessive alcohol consumption among people with psychotic disorders. Acta Psychiatr Scand. 2012;126(4):243-255.
50. Baker AL, Hides L, Lubman DI. Treatment of cannabis use among people with psychotic or depressive disorders: a systematic review. J Clin Psychiatry. 2010;71(3):247-254.
51. Berenz EC, Coffey SF. Treatment of co-occurring posttraumatic stress disorder and substance use disorders. Curr Psychiatry Rep. 2012;14(5):469-477.
52. Pickard H, Fazel S. Substance abuse as a risk factor for violence in mental illness: some implications for forensic psychiatric practice and clinical ethics. Curr Opin Psychiatry. 2013;26(4):349-354.
53. Fazel S, Gulati G, Linsell L, Geddes JR, Grann M. Schizophrenia and violence: systematic review and meta-analysis. PLoS Med. 2009;6(8):e1000120.
54. Van Dorn R, Volavka J, Johnson N. Mental disorder and violence: Is there a relationship beyond substance use? Soc Psychiatry Psychiatr Epidemiol. 2012;47(3):487-503.
55. Centers for Medicare & Medicaid Services. 2019 Core set of adult health care quality measures for Medicaid (adult core set). Adult health care quality measures. Accessed January 11, 2021. https://www.medicaid.gov/medicaid/quality-of-care/performance-measurement/adult-core-set/index.html.
56. Jha P, Peto R. Global effects of smoking, of quitting, and of taxing tobacco. N Engl J Med. 2014;370(1):60-68.
57. Lê Cook B, Wayne GF, Kafali EN, et al. Trends in smoking among adults with mental illness and association between mental health treatment and smoking cessation. JAMA. 2014;311(2):172-182.
58. Smith PH, Mazure CM, McKee SA. Smoking and mental illness in the US population. Tob Control. 2014;23(0):e147-e153.
59. Taylor G, McNeill A, Girling A, et al. Change in mental health after smoking cessation: systematic review and meta-analysis. BMJ. 2014;348:g1151.
60. McNeely J, Cleland CM, Strauss SM, et al. Validation of self-administered single-item screening questions (SISQs) for unhealthy alcohol and drug use in primary care patients. J Gen Intern Med. 2015;30(12):1757-1764.
61. Saitz R, Cheng DM, Allensworth-Davies D, et al. The ability of single screening questions for unhealthy alcohol and other drug use to identify substance dependence in primary care. J Stud Alcohol Drugs. 2014;75(1):153-157.
62. Smith PC, Schmidt SM, Allensworth-Davies D, Saitz R. Primary care validation of a single-question alcohol screening test. J Gen Intern Med. 2009;24(7):783-788.
63. Dawson DA, Compton WM, Grant BF. Frequency of 5+/4+ drinks as a screener for drug use and drug-use disorders. J Stud Alcohol Drugs. 2010;71(5):751-760.
64. Smith PC, Schmidt SM, Allensworth-Davies D, Saitz R. A single-question screening test for drug use in primary care. Arch Intern Med. 2010;170(13):1155-1160.
65. McNeely J, Strauss SM, Saitz R, et al. A brief patient self-administered substance use screening tool for primary care: two-site validation study of the Substance Use Brief Screen (SUBS). Am J Med. 2015;128(7):784.e9-784.e19.
66. McNeely J, Wu L-T, Subramaniam G, et al. Performance of the Tobacco, Alcohol, Prescription medication, and other Substance use (TAPS) tool for substance use screening in primary care patients. Ann Intern Med. 2016;165(10):690-699.
67. Gryczynski J, McNeely J, Wu L-T, et al. Validation of the TAPS-1: a four-item screening tool to identify unhealthy substance use in primary care. J Gen Intern Med. 2017;32(9):990-996.
68. Roelfs DJ, Shor E, Davidson KW, Schwartz JE. Losing life and livelihood: a systematic review and meta-analysis of unemployment and all-cause mortality. Soc Sci Med. 2011;72(6):840-854.
69. McKee-Ryan F, Song Z, Wanberg CR, Kinicki AJ. Psychological and physical well-being during unemployment: a meta-analytic study. J Appl Psychol. 2005;90(1):53-76.
70. Zhang S, Bhavsar V. Unemployment as a risk factor for mental illness: combining social and psychiatric literature. Adv Appl Sociol. 2013;03(02):131-136.
71. Goodman N. The impact of employment on the health status and health care costs of working-age people with disabilities. The Lead Center. November 2015. Accessed January 11, 2021. http://www.leadcenter.org/system/files/resource/downloadable_version/impact_of_employment_health_status_health_care_costs_0.pdf
72. Hergenrather KC, Zeglin RJ, McGuire-Kuletz M, Rhodes SD. Employment as a social determinant of health: a systematic review of longitudinal studies exploring the relationship between employment status and physical health. Rehabil Res Policy Educ. 2015;29(1):2-26.
73. Walton MT, Hall MT. The effects of employment interventions on addiction treatment outcomes: a review of the literature. J Soc Work Pract Addict. 2016;16(4):358-384.
74. Schuring M, Robroek SJ, Burdorf A. The benefits of paid employment among persons with common mental health problems: evidence for the selection and causation mechanism. Scand J Work Environ Health. 2017;43(6):540-549.
75. Burns T, Catty J, White S, et al. The impact of supported employment and working on clinical and social functioning: results of an international study of individual placement and support. Schizophr Bull. 2009;35(5):949-958.
76. Kilian R, Lauber C, Kalkan R, et al. The relationships between employment, clinical status, and psychiatric hospitalisation in patients with schizophrenia receiving either IPS or a conventional vocational rehabilitation programme. Soc Psychiatry Psychiatr Epidemiol. 2012;47(9):1381-1389.
77. Marwaha S, Johnson S. Schizophrenia and employment. Soc Psychiatry Psychiatr Epidemiol. 2004;39(5):337-349.
78. Mueser KT, Drake RE, Bond GR. Recent advances in supported employment for people with serious mental illness. Curr Opin Psychiatry. 2016;29(3):196-201.
79. Bush PW, Drake RE, Xie H, et al. The long-term impact of employment on mental health service use and costs for persons with severe mental illness. Psychiatr Serv. 2009;60(8):1024-1031.
80. Drake RE, Skinner JS, Bond GR, Goldman HH. Social security and mental illness: reducing disability with supported employment. Health Aff (Millwood). 2009;28(3):761-770.
81. Substance Abuse and Mental Health Services Administration. SAMHSA’s working definition of recovery. Substance Abuse and Mental Health Services Administration. Published 2012. Accessed January 11, 2021. https://store.samhsa.gov/system/files/pep12-recdef.pdf
82. Drake RE, Whitley R. Recovery and severe mental illness: description and analysis. Can J Psychiatry. 2014;59(5):236-242.
83. Wisconsin Department of Health Services. PPS Mental Health Module Handbook. Published 2018. Accessed January 11, 2021. https://www.dhs.wisconsin.gov/publications/p02182.pdf
84. Robert Wood Johnson Foundation. Housing and Health. How does housing affect health? May 1, 2011. Accessed January 11, 2021. https://www.rwjf.org/en/library/research/2011/05/housing-and-health.html
85. Cunningham MK, MacDonald G. Housing as a platform for improving education outcomes among low-income children. Urban Institute. May 2012. Accessed January 11, 2021. https://www.urban.org/sites/default/files/publication/25331/412554-Housing-as-a-Platform-for-Improving-Education-Outcomes-among-Low-Income-Children.PDF
86. Friedman D. Social impact of poor housing. ECOTEC. March 2010. Accessed January 11, 2021. https://southdevonrural.co.uk/userfiles/file/JC-JC13-Social-impact-of-poor-housing.pdf
87. Fazel S, Geddes JR, Kushel M. The health of homeless people in high-income countries: descriptive epidemiology, health consequences, and clinical and policy recommendations. Lancet Lond Engl. 2014;384(9953):1529-1540.
88. Nusselder WJ, Slockers MT, Krol L, et al. Mortality and life expectancy in homeless men and women in Rotterdam: 2001–2010. PLoS One. 2013;8(10):e73979.
89. O’Connell JJ. Premature mortality in homeless populations: a review of the literature. National Health Care for the Homeless Council. Published 2005. Accessed April 23, 2019. http://sbdww.org/wp-content/uploads/2011/04/PrematureMortalityFinal.pdf
90. Hwang SW, Wilkins R, Tjepkema M, et al. Mortality among residents of shelters, rooming houses, and hotels in Canada: 11 year follow-up study. BMJ. 2009;339:b4036.
91. Roncarati JS, Baggett TP, O’Connell JJ, et al. Mortality among unsheltered homeless adults in Boston, Massachusetts, 2000-2009. JAMA Intern Med. 2018;178(9):1242-1248.
92. Thomson H, Thomas S, Sellstrom E, Petticrew M. Housing improvements for health and associated socio-economic outcomes. Cochrane Database Syst Rev. 2013;(2):CD008657.
93. Fenelon A, Mayne P, Simon AE, et al. Housing assistance programs and adult health in the United States. Am J Public Health. 2017;107(4):571-578.
94. Fitzpatrick-Lewis D, Ganann R, Krishnaratne S, et al. Effectiveness of interventions to improve the health and housing status of homeless people: a rapid systematic review. BMC Public Health. 2011;11(1):638.
95. Rog DJ, Marshall T, Dougherty RH, et al. Permanent supportive housing: assessing the evidence. Psychiatr Serv. 2014;65(3):287-294.
96. Kirst M, Zerger S, Wise Harris D, et al. The promise of recovery: narratives of hope among homeless individuals with mental illness participating in a Housing First randomised controlled trial in Toronto, Canada: Table 1. BMJ Open. 2014;4(3):e004379.
97. Cohen S, Janicki-Deverts D. Can we improve our physical health by altering our social networks? Perspect Psychol Sci. 2009;4(4):375-378.
98. House JS, Landis KR, Umberson D. Social relationships and health. Science. 1988;241(4865):540-545.
99. Kawachi I, Berkman LF. Social ties and mental health. J Urban Health. 2001;78(3):458-467.
100. Schön U-K, Denhov A, Topor A. Social relationships as a decisive factor in recovering from severe mental illness. Int J Soc Psychiatry. 2009;55(4):336-347.
101. Soundy A, Stubbs B, Roskell C, et al. Identifying the facilitators and processes which influence recovery in individuals with schizophrenia: a systematic review and thematic synthesis. J Ment Health. 2015;24(2):103-110.
102. Tew J, Ramon S, Slade M, et al. Social factors and recovery from mental health difficulties: a review of the evidence. Br J Soc Work. 2012;42(3):443-460.
103. Moos RH. Theory-based processes that promote the remission of substance use disorders. Clin Psychol Rev. 2007;27(5):537-551.
104. Stiefel MC, Riley CL, Roy B, et al. 100 Million Healthier Lives Measurement System: Progress to date. Institute for Healthcare Improvement; 2016:41. Accessed January 11, 2021. http://www.100mlives.org
105. Swanson JW, Frisman LK, Robertson AG, et al. Costs of criminal justice involvement among persons with serious mental illness in Connecticut. Psychiatr Serv. 2013;64(7):630-637.
106. Fazel S, Bains P, Doll H. Substance abuse and dependence in prisoners: a systematic review. Addiction. 2006;101(2):181-191.
107. Fazel S, Seewald K. Severe mental illness in 33 588 prisoners worldwide: systematic review and meta-regression analysis. Br J Psychiatry. 2012;200(05):364-373.
108. Fazel S, Baillargeon J. The health of prisoners. Lancet. 2011;377(9769):956-965.
109. Wildeman C, Wang EA. Mass incarceration, public health, and widening inequality in the USA. Lancet. 2017;389(10077):1464-1474.
110. Zlodre J, Fazel S. All-cause and external mortality in released prisoners: systematic review and meta-analysis. Am J Public Health. 2012;102(12):e67-e75.
111. Massoglia M, Pridemore WA. Incarceration and health. Annu Rev Sociol. 2015;41:291-310.
112. Kouyoumdjian FG, Cheng SY, Fung K, et al. The health care utilization of people in prison and after prison release: a population-based cohort study in Ontario, Canada. PLoS One. 2018;13(8):e0201592.
113. Winkelman TNA, Genao I, Wildeman C, Wang EA. Emergency department and hospital use among adolescents with justice system involvement. Pediatrics. 2017;140(5):e20171144.
114. Wildeman C. Imprisonment and infant mortality. Soc Prob. 2012;59:228-257.
115. Geller A, Cooper CE, Garfinkel I, et al. Beyond absenteeism: father incarceration and child development. Demography. 2012;49(1):49-76.
116. Turney K. Stress proliferation across generations? Examining the relationship between parental incarceration and childhood health. J Health Soc Behav. 2014;55(3):302-319.
117. Geller A, Garfinkel I, Western B. Paternal incarceration and support for children in fragile families. Demography. 2011;48(1):25-47.
118. Lee H, Wildeman C, Wang EA, et al. A heavy burden: the cardiovascular health consequences of having a family member incarcerated. Am J Public Health. 2014;104(3):421-427.
119. Western B. The impact of incarceration on wage mobility and inequality. Am Sociol Rev. 2002;67(4):526-546.
120. Constantine R, Andel R, Petrila J, et al. Characteristics and experiences of adults with a serious mental illness who were involved in the criminal justice system. Psychiatr Serv. 2010;61(5):451-457.
121. Garnick DW, Horgan CM, Acevedo A, et al. Criminal justice outcomes after engagement in outpatient substance abuse treatment. J Subst Abuse Treat. 2014;46(3):295-305.
122. Adler NE, Ostrove JM. Socioeconomic status and health: what we know and what we don’t. Ann N Y Acad Sci. 1999;896(1):3-15.
123. Luo Y, Waite LJ. The impact of childhood and adult SES on physical, mental, and cognitive well-being in later life. J Gerontol Ser B. 2005;60(2):S93-S101.
124. Mackenbach JP, Stirbu I, Roskam A-JR, et al. Socioeconomic inequalities in health in 22 European countries. N Engl J Med. 2008;358(23):2468-2481.
125. Hudson CG. Socioeconomic status and mental illness: tests of the social causation and selection hypotheses. Am J Orthopsychiatry. 2005;75(1):3-18.
126. McLaughlin KA, Breslau J, Green JG, et al. Childhood socio-economic status and the onset, persistence, and severity of DSM-IV mental disorders in a US national sample. Soc Sci Med. 2011;73(7):1088-1096.
127. Muntaner C. Socioeconomic position and major mental disorders. Epidemiol Rev. 2004;26(1):53-62.
128. Präg P, Mills MC, Wittek R. Subjective socioeconomic status and health in cross-national comparison. Soc Sci Med. 2016;149:84-92.
129. Shaked D, Williams M, Evans MK, Zonderman AB. Indicators of subjective social status: differential associations across race and sex. SSM Popul Health. 2016;2:700-707.
130. Singh-Manoux A, Adler NE, Marmot MG. Subjective social status: its determinants and its association with measures of ill-health in the Whitehall II study. Soc Sci Med. 2003;56(6):1321-1333.
131. Adler NE, Epel ES, Castellazzo G, Ickovics JR. Relationship of subjective and objective social status with psychological and physiological functioning: preliminary data in healthy, white women. Health Psychol. 2000;19(6):586-592.
132. Cundiff JM, Matthews KA. Is subjective social status a unique correlate of physical health? A meta-analysis. Health Psychol. 2017;36(12):1109-1125.
133. Demakakos P, Biddulph JP, de Oliveira C, et al. Subjective social status and mortality: The English Longitudinal Study of Ageing. Eur J Epidemiol. 2018;33(8):729-739.
134. Quon EC, McGrath JJ. Subjective socioeconomic status and adolescent health: a meta-analysis. Health Psychol. 2014;33(5):433-447.
135. Scott KM, Al-Hamzawi AO, Andrade LH, et al. Associations between subjective social status and DSM-IV mental disorders: Results from the World Mental Health Surveys. JAMA Psychiatry. 2014;71(12):1400-1408.
136. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare Fee-for-Service Program. N Engl J Med. 2009;360(14):1418-1428.
137. Krumholz HM, Wang K, Lin Z, et al. Hospital-readmission risk—Isolating hospital effects from patient effects. N Engl J Med. 2017;377(11):1055-1064.
138. Krumholz HM, Lin Z, Keenan PS, et al. Relationship of hospital performance with readmission and mortality rates for patients hospitalized with acute myocardial infarction, heart failure, or pneumonia. JAMA. 2013;309(6):587-593.
139. Chin DL, Bang H, Manickam RN, Romano PS. Rethinking thirty-day hospital readmissions: Shorter intervals might be better indicators of quality of care. Health Aff (Millwood). 2016;35(10):1867-1875.
140. Feltner C, Jones CD, Cené CW, et al. Transitional care interventions to prevent readmissions for persons with heart failure: a systematic review and meta-analysis. Ann Intern Med. 2014;160(11):774-784.
141. Hudon C, Chouinard M-C, Lambert M, et al. Effectiveness of case management interventions for frequent users of healthcare services: a scoping review. BMJ Open. 2016;6(9):e012353.
142. Kripalani S, Theobald CN, Anctil B, Vasilevskis EE. Reducing hospital readmission: current strategies and future directions. Annu Rev Med. 2014;65:471-485.
143. Verhaegh KJ, MacNeil-Vroomen JL, Eslami S, et al. Transitional care interventions prevent hospital readmissions for adults with chronic illnesses. Health Aff (Millwood). 2014;33(9):1531-1539.
144. Duhig M, Gunasekara I, Patterson S. Understanding readmission to psychiatric hospital in Australia from the service users’ perspective: a qualitative study. Health Soc Care Community. 2017;25(1):75-82.
145. New England Healthcare Institute. A matter of urgency: Reducing emergency department overuse. Published 2010. Accessed January 11, 2021. https://www.nehi.net/writable/publication_files/file/nehi_ed_overuse_issue_brief_032610finaledits.pdf
146. Billings J, Raven MC. Dispelling an urban legend: Frequent emergency department users have substantial burden of disease. Health Aff (Millwood). 2013;32(12):2099-2108.
147. Moe J, Kirkland S, Ospina MB, et al. Mortality, admission rates and outpatient use among frequent users of emergency departments: a systematic review. Emerg Med J. 2016;33(3):230-236.
148. Carret MLV, Fassa ACG, Domingues MR. Inappropriate use of emergency services: a systematic review of prevalence and associated factors. Cad Saúde Pública. 2009;25(1):7-28.
149. Durand A-C, Palazzolo S, Tanti-Hardouin N, et al. Nonurgent patients in emergency departments: rational or irresponsible consumers? Perceptions of professionals and patients. BMC Res Notes. 2012;5(1):525.
150. Uscher-Pines L, Pines J, Kellermann A, et al. Deciding to visit the emergency department for non-urgent conditions: a systematic review of the literature. Am J Manag Care. 2013;19(1):47-59.
151. Kumar GS, Klein R. Effectiveness of case management strategies in reducing emergency department visits in frequent user patient populations: a systematic review. J Emerg Med. 2013;44(3):717-729.
152. Moe J, Kirkland SW, Rawe E, et al. Effectiveness of interventions to decrease emergency department visits by adult frequent users: a systematic review. Acad Emerg Med. 2017;24(1):40-52.
153. Raven MC, Kushel M, Ko MJ, et al. The effectiveness of emergency department visit reduction programs: a systematic review. Ann Emerg Med. 2016;68(4):467-483.e15.
154. Soril LJJ, Leggett LE, Lorenzetti DL, et al. Reducing frequent visits to the emergency department: a systematic review of interventions. PLoS One. 2015;10(4):e0123660.
155. Van den Heede K, Van de Voorde C. Interventions to reduce emergency department utilisation: a review of reviews. Health Policy. 2016;120(12):1337-1349.
156. Onken SJ, Dumont JM, Ridgway P, et al. Mental health recovery: What helps and what hinders? October 2002. Accessed January 11, 2021. https://www.nasmhpd.org/sites/default/files//MHSIPReport%281%29.pdf
157. Leggett LE, Khadaroo RG, Holroyd-Leduc J, et al. Measuring resource utilization. Medicine (Baltimore). 2016;95(10):e2759.
158. Bhandari A, Wagner T. Self-reported utilization of health care services: improving measurement and accuracy. Med Care Res Rev. 2006;63(2):217-235.
159. Short ME, Goetzel RZ, Pei X, et al. How accurate are self-reports? An analysis of self-reported healthcare utilization and absence when compared to administrative data. J Occup Environ Med. 2009;51(7):786-796.
160. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. National Academies Press; 2001.
161. Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med. 1991;324(6):370-376.
162. Rafter N, Hickey A, Condell S, et al. Adverse events in healthcare: learning from mistakes. QJM. 2015;108(4):273-277.
163. Makary MA, Daniel M. Medical error—the third leading cause of death in the US. BMJ. 2016;3(353):i2139.
164. D’Lima D, Crawford MJ, Darzi A, Archer S. Patient safety and quality of care in mental health: A world of its own? BJPsych Bull. 2017;41(5):241-243.
165. Maidment ID, Lelliott P, Paton C. Medication errors in mental healthcare: a systematic review. Qual Saf Health Care. 2006;15(6):409-413.
166. Mills PD, Watts BV, Shiner B, Hemphill RR. Adverse events occurring on mental health units. Gen Hosp Psychiatry. 2018;50:63-68.
167. Marcus SC, Hermann RC, Frankel MR, Cullen SW. Safety of psychiatric inpatients at the Veterans Health Administration. Psychiatr Serv. 2017;69(2):204-210.
168. Reilly CA, Cullen SW, Watts BV, et al. How well do incident reporting systems work on inpatient psychiatric units? Jt Comm J Qual Patient Saf. 2019;45:63-69.
169. Rosano A, Loha CA, Falvo R, et al. The relationship between avoidable hospitalization and accessibility to primary care: a systematic review. Eur J Public Health. 2013;23(3):356-360.
170. Dupre ME, Xu H, Granger BB, et al. Access to routine care and risks for 30-day readmission in patients with cardiovascular disease. Am Heart J. 2018;196:9-17.
171. Pizer SD, Prentice JC. What are the consequences of waiting for health care in the veteran population? J Gen Intern Med. 2011;26(S2):676-682.
172. Festinger DS, Lamb RJ, Kirby KC, Marlowe DB. The accelerated intake: a method for increasing initial attendance to outpatient cocaine treatment. J Appl Behav Anal. 1996;29(3):387-389.
173. Festinger DS, Lamb RJ, Marlowe DB, Kirby KC. From telephone to office: intake attendance as a function of appointment delay. Addict Behav. 2002;27(1):131-137.
174. Gallucci G, Swartz W, Hackerman F. Brief reports: Impact of the wait for an initial appointment on the rate of kept appointments at a mental health center. Psychiatr Serv. 2005;56(3):344-346.
175. Bleustein C, Rothschild DB, Valen A, et al. Wait times, patient satisfaction scores, and the perception of care. Am J Manag Care. 2014;20(5):393-400.
176. Reichert A, Jacobs R. The impact of waiting time on patient outcomes: Evidence from early intervention in psychosis services in England. Health Econ. 2018;27(11):1772-1787.
177. Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013;3(1):e001570.
178. Horvath AO, Del Re AC, Flückiger C, Symonds D. Alliance in individual psychotherapy. Psychotherapy. 2011;48(1):9-16.
179. Farley H, Enguidanos ER, Coletti CM, et al. Patient satisfaction surveys and quality of care: an information paper. Ann Emerg Med. 2014;64(4):351-357.
180. Aiken LH, Sermeus W, Van den Heede K, et al. Patient safety, satisfaction, and quality of hospital care: cross sectional surveys of nurses and patients in 12 countries in Europe and the United States. BMJ. 2012;344:e1717.
181. Boulding W, Glickman SW, Manary MP, Schulman KA, Staelin R. Relationship between patient satisfaction with inpatient care and hospital readmission within 30 days. Am J Manag Care. 2011;17(1):41-48.
182. Rohland BM, Langbehn DR, Rohrer JE. Relationship between service effectiveness and satisfaction among persons receiving Medicaid mental health services. Psychiatr Serv. 2000;51(2):248-250.
183. Zendjidjian X-Y, Baumstarck K, Auquier P, et al. Satisfaction of hospitalized psychiatry patients: Why should clinicians care? Patient Prefer Adherence. 2014;8:575-583.
184. Centers for Medicare & Medicaid Services. National health expenditures; aggregate and per capita amounts, annual percent change and percent distribution: Calendar years 1960-2016. National Health Expenditure Data. https://www.cms.gov/research-statistics-data-and-systems/statistics-trends-and-reports/nationalhealthexpenddata/nationalhealthaccountshistorical.html. Published 2018. Accessed August 21, 2018.
185. DuBard CA. Running the numbers. N C Med J. 2016;77(4):297-300.
186. Peikes D, Chen A, Schore J, Brown R. Effects of care coordination on hospitalization, quality of care, and health care expenditures among Medicare beneficiaries: 15 randomized trials. JAMA. 2009;301(6):603-618.
187. Seow H-Y, Sibley LM. Developing a dashboard to help measure and achieve the triple aim: A population-based cohort study. BMC Health Serv Res. 2014;14(1):363.
188. Lee VS, Kawamoto K, Hess R, et al. Implementation of a value-driven outcomes program to identify high variability in clinical costs and outcomes and association with reduced cost and improved quality. JAMA. 2016;316(10):1061.
189. Dyrbye LN, Shanafelt TD, Sinsky CA, et al. Burnout among health care professionals: A call to explore and address this underrecognized threat to safe, high-quality care discussion paper. National Academy of Medicine. July 5, 2017. Accessed January 11, 2021. https://nam.edu/burnout-among-health-care-professionals-a-call-to-explore-and-address-this-underrecognized-threat-to-safe-high-quality-care/
190. Johnson J, Hall LH, Berzins K, et al. Mental healthcare staff well-being and burnout: a narrative review of trends, causes, implications, and recommendations for future interventions. Int J Ment Health Nurs. 2018;27(1):20-32.
191. Morse G, Salyers MP, Rollins AL, et al. Burnout in mental health services: a review of the problem and its remediation. Adm Policy Ment Health. 2012;39(5):341-352.
192. Hall LH, Johnson J, Watt I, et al. Healthcare staff wellbeing, burnout, and patient safety: a systematic review. PLoS One. 2016;11(7):e0159015.
193. Garman AN, Corrigan PW, Morris S. Staff burnout and patient satisfaction: evidence of relationships at the care unit level. J Occup Health Psychol. 2002;7(3):235-241.
194. Wilkinson H, Whittington R, Perry L, Eames C. Examining the relationship between burnout and empathy in healthcare professionals: a systematic review. Burn Res. 2017;6:18-29.
195. Holmqvist R, Jeanneau M. Burnout and psychiatric staff’s feelings towards patients. Psychiatry Res. 2006;145(2-3):207-213.
196. Leiter MP, Maslach C. Nurse turnover: the mediating role of burnout. J Nurs Manag. 2009;17(3):331-339.
197. Zhang Y, Feng X. The relationship between job satisfaction, burnout, and turnover intention among physicians from urban state-owned medical institutions in Hubei, China: a cross-sectional study. BMC Health Serv Res. 2011;11(1):235.
198. Halter M, Boiko O, Pelone F, et al. The determinants and consequences of adult nursing staff turnover: a systematic review of systematic reviews. BMC Health Serv Res. 2017;17(1):824.
199. Hamidi MS, Bohman B, Sandborg C, et al. Estimating institutional physician turnover attributable to self-reported burnout and associated financial burden: a case study. BMC Health Serv Res. 2018;18(1):851.
200. National Taskforce for Humanity in Healthcare. The business case for humanity in healthcare position paper. April 2018. Accessed January 11, 2021. https://www.vocera.com/public/pdf/NTHBusinessCase_final003.pdf
201. Waldman JD, Kelly F, Arora S, Smith HL. The shocking cost of turnover in health care. Health Care Manage Rev. 2004;29(1):2-7.
202. Brunette MF, Asher D, Whitley R, et al. Implementation of integrated dual disorders treatment: a qualitative analysis of facilitators and barriers. Psychiatr Serv. 2008;59(9):989-995.
203. Mancini AD, Moser LL, Whitley R, et al. Assertive community treatment: facilitators and barriers to implementation in routine mental health settings. Psychiatr Serv. 2009;60(2):189-195.
204. Rollins AL, Salyers MP, Tsai J, Lydick JM. Staff turnover in statewide implementation of ACT: Relationship with ACT fidelity and other team characteristics. Adm Policy Ment Health. 2010;37(5):417-426.
205. Woltmann EM, Whitley R, McHugo GJ, et al. The role of staff turnover in the implementation of evidence-based practices in mental health care. Psychiatr Serv. 2008;59(7):732-737.
206. Alegría M, NeMoyer A, Falgàs Bagué I, et al. Social determinants of mental health: Where we are and where we need to go. Curr Psychiatry Rep. 2018;20(11):95.
207. Daniel H, Bornstein SS, Kane GC. Addressing social determinants to improve patient care and promote health equity: an American College of Physicians position paper. Ann Intern Med. 2018;168(8):577-578.
208. Park H, Roubal AM, Jovaag A, Gennuso KP, Catlin BB. Relative contributions of a set of health factors to selected health outcomes. Am J Prev Med. 2015;49(6):961-969.
209. Ash AS, Mick EO, Ellis RP, et al. Social determinants of health in managed care payment formulas. JAMA Intern Med. 2017;177(10):1424.
210. de Beurs E, Warmerdam EH, Oudejans SCC, et al. Treatment outcome, duration, and costs: a comparison of performance indicators using data from eight mental health care providers in the Netherlands. Adm Policy Ment Health. 2018;45(2):212-223.
211. Dunbar-Rees R. Paying for what matters most: the future of outcomes-based payments in healthcare. Future Healthc J. 2018;5(2):98-102.
212. Woolhandler S, Himmelstein DU. Administrative work consumes one-sixth of U.S. physicians’ working hours and lowers their career satisfaction. Int J Health Serv. 2014;44(4):635-642.
From the Milwaukee County Behavioral Health Division, Milwaukee, WI.
Abstract
Objective: Implementation of the Quadruple Aim of health care must begin with a clearly articulated set of concepts, or core domains (CDs), that comprise each aim. These CDs can then be operationalized with existing or new measures. If aligned to the organization’s mission and strategic goals, these CDs have the potential to focus quality improvement activities and reduce measurement burden. This article represents the efforts of a publicly funded behavioral health system to operationalize the Quadruple Aim through the development of CDs.
Methods: Various stakeholders across the organization were consulted on their perceptions of the Quadruple Aim and the CDs they believed should support it. Then, a review of existing literature on core metrics for health care and population health was completed, summarized, and integrated with the stakeholder feedback.
Results: These efforts led to the development and adoption of 15 CDs, with an accompanying literature review and set of recommendations of new and existing measures for each domain.
Conclusions: It is possible to create a comprehensive yet economical set of CDs and attendant measures that can be implemented in a staged, scalable, enterprise manner. It is hoped that the process articulated here, and the accompanying literature review, may be of some benefit to other public or government-run health systems in their own quality improvement journey to operationalize the Quadruple Aim by developing a set of CDs.
Keywords: quality measures; quality improvement; adult behavioral health.
First articulated in 2008, the Triple Aim proposes that health care systems should simultaneously seek to improve the patient’s experience of care, improve the health of populations, and reduce the per capita costs of care for populations.1 More recently, some have argued that health care provider burnout can deleteriously impact the attainment of the Triple Aim and have therefore advocated for an expanded focus to include a fourth Aim, the work life quality of the staff.2 Milwaukee County Behavioral Health Division (BHD), a publicly funded, county-based behavioral health care system in Milwaukee, Wisconsin, recently adopted the Quadruple Aim as the framework by which it will organize its quality activities.
Although originally developed for medical organizations, BHD believes that the Quadruple Aim has strong applicability to county-level behavioral health services. Many county-based behavioral health divisions provide a variety of programs to large segments of the county based on financial eligibility and/or clinical need, and thus often have responsibilities to populations or subpopulations, rather than programs. County health divisions, such as Milwaukee County’s Department of Health and Human Services, are often asked to improve outcomes and client experience of care with neutral growth budgets and less reliance on taxes to fund programs, while simultaneously attracting and retaining competent staff.
Crucial to the effective implementation of the Quadruple Aim, however, is a clear set of population- level measures that help organizations assess their progress.3 Unfortunately, as some authors have noted, evaluation of the Quadruple Aim remains a challenge because the “concepts of (population) health, quality of care and costs are not unanimously defined and measures for these concepts are under construction.”4 Several authors have provided some guidance to assist in the development of a set of measures that effectively capture the elements of the Quadruple Aim.5,6 However, the recent rapid proliferation of quality measures in health care7,8 has been both burdensome and costly for providers.9,10 Any measures adopted should not only be as meaningful as possible with regards to assessing progress towards the basic aims of health care, but should also be parsimonious, to limit measurement burden for providers (and patients) and focus attention on important issues.11,12
To select the most effective, parsimonious set of measures possible, one must first select a set of key foci from among the many possible areas of focus that the core measure is intended to represent. The core domains (CDs), if appropriately consistent with the strategic goals of the organization, provide a mechanism to orient the efforts of the organization at every level and help every staff member of the organization understand how his or her work impacts the progress towards these goals.11 The CDs, therefore, represent the opportunity to affect a greater integration of efforts across the organization toward these shared aims, creating uniformity of purpose at every level. Further, increasing organizational attention on the CDs can also help to reduce measurement burden by streamlining and focusing the data capture processes on the most valuable elements of quality and health, and discarding other extraneous measures (albeit not at the expense of other reporting requirements).11 The remainder of this article describes the CDs selected by BHD to assess its progress toward implementation of the Quadruple Aim and are organized by the Aim which they best represent.
Methods
To effectively implement the Quadruple Aim at BHD, it was necessary to clearly define the subpopulation of focus for our efforts.6 In this case, the subpopulation of interest was defined as all adult clients (18 years and older) who received at least 1 service encounter within a specified time frame from a program that BHD either operated or contracted with to provide care. Services provided by the BHD network include everything from psychiatric inpatient services to mental health and addiction treatment and care management. A limited array of social services, including housing and employment services, is also available to eligible consumers. BHD is the county-run behavioral health provider for individuals who are uninsured or underinsured in Milwaukee County, a demographically diverse, primarily urban county of approximately 950,000 people located in Wisconsin. Approximately 15,000 adults receive services at BHD each year.
This work began by obtaining executive sponsorship for the project, in this case from the Chief Operations Officer and Executive Medical Director of BHD. With their backing, an initial review of the literature produced a preliminary set of possible domains, for which we created working definitions. We then made a list of key stakeholders throughout BHD to whom we needed to present the idea of the Quadruple Aim, and the CDs under each Aim, to secure their support. These stakeholders, which included individuals involved in quality activities, program managers, and executive leadership, were strategically selected based on their relative influence within the organization. A set of brief presentations and handouts explaining the project were then developed and shared at different focus groups with these stakeholders over the course of 6 months. These focus groups served to not only educate the organization about the Quadruple Aim and the CDs but afforded participants an opportunity to provide feedback as well.
During the focus groups, we asked participants which domains they believed were most important (were “core”) when operationalizing the Quadruple Aim. The focus groups provided feedback on the domain definitions, feedback that was used to develop uniform, mutually agreed upon definitions for the CDs that were generalizable to all departments at BHD, regardless of the focus of their services within the continuum of care or the continuum of age. This was a crucial step, as it will eventually enable BHD to aggregate data across departments, even if there are minor discrepancies in the specific items they use to assess the CDs. Comments from the focus groups ultimately resulted in a truncated list of domains and definitions, which, coupled with the literature review, resulted in our final set of CDs.
During our review of the literature, we also looked for items that we felt could best represent each CD in the briefest, most meaningful way. (These items were not meant to supersede existing data, but to provide examples that could be implemented with existing data or recommendations that could be utilized in the absence of existing data.) During this process, we made every effort to make use of existing data-reporting requirements. For example, if we had a state mandate to collect data on housing status, we attempted to leverage this required data point to represent the CD related to housing. In other cases, we attempted to utilize claims or other administrative data to operationalize the CD, such as in the cost-of-care metric articulated in the section the Third Aim. For CDs for which no data existed or were insufficient, we emphasized the use of single- versus multi-item scales. For example, if we found a single-item global assessment of quality of life that had good psychometric properties relative to its longer parent scale, we selected the single item. This approach to item selection allowed us to create the most efficient, parsimonious set of measures possible, which we believed would enable us to comprehensively assess all the CDs with the least amount of burden to staff and clients. These items were presented at stakeholder focus groups, during which we asked for comments on the existing measures in their program or department and gave them the opportunity to comment on the new recommended measures.
A working definition is provided for each CD, followed by a brief review of the research base supporting its inclusion in the final list. The item(s) selected by BHD to represent each CD and the source of the item(s) are then supplied. These items were based either on measures currently collected because of existing reporting mandates or, in the case where extant measures were not available, on new items that demonstrated acceptable psychometric properties in the research literature. The CDs and items are organized by the Aim they best represent. A full list of the CDs by Quadruple Aim and items by CD is provided in the Appendix of the online version of this article. This article concludes with a brief summary of this effort and a discussion of how staff will utilize these items at different levels throughout the BHD system.
The First Aim: Population Health
Health Outcomes
Deaths. This can be defined as the cause of death, as determined by the medical examiner’s office (where appropriate) or as the age at time of death. This CD can also be reported as proportion of deaths considered premature (eg, before age 75) or calculated as total years of potential life lost.
Brief review and suggested item(s). Rates and causes of premature mortality are critical foci for the County Health Rankings & Roadmaps,13 the Institute for Healthcare Improvement’s “Guide to Measuring the Triple Aim,”6 the Centers for Disease Control and Prevention’s “Community Health Assessment for Population Health Improvement,”14 and the Institute of Medicine’s (IOM) “Vital Signs: Core Metrics for Health and Health Care Progress.”11 There is ample evidence that individuals with serious mental illness are at increased risk of early mortality relative to the general population,15-18 and this risk applies to those with substance use disorders as well.15,19-20 BHD tracks all deaths that occur while patients are receiving BHD-funded, community-based services.
Self-Reported Health and Well-Being. This CD asks patients to rate their current physical and mental health status, as well as their overall quality of life.
Brief review and suggested item(s): Self-rated physical health. Premature mortality among individuals with behavioral health issues appears to be due, in large part, to their increased vulnerability to the development of medical comorbidities.16,21 A single self-rating question has demonstrated considerable sensitivity to premature mortality,22,23 with predictive properties up to a decade prior to death.24,25 Further, self-rated health has been associated with subsequent functional decline,26,27 acute service utilization,28,29 and overall health care costs.28
Brief review and suggested item(s): Self-rated mental health. Mental health disorders are associated with significant disability worldwide,30 and comorbid mental health issues can exacerbate the course of other medical problems. For example, depression is associated with increased rates of mortality among individuals with diabetes and31 cardiovascular disease,32 as well as with rates of overall mortality,33 and psychiatric comorbidity is associated with longer lengths of stay and higher costs among patients hospitalized for medical problems.34 Research has found that a single-item measure of self-rated mental health is associated with the presence of psychiatric diagnoses, psychiatric symptoms, and subsequent depression and serious mental illness up to 1 year post-assessment.35,36 There is even evidence that self-rated mental health may be more strongly associated with self-ratings of overall health than self-ratings of physical health.37
Brief review and suggested item(s): Self-rated quality of life. Quality of life is a critical component of the recovery journey and overall health.38 For example, the County Health Rankings & Roadmaps lists “quality of life” as 1 of its key “health outcomes” in its County Health Rankings.13 As some authors have noted, quality of life is often inferred from other “objective” recovery domains, such as employment, health status, or housing status. However, there is evidence that these objective domains are functionally distinct from the inherently subjective construct of quality of life.39 This has led other authors to conclude that these domains should be assessed separately when evaluating outcomes.40 Single-item quality of life assessments have been used in research with individuals with cancer,41 adults with disabilities,42 patients with cystic fibrosis,43 and children with epilepsy.44 For this effort, BHD selected the first global quality of life item from the World Health Organization’s WHOQOL-BREF quality of life assessment,45 an item used in other quality of life research.46
Health Factors
Substance Use. This CD is a composite of 4 different types of substance use, any recent heavy alcohol use (defined as 5 or more drinks in one sitting), any recent drug use, any recent prescription drug abuse, and any recent tobacco use.
Brief review and suggested item(s). As noted, substance use disorders confer an increased risk for early mortality15,19 and are significantly implicated in disease disability burden worldwide.30 Substance use has also been associated with both the onset47,48 and exacerbation of mental health diagnoses.49-51 Further, substance use appears to heighten the risk of violence in the general population52 and especially among those with a co-occurring mental illness.53,54 The County Health Rankings & Roadmaps list alcohol and drug use as key behaviors to address to improve the overall health of a given county,13 and the Centers for Medicare & Medicaid Services (CMS) has endorsed initiation and engagement in addiction treatment as one of the measures in its Adult Core Set.55
Tobacco use continues to be one of the most significant risk factors for early mortality worldwide, and evidence indicates that it is associated with a lower life expectancy of nearly 10 years.56 Unfortunately, rates of tobacco use are even higher among those with severe mental illness relative to the general population, and their rates of smoking cessation are lower.57,58 Tobacco use is a significant risk factor for the high rates of early mortality in individuals with severe mental illness.18 Further, a recent meta-analysis noted that, relative to those who continued to smoke, those who ceased smoking had reduced rates of psychological distress and increased quality of life rankings.59 Reducing tobacco use is one of the key components of the County Health Rankings & Roadmaps, and medication assistance with smoking and tobacco use cessation is also listed in the CMS Adult Core Set.13,55
An accumulating body of evidence suggests that single-item measures can adequately detect alcohol60-62 and drug use disorders.60-64 McNeely and colleagues recently developed and tested a brief 4-item screen, the Tobacco, Alcohol, Prescription medication, and other Substance use (TAPS) tool.65,66 Preliminary evidence suggests that the TAPS tool can effectively identify the presence of problematic and disordered use of tobacco, alcohol, prescription medications, and other drugs.65-67 BHD will use the 4 items from the TAPS tool to represent its substance use CD.
Education/Employment Status. This CD assesses the proportion of BHD members who have completed high school, who are in some type of educational or training program, or who are engaged in some type of employment activity (defined as full-time, part-time, supported, sheltered workshop, or as a full-time homemaker).
Brief review and suggested item(s). Research indicates that unemployment is a risk factor for mortality, even after controlling for other risk factors (eg, age, sex, socioeconomic status [SES], health).68 Unemployment is associated with poorer physical and mental health in the general population and among those with disabilities.69-71 Promisingly, evidence suggests that gaining employment or re-employment is associated with better health,72 even for individuals with substance use disorders73 or moderate74 to severe mental health disorders.75-78 Some authors have even proposed that, above and beyond the associated health benefits, employment may also help to realize a modest cost savings due to reduced service utilization and disability.79,80 Employment is a core tenet in the Substance Abuse and Mental Health Services Administration’s (SAMHSA’s) model of recovery,81 and is also listed as an important recovery goal for individuals with behavioral health issues.82 BHD collects data on employment status on all the patients it serves as part of its state-mandated reporting requirements and will use this item in the CD data set.83
Living Situation. This is measured as the proportion of people who live in permanent, supportive, stable housing; it may also be measured as the percentage of the population living with severe housing problems or who are homeless.
Brief review and suggested item(s). Housing problems can be conceptualized as 3 inter-related components: conditions within the home, neighborhood conditions, and housing affordability, each of which can contribute uniquely to poorer physical and mental health of individuals and families84 and to educational outcomes for children.85,86 Further, individuals who are homeless have a standardized mortality ratio 2 to 5 times that of the general population,87-89 even after controlling for low income status,90 and some evidence suggests these rates are even higher among unsheltered versus sheltered homeless individuals.91 Interventions to improve the condition of housing have demonstrated positive impacts on both physical and mental health,92 and a recent study found that individuals receiving housing assistance in the form of public housing or multifamily housing from the Department of Housing and Urban Development had better self-rated physical and mental health relative to individuals on the wait list for housing assistance.93 Moreover, the provision of housing has been shown to promote reductions in substance use and health service utilization among homeless individuals with substance use disorders.94 Rog and colleagues reviewed the literature on permanent supportive housing for individuals with substance use or mental health disorders who were homeless or disabled, and found that provision of housing led to reduced rates of homelessness, emergency department (ED) and inpatient utilization and increased consumer satisfaction.95
Importantly, evidence suggests that housing is viewed as facilitative of recovery. For example, in a recent qualitative study of homeless individuals with mental illness, housing was seen as a critical first step in recovery, providing a sense of security, increasing feelings of personal independence and autonomy, improving perceptions of health and well-being, and affording a stable environment to rebuild relationships with important others.96 BHD collects data on housing status on all the patients it serves as part of its state-mandated reporting requirements and will utilize this item in the CD data set.83
Social Relationships. This is defined as recent interactions with family, supportive networks (formal and informal), and other recovery services.
Brief review and suggested item(s). Research has long established that social relationships have a significant impact on health, including rates of mortality as well as physical and mental health morbidity.97-99 Social connectedness is another of the pillars supporting an individual’s recovery in SAMHSA’s formulation. Several reviews of the recovery literature38,82 support its importance to the recovery process and inclusion in any assessment of holistic recovery. Social support has been shown to promote recovery among individuals with severe mental illness100-102 and substance use disorders,103 and may mitigate the progression of chronic, life-threatening physical illnesses.97 For the purposes of BHD’s CD data set, the social support question from the “100 Million Healthier Lives Common Questionnaire for Adults” will be used to assess individuals’ perceived adequacy of social support.104
Legal Involvement. Defined as involvement with the civil or criminal justice system, including arrests, imprisonment, or detainment.
Brief review and suggested item(s). Involvement in the criminal justice system is both disruptive for the individual in recovery and expensive to the larger health care system.105 Individuals with substance use106 and severe mental health disorders107 are over-represented in the prison system, and evidence suggests that general physical and mental health declines while individuals are in prison.108,109 Perhaps even more concerning, numerous studies have demonstrated an increase in mortality rates for individuals recently released from prison relative to the general population, particularly during the period immediately following release.108-110 This relationship may even persist long term.111 Further, research indicates that individuals recently released from prison have increased emergency care and hospital utilization.112,113
Incarceration can have significant impacts on the health of the broader community as well. For example, research has found an association between parental incarceration to rates of infant mortality,114 increased behavioral and developmental problems of children of incarcerated parents,115,116 lower rates of child support payments,117 and poorer cardiovascular health of female partners of incarcerated individuals.118 Formerly incarcerated individuals experience slower wage growth as well.119 However, evidence also indicates that engagement in mental health120 and substance abuse121 treatment can reduce the likelihood of subsequent recidivism. As part of its state-mandated reporting, BHD is required to provide information on the criminal justice system involvement of its clients in the previous 6 months, including whether they have been jailed or imprisoned,83 and this will function as its measure of legal involvement in its CD data set.
Socioeconomic Status. Socioeconomic status is the social standing or class of an individual or group. It is often measured as a combination of education, income, and occupation. It can also be defined subjectively, such as one’s evaluation of status relative to similar others or based on an individual’s interpretation of her or his financial needs.
Brief review and suggested item(s). A large body of evidence supports the existence of a robust relationship between lower SES and poor health, including mortality and chronic medical diseases,122-124 as well as mental illness.125-127 Although previous research has examined this relationship using objective indicators of SES (eg, income, education level, occupation), there has recently been an increased interest in exploring the relationship of subjective SES with health indices. Subjective SES is generally assessed by asking individuals to rate themselves relative to others in the society in which they live, in terms of wealth, occupation, educational level, or other indicators of social status. Evidence suggests that subjective SES is associated with objective measures of SES,128-130 and relates to measures of physical and mental health as well, even after controlling for objective SES.130-135 BHD will be using a modified version of the Subject SES Scale,131,135 which is deployed in the “100 Million Healthier Lives Common Questionnaire for Adults.”104
Acute Service Use. This is defined as an admission to a medical or psychiatric emergency room or to a medical or psychiatric hospital or to a detoxification facility.
Brief review and suggested item(s). The CMS Adult Core Set includes “plan all cause readmissions” as a key quality metric.55 Hospital readmissions are also endorsed by the National Committee on Quality Assurance as one of its Health Effectiveness Data and Information Set (HEDIS) measures and by the National Quality Forum. Readmissions, despite their widespread endorsement, are a somewhat controversial measure. Although readmissions are costly to the health care system,136 the relationship between readmissions and quality is inconsistent. For example, Krumholz and colleagues137 found differential rates of readmission for the same patient discharged from 2 different hospitals, which were categorized based on previous readmission rates, suggesting that hospitals do have different levels of performance even when treating the same patient. However, other data indicate that 30-day, all-cause, risk-standardized readmission rates are not associated with hospital 30-day, all-cause, risk-standardized mortality rates.138
Chin and colleague found that readmissions to the hospital that occurred more than 7 days post-discharge were likely due to community- and household-related factors, rather than hospital-related quality factors.139 Transitional care interventions that have successfully reduced 30-day readmission rates are most often multicomponent and focus not just on hospital-based interventions (eg, discharge planning, education) but on follow-up care in the community by formal supports (eg, in-home visits, telephone calls, outpatient clinic appointments, case management) and informal supports (eg, family and friends).140-143 Further, qualitative evidence suggests that some individuals perceive psychiatric hospitalizations to be the result of insufficient resources or unsuccessful attempts to maintain their stability in the community.144 Thus, unplanned or avoidable hospital readmissions may represent a failure of the continuum of care not only from the perspective of the health care system, but from the patient perspective as well.
Frequent or nonurgent use of EDs is conceptually similar to excessive or avoidable inpatient utilization in several ways. For example, overuse of EDs is costly, with some estimates suggesting that it is responsible for up to $38 billion in wasteful spending each year.145 Individuals with frequent ED visits have a greater disease burden146 and an increased risk of mortality compared to nonfrequent users.147 Research suggests that individuals who visit the ED for non-urgent issues do so because of perceived difficulties associated with accessing primary care, and the convenience of EDs relative to primary care.148-150 Moreover, similar to the hospital readmission literature discussed earlier, successful strategies to reduce high rates of ED utilization generally focus on continuum of care interventions, such as provision of case management services.151-155
This evidence implies that frequent ED utilization and hospital readmissions may not be a fundamental issue of quality (or lack thereof) in hospitals or EDs but rather a lack of, or ineffectual, transitional and continuum of care strategies and services. To underscore this point, some authors have argued that a system that is excessively crisis-oriented hinders recovery because it is reactive rather than proactive, predicated on the notion that one’s condition must deteriorate to receive care.156
Although some organizations may have access to claims data or may function as self-contained health systems (eg, the Veterans Health Administration [VHA] ), others may not have access to such data. In the absence of claims data, patient self-report of service utilization has been used as a proxy for actual agency records.157 Although concordance between medical and/or agency records and patient self-report has been variable,157 evidence generally suggests that rates of agreement are higher the shorter the recall time interval.158,159 BHD does not have access to comprehensive claims data and has therefore chosen to use 5 dichotomously scored (yes/no) questions—related to medical inpatient, medical ED, psychiatric inpatient, psychiatric ED, and detoxification use in the last 30 days—to represent the CD of acute service utilization.
The Second Aim: Quality of Care
Safety
Safety is defined as avoiding injuries to patients from the care that is intended to help them.
Brief review and suggested item(s). As noted in “Crossing the Quality Chasm,” the IOM’s seminal document, “the health care environment should be safe for all patients, in all of its processes, all the time.”160 The landmark Harvard Medical Practice Study in 1991 found that adverse events occurred in nearly 4% of all hospital admissions and, among these, over a quarter were due to negligence.161 Other estimates of adverse events range as high as 17%.162 Indeed, a recent article by Makary and Daniel estimated that medical errors may be the third leading cause of death in the United States.163 Unfortunately, research on safety in the mental health field has lagged behind that of physical health,164 with evidence indicating that research in nonhospital settings in mental health care may be particularly scarce.165 In a study of adverse events that occurred in psychiatric inpatient units in the VHA system between 2015 and 2016, Mills and colleagues found that of the 87 root cause analysis reports, suicide attempts were the most frequent, and, among safety events, falls were the most frequently reported, followed by medication events.166 Another report on data collected from psychiatric inpatient units in the VHA revealed that nearly one-fifth of patients experienced a safety event, over half of which were deemed preventable.167 These numbers likely represent an underestimation of the true volume of safety events, as another study by the same research group found that less than 40% of safety events described in patient medical records were documented in the incident reporting system.168 BHD will utilize the total number of complaints and incident reports submitted within a given time frame as its “safety” metric in the CD data set.
Wait Time for Service
The CD is defined as the length of time between the date a patient first contacted BHD for services and the date of their first clinical service.
Brief review and suggested item(s). “Timeliness” was listed among the 6 aims for improvement in “Crossing the Quality Chasm” in 2001, and it remains no less relevant today.160 For example, evidence indicates that access to primary care is inversely related to avoidable hospitalizations.169 One study found that, of patients hospitalized for cardiovascular problems, those who had difficulty accessing routine care post discharge had higher 30-day readmission rates.170 Among VHA patients, longer wait times are associated with more avoidable hospitalizations and higher rates of mortality.171 Longer wait times appear to decrease the likelihood of attending a first appointment for individuals with substance use172,173 and mental health disorders.174 Importantly, longer wait times are associated with lower ratings of the patient experience of care, including perceptions of the quality of and satisfaction with care,175 and may be associated with worse outcomes for individuals in early intervention for psychosis treatment.176 For the purposes of the CD data set, BHD will monitor the length of time between the date a patient first contacted BHD for services and the date of their first clinical service.
Patient Satisfaction
Patient satisfaction is defined as the degree of patients’ satisfaction with the care they have received.
Brief review and suggested item(s). Research has consistently demonstrated the relationship of the patient’s experience of care to a variety of safety and clinical effectiveness measures in medical health care,177 and the therapeutic alliance is one of the most consistent predictors of outcomes in behavioral health, regardless of therapeutic modality.178 Patient satisfaction is a commonly assessed aspect of the patient experience of care. Patient satisfaction scores have been correlated with patient adherence to recommended treatment regimens, care quality, and health outcomes.179 For example, Aiken et al found that patient satisfaction with hospital care was associated with higher ratings of the quality and safety of nursing care in these hospitals.180 Increased satisfaction with inpatient care has been associated with lower 30-day readmission rates for patients with acute myocardial infarction, heart failure, and pneumonia,181 and patients with schizophrenia who reported higher treatment satisfaction also reported better quality of life.182,183 Many satisfaction survey options exist to evaluate this CD, including the Consumer Assessment of Healthcare Providers and Systems and the Client Satisfaction Questionnaire; BHD will utilize an outpatient behavioral health survey from a third-party vendor.
The Third Aim: Cost of Care
Cost of Care
This can be defined as the average cost to provide care per patient per month.
Brief review and suggested item(s). Per capita cost, or rather, the total cost of providing care to a circumscribed population divided by the total population, has been espoused as an important metric for the Triple Aim and the County Health Rankings.6,13 Indeed, between 1960 and 2016, per capita expenditures for health care have grown 70-fold, and the percent of the national gross domestic product accounted for by health expenditures has more than tripled (5.0% to 17.9%).184 One of the more common metrics deployed for assessing health care cost is the per capita per month cost, or rather, the per member per month cost of the predefined population for a given health care system.6,185,186 In fact, some authors have proposed that cost of care can be used not only to track efficient resource allocation, but can also be a proxy for a healthier population as well (ie, as health improves, individuals use fewer and less-expensive services, thus costing the system less).187 To assess this metric, BHD will calculate the total amount billed for patient care provided within BHD’s health network each month (irrespective of funding source) and then divide this sum by the number of members served each month. Although this measure does not account for care received at other health care facilities outside BHD’s provider network, nor does it include all the overhead costs associated with the care provided by BHD itself, it is consistent with the claims-based approach used or recommended by other authors.6,188
The Fourth Aim: Staff Well-being
Staff Quality of Work Life
This can be defined as the quality of the work life of health care clinicians and staff.
Brief review and suggested item(s). Some authors have suggested that the Triple Aim framework is incomplete and have proffered compelling arguments that provider well-being and the quality of work life constitutes a fourth aim.2 Provider burnout is prevalent in both medical2,189 and behavioral health care.190,191 Burnout among health care professionals has been associated with higher rates of perceived medical errors,192 lower patient satisfaction scores,189,193 lower rates of provider empathy,194 more negative attitudes towards patients,195 and poorer staff mental and physical health.191
Burnout is also associated with higher rates of absenteeism, turnover intentions, and turnover.190,191,196,197 However, burnout is not the only predictor of staff turnover; for example, turnover rates are a useful proxy for staff quality of work life for several reasons.198 First, turnover is associated with substantial direct and indirect costs, including lost productivity, increased errors, and lost revenue and recruitment costs, with some turnover cost estimates as high as $17 billion for physicians and $14 billion for nurses annually.199-201 Second, research indicates that staff turnover can have a deleterious impact on implementation of evidence-based interventions.202-205 Finally, consistent with the philosophy of utilizing existing data sources for the CD measures, turnover can be relatively easily extracted from administrative data for operated or contracted programs, and its collection does not place any additional burden on staff. As a large behavioral health system that is both a provider and payer of care, BHD will therefore examine the turnover rates of its internal administrative and clinical staff as well as the turnover of staff in its contracted provider network as its measures for the Staff Quality of Work Life CD.
Clinical Implications
These metrics can be deployed at any level of the organization. Clinicians may use 1 or more of the measures to track the recovery of individual clients, or in aggregate for their entire caseload. Similarly, managers can use these measures to assess the overall effectiveness of the programs for which they are responsible. Executive leaders can evaluate the impact of several programs or the system of care on the health of a subpopulation of clients with a specific condition, or for all their enrolled members. Further, not all measures need be utilized for every dashboard or evaluative effort. The benefit of a comprehensive set of measures lies in their flexibility—1 or more of the measures may be selected depending on the project being implemented or the interests of the stakeholder.
It is important to note that many of the CDs (and their accompanying measures) are aligned to/consistent with social determinants of health.206,207 Evidence suggests that social determinants make substantial contributions to the overall health of individuals and populations and may even account for a greater proportion of variance in health outcomes than health care itself.208 The measures articulated here, therefore, can be used to assess whether and how effectively care provision has addressed these social determinants, as well as the relative impact their resolution may have on other health outcomes (eg, mortality, self-rated health).
These measures can also be used to stratify clients by clinical severity or degree of socioeconomic deprivation. The ability to adjust for risk has many applications in health care, particularly when organizations are attempting to implement value-based purchasing models, such as pay-for-performance contracts or other alternative payment models (population health-based payment models).209 Indeed, once fully implemented, the CDs and measures will enable BHD to more effectively build and execute different conceptual models of “value” (see references 210 and 211 for examples). We will be able to assess the progress our clients have made in care, the cost associated with that degree of improvement, the experience of those clients receiving that care, and the clinical and social variables that may influence the relative degree of improvement (or lack thereof). Thus, the CDs provide a conceptual and data-driven foundation for the Quadruple Aim and any quality initiatives that either catalyze or augment its implementation.
Conclusion
This article provides an overview of the CDs selected by BHD to help organize, focus, advance, and track its quality efforts within the framework of the Quadruple Aim. Although items aligned to each of these CDs are offered, the CDs themselves have been broadly conceptualized such that they can flexibly admit a variety of possible items and/or assessments to operationalize each CD and thus have potential applicability to other behavioral health systems, particularly public systems that have state-mandated and other data reporting requirements.
Bearing in mind the burden that growing data collection requirements can have on the provision of quality care and staff work satisfaction and burnout,10,212 the CDs (and the items selected to represent each) are designed with “strategic parsimony” in mind. Although the CDs are inclusive in that they cover care quality, cost of care, staff quality of life, and general population health, only CDs and items undergirded by a solid evidence base and high value with regards to BHD’s mission and values, as determined by key stakeholders, were selected. Moreover, BHD attempted to make use of existing data collection and reporting mandates when selecting the final pool of items to reduce the measurement burden on staff and clients. Thus, the final set of CDs and items are designed to be comprehensive yet economical.
The CDs are deeply interrelated. Although each CD may be individually viewed as a valuable metric, improvements in any 1 CD will impact the others (eg, increasing care quality should impact population health, increasing staff quality of life should impact the quality of care). Moreover, this idea of interrelatedness acknowledges the need to view health systems and the populations they serve holistically, in that improvement is not simply the degree of change in any given metric (whether individually or collectively), but rather something more entirely. The concepts of value, quality, and health are complex, multidimensional, and dynamic, and the CDs that comprise these concepts should not be considered independently from one another. The CDs (and items) offered in this article are scalable in that they can be used at different levels of an organization depending on the question or stakeholder, and can be used individually or in combination with one another. Moreover, they are adaptable to a variety of risk-adjusted program, population health, and value-based evaluation models. It is hoped that the process articulated here, and the accompanying literature review, may benefit other public or government-run health systems in their own quality journey to operationalize the Quadruple Aim by developing a set of CDs.
Corresponding author: Walter Matthew Drymalski, PhD; walter.drymalski@milwaukeecountywi.gov.
Financial disclosures: None.
From the Milwaukee County Behavioral Health Division, Milwaukee, WI.
Abstract
Objective: Implementation of the Quadruple Aim of health care must begin with a clearly articulated set of concepts, or core domains (CDs), that comprise each aim. These CDs can then be operationalized with existing or new measures. If aligned to the organization’s mission and strategic goals, these CDs have the potential to focus quality improvement activities and reduce measurement burden. This article represents the efforts of a publicly funded behavioral health system to operationalize the Quadruple Aim through the development of CDs.
Methods: Various stakeholders across the organization were consulted on their perceptions of the Quadruple Aim and the CDs they believed should support it. Then, a review of existing literature on core metrics for health care and population health was completed, summarized, and integrated with the stakeholder feedback.
Results: These efforts led to the development and adoption of 15 CDs, with an accompanying literature review and set of recommendations of new and existing measures for each domain.
Conclusions: It is possible to create a comprehensive yet economical set of CDs and attendant measures that can be implemented in a staged, scalable, enterprise manner. It is hoped that the process articulated here, and the accompanying literature review, may be of some benefit to other public or government-run health systems in their own quality improvement journey to operationalize the Quadruple Aim by developing a set of CDs.
Keywords: quality measures; quality improvement; adult behavioral health.
First articulated in 2008, the Triple Aim proposes that health care systems should simultaneously seek to improve the patient’s experience of care, improve the health of populations, and reduce the per capita costs of care for populations.1 More recently, some have argued that health care provider burnout can deleteriously impact the attainment of the Triple Aim and have therefore advocated for an expanded focus to include a fourth Aim, the work life quality of the staff.2 Milwaukee County Behavioral Health Division (BHD), a publicly funded, county-based behavioral health care system in Milwaukee, Wisconsin, recently adopted the Quadruple Aim as the framework by which it will organize its quality activities.
Although originally developed for medical organizations, BHD believes that the Quadruple Aim has strong applicability to county-level behavioral health services. Many county-based behavioral health divisions provide a variety of programs to large segments of the county based on financial eligibility and/or clinical need, and thus often have responsibilities to populations or subpopulations, rather than programs. County health divisions, such as Milwaukee County’s Department of Health and Human Services, are often asked to improve outcomes and client experience of care with neutral growth budgets and less reliance on taxes to fund programs, while simultaneously attracting and retaining competent staff.
Crucial to the effective implementation of the Quadruple Aim, however, is a clear set of population- level measures that help organizations assess their progress.3 Unfortunately, as some authors have noted, evaluation of the Quadruple Aim remains a challenge because the “concepts of (population) health, quality of care and costs are not unanimously defined and measures for these concepts are under construction.”4 Several authors have provided some guidance to assist in the development of a set of measures that effectively capture the elements of the Quadruple Aim.5,6 However, the recent rapid proliferation of quality measures in health care7,8 has been both burdensome and costly for providers.9,10 Any measures adopted should not only be as meaningful as possible with regards to assessing progress towards the basic aims of health care, but should also be parsimonious, to limit measurement burden for providers (and patients) and focus attention on important issues.11,12
To select the most effective, parsimonious set of measures possible, one must first select a set of key foci from among the many possible areas of focus that the core measure is intended to represent. The core domains (CDs), if appropriately consistent with the strategic goals of the organization, provide a mechanism to orient the efforts of the organization at every level and help every staff member of the organization understand how his or her work impacts the progress towards these goals.11 The CDs, therefore, represent the opportunity to affect a greater integration of efforts across the organization toward these shared aims, creating uniformity of purpose at every level. Further, increasing organizational attention on the CDs can also help to reduce measurement burden by streamlining and focusing the data capture processes on the most valuable elements of quality and health, and discarding other extraneous measures (albeit not at the expense of other reporting requirements).11 The remainder of this article describes the CDs selected by BHD to assess its progress toward implementation of the Quadruple Aim and are organized by the Aim which they best represent.
Methods
To effectively implement the Quadruple Aim at BHD, it was necessary to clearly define the subpopulation of focus for our efforts.6 In this case, the subpopulation of interest was defined as all adult clients (18 years and older) who received at least 1 service encounter within a specified time frame from a program that BHD either operated or contracted with to provide care. Services provided by the BHD network include everything from psychiatric inpatient services to mental health and addiction treatment and care management. A limited array of social services, including housing and employment services, is also available to eligible consumers. BHD is the county-run behavioral health provider for individuals who are uninsured or underinsured in Milwaukee County, a demographically diverse, primarily urban county of approximately 950,000 people located in Wisconsin. Approximately 15,000 adults receive services at BHD each year.
This work began by obtaining executive sponsorship for the project, in this case from the Chief Operations Officer and Executive Medical Director of BHD. With their backing, an initial review of the literature produced a preliminary set of possible domains, for which we created working definitions. We then made a list of key stakeholders throughout BHD to whom we needed to present the idea of the Quadruple Aim, and the CDs under each Aim, to secure their support. These stakeholders, which included individuals involved in quality activities, program managers, and executive leadership, were strategically selected based on their relative influence within the organization. A set of brief presentations and handouts explaining the project were then developed and shared at different focus groups with these stakeholders over the course of 6 months. These focus groups served to not only educate the organization about the Quadruple Aim and the CDs but afforded participants an opportunity to provide feedback as well.
During the focus groups, we asked participants which domains they believed were most important (were “core”) when operationalizing the Quadruple Aim. The focus groups provided feedback on the domain definitions, feedback that was used to develop uniform, mutually agreed upon definitions for the CDs that were generalizable to all departments at BHD, regardless of the focus of their services within the continuum of care or the continuum of age. This was a crucial step, as it will eventually enable BHD to aggregate data across departments, even if there are minor discrepancies in the specific items they use to assess the CDs. Comments from the focus groups ultimately resulted in a truncated list of domains and definitions, which, coupled with the literature review, resulted in our final set of CDs.
During our review of the literature, we also looked for items that we felt could best represent each CD in the briefest, most meaningful way. (These items were not meant to supersede existing data, but to provide examples that could be implemented with existing data or recommendations that could be utilized in the absence of existing data.) During this process, we made every effort to make use of existing data-reporting requirements. For example, if we had a state mandate to collect data on housing status, we attempted to leverage this required data point to represent the CD related to housing. In other cases, we attempted to utilize claims or other administrative data to operationalize the CD, such as in the cost-of-care metric articulated in the section the Third Aim. For CDs for which no data existed or were insufficient, we emphasized the use of single- versus multi-item scales. For example, if we found a single-item global assessment of quality of life that had good psychometric properties relative to its longer parent scale, we selected the single item. This approach to item selection allowed us to create the most efficient, parsimonious set of measures possible, which we believed would enable us to comprehensively assess all the CDs with the least amount of burden to staff and clients. These items were presented at stakeholder focus groups, during which we asked for comments on the existing measures in their program or department and gave them the opportunity to comment on the new recommended measures.
A working definition is provided for each CD, followed by a brief review of the research base supporting its inclusion in the final list. The item(s) selected by BHD to represent each CD and the source of the item(s) are then supplied. These items were based either on measures currently collected because of existing reporting mandates or, in the case where extant measures were not available, on new items that demonstrated acceptable psychometric properties in the research literature. The CDs and items are organized by the Aim they best represent. A full list of the CDs by Quadruple Aim and items by CD is provided in the Appendix of the online version of this article. This article concludes with a brief summary of this effort and a discussion of how staff will utilize these items at different levels throughout the BHD system.
The First Aim: Population Health
Health Outcomes
Deaths. This can be defined as the cause of death, as determined by the medical examiner’s office (where appropriate) or as the age at time of death. This CD can also be reported as proportion of deaths considered premature (eg, before age 75) or calculated as total years of potential life lost.
Brief review and suggested item(s). Rates and causes of premature mortality are critical foci for the County Health Rankings & Roadmaps,13 the Institute for Healthcare Improvement’s “Guide to Measuring the Triple Aim,”6 the Centers for Disease Control and Prevention’s “Community Health Assessment for Population Health Improvement,”14 and the Institute of Medicine’s (IOM) “Vital Signs: Core Metrics for Health and Health Care Progress.”11 There is ample evidence that individuals with serious mental illness are at increased risk of early mortality relative to the general population,15-18 and this risk applies to those with substance use disorders as well.15,19-20 BHD tracks all deaths that occur while patients are receiving BHD-funded, community-based services.
Self-Reported Health and Well-Being. This CD asks patients to rate their current physical and mental health status, as well as their overall quality of life.
Brief review and suggested item(s): Self-rated physical health. Premature mortality among individuals with behavioral health issues appears to be due, in large part, to their increased vulnerability to the development of medical comorbidities.16,21 A single self-rating question has demonstrated considerable sensitivity to premature mortality,22,23 with predictive properties up to a decade prior to death.24,25 Further, self-rated health has been associated with subsequent functional decline,26,27 acute service utilization,28,29 and overall health care costs.28
Brief review and suggested item(s): Self-rated mental health. Mental health disorders are associated with significant disability worldwide,30 and comorbid mental health issues can exacerbate the course of other medical problems. For example, depression is associated with increased rates of mortality among individuals with diabetes and31 cardiovascular disease,32 as well as with rates of overall mortality,33 and psychiatric comorbidity is associated with longer lengths of stay and higher costs among patients hospitalized for medical problems.34 Research has found that a single-item measure of self-rated mental health is associated with the presence of psychiatric diagnoses, psychiatric symptoms, and subsequent depression and serious mental illness up to 1 year post-assessment.35,36 There is even evidence that self-rated mental health may be more strongly associated with self-ratings of overall health than self-ratings of physical health.37
Brief review and suggested item(s): Self-rated quality of life. Quality of life is a critical component of the recovery journey and overall health.38 For example, the County Health Rankings & Roadmaps lists “quality of life” as 1 of its key “health outcomes” in its County Health Rankings.13 As some authors have noted, quality of life is often inferred from other “objective” recovery domains, such as employment, health status, or housing status. However, there is evidence that these objective domains are functionally distinct from the inherently subjective construct of quality of life.39 This has led other authors to conclude that these domains should be assessed separately when evaluating outcomes.40 Single-item quality of life assessments have been used in research with individuals with cancer,41 adults with disabilities,42 patients with cystic fibrosis,43 and children with epilepsy.44 For this effort, BHD selected the first global quality of life item from the World Health Organization’s WHOQOL-BREF quality of life assessment,45 an item used in other quality of life research.46
Health Factors
Substance Use. This CD is a composite of 4 different types of substance use, any recent heavy alcohol use (defined as 5 or more drinks in one sitting), any recent drug use, any recent prescription drug abuse, and any recent tobacco use.
Brief review and suggested item(s). As noted, substance use disorders confer an increased risk for early mortality15,19 and are significantly implicated in disease disability burden worldwide.30 Substance use has also been associated with both the onset47,48 and exacerbation of mental health diagnoses.49-51 Further, substance use appears to heighten the risk of violence in the general population52 and especially among those with a co-occurring mental illness.53,54 The County Health Rankings & Roadmaps list alcohol and drug use as key behaviors to address to improve the overall health of a given county,13 and the Centers for Medicare & Medicaid Services (CMS) has endorsed initiation and engagement in addiction treatment as one of the measures in its Adult Core Set.55
Tobacco use continues to be one of the most significant risk factors for early mortality worldwide, and evidence indicates that it is associated with a lower life expectancy of nearly 10 years.56 Unfortunately, rates of tobacco use are even higher among those with severe mental illness relative to the general population, and their rates of smoking cessation are lower.57,58 Tobacco use is a significant risk factor for the high rates of early mortality in individuals with severe mental illness.18 Further, a recent meta-analysis noted that, relative to those who continued to smoke, those who ceased smoking had reduced rates of psychological distress and increased quality of life rankings.59 Reducing tobacco use is one of the key components of the County Health Rankings & Roadmaps, and medication assistance with smoking and tobacco use cessation is also listed in the CMS Adult Core Set.13,55
An accumulating body of evidence suggests that single-item measures can adequately detect alcohol60-62 and drug use disorders.60-64 McNeely and colleagues recently developed and tested a brief 4-item screen, the Tobacco, Alcohol, Prescription medication, and other Substance use (TAPS) tool.65,66 Preliminary evidence suggests that the TAPS tool can effectively identify the presence of problematic and disordered use of tobacco, alcohol, prescription medications, and other drugs.65-67 BHD will use the 4 items from the TAPS tool to represent its substance use CD.
Education/Employment Status. This CD assesses the proportion of BHD members who have completed high school, who are in some type of educational or training program, or who are engaged in some type of employment activity (defined as full-time, part-time, supported, sheltered workshop, or as a full-time homemaker).
Brief review and suggested item(s). Research indicates that unemployment is a risk factor for mortality, even after controlling for other risk factors (eg, age, sex, socioeconomic status [SES], health).68 Unemployment is associated with poorer physical and mental health in the general population and among those with disabilities.69-71 Promisingly, evidence suggests that gaining employment or re-employment is associated with better health,72 even for individuals with substance use disorders73 or moderate74 to severe mental health disorders.75-78 Some authors have even proposed that, above and beyond the associated health benefits, employment may also help to realize a modest cost savings due to reduced service utilization and disability.79,80 Employment is a core tenet in the Substance Abuse and Mental Health Services Administration’s (SAMHSA’s) model of recovery,81 and is also listed as an important recovery goal for individuals with behavioral health issues.82 BHD collects data on employment status on all the patients it serves as part of its state-mandated reporting requirements and will use this item in the CD data set.83
Living Situation. This is measured as the proportion of people who live in permanent, supportive, stable housing; it may also be measured as the percentage of the population living with severe housing problems or who are homeless.
Brief review and suggested item(s). Housing problems can be conceptualized as 3 inter-related components: conditions within the home, neighborhood conditions, and housing affordability, each of which can contribute uniquely to poorer physical and mental health of individuals and families84 and to educational outcomes for children.85,86 Further, individuals who are homeless have a standardized mortality ratio 2 to 5 times that of the general population,87-89 even after controlling for low income status,90 and some evidence suggests these rates are even higher among unsheltered versus sheltered homeless individuals.91 Interventions to improve the condition of housing have demonstrated positive impacts on both physical and mental health,92 and a recent study found that individuals receiving housing assistance in the form of public housing or multifamily housing from the Department of Housing and Urban Development had better self-rated physical and mental health relative to individuals on the wait list for housing assistance.93 Moreover, the provision of housing has been shown to promote reductions in substance use and health service utilization among homeless individuals with substance use disorders.94 Rog and colleagues reviewed the literature on permanent supportive housing for individuals with substance use or mental health disorders who were homeless or disabled, and found that provision of housing led to reduced rates of homelessness, emergency department (ED) and inpatient utilization and increased consumer satisfaction.95
Importantly, evidence suggests that housing is viewed as facilitative of recovery. For example, in a recent qualitative study of homeless individuals with mental illness, housing was seen as a critical first step in recovery, providing a sense of security, increasing feelings of personal independence and autonomy, improving perceptions of health and well-being, and affording a stable environment to rebuild relationships with important others.96 BHD collects data on housing status on all the patients it serves as part of its state-mandated reporting requirements and will utilize this item in the CD data set.83
Social Relationships. This is defined as recent interactions with family, supportive networks (formal and informal), and other recovery services.
Brief review and suggested item(s). Research has long established that social relationships have a significant impact on health, including rates of mortality as well as physical and mental health morbidity.97-99 Social connectedness is another of the pillars supporting an individual’s recovery in SAMHSA’s formulation. Several reviews of the recovery literature38,82 support its importance to the recovery process and inclusion in any assessment of holistic recovery. Social support has been shown to promote recovery among individuals with severe mental illness100-102 and substance use disorders,103 and may mitigate the progression of chronic, life-threatening physical illnesses.97 For the purposes of BHD’s CD data set, the social support question from the “100 Million Healthier Lives Common Questionnaire for Adults” will be used to assess individuals’ perceived adequacy of social support.104
Legal Involvement. Defined as involvement with the civil or criminal justice system, including arrests, imprisonment, or detainment.
Brief review and suggested item(s). Involvement in the criminal justice system is both disruptive for the individual in recovery and expensive to the larger health care system.105 Individuals with substance use106 and severe mental health disorders107 are over-represented in the prison system, and evidence suggests that general physical and mental health declines while individuals are in prison.108,109 Perhaps even more concerning, numerous studies have demonstrated an increase in mortality rates for individuals recently released from prison relative to the general population, particularly during the period immediately following release.108-110 This relationship may even persist long term.111 Further, research indicates that individuals recently released from prison have increased emergency care and hospital utilization.112,113
Incarceration can have significant impacts on the health of the broader community as well. For example, research has found an association between parental incarceration to rates of infant mortality,114 increased behavioral and developmental problems of children of incarcerated parents,115,116 lower rates of child support payments,117 and poorer cardiovascular health of female partners of incarcerated individuals.118 Formerly incarcerated individuals experience slower wage growth as well.119 However, evidence also indicates that engagement in mental health120 and substance abuse121 treatment can reduce the likelihood of subsequent recidivism. As part of its state-mandated reporting, BHD is required to provide information on the criminal justice system involvement of its clients in the previous 6 months, including whether they have been jailed or imprisoned,83 and this will function as its measure of legal involvement in its CD data set.
Socioeconomic Status. Socioeconomic status is the social standing or class of an individual or group. It is often measured as a combination of education, income, and occupation. It can also be defined subjectively, such as one’s evaluation of status relative to similar others or based on an individual’s interpretation of her or his financial needs.
Brief review and suggested item(s). A large body of evidence supports the existence of a robust relationship between lower SES and poor health, including mortality and chronic medical diseases,122-124 as well as mental illness.125-127 Although previous research has examined this relationship using objective indicators of SES (eg, income, education level, occupation), there has recently been an increased interest in exploring the relationship of subjective SES with health indices. Subjective SES is generally assessed by asking individuals to rate themselves relative to others in the society in which they live, in terms of wealth, occupation, educational level, or other indicators of social status. Evidence suggests that subjective SES is associated with objective measures of SES,128-130 and relates to measures of physical and mental health as well, even after controlling for objective SES.130-135 BHD will be using a modified version of the Subject SES Scale,131,135 which is deployed in the “100 Million Healthier Lives Common Questionnaire for Adults.”104
Acute Service Use. This is defined as an admission to a medical or psychiatric emergency room or to a medical or psychiatric hospital or to a detoxification facility.
Brief review and suggested item(s). The CMS Adult Core Set includes “plan all cause readmissions” as a key quality metric.55 Hospital readmissions are also endorsed by the National Committee on Quality Assurance as one of its Health Effectiveness Data and Information Set (HEDIS) measures and by the National Quality Forum. Readmissions, despite their widespread endorsement, are a somewhat controversial measure. Although readmissions are costly to the health care system,136 the relationship between readmissions and quality is inconsistent. For example, Krumholz and colleagues137 found differential rates of readmission for the same patient discharged from 2 different hospitals, which were categorized based on previous readmission rates, suggesting that hospitals do have different levels of performance even when treating the same patient. However, other data indicate that 30-day, all-cause, risk-standardized readmission rates are not associated with hospital 30-day, all-cause, risk-standardized mortality rates.138
Chin and colleague found that readmissions to the hospital that occurred more than 7 days post-discharge were likely due to community- and household-related factors, rather than hospital-related quality factors.139 Transitional care interventions that have successfully reduced 30-day readmission rates are most often multicomponent and focus not just on hospital-based interventions (eg, discharge planning, education) but on follow-up care in the community by formal supports (eg, in-home visits, telephone calls, outpatient clinic appointments, case management) and informal supports (eg, family and friends).140-143 Further, qualitative evidence suggests that some individuals perceive psychiatric hospitalizations to be the result of insufficient resources or unsuccessful attempts to maintain their stability in the community.144 Thus, unplanned or avoidable hospital readmissions may represent a failure of the continuum of care not only from the perspective of the health care system, but from the patient perspective as well.
Frequent or nonurgent use of EDs is conceptually similar to excessive or avoidable inpatient utilization in several ways. For example, overuse of EDs is costly, with some estimates suggesting that it is responsible for up to $38 billion in wasteful spending each year.145 Individuals with frequent ED visits have a greater disease burden146 and an increased risk of mortality compared to nonfrequent users.147 Research suggests that individuals who visit the ED for non-urgent issues do so because of perceived difficulties associated with accessing primary care, and the convenience of EDs relative to primary care.148-150 Moreover, similar to the hospital readmission literature discussed earlier, successful strategies to reduce high rates of ED utilization generally focus on continuum of care interventions, such as provision of case management services.151-155
This evidence implies that frequent ED utilization and hospital readmissions may not be a fundamental issue of quality (or lack thereof) in hospitals or EDs but rather a lack of, or ineffectual, transitional and continuum of care strategies and services. To underscore this point, some authors have argued that a system that is excessively crisis-oriented hinders recovery because it is reactive rather than proactive, predicated on the notion that one’s condition must deteriorate to receive care.156
Although some organizations may have access to claims data or may function as self-contained health systems (eg, the Veterans Health Administration [VHA] ), others may not have access to such data. In the absence of claims data, patient self-report of service utilization has been used as a proxy for actual agency records.157 Although concordance between medical and/or agency records and patient self-report has been variable,157 evidence generally suggests that rates of agreement are higher the shorter the recall time interval.158,159 BHD does not have access to comprehensive claims data and has therefore chosen to use 5 dichotomously scored (yes/no) questions—related to medical inpatient, medical ED, psychiatric inpatient, psychiatric ED, and detoxification use in the last 30 days—to represent the CD of acute service utilization.
The Second Aim: Quality of Care
Safety
Safety is defined as avoiding injuries to patients from the care that is intended to help them.
Brief review and suggested item(s). As noted in “Crossing the Quality Chasm,” the IOM’s seminal document, “the health care environment should be safe for all patients, in all of its processes, all the time.”160 The landmark Harvard Medical Practice Study in 1991 found that adverse events occurred in nearly 4% of all hospital admissions and, among these, over a quarter were due to negligence.161 Other estimates of adverse events range as high as 17%.162 Indeed, a recent article by Makary and Daniel estimated that medical errors may be the third leading cause of death in the United States.163 Unfortunately, research on safety in the mental health field has lagged behind that of physical health,164 with evidence indicating that research in nonhospital settings in mental health care may be particularly scarce.165 In a study of adverse events that occurred in psychiatric inpatient units in the VHA system between 2015 and 2016, Mills and colleagues found that of the 87 root cause analysis reports, suicide attempts were the most frequent, and, among safety events, falls were the most frequently reported, followed by medication events.166 Another report on data collected from psychiatric inpatient units in the VHA revealed that nearly one-fifth of patients experienced a safety event, over half of which were deemed preventable.167 These numbers likely represent an underestimation of the true volume of safety events, as another study by the same research group found that less than 40% of safety events described in patient medical records were documented in the incident reporting system.168 BHD will utilize the total number of complaints and incident reports submitted within a given time frame as its “safety” metric in the CD data set.
Wait Time for Service
The CD is defined as the length of time between the date a patient first contacted BHD for services and the date of their first clinical service.
Brief review and suggested item(s). “Timeliness” was listed among the 6 aims for improvement in “Crossing the Quality Chasm” in 2001, and it remains no less relevant today.160 For example, evidence indicates that access to primary care is inversely related to avoidable hospitalizations.169 One study found that, of patients hospitalized for cardiovascular problems, those who had difficulty accessing routine care post discharge had higher 30-day readmission rates.170 Among VHA patients, longer wait times are associated with more avoidable hospitalizations and higher rates of mortality.171 Longer wait times appear to decrease the likelihood of attending a first appointment for individuals with substance use172,173 and mental health disorders.174 Importantly, longer wait times are associated with lower ratings of the patient experience of care, including perceptions of the quality of and satisfaction with care,175 and may be associated with worse outcomes for individuals in early intervention for psychosis treatment.176 For the purposes of the CD data set, BHD will monitor the length of time between the date a patient first contacted BHD for services and the date of their first clinical service.
Patient Satisfaction
Patient satisfaction is defined as the degree of patients’ satisfaction with the care they have received.
Brief review and suggested item(s). Research has consistently demonstrated the relationship of the patient’s experience of care to a variety of safety and clinical effectiveness measures in medical health care,177 and the therapeutic alliance is one of the most consistent predictors of outcomes in behavioral health, regardless of therapeutic modality.178 Patient satisfaction is a commonly assessed aspect of the patient experience of care. Patient satisfaction scores have been correlated with patient adherence to recommended treatment regimens, care quality, and health outcomes.179 For example, Aiken et al found that patient satisfaction with hospital care was associated with higher ratings of the quality and safety of nursing care in these hospitals.180 Increased satisfaction with inpatient care has been associated with lower 30-day readmission rates for patients with acute myocardial infarction, heart failure, and pneumonia,181 and patients with schizophrenia who reported higher treatment satisfaction also reported better quality of life.182,183 Many satisfaction survey options exist to evaluate this CD, including the Consumer Assessment of Healthcare Providers and Systems and the Client Satisfaction Questionnaire; BHD will utilize an outpatient behavioral health survey from a third-party vendor.
The Third Aim: Cost of Care
Cost of Care
This can be defined as the average cost to provide care per patient per month.
Brief review and suggested item(s). Per capita cost, or rather, the total cost of providing care to a circumscribed population divided by the total population, has been espoused as an important metric for the Triple Aim and the County Health Rankings.6,13 Indeed, between 1960 and 2016, per capita expenditures for health care have grown 70-fold, and the percent of the national gross domestic product accounted for by health expenditures has more than tripled (5.0% to 17.9%).184 One of the more common metrics deployed for assessing health care cost is the per capita per month cost, or rather, the per member per month cost of the predefined population for a given health care system.6,185,186 In fact, some authors have proposed that cost of care can be used not only to track efficient resource allocation, but can also be a proxy for a healthier population as well (ie, as health improves, individuals use fewer and less-expensive services, thus costing the system less).187 To assess this metric, BHD will calculate the total amount billed for patient care provided within BHD’s health network each month (irrespective of funding source) and then divide this sum by the number of members served each month. Although this measure does not account for care received at other health care facilities outside BHD’s provider network, nor does it include all the overhead costs associated with the care provided by BHD itself, it is consistent with the claims-based approach used or recommended by other authors.6,188
The Fourth Aim: Staff Well-being
Staff Quality of Work Life
This can be defined as the quality of the work life of health care clinicians and staff.
Brief review and suggested item(s). Some authors have suggested that the Triple Aim framework is incomplete and have proffered compelling arguments that provider well-being and the quality of work life constitutes a fourth aim.2 Provider burnout is prevalent in both medical2,189 and behavioral health care.190,191 Burnout among health care professionals has been associated with higher rates of perceived medical errors,192 lower patient satisfaction scores,189,193 lower rates of provider empathy,194 more negative attitudes towards patients,195 and poorer staff mental and physical health.191
Burnout is also associated with higher rates of absenteeism, turnover intentions, and turnover.190,191,196,197 However, burnout is not the only predictor of staff turnover; for example, turnover rates are a useful proxy for staff quality of work life for several reasons.198 First, turnover is associated with substantial direct and indirect costs, including lost productivity, increased errors, and lost revenue and recruitment costs, with some turnover cost estimates as high as $17 billion for physicians and $14 billion for nurses annually.199-201 Second, research indicates that staff turnover can have a deleterious impact on implementation of evidence-based interventions.202-205 Finally, consistent with the philosophy of utilizing existing data sources for the CD measures, turnover can be relatively easily extracted from administrative data for operated or contracted programs, and its collection does not place any additional burden on staff. As a large behavioral health system that is both a provider and payer of care, BHD will therefore examine the turnover rates of its internal administrative and clinical staff as well as the turnover of staff in its contracted provider network as its measures for the Staff Quality of Work Life CD.
Clinical Implications
These metrics can be deployed at any level of the organization. Clinicians may use 1 or more of the measures to track the recovery of individual clients, or in aggregate for their entire caseload. Similarly, managers can use these measures to assess the overall effectiveness of the programs for which they are responsible. Executive leaders can evaluate the impact of several programs or the system of care on the health of a subpopulation of clients with a specific condition, or for all their enrolled members. Further, not all measures need be utilized for every dashboard or evaluative effort. The benefit of a comprehensive set of measures lies in their flexibility—1 or more of the measures may be selected depending on the project being implemented or the interests of the stakeholder.
It is important to note that many of the CDs (and their accompanying measures) are aligned to/consistent with social determinants of health.206,207 Evidence suggests that social determinants make substantial contributions to the overall health of individuals and populations and may even account for a greater proportion of variance in health outcomes than health care itself.208 The measures articulated here, therefore, can be used to assess whether and how effectively care provision has addressed these social determinants, as well as the relative impact their resolution may have on other health outcomes (eg, mortality, self-rated health).
These measures can also be used to stratify clients by clinical severity or degree of socioeconomic deprivation. The ability to adjust for risk has many applications in health care, particularly when organizations are attempting to implement value-based purchasing models, such as pay-for-performance contracts or other alternative payment models (population health-based payment models).209 Indeed, once fully implemented, the CDs and measures will enable BHD to more effectively build and execute different conceptual models of “value” (see references 210 and 211 for examples). We will be able to assess the progress our clients have made in care, the cost associated with that degree of improvement, the experience of those clients receiving that care, and the clinical and social variables that may influence the relative degree of improvement (or lack thereof). Thus, the CDs provide a conceptual and data-driven foundation for the Quadruple Aim and any quality initiatives that either catalyze or augment its implementation.
Conclusion
This article provides an overview of the CDs selected by BHD to help organize, focus, advance, and track its quality efforts within the framework of the Quadruple Aim. Although items aligned to each of these CDs are offered, the CDs themselves have been broadly conceptualized such that they can flexibly admit a variety of possible items and/or assessments to operationalize each CD and thus have potential applicability to other behavioral health systems, particularly public systems that have state-mandated and other data reporting requirements.
Bearing in mind the burden that growing data collection requirements can have on the provision of quality care and staff work satisfaction and burnout,10,212 the CDs (and the items selected to represent each) are designed with “strategic parsimony” in mind. Although the CDs are inclusive in that they cover care quality, cost of care, staff quality of life, and general population health, only CDs and items undergirded by a solid evidence base and high value with regards to BHD’s mission and values, as determined by key stakeholders, were selected. Moreover, BHD attempted to make use of existing data collection and reporting mandates when selecting the final pool of items to reduce the measurement burden on staff and clients. Thus, the final set of CDs and items are designed to be comprehensive yet economical.
The CDs are deeply interrelated. Although each CD may be individually viewed as a valuable metric, improvements in any 1 CD will impact the others (eg, increasing care quality should impact population health, increasing staff quality of life should impact the quality of care). Moreover, this idea of interrelatedness acknowledges the need to view health systems and the populations they serve holistically, in that improvement is not simply the degree of change in any given metric (whether individually or collectively), but rather something more entirely. The concepts of value, quality, and health are complex, multidimensional, and dynamic, and the CDs that comprise these concepts should not be considered independently from one another. The CDs (and items) offered in this article are scalable in that they can be used at different levels of an organization depending on the question or stakeholder, and can be used individually or in combination with one another. Moreover, they are adaptable to a variety of risk-adjusted program, population health, and value-based evaluation models. It is hoped that the process articulated here, and the accompanying literature review, may benefit other public or government-run health systems in their own quality journey to operationalize the Quadruple Aim by developing a set of CDs.
Corresponding author: Walter Matthew Drymalski, PhD; walter.drymalski@milwaukeecountywi.gov.
Financial disclosures: None.
1. Berwick DM, Nolan TW, Whittington J. The Triple Aim: Care, health, and cost. Health Aff (Millwood). 2008;27(3):759-769.
2. Bodenheimer T, Sinsky C. From Triple to Quadruple Aim: Care of the patient requires care of the provider. Ann Fam Med. 2014;12(6):573-576.
3. Whittington JW, Nolan K, Lewis N, Torres T. Pursuing the Triple Aim: The first 7 years. Milbank Q. 2015;93(2):263-300.
4. Hendrikx RJP, Drewes HW, Spreeuwenberg M, et al. Which Triple Aim related measures are being used to evaluate population management initiatives? An international comparative analysis. Health Policy. 2016;120(5):471-485.
5. Kassler WJ, Howerton M, Thompson A, et al. Population Health Measurement at Centers for Medicare & Medicaid Services: Bridging the gap between public health and clinical quality. Popul Health Manag. 2017;20(3):173-180.
6. Stiefel MC, Nolan K. A Guide to Measuring the Triple Aim: Population health, experience of care, and per capita cost. IHI Innovation Series white paper. Institute for Healthcare Improvement; 2012.
7. Panzer RJ, Gitomer RS, Greene WH, et al. Increasing demands for quality measurement. JAMA. 2013;310(18):1971-1980.
8. Schuster MA, Onorato SE, Meltzer DO. Measuring the cost of quality measurement: a missing link in quality strategy. JAMA. 2017;318(13):1219-1220.
9. Casalino LP, Gans D, Weber R, et al. US physician practices spend more than $15.4 billion annually to report quality measures. Health Aff (Millwood). 2016;35(3):401-406.
10. Rao SK, Kimball AB, Lehrhoff SR, et al. The impact of administrative burden on academic physicians: results of a hospital-wide physician survey. Acad Med. 2017;92(2):237-243.
11. Institute of Medicine. Vital signs: Core metrics for health and health care progress. National Academies Press; 2015.
12. Meyer GS, Nelson EC, Pryor DB, et al. More quality measures versus measuring what matters: a call for balance and parsimony: Table 1. BMJ Qual Saf. 2012;21(11):964-968.
13. County Health Rankings. Measures & data sources. County Health Rankings & Roadmaps. Accessed January 11, 2021. https://www.countyhealthrankings.org/explore-health-rankings/measures-data-sources
14. U.S. Centers for Disease Control and Prevention. Community Health Assessment for population health improvement: Resource of most frequently recommended health outcomes and determinants. Office of Surveillance, Epidemiology, and Laboratory Services; 2013.
15. Chang C-K, Hayes RD, Perera G, et al. Life expectancy at birth for people with serious mental illness and other major disorders from a secondary mental health care case register in London. PLoS One. 2011;6(5):e19590.
16. De Hert M, Correll CU, Bobes J, et al. Physical illness in patients with severe mental disorders. I. Prevalence, impact of medications and disparities in health care. World Psychiatry. 2011;10(1):52-77.
17. Druss BG, Zhao L, Von Esenwein S, Morrato EH, Marcus SC. Understanding excess mortality in persons with mental illness: 17-year follow up of a nationally representative US survey. Med Care. 2011;49(6):599-604.
18. National Association of State Mental Health Program Directors, (NASMHPD) Medical Directors Council. Morbidity and mortality in people with serious mental illness. National Association of State Mental Health Program Directors, (NASMHPD) Medical Directors Council; 2006.
19. Nordentoft M, Wahlbeck K, Hällgren J, et al. Excess mortality, causes of death and life expectancy in 270,770 patients with recent onset of mental disorders in Denmark, Finland and Sweden. PLoS One. 2013;8(1):e55176.
20. Griswold MG, Fullman N, Hawley C, et al. Alcohol use and burden for 195 countries and territories, 1990–2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet. 2018;392(10152):1015-1035.
21. Walker ER, Druss BG. A public health perspective on mental and medical comorbidity. JAMA. 2016;316(10):1104-1105.
22. DeSalvo KB, Bloser N, Reynolds K, et al. Mortality prediction with a single general self-rated health question: a meta-analysis. J Gen Intern Med. 2005;21(3):267-275.
23. Mavaddat N, Parker RA, Sanderson S, et al. Relationship of self-rated health with fatal and non-fatal outcomes in cardiovascular disease: a systematic review and meta-analysis. PLoS One. 2014;9(7):e103509.
24. Lima-Costa MF, Cesar CC, Chor D, Proietti FA. Self-rated health compared with objectively measured health status as a tool for mortality risk screening in older adults: 10-year follow-up of the Bambui Cohort Study of Aging. Am J Epidemiol. 2012;175(3):228-235.
25. Stenholm S, Pentti J, Kawachi I, et al. Self-rated health in the last 12 years of life compared to matched surviving controls: the Health and Retirement Study. PLoS One. 2014;9(9):e107879.
26. Lee Y. The predictive value of self-assessed general, physical, and mental health on functional decline and mortality in older adults. J Epidemiol Community Health. 2000;54(2):123-129.
27. Tomioka K, Kurumatani N, Hosoi H. Self-rated health predicts decline in instrumental activities of daily living among high-functioning community-dwelling older people. Age Ageing. 2017;46(2):265-270.
28. DeSalvo KB, Jones TM, Peabody J, et al. Health care expenditure prediction with a single item, self-rated health measure. Med Care. 2009;47(4):440-447.
29. Farkas J, Kosnik M, Flezar M, Suskovic S, Lainscak M. Self-rated health predicts acute exacerbations and hospitalizations in patients with COPD. Chest. 2010;138(2):323-330.
30. Whiteford HA, Degenhardt L, Rehm J, et al. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet. 2013;382(9904):1575-1586.
31. Park M, Katon WJ, Wolf FM. Depression and risk of mortality in individuals with diabetes: a meta-analysis and systematic review. Gen Hosp Psychiatry. 2013;35(3):217-225.
32. Hare DL, Toukhsati SR, Johansson P, Jaarsma T. Depression and cardiovascular disease: a clinical review. Eur Heart J. 2014;35(21):1365-1372.
33. Cuijpers P, Schoevers RA. Increased mortality in depressive disorders: a review. Curr Psychiatry Rep. 2004;6(6):430-437.
34. Jansen L, van Schijndel M, van Waarde J, van Busschbach J. Health-economic outcomes in hospital patients with medical-psychiatric comorbidity: a systematic review and meta-analysis. PLoS One. 2018;13(3):e0194029.
35. Ahmad F, Jhajj AK, Stewart DE, et al. Single item measures of self-rated mental health: a scoping review. BMC Health Serv Res. 2014;14:398.
36. McAlpine DD, McCreedy E, Alang S. The meaning and predictive value of self-rated mental health among persons with a mental health problem. J Health Soc Behav. 2018;59(2):200-214.
37. Levinson D, Kaplan G. What does self-rated mental health represent? J Public Health Res. 2014;3(3):287.
38. Leamy M, Bird V, Boutillier CL, et al. Conceptual framework for personal recovery in mental health: systematic review and narrative synthesis. Br J Psychiatry. 2011;199(6):445-452.
39. Smith KW, Avis NE, Assmann SF. Distinguishing between quality of life and health status in quality of life research: a meta-analysis. Qual Life Res. 1999;8(5):447-459.
40. Hamming JF, De Vries J. Measuring quality of life. Br J Surg. 2007;94(8):923-924.
41. Singh JA, Satele D, Pattabasavaiah S, et al. Normative data and clinically significant effect sizes for single-item numerical linear analogue self-assessment (LASA) scales. Health Qual Life Outcomes. 2014;12(1):187.
42. Siebens HC, Tsukerman D, Adkins RH, et al. Correlates of a single-item quality-of-life measure in people aging with disabilities. Am J Phys Med Rehabil. 2015;94(12):1065-1074.
43. Yohannes AM, Dodd M, Morris J, Webb K. Reliability and validity of a single item measure of quality of life scale for adult patients with cystic fibrosis. Health Qual Life Outcomes. 2011;9(1):105.
44. Conway L, Widjaja E, Smith ML. Single-item measure for assessing quality of life in children with drug-resistant epilepsy. Epilepsia Open. 2018;3(1):46-54.
45. Skevington SM, Lofty M, O’Connell KA. The World Health Organization’s WHOQOL-BREF quality of life assessment: psychometric properties and results of the international field trial. Qual Life Res. 2004;13:299-310.
46. Atroszko PA, Baginska P, Mokosinska M, et al. Validity and reliability of single-item self-report measures of general quality of life, general health and sleep quality. In: CER Comparative European Research Conference: Research Track. Vol 2. Sciemcee Publishing; 2015:207-211.
47. Beaulieu S, Saury S, Sareen J, et al.
48. Marconi A, Di Forti M, Lewis CM, et al. Meta-analysis of the association between the level of cannabis use and risk of psychosis. Schizophr Bull. 2016;42(5):1262-1269.
49. Baker AL, Hiles SA, Thornton LK, et al. A systematic review of psychological interventions for excessive alcohol consumption among people with psychotic disorders. Acta Psychiatr Scand. 2012;126(4):243-255.
50. Baker AL, Hides L, Lubman DI. Treatment of cannabis use among people with psychotic or depressive disorders: a systematic review. J Clin Psychiatry. 2010;71(3):247-254.
51. Berenz EC, Coffey SF. Treatment of co-occurring posttraumatic stress disorder and substance use disorders. Curr Psychiatry Rep. 2012;14(5):469-477.
52. Pickard H, Fazel S. Substance abuse as a risk factor for violence in mental illness: some implications for forensic psychiatric practice and clinical ethics. Curr Opin Psychiatry. 2013;26(4):349-354.
53. Fazel S, Gulati G, Linsell L, Geddes JR, Grann M. Schizophrenia and violence: systematic review and meta-analysis. PLoS Med. 2009;6(8):e1000120.
54. Van Dorn R, Volavka J, Johnson N. Mental disorder and violence: Is there a relationship beyond substance use? Soc Psychiatry Psychiatr Epidemiol. 2012;47(3):487-503.
55. Centers for Medicare & Medicaid Services. 2019 Core set of adult health care quality measures for Medicaid (adult core set). Adult health care quality measures. Accessed January 11, 2021. https://www.medicaid.gov/medicaid/quality-of-care/performance-measurement/adult-core-set/index.html.
56. Jha P, Peto R. Global effects of smoking, of quitting, and of taxing tobacco. N Engl J Med. 2014;370(1):60-68.
57. Lê Cook B, Wayne GF, Kafali EN, et al. Trends in smoking among adults with mental illness and association between mental health treatment and smoking cessation. JAMA. 2014;311(2):172-182.
58. Smith PH, Mazure CM, McKee SA. Smoking and mental illness in the US population. Tob Control. 2014;23(0):e147-e153.
59. Taylor G, McNeill A, Girling A, et al. Change in mental health after smoking cessation: systematic review and meta-analysis. BMJ. 2014;348:g1151.
60. McNeely J, Cleland CM, Strauss SM, et al. Validation of self-administered single-item screening questions (SISQs) for unhealthy alcohol and drug use in primary care patients. J Gen Intern Med. 2015;30(12):1757-1764.
61. Saitz R, Cheng DM, Allensworth-Davies D, et al. The ability of single screening questions for unhealthy alcohol and other drug use to identify substance dependence in primary care. J Stud Alcohol Drugs. 2014;75(1):153-157.
62. Smith PC, Schmidt SM, Allensworth-Davies D, Saitz R. Primary care validation of a single-question alcohol screening test. J Gen Intern Med. 2009;24(7):783-788.
63. Dawson DA, Compton WM, Grant BF. Frequency of 5+/4+ drinks as a screener for drug use and drug-use disorders. J Stud Alcohol Drugs. 2010;71(5):751-760.
64. Smith PC, Schmidt SM, Allensworth-Davies D, Saitz R. A single-question screening test for drug use in primary care. Arch Intern Med. 2010;170(13):1155-1160.
65. McNeely J, Strauss SM, Saitz R, et al. A brief patient self-administered substance use screening tool for primary care: two-site validation study of the Substance Use Brief Screen (SUBS). Am J Med. 2015;128(7):784.e9-784.e19.
66. McNeely J, Wu L-T, Subramaniam G, et al. Performance of the Tobacco, Alcohol, Prescription medication, and other Substance use (TAPS) tool for substance use screening in primary care patients. Ann Intern Med. 2016;165(10):690-699.
67. Gryczynski J, McNeely J, Wu L-T, et al. Validation of the TAPS-1: a four-item screening tool to identify unhealthy substance use in primary care. J Gen Intern Med. 2017;32(9):990-996.
68. Roelfs DJ, Shor E, Davidson KW, Schwartz JE. Losing life and livelihood: a systematic review and meta-analysis of unemployment and all-cause mortality. Soc Sci Med. 2011;72(6):840-854.
69. McKee-Ryan F, Song Z, Wanberg CR, Kinicki AJ. Psychological and physical well-being during unemployment: a meta-analytic study. J Appl Psychol. 2005;90(1):53-76.
70. Zhang S, Bhavsar V. Unemployment as a risk factor for mental illness: combining social and psychiatric literature. Adv Appl Sociol. 2013;03(02):131-136.
71. Goodman N. The impact of employment on the health status and health care costs of working-age people with disabilities. The Lead Center. November 2015. Accessed January 11, 2021. http://www.leadcenter.org/system/files/resource/downloadable_version/impact_of_employment_health_status_health_care_costs_0.pdf
72. Hergenrather KC, Zeglin RJ, McGuire-Kuletz M, Rhodes SD. Employment as a social determinant of health: a systematic review of longitudinal studies exploring the relationship between employment status and physical health. Rehabil Res Policy Educ. 2015;29(1):2-26.
73. Walton MT, Hall MT. The effects of employment interventions on addiction treatment outcomes: a review of the literature. J Soc Work Pract Addict. 2016;16(4):358-384.
74. Schuring M, Robroek SJ, Burdorf A. The benefits of paid employment among persons with common mental health problems: evidence for the selection and causation mechanism. Scand J Work Environ Health. 2017;43(6):540-549.
75. Burns T, Catty J, White S, et al. The impact of supported employment and working on clinical and social functioning: results of an international study of individual placement and support. Schizophr Bull. 2009;35(5):949-958.
76. Kilian R, Lauber C, Kalkan R, et al. The relationships between employment, clinical status, and psychiatric hospitalisation in patients with schizophrenia receiving either IPS or a conventional vocational rehabilitation programme. Soc Psychiatry Psychiatr Epidemiol. 2012;47(9):1381-1389.
77. Marwaha S, Johnson S. Schizophrenia and employment. Soc Psychiatry Psychiatr Epidemiol. 2004;39(5):337-349.
78. Mueser KT, Drake RE, Bond GR. Recent advances in supported employment for people with serious mental illness. Curr Opin Psychiatry. 2016;29(3):196-201.
79. Bush PW, Drake RE, Xie H, et al. The long-term impact of employment on mental health service use and costs for persons with severe mental illness. Psychiatr Serv. 2009;60(8):1024-1031.
80. Drake RE, Skinner JS, Bond GR, Goldman HH. Social security and mental illness: reducing disability with supported employment. Health Aff (Millwood). 2009;28(3):761-770.
81. Substance Abuse and Mental Health Services Administration. SAMHSA’s working definition of recovery. Substance Abuse and Mental Health Services Administration. Published 2012. Accessed January 11, 2021. https://store.samhsa.gov/system/files/pep12-recdef.pdf
82. Drake RE, Whitley R. Recovery and severe mental illness: description and analysis. Can J Psychiatry. 2014;59(5):236-242.
83. Wisconsin Department of Health Services. PPS Mental Health Module Handbook. Published 2018. Accessed January 11, 2021. https://www.dhs.wisconsin.gov/publications/p02182.pdf
84. Robert Wood Johnson Foundation. Housing and Health. How does housing affect health? May 1, 2011. Accessed January 11, 2021. https://www.rwjf.org/en/library/research/2011/05/housing-and-health.html
85. Cunningham MK, MacDonald G. Housing as a platform for improving education outcomes among low-income children. Urban Institute. May 2012. Accessed January 11, 2021. https://www.urban.org/sites/default/files/publication/25331/412554-Housing-as-a-Platform-for-Improving-Education-Outcomes-among-Low-Income-Children.PDF
86. Friedman D. Social impact of poor housing. ECOTEC. March 2010. Accessed January 11, 2021. https://southdevonrural.co.uk/userfiles/file/JC-JC13-Social-impact-of-poor-housing.pdf
87. Fazel S, Geddes JR, Kushel M. The health of homeless people in high-income countries: descriptive epidemiology, health consequences, and clinical and policy recommendations. Lancet Lond Engl. 2014;384(9953):1529-1540.
88. Nusselder WJ, Slockers MT, Krol L, et al. Mortality and life expectancy in homeless men and women in Rotterdam: 2001–2010. PLoS One. 2013;8(10):e73979.
89. O’Connell JJ. Premature mortality in homeless populations: a review of the literature. National Health Care for the Homeless Council. Published 2005. Accessed April 23, 2019. http://sbdww.org/wp-content/uploads/2011/04/PrematureMortalityFinal.pdf
90. Hwang SW, Wilkins R, Tjepkema M, et al. Mortality among residents of shelters, rooming houses, and hotels in Canada: 11 year follow-up study. BMJ. 2009;339:b4036.
91. Roncarati JS, Baggett TP, O’Connell JJ, et al. Mortality among unsheltered homeless adults in Boston, Massachusetts, 2000-2009. JAMA Intern Med. 2018;178(9):1242-1248.
92. Thomson H, Thomas S, Sellstrom E, Petticrew M. Housing improvements for health and associated socio-economic outcomes. Cochrane Database Syst Rev. 2013;(2):CD008657.
93. Fenelon A, Mayne P, Simon AE, et al. Housing assistance programs and adult health in the United States. Am J Public Health. 2017;107(4):571-578.
94. Fitzpatrick-Lewis D, Ganann R, Krishnaratne S, et al. Effectiveness of interventions to improve the health and housing status of homeless people: a rapid systematic review. BMC Public Health. 2011;11(1):638.
95. Rog DJ, Marshall T, Dougherty RH, et al. Permanent supportive housing: assessing the evidence. Psychiatr Serv. 2014;65(3):287-294.
96. Kirst M, Zerger S, Wise Harris D, et al. The promise of recovery: narratives of hope among homeless individuals with mental illness participating in a Housing First randomised controlled trial in Toronto, Canada: Table 1. BMJ Open. 2014;4(3):e004379.
97. Cohen S, Janicki-Deverts D. Can we improve our physical health by altering our social networks? Perspect Psychol Sci. 2009;4(4):375-378.
98. House JS, Landis KR, Umberson D. Social relationships and health. Science. 1988;241(4865):540-545.
99. Kawachi I, Berkman LF. Social ties and mental health. J Urban Health. 2001;78(3):458-467.
100. Schön U-K, Denhov A, Topor A. Social relationships as a decisive factor in recovering from severe mental illness. Int J Soc Psychiatry. 2009;55(4):336-347.
101. Soundy A, Stubbs B, Roskell C, et al. Identifying the facilitators and processes which influence recovery in individuals with schizophrenia: a systematic review and thematic synthesis. J Ment Health. 2015;24(2):103-110.
102. Tew J, Ramon S, Slade M, et al. Social factors and recovery from mental health difficulties: a review of the evidence. Br J Soc Work. 2012;42(3):443-460.
103. Moos RH. Theory-based processes that promote the remission of substance use disorders. Clin Psychol Rev. 2007;27(5):537-551.
104. Stiefel MC, Riley CL, Roy B, et al. 100 Million Healthier Lives Measurement System: Progress to date. Institute for Healthcare Improvement; 2016:41. Accessed January 11, 2021. http://www.100mlives.org
105. Swanson JW, Frisman LK, Robertson AG, et al. Costs of criminal justice involvement among persons with serious mental illness in Connecticut. Psychiatr Serv. 2013;64(7):630-637.
106. Fazel S, Bains P, Doll H. Substance abuse and dependence in prisoners: a systematic review. Addiction. 2006;101(2):181-191.
107. Fazel S, Seewald K. Severe mental illness in 33 588 prisoners worldwide: systematic review and meta-regression analysis. Br J Psychiatry. 2012;200(05):364-373.
108. Fazel S, Baillargeon J. The health of prisoners. Lancet. 2011;377(9769):956-965.
109. Wildeman C, Wang EA. Mass incarceration, public health, and widening inequality in the USA. Lancet. 2017;389(10077):1464-1474.
110. Zlodre J, Fazel S. All-cause and external mortality in released prisoners: systematic review and meta-analysis. Am J Public Health. 2012;102(12):e67-e75.
111. Massoglia M, Pridemore WA. Incarceration and health. Annu Rev Sociol. 2015;41:291-310.
112. Kouyoumdjian FG, Cheng SY, Fung K, et al. The health care utilization of people in prison and after prison release: a population-based cohort study in Ontario, Canada. PLoS One. 2018;13(8):e0201592.
113. Winkelman TNA, Genao I, Wildeman C, Wang EA. Emergency department and hospital use among adolescents with justice system involvement. Pediatrics. 2017;140(5):e20171144.
114. Wildeman C. Imprisonment and infant mortality. Soc Prob. 2012;59:228-257.
115. Geller A, Cooper CE, Garfinkel I, et al. Beyond absenteeism: father incarceration and child development. Demography. 2012;49(1):49-76.
116. Turney K. Stress proliferation across generations? Examining the relationship between parental incarceration and childhood health. J Health Soc Behav. 2014;55(3):302-319.
117. Geller A, Garfinkel I, Western B. Paternal incarceration and support for children in fragile families. Demography. 2011;48(1):25-47.
118. Lee H, Wildeman C, Wang EA, et al. A heavy burden: the cardiovascular health consequences of having a family member incarcerated. Am J Public Health. 2014;104(3):421-427.
119. Western B. The impact of incarceration on wage mobility and inequality. Am Sociol Rev. 2002;67(4):526-546.
120. Constantine R, Andel R, Petrila J, et al. Characteristics and experiences of adults with a serious mental illness who were involved in the criminal justice system. Psychiatr Serv. 2010;61(5):451-457.
121. Garnick DW, Horgan CM, Acevedo A, et al. Criminal justice outcomes after engagement in outpatient substance abuse treatment. J Subst Abuse Treat. 2014;46(3):295-305.
122. Adler NE, Ostrove JM. Socioeconomic status and health: what we know and what we don’t. Ann N Y Acad Sci. 1999;896(1):3-15.
123. Luo Y, Waite LJ. The impact of childhood and adult SES on physical, mental, and cognitive well-being in later life. J Gerontol Ser B. 2005;60(2):S93-S101.
124. Mackenbach JP, Stirbu I, Roskam A-JR, et al. Socioeconomic inequalities in health in 22 European countries. N Engl J Med. 2008;358(23):2468-2481.
125. Hudson CG. Socioeconomic status and mental illness: tests of the social causation and selection hypotheses. Am J Orthopsychiatry. 2005;75(1):3-18.
126. McLaughlin KA, Breslau J, Green JG, et al. Childhood socio-economic status and the onset, persistence, and severity of DSM-IV mental disorders in a US national sample. Soc Sci Med. 2011;73(7):1088-1096.
127. Muntaner C. Socioeconomic position and major mental disorders. Epidemiol Rev. 2004;26(1):53-62.
128. Präg P, Mills MC, Wittek R. Subjective socioeconomic status and health in cross-national comparison. Soc Sci Med. 2016;149:84-92.
129. Shaked D, Williams M, Evans MK, Zonderman AB. Indicators of subjective social status: differential associations across race and sex. SSM Popul Health. 2016;2:700-707.
130. Singh-Manoux A, Adler NE, Marmot MG. Subjective social status: its determinants and its association with measures of ill-health in the Whitehall II study. Soc Sci Med. 2003;56(6):1321-1333.
131. Adler NE, Epel ES, Castellazzo G, Ickovics JR. Relationship of subjective and objective social status with psychological and physiological functioning: preliminary data in healthy, white women. Health Psychol. 2000;19(6):586-592.
132. Cundiff JM, Matthews KA. Is subjective social status a unique correlate of physical health? A meta-analysis. Health Psychol. 2017;36(12):1109-1125.
133. Demakakos P, Biddulph JP, de Oliveira C, et al. Subjective social status and mortality: The English Longitudinal Study of Ageing. Eur J Epidemiol. 2018;33(8):729-739.
134. Quon EC, McGrath JJ. Subjective socioeconomic status and adolescent health: a meta-analysis. Health Psychol. 2014;33(5):433-447.
135. Scott KM, Al-Hamzawi AO, Andrade LH, et al. Associations between subjective social status and DSM-IV mental disorders: Results from the World Mental Health Surveys. JAMA Psychiatry. 2014;71(12):1400-1408.
136. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare Fee-for-Service Program. N Engl J Med. 2009;360(14):1418-1428.
137. Krumholz HM, Wang K, Lin Z, et al. Hospital-readmission risk—Isolating hospital effects from patient effects. N Engl J Med. 2017;377(11):1055-1064.
138. Krumholz HM, Lin Z, Keenan PS, et al. Relationship of hospital performance with readmission and mortality rates for patients hospitalized with acute myocardial infarction, heart failure, or pneumonia. JAMA. 2013;309(6):587-593.
139. Chin DL, Bang H, Manickam RN, Romano PS. Rethinking thirty-day hospital readmissions: Shorter intervals might be better indicators of quality of care. Health Aff (Millwood). 2016;35(10):1867-1875.
140. Feltner C, Jones CD, Cené CW, et al. Transitional care interventions to prevent readmissions for persons with heart failure: a systematic review and meta-analysis. Ann Intern Med. 2014;160(11):774-784.
141. Hudon C, Chouinard M-C, Lambert M, et al. Effectiveness of case management interventions for frequent users of healthcare services: a scoping review. BMJ Open. 2016;6(9):e012353.
142. Kripalani S, Theobald CN, Anctil B, Vasilevskis EE. Reducing hospital readmission: current strategies and future directions. Annu Rev Med. 2014;65:471-485.
143. Verhaegh KJ, MacNeil-Vroomen JL, Eslami S, et al. Transitional care interventions prevent hospital readmissions for adults with chronic illnesses. Health Aff (Millwood). 2014;33(9):1531-1539.
144. Duhig M, Gunasekara I, Patterson S. Understanding readmission to psychiatric hospital in Australia from the service users’ perspective: a qualitative study. Health Soc Care Community. 2017;25(1):75-82.
145. New England Healthcare Institute. A matter of urgency: Reducing emergency department overuse. Published 2010. Accessed January 11, 2021. https://www.nehi.net/writable/publication_files/file/nehi_ed_overuse_issue_brief_032610finaledits.pdf
146. Billings J, Raven MC. Dispelling an urban legend: Frequent emergency department users have substantial burden of disease. Health Aff (Millwood). 2013;32(12):2099-2108.
147. Moe J, Kirkland S, Ospina MB, et al. Mortality, admission rates and outpatient use among frequent users of emergency departments: a systematic review. Emerg Med J. 2016;33(3):230-236.
148. Carret MLV, Fassa ACG, Domingues MR. Inappropriate use of emergency services: a systematic review of prevalence and associated factors. Cad Saúde Pública. 2009;25(1):7-28.
149. Durand A-C, Palazzolo S, Tanti-Hardouin N, et al. Nonurgent patients in emergency departments: rational or irresponsible consumers? Perceptions of professionals and patients. BMC Res Notes. 2012;5(1):525.
150. Uscher-Pines L, Pines J, Kellermann A, et al. Deciding to visit the emergency department for non-urgent conditions: a systematic review of the literature. Am J Manag Care. 2013;19(1):47-59.
151. Kumar GS, Klein R. Effectiveness of case management strategies in reducing emergency department visits in frequent user patient populations: a systematic review. J Emerg Med. 2013;44(3):717-729.
152. Moe J, Kirkland SW, Rawe E, et al. Effectiveness of interventions to decrease emergency department visits by adult frequent users: a systematic review. Acad Emerg Med. 2017;24(1):40-52.
153. Raven MC, Kushel M, Ko MJ, et al. The effectiveness of emergency department visit reduction programs: a systematic review. Ann Emerg Med. 2016;68(4):467-483.e15.
154. Soril LJJ, Leggett LE, Lorenzetti DL, et al. Reducing frequent visits to the emergency department: a systematic review of interventions. PLoS One. 2015;10(4):e0123660.
155. Van den Heede K, Van de Voorde C. Interventions to reduce emergency department utilisation: a review of reviews. Health Policy. 2016;120(12):1337-1349.
156. Onken SJ, Dumont JM, Ridgway P, et al. Mental health recovery: What helps and what hinders? October 2002. Accessed January 11, 2021. https://www.nasmhpd.org/sites/default/files//MHSIPReport%281%29.pdf
157. Leggett LE, Khadaroo RG, Holroyd-Leduc J, et al. Measuring resource utilization. Medicine (Baltimore). 2016;95(10):e2759.
158. Bhandari A, Wagner T. Self-reported utilization of health care services: improving measurement and accuracy. Med Care Res Rev. 2006;63(2):217-235.
159. Short ME, Goetzel RZ, Pei X, et al. How accurate are self-reports? An analysis of self-reported healthcare utilization and absence when compared to administrative data. J Occup Environ Med. 2009;51(7):786-796.
160. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. National Academies Press; 2001.
161. Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med. 1991;324(6):370-376.
162. Rafter N, Hickey A, Condell S, et al. Adverse events in healthcare: learning from mistakes. QJM. 2015;108(4):273-277.
163. Makary MA, Daniel M. Medical error—the third leading cause of death in the US. BMJ. 2016;3(353):i2139.
164. D’Lima D, Crawford MJ, Darzi A, Archer S. Patient safety and quality of care in mental health: A world of its own? BJPsych Bull. 2017;41(5):241-243.
165. Maidment ID, Lelliott P, Paton C. Medication errors in mental healthcare: a systematic review. Qual Saf Health Care. 2006;15(6):409-413.
166. Mills PD, Watts BV, Shiner B, Hemphill RR. Adverse events occurring on mental health units. Gen Hosp Psychiatry. 2018;50:63-68.
167. Marcus SC, Hermann RC, Frankel MR, Cullen SW. Safety of psychiatric inpatients at the Veterans Health Administration. Psychiatr Serv. 2017;69(2):204-210.
168. Reilly CA, Cullen SW, Watts BV, et al. How well do incident reporting systems work on inpatient psychiatric units? Jt Comm J Qual Patient Saf. 2019;45:63-69.
169. Rosano A, Loha CA, Falvo R, et al. The relationship between avoidable hospitalization and accessibility to primary care: a systematic review. Eur J Public Health. 2013;23(3):356-360.
170. Dupre ME, Xu H, Granger BB, et al. Access to routine care and risks for 30-day readmission in patients with cardiovascular disease. Am Heart J. 2018;196:9-17.
171. Pizer SD, Prentice JC. What are the consequences of waiting for health care in the veteran population? J Gen Intern Med. 2011;26(S2):676-682.
172. Festinger DS, Lamb RJ, Kirby KC, Marlowe DB. The accelerated intake: a method for increasing initial attendance to outpatient cocaine treatment. J Appl Behav Anal. 1996;29(3):387-389.
173. Festinger DS, Lamb RJ, Marlowe DB, Kirby KC. From telephone to office: intake attendance as a function of appointment delay. Addict Behav. 2002;27(1):131-137.
174. Gallucci G, Swartz W, Hackerman F. Brief reports: Impact of the wait for an initial appointment on the rate of kept appointments at a mental health center. Psychiatr Serv. 2005;56(3):344-346.
175. Bleustein C, Rothschild DB, Valen A, et al. Wait times, patient satisfaction scores, and the perception of care. Am J Manag Care. 2014;20(5):393-400.
176. Reichert A, Jacobs R. The impact of waiting time on patient outcomes: Evidence from early intervention in psychosis services in England. Health Econ. 2018;27(11):1772-1787.
177. Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013;3(1):e001570.
178. Horvath AO, Del Re AC, Flückiger C, Symonds D. Alliance in individual psychotherapy. Psychotherapy. 2011;48(1):9-16.
179. Farley H, Enguidanos ER, Coletti CM, et al. Patient satisfaction surveys and quality of care: an information paper. Ann Emerg Med. 2014;64(4):351-357.
180. Aiken LH, Sermeus W, Van den Heede K, et al. Patient safety, satisfaction, and quality of hospital care: cross sectional surveys of nurses and patients in 12 countries in Europe and the United States. BMJ. 2012;344:e1717.
181. Boulding W, Glickman SW, Manary MP, Schulman KA, Staelin R. Relationship between patient satisfaction with inpatient care and hospital readmission within 30 days. Am J Manag Care. 2011;17(1):41-48.
182. Rohland BM, Langbehn DR, Rohrer JE. Relationship between service effectiveness and satisfaction among persons receiving Medicaid mental health services. Psychiatr Serv. 2000;51(2):248-250.
183. Zendjidjian X-Y, Baumstarck K, Auquier P, et al. Satisfaction of hospitalized psychiatry patients: Why should clinicians care? Patient Prefer Adherence. 2014;8:575-583.
184. Centers for Medicare & Medicaid Services. National health expenditures; aggregate and per capita amounts, annual percent change and percent distribution: Calendar years 1960-2016. National Health Expenditure Data. https://www.cms.gov/research-statistics-data-and-systems/statistics-trends-and-reports/nationalhealthexpenddata/nationalhealthaccountshistorical.html. Published 2018. Accessed August 21, 2018.
185. DuBard CA. Running the numbers. N C Med J. 2016;77(4):297-300.
186. Peikes D, Chen A, Schore J, Brown R. Effects of care coordination on hospitalization, quality of care, and health care expenditures among Medicare beneficiaries: 15 randomized trials. JAMA. 2009;301(6):603-618.
187. Seow H-Y, Sibley LM. Developing a dashboard to help measure and achieve the triple aim: A population-based cohort study. BMC Health Serv Res. 2014;14(1):363.
188. Lee VS, Kawamoto K, Hess R, et al. Implementation of a value-driven outcomes program to identify high variability in clinical costs and outcomes and association with reduced cost and improved quality. JAMA. 2016;316(10):1061.
189. Dyrbye LN, Shanafelt TD, Sinsky CA, et al. Burnout among health care professionals: A call to explore and address this underrecognized threat to safe, high-quality care discussion paper. National Academy of Medicine. July 5, 2017. Accessed January 11, 2021. https://nam.edu/burnout-among-health-care-professionals-a-call-to-explore-and-address-this-underrecognized-threat-to-safe-high-quality-care/
190. Johnson J, Hall LH, Berzins K, et al. Mental healthcare staff well-being and burnout: a narrative review of trends, causes, implications, and recommendations for future interventions. Int J Ment Health Nurs. 2018;27(1):20-32.
191. Morse G, Salyers MP, Rollins AL, et al. Burnout in mental health services: a review of the problem and its remediation. Adm Policy Ment Health. 2012;39(5):341-352.
192. Hall LH, Johnson J, Watt I, et al. Healthcare staff wellbeing, burnout, and patient safety: a systematic review. PLoS One. 2016;11(7):e0159015.
193. Garman AN, Corrigan PW, Morris S. Staff burnout and patient satisfaction: evidence of relationships at the care unit level. J Occup Health Psychol. 2002;7(3):235-241.
194. Wilkinson H, Whittington R, Perry L, Eames C. Examining the relationship between burnout and empathy in healthcare professionals: a systematic review. Burn Res. 2017;6:18-29.
195. Holmqvist R, Jeanneau M. Burnout and psychiatric staff’s feelings towards patients. Psychiatry Res. 2006;145(2-3):207-213.
196. Leiter MP, Maslach C. Nurse turnover: the mediating role of burnout. J Nurs Manag. 2009;17(3):331-339.
197. Zhang Y, Feng X. The relationship between job satisfaction, burnout, and turnover intention among physicians from urban state-owned medical institutions in Hubei, China: a cross-sectional study. BMC Health Serv Res. 2011;11(1):235.
198. Halter M, Boiko O, Pelone F, et al. The determinants and consequences of adult nursing staff turnover: a systematic review of systematic reviews. BMC Health Serv Res. 2017;17(1):824.
199. Hamidi MS, Bohman B, Sandborg C, et al. Estimating institutional physician turnover attributable to self-reported burnout and associated financial burden: a case study. BMC Health Serv Res. 2018;18(1):851.
200. National Taskforce for Humanity in Healthcare. The business case for humanity in healthcare position paper. April 2018. Accessed January 11, 2021. https://www.vocera.com/public/pdf/NTHBusinessCase_final003.pdf
201. Waldman JD, Kelly F, Arora S, Smith HL. The shocking cost of turnover in health care. Health Care Manage Rev. 2004;29(1):2-7.
202. Brunette MF, Asher D, Whitley R, et al. Implementation of integrated dual disorders treatment: a qualitative analysis of facilitators and barriers. Psychiatr Serv. 2008;59(9):989-995.
203. Mancini AD, Moser LL, Whitley R, et al. Assertive community treatment: facilitators and barriers to implementation in routine mental health settings. Psychiatr Serv. 2009;60(2):189-195.
204. Rollins AL, Salyers MP, Tsai J, Lydick JM. Staff turnover in statewide implementation of ACT: Relationship with ACT fidelity and other team characteristics. Adm Policy Ment Health. 2010;37(5):417-426.
205. Woltmann EM, Whitley R, McHugo GJ, et al. The role of staff turnover in the implementation of evidence-based practices in mental health care. Psychiatr Serv. 2008;59(7):732-737.
206. Alegría M, NeMoyer A, Falgàs Bagué I, et al. Social determinants of mental health: Where we are and where we need to go. Curr Psychiatry Rep. 2018;20(11):95.
207. Daniel H, Bornstein SS, Kane GC. Addressing social determinants to improve patient care and promote health equity: an American College of Physicians position paper. Ann Intern Med. 2018;168(8):577-578.
208. Park H, Roubal AM, Jovaag A, Gennuso KP, Catlin BB. Relative contributions of a set of health factors to selected health outcomes. Am J Prev Med. 2015;49(6):961-969.
209. Ash AS, Mick EO, Ellis RP, et al. Social determinants of health in managed care payment formulas. JAMA Intern Med. 2017;177(10):1424.
210. de Beurs E, Warmerdam EH, Oudejans SCC, et al. Treatment outcome, duration, and costs: a comparison of performance indicators using data from eight mental health care providers in the Netherlands. Adm Policy Ment Health. 2018;45(2):212-223.
211. Dunbar-Rees R. Paying for what matters most: the future of outcomes-based payments in healthcare. Future Healthc J. 2018;5(2):98-102.
212. Woolhandler S, Himmelstein DU. Administrative work consumes one-sixth of U.S. physicians’ working hours and lowers their career satisfaction. Int J Health Serv. 2014;44(4):635-642.
1. Berwick DM, Nolan TW, Whittington J. The Triple Aim: Care, health, and cost. Health Aff (Millwood). 2008;27(3):759-769.
2. Bodenheimer T, Sinsky C. From Triple to Quadruple Aim: Care of the patient requires care of the provider. Ann Fam Med. 2014;12(6):573-576.
3. Whittington JW, Nolan K, Lewis N, Torres T. Pursuing the Triple Aim: The first 7 years. Milbank Q. 2015;93(2):263-300.
4. Hendrikx RJP, Drewes HW, Spreeuwenberg M, et al. Which Triple Aim related measures are being used to evaluate population management initiatives? An international comparative analysis. Health Policy. 2016;120(5):471-485.
5. Kassler WJ, Howerton M, Thompson A, et al. Population Health Measurement at Centers for Medicare & Medicaid Services: Bridging the gap between public health and clinical quality. Popul Health Manag. 2017;20(3):173-180.
6. Stiefel MC, Nolan K. A Guide to Measuring the Triple Aim: Population health, experience of care, and per capita cost. IHI Innovation Series white paper. Institute for Healthcare Improvement; 2012.
7. Panzer RJ, Gitomer RS, Greene WH, et al. Increasing demands for quality measurement. JAMA. 2013;310(18):1971-1980.
8. Schuster MA, Onorato SE, Meltzer DO. Measuring the cost of quality measurement: a missing link in quality strategy. JAMA. 2017;318(13):1219-1220.
9. Casalino LP, Gans D, Weber R, et al. US physician practices spend more than $15.4 billion annually to report quality measures. Health Aff (Millwood). 2016;35(3):401-406.
10. Rao SK, Kimball AB, Lehrhoff SR, et al. The impact of administrative burden on academic physicians: results of a hospital-wide physician survey. Acad Med. 2017;92(2):237-243.
11. Institute of Medicine. Vital signs: Core metrics for health and health care progress. National Academies Press; 2015.
12. Meyer GS, Nelson EC, Pryor DB, et al. More quality measures versus measuring what matters: a call for balance and parsimony: Table 1. BMJ Qual Saf. 2012;21(11):964-968.
13. County Health Rankings. Measures & data sources. County Health Rankings & Roadmaps. Accessed January 11, 2021. https://www.countyhealthrankings.org/explore-health-rankings/measures-data-sources
14. U.S. Centers for Disease Control and Prevention. Community Health Assessment for population health improvement: Resource of most frequently recommended health outcomes and determinants. Office of Surveillance, Epidemiology, and Laboratory Services; 2013.
15. Chang C-K, Hayes RD, Perera G, et al. Life expectancy at birth for people with serious mental illness and other major disorders from a secondary mental health care case register in London. PLoS One. 2011;6(5):e19590.
16. De Hert M, Correll CU, Bobes J, et al. Physical illness in patients with severe mental disorders. I. Prevalence, impact of medications and disparities in health care. World Psychiatry. 2011;10(1):52-77.
17. Druss BG, Zhao L, Von Esenwein S, Morrato EH, Marcus SC. Understanding excess mortality in persons with mental illness: 17-year follow up of a nationally representative US survey. Med Care. 2011;49(6):599-604.
18. National Association of State Mental Health Program Directors, (NASMHPD) Medical Directors Council. Morbidity and mortality in people with serious mental illness. National Association of State Mental Health Program Directors, (NASMHPD) Medical Directors Council; 2006.
19. Nordentoft M, Wahlbeck K, Hällgren J, et al. Excess mortality, causes of death and life expectancy in 270,770 patients with recent onset of mental disorders in Denmark, Finland and Sweden. PLoS One. 2013;8(1):e55176.
20. Griswold MG, Fullman N, Hawley C, et al. Alcohol use and burden for 195 countries and territories, 1990–2016: a systematic analysis for the Global Burden of Disease Study 2016. Lancet. 2018;392(10152):1015-1035.
21. Walker ER, Druss BG. A public health perspective on mental and medical comorbidity. JAMA. 2016;316(10):1104-1105.
22. DeSalvo KB, Bloser N, Reynolds K, et al. Mortality prediction with a single general self-rated health question: a meta-analysis. J Gen Intern Med. 2005;21(3):267-275.
23. Mavaddat N, Parker RA, Sanderson S, et al. Relationship of self-rated health with fatal and non-fatal outcomes in cardiovascular disease: a systematic review and meta-analysis. PLoS One. 2014;9(7):e103509.
24. Lima-Costa MF, Cesar CC, Chor D, Proietti FA. Self-rated health compared with objectively measured health status as a tool for mortality risk screening in older adults: 10-year follow-up of the Bambui Cohort Study of Aging. Am J Epidemiol. 2012;175(3):228-235.
25. Stenholm S, Pentti J, Kawachi I, et al. Self-rated health in the last 12 years of life compared to matched surviving controls: the Health and Retirement Study. PLoS One. 2014;9(9):e107879.
26. Lee Y. The predictive value of self-assessed general, physical, and mental health on functional decline and mortality in older adults. J Epidemiol Community Health. 2000;54(2):123-129.
27. Tomioka K, Kurumatani N, Hosoi H. Self-rated health predicts decline in instrumental activities of daily living among high-functioning community-dwelling older people. Age Ageing. 2017;46(2):265-270.
28. DeSalvo KB, Jones TM, Peabody J, et al. Health care expenditure prediction with a single item, self-rated health measure. Med Care. 2009;47(4):440-447.
29. Farkas J, Kosnik M, Flezar M, Suskovic S, Lainscak M. Self-rated health predicts acute exacerbations and hospitalizations in patients with COPD. Chest. 2010;138(2):323-330.
30. Whiteford HA, Degenhardt L, Rehm J, et al. Global burden of disease attributable to mental and substance use disorders: findings from the Global Burden of Disease Study 2010. Lancet. 2013;382(9904):1575-1586.
31. Park M, Katon WJ, Wolf FM. Depression and risk of mortality in individuals with diabetes: a meta-analysis and systematic review. Gen Hosp Psychiatry. 2013;35(3):217-225.
32. Hare DL, Toukhsati SR, Johansson P, Jaarsma T. Depression and cardiovascular disease: a clinical review. Eur Heart J. 2014;35(21):1365-1372.
33. Cuijpers P, Schoevers RA. Increased mortality in depressive disorders: a review. Curr Psychiatry Rep. 2004;6(6):430-437.
34. Jansen L, van Schijndel M, van Waarde J, van Busschbach J. Health-economic outcomes in hospital patients with medical-psychiatric comorbidity: a systematic review and meta-analysis. PLoS One. 2018;13(3):e0194029.
35. Ahmad F, Jhajj AK, Stewart DE, et al. Single item measures of self-rated mental health: a scoping review. BMC Health Serv Res. 2014;14:398.
36. McAlpine DD, McCreedy E, Alang S. The meaning and predictive value of self-rated mental health among persons with a mental health problem. J Health Soc Behav. 2018;59(2):200-214.
37. Levinson D, Kaplan G. What does self-rated mental health represent? J Public Health Res. 2014;3(3):287.
38. Leamy M, Bird V, Boutillier CL, et al. Conceptual framework for personal recovery in mental health: systematic review and narrative synthesis. Br J Psychiatry. 2011;199(6):445-452.
39. Smith KW, Avis NE, Assmann SF. Distinguishing between quality of life and health status in quality of life research: a meta-analysis. Qual Life Res. 1999;8(5):447-459.
40. Hamming JF, De Vries J. Measuring quality of life. Br J Surg. 2007;94(8):923-924.
41. Singh JA, Satele D, Pattabasavaiah S, et al. Normative data and clinically significant effect sizes for single-item numerical linear analogue self-assessment (LASA) scales. Health Qual Life Outcomes. 2014;12(1):187.
42. Siebens HC, Tsukerman D, Adkins RH, et al. Correlates of a single-item quality-of-life measure in people aging with disabilities. Am J Phys Med Rehabil. 2015;94(12):1065-1074.
43. Yohannes AM, Dodd M, Morris J, Webb K. Reliability and validity of a single item measure of quality of life scale for adult patients with cystic fibrosis. Health Qual Life Outcomes. 2011;9(1):105.
44. Conway L, Widjaja E, Smith ML. Single-item measure for assessing quality of life in children with drug-resistant epilepsy. Epilepsia Open. 2018;3(1):46-54.
45. Skevington SM, Lofty M, O’Connell KA. The World Health Organization’s WHOQOL-BREF quality of life assessment: psychometric properties and results of the international field trial. Qual Life Res. 2004;13:299-310.
46. Atroszko PA, Baginska P, Mokosinska M, et al. Validity and reliability of single-item self-report measures of general quality of life, general health and sleep quality. In: CER Comparative European Research Conference: Research Track. Vol 2. Sciemcee Publishing; 2015:207-211.
47. Beaulieu S, Saury S, Sareen J, et al.
48. Marconi A, Di Forti M, Lewis CM, et al. Meta-analysis of the association between the level of cannabis use and risk of psychosis. Schizophr Bull. 2016;42(5):1262-1269.
49. Baker AL, Hiles SA, Thornton LK, et al. A systematic review of psychological interventions for excessive alcohol consumption among people with psychotic disorders. Acta Psychiatr Scand. 2012;126(4):243-255.
50. Baker AL, Hides L, Lubman DI. Treatment of cannabis use among people with psychotic or depressive disorders: a systematic review. J Clin Psychiatry. 2010;71(3):247-254.
51. Berenz EC, Coffey SF. Treatment of co-occurring posttraumatic stress disorder and substance use disorders. Curr Psychiatry Rep. 2012;14(5):469-477.
52. Pickard H, Fazel S. Substance abuse as a risk factor for violence in mental illness: some implications for forensic psychiatric practice and clinical ethics. Curr Opin Psychiatry. 2013;26(4):349-354.
53. Fazel S, Gulati G, Linsell L, Geddes JR, Grann M. Schizophrenia and violence: systematic review and meta-analysis. PLoS Med. 2009;6(8):e1000120.
54. Van Dorn R, Volavka J, Johnson N. Mental disorder and violence: Is there a relationship beyond substance use? Soc Psychiatry Psychiatr Epidemiol. 2012;47(3):487-503.
55. Centers for Medicare & Medicaid Services. 2019 Core set of adult health care quality measures for Medicaid (adult core set). Adult health care quality measures. Accessed January 11, 2021. https://www.medicaid.gov/medicaid/quality-of-care/performance-measurement/adult-core-set/index.html.
56. Jha P, Peto R. Global effects of smoking, of quitting, and of taxing tobacco. N Engl J Med. 2014;370(1):60-68.
57. Lê Cook B, Wayne GF, Kafali EN, et al. Trends in smoking among adults with mental illness and association between mental health treatment and smoking cessation. JAMA. 2014;311(2):172-182.
58. Smith PH, Mazure CM, McKee SA. Smoking and mental illness in the US population. Tob Control. 2014;23(0):e147-e153.
59. Taylor G, McNeill A, Girling A, et al. Change in mental health after smoking cessation: systematic review and meta-analysis. BMJ. 2014;348:g1151.
60. McNeely J, Cleland CM, Strauss SM, et al. Validation of self-administered single-item screening questions (SISQs) for unhealthy alcohol and drug use in primary care patients. J Gen Intern Med. 2015;30(12):1757-1764.
61. Saitz R, Cheng DM, Allensworth-Davies D, et al. The ability of single screening questions for unhealthy alcohol and other drug use to identify substance dependence in primary care. J Stud Alcohol Drugs. 2014;75(1):153-157.
62. Smith PC, Schmidt SM, Allensworth-Davies D, Saitz R. Primary care validation of a single-question alcohol screening test. J Gen Intern Med. 2009;24(7):783-788.
63. Dawson DA, Compton WM, Grant BF. Frequency of 5+/4+ drinks as a screener for drug use and drug-use disorders. J Stud Alcohol Drugs. 2010;71(5):751-760.
64. Smith PC, Schmidt SM, Allensworth-Davies D, Saitz R. A single-question screening test for drug use in primary care. Arch Intern Med. 2010;170(13):1155-1160.
65. McNeely J, Strauss SM, Saitz R, et al. A brief patient self-administered substance use screening tool for primary care: two-site validation study of the Substance Use Brief Screen (SUBS). Am J Med. 2015;128(7):784.e9-784.e19.
66. McNeely J, Wu L-T, Subramaniam G, et al. Performance of the Tobacco, Alcohol, Prescription medication, and other Substance use (TAPS) tool for substance use screening in primary care patients. Ann Intern Med. 2016;165(10):690-699.
67. Gryczynski J, McNeely J, Wu L-T, et al. Validation of the TAPS-1: a four-item screening tool to identify unhealthy substance use in primary care. J Gen Intern Med. 2017;32(9):990-996.
68. Roelfs DJ, Shor E, Davidson KW, Schwartz JE. Losing life and livelihood: a systematic review and meta-analysis of unemployment and all-cause mortality. Soc Sci Med. 2011;72(6):840-854.
69. McKee-Ryan F, Song Z, Wanberg CR, Kinicki AJ. Psychological and physical well-being during unemployment: a meta-analytic study. J Appl Psychol. 2005;90(1):53-76.
70. Zhang S, Bhavsar V. Unemployment as a risk factor for mental illness: combining social and psychiatric literature. Adv Appl Sociol. 2013;03(02):131-136.
71. Goodman N. The impact of employment on the health status and health care costs of working-age people with disabilities. The Lead Center. November 2015. Accessed January 11, 2021. http://www.leadcenter.org/system/files/resource/downloadable_version/impact_of_employment_health_status_health_care_costs_0.pdf
72. Hergenrather KC, Zeglin RJ, McGuire-Kuletz M, Rhodes SD. Employment as a social determinant of health: a systematic review of longitudinal studies exploring the relationship between employment status and physical health. Rehabil Res Policy Educ. 2015;29(1):2-26.
73. Walton MT, Hall MT. The effects of employment interventions on addiction treatment outcomes: a review of the literature. J Soc Work Pract Addict. 2016;16(4):358-384.
74. Schuring M, Robroek SJ, Burdorf A. The benefits of paid employment among persons with common mental health problems: evidence for the selection and causation mechanism. Scand J Work Environ Health. 2017;43(6):540-549.
75. Burns T, Catty J, White S, et al. The impact of supported employment and working on clinical and social functioning: results of an international study of individual placement and support. Schizophr Bull. 2009;35(5):949-958.
76. Kilian R, Lauber C, Kalkan R, et al. The relationships between employment, clinical status, and psychiatric hospitalisation in patients with schizophrenia receiving either IPS or a conventional vocational rehabilitation programme. Soc Psychiatry Psychiatr Epidemiol. 2012;47(9):1381-1389.
77. Marwaha S, Johnson S. Schizophrenia and employment. Soc Psychiatry Psychiatr Epidemiol. 2004;39(5):337-349.
78. Mueser KT, Drake RE, Bond GR. Recent advances in supported employment for people with serious mental illness. Curr Opin Psychiatry. 2016;29(3):196-201.
79. Bush PW, Drake RE, Xie H, et al. The long-term impact of employment on mental health service use and costs for persons with severe mental illness. Psychiatr Serv. 2009;60(8):1024-1031.
80. Drake RE, Skinner JS, Bond GR, Goldman HH. Social security and mental illness: reducing disability with supported employment. Health Aff (Millwood). 2009;28(3):761-770.
81. Substance Abuse and Mental Health Services Administration. SAMHSA’s working definition of recovery. Substance Abuse and Mental Health Services Administration. Published 2012. Accessed January 11, 2021. https://store.samhsa.gov/system/files/pep12-recdef.pdf
82. Drake RE, Whitley R. Recovery and severe mental illness: description and analysis. Can J Psychiatry. 2014;59(5):236-242.
83. Wisconsin Department of Health Services. PPS Mental Health Module Handbook. Published 2018. Accessed January 11, 2021. https://www.dhs.wisconsin.gov/publications/p02182.pdf
84. Robert Wood Johnson Foundation. Housing and Health. How does housing affect health? May 1, 2011. Accessed January 11, 2021. https://www.rwjf.org/en/library/research/2011/05/housing-and-health.html
85. Cunningham MK, MacDonald G. Housing as a platform for improving education outcomes among low-income children. Urban Institute. May 2012. Accessed January 11, 2021. https://www.urban.org/sites/default/files/publication/25331/412554-Housing-as-a-Platform-for-Improving-Education-Outcomes-among-Low-Income-Children.PDF
86. Friedman D. Social impact of poor housing. ECOTEC. March 2010. Accessed January 11, 2021. https://southdevonrural.co.uk/userfiles/file/JC-JC13-Social-impact-of-poor-housing.pdf
87. Fazel S, Geddes JR, Kushel M. The health of homeless people in high-income countries: descriptive epidemiology, health consequences, and clinical and policy recommendations. Lancet Lond Engl. 2014;384(9953):1529-1540.
88. Nusselder WJ, Slockers MT, Krol L, et al. Mortality and life expectancy in homeless men and women in Rotterdam: 2001–2010. PLoS One. 2013;8(10):e73979.
89. O’Connell JJ. Premature mortality in homeless populations: a review of the literature. National Health Care for the Homeless Council. Published 2005. Accessed April 23, 2019. http://sbdww.org/wp-content/uploads/2011/04/PrematureMortalityFinal.pdf
90. Hwang SW, Wilkins R, Tjepkema M, et al. Mortality among residents of shelters, rooming houses, and hotels in Canada: 11 year follow-up study. BMJ. 2009;339:b4036.
91. Roncarati JS, Baggett TP, O’Connell JJ, et al. Mortality among unsheltered homeless adults in Boston, Massachusetts, 2000-2009. JAMA Intern Med. 2018;178(9):1242-1248.
92. Thomson H, Thomas S, Sellstrom E, Petticrew M. Housing improvements for health and associated socio-economic outcomes. Cochrane Database Syst Rev. 2013;(2):CD008657.
93. Fenelon A, Mayne P, Simon AE, et al. Housing assistance programs and adult health in the United States. Am J Public Health. 2017;107(4):571-578.
94. Fitzpatrick-Lewis D, Ganann R, Krishnaratne S, et al. Effectiveness of interventions to improve the health and housing status of homeless people: a rapid systematic review. BMC Public Health. 2011;11(1):638.
95. Rog DJ, Marshall T, Dougherty RH, et al. Permanent supportive housing: assessing the evidence. Psychiatr Serv. 2014;65(3):287-294.
96. Kirst M, Zerger S, Wise Harris D, et al. The promise of recovery: narratives of hope among homeless individuals with mental illness participating in a Housing First randomised controlled trial in Toronto, Canada: Table 1. BMJ Open. 2014;4(3):e004379.
97. Cohen S, Janicki-Deverts D. Can we improve our physical health by altering our social networks? Perspect Psychol Sci. 2009;4(4):375-378.
98. House JS, Landis KR, Umberson D. Social relationships and health. Science. 1988;241(4865):540-545.
99. Kawachi I, Berkman LF. Social ties and mental health. J Urban Health. 2001;78(3):458-467.
100. Schön U-K, Denhov A, Topor A. Social relationships as a decisive factor in recovering from severe mental illness. Int J Soc Psychiatry. 2009;55(4):336-347.
101. Soundy A, Stubbs B, Roskell C, et al. Identifying the facilitators and processes which influence recovery in individuals with schizophrenia: a systematic review and thematic synthesis. J Ment Health. 2015;24(2):103-110.
102. Tew J, Ramon S, Slade M, et al. Social factors and recovery from mental health difficulties: a review of the evidence. Br J Soc Work. 2012;42(3):443-460.
103. Moos RH. Theory-based processes that promote the remission of substance use disorders. Clin Psychol Rev. 2007;27(5):537-551.
104. Stiefel MC, Riley CL, Roy B, et al. 100 Million Healthier Lives Measurement System: Progress to date. Institute for Healthcare Improvement; 2016:41. Accessed January 11, 2021. http://www.100mlives.org
105. Swanson JW, Frisman LK, Robertson AG, et al. Costs of criminal justice involvement among persons with serious mental illness in Connecticut. Psychiatr Serv. 2013;64(7):630-637.
106. Fazel S, Bains P, Doll H. Substance abuse and dependence in prisoners: a systematic review. Addiction. 2006;101(2):181-191.
107. Fazel S, Seewald K. Severe mental illness in 33 588 prisoners worldwide: systematic review and meta-regression analysis. Br J Psychiatry. 2012;200(05):364-373.
108. Fazel S, Baillargeon J. The health of prisoners. Lancet. 2011;377(9769):956-965.
109. Wildeman C, Wang EA. Mass incarceration, public health, and widening inequality in the USA. Lancet. 2017;389(10077):1464-1474.
110. Zlodre J, Fazel S. All-cause and external mortality in released prisoners: systematic review and meta-analysis. Am J Public Health. 2012;102(12):e67-e75.
111. Massoglia M, Pridemore WA. Incarceration and health. Annu Rev Sociol. 2015;41:291-310.
112. Kouyoumdjian FG, Cheng SY, Fung K, et al. The health care utilization of people in prison and after prison release: a population-based cohort study in Ontario, Canada. PLoS One. 2018;13(8):e0201592.
113. Winkelman TNA, Genao I, Wildeman C, Wang EA. Emergency department and hospital use among adolescents with justice system involvement. Pediatrics. 2017;140(5):e20171144.
114. Wildeman C. Imprisonment and infant mortality. Soc Prob. 2012;59:228-257.
115. Geller A, Cooper CE, Garfinkel I, et al. Beyond absenteeism: father incarceration and child development. Demography. 2012;49(1):49-76.
116. Turney K. Stress proliferation across generations? Examining the relationship between parental incarceration and childhood health. J Health Soc Behav. 2014;55(3):302-319.
117. Geller A, Garfinkel I, Western B. Paternal incarceration and support for children in fragile families. Demography. 2011;48(1):25-47.
118. Lee H, Wildeman C, Wang EA, et al. A heavy burden: the cardiovascular health consequences of having a family member incarcerated. Am J Public Health. 2014;104(3):421-427.
119. Western B. The impact of incarceration on wage mobility and inequality. Am Sociol Rev. 2002;67(4):526-546.
120. Constantine R, Andel R, Petrila J, et al. Characteristics and experiences of adults with a serious mental illness who were involved in the criminal justice system. Psychiatr Serv. 2010;61(5):451-457.
121. Garnick DW, Horgan CM, Acevedo A, et al. Criminal justice outcomes after engagement in outpatient substance abuse treatment. J Subst Abuse Treat. 2014;46(3):295-305.
122. Adler NE, Ostrove JM. Socioeconomic status and health: what we know and what we don’t. Ann N Y Acad Sci. 1999;896(1):3-15.
123. Luo Y, Waite LJ. The impact of childhood and adult SES on physical, mental, and cognitive well-being in later life. J Gerontol Ser B. 2005;60(2):S93-S101.
124. Mackenbach JP, Stirbu I, Roskam A-JR, et al. Socioeconomic inequalities in health in 22 European countries. N Engl J Med. 2008;358(23):2468-2481.
125. Hudson CG. Socioeconomic status and mental illness: tests of the social causation and selection hypotheses. Am J Orthopsychiatry. 2005;75(1):3-18.
126. McLaughlin KA, Breslau J, Green JG, et al. Childhood socio-economic status and the onset, persistence, and severity of DSM-IV mental disorders in a US national sample. Soc Sci Med. 2011;73(7):1088-1096.
127. Muntaner C. Socioeconomic position and major mental disorders. Epidemiol Rev. 2004;26(1):53-62.
128. Präg P, Mills MC, Wittek R. Subjective socioeconomic status and health in cross-national comparison. Soc Sci Med. 2016;149:84-92.
129. Shaked D, Williams M, Evans MK, Zonderman AB. Indicators of subjective social status: differential associations across race and sex. SSM Popul Health. 2016;2:700-707.
130. Singh-Manoux A, Adler NE, Marmot MG. Subjective social status: its determinants and its association with measures of ill-health in the Whitehall II study. Soc Sci Med. 2003;56(6):1321-1333.
131. Adler NE, Epel ES, Castellazzo G, Ickovics JR. Relationship of subjective and objective social status with psychological and physiological functioning: preliminary data in healthy, white women. Health Psychol. 2000;19(6):586-592.
132. Cundiff JM, Matthews KA. Is subjective social status a unique correlate of physical health? A meta-analysis. Health Psychol. 2017;36(12):1109-1125.
133. Demakakos P, Biddulph JP, de Oliveira C, et al. Subjective social status and mortality: The English Longitudinal Study of Ageing. Eur J Epidemiol. 2018;33(8):729-739.
134. Quon EC, McGrath JJ. Subjective socioeconomic status and adolescent health: a meta-analysis. Health Psychol. 2014;33(5):433-447.
135. Scott KM, Al-Hamzawi AO, Andrade LH, et al. Associations between subjective social status and DSM-IV mental disorders: Results from the World Mental Health Surveys. JAMA Psychiatry. 2014;71(12):1400-1408.
136. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare Fee-for-Service Program. N Engl J Med. 2009;360(14):1418-1428.
137. Krumholz HM, Wang K, Lin Z, et al. Hospital-readmission risk—Isolating hospital effects from patient effects. N Engl J Med. 2017;377(11):1055-1064.
138. Krumholz HM, Lin Z, Keenan PS, et al. Relationship of hospital performance with readmission and mortality rates for patients hospitalized with acute myocardial infarction, heart failure, or pneumonia. JAMA. 2013;309(6):587-593.
139. Chin DL, Bang H, Manickam RN, Romano PS. Rethinking thirty-day hospital readmissions: Shorter intervals might be better indicators of quality of care. Health Aff (Millwood). 2016;35(10):1867-1875.
140. Feltner C, Jones CD, Cené CW, et al. Transitional care interventions to prevent readmissions for persons with heart failure: a systematic review and meta-analysis. Ann Intern Med. 2014;160(11):774-784.
141. Hudon C, Chouinard M-C, Lambert M, et al. Effectiveness of case management interventions for frequent users of healthcare services: a scoping review. BMJ Open. 2016;6(9):e012353.
142. Kripalani S, Theobald CN, Anctil B, Vasilevskis EE. Reducing hospital readmission: current strategies and future directions. Annu Rev Med. 2014;65:471-485.
143. Verhaegh KJ, MacNeil-Vroomen JL, Eslami S, et al. Transitional care interventions prevent hospital readmissions for adults with chronic illnesses. Health Aff (Millwood). 2014;33(9):1531-1539.
144. Duhig M, Gunasekara I, Patterson S. Understanding readmission to psychiatric hospital in Australia from the service users’ perspective: a qualitative study. Health Soc Care Community. 2017;25(1):75-82.
145. New England Healthcare Institute. A matter of urgency: Reducing emergency department overuse. Published 2010. Accessed January 11, 2021. https://www.nehi.net/writable/publication_files/file/nehi_ed_overuse_issue_brief_032610finaledits.pdf
146. Billings J, Raven MC. Dispelling an urban legend: Frequent emergency department users have substantial burden of disease. Health Aff (Millwood). 2013;32(12):2099-2108.
147. Moe J, Kirkland S, Ospina MB, et al. Mortality, admission rates and outpatient use among frequent users of emergency departments: a systematic review. Emerg Med J. 2016;33(3):230-236.
148. Carret MLV, Fassa ACG, Domingues MR. Inappropriate use of emergency services: a systematic review of prevalence and associated factors. Cad Saúde Pública. 2009;25(1):7-28.
149. Durand A-C, Palazzolo S, Tanti-Hardouin N, et al. Nonurgent patients in emergency departments: rational or irresponsible consumers? Perceptions of professionals and patients. BMC Res Notes. 2012;5(1):525.
150. Uscher-Pines L, Pines J, Kellermann A, et al. Deciding to visit the emergency department for non-urgent conditions: a systematic review of the literature. Am J Manag Care. 2013;19(1):47-59.
151. Kumar GS, Klein R. Effectiveness of case management strategies in reducing emergency department visits in frequent user patient populations: a systematic review. J Emerg Med. 2013;44(3):717-729.
152. Moe J, Kirkland SW, Rawe E, et al. Effectiveness of interventions to decrease emergency department visits by adult frequent users: a systematic review. Acad Emerg Med. 2017;24(1):40-52.
153. Raven MC, Kushel M, Ko MJ, et al. The effectiveness of emergency department visit reduction programs: a systematic review. Ann Emerg Med. 2016;68(4):467-483.e15.
154. Soril LJJ, Leggett LE, Lorenzetti DL, et al. Reducing frequent visits to the emergency department: a systematic review of interventions. PLoS One. 2015;10(4):e0123660.
155. Van den Heede K, Van de Voorde C. Interventions to reduce emergency department utilisation: a review of reviews. Health Policy. 2016;120(12):1337-1349.
156. Onken SJ, Dumont JM, Ridgway P, et al. Mental health recovery: What helps and what hinders? October 2002. Accessed January 11, 2021. https://www.nasmhpd.org/sites/default/files//MHSIPReport%281%29.pdf
157. Leggett LE, Khadaroo RG, Holroyd-Leduc J, et al. Measuring resource utilization. Medicine (Baltimore). 2016;95(10):e2759.
158. Bhandari A, Wagner T. Self-reported utilization of health care services: improving measurement and accuracy. Med Care Res Rev. 2006;63(2):217-235.
159. Short ME, Goetzel RZ, Pei X, et al. How accurate are self-reports? An analysis of self-reported healthcare utilization and absence when compared to administrative data. J Occup Environ Med. 2009;51(7):786-796.
160. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. National Academies Press; 2001.
161. Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med. 1991;324(6):370-376.
162. Rafter N, Hickey A, Condell S, et al. Adverse events in healthcare: learning from mistakes. QJM. 2015;108(4):273-277.
163. Makary MA, Daniel M. Medical error—the third leading cause of death in the US. BMJ. 2016;3(353):i2139.
164. D’Lima D, Crawford MJ, Darzi A, Archer S. Patient safety and quality of care in mental health: A world of its own? BJPsych Bull. 2017;41(5):241-243.
165. Maidment ID, Lelliott P, Paton C. Medication errors in mental healthcare: a systematic review. Qual Saf Health Care. 2006;15(6):409-413.
166. Mills PD, Watts BV, Shiner B, Hemphill RR. Adverse events occurring on mental health units. Gen Hosp Psychiatry. 2018;50:63-68.
167. Marcus SC, Hermann RC, Frankel MR, Cullen SW. Safety of psychiatric inpatients at the Veterans Health Administration. Psychiatr Serv. 2017;69(2):204-210.
168. Reilly CA, Cullen SW, Watts BV, et al. How well do incident reporting systems work on inpatient psychiatric units? Jt Comm J Qual Patient Saf. 2019;45:63-69.
169. Rosano A, Loha CA, Falvo R, et al. The relationship between avoidable hospitalization and accessibility to primary care: a systematic review. Eur J Public Health. 2013;23(3):356-360.
170. Dupre ME, Xu H, Granger BB, et al. Access to routine care and risks for 30-day readmission in patients with cardiovascular disease. Am Heart J. 2018;196:9-17.
171. Pizer SD, Prentice JC. What are the consequences of waiting for health care in the veteran population? J Gen Intern Med. 2011;26(S2):676-682.
172. Festinger DS, Lamb RJ, Kirby KC, Marlowe DB. The accelerated intake: a method for increasing initial attendance to outpatient cocaine treatment. J Appl Behav Anal. 1996;29(3):387-389.
173. Festinger DS, Lamb RJ, Marlowe DB, Kirby KC. From telephone to office: intake attendance as a function of appointment delay. Addict Behav. 2002;27(1):131-137.
174. Gallucci G, Swartz W, Hackerman F. Brief reports: Impact of the wait for an initial appointment on the rate of kept appointments at a mental health center. Psychiatr Serv. 2005;56(3):344-346.
175. Bleustein C, Rothschild DB, Valen A, et al. Wait times, patient satisfaction scores, and the perception of care. Am J Manag Care. 2014;20(5):393-400.
176. Reichert A, Jacobs R. The impact of waiting time on patient outcomes: Evidence from early intervention in psychosis services in England. Health Econ. 2018;27(11):1772-1787.
177. Doyle C, Lennox L, Bell D. A systematic review of evidence on the links between patient experience and clinical safety and effectiveness. BMJ Open. 2013;3(1):e001570.
178. Horvath AO, Del Re AC, Flückiger C, Symonds D. Alliance in individual psychotherapy. Psychotherapy. 2011;48(1):9-16.
179. Farley H, Enguidanos ER, Coletti CM, et al. Patient satisfaction surveys and quality of care: an information paper. Ann Emerg Med. 2014;64(4):351-357.
180. Aiken LH, Sermeus W, Van den Heede K, et al. Patient safety, satisfaction, and quality of hospital care: cross sectional surveys of nurses and patients in 12 countries in Europe and the United States. BMJ. 2012;344:e1717.
181. Boulding W, Glickman SW, Manary MP, Schulman KA, Staelin R. Relationship between patient satisfaction with inpatient care and hospital readmission within 30 days. Am J Manag Care. 2011;17(1):41-48.
182. Rohland BM, Langbehn DR, Rohrer JE. Relationship between service effectiveness and satisfaction among persons receiving Medicaid mental health services. Psychiatr Serv. 2000;51(2):248-250.
183. Zendjidjian X-Y, Baumstarck K, Auquier P, et al. Satisfaction of hospitalized psychiatry patients: Why should clinicians care? Patient Prefer Adherence. 2014;8:575-583.
184. Centers for Medicare & Medicaid Services. National health expenditures; aggregate and per capita amounts, annual percent change and percent distribution: Calendar years 1960-2016. National Health Expenditure Data. https://www.cms.gov/research-statistics-data-and-systems/statistics-trends-and-reports/nationalhealthexpenddata/nationalhealthaccountshistorical.html. Published 2018. Accessed August 21, 2018.
185. DuBard CA. Running the numbers. N C Med J. 2016;77(4):297-300.
186. Peikes D, Chen A, Schore J, Brown R. Effects of care coordination on hospitalization, quality of care, and health care expenditures among Medicare beneficiaries: 15 randomized trials. JAMA. 2009;301(6):603-618.
187. Seow H-Y, Sibley LM. Developing a dashboard to help measure and achieve the triple aim: A population-based cohort study. BMC Health Serv Res. 2014;14(1):363.
188. Lee VS, Kawamoto K, Hess R, et al. Implementation of a value-driven outcomes program to identify high variability in clinical costs and outcomes and association with reduced cost and improved quality. JAMA. 2016;316(10):1061.
189. Dyrbye LN, Shanafelt TD, Sinsky CA, et al. Burnout among health care professionals: A call to explore and address this underrecognized threat to safe, high-quality care discussion paper. National Academy of Medicine. July 5, 2017. Accessed January 11, 2021. https://nam.edu/burnout-among-health-care-professionals-a-call-to-explore-and-address-this-underrecognized-threat-to-safe-high-quality-care/
190. Johnson J, Hall LH, Berzins K, et al. Mental healthcare staff well-being and burnout: a narrative review of trends, causes, implications, and recommendations for future interventions. Int J Ment Health Nurs. 2018;27(1):20-32.
191. Morse G, Salyers MP, Rollins AL, et al. Burnout in mental health services: a review of the problem and its remediation. Adm Policy Ment Health. 2012;39(5):341-352.
192. Hall LH, Johnson J, Watt I, et al. Healthcare staff wellbeing, burnout, and patient safety: a systematic review. PLoS One. 2016;11(7):e0159015.
193. Garman AN, Corrigan PW, Morris S. Staff burnout and patient satisfaction: evidence of relationships at the care unit level. J Occup Health Psychol. 2002;7(3):235-241.
194. Wilkinson H, Whittington R, Perry L, Eames C. Examining the relationship between burnout and empathy in healthcare professionals: a systematic review. Burn Res. 2017;6:18-29.
195. Holmqvist R, Jeanneau M. Burnout and psychiatric staff’s feelings towards patients. Psychiatry Res. 2006;145(2-3):207-213.
196. Leiter MP, Maslach C. Nurse turnover: the mediating role of burnout. J Nurs Manag. 2009;17(3):331-339.
197. Zhang Y, Feng X. The relationship between job satisfaction, burnout, and turnover intention among physicians from urban state-owned medical institutions in Hubei, China: a cross-sectional study. BMC Health Serv Res. 2011;11(1):235.
198. Halter M, Boiko O, Pelone F, et al. The determinants and consequences of adult nursing staff turnover: a systematic review of systematic reviews. BMC Health Serv Res. 2017;17(1):824.
199. Hamidi MS, Bohman B, Sandborg C, et al. Estimating institutional physician turnover attributable to self-reported burnout and associated financial burden: a case study. BMC Health Serv Res. 2018;18(1):851.
200. National Taskforce for Humanity in Healthcare. The business case for humanity in healthcare position paper. April 2018. Accessed January 11, 2021. https://www.vocera.com/public/pdf/NTHBusinessCase_final003.pdf
201. Waldman JD, Kelly F, Arora S, Smith HL. The shocking cost of turnover in health care. Health Care Manage Rev. 2004;29(1):2-7.
202. Brunette MF, Asher D, Whitley R, et al. Implementation of integrated dual disorders treatment: a qualitative analysis of facilitators and barriers. Psychiatr Serv. 2008;59(9):989-995.
203. Mancini AD, Moser LL, Whitley R, et al. Assertive community treatment: facilitators and barriers to implementation in routine mental health settings. Psychiatr Serv. 2009;60(2):189-195.
204. Rollins AL, Salyers MP, Tsai J, Lydick JM. Staff turnover in statewide implementation of ACT: Relationship with ACT fidelity and other team characteristics. Adm Policy Ment Health. 2010;37(5):417-426.
205. Woltmann EM, Whitley R, McHugo GJ, et al. The role of staff turnover in the implementation of evidence-based practices in mental health care. Psychiatr Serv. 2008;59(7):732-737.
206. Alegría M, NeMoyer A, Falgàs Bagué I, et al. Social determinants of mental health: Where we are and where we need to go. Curr Psychiatry Rep. 2018;20(11):95.
207. Daniel H, Bornstein SS, Kane GC. Addressing social determinants to improve patient care and promote health equity: an American College of Physicians position paper. Ann Intern Med. 2018;168(8):577-578.
208. Park H, Roubal AM, Jovaag A, Gennuso KP, Catlin BB. Relative contributions of a set of health factors to selected health outcomes. Am J Prev Med. 2015;49(6):961-969.
209. Ash AS, Mick EO, Ellis RP, et al. Social determinants of health in managed care payment formulas. JAMA Intern Med. 2017;177(10):1424.
210. de Beurs E, Warmerdam EH, Oudejans SCC, et al. Treatment outcome, duration, and costs: a comparison of performance indicators using data from eight mental health care providers in the Netherlands. Adm Policy Ment Health. 2018;45(2):212-223.
211. Dunbar-Rees R. Paying for what matters most: the future of outcomes-based payments in healthcare. Future Healthc J. 2018;5(2):98-102.
212. Woolhandler S, Himmelstein DU. Administrative work consumes one-sixth of U.S. physicians’ working hours and lowers their career satisfaction. Int J Health Serv. 2014;44(4):635-642.