User login
Dementia Evaluation, Management, and Outreach
Dementia is a common, multifaceted problem with significant implications for function and quality of life among older individuals. Current demographic shifts are magnifying this problem. Particularly troubling are neuropsychiatric symptoms, which, in addition to creating caregiver distress, also are linked to functional decline, institutionalization, higher health care costs, and mortality (even after controlling for other potential confounders and severity of cognitive impairment). Furthermore, dementia is often unrecognized and underdiagnosed, and patients with dementia historically have poor access to care, particularly those living in rural areas.
The Geriatrics/Dementia Clinic at the Baltimore Veterans Affairs Medical Center (BVAMC) is a referral resource that provides extensive, multidisciplinary evaluations as well as coordinated subspecialist and interprofessional case review. A diverse group of clinicians (representing geriatrics, geriatric psychiatry, neuropsychology, clinical pharmacy, nursing, and social work) perform a half-day evaluation, followed by a meeting of the interdisciplinary team, and feedback session where patients and their families are given the results of the testing and diagnostic impressions as well as plans for further evaluation and treatment. Finally, in addition to the benefits it provides to veterans and their families, this clinic has proven to be an important resource for professional trainees.
Yet this model can be difficult to access, and those living in more remote regions had challenges availing themselves of this resource. Often, they would have to wake before dawn to drive 2 to 4 hours to the medical center. Furthermore, despite the many obvious benefits of this comprehensive approach, veterans and their families often left the clinic with a staggering amount of information and numerous recommendations for future care but without the assurance of integrated follow-up (a burden borne particularly by those living remotely).
The DEMO Program
In response to the dual challenges of access to and coordination of care, existing resources were leveraged with about $250,000 of VA T21 funds (adding both a full-time geriatric nurse practitioner and a psychology technician to the multidisciplinary team) to design and implement the novel DEMO (Dementia Evaluation, Management and Outreach) program. This program aimed to (1) extend dementia evaluations to regional community-based outpatient clinics (CBOCs) that serve veterans in outlying regions; and (2) improve the management and the follow-up that these veterans receive, with a focus on containing costs, while improving both the quality of and veterans’ (and their families’) satisfaction with health care.
Methods
Dementia evaluations were conducted by a geriatric nurse practitioner and psychology technician teamlet at the CBOCs. In addition to neuropsychological testing, medical records were reviewed, caregivers were interviewed, and the patients were examined. The data were then brought back to the full BVAMC Geriatrics/Dementia Clinic multidisciplinary team for discussion. The team reached a consensus diagnosis and then made a comprehensive plan for the further evaluation and management of these complex patients. The plan was entered into the Computerized Patient Record System and communicated to the patient and caregiver during a follow-up CBOC visit.
The teamlet frequently provided informal education during CBOC visits in addition to formal lectures given by experts from the BVAMC and the University of Maryland. The DEMO program was introduced to providers at the CBOCs by e-mail with follow-up information sessions provided on site.
Rather than having patients simply return to their CBOC primary care providers (PCPs) and exposing patients to the risks of poor communication/coordination of care, services were expanded to include regular phone follow-up calls with case management services that augmented those of their PCPs. The goal was to improve outcomes for these patients and provide alternatives to institutionalization.
Standardized instruments were used to gauge patient and caregiver satisfaction, obtain cost data from the VA and the Centers for Medicare & Medicaid Services, and medication data from the Pharmacy Benefits files.
The institutional review board provided approval to collect data on participants to assess the program’s clinical and economic impacts. Since all patients were suspected to have dementia, the informed consent procedures included additional protections. The patient’s understanding of the pertinent information related to participation in the study was assessed to help ensure that participants with dementia truly understood the conditions to which they were consenting. If the potential participant could not provide informed consent, it was obtained from a surrogate with durable power of attorney (the person recognized by Maryland law as the substitute decision maker or the veteran’s legal guardian). Consent was assessed on an ongoing basis regardless of the patient’s capacity to give informed consent, and those willing to have their data collected were enrolled in a “research” arm. These participants were compared with veterans in the dementia clinic during the enrollment period but who did not consent to participate in the research arm, controlling for sociodemographic characteristics and prior health care utilization.
Data Analysis
Patterns of health care utilization may fluctuate with time, and enrollment may identify potential problems that otherwise would not have been found. The authors looked at health care use over 6-month and 1-year intervals before and after enrollment, examining Occupational Physical Assessment Test (OPAT) data and fee-based outpatient data for both inpatient events (including nursing home utilization and hospitalization events) and outpatient visits (including primary, specialty and mental health care services; home care visits, and emergency department [ED]). The total cost incurred by outpatient visits and inpatient care was then adjusted to 2011 dollars. The relationship between program enrollment and health care use and costs was examined with a multivariate regression analyses, controlling for age and health care use in the prior year among veterans who were in the “consent” group or “nonconsent” group.
Outcome variables included the number of primary care, ED, specialty care, home health care, mental health care clinic, and inpatient visits; total inpatient bed days; and the total costs of all the events. The authors fit different multivariate models for these events according to their distributions. Specifically, the Poisson model was used if the distribution of the outcome variable was not overdispersed (eg, ED, primary care).
A negative binomial was used if the distribution of these events was overdispersed (eg, specialty care visits). Further, because the occurrence of inpatient events is relatively rare, a logit model was used to examine the relationship between enrollment status and probability of any inpatient events, regardless of the number of events. Generalized estimating equation (GEE) model with gamma distribution and log link function was used to examine the relationship between cost and program enrollment. The authors also examined the program’s effect on medication use, focusing on high-risk medications in older adults, comparing both the number of unique medications, as well as frequency such medications prescribed 1 year before and after the enrollment.1
Results
Two hundred ninety-eight (298) veterans were referred to DEMO from a 150-mile radius of Baltimore. Of these veterans, 132 consented to participate in this study. The study participants largely were representative of the total group as well as both the overall veteran population and the more general population of community-dwelling individuals with dementia (Table). 
Veterans in the DEMO program largely had mild-to-moderate cognitive impairment with mean Mini-Mental State Examination (MMSE) score of 22 and significant functional limitations (Table). Only 3% displayed a pattern of “pure” dementia typical of Alzheimer disease, and 11% of those referred did not have significant abnormalities on neurocognitive testing.
The team averaged 10.3 recommendations (range 3-22), which focused on a diverse set of issues related to additional diagnostic and therapeutic concerns. Although screening data, including basic laboratory results and imaging, was requested in the referral form, in 71% of cases further diagnostic investigations were suggested. With regard to therapeutic suggestions, not surprisingly, medicines were cited as targets in a majority of cases (eg, discontinuing high-risk medications, initiating/titrating medications to minimize cardiovascular risk). While remaining mindful of the time to benefit and competing morbidities, measures to modify cardiovascular risk factors were suggested in more than half and treatment of depression in 15% of cases. Similarly, addressing poor sensory input was suggested in 38% of cases, with other common recommendations focusing on multiple environmental and social interventions (> 50%) as well as supports/outlets/respite for the caregivers.
The full multidisciplinary DEMO group met only weekly to review cases, and due to travel and scheduling difficulties, feedback to the patients and their families often was delayed for weeks. Although initial plans included regularly scheduled follow-up phone calls, demand quickly outstripped program resources. Nonetheless, chart reviews and abstracted adherence and utilization data revealed that PCPs successfully implemented 52% of recommendations within 2 weeks, rising to > 60% by 3 months. When patients were reevaluated at 1 year, they were remarkably stable: Mini–Mental State Examination (baseline 22.2 ± 5.0 → 22.3 ± 5.7 at follow-up) and Instrumental Activities of Daily Living scores (15.5 ± 10.6 → 17.7 ± 11.4).
Feedback
This program was enthusiastically received by both patients and their caregivers—100% and 98%, respectively—reporting overall satisfaction with the services received and 93% of caregivers indicating satisfaction with how the program met their needs. Caregivers were happy with the amount of time the provider took to answer questions (100% satisfied with the amount of time the DEMO provider spent and that they explained “what they wanted to know,” with 98% responding “good” or “great” for both), as well as with services and amount of help received (83% and 77% “very satisfied,” respectively).
In the survey, 98% of caregivers and patients felt that the program helped them deal more effectively with their problems, 97% would recommend the program to a friend in need of similar help, and 100% would come back if they were to seek help again. In keeping with DEMO’s initial aim of increasing access, there was favorable feedback on the ability to get in and be seen and convenience of location. In addition, referring providers universally expressed satisfaction with the referral process (referrals increased linearly); timeliness of scheduling; usefulness of the recommendations; and they planned on continuing to refer patients.
Although there was great variability, controlling for age and prior utilization, veterans in DEMO had statistically significant (all P < .05) fewer ED and specialty care visits and more mental health care clinic visits 183 to 365 daysafter referral dates compared with those in the nonconsented group. Veterans in the consented group also were less likely to use inpatient care than were veterans in the nonconsented group 183 to 365 days after referral dates. These trends were similarly reflected after controlling for age and prior utilization as well as when examining health care costs (data not shown). Nonetheless, DEMO did not seem to have any effect on overall inpatient bed days, primary/home-based care visits, or total costs. In fact, utilization of mental health care resources increased (Figures 1, 2, and 3).
Discussion
Cognitive issues in patients within the general population are common, and the patients cared for by the VA are no exception. Dementia is more common in rural compared with urban areas, and those living in more remote locations have reduced access to specialized evaluation, management, and support services.2 The authors describe a novel program that dramatically increased patient access, bringing the normally tertiary referral services to geographically remote CBOCs at a minimal investment. These services were well received by patients, caregivers, and PCPs. As anticipated, patients and their caregivers especially appreciated the ease and convenience of access. Considering the already significant burden(s) borne every day by those caring for patients with dementia, the benefit of this approach is evident.
Clinicians often feel uncomfortable in evaluating and managing patients with cognitive deficits. Nonetheless, the role of specialized clinics in diagnosing dementia has been demonstrated previously, and the present results are in agreement with previous studies.3 The novelty here is the provision of specialized care usually found only in large, academic medical centers to local CBOCs. By bringing specialized services to geographically isolated patients, the DEMO program was able to increase both access and utilization. Furthermore, providing coordination and ongoing, focused follow-up provided increases in satisfaction and efficiency.
An additional benefit of this approach is the opportunity for PCP education. The authors even found anecdotal reductions in ED usage as well as acute hospitalization and long-term placement—although it was not a statistical significant difference. The relatively high use of mental health care services in this population is in line with previous reports in similar populations, and greater utilization of mental health care services may be one explanation why overall costs did not differ between the 2 groups.4 Nonetheless, this intimates that such a program may yield savings over a longer term, as has been demonstrated in patients with a variety of other psychiatric diagnoses cared for in the community rather than in institutions.5,6
The prevalence of dementia and its associated costs are nearly $50,000 per year per person—suggesting a total cost in the hundreds of billions of dollars.7 Similarly, the importance of caregiver support (including psychosocial interventions, such as the one piloted here) has been demonstrated in a variety of settings (even without improvement in caregiver burden itself).8
There were a number of challenges in the rollout and delivery of DEMO. Although CBOC PCPs
Unfortunately, DEMO was underresourced to provide either real-time feedback true or first responder services. Misunderstandings concerning this were an early challenge to PCP acceptance. However, the longitudinal presence and close working relationships of the DEMO teamlet in each CBOC allowed their use as an adjunct to primary care, and increased the efficacy of both.
Limitations
A number of additional caveats must be made. First, this study had a relatively small number of participants, and there was great variability in health care utilization. This is particularly germane in this population of patients with dementia, which typically has an asymmetrically high use of health care resources. Additionally, the relatively limited follow-up period may have blunted the programs true effect(s). Further, although veterans in the nonconsented group were not officially enrolled in the program, there was likely spillover of the effects of the program on practice patterns, leading to an underestimation of the program’s impact.
Conclusion
With minimal resources, DEMO successfully brought expert evaluation (usually tertiary referral) services, and provided specialized case management in coordination with existing primary care to remote patients. Although there were a number of features rather unique to this setting (eg, infrastructural support; close working interdisciplinary and interprofessional relationships, buy-in at all levels, relative geographic density/demographic homogeneity of participants), specialized case management is increasingly being adopted throughout the VA (and elsewhere). Although the value of collaborative, interdisciplinary interventions has been shown in a variety of settings and conditions—nursing homes,9 chronic low back pain,10 safety among hospital inpatients11—its utility for dementia care is relatively underexplored.
Yet the effectiveness of team-based care for individuals has been demonstrated in a number of settings, including Alzheimer disease.12,13 In addition to involving a number of disciplines, collaborative care is marked by coordination. A number of recent systematic reviews have found that behavioral and multicomponent interventions directed towards the caregiver as well as case management were beneficial in improving some outcomes, although there is considerable heterogeneity in the effects.14,15 Future work will focus on examining methods to focus/optimize interventions based on individual patient characteristics.
Given the epidemiologic trends, care for patients with dementia is expected to grow. Novel interventions, like DEMO, are a particularly promising option to meet this challenge. In fact, just such a collaborative practice-ready workforce has been identified by the World Health Organization as crucial to meeting the challenges of the health needs in the 21st century.16 With the feasibility of such an approach in this population now evident, further studies (including larger sample sizes, across greater geographic regions, as well as among more diverse populations) should be undertaken. These results, if replicated, suggest a novel approach to the particularly vexing problem of caring for patients with dementia with potentially far-reaching public health implications.
Acknowledgments
Supported with T21 funds from VA to expand noninstitutional alternatives to institutional extended care for veterans, as well as the Geriatrics Research and Clinical Center (GRECC) at the Baltimore VAMC.
1. National Committee for Quality Assurance. 2011 HEDIS List. http://www.ncqa.org/tabid/1274/Default.aspx. Accessed December 16, 2016.
2. Russ TC, Batty GD, Hearnshaw GF, Fenton C, Starr JM. Geographical variation in dementia: systemic review with meta-analysis. Int J Epidemiol. 2012;41(4):1012-1032.
3. Wolfs CA, Kessels A, Dirksen CD, Severens JL, Verhey FR. Integrated multidisciplinary diagnostic approach for dementia care: randomized controlled trial. Br J Psychiatry. 2008;192(4):300-305.
4. King PR, Vair CL, Wade M, et al. Outpatient health care utilization in a sample of cognitively impaired veterans receiving care in VHA geriatric evaluation and management clinics. Psychol Serv. 2015;12(1):66-72.
5. Tam-Tham H, Cepoiu-Martin M, Ronksley PE, Maxwell CJ, Hemmelgarn BR. Dementia case management and risk of long-term care placement: a systemic review and meta-analysis. Int J Geriatr Psychiatr. 2013;28(9):889-902.
6. Rothbard AB, Kuno E, Schinnar AP, Hadley TR, Turk R. Service utilization and cost of community care for discharged state hospital patients: a 3-year follow-up study. Am J Psychiatry. 1999;156(6):920-927.
7. Hurd MD, Martorell P, Delavande A, Mullen KJ, Langa KM. Monetary costs of dementia in the United States. N Engl J Med. 2013;368(14):1326-1334.
8. Adelman RD, Tmanova LL, Delgado D, Dion S, Lachs MS. Caregiver burden: a clinical review. JAMA. 2014;311(10):1052-1060.
9. Nazir A, Unroe K, Tegeler M, Khan B, Azar J, Boustani M. Systematic review of interdisciplinary interventions in nursing homes. J Am Med Dir Assoc. 2013;14(7):471-478.
10. Kamper SJ, Apeldoorn AT, Chiarotto A, et al. Multidisciplinary biopsychosocial rehabilitation for chronic low back pain: Cochrane systematic review and meta-analysis. BMJ. 2015;350:h444.
11. O’Leary KJ, Buck R, Fligiel HM, et al. Structured interdisciplinary rounds in a medical teaching unit: improving patient safety. Arch Intern Med. 2011;171(7):678-684.
12. Counsell SR, Callahan CM, Clark DO, et al. Geriatric care management for low-income seniors: a randomized controlled trial. JAMA. 2007;298(22):2623-2633.
13. Callahan CM, Boustani MA, Unverzagt FW, et al. Effectiveness of collaborative care for older adults with Alzheimer disease in primary care: a randomized controlled trial. JAMA. 2006;295(18):2148-2157.
14. Health Quality Ontario. Caregiver- and patient-directed interventions for dementia: an evidence-based analysis. Ont Health Technol Assess Ser. 2008;8(4):1-98.
15. Reilly S, Miranda-Castillo C, Malouf R, et al. Case management approaches to home support for people with dementia. Cochrane Database Syst Rev. 2015;1:CD008345.
16. World Health Organization. Framework for action on interprofessional education and collaborative practice. http://apps.who.int/iris/bitstream/10665/70185/1/WHO_HRH_HPN_10.3_eng.pdf?ua=1. Published 2010. Accessed November 17, 2016.
Dementia is a common, multifaceted problem with significant implications for function and quality of life among older individuals. Current demographic shifts are magnifying this problem. Particularly troubling are neuropsychiatric symptoms, which, in addition to creating caregiver distress, also are linked to functional decline, institutionalization, higher health care costs, and mortality (even after controlling for other potential confounders and severity of cognitive impairment). Furthermore, dementia is often unrecognized and underdiagnosed, and patients with dementia historically have poor access to care, particularly those living in rural areas.
The Geriatrics/Dementia Clinic at the Baltimore Veterans Affairs Medical Center (BVAMC) is a referral resource that provides extensive, multidisciplinary evaluations as well as coordinated subspecialist and interprofessional case review. A diverse group of clinicians (representing geriatrics, geriatric psychiatry, neuropsychology, clinical pharmacy, nursing, and social work) perform a half-day evaluation, followed by a meeting of the interdisciplinary team, and feedback session where patients and their families are given the results of the testing and diagnostic impressions as well as plans for further evaluation and treatment. Finally, in addition to the benefits it provides to veterans and their families, this clinic has proven to be an important resource for professional trainees.
Yet this model can be difficult to access, and those living in more remote regions had challenges availing themselves of this resource. Often, they would have to wake before dawn to drive 2 to 4 hours to the medical center. Furthermore, despite the many obvious benefits of this comprehensive approach, veterans and their families often left the clinic with a staggering amount of information and numerous recommendations for future care but without the assurance of integrated follow-up (a burden borne particularly by those living remotely).
The DEMO Program
In response to the dual challenges of access to and coordination of care, existing resources were leveraged with about $250,000 of VA T21 funds (adding both a full-time geriatric nurse practitioner and a psychology technician to the multidisciplinary team) to design and implement the novel DEMO (Dementia Evaluation, Management and Outreach) program. This program aimed to (1) extend dementia evaluations to regional community-based outpatient clinics (CBOCs) that serve veterans in outlying regions; and (2) improve the management and the follow-up that these veterans receive, with a focus on containing costs, while improving both the quality of and veterans’ (and their families’) satisfaction with health care.
Methods
Dementia evaluations were conducted by a geriatric nurse practitioner and psychology technician teamlet at the CBOCs. In addition to neuropsychological testing, medical records were reviewed, caregivers were interviewed, and the patients were examined. The data were then brought back to the full BVAMC Geriatrics/Dementia Clinic multidisciplinary team for discussion. The team reached a consensus diagnosis and then made a comprehensive plan for the further evaluation and management of these complex patients. The plan was entered into the Computerized Patient Record System and communicated to the patient and caregiver during a follow-up CBOC visit.
The teamlet frequently provided informal education during CBOC visits in addition to formal lectures given by experts from the BVAMC and the University of Maryland. The DEMO program was introduced to providers at the CBOCs by e-mail with follow-up information sessions provided on site.
Rather than having patients simply return to their CBOC primary care providers (PCPs) and exposing patients to the risks of poor communication/coordination of care, services were expanded to include regular phone follow-up calls with case management services that augmented those of their PCPs. The goal was to improve outcomes for these patients and provide alternatives to institutionalization.
Standardized instruments were used to gauge patient and caregiver satisfaction, obtain cost data from the VA and the Centers for Medicare & Medicaid Services, and medication data from the Pharmacy Benefits files.
The institutional review board provided approval to collect data on participants to assess the program’s clinical and economic impacts. Since all patients were suspected to have dementia, the informed consent procedures included additional protections. The patient’s understanding of the pertinent information related to participation in the study was assessed to help ensure that participants with dementia truly understood the conditions to which they were consenting. If the potential participant could not provide informed consent, it was obtained from a surrogate with durable power of attorney (the person recognized by Maryland law as the substitute decision maker or the veteran’s legal guardian). Consent was assessed on an ongoing basis regardless of the patient’s capacity to give informed consent, and those willing to have their data collected were enrolled in a “research” arm. These participants were compared with veterans in the dementia clinic during the enrollment period but who did not consent to participate in the research arm, controlling for sociodemographic characteristics and prior health care utilization.
Data Analysis
Patterns of health care utilization may fluctuate with time, and enrollment may identify potential problems that otherwise would not have been found. The authors looked at health care use over 6-month and 1-year intervals before and after enrollment, examining Occupational Physical Assessment Test (OPAT) data and fee-based outpatient data for both inpatient events (including nursing home utilization and hospitalization events) and outpatient visits (including primary, specialty and mental health care services; home care visits, and emergency department [ED]). The total cost incurred by outpatient visits and inpatient care was then adjusted to 2011 dollars. The relationship between program enrollment and health care use and costs was examined with a multivariate regression analyses, controlling for age and health care use in the prior year among veterans who were in the “consent” group or “nonconsent” group.
Outcome variables included the number of primary care, ED, specialty care, home health care, mental health care clinic, and inpatient visits; total inpatient bed days; and the total costs of all the events. The authors fit different multivariate models for these events according to their distributions. Specifically, the Poisson model was used if the distribution of the outcome variable was not overdispersed (eg, ED, primary care).
A negative binomial was used if the distribution of these events was overdispersed (eg, specialty care visits). Further, because the occurrence of inpatient events is relatively rare, a logit model was used to examine the relationship between enrollment status and probability of any inpatient events, regardless of the number of events. Generalized estimating equation (GEE) model with gamma distribution and log link function was used to examine the relationship between cost and program enrollment. The authors also examined the program’s effect on medication use, focusing on high-risk medications in older adults, comparing both the number of unique medications, as well as frequency such medications prescribed 1 year before and after the enrollment.1
Results
Two hundred ninety-eight (298) veterans were referred to DEMO from a 150-mile radius of Baltimore. Of these veterans, 132 consented to participate in this study. The study participants largely were representative of the total group as well as both the overall veteran population and the more general population of community-dwelling individuals with dementia (Table).
Veterans in the DEMO program largely had mild-to-moderate cognitive impairment with mean Mini-Mental State Examination (MMSE) score of 22 and significant functional limitations (Table). Only 3% displayed a pattern of “pure” dementia typical of Alzheimer disease, and 11% of those referred did not have significant abnormalities on neurocognitive testing.
The team averaged 10.3 recommendations (range 3-22), which focused on a diverse set of issues related to additional diagnostic and therapeutic concerns. Although screening data, including basic laboratory results and imaging, was requested in the referral form, in 71% of cases further diagnostic investigations were suggested. With regard to therapeutic suggestions, not surprisingly, medicines were cited as targets in a majority of cases (eg, discontinuing high-risk medications, initiating/titrating medications to minimize cardiovascular risk). While remaining mindful of the time to benefit and competing morbidities, measures to modify cardiovascular risk factors were suggested in more than half and treatment of depression in 15% of cases. Similarly, addressing poor sensory input was suggested in 38% of cases, with other common recommendations focusing on multiple environmental and social interventions (> 50%) as well as supports/outlets/respite for the caregivers.
The full multidisciplinary DEMO group met only weekly to review cases, and due to travel and scheduling difficulties, feedback to the patients and their families often was delayed for weeks. Although initial plans included regularly scheduled follow-up phone calls, demand quickly outstripped program resources. Nonetheless, chart reviews and abstracted adherence and utilization data revealed that PCPs successfully implemented 52% of recommendations within 2 weeks, rising to > 60% by 3 months. When patients were reevaluated at 1 year, they were remarkably stable: Mini–Mental State Examination (baseline 22.2 ± 5.0 → 22.3 ± 5.7 at follow-up) and Instrumental Activities of Daily Living scores (15.5 ± 10.6 → 17.7 ± 11.4).
Feedback
This program was enthusiastically received by both patients and their caregivers—100% and 98%, respectively—reporting overall satisfaction with the services received and 93% of caregivers indicating satisfaction with how the program met their needs. Caregivers were happy with the amount of time the provider took to answer questions (100% satisfied with the amount of time the DEMO provider spent and that they explained “what they wanted to know,” with 98% responding “good” or “great” for both), as well as with services and amount of help received (83% and 77% “very satisfied,” respectively).
In the survey, 98% of caregivers and patients felt that the program helped them deal more effectively with their problems, 97% would recommend the program to a friend in need of similar help, and 100% would come back if they were to seek help again. In keeping with DEMO’s initial aim of increasing access, there was favorable feedback on the ability to get in and be seen and convenience of location. In addition, referring providers universally expressed satisfaction with the referral process (referrals increased linearly); timeliness of scheduling; usefulness of the recommendations; and they planned on continuing to refer patients.
Although there was great variability, controlling for age and prior utilization, veterans in DEMO had statistically significant (all P < .05) fewer ED and specialty care visits and more mental health care clinic visits 183 to 365 daysafter referral dates compared with those in the nonconsented group. Veterans in the consented group also were less likely to use inpatient care than were veterans in the nonconsented group 183 to 365 days after referral dates. These trends were similarly reflected after controlling for age and prior utilization as well as when examining health care costs (data not shown). Nonetheless, DEMO did not seem to have any effect on overall inpatient bed days, primary/home-based care visits, or total costs. In fact, utilization of mental health care resources increased (Figures 1, 2, and 3).
Discussion
Cognitive issues in patients within the general population are common, and the patients cared for by the VA are no exception. Dementia is more common in rural compared with urban areas, and those living in more remote locations have reduced access to specialized evaluation, management, and support services.2 The authors describe a novel program that dramatically increased patient access, bringing the normally tertiary referral services to geographically remote CBOCs at a minimal investment. These services were well received by patients, caregivers, and PCPs. As anticipated, patients and their caregivers especially appreciated the ease and convenience of access. Considering the already significant burden(s) borne every day by those caring for patients with dementia, the benefit of this approach is evident.
Clinicians often feel uncomfortable in evaluating and managing patients with cognitive deficits. Nonetheless, the role of specialized clinics in diagnosing dementia has been demonstrated previously, and the present results are in agreement with previous studies.3 The novelty here is the provision of specialized care usually found only in large, academic medical centers to local CBOCs. By bringing specialized services to geographically isolated patients, the DEMO program was able to increase both access and utilization. Furthermore, providing coordination and ongoing, focused follow-up provided increases in satisfaction and efficiency.
An additional benefit of this approach is the opportunity for PCP education. The authors even found anecdotal reductions in ED usage as well as acute hospitalization and long-term placement—although it was not a statistical significant difference. The relatively high use of mental health care services in this population is in line with previous reports in similar populations, and greater utilization of mental health care services may be one explanation why overall costs did not differ between the 2 groups.4 Nonetheless, this intimates that such a program may yield savings over a longer term, as has been demonstrated in patients with a variety of other psychiatric diagnoses cared for in the community rather than in institutions.5,6
The prevalence of dementia and its associated costs are nearly $50,000 per year per person—suggesting a total cost in the hundreds of billions of dollars.7 Similarly, the importance of caregiver support (including psychosocial interventions, such as the one piloted here) has been demonstrated in a variety of settings (even without improvement in caregiver burden itself).8
There were a number of challenges in the rollout and delivery of DEMO. Although CBOC PCPs
Unfortunately, DEMO was underresourced to provide either real-time feedback true or first responder services. Misunderstandings concerning this were an early challenge to PCP acceptance. However, the longitudinal presence and close working relationships of the DEMO teamlet in each CBOC allowed their use as an adjunct to primary care, and increased the efficacy of both.
Limitations
A number of additional caveats must be made. First, this study had a relatively small number of participants, and there was great variability in health care utilization. This is particularly germane in this population of patients with dementia, which typically has an asymmetrically high use of health care resources. Additionally, the relatively limited follow-up period may have blunted the programs true effect(s). Further, although veterans in the nonconsented group were not officially enrolled in the program, there was likely spillover of the effects of the program on practice patterns, leading to an underestimation of the program’s impact.
Conclusion
With minimal resources, DEMO successfully brought expert evaluation (usually tertiary referral) services, and provided specialized case management in coordination with existing primary care to remote patients. Although there were a number of features rather unique to this setting (eg, infrastructural support; close working interdisciplinary and interprofessional relationships, buy-in at all levels, relative geographic density/demographic homogeneity of participants), specialized case management is increasingly being adopted throughout the VA (and elsewhere). Although the value of collaborative, interdisciplinary interventions has been shown in a variety of settings and conditions—nursing homes,9 chronic low back pain,10 safety among hospital inpatients11—its utility for dementia care is relatively underexplored.
Yet the effectiveness of team-based care for individuals has been demonstrated in a number of settings, including Alzheimer disease.12,13 In addition to involving a number of disciplines, collaborative care is marked by coordination. A number of recent systematic reviews have found that behavioral and multicomponent interventions directed towards the caregiver as well as case management were beneficial in improving some outcomes, although there is considerable heterogeneity in the effects.14,15 Future work will focus on examining methods to focus/optimize interventions based on individual patient characteristics.
Given the epidemiologic trends, care for patients with dementia is expected to grow. Novel interventions, like DEMO, are a particularly promising option to meet this challenge. In fact, just such a collaborative practice-ready workforce has been identified by the World Health Organization as crucial to meeting the challenges of the health needs in the 21st century.16 With the feasibility of such an approach in this population now evident, further studies (including larger sample sizes, across greater geographic regions, as well as among more diverse populations) should be undertaken. These results, if replicated, suggest a novel approach to the particularly vexing problem of caring for patients with dementia with potentially far-reaching public health implications.
Acknowledgments
Supported with T21 funds from VA to expand noninstitutional alternatives to institutional extended care for veterans, as well as the Geriatrics Research and Clinical Center (GRECC) at the Baltimore VAMC.
Dementia is a common, multifaceted problem with significant implications for function and quality of life among older individuals. Current demographic shifts are magnifying this problem. Particularly troubling are neuropsychiatric symptoms, which, in addition to creating caregiver distress, also are linked to functional decline, institutionalization, higher health care costs, and mortality (even after controlling for other potential confounders and severity of cognitive impairment). Furthermore, dementia is often unrecognized and underdiagnosed, and patients with dementia historically have poor access to care, particularly those living in rural areas.
The Geriatrics/Dementia Clinic at the Baltimore Veterans Affairs Medical Center (BVAMC) is a referral resource that provides extensive, multidisciplinary evaluations as well as coordinated subspecialist and interprofessional case review. A diverse group of clinicians (representing geriatrics, geriatric psychiatry, neuropsychology, clinical pharmacy, nursing, and social work) perform a half-day evaluation, followed by a meeting of the interdisciplinary team, and feedback session where patients and their families are given the results of the testing and diagnostic impressions as well as plans for further evaluation and treatment. Finally, in addition to the benefits it provides to veterans and their families, this clinic has proven to be an important resource for professional trainees.
Yet this model can be difficult to access, and those living in more remote regions had challenges availing themselves of this resource. Often, they would have to wake before dawn to drive 2 to 4 hours to the medical center. Furthermore, despite the many obvious benefits of this comprehensive approach, veterans and their families often left the clinic with a staggering amount of information and numerous recommendations for future care but without the assurance of integrated follow-up (a burden borne particularly by those living remotely).
The DEMO Program
In response to the dual challenges of access to and coordination of care, existing resources were leveraged with about $250,000 of VA T21 funds (adding both a full-time geriatric nurse practitioner and a psychology technician to the multidisciplinary team) to design and implement the novel DEMO (Dementia Evaluation, Management and Outreach) program. This program aimed to (1) extend dementia evaluations to regional community-based outpatient clinics (CBOCs) that serve veterans in outlying regions; and (2) improve the management and the follow-up that these veterans receive, with a focus on containing costs, while improving both the quality of and veterans’ (and their families’) satisfaction with health care.
Methods
Dementia evaluations were conducted by a geriatric nurse practitioner and psychology technician teamlet at the CBOCs. In addition to neuropsychological testing, medical records were reviewed, caregivers were interviewed, and the patients were examined. The data were then brought back to the full BVAMC Geriatrics/Dementia Clinic multidisciplinary team for discussion. The team reached a consensus diagnosis and then made a comprehensive plan for the further evaluation and management of these complex patients. The plan was entered into the Computerized Patient Record System and communicated to the patient and caregiver during a follow-up CBOC visit.
The teamlet frequently provided informal education during CBOC visits in addition to formal lectures given by experts from the BVAMC and the University of Maryland. The DEMO program was introduced to providers at the CBOCs by e-mail with follow-up information sessions provided on site.
Rather than having patients simply return to their CBOC primary care providers (PCPs) and exposing patients to the risks of poor communication/coordination of care, services were expanded to include regular phone follow-up calls with case management services that augmented those of their PCPs. The goal was to improve outcomes for these patients and provide alternatives to institutionalization.
Standardized instruments were used to gauge patient and caregiver satisfaction, obtain cost data from the VA and the Centers for Medicare & Medicaid Services, and medication data from the Pharmacy Benefits files.
The institutional review board provided approval to collect data on participants to assess the program’s clinical and economic impacts. Since all patients were suspected to have dementia, the informed consent procedures included additional protections. The patient’s understanding of the pertinent information related to participation in the study was assessed to help ensure that participants with dementia truly understood the conditions to which they were consenting. If the potential participant could not provide informed consent, it was obtained from a surrogate with durable power of attorney (the person recognized by Maryland law as the substitute decision maker or the veteran’s legal guardian). Consent was assessed on an ongoing basis regardless of the patient’s capacity to give informed consent, and those willing to have their data collected were enrolled in a “research” arm. These participants were compared with veterans in the dementia clinic during the enrollment period but who did not consent to participate in the research arm, controlling for sociodemographic characteristics and prior health care utilization.
Data Analysis
Patterns of health care utilization may fluctuate with time, and enrollment may identify potential problems that otherwise would not have been found. The authors looked at health care use over 6-month and 1-year intervals before and after enrollment, examining Occupational Physical Assessment Test (OPAT) data and fee-based outpatient data for both inpatient events (including nursing home utilization and hospitalization events) and outpatient visits (including primary, specialty and mental health care services; home care visits, and emergency department [ED]). The total cost incurred by outpatient visits and inpatient care was then adjusted to 2011 dollars. The relationship between program enrollment and health care use and costs was examined with a multivariate regression analyses, controlling for age and health care use in the prior year among veterans who were in the “consent” group or “nonconsent” group.
Outcome variables included the number of primary care, ED, specialty care, home health care, mental health care clinic, and inpatient visits; total inpatient bed days; and the total costs of all the events. The authors fit different multivariate models for these events according to their distributions. Specifically, the Poisson model was used if the distribution of the outcome variable was not overdispersed (eg, ED, primary care).
A negative binomial was used if the distribution of these events was overdispersed (eg, specialty care visits). Further, because the occurrence of inpatient events is relatively rare, a logit model was used to examine the relationship between enrollment status and probability of any inpatient events, regardless of the number of events. Generalized estimating equation (GEE) model with gamma distribution and log link function was used to examine the relationship between cost and program enrollment. The authors also examined the program’s effect on medication use, focusing on high-risk medications in older adults, comparing both the number of unique medications, as well as frequency such medications prescribed 1 year before and after the enrollment.1
Results
Two hundred ninety-eight (298) veterans were referred to DEMO from a 150-mile radius of Baltimore. Of these veterans, 132 consented to participate in this study. The study participants largely were representative of the total group as well as both the overall veteran population and the more general population of community-dwelling individuals with dementia (Table).
Veterans in the DEMO program largely had mild-to-moderate cognitive impairment with mean Mini-Mental State Examination (MMSE) score of 22 and significant functional limitations (Table). Only 3% displayed a pattern of “pure” dementia typical of Alzheimer disease, and 11% of those referred did not have significant abnormalities on neurocognitive testing.
The team averaged 10.3 recommendations (range 3-22), which focused on a diverse set of issues related to additional diagnostic and therapeutic concerns. Although screening data, including basic laboratory results and imaging, was requested in the referral form, in 71% of cases further diagnostic investigations were suggested. With regard to therapeutic suggestions, not surprisingly, medicines were cited as targets in a majority of cases (eg, discontinuing high-risk medications, initiating/titrating medications to minimize cardiovascular risk). While remaining mindful of the time to benefit and competing morbidities, measures to modify cardiovascular risk factors were suggested in more than half and treatment of depression in 15% of cases. Similarly, addressing poor sensory input was suggested in 38% of cases, with other common recommendations focusing on multiple environmental and social interventions (> 50%) as well as supports/outlets/respite for the caregivers.
The full multidisciplinary DEMO group met only weekly to review cases, and due to travel and scheduling difficulties, feedback to the patients and their families often was delayed for weeks. Although initial plans included regularly scheduled follow-up phone calls, demand quickly outstripped program resources. Nonetheless, chart reviews and abstracted adherence and utilization data revealed that PCPs successfully implemented 52% of recommendations within 2 weeks, rising to > 60% by 3 months. When patients were reevaluated at 1 year, they were remarkably stable: Mini–Mental State Examination (baseline 22.2 ± 5.0 → 22.3 ± 5.7 at follow-up) and Instrumental Activities of Daily Living scores (15.5 ± 10.6 → 17.7 ± 11.4).
Feedback
This program was enthusiastically received by both patients and their caregivers—100% and 98%, respectively—reporting overall satisfaction with the services received and 93% of caregivers indicating satisfaction with how the program met their needs. Caregivers were happy with the amount of time the provider took to answer questions (100% satisfied with the amount of time the DEMO provider spent and that they explained “what they wanted to know,” with 98% responding “good” or “great” for both), as well as with services and amount of help received (83% and 77% “very satisfied,” respectively).
In the survey, 98% of caregivers and patients felt that the program helped them deal more effectively with their problems, 97% would recommend the program to a friend in need of similar help, and 100% would come back if they were to seek help again. In keeping with DEMO’s initial aim of increasing access, there was favorable feedback on the ability to get in and be seen and convenience of location. In addition, referring providers universally expressed satisfaction with the referral process (referrals increased linearly); timeliness of scheduling; usefulness of the recommendations; and they planned on continuing to refer patients.
Although there was great variability, controlling for age and prior utilization, veterans in DEMO had statistically significant (all P < .05) fewer ED and specialty care visits and more mental health care clinic visits 183 to 365 daysafter referral dates compared with those in the nonconsented group. Veterans in the consented group also were less likely to use inpatient care than were veterans in the nonconsented group 183 to 365 days after referral dates. These trends were similarly reflected after controlling for age and prior utilization as well as when examining health care costs (data not shown). Nonetheless, DEMO did not seem to have any effect on overall inpatient bed days, primary/home-based care visits, or total costs. In fact, utilization of mental health care resources increased (Figures 1, 2, and 3).
Discussion
Cognitive issues in patients within the general population are common, and the patients cared for by the VA are no exception. Dementia is more common in rural compared with urban areas, and those living in more remote locations have reduced access to specialized evaluation, management, and support services.2 The authors describe a novel program that dramatically increased patient access, bringing the normally tertiary referral services to geographically remote CBOCs at a minimal investment. These services were well received by patients, caregivers, and PCPs. As anticipated, patients and their caregivers especially appreciated the ease and convenience of access. Considering the already significant burden(s) borne every day by those caring for patients with dementia, the benefit of this approach is evident.
Clinicians often feel uncomfortable in evaluating and managing patients with cognitive deficits. Nonetheless, the role of specialized clinics in diagnosing dementia has been demonstrated previously, and the present results are in agreement with previous studies.3 The novelty here is the provision of specialized care usually found only in large, academic medical centers to local CBOCs. By bringing specialized services to geographically isolated patients, the DEMO program was able to increase both access and utilization. Furthermore, providing coordination and ongoing, focused follow-up provided increases in satisfaction and efficiency.
An additional benefit of this approach is the opportunity for PCP education. The authors even found anecdotal reductions in ED usage as well as acute hospitalization and long-term placement—although it was not a statistical significant difference. The relatively high use of mental health care services in this population is in line with previous reports in similar populations, and greater utilization of mental health care services may be one explanation why overall costs did not differ between the 2 groups.4 Nonetheless, this intimates that such a program may yield savings over a longer term, as has been demonstrated in patients with a variety of other psychiatric diagnoses cared for in the community rather than in institutions.5,6
The prevalence of dementia and its associated costs are nearly $50,000 per year per person—suggesting a total cost in the hundreds of billions of dollars.7 Similarly, the importance of caregiver support (including psychosocial interventions, such as the one piloted here) has been demonstrated in a variety of settings (even without improvement in caregiver burden itself).8
There were a number of challenges in the rollout and delivery of DEMO. Although CBOC PCPs
Unfortunately, DEMO was underresourced to provide either real-time feedback true or first responder services. Misunderstandings concerning this were an early challenge to PCP acceptance. However, the longitudinal presence and close working relationships of the DEMO teamlet in each CBOC allowed their use as an adjunct to primary care, and increased the efficacy of both.
Limitations
A number of additional caveats must be made. First, this study had a relatively small number of participants, and there was great variability in health care utilization. This is particularly germane in this population of patients with dementia, which typically has an asymmetrically high use of health care resources. Additionally, the relatively limited follow-up period may have blunted the programs true effect(s). Further, although veterans in the nonconsented group were not officially enrolled in the program, there was likely spillover of the effects of the program on practice patterns, leading to an underestimation of the program’s impact.
Conclusion
With minimal resources, DEMO successfully brought expert evaluation (usually tertiary referral) services, and provided specialized case management in coordination with existing primary care to remote patients. Although there were a number of features rather unique to this setting (eg, infrastructural support; close working interdisciplinary and interprofessional relationships, buy-in at all levels, relative geographic density/demographic homogeneity of participants), specialized case management is increasingly being adopted throughout the VA (and elsewhere). Although the value of collaborative, interdisciplinary interventions has been shown in a variety of settings and conditions—nursing homes,9 chronic low back pain,10 safety among hospital inpatients11—its utility for dementia care is relatively underexplored.
Yet the effectiveness of team-based care for individuals has been demonstrated in a number of settings, including Alzheimer disease.12,13 In addition to involving a number of disciplines, collaborative care is marked by coordination. A number of recent systematic reviews have found that behavioral and multicomponent interventions directed towards the caregiver as well as case management were beneficial in improving some outcomes, although there is considerable heterogeneity in the effects.14,15 Future work will focus on examining methods to focus/optimize interventions based on individual patient characteristics.
Given the epidemiologic trends, care for patients with dementia is expected to grow. Novel interventions, like DEMO, are a particularly promising option to meet this challenge. In fact, just such a collaborative practice-ready workforce has been identified by the World Health Organization as crucial to meeting the challenges of the health needs in the 21st century.16 With the feasibility of such an approach in this population now evident, further studies (including larger sample sizes, across greater geographic regions, as well as among more diverse populations) should be undertaken. These results, if replicated, suggest a novel approach to the particularly vexing problem of caring for patients with dementia with potentially far-reaching public health implications.
Acknowledgments
Supported with T21 funds from VA to expand noninstitutional alternatives to institutional extended care for veterans, as well as the Geriatrics Research and Clinical Center (GRECC) at the Baltimore VAMC.
1. National Committee for Quality Assurance. 2011 HEDIS List. http://www.ncqa.org/tabid/1274/Default.aspx. Accessed December 16, 2016.
2. Russ TC, Batty GD, Hearnshaw GF, Fenton C, Starr JM. Geographical variation in dementia: systemic review with meta-analysis. Int J Epidemiol. 2012;41(4):1012-1032.
3. Wolfs CA, Kessels A, Dirksen CD, Severens JL, Verhey FR. Integrated multidisciplinary diagnostic approach for dementia care: randomized controlled trial. Br J Psychiatry. 2008;192(4):300-305.
4. King PR, Vair CL, Wade M, et al. Outpatient health care utilization in a sample of cognitively impaired veterans receiving care in VHA geriatric evaluation and management clinics. Psychol Serv. 2015;12(1):66-72.
5. Tam-Tham H, Cepoiu-Martin M, Ronksley PE, Maxwell CJ, Hemmelgarn BR. Dementia case management and risk of long-term care placement: a systemic review and meta-analysis. Int J Geriatr Psychiatr. 2013;28(9):889-902.
6. Rothbard AB, Kuno E, Schinnar AP, Hadley TR, Turk R. Service utilization and cost of community care for discharged state hospital patients: a 3-year follow-up study. Am J Psychiatry. 1999;156(6):920-927.
7. Hurd MD, Martorell P, Delavande A, Mullen KJ, Langa KM. Monetary costs of dementia in the United States. N Engl J Med. 2013;368(14):1326-1334.
8. Adelman RD, Tmanova LL, Delgado D, Dion S, Lachs MS. Caregiver burden: a clinical review. JAMA. 2014;311(10):1052-1060.
9. Nazir A, Unroe K, Tegeler M, Khan B, Azar J, Boustani M. Systematic review of interdisciplinary interventions in nursing homes. J Am Med Dir Assoc. 2013;14(7):471-478.
10. Kamper SJ, Apeldoorn AT, Chiarotto A, et al. Multidisciplinary biopsychosocial rehabilitation for chronic low back pain: Cochrane systematic review and meta-analysis. BMJ. 2015;350:h444.
11. O’Leary KJ, Buck R, Fligiel HM, et al. Structured interdisciplinary rounds in a medical teaching unit: improving patient safety. Arch Intern Med. 2011;171(7):678-684.
12. Counsell SR, Callahan CM, Clark DO, et al. Geriatric care management for low-income seniors: a randomized controlled trial. JAMA. 2007;298(22):2623-2633.
13. Callahan CM, Boustani MA, Unverzagt FW, et al. Effectiveness of collaborative care for older adults with Alzheimer disease in primary care: a randomized controlled trial. JAMA. 2006;295(18):2148-2157.
14. Health Quality Ontario. Caregiver- and patient-directed interventions for dementia: an evidence-based analysis. Ont Health Technol Assess Ser. 2008;8(4):1-98.
15. Reilly S, Miranda-Castillo C, Malouf R, et al. Case management approaches to home support for people with dementia. Cochrane Database Syst Rev. 2015;1:CD008345.
16. World Health Organization. Framework for action on interprofessional education and collaborative practice. http://apps.who.int/iris/bitstream/10665/70185/1/WHO_HRH_HPN_10.3_eng.pdf?ua=1. Published 2010. Accessed November 17, 2016.
1. National Committee for Quality Assurance. 2011 HEDIS List. http://www.ncqa.org/tabid/1274/Default.aspx. Accessed December 16, 2016.
2. Russ TC, Batty GD, Hearnshaw GF, Fenton C, Starr JM. Geographical variation in dementia: systemic review with meta-analysis. Int J Epidemiol. 2012;41(4):1012-1032.
3. Wolfs CA, Kessels A, Dirksen CD, Severens JL, Verhey FR. Integrated multidisciplinary diagnostic approach for dementia care: randomized controlled trial. Br J Psychiatry. 2008;192(4):300-305.
4. King PR, Vair CL, Wade M, et al. Outpatient health care utilization in a sample of cognitively impaired veterans receiving care in VHA geriatric evaluation and management clinics. Psychol Serv. 2015;12(1):66-72.
5. Tam-Tham H, Cepoiu-Martin M, Ronksley PE, Maxwell CJ, Hemmelgarn BR. Dementia case management and risk of long-term care placement: a systemic review and meta-analysis. Int J Geriatr Psychiatr. 2013;28(9):889-902.
6. Rothbard AB, Kuno E, Schinnar AP, Hadley TR, Turk R. Service utilization and cost of community care for discharged state hospital patients: a 3-year follow-up study. Am J Psychiatry. 1999;156(6):920-927.
7. Hurd MD, Martorell P, Delavande A, Mullen KJ, Langa KM. Monetary costs of dementia in the United States. N Engl J Med. 2013;368(14):1326-1334.
8. Adelman RD, Tmanova LL, Delgado D, Dion S, Lachs MS. Caregiver burden: a clinical review. JAMA. 2014;311(10):1052-1060.
9. Nazir A, Unroe K, Tegeler M, Khan B, Azar J, Boustani M. Systematic review of interdisciplinary interventions in nursing homes. J Am Med Dir Assoc. 2013;14(7):471-478.
10. Kamper SJ, Apeldoorn AT, Chiarotto A, et al. Multidisciplinary biopsychosocial rehabilitation for chronic low back pain: Cochrane systematic review and meta-analysis. BMJ. 2015;350:h444.
11. O’Leary KJ, Buck R, Fligiel HM, et al. Structured interdisciplinary rounds in a medical teaching unit: improving patient safety. Arch Intern Med. 2011;171(7):678-684.
12. Counsell SR, Callahan CM, Clark DO, et al. Geriatric care management for low-income seniors: a randomized controlled trial. JAMA. 2007;298(22):2623-2633.
13. Callahan CM, Boustani MA, Unverzagt FW, et al. Effectiveness of collaborative care for older adults with Alzheimer disease in primary care: a randomized controlled trial. JAMA. 2006;295(18):2148-2157.
14. Health Quality Ontario. Caregiver- and patient-directed interventions for dementia: an evidence-based analysis. Ont Health Technol Assess Ser. 2008;8(4):1-98.
15. Reilly S, Miranda-Castillo C, Malouf R, et al. Case management approaches to home support for people with dementia. Cochrane Database Syst Rev. 2015;1:CD008345.
16. World Health Organization. Framework for action on interprofessional education and collaborative practice. http://apps.who.int/iris/bitstream/10665/70185/1/WHO_HRH_HPN_10.3_eng.pdf?ua=1. Published 2010. Accessed November 17, 2016.