User login
- Most studies of asthma self-management do not permit retrospective isolation of the independent effects of a written action plan or peak flow meter use.
- Studies designed to isolate the effect of these self-care activities are generally underpowered or prone to systematic bias.
- Available evidence suggests that peak flow meters and written action plans do not have a large impact on outcomes when applied to the general population of asthmatics.
- These interventions are most likely to have beneficial effect when applied to selected populations, particularly patients with high baseline utilization.
Self-management skills are widely promoted by health plans and specialty societies with the expectation that they will improve care. The 1997 National Heart, Lung, and Blood Institute guidelines on treating asthma emphasize self-management,1 although they do not recommend specific programs. To maximize therapeutic effectiveness, it would be useful to know which components of patient self-management improve outcomes. Written action plans and peak flow meters are commonly used in asthma self-management programs. While these are simple, low-cost interventions for an individual, the aggregate cost for the entire population of asthmatics may be high.2
Much literature has accumulated on the effectiveness of providing asthma education alone and on programs that actively engage patients in their own care.Several systematic reviews have found that providing educational information alone has had little effect on asthma outcomes.3-5 There is evidence, though, that self-management activities are more effective than educational information alone. A recent Cochrane review of 24 trials found that self-management with regular practitioner review reduces hospi-talizations and emergency room visits.6 This review did not identify specific components contributing to improved outcomes. In contrast to the aforementioned studies on patient education, a large case-control study of children in the Kaiser Permanente System,7 found that written action plans were associated with lower rates of hospitalization and emergency room use. However, such observational studies often include confounding factors and are not sufficient to establish a cause-effect relationship between written action plans and improved outcomes.
We report on a systematic review that attempts to isolate the independent effect of a written action plan on asthma outcomes. We address two key questions:
- Compared with medical management alone, does the addition of a written asthma action plan (with or without peak flow meter use) improve outcomes?
- Compared with a written action plan based on symptoms, does a written action plan based on peak flow monitoring improve outcomes?
Methods
This study is part of a broader evidence report on the management of chronic asthma prepared for the Agency of Health Care Research and Quality8. Complete details of the methodology are available in the full report8 (http://www.ahcpr.gov/clinic/epcix.htm).
Literature search and study selection
We performed a comprehensive literature search from 1980 to August 2000 using MEDLINE, Embase, the Cochrane Library, and a hand search of recent bibliographies. The search was limited to full-length, peer-reviewed articles with an English abstract. Two independent reviewers carried out each step of study selection and data abstraction. Disagreements were resolved by consensus of the two reviewers or, if necessary, by the decision of a third reviewer.
Initial study selection was limited to comparative full-length reports or abstracts in peer-reviewed medical journals, with at least 25 evaluable children or adults per arm, treated for at least 12 weeks. Relevant comparisons included a written action plan and no written action plan; a written action plan based on peak flow readings and a written action plan based on symptoms. Study designs varied: clinical trials, cohort comparisons, case-control analyses, cross-sectional evaluations, and before-after comparisons. Specific components of the management plan had to be described.
Relevant outcomes included measures of inpatient and outpatient utilization, lung function, symptoms, rescue medication or oral steroid use, and quality of life. Outcomes of greatest interest were utilization parameters, as the goals of self-management usually focus on improving these outcomes.
These initial selection criteria yielded many studies that were confounded by multiple asthma management interventions and thus did not isolate the comparisons of interest. Therefore, the research team collectively determined the study design features that would best isolate the effects of written action plans and used them as new criteria in a second round of study selection. The studies thus selected satisfied 4 criteria: 1) randomization of patients; (2) delivery of the same interventions to experimental and control groups, except that the experimental group also received a written action plan; (3) delivery of the same interventions to experimental and control groups, except that one group received a written action plan based on peak flow meter readings, and the comparison group received a written action plan based on symptom monitoring; and 4) inclusion of a written action plan that met our specified definition.
A written action plan, by our definition, had two components: an algorithm that identified specific clinical indicators signaling the need for adjustments in medication; and specific instructions on how to adjust medications in response to such indicators. Many publications lacked sufficient detail on the written plan, so a brief survey was sent to the primary author of each of the 36 studies. If no response was obtained (36%), the article was excluded only when it was clear from the publication that our definition was not met.
Assessment of study quality
High-quality studies were randomized controlled trials that met the 3 domains of study quality that have been demonstrated empirically to impact effect size: concealment of treatment allocation; double-blinding; and minimization of exclusion bias.9,10 However, we doubted the feasibility of double-blinding a written asthma plan intervention, and so relaxed this requirement. We considered exclusion bias to be minimized when a study either reported intent-to-treat analysis or excluded fewer than 10% of subjects from analysis, with the ratio of subjects excluded from each arm being less than 2:1.
To more fully evaluate study design issues that may be particularly important in asthma research,11,12 we constructed asthma-specific quality indicators in consultation with an expert panel. Controls for potential confounders of treatment effect included establishing reversibility of airway obstruction, controlling for other medication use, reporting compliance, and addressing seasonality. In addition, a priori reporting of power calculations and accounting for exclusions and withdrawals were judged to be study quality characteristics pertinent to this body of evidence.
Data analysis
We constructed evidence tables for the outcomes of interest, and performed a qualitative synthesis of the data. Meta-analysis was not appropriate due to wide discrepancies in the patient populations studied, the interventions employed, and measurement and reporting of outcomes.
Results
Our literature search yielded a total of 4578 citations. Of these, 36 studies met the initial selection criteria. Many of these qualifying studies, however, were confounded by multiple asthma management interventions applied inconsistently across treatment arms. For example, a common confounder was review of and change in long-term medication use in the treatment group, but not in the control group. This necessitated a refinement in our selection criteria to focus on studies that largely isolated the effect of written action plans.13-21 This step yielded a final evidence base of 9 randomized controlled trials with a total enrollment of 1501 patients.
Table 1 summarizes the characteristics, interventions, and outcomes of the 9 studies. Two studies were 3-arm trials,16,17 which raised the total number of comparisons among the 9 studies to 11. The largest study was the Grampian Asthma Study of Integrated Care (n=569),14 a community study conducted in the UK. Enrollment in the other 8 studies ranged from 43 to 64 patients per arm. Treatment duration ranged from 24 to 52 weeks.
None of the studies met our definition of high quality. In fact, no study met any of the generic quality criteria—none was blinded, none described concealment of allocation, and all excluded more than 10% of subjects. Furthermore, none reported an intention-to-treat analysis. Thus these trials were prone to withdrawal bias as well as overestimation of treatment effect due to lack of allocation concealment.
No study met the majority of asthma-specific indicators (Table 1). Of the 9 studies, only 5 met any asthma-specific indicator. Three reported prospective power calculations,13-15 but 2 of these substantially overestimated the expected effect.13,15 Two studies established reversibility;14,17 2 controlled for other medication use;13,15 and 2 reported compliance.17,21 Thus, the studies were also prone to a type II error (failing to detect a true effect) and to potential confounding of outcomes.
We performed sample power calculations for hospitalizations (Table 2), derived from baseline rates reported in 4 studies14,16-18 and standard deviations reported in 2.14,17 A study with 250 patients per arm could detect a reduction of 50% or more in hospitalization, given a control rate of at least 0.2 hospitalizations/patient/year. In actuality, GRASSIC,14 which is the largest available trial (N=569), had baseline hospitalization rates of 0.12 and 0.13. With this baseline rate, over 700 patients per arm are required, higher than the actual enrollment in GRASSIC. The other studies in this review would be adequately powered to detect a 50% difference only in the setting of even higher baseline utilization (eg, 0.30 hospitalizations/patient/year).
Table 3 displays utilization outcomes for the 11 comparisons in the 9 trials. In 5 studies (N=1019), medical management with a written action plan was compared with medical management without a written action plan.13-17 Two trials (N=185) compared a peak flow meter plus a written action plan with a peak flow meter and no written action plan18,19 In 4 studies (N=393), a written action plan based on peak flow monitoring was compared with a written action plan based on symptoms.
TABLE 1
Study characteristics
| Study | Patient popultation | Study Arms | Intervention components | Outcomes reported | Asthma quality indicators met |
|---|---|---|---|---|---|
| Optimal medical management vs. optimal medical management + PFM action plan | |||||
| Jones 199514 | Inclusions: patients using ICS | Usual care | SxD, FU | Ut, LF, Sx | Pow, Med |
| Exclusions: patients on oral steroids or using peak flow meters at home | PFM action plan | AP, PF, SxD, FU | |||
| Mean age: 29.5 years | |||||
| Severity level: Mild–moderate | |||||
| Drummond 1994 (GRASSIC)15 | Inclusion: FEV1 reversibility 20% or greater | Usual care | FU | Ut, LF, Med Ex | Pow, Rev |
| Exclusions: patients who already owned a PFM | PFM action plan | AP, PF, FU | |||
| Mean age: 50.8 years | |||||
| Severity level: Mild–severe | |||||
| Ayres 199516 | Inclusions: maximum PEF variability, 0.15%; minimum nights/week with symptoms, 3; minimum use of ICS or sodium cromoglycate, 3 months | Usual care | SxD, FU | LF, Sx, Ex | Pow, Med |
| Mean age: 45 years | PFM action plan | AP, PF, SxD, FU | |||
| Severity level: Moderate–severe | |||||
| Cowie 199713 | Inclusions: treatment for an exacerbation of asthma in an ER asthma clinic; history of receiving urgent treatment for asthma in the previous 12 months | Usual care | Ed, SxD, FU | Ut, PF, Med, Ex | None |
| Mean age: 37.8 years | PFM action plan | AP, PF, Ed, SxD, FU | |||
| Severity level: Mild–severe | |||||
| Cote 199717 | Inclusions: FEV1postbronchodilator 85-100 % of predicted; PEF, at minimum, 85 % of predicted; minimum PEF variability, 0%; Methacholine | Usual care | Ed | Ut, LF, Med | Exc, Rev, Com |
| Exclusions: patients having previously taken an asthma educational program | PFM action plan | Ed, Cn, AP, PF | |||
| Mean age: 36.5 years Severity level: Mild | |||||
| Usual care + PFM use alone vs. usual care + PFM action plan | |||||
| Ignacio-Garcia 199518 | Inclusions: patients from outpatient asthma clinic with asthma for 2 years | Usual care + PFM | PF, SxD, FU | Ut, LF, Med | None |
| Mean age: 41.9 years | Usual care + PFM action plan | PF, AP, Ed, SxD, FU | |||
| Severity level: Mild–severe | |||||
| Charlton 199419 | Inclusion: patients with inpatient or outpatient visit for asthma | Usual care + PFM | PF, Ed, SxD, FU | Ut, Sx, Med, Ex | None |
| Mean age: 6.5 years | Usual care + PFM action plan | PF, AP, Ed, SxD, FU | |||
| Severity level: Mild–moderate | |||||
| PFM action plan vs. Symptom action plan | |||||
| Turner 199820 | Inclusions: Maximum methacholine PC20, 7.9; using ICS | Symptom action plan | AP, Ed, SxD, Cn BM, EM | Ut, LF, Sx, Med | Exc, Com |
| Exclusions: previous PFM use; significant comorbid conditions | PFM action plan | PF, AP, Ed, SxD, Cn BM, EM | |||
| Mean age: 34.1 years | |||||
| Severity level: Mild–severe | |||||
| Charlton 199021 | Inclusions: patients on repeat prescribing register | Symptom action plan | AP, Ed, FU | Ut, Med | None |
| Mean age: NR | PFM action plan | PF, AP, Ed, FU, Cn | Ut, PF, Med, Ex | None | |
| Severity level: Mild–severe (?) | |||||
| Cowie 199716 | Inclusions: treatment for an exacerbation of asthma in an ER, or asthma clinic; history of receiving urgent treatment for asthma in the previous 12 months | Symptom action plan | AP, Ed, SxD, FU | ||
| PFM action plan | AP, PF, Ed, SxD, FU | ||||
| Cote 199717 | Inclusions: FEV1postbronchodilator, 85-100 % of predicted; PEF, at minimum, 85 % of predicted; minimum PEF variability, 0%; Methacholine | Symptom action plan | Ed, AP | Ut, LF, Med | Exc, Rev, Com |
| Exclusions: previous enrollment in an asthma educational program | PFM action plan | Ed, Cn, AP, PF | |||
| Eligibility criteria: ICS = inhaled corticosteroid; FEV1 = forced expiratory volume in 1 second; PEF = peak expiratory flow; PFM = peak flow meter; ER = emergency room; PC20 = 20% fall in FEV1 Intervention components: PF = Peak flow meter; AP = Written Action Plan; Ed = Education; SxD = Symptom diary; FU = Follow-up visits; Cn = Counseling; BM = Behavior modification; EM = Environmental modification | |||||
| Outcomes: Ut = Utilization measures; LF= Lung function measurements; Sx = Symptom=based measurements; Med = Medication use; Ex = Exacerbations of asth ma Asthma Quality Indicators: Exc = Accounted for excluded patients; Pow = Reported power calculations; Rev = Established reversibility of airway obstruction; Med = Controlled for other medication use; Com = Reported compliance; Sea = Addressed seasonality. | |||||
TABLE 2
Power calculations for hospitalizations per patient per year
| Assumed control mean | Possible treatment mean | % decrease | N needed per study arm |
|---|---|---|---|
| 0.10 | 0.075 | 25 | 3077 |
| 0.10 | 0.05 | 50 | 770 |
| 0.10 | 0.025 | 75 | 342 |
| 0.20 | 0.015 | 25 | 770 |
| 0.20 | 0.10 | 50 | 193 |
| 0.20 | 0.05 | 75 | 86 |
| 0.30 | 0.225 | 25 | 342 |
| 0.30 | 0.15 | 50 | 86 |
| 0.30 | 0.075 | 75 | 38 |
| Studies were identified that contained baseline rates on hospitalizations/patient/year, or information that allowed calculation of this parameter (Drummond, Abdalla, Beattie et al., 1994; Cote, Cartier, Robichaud et al. 1997; Cowie, Revitt, Underwood et al., 1997; Ignacio-Garcia and Gonzalez-Santos, 1995). Baseline rates of hospitalization varied in these studies from 0.04-0.29/patient/year. Standard deviations for this outcome were available only in two studies; Cote, Cartier, Robichaud et al. (1997) reported an SD of 0.30 for this variable, and an SD of 0.35 was calculated from the confidence intervals reported in GRASSIC (Drummond, Abdalla, Beattie et al., 1994). For the calculations, the more conservative 0.35 estimate for SD was used. | |||
| Number of patients per study arm were estimated for 80 percent power at the 5 percent significance level using control arm means of 0.10, 0.20, and 0.30 hospitalizations/patient/year. The expected reduction in this variable was tested along a spectrum from 25-75 percent. | |||
Written action plan versus no written action plan
All 5 studies used a peak flow meter based written action plan. All reported utilization outcomes, but the types and units of measurement were not consistent across studies (Table 2). Additionally, 4 studies reported on symptoms,13-16 and 3 reported lung function outcomes.13-15
With one notable exception, there were no statistically significant differences in outcomes among groups. Cowie et al16 reported an 11-fold decrease in total emergency room visits for the group using a peak-flow action plan (5 vs 55, P = .02), and also reported a reduction in hospitalizations of a similar magnitude (2 vs 12) that did not reach statistical significance. However, this study suffers from notable flaws that diminish confidence in the results. It is a post-intervention comparison among groups, which does not compare change from baseline, or incorporate baseline values as covariates in the analysis. Moreover baseline utilization data were provided by patient recall and not corroborated by medical records. There was a substantially larger variability in the baseline utilization rates for the peak flow group compared with the control group. This suggests that a subset of very high frequency users may have been over-represented in the peak flow group, and the reduction in emergency room visits may be concentrated in this subset.
Peak-flow meter-based written action plan versus peak flow meter with no written action plan
Two studies18,19 addressed the independent effect of a written action plan when added to peak flow self-monitoring (Table 3). Charlton19 reported no significant group differences for main outcomes, while Ignacio-Garcia18 reported large and statistically significant differences in most of the outcomes, favoring the group that used the written action plan.
The Ignacio-Garcia study, however, suffers from notable flaws suggesting the results may be attributable to bias. The sole participating physician, not blinded to treatment assignment, was highly involved in all phases of patient assessment, monitoring, and treatment. There was evidence of baseline differences between the two groups. A total of 25% of patients were withdrawn after randomization, and an unexplained decline in lung function occurred in the control group. Thus, the potential for selection bias, withdrawal bias, and ascertainment bias limits confidence in the results of this study
Symptom-based written action plan compared with peak flow-based written action plan
In 4 studies,16,17,20,21 reported outcomes were generally equivalent between groups and comparisons were not statistically significant, with one exception (Table 3). The 3-arm study by Cowie et al16 reported a striking reduction in the total number of emergency room visits with a peak flow meter-based written action plan compared with a symptom-based written action plan (5 versus 45, P
Discussion
The objective of this systematic review was to assess the independent effects of 2 specific components commonly included in asthma self-management plans—a written action plan and a peak flow meter. Few studies, however, are designed to permit reviewers to isolate the effects of these components. Moreover, the studies we reviewed did not clearly identify the population expected to benefit from interventions or specify the primary outcomes of interest; nor was the level of clinically meaningful improvement prospectively defined.
Most of the trials we reviewed, including the largest community study of 569 patients, did not demonstrate improved outcomes. The 2 trials that reported statistically significant results favoring a peak flow-based written action plan suffer from notable flaws suggesting the results may be attributable to bias. In the other 7 trials, there was little difference in outcomes between groups. However, these studies had insufficient power to detect group differences or confidently conclude equivalence between groups.
Thus, available evidence is insufficient to demonstrate that asthma outcomes are improved by use of a written asthma action plan, with or without peak flow monitoring. While this body of literature does not establish that these interventions are ineffective, it suggests they will not have a large effect on outcomes when applied to the general asthmatic population. The application of written action plans to all asthmatics indiscriminately may be a wasteful use of resources. This systematic review also questions the validity of written action plans as an indicator of asthma quality of care, or as a means to achieve quality improvement.
This analysis also highlights several obstacles to assessing the effects of disease management interventions. First, while the impact of whole intervention programs can be evaluated in controlled trials, it may be unfeasible to isolate each component of such programs and subject it to a rigorous analysis. Furthermore, as a behavioral intervention, the general principle of engaging patients in self-management may be more important that the specific components of these programs. Finally, regarding the optimization of medications (most obviously initiation of inhaled steroids) the impact of written action plans is likely to be relatively small, particularly on lung function or symptom control.
Future clinical trials should be done selectively, aimed at producing rigorous results that can improve the effectiveness of self-management interventions. Further study is warranted for specific subpopulations, such as those with higher baseline severity of illness or those with high baseline utilization rates. Available data suggest that, if there is benefit to be gained from self-management interventions, it will most likely be seen among these patients. Specific components of self-management that might be tested individually are those that are relatively high-cost, resource intensive, or risky for the patient.
Existing trials have tended to over-estimate the effects of action plan-based interventions, thus having invested resources for results inadequate for optimizing self-management strategies. Careful consideration needs to be taken in future trials to realistically estimate the expected impact of each intervention, and to specify the primary outcomes of interest and their baseline frequencies. Future trials should be large enough to detect a difference if one exists, or to confidently conclude that the intervention is ineffective.
Attention to these principles will help to advance our knowledge in this area most efficiently and to ultimately improve the quality of care for the entire population of patients with asthma.
· Acknowledgments ·
We acknowledge Kathleen Ziegler, Pharm.D, and Claudia Bonnell, RN, MSL, for their assistance in the research and preparation of this manuscript.
1. National Heart, Lung and Blood Institute. Expert panel report 2: guidelines for the diagnosis and management of asthma. Bethesda, MD: National Institutes of Health; 1997. NIH publication 97-4051.
2. Ruffin RE, Pierce RJ. Peak flow monitoring—which asthmatics, when, and how? Aust N Z J Med 1994;24:519-20.
3. Devine EC. Meta-analysis of the effects of psychoeducational care in adults with asthma. Res Nursing Health 1996;19:367-76.
4. Bernard-Bonnin AC, Stachenko S, Bonin D, et al. Self-management teaching programs and morbidity of pediatric asthma: a meta-analysis. J Allergy Clin Immunol 1995;95(1 Pt 1):34-41.
5. Gibson PG, Coughlan J, Wilson AJ, et al. Limited (information only) patient education programs for adults with asthma. Cochrane Database Syst Rev 2000a;(2):CD001005.-
6. Gibson PG, Coughlan J, Wilson AJ, et al. Self-management education and regular practitioner review for adults with asthma. Cochrane Database Syst Rev 2000b (2):CD001117.-
7. Lieu TA, Quesenberry CP, Jr, Capra AM, et al. Outpatient management practices associated with reduced risk of pediatric asthma hospitalization and emergency department visits. Pediatrics 1997;100(3 Pt 1):334-41.
8. Lefevre F, Piper M, Mark D, et al. Management of Chronic Asthma. AHRQ evidence report, contract number 290-97-001-5, 2001, http://www.ahcpr.gov/clinic/epcix.htm.
9. Mulrow CD, Oxman AD, editors. Cochrane Collaboration Handbook. Available in the Cochrane Library [database on disk and CD-ROM]. The Cochrane Collaboration; Issue 1. Oxford: Update Software; 1997.
10. Schulz KF, Chalmers I, Hayes RJ, et al. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273(5):408-12.
11. Berlin JA, Rennie D. Measuring the quality of trials: the quality of the quality scales. JAMA 1999;282(11):1083-5.
12. Juni P, Witschi A, Bloch R, et al. The hazards of scoring the quality of clinical trials for meta-analysis. JAMA 1999;282(11):1054-60.
13. Jones KP, Mullee MA, Middleton M, et al. Peak flow based asthma self-management: a randomised controlled study in general practice. British Thoracic Society Research Committee. Thorax 1995;50(8):851-7.
14. Drummond N, Abdalla M, Beattie JAG, et al. Effectiveness of routine self monitoring of peak flow in patients with asthma. Grampian Asthma Study of Integrated Care GRASSIC). BMJ 1994 Feb. 26;308(6928):564-7.
15. Ayres JG, Campbell LM. A controlled assessment of an asthma self-management plan involving a budesonide dose regimen. OPTIONS Research Group. Eur Respir J 1996;886-92.
16. Cowie RL, Revitt SG, Underwood MF, et al. The effect of a peak flow-based action plan in the prevention of exacerbations of asthma. Chest 1997;112(6):1534-8.
17. Cote J, Cartier A, Robichaud P, et al. Influence on asthma morbidity of asthma education programs based on self-management plans following treatment optimization. Am J Respir Crit Care Med 1997;155(5):1509-14.
18. Ignacio-Garcia JM, Gonzalez-Santos P. Asthma self-management education program by home monitoring of peak expiratory flow. Am J Respir Crit Care Med 1995;151(2 Pt 1):353-9.
19. Charlton I, Antoniou AG, Atkinson J, et al. Asthma at the interface: bridging the gap between general practice and a district general hospital. Arch Dis Child 1994;70(4):313-8.
20. Turner MO, Taylor D, Bennett R, et al. A randomized trial comparing peak expiratory flow and symptom self-management plans for patients with asthma attending a primary care clinic. Am J Respir Crit Care Med 1998;157(2):540-6.
21. Charlton I, Charlton G, Broomfield J, et al. Evaluation of peak flow and symptoms only self-management plans for control of asthma in general practice. BMJ 1990;301(6765):1355-9.
- Most studies of asthma self-management do not permit retrospective isolation of the independent effects of a written action plan or peak flow meter use.
- Studies designed to isolate the effect of these self-care activities are generally underpowered or prone to systematic bias.
- Available evidence suggests that peak flow meters and written action plans do not have a large impact on outcomes when applied to the general population of asthmatics.
- These interventions are most likely to have beneficial effect when applied to selected populations, particularly patients with high baseline utilization.
Self-management skills are widely promoted by health plans and specialty societies with the expectation that they will improve care. The 1997 National Heart, Lung, and Blood Institute guidelines on treating asthma emphasize self-management,1 although they do not recommend specific programs. To maximize therapeutic effectiveness, it would be useful to know which components of patient self-management improve outcomes. Written action plans and peak flow meters are commonly used in asthma self-management programs. While these are simple, low-cost interventions for an individual, the aggregate cost for the entire population of asthmatics may be high.2
Much literature has accumulated on the effectiveness of providing asthma education alone and on programs that actively engage patients in their own care.Several systematic reviews have found that providing educational information alone has had little effect on asthma outcomes.3-5 There is evidence, though, that self-management activities are more effective than educational information alone. A recent Cochrane review of 24 trials found that self-management with regular practitioner review reduces hospi-talizations and emergency room visits.6 This review did not identify specific components contributing to improved outcomes. In contrast to the aforementioned studies on patient education, a large case-control study of children in the Kaiser Permanente System,7 found that written action plans were associated with lower rates of hospitalization and emergency room use. However, such observational studies often include confounding factors and are not sufficient to establish a cause-effect relationship between written action plans and improved outcomes.
We report on a systematic review that attempts to isolate the independent effect of a written action plan on asthma outcomes. We address two key questions:
- Compared with medical management alone, does the addition of a written asthma action plan (with or without peak flow meter use) improve outcomes?
- Compared with a written action plan based on symptoms, does a written action plan based on peak flow monitoring improve outcomes?
Methods
This study is part of a broader evidence report on the management of chronic asthma prepared for the Agency of Health Care Research and Quality8. Complete details of the methodology are available in the full report8 (http://www.ahcpr.gov/clinic/epcix.htm).
Literature search and study selection
We performed a comprehensive literature search from 1980 to August 2000 using MEDLINE, Embase, the Cochrane Library, and a hand search of recent bibliographies. The search was limited to full-length, peer-reviewed articles with an English abstract. Two independent reviewers carried out each step of study selection and data abstraction. Disagreements were resolved by consensus of the two reviewers or, if necessary, by the decision of a third reviewer.
Initial study selection was limited to comparative full-length reports or abstracts in peer-reviewed medical journals, with at least 25 evaluable children or adults per arm, treated for at least 12 weeks. Relevant comparisons included a written action plan and no written action plan; a written action plan based on peak flow readings and a written action plan based on symptoms. Study designs varied: clinical trials, cohort comparisons, case-control analyses, cross-sectional evaluations, and before-after comparisons. Specific components of the management plan had to be described.
Relevant outcomes included measures of inpatient and outpatient utilization, lung function, symptoms, rescue medication or oral steroid use, and quality of life. Outcomes of greatest interest were utilization parameters, as the goals of self-management usually focus on improving these outcomes.
These initial selection criteria yielded many studies that were confounded by multiple asthma management interventions and thus did not isolate the comparisons of interest. Therefore, the research team collectively determined the study design features that would best isolate the effects of written action plans and used them as new criteria in a second round of study selection. The studies thus selected satisfied 4 criteria: 1) randomization of patients; (2) delivery of the same interventions to experimental and control groups, except that the experimental group also received a written action plan; (3) delivery of the same interventions to experimental and control groups, except that one group received a written action plan based on peak flow meter readings, and the comparison group received a written action plan based on symptom monitoring; and 4) inclusion of a written action plan that met our specified definition.
A written action plan, by our definition, had two components: an algorithm that identified specific clinical indicators signaling the need for adjustments in medication; and specific instructions on how to adjust medications in response to such indicators. Many publications lacked sufficient detail on the written plan, so a brief survey was sent to the primary author of each of the 36 studies. If no response was obtained (36%), the article was excluded only when it was clear from the publication that our definition was not met.
Assessment of study quality
High-quality studies were randomized controlled trials that met the 3 domains of study quality that have been demonstrated empirically to impact effect size: concealment of treatment allocation; double-blinding; and minimization of exclusion bias.9,10 However, we doubted the feasibility of double-blinding a written asthma plan intervention, and so relaxed this requirement. We considered exclusion bias to be minimized when a study either reported intent-to-treat analysis or excluded fewer than 10% of subjects from analysis, with the ratio of subjects excluded from each arm being less than 2:1.
To more fully evaluate study design issues that may be particularly important in asthma research,11,12 we constructed asthma-specific quality indicators in consultation with an expert panel. Controls for potential confounders of treatment effect included establishing reversibility of airway obstruction, controlling for other medication use, reporting compliance, and addressing seasonality. In addition, a priori reporting of power calculations and accounting for exclusions and withdrawals were judged to be study quality characteristics pertinent to this body of evidence.
Data analysis
We constructed evidence tables for the outcomes of interest, and performed a qualitative synthesis of the data. Meta-analysis was not appropriate due to wide discrepancies in the patient populations studied, the interventions employed, and measurement and reporting of outcomes.
Results
Our literature search yielded a total of 4578 citations. Of these, 36 studies met the initial selection criteria. Many of these qualifying studies, however, were confounded by multiple asthma management interventions applied inconsistently across treatment arms. For example, a common confounder was review of and change in long-term medication use in the treatment group, but not in the control group. This necessitated a refinement in our selection criteria to focus on studies that largely isolated the effect of written action plans.13-21 This step yielded a final evidence base of 9 randomized controlled trials with a total enrollment of 1501 patients.
Table 1 summarizes the characteristics, interventions, and outcomes of the 9 studies. Two studies were 3-arm trials,16,17 which raised the total number of comparisons among the 9 studies to 11. The largest study was the Grampian Asthma Study of Integrated Care (n=569),14 a community study conducted in the UK. Enrollment in the other 8 studies ranged from 43 to 64 patients per arm. Treatment duration ranged from 24 to 52 weeks.
None of the studies met our definition of high quality. In fact, no study met any of the generic quality criteria—none was blinded, none described concealment of allocation, and all excluded more than 10% of subjects. Furthermore, none reported an intention-to-treat analysis. Thus these trials were prone to withdrawal bias as well as overestimation of treatment effect due to lack of allocation concealment.
No study met the majority of asthma-specific indicators (Table 1). Of the 9 studies, only 5 met any asthma-specific indicator. Three reported prospective power calculations,13-15 but 2 of these substantially overestimated the expected effect.13,15 Two studies established reversibility;14,17 2 controlled for other medication use;13,15 and 2 reported compliance.17,21 Thus, the studies were also prone to a type II error (failing to detect a true effect) and to potential confounding of outcomes.
We performed sample power calculations for hospitalizations (Table 2), derived from baseline rates reported in 4 studies14,16-18 and standard deviations reported in 2.14,17 A study with 250 patients per arm could detect a reduction of 50% or more in hospitalization, given a control rate of at least 0.2 hospitalizations/patient/year. In actuality, GRASSIC,14 which is the largest available trial (N=569), had baseline hospitalization rates of 0.12 and 0.13. With this baseline rate, over 700 patients per arm are required, higher than the actual enrollment in GRASSIC. The other studies in this review would be adequately powered to detect a 50% difference only in the setting of even higher baseline utilization (eg, 0.30 hospitalizations/patient/year).
Table 3 displays utilization outcomes for the 11 comparisons in the 9 trials. In 5 studies (N=1019), medical management with a written action plan was compared with medical management without a written action plan.13-17 Two trials (N=185) compared a peak flow meter plus a written action plan with a peak flow meter and no written action plan18,19 In 4 studies (N=393), a written action plan based on peak flow monitoring was compared with a written action plan based on symptoms.
TABLE 1
Study characteristics
| Study | Patient popultation | Study Arms | Intervention components | Outcomes reported | Asthma quality indicators met |
|---|---|---|---|---|---|
| Optimal medical management vs. optimal medical management + PFM action plan | |||||
| Jones 199514 | Inclusions: patients using ICS | Usual care | SxD, FU | Ut, LF, Sx | Pow, Med |
| Exclusions: patients on oral steroids or using peak flow meters at home | PFM action plan | AP, PF, SxD, FU | |||
| Mean age: 29.5 years | |||||
| Severity level: Mild–moderate | |||||
| Drummond 1994 (GRASSIC)15 | Inclusion: FEV1 reversibility 20% or greater | Usual care | FU | Ut, LF, Med Ex | Pow, Rev |
| Exclusions: patients who already owned a PFM | PFM action plan | AP, PF, FU | |||
| Mean age: 50.8 years | |||||
| Severity level: Mild–severe | |||||
| Ayres 199516 | Inclusions: maximum PEF variability, 0.15%; minimum nights/week with symptoms, 3; minimum use of ICS or sodium cromoglycate, 3 months | Usual care | SxD, FU | LF, Sx, Ex | Pow, Med |
| Mean age: 45 years | PFM action plan | AP, PF, SxD, FU | |||
| Severity level: Moderate–severe | |||||
| Cowie 199713 | Inclusions: treatment for an exacerbation of asthma in an ER asthma clinic; history of receiving urgent treatment for asthma in the previous 12 months | Usual care | Ed, SxD, FU | Ut, PF, Med, Ex | None |
| Mean age: 37.8 years | PFM action plan | AP, PF, Ed, SxD, FU | |||
| Severity level: Mild–severe | |||||
| Cote 199717 | Inclusions: FEV1postbronchodilator 85-100 % of predicted; PEF, at minimum, 85 % of predicted; minimum PEF variability, 0%; Methacholine | Usual care | Ed | Ut, LF, Med | Exc, Rev, Com |
| Exclusions: patients having previously taken an asthma educational program | PFM action plan | Ed, Cn, AP, PF | |||
| Mean age: 36.5 years Severity level: Mild | |||||
| Usual care + PFM use alone vs. usual care + PFM action plan | |||||
| Ignacio-Garcia 199518 | Inclusions: patients from outpatient asthma clinic with asthma for 2 years | Usual care + PFM | PF, SxD, FU | Ut, LF, Med | None |
| Mean age: 41.9 years | Usual care + PFM action plan | PF, AP, Ed, SxD, FU | |||
| Severity level: Mild–severe | |||||
| Charlton 199419 | Inclusion: patients with inpatient or outpatient visit for asthma | Usual care + PFM | PF, Ed, SxD, FU | Ut, Sx, Med, Ex | None |
| Mean age: 6.5 years | Usual care + PFM action plan | PF, AP, Ed, SxD, FU | |||
| Severity level: Mild–moderate | |||||
| PFM action plan vs. Symptom action plan | |||||
| Turner 199820 | Inclusions: Maximum methacholine PC20, 7.9; using ICS | Symptom action plan | AP, Ed, SxD, Cn BM, EM | Ut, LF, Sx, Med | Exc, Com |
| Exclusions: previous PFM use; significant comorbid conditions | PFM action plan | PF, AP, Ed, SxD, Cn BM, EM | |||
| Mean age: 34.1 years | |||||
| Severity level: Mild–severe | |||||
| Charlton 199021 | Inclusions: patients on repeat prescribing register | Symptom action plan | AP, Ed, FU | Ut, Med | None |
| Mean age: NR | PFM action plan | PF, AP, Ed, FU, Cn | Ut, PF, Med, Ex | None | |
| Severity level: Mild–severe (?) | |||||
| Cowie 199716 | Inclusions: treatment for an exacerbation of asthma in an ER, or asthma clinic; history of receiving urgent treatment for asthma in the previous 12 months | Symptom action plan | AP, Ed, SxD, FU | ||
| PFM action plan | AP, PF, Ed, SxD, FU | ||||
| Cote 199717 | Inclusions: FEV1postbronchodilator, 85-100 % of predicted; PEF, at minimum, 85 % of predicted; minimum PEF variability, 0%; Methacholine | Symptom action plan | Ed, AP | Ut, LF, Med | Exc, Rev, Com |
| Exclusions: previous enrollment in an asthma educational program | PFM action plan | Ed, Cn, AP, PF | |||
| Eligibility criteria: ICS = inhaled corticosteroid; FEV1 = forced expiratory volume in 1 second; PEF = peak expiratory flow; PFM = peak flow meter; ER = emergency room; PC20 = 20% fall in FEV1 Intervention components: PF = Peak flow meter; AP = Written Action Plan; Ed = Education; SxD = Symptom diary; FU = Follow-up visits; Cn = Counseling; BM = Behavior modification; EM = Environmental modification | |||||
| Outcomes: Ut = Utilization measures; LF= Lung function measurements; Sx = Symptom=based measurements; Med = Medication use; Ex = Exacerbations of asth ma Asthma Quality Indicators: Exc = Accounted for excluded patients; Pow = Reported power calculations; Rev = Established reversibility of airway obstruction; Med = Controlled for other medication use; Com = Reported compliance; Sea = Addressed seasonality. | |||||
TABLE 2
Power calculations for hospitalizations per patient per year
| Assumed control mean | Possible treatment mean | % decrease | N needed per study arm |
|---|---|---|---|
| 0.10 | 0.075 | 25 | 3077 |
| 0.10 | 0.05 | 50 | 770 |
| 0.10 | 0.025 | 75 | 342 |
| 0.20 | 0.015 | 25 | 770 |
| 0.20 | 0.10 | 50 | 193 |
| 0.20 | 0.05 | 75 | 86 |
| 0.30 | 0.225 | 25 | 342 |
| 0.30 | 0.15 | 50 | 86 |
| 0.30 | 0.075 | 75 | 38 |
| Studies were identified that contained baseline rates on hospitalizations/patient/year, or information that allowed calculation of this parameter (Drummond, Abdalla, Beattie et al., 1994; Cote, Cartier, Robichaud et al. 1997; Cowie, Revitt, Underwood et al., 1997; Ignacio-Garcia and Gonzalez-Santos, 1995). Baseline rates of hospitalization varied in these studies from 0.04-0.29/patient/year. Standard deviations for this outcome were available only in two studies; Cote, Cartier, Robichaud et al. (1997) reported an SD of 0.30 for this variable, and an SD of 0.35 was calculated from the confidence intervals reported in GRASSIC (Drummond, Abdalla, Beattie et al., 1994). For the calculations, the more conservative 0.35 estimate for SD was used. | |||
| Number of patients per study arm were estimated for 80 percent power at the 5 percent significance level using control arm means of 0.10, 0.20, and 0.30 hospitalizations/patient/year. The expected reduction in this variable was tested along a spectrum from 25-75 percent. | |||
Written action plan versus no written action plan
All 5 studies used a peak flow meter based written action plan. All reported utilization outcomes, but the types and units of measurement were not consistent across studies (Table 2). Additionally, 4 studies reported on symptoms,13-16 and 3 reported lung function outcomes.13-15
With one notable exception, there were no statistically significant differences in outcomes among groups. Cowie et al16 reported an 11-fold decrease in total emergency room visits for the group using a peak-flow action plan (5 vs 55, P = .02), and also reported a reduction in hospitalizations of a similar magnitude (2 vs 12) that did not reach statistical significance. However, this study suffers from notable flaws that diminish confidence in the results. It is a post-intervention comparison among groups, which does not compare change from baseline, or incorporate baseline values as covariates in the analysis. Moreover baseline utilization data were provided by patient recall and not corroborated by medical records. There was a substantially larger variability in the baseline utilization rates for the peak flow group compared with the control group. This suggests that a subset of very high frequency users may have been over-represented in the peak flow group, and the reduction in emergency room visits may be concentrated in this subset.
Peak-flow meter-based written action plan versus peak flow meter with no written action plan
Two studies18,19 addressed the independent effect of a written action plan when added to peak flow self-monitoring (Table 3). Charlton19 reported no significant group differences for main outcomes, while Ignacio-Garcia18 reported large and statistically significant differences in most of the outcomes, favoring the group that used the written action plan.
The Ignacio-Garcia study, however, suffers from notable flaws suggesting the results may be attributable to bias. The sole participating physician, not blinded to treatment assignment, was highly involved in all phases of patient assessment, monitoring, and treatment. There was evidence of baseline differences between the two groups. A total of 25% of patients were withdrawn after randomization, and an unexplained decline in lung function occurred in the control group. Thus, the potential for selection bias, withdrawal bias, and ascertainment bias limits confidence in the results of this study
Symptom-based written action plan compared with peak flow-based written action plan
In 4 studies,16,17,20,21 reported outcomes were generally equivalent between groups and comparisons were not statistically significant, with one exception (Table 3). The 3-arm study by Cowie et al16 reported a striking reduction in the total number of emergency room visits with a peak flow meter-based written action plan compared with a symptom-based written action plan (5 versus 45, P
Discussion
The objective of this systematic review was to assess the independent effects of 2 specific components commonly included in asthma self-management plans—a written action plan and a peak flow meter. Few studies, however, are designed to permit reviewers to isolate the effects of these components. Moreover, the studies we reviewed did not clearly identify the population expected to benefit from interventions or specify the primary outcomes of interest; nor was the level of clinically meaningful improvement prospectively defined.
Most of the trials we reviewed, including the largest community study of 569 patients, did not demonstrate improved outcomes. The 2 trials that reported statistically significant results favoring a peak flow-based written action plan suffer from notable flaws suggesting the results may be attributable to bias. In the other 7 trials, there was little difference in outcomes between groups. However, these studies had insufficient power to detect group differences or confidently conclude equivalence between groups.
Thus, available evidence is insufficient to demonstrate that asthma outcomes are improved by use of a written asthma action plan, with or without peak flow monitoring. While this body of literature does not establish that these interventions are ineffective, it suggests they will not have a large effect on outcomes when applied to the general asthmatic population. The application of written action plans to all asthmatics indiscriminately may be a wasteful use of resources. This systematic review also questions the validity of written action plans as an indicator of asthma quality of care, or as a means to achieve quality improvement.
This analysis also highlights several obstacles to assessing the effects of disease management interventions. First, while the impact of whole intervention programs can be evaluated in controlled trials, it may be unfeasible to isolate each component of such programs and subject it to a rigorous analysis. Furthermore, as a behavioral intervention, the general principle of engaging patients in self-management may be more important that the specific components of these programs. Finally, regarding the optimization of medications (most obviously initiation of inhaled steroids) the impact of written action plans is likely to be relatively small, particularly on lung function or symptom control.
Future clinical trials should be done selectively, aimed at producing rigorous results that can improve the effectiveness of self-management interventions. Further study is warranted for specific subpopulations, such as those with higher baseline severity of illness or those with high baseline utilization rates. Available data suggest that, if there is benefit to be gained from self-management interventions, it will most likely be seen among these patients. Specific components of self-management that might be tested individually are those that are relatively high-cost, resource intensive, or risky for the patient.
Existing trials have tended to over-estimate the effects of action plan-based interventions, thus having invested resources for results inadequate for optimizing self-management strategies. Careful consideration needs to be taken in future trials to realistically estimate the expected impact of each intervention, and to specify the primary outcomes of interest and their baseline frequencies. Future trials should be large enough to detect a difference if one exists, or to confidently conclude that the intervention is ineffective.
Attention to these principles will help to advance our knowledge in this area most efficiently and to ultimately improve the quality of care for the entire population of patients with asthma.
· Acknowledgments ·
We acknowledge Kathleen Ziegler, Pharm.D, and Claudia Bonnell, RN, MSL, for their assistance in the research and preparation of this manuscript.
- Most studies of asthma self-management do not permit retrospective isolation of the independent effects of a written action plan or peak flow meter use.
- Studies designed to isolate the effect of these self-care activities are generally underpowered or prone to systematic bias.
- Available evidence suggests that peak flow meters and written action plans do not have a large impact on outcomes when applied to the general population of asthmatics.
- These interventions are most likely to have beneficial effect when applied to selected populations, particularly patients with high baseline utilization.
Self-management skills are widely promoted by health plans and specialty societies with the expectation that they will improve care. The 1997 National Heart, Lung, and Blood Institute guidelines on treating asthma emphasize self-management,1 although they do not recommend specific programs. To maximize therapeutic effectiveness, it would be useful to know which components of patient self-management improve outcomes. Written action plans and peak flow meters are commonly used in asthma self-management programs. While these are simple, low-cost interventions for an individual, the aggregate cost for the entire population of asthmatics may be high.2
Much literature has accumulated on the effectiveness of providing asthma education alone and on programs that actively engage patients in their own care.Several systematic reviews have found that providing educational information alone has had little effect on asthma outcomes.3-5 There is evidence, though, that self-management activities are more effective than educational information alone. A recent Cochrane review of 24 trials found that self-management with regular practitioner review reduces hospi-talizations and emergency room visits.6 This review did not identify specific components contributing to improved outcomes. In contrast to the aforementioned studies on patient education, a large case-control study of children in the Kaiser Permanente System,7 found that written action plans were associated with lower rates of hospitalization and emergency room use. However, such observational studies often include confounding factors and are not sufficient to establish a cause-effect relationship between written action plans and improved outcomes.
We report on a systematic review that attempts to isolate the independent effect of a written action plan on asthma outcomes. We address two key questions:
- Compared with medical management alone, does the addition of a written asthma action plan (with or without peak flow meter use) improve outcomes?
- Compared with a written action plan based on symptoms, does a written action plan based on peak flow monitoring improve outcomes?
Methods
This study is part of a broader evidence report on the management of chronic asthma prepared for the Agency of Health Care Research and Quality8. Complete details of the methodology are available in the full report8 (http://www.ahcpr.gov/clinic/epcix.htm).
Literature search and study selection
We performed a comprehensive literature search from 1980 to August 2000 using MEDLINE, Embase, the Cochrane Library, and a hand search of recent bibliographies. The search was limited to full-length, peer-reviewed articles with an English abstract. Two independent reviewers carried out each step of study selection and data abstraction. Disagreements were resolved by consensus of the two reviewers or, if necessary, by the decision of a third reviewer.
Initial study selection was limited to comparative full-length reports or abstracts in peer-reviewed medical journals, with at least 25 evaluable children or adults per arm, treated for at least 12 weeks. Relevant comparisons included a written action plan and no written action plan; a written action plan based on peak flow readings and a written action plan based on symptoms. Study designs varied: clinical trials, cohort comparisons, case-control analyses, cross-sectional evaluations, and before-after comparisons. Specific components of the management plan had to be described.
Relevant outcomes included measures of inpatient and outpatient utilization, lung function, symptoms, rescue medication or oral steroid use, and quality of life. Outcomes of greatest interest were utilization parameters, as the goals of self-management usually focus on improving these outcomes.
These initial selection criteria yielded many studies that were confounded by multiple asthma management interventions and thus did not isolate the comparisons of interest. Therefore, the research team collectively determined the study design features that would best isolate the effects of written action plans and used them as new criteria in a second round of study selection. The studies thus selected satisfied 4 criteria: 1) randomization of patients; (2) delivery of the same interventions to experimental and control groups, except that the experimental group also received a written action plan; (3) delivery of the same interventions to experimental and control groups, except that one group received a written action plan based on peak flow meter readings, and the comparison group received a written action plan based on symptom monitoring; and 4) inclusion of a written action plan that met our specified definition.
A written action plan, by our definition, had two components: an algorithm that identified specific clinical indicators signaling the need for adjustments in medication; and specific instructions on how to adjust medications in response to such indicators. Many publications lacked sufficient detail on the written plan, so a brief survey was sent to the primary author of each of the 36 studies. If no response was obtained (36%), the article was excluded only when it was clear from the publication that our definition was not met.
Assessment of study quality
High-quality studies were randomized controlled trials that met the 3 domains of study quality that have been demonstrated empirically to impact effect size: concealment of treatment allocation; double-blinding; and minimization of exclusion bias.9,10 However, we doubted the feasibility of double-blinding a written asthma plan intervention, and so relaxed this requirement. We considered exclusion bias to be minimized when a study either reported intent-to-treat analysis or excluded fewer than 10% of subjects from analysis, with the ratio of subjects excluded from each arm being less than 2:1.
To more fully evaluate study design issues that may be particularly important in asthma research,11,12 we constructed asthma-specific quality indicators in consultation with an expert panel. Controls for potential confounders of treatment effect included establishing reversibility of airway obstruction, controlling for other medication use, reporting compliance, and addressing seasonality. In addition, a priori reporting of power calculations and accounting for exclusions and withdrawals were judged to be study quality characteristics pertinent to this body of evidence.
Data analysis
We constructed evidence tables for the outcomes of interest, and performed a qualitative synthesis of the data. Meta-analysis was not appropriate due to wide discrepancies in the patient populations studied, the interventions employed, and measurement and reporting of outcomes.
Results
Our literature search yielded a total of 4578 citations. Of these, 36 studies met the initial selection criteria. Many of these qualifying studies, however, were confounded by multiple asthma management interventions applied inconsistently across treatment arms. For example, a common confounder was review of and change in long-term medication use in the treatment group, but not in the control group. This necessitated a refinement in our selection criteria to focus on studies that largely isolated the effect of written action plans.13-21 This step yielded a final evidence base of 9 randomized controlled trials with a total enrollment of 1501 patients.
Table 1 summarizes the characteristics, interventions, and outcomes of the 9 studies. Two studies were 3-arm trials,16,17 which raised the total number of comparisons among the 9 studies to 11. The largest study was the Grampian Asthma Study of Integrated Care (n=569),14 a community study conducted in the UK. Enrollment in the other 8 studies ranged from 43 to 64 patients per arm. Treatment duration ranged from 24 to 52 weeks.
None of the studies met our definition of high quality. In fact, no study met any of the generic quality criteria—none was blinded, none described concealment of allocation, and all excluded more than 10% of subjects. Furthermore, none reported an intention-to-treat analysis. Thus these trials were prone to withdrawal bias as well as overestimation of treatment effect due to lack of allocation concealment.
No study met the majority of asthma-specific indicators (Table 1). Of the 9 studies, only 5 met any asthma-specific indicator. Three reported prospective power calculations,13-15 but 2 of these substantially overestimated the expected effect.13,15 Two studies established reversibility;14,17 2 controlled for other medication use;13,15 and 2 reported compliance.17,21 Thus, the studies were also prone to a type II error (failing to detect a true effect) and to potential confounding of outcomes.
We performed sample power calculations for hospitalizations (Table 2), derived from baseline rates reported in 4 studies14,16-18 and standard deviations reported in 2.14,17 A study with 250 patients per arm could detect a reduction of 50% or more in hospitalization, given a control rate of at least 0.2 hospitalizations/patient/year. In actuality, GRASSIC,14 which is the largest available trial (N=569), had baseline hospitalization rates of 0.12 and 0.13. With this baseline rate, over 700 patients per arm are required, higher than the actual enrollment in GRASSIC. The other studies in this review would be adequately powered to detect a 50% difference only in the setting of even higher baseline utilization (eg, 0.30 hospitalizations/patient/year).
Table 3 displays utilization outcomes for the 11 comparisons in the 9 trials. In 5 studies (N=1019), medical management with a written action plan was compared with medical management without a written action plan.13-17 Two trials (N=185) compared a peak flow meter plus a written action plan with a peak flow meter and no written action plan18,19 In 4 studies (N=393), a written action plan based on peak flow monitoring was compared with a written action plan based on symptoms.
TABLE 1
Study characteristics
| Study | Patient popultation | Study Arms | Intervention components | Outcomes reported | Asthma quality indicators met |
|---|---|---|---|---|---|
| Optimal medical management vs. optimal medical management + PFM action plan | |||||
| Jones 199514 | Inclusions: patients using ICS | Usual care | SxD, FU | Ut, LF, Sx | Pow, Med |
| Exclusions: patients on oral steroids or using peak flow meters at home | PFM action plan | AP, PF, SxD, FU | |||
| Mean age: 29.5 years | |||||
| Severity level: Mild–moderate | |||||
| Drummond 1994 (GRASSIC)15 | Inclusion: FEV1 reversibility 20% or greater | Usual care | FU | Ut, LF, Med Ex | Pow, Rev |
| Exclusions: patients who already owned a PFM | PFM action plan | AP, PF, FU | |||
| Mean age: 50.8 years | |||||
| Severity level: Mild–severe | |||||
| Ayres 199516 | Inclusions: maximum PEF variability, 0.15%; minimum nights/week with symptoms, 3; minimum use of ICS or sodium cromoglycate, 3 months | Usual care | SxD, FU | LF, Sx, Ex | Pow, Med |
| Mean age: 45 years | PFM action plan | AP, PF, SxD, FU | |||
| Severity level: Moderate–severe | |||||
| Cowie 199713 | Inclusions: treatment for an exacerbation of asthma in an ER asthma clinic; history of receiving urgent treatment for asthma in the previous 12 months | Usual care | Ed, SxD, FU | Ut, PF, Med, Ex | None |
| Mean age: 37.8 years | PFM action plan | AP, PF, Ed, SxD, FU | |||
| Severity level: Mild–severe | |||||
| Cote 199717 | Inclusions: FEV1postbronchodilator 85-100 % of predicted; PEF, at minimum, 85 % of predicted; minimum PEF variability, 0%; Methacholine | Usual care | Ed | Ut, LF, Med | Exc, Rev, Com |
| Exclusions: patients having previously taken an asthma educational program | PFM action plan | Ed, Cn, AP, PF | |||
| Mean age: 36.5 years Severity level: Mild | |||||
| Usual care + PFM use alone vs. usual care + PFM action plan | |||||
| Ignacio-Garcia 199518 | Inclusions: patients from outpatient asthma clinic with asthma for 2 years | Usual care + PFM | PF, SxD, FU | Ut, LF, Med | None |
| Mean age: 41.9 years | Usual care + PFM action plan | PF, AP, Ed, SxD, FU | |||
| Severity level: Mild–severe | |||||
| Charlton 199419 | Inclusion: patients with inpatient or outpatient visit for asthma | Usual care + PFM | PF, Ed, SxD, FU | Ut, Sx, Med, Ex | None |
| Mean age: 6.5 years | Usual care + PFM action plan | PF, AP, Ed, SxD, FU | |||
| Severity level: Mild–moderate | |||||
| PFM action plan vs. Symptom action plan | |||||
| Turner 199820 | Inclusions: Maximum methacholine PC20, 7.9; using ICS | Symptom action plan | AP, Ed, SxD, Cn BM, EM | Ut, LF, Sx, Med | Exc, Com |
| Exclusions: previous PFM use; significant comorbid conditions | PFM action plan | PF, AP, Ed, SxD, Cn BM, EM | |||
| Mean age: 34.1 years | |||||
| Severity level: Mild–severe | |||||
| Charlton 199021 | Inclusions: patients on repeat prescribing register | Symptom action plan | AP, Ed, FU | Ut, Med | None |
| Mean age: NR | PFM action plan | PF, AP, Ed, FU, Cn | Ut, PF, Med, Ex | None | |
| Severity level: Mild–severe (?) | |||||
| Cowie 199716 | Inclusions: treatment for an exacerbation of asthma in an ER, or asthma clinic; history of receiving urgent treatment for asthma in the previous 12 months | Symptom action plan | AP, Ed, SxD, FU | ||
| PFM action plan | AP, PF, Ed, SxD, FU | ||||
| Cote 199717 | Inclusions: FEV1postbronchodilator, 85-100 % of predicted; PEF, at minimum, 85 % of predicted; minimum PEF variability, 0%; Methacholine | Symptom action plan | Ed, AP | Ut, LF, Med | Exc, Rev, Com |
| Exclusions: previous enrollment in an asthma educational program | PFM action plan | Ed, Cn, AP, PF | |||
| Eligibility criteria: ICS = inhaled corticosteroid; FEV1 = forced expiratory volume in 1 second; PEF = peak expiratory flow; PFM = peak flow meter; ER = emergency room; PC20 = 20% fall in FEV1 Intervention components: PF = Peak flow meter; AP = Written Action Plan; Ed = Education; SxD = Symptom diary; FU = Follow-up visits; Cn = Counseling; BM = Behavior modification; EM = Environmental modification | |||||
| Outcomes: Ut = Utilization measures; LF= Lung function measurements; Sx = Symptom=based measurements; Med = Medication use; Ex = Exacerbations of asth ma Asthma Quality Indicators: Exc = Accounted for excluded patients; Pow = Reported power calculations; Rev = Established reversibility of airway obstruction; Med = Controlled for other medication use; Com = Reported compliance; Sea = Addressed seasonality. | |||||
TABLE 2
Power calculations for hospitalizations per patient per year
| Assumed control mean | Possible treatment mean | % decrease | N needed per study arm |
|---|---|---|---|
| 0.10 | 0.075 | 25 | 3077 |
| 0.10 | 0.05 | 50 | 770 |
| 0.10 | 0.025 | 75 | 342 |
| 0.20 | 0.015 | 25 | 770 |
| 0.20 | 0.10 | 50 | 193 |
| 0.20 | 0.05 | 75 | 86 |
| 0.30 | 0.225 | 25 | 342 |
| 0.30 | 0.15 | 50 | 86 |
| 0.30 | 0.075 | 75 | 38 |
| Studies were identified that contained baseline rates on hospitalizations/patient/year, or information that allowed calculation of this parameter (Drummond, Abdalla, Beattie et al., 1994; Cote, Cartier, Robichaud et al. 1997; Cowie, Revitt, Underwood et al., 1997; Ignacio-Garcia and Gonzalez-Santos, 1995). Baseline rates of hospitalization varied in these studies from 0.04-0.29/patient/year. Standard deviations for this outcome were available only in two studies; Cote, Cartier, Robichaud et al. (1997) reported an SD of 0.30 for this variable, and an SD of 0.35 was calculated from the confidence intervals reported in GRASSIC (Drummond, Abdalla, Beattie et al., 1994). For the calculations, the more conservative 0.35 estimate for SD was used. | |||
| Number of patients per study arm were estimated for 80 percent power at the 5 percent significance level using control arm means of 0.10, 0.20, and 0.30 hospitalizations/patient/year. The expected reduction in this variable was tested along a spectrum from 25-75 percent. | |||
Written action plan versus no written action plan
All 5 studies used a peak flow meter based written action plan. All reported utilization outcomes, but the types and units of measurement were not consistent across studies (Table 2). Additionally, 4 studies reported on symptoms,13-16 and 3 reported lung function outcomes.13-15
With one notable exception, there were no statistically significant differences in outcomes among groups. Cowie et al16 reported an 11-fold decrease in total emergency room visits for the group using a peak-flow action plan (5 vs 55, P = .02), and also reported a reduction in hospitalizations of a similar magnitude (2 vs 12) that did not reach statistical significance. However, this study suffers from notable flaws that diminish confidence in the results. It is a post-intervention comparison among groups, which does not compare change from baseline, or incorporate baseline values as covariates in the analysis. Moreover baseline utilization data were provided by patient recall and not corroborated by medical records. There was a substantially larger variability in the baseline utilization rates for the peak flow group compared with the control group. This suggests that a subset of very high frequency users may have been over-represented in the peak flow group, and the reduction in emergency room visits may be concentrated in this subset.
Peak-flow meter-based written action plan versus peak flow meter with no written action plan
Two studies18,19 addressed the independent effect of a written action plan when added to peak flow self-monitoring (Table 3). Charlton19 reported no significant group differences for main outcomes, while Ignacio-Garcia18 reported large and statistically significant differences in most of the outcomes, favoring the group that used the written action plan.
The Ignacio-Garcia study, however, suffers from notable flaws suggesting the results may be attributable to bias. The sole participating physician, not blinded to treatment assignment, was highly involved in all phases of patient assessment, monitoring, and treatment. There was evidence of baseline differences between the two groups. A total of 25% of patients were withdrawn after randomization, and an unexplained decline in lung function occurred in the control group. Thus, the potential for selection bias, withdrawal bias, and ascertainment bias limits confidence in the results of this study
Symptom-based written action plan compared with peak flow-based written action plan
In 4 studies,16,17,20,21 reported outcomes were generally equivalent between groups and comparisons were not statistically significant, with one exception (Table 3). The 3-arm study by Cowie et al16 reported a striking reduction in the total number of emergency room visits with a peak flow meter-based written action plan compared with a symptom-based written action plan (5 versus 45, P
Discussion
The objective of this systematic review was to assess the independent effects of 2 specific components commonly included in asthma self-management plans—a written action plan and a peak flow meter. Few studies, however, are designed to permit reviewers to isolate the effects of these components. Moreover, the studies we reviewed did not clearly identify the population expected to benefit from interventions or specify the primary outcomes of interest; nor was the level of clinically meaningful improvement prospectively defined.
Most of the trials we reviewed, including the largest community study of 569 patients, did not demonstrate improved outcomes. The 2 trials that reported statistically significant results favoring a peak flow-based written action plan suffer from notable flaws suggesting the results may be attributable to bias. In the other 7 trials, there was little difference in outcomes between groups. However, these studies had insufficient power to detect group differences or confidently conclude equivalence between groups.
Thus, available evidence is insufficient to demonstrate that asthma outcomes are improved by use of a written asthma action plan, with or without peak flow monitoring. While this body of literature does not establish that these interventions are ineffective, it suggests they will not have a large effect on outcomes when applied to the general asthmatic population. The application of written action plans to all asthmatics indiscriminately may be a wasteful use of resources. This systematic review also questions the validity of written action plans as an indicator of asthma quality of care, or as a means to achieve quality improvement.
This analysis also highlights several obstacles to assessing the effects of disease management interventions. First, while the impact of whole intervention programs can be evaluated in controlled trials, it may be unfeasible to isolate each component of such programs and subject it to a rigorous analysis. Furthermore, as a behavioral intervention, the general principle of engaging patients in self-management may be more important that the specific components of these programs. Finally, regarding the optimization of medications (most obviously initiation of inhaled steroids) the impact of written action plans is likely to be relatively small, particularly on lung function or symptom control.
Future clinical trials should be done selectively, aimed at producing rigorous results that can improve the effectiveness of self-management interventions. Further study is warranted for specific subpopulations, such as those with higher baseline severity of illness or those with high baseline utilization rates. Available data suggest that, if there is benefit to be gained from self-management interventions, it will most likely be seen among these patients. Specific components of self-management that might be tested individually are those that are relatively high-cost, resource intensive, or risky for the patient.
Existing trials have tended to over-estimate the effects of action plan-based interventions, thus having invested resources for results inadequate for optimizing self-management strategies. Careful consideration needs to be taken in future trials to realistically estimate the expected impact of each intervention, and to specify the primary outcomes of interest and their baseline frequencies. Future trials should be large enough to detect a difference if one exists, or to confidently conclude that the intervention is ineffective.
Attention to these principles will help to advance our knowledge in this area most efficiently and to ultimately improve the quality of care for the entire population of patients with asthma.
· Acknowledgments ·
We acknowledge Kathleen Ziegler, Pharm.D, and Claudia Bonnell, RN, MSL, for their assistance in the research and preparation of this manuscript.
1. National Heart, Lung and Blood Institute. Expert panel report 2: guidelines for the diagnosis and management of asthma. Bethesda, MD: National Institutes of Health; 1997. NIH publication 97-4051.
2. Ruffin RE, Pierce RJ. Peak flow monitoring—which asthmatics, when, and how? Aust N Z J Med 1994;24:519-20.
3. Devine EC. Meta-analysis of the effects of psychoeducational care in adults with asthma. Res Nursing Health 1996;19:367-76.
4. Bernard-Bonnin AC, Stachenko S, Bonin D, et al. Self-management teaching programs and morbidity of pediatric asthma: a meta-analysis. J Allergy Clin Immunol 1995;95(1 Pt 1):34-41.
5. Gibson PG, Coughlan J, Wilson AJ, et al. Limited (information only) patient education programs for adults with asthma. Cochrane Database Syst Rev 2000a;(2):CD001005.-
6. Gibson PG, Coughlan J, Wilson AJ, et al. Self-management education and regular practitioner review for adults with asthma. Cochrane Database Syst Rev 2000b (2):CD001117.-
7. Lieu TA, Quesenberry CP, Jr, Capra AM, et al. Outpatient management practices associated with reduced risk of pediatric asthma hospitalization and emergency department visits. Pediatrics 1997;100(3 Pt 1):334-41.
8. Lefevre F, Piper M, Mark D, et al. Management of Chronic Asthma. AHRQ evidence report, contract number 290-97-001-5, 2001, http://www.ahcpr.gov/clinic/epcix.htm.
9. Mulrow CD, Oxman AD, editors. Cochrane Collaboration Handbook. Available in the Cochrane Library [database on disk and CD-ROM]. The Cochrane Collaboration; Issue 1. Oxford: Update Software; 1997.
10. Schulz KF, Chalmers I, Hayes RJ, et al. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273(5):408-12.
11. Berlin JA, Rennie D. Measuring the quality of trials: the quality of the quality scales. JAMA 1999;282(11):1083-5.
12. Juni P, Witschi A, Bloch R, et al. The hazards of scoring the quality of clinical trials for meta-analysis. JAMA 1999;282(11):1054-60.
13. Jones KP, Mullee MA, Middleton M, et al. Peak flow based asthma self-management: a randomised controlled study in general practice. British Thoracic Society Research Committee. Thorax 1995;50(8):851-7.
14. Drummond N, Abdalla M, Beattie JAG, et al. Effectiveness of routine self monitoring of peak flow in patients with asthma. Grampian Asthma Study of Integrated Care GRASSIC). BMJ 1994 Feb. 26;308(6928):564-7.
15. Ayres JG, Campbell LM. A controlled assessment of an asthma self-management plan involving a budesonide dose regimen. OPTIONS Research Group. Eur Respir J 1996;886-92.
16. Cowie RL, Revitt SG, Underwood MF, et al. The effect of a peak flow-based action plan in the prevention of exacerbations of asthma. Chest 1997;112(6):1534-8.
17. Cote J, Cartier A, Robichaud P, et al. Influence on asthma morbidity of asthma education programs based on self-management plans following treatment optimization. Am J Respir Crit Care Med 1997;155(5):1509-14.
18. Ignacio-Garcia JM, Gonzalez-Santos P. Asthma self-management education program by home monitoring of peak expiratory flow. Am J Respir Crit Care Med 1995;151(2 Pt 1):353-9.
19. Charlton I, Antoniou AG, Atkinson J, et al. Asthma at the interface: bridging the gap between general practice and a district general hospital. Arch Dis Child 1994;70(4):313-8.
20. Turner MO, Taylor D, Bennett R, et al. A randomized trial comparing peak expiratory flow and symptom self-management plans for patients with asthma attending a primary care clinic. Am J Respir Crit Care Med 1998;157(2):540-6.
21. Charlton I, Charlton G, Broomfield J, et al. Evaluation of peak flow and symptoms only self-management plans for control of asthma in general practice. BMJ 1990;301(6765):1355-9.
1. National Heart, Lung and Blood Institute. Expert panel report 2: guidelines for the diagnosis and management of asthma. Bethesda, MD: National Institutes of Health; 1997. NIH publication 97-4051.
2. Ruffin RE, Pierce RJ. Peak flow monitoring—which asthmatics, when, and how? Aust N Z J Med 1994;24:519-20.
3. Devine EC. Meta-analysis of the effects of psychoeducational care in adults with asthma. Res Nursing Health 1996;19:367-76.
4. Bernard-Bonnin AC, Stachenko S, Bonin D, et al. Self-management teaching programs and morbidity of pediatric asthma: a meta-analysis. J Allergy Clin Immunol 1995;95(1 Pt 1):34-41.
5. Gibson PG, Coughlan J, Wilson AJ, et al. Limited (information only) patient education programs for adults with asthma. Cochrane Database Syst Rev 2000a;(2):CD001005.-
6. Gibson PG, Coughlan J, Wilson AJ, et al. Self-management education and regular practitioner review for adults with asthma. Cochrane Database Syst Rev 2000b (2):CD001117.-
7. Lieu TA, Quesenberry CP, Jr, Capra AM, et al. Outpatient management practices associated with reduced risk of pediatric asthma hospitalization and emergency department visits. Pediatrics 1997;100(3 Pt 1):334-41.
8. Lefevre F, Piper M, Mark D, et al. Management of Chronic Asthma. AHRQ evidence report, contract number 290-97-001-5, 2001, http://www.ahcpr.gov/clinic/epcix.htm.
9. Mulrow CD, Oxman AD, editors. Cochrane Collaboration Handbook. Available in the Cochrane Library [database on disk and CD-ROM]. The Cochrane Collaboration; Issue 1. Oxford: Update Software; 1997.
10. Schulz KF, Chalmers I, Hayes RJ, et al. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273(5):408-12.
11. Berlin JA, Rennie D. Measuring the quality of trials: the quality of the quality scales. JAMA 1999;282(11):1083-5.
12. Juni P, Witschi A, Bloch R, et al. The hazards of scoring the quality of clinical trials for meta-analysis. JAMA 1999;282(11):1054-60.
13. Jones KP, Mullee MA, Middleton M, et al. Peak flow based asthma self-management: a randomised controlled study in general practice. British Thoracic Society Research Committee. Thorax 1995;50(8):851-7.
14. Drummond N, Abdalla M, Beattie JAG, et al. Effectiveness of routine self monitoring of peak flow in patients with asthma. Grampian Asthma Study of Integrated Care GRASSIC). BMJ 1994 Feb. 26;308(6928):564-7.
15. Ayres JG, Campbell LM. A controlled assessment of an asthma self-management plan involving a budesonide dose regimen. OPTIONS Research Group. Eur Respir J 1996;886-92.
16. Cowie RL, Revitt SG, Underwood MF, et al. The effect of a peak flow-based action plan in the prevention of exacerbations of asthma. Chest 1997;112(6):1534-8.
17. Cote J, Cartier A, Robichaud P, et al. Influence on asthma morbidity of asthma education programs based on self-management plans following treatment optimization. Am J Respir Crit Care Med 1997;155(5):1509-14.
18. Ignacio-Garcia JM, Gonzalez-Santos P. Asthma self-management education program by home monitoring of peak expiratory flow. Am J Respir Crit Care Med 1995;151(2 Pt 1):353-9.
19. Charlton I, Antoniou AG, Atkinson J, et al. Asthma at the interface: bridging the gap between general practice and a district general hospital. Arch Dis Child 1994;70(4):313-8.
20. Turner MO, Taylor D, Bennett R, et al. A randomized trial comparing peak expiratory flow and symptom self-management plans for patients with asthma attending a primary care clinic. Am J Respir Crit Care Med 1998;157(2):540-6.
21. Charlton I, Charlton G, Broomfield J, et al. Evaluation of peak flow and symptoms only self-management plans for control of asthma in general practice. BMJ 1990;301(6765):1355-9.