S. Claiborne Johnston, MD, PhD

Delirium is characterized by fluctuating disturbances in cognition and consciousness and is a common complication of hospitalization in medical and surgical patients. Studies estimate the prevalence of delirium in hospitalized patients[1] to be 14% to 56%, and up to 70% in critically ill elderly patients.[2] Estimates of total healthcare costs associated with delirium range from $38 to $152 billion per year in the United States.[3] Delirious patients are more likely to be discharged to a nursing home and have increased hospital mortality and longer lengths of stay.[4, 5, 6] Recent data suggest long‐term effects of delirium including cognitive impairments up to 1 year following the illness[7] and an increased likelihood of developing[8] or worsening dementia.[9]

It is estimated that one‐third of hospital‐acquired delirium cases could be prevented with appropriate interventions.[10] A prediction rule that easily and accurately identifies high‐risk patients upon admission could therefore have a substantial clinical impact. In addition, a prediction rule could be used to identify patients in whom new targeted interventions for delirium prevention could be investigated. A number of risk factors for delirium have been identified, including older age, preexisting cognitive dysfunction, vision and hearing impairment, severe illness, dehydration, electrolyte abnormalities, overmedication, and alcohol abuse.[11, 12, 13, 14, 15, 16] Existing prediction rules using various combinations of these measures have been limited by their complexity,[17] do not predict incident delirium,[18, 19] or are restricted to surgical[20, 21, 22] or intensive care[23] patients and therefore are not broadly applicable to the general medical population, which is particularly susceptible to developing delirium.

We conducted this study to develop a simple, efficient, and accurate prediction rule for hospital‐acquired delirium in adult medical inpatients assessed at the time of admission. Our a priori hypothesis was that a delirium prediction rule would consist of a combination of known risk factors and most likely incorporate old age, illness severity, and preexisting cognitive dysfunction.

METHODS

RESULTS

The derivation cohort consisted of elderly patients (mean age, 68.0811.96 years; interquartile range, 5096 years), and included more males than females (54.1% vs 45.9%). Subjects were predominantly white (73.7%) and lived at home (90%) (Table 1). The mean admission MMSE score was 27.0 (standard deviation [SD], 3.4; range, 730). Median follow‐up was 2 days (interquartile range, 13). Delirium developed in 12% (n=25) of the cohort.

Univariate analysis of the derivation study identified 10 variables significantly associated (P<0.05) with delirium (Table 2). Predictors of delirium included abnormal scores on 4 subtests of the MMSE, low score on the Mini‐Cog, living in an assisted living or skilled nursing facility, moderate to severe illness, old age, a past history of dementia, and hearing loss as assessed by the whisper test. These predictors were then entered into a stepwise logistic regression analysis that identified 4 independent predictors of delirium (Table 3).

These 4 independent predictors were assigned 1 point each if present to create a prediction rule with a range of possible scores from 0 to 4. There was a significant trend predicting higher rates of delirium with higher scores, with no subjects who scored 0 becoming delirious, compared to 40% of those subjects scoring 3 or 4 (P for trend<0.001) (Table 4).

The validation cohort consisted of adults with a mean age of 70.7210.6 years, (interquartile range, 5194 years), who were predominantly white (75.8%) and overwhelmingly male (95.2%) (Table 1). The mean admission MMSE score was 26.75 (SD, 2.8; range, 1730). Median follow‐up was 2 days (interquartile range, 15). Delirium developed in 8.5% (n=14) of the cohort. In the validation cohort, 4% of subjects with a score of 0 became delirious, whereas 19% of those scoring 3 or 4 became delirious (P for trend 0.025) (Table 4).

ROC curves were compared for the derivation and validation cohorts. The area under the ROC curve for the derivation cohort (0.81, 95% confidence interval [CI]: 0.720.90) was slightly better than that in the validation cohort (0.69, 95% CI: 0.540.83), but the difference did not reach statistical significance (P=0.14) (Figure 1).

Figure 1

DISCUSSION

We derived and validated a prediction rule to assess the risk of developing delirium in hospitalized adult medical patients. Four variables easily assessed on admission in a screen lasting less than 2 minutes were independently associated with the development of delirium. The prediction rule can be remembered with the following mnemonic: AWOL (Age80 years; unable to spell World backward; not fully Oriented to place; and moderate or severe iLlness severity).

It is estimated up to a third of hospital acquired delirium cases can be prevented.[10] Recent guidelines recommend the use of a multicomponent intervention to prevent delirium and provide evidence that such a strategy would be cost‐effective.[39] Nevertheless, such interventions are resource intense, requiring specialized nurse training and staffing[40] and have not been widely implemented. Acute care for the elderly units, where interventions to prevent delirium might logically be implemented, also require physical remodeling to provide carpeted hallways, handrails, and elevated toilet seats and door levers.[41] A method of risk stratification to identify the patients who would benefit most from resource‐intensive prevention strategies would be valuable.

The AWOL tool may provide a practical alternative to existing delirium prediction rules for adult medical inpatients. Because it can be completed by a nurse in <2 minutes, the AWOL tool may be easier to apply and disseminate than a previously described score relying on the MMSE, Acute Physiology and Chronic Health Evaluation scores, and measured visual acuity.[17] Two other tools, 1 based on chart abstraction[18] and the other based on clinical variables measured at admission,[19] are similarly easy to apply but only predict prevalent and not incident delirium, making them less clinically useful.

This study's strengths include its prospective cohort design and the derivation and validation being performed in different hospitals. The derivation cohort consisted of patients admitted to a tertiary care academic medical center or an affiliated hospital where routine mixed gender general medical patients are treated, whereas validation was performed at the SFVAMC, where patients are predominantly older men with a high incidence of vascular risk factors. The outcome was assessed on a daily basis, and the likelihood any cases were missed was low. Although there is some potential for bias because the outcome was assessed by a research assistant not blinded to baseline characteristics, this was mitigated by having each outcome validated by a blinded neurologist and in the validation cohort having the research assistant blinded to the AWOL score. Other strengths are the broad inclusion criteria, with both middle‐aged and elderly patients having a wide range of medical and neurological conditions, allowing for wide application of the results. Although many studies of delirium focus on patients over age 70 years, we chose to include patients aged 50 years or older because hospital‐acquired delirium still occurs in this age group (17 of 195 [8%] patients aged 5069 years became delirious in this study), and risk factors such as severe illness and cognitive dysfunction are likely to be predictors of delirium even at younger ages. Additionally, the inclusion of nurses' clinical judgment to assess illness severity using a straightforward rating scale allows bedside nurses to readily administer the prediction rule in practice.[34]

This study has several potential limitations. The number of outcomes in the derivation cohort was small compared to the number of predictors chosen for the prediction rule. This could potentially have led to overfitting the model in the derivation cohort and thus an overly optimistic estimation of the model's performance. In the validation cohort, the area under the ROC curve was lower than in the derivation cohort, and although the difference did not reach statistical significance, this may have been due to the small sample size. In addition, none of the 4 subjects with an AWOL score of 4 became delirious, potentially reflecting poor calibration of the prediction rule. However, the trend of higher rates of delirium among subjects with higher AWOL scores still reached statistical significance, and the prediction rule demonstrated good discrimination between patients at high and low risk for developing delirium.

To test whether a better prediction tool could be derived from our data, we combined the derivation and validation cohorts and repeated a stepwise multivariable logistic regression with the same variables used for derivation of the AWOL tool (with the exception of the whisper test of hearing and a past medical history of dementia, because these data were not collected in the validation cohort). This model produced the same 4 independent predictors of delirium used in the AWOL tool. We then used bootstrapping to internally validate the prediction rule, suggesting that the predictors in the AWOL tool were the best fit for the available data. However, given the small number of outcomes in our study, the AWOL tool may benefit from further validation in a larger independent cohort to more precisely calibrate the number of expected outcomes with each score.

Although the majority of medical inpatients were eligible for enrollment in our study, some populations were excluded, and our results may not generalize to these populations. Non‐English speaking patients were excluded to preserve the validity of our study instruments. In addition, patients with profound aphasia or an admission diagnosis of alcohol withdrawal were excluded. Patients discharged on the first day of their hospitalization were excluded either because they were discharged prior to screening or prior to their first follow‐up visit. Therefore, our results may only be valid in patients who remained in the hospital for over 24 hours. In addition, because we only included medical patients, our results cannot necessarily be generalized to the surgical population.

Finally, parts of the prediction rule (orientation and spelling world backward) are also components of the CAM and were used in the assessment of the outcome, and this may introduce a potential tautology: if patients are disoriented or have poor attention because they cannot spell world backward at admission, they already have fulfilled part of the criteria for delirium. However, a diagnosis of delirium using the CAM involves a comprehensive patient and caregiver interview, and in addition to poor attention, requires the presence of an acute change in mental status and disorganized thinking or altered level of consciousness. Therefore, it is possible, and common, for patients to be disoriented to place and/or unable to spell world backward, yet not be delirious, and predicting a subsequent change in cognition during the hospitalization is still clinically important. It is possible the AWOL tool works by identifying patients with impaired attention and subclinical delirium, but one could argue this makes a strong case for its validity because these patients especially should be triaged to an inpatient unit that specializes in delirium prevention. It is also possible the cognitive tasks that are part of the AWOL tool detect preexisting cognitive impairment, which is in turn a major risk factor for delirium.

Recognizing and classifying the risk of delirium during hospitalization is imperative, considering the illness' significant contribution to healthcare costs, morbidity, and mortality. The cost‐effectiveness of proven interventions to detect and prevent delirium could be magnified with focused implementation in those patients at highest risk.[39, 40, 41] Further research is required to determine whether the combination of delirium prediction rules such as those developed here and prevention strategies will result in decreased rates of delirium and economic savings for the healthcare system.

Disclosures

Drs. Douglas, Hessler, Dhaliwal, Betjemann, Lucatorto, Johnston, Josephson, and Ms. Fukuda and Ms. Alameddine have no conflicts of interest or financial disclosures. This research was made possible by the Ruth E. Raskin Fund and a UCSF Dean's Research Scholarship. These funding agencies had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.

Delirium is characterized by fluctuating disturbances in cognition and consciousness and is a common complication of hospitalization in medical and surgical patients. Studies estimate the prevalence of delirium in hospitalized patients[1] to be 14% to 56%, and up to 70% in critically ill elderly patients.[2] Estimates of total healthcare costs associated with delirium range from $38 to $152 billion per year in the United States.[3] Delirious patients are more likely to be discharged to a nursing home and have increased hospital mortality and longer lengths of stay.[4, 5, 6] Recent data suggest long‐term effects of delirium including cognitive impairments up to 1 year following the illness[7] and an increased likelihood of developing[8] or worsening dementia.[9]

It is estimated that one‐third of hospital‐acquired delirium cases could be prevented with appropriate interventions.[10] A prediction rule that easily and accurately identifies high‐risk patients upon admission could therefore have a substantial clinical impact. In addition, a prediction rule could be used to identify patients in whom new targeted interventions for delirium prevention could be investigated. A number of risk factors for delirium have been identified, including older age, preexisting cognitive dysfunction, vision and hearing impairment, severe illness, dehydration, electrolyte abnormalities, overmedication, and alcohol abuse.[11, 12, 13, 14, 15, 16] Existing prediction rules using various combinations of these measures have been limited by their complexity,[17] do not predict incident delirium,[18, 19] or are restricted to surgical[20, 21, 22] or intensive care[23] patients and therefore are not broadly applicable to the general medical population, which is particularly susceptible to developing delirium.

We conducted this study to develop a simple, efficient, and accurate prediction rule for hospital‐acquired delirium in adult medical inpatients assessed at the time of admission. Our a priori hypothesis was that a delirium prediction rule would consist of a combination of known risk factors and most likely incorporate old age, illness severity, and preexisting cognitive dysfunction.

METHODS

RESULTS

The derivation cohort consisted of elderly patients (mean age, 68.0811.96 years; interquartile range, 5096 years), and included more males than females (54.1% vs 45.9%). Subjects were predominantly white (73.7%) and lived at home (90%) (Table 1). The mean admission MMSE score was 27.0 (standard deviation [SD], 3.4; range, 730). Median follow‐up was 2 days (interquartile range, 13). Delirium developed in 12% (n=25) of the cohort.

Univariate analysis of the derivation study identified 10 variables significantly associated (P<0.05) with delirium (Table 2). Predictors of delirium included abnormal scores on 4 subtests of the MMSE, low score on the Mini‐Cog, living in an assisted living or skilled nursing facility, moderate to severe illness, old age, a past history of dementia, and hearing loss as assessed by the whisper test. These predictors were then entered into a stepwise logistic regression analysis that identified 4 independent predictors of delirium (Table 3).

These 4 independent predictors were assigned 1 point each if present to create a prediction rule with a range of possible scores from 0 to 4. There was a significant trend predicting higher rates of delirium with higher scores, with no subjects who scored 0 becoming delirious, compared to 40% of those subjects scoring 3 or 4 (P for trend<0.001) (Table 4).

The validation cohort consisted of adults with a mean age of 70.7210.6 years, (interquartile range, 5194 years), who were predominantly white (75.8%) and overwhelmingly male (95.2%) (Table 1). The mean admission MMSE score was 26.75 (SD, 2.8; range, 1730). Median follow‐up was 2 days (interquartile range, 15). Delirium developed in 8.5% (n=14) of the cohort. In the validation cohort, 4% of subjects with a score of 0 became delirious, whereas 19% of those scoring 3 or 4 became delirious (P for trend 0.025) (Table 4).

ROC curves were compared for the derivation and validation cohorts. The area under the ROC curve for the derivation cohort (0.81, 95% confidence interval [CI]: 0.720.90) was slightly better than that in the validation cohort (0.69, 95% CI: 0.540.83), but the difference did not reach statistical significance (P=0.14) (Figure 1).

Figure 1

DISCUSSION

We derived and validated a prediction rule to assess the risk of developing delirium in hospitalized adult medical patients. Four variables easily assessed on admission in a screen lasting less than 2 minutes were independently associated with the development of delirium. The prediction rule can be remembered with the following mnemonic: AWOL (Age80 years; unable to spell World backward; not fully Oriented to place; and moderate or severe iLlness severity).

It is estimated up to a third of hospital acquired delirium cases can be prevented.[10] Recent guidelines recommend the use of a multicomponent intervention to prevent delirium and provide evidence that such a strategy would be cost‐effective.[39] Nevertheless, such interventions are resource intense, requiring specialized nurse training and staffing[40] and have not been widely implemented. Acute care for the elderly units, where interventions to prevent delirium might logically be implemented, also require physical remodeling to provide carpeted hallways, handrails, and elevated toilet seats and door levers.[41] A method of risk stratification to identify the patients who would benefit most from resource‐intensive prevention strategies would be valuable.

The AWOL tool may provide a practical alternative to existing delirium prediction rules for adult medical inpatients. Because it can be completed by a nurse in <2 minutes, the AWOL tool may be easier to apply and disseminate than a previously described score relying on the MMSE, Acute Physiology and Chronic Health Evaluation scores, and measured visual acuity.[17] Two other tools, 1 based on chart abstraction[18] and the other based on clinical variables measured at admission,[19] are similarly easy to apply but only predict prevalent and not incident delirium, making them less clinically useful.

This study's strengths include its prospective cohort design and the derivation and validation being performed in different hospitals. The derivation cohort consisted of patients admitted to a tertiary care academic medical center or an affiliated hospital where routine mixed gender general medical patients are treated, whereas validation was performed at the SFVAMC, where patients are predominantly older men with a high incidence of vascular risk factors. The outcome was assessed on a daily basis, and the likelihood any cases were missed was low. Although there is some potential for bias because the outcome was assessed by a research assistant not blinded to baseline characteristics, this was mitigated by having each outcome validated by a blinded neurologist and in the validation cohort having the research assistant blinded to the AWOL score. Other strengths are the broad inclusion criteria, with both middle‐aged and elderly patients having a wide range of medical and neurological conditions, allowing for wide application of the results. Although many studies of delirium focus on patients over age 70 years, we chose to include patients aged 50 years or older because hospital‐acquired delirium still occurs in this age group (17 of 195 [8%] patients aged 5069 years became delirious in this study), and risk factors such as severe illness and cognitive dysfunction are likely to be predictors of delirium even at younger ages. Additionally, the inclusion of nurses' clinical judgment to assess illness severity using a straightforward rating scale allows bedside nurses to readily administer the prediction rule in practice.[34]

This study has several potential limitations. The number of outcomes in the derivation cohort was small compared to the number of predictors chosen for the prediction rule. This could potentially have led to overfitting the model in the derivation cohort and thus an overly optimistic estimation of the model's performance. In the validation cohort, the area under the ROC curve was lower than in the derivation cohort, and although the difference did not reach statistical significance, this may have been due to the small sample size. In addition, none of the 4 subjects with an AWOL score of 4 became delirious, potentially reflecting poor calibration of the prediction rule. However, the trend of higher rates of delirium among subjects with higher AWOL scores still reached statistical significance, and the prediction rule demonstrated good discrimination between patients at high and low risk for developing delirium.

To test whether a better prediction tool could be derived from our data, we combined the derivation and validation cohorts and repeated a stepwise multivariable logistic regression with the same variables used for derivation of the AWOL tool (with the exception of the whisper test of hearing and a past medical history of dementia, because these data were not collected in the validation cohort). This model produced the same 4 independent predictors of delirium used in the AWOL tool. We then used bootstrapping to internally validate the prediction rule, suggesting that the predictors in the AWOL tool were the best fit for the available data. However, given the small number of outcomes in our study, the AWOL tool may benefit from further validation in a larger independent cohort to more precisely calibrate the number of expected outcomes with each score.

Although the majority of medical inpatients were eligible for enrollment in our study, some populations were excluded, and our results may not generalize to these populations. Non‐English speaking patients were excluded to preserve the validity of our study instruments. In addition, patients with profound aphasia or an admission diagnosis of alcohol withdrawal were excluded. Patients discharged on the first day of their hospitalization were excluded either because they were discharged prior to screening or prior to their first follow‐up visit. Therefore, our results may only be valid in patients who remained in the hospital for over 24 hours. In addition, because we only included medical patients, our results cannot necessarily be generalized to the surgical population.

Finally, parts of the prediction rule (orientation and spelling world backward) are also components of the CAM and were used in the assessment of the outcome, and this may introduce a potential tautology: if patients are disoriented or have poor attention because they cannot spell world backward at admission, they already have fulfilled part of the criteria for delirium. However, a diagnosis of delirium using the CAM involves a comprehensive patient and caregiver interview, and in addition to poor attention, requires the presence of an acute change in mental status and disorganized thinking or altered level of consciousness. Therefore, it is possible, and common, for patients to be disoriented to place and/or unable to spell world backward, yet not be delirious, and predicting a subsequent change in cognition during the hospitalization is still clinically important. It is possible the AWOL tool works by identifying patients with impaired attention and subclinical delirium, but one could argue this makes a strong case for its validity because these patients especially should be triaged to an inpatient unit that specializes in delirium prevention. It is also possible the cognitive tasks that are part of the AWOL tool detect preexisting cognitive impairment, which is in turn a major risk factor for delirium.

Recognizing and classifying the risk of delirium during hospitalization is imperative, considering the illness' significant contribution to healthcare costs, morbidity, and mortality. The cost‐effectiveness of proven interventions to detect and prevent delirium could be magnified with focused implementation in those patients at highest risk.[39, 40, 41] Further research is required to determine whether the combination of delirium prediction rules such as those developed here and prevention strategies will result in decreased rates of delirium and economic savings for the healthcare system.

Disclosures

Drs. Douglas, Hessler, Dhaliwal, Betjemann, Lucatorto, Johnston, Josephson, and Ms. Fukuda and Ms. Alameddine have no conflicts of interest or financial disclosures. This research was made possible by the Ruth E. Raskin Fund and a UCSF Dean's Research Scholarship. These funding agencies had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.

Delirium is characterized by fluctuating disturbances in cognition and consciousness and is a common complication of hospitalization in medical and surgical patients. Studies estimate the prevalence of delirium in hospitalized patients[1] to be 14% to 56%, and up to 70% in critically ill elderly patients.[2] Estimates of total healthcare costs associated with delirium range from $38 to $152 billion per year in the United States.[3] Delirious patients are more likely to be discharged to a nursing home and have increased hospital mortality and longer lengths of stay.[4, 5, 6] Recent data suggest long‐term effects of delirium including cognitive impairments up to 1 year following the illness[7] and an increased likelihood of developing[8] or worsening dementia.[9]

It is estimated that one‐third of hospital‐acquired delirium cases could be prevented with appropriate interventions.[10] A prediction rule that easily and accurately identifies high‐risk patients upon admission could therefore have a substantial clinical impact. In addition, a prediction rule could be used to identify patients in whom new targeted interventions for delirium prevention could be investigated. A number of risk factors for delirium have been identified, including older age, preexisting cognitive dysfunction, vision and hearing impairment, severe illness, dehydration, electrolyte abnormalities, overmedication, and alcohol abuse.[11, 12, 13, 14, 15, 16] Existing prediction rules using various combinations of these measures have been limited by their complexity,[17] do not predict incident delirium,[18, 19] or are restricted to surgical[20, 21, 22] or intensive care[23] patients and therefore are not broadly applicable to the general medical population, which is particularly susceptible to developing delirium.

We conducted this study to develop a simple, efficient, and accurate prediction rule for hospital‐acquired delirium in adult medical inpatients assessed at the time of admission. Our a priori hypothesis was that a delirium prediction rule would consist of a combination of known risk factors and most likely incorporate old age, illness severity, and preexisting cognitive dysfunction.

METHODS

RESULTS

The derivation cohort consisted of elderly patients (mean age, 68.0811.96 years; interquartile range, 5096 years), and included more males than females (54.1% vs 45.9%). Subjects were predominantly white (73.7%) and lived at home (90%) (Table 1). The mean admission MMSE score was 27.0 (standard deviation [SD], 3.4; range, 730). Median follow‐up was 2 days (interquartile range, 13). Delirium developed in 12% (n=25) of the cohort.

Univariate analysis of the derivation study identified 10 variables significantly associated (P<0.05) with delirium (Table 2). Predictors of delirium included abnormal scores on 4 subtests of the MMSE, low score on the Mini‐Cog, living in an assisted living or skilled nursing facility, moderate to severe illness, old age, a past history of dementia, and hearing loss as assessed by the whisper test. These predictors were then entered into a stepwise logistic regression analysis that identified 4 independent predictors of delirium (Table 3).

These 4 independent predictors were assigned 1 point each if present to create a prediction rule with a range of possible scores from 0 to 4. There was a significant trend predicting higher rates of delirium with higher scores, with no subjects who scored 0 becoming delirious, compared to 40% of those subjects scoring 3 or 4 (P for trend<0.001) (Table 4).

The validation cohort consisted of adults with a mean age of 70.7210.6 years, (interquartile range, 5194 years), who were predominantly white (75.8%) and overwhelmingly male (95.2%) (Table 1). The mean admission MMSE score was 26.75 (SD, 2.8; range, 1730). Median follow‐up was 2 days (interquartile range, 15). Delirium developed in 8.5% (n=14) of the cohort. In the validation cohort, 4% of subjects with a score of 0 became delirious, whereas 19% of those scoring 3 or 4 became delirious (P for trend 0.025) (Table 4).

ROC curves were compared for the derivation and validation cohorts. The area under the ROC curve for the derivation cohort (0.81, 95% confidence interval [CI]: 0.720.90) was slightly better than that in the validation cohort (0.69, 95% CI: 0.540.83), but the difference did not reach statistical significance (P=0.14) (Figure 1).

Figure 1

DISCUSSION

We derived and validated a prediction rule to assess the risk of developing delirium in hospitalized adult medical patients. Four variables easily assessed on admission in a screen lasting less than 2 minutes were independently associated with the development of delirium. The prediction rule can be remembered with the following mnemonic: AWOL (Age80 years; unable to spell World backward; not fully Oriented to place; and moderate or severe iLlness severity).

It is estimated up to a third of hospital acquired delirium cases can be prevented.[10] Recent guidelines recommend the use of a multicomponent intervention to prevent delirium and provide evidence that such a strategy would be cost‐effective.[39] Nevertheless, such interventions are resource intense, requiring specialized nurse training and staffing[40] and have not been widely implemented. Acute care for the elderly units, where interventions to prevent delirium might logically be implemented, also require physical remodeling to provide carpeted hallways, handrails, and elevated toilet seats and door levers.[41] A method of risk stratification to identify the patients who would benefit most from resource‐intensive prevention strategies would be valuable.

The AWOL tool may provide a practical alternative to existing delirium prediction rules for adult medical inpatients. Because it can be completed by a nurse in <2 minutes, the AWOL tool may be easier to apply and disseminate than a previously described score relying on the MMSE, Acute Physiology and Chronic Health Evaluation scores, and measured visual acuity.[17] Two other tools, 1 based on chart abstraction[18] and the other based on clinical variables measured at admission,[19] are similarly easy to apply but only predict prevalent and not incident delirium, making them less clinically useful.

This study's strengths include its prospective cohort design and the derivation and validation being performed in different hospitals. The derivation cohort consisted of patients admitted to a tertiary care academic medical center or an affiliated hospital where routine mixed gender general medical patients are treated, whereas validation was performed at the SFVAMC, where patients are predominantly older men with a high incidence of vascular risk factors. The outcome was assessed on a daily basis, and the likelihood any cases were missed was low. Although there is some potential for bias because the outcome was assessed by a research assistant not blinded to baseline characteristics, this was mitigated by having each outcome validated by a blinded neurologist and in the validation cohort having the research assistant blinded to the AWOL score. Other strengths are the broad inclusion criteria, with both middle‐aged and elderly patients having a wide range of medical and neurological conditions, allowing for wide application of the results. Although many studies of delirium focus on patients over age 70 years, we chose to include patients aged 50 years or older because hospital‐acquired delirium still occurs in this age group (17 of 195 [8%] patients aged 5069 years became delirious in this study), and risk factors such as severe illness and cognitive dysfunction are likely to be predictors of delirium even at younger ages. Additionally, the inclusion of nurses' clinical judgment to assess illness severity using a straightforward rating scale allows bedside nurses to readily administer the prediction rule in practice.[34]

This study has several potential limitations. The number of outcomes in the derivation cohort was small compared to the number of predictors chosen for the prediction rule. This could potentially have led to overfitting the model in the derivation cohort and thus an overly optimistic estimation of the model's performance. In the validation cohort, the area under the ROC curve was lower than in the derivation cohort, and although the difference did not reach statistical significance, this may have been due to the small sample size. In addition, none of the 4 subjects with an AWOL score of 4 became delirious, potentially reflecting poor calibration of the prediction rule. However, the trend of higher rates of delirium among subjects with higher AWOL scores still reached statistical significance, and the prediction rule demonstrated good discrimination between patients at high and low risk for developing delirium.

To test whether a better prediction tool could be derived from our data, we combined the derivation and validation cohorts and repeated a stepwise multivariable logistic regression with the same variables used for derivation of the AWOL tool (with the exception of the whisper test of hearing and a past medical history of dementia, because these data were not collected in the validation cohort). This model produced the same 4 independent predictors of delirium used in the AWOL tool. We then used bootstrapping to internally validate the prediction rule, suggesting that the predictors in the AWOL tool were the best fit for the available data. However, given the small number of outcomes in our study, the AWOL tool may benefit from further validation in a larger independent cohort to more precisely calibrate the number of expected outcomes with each score.

Although the majority of medical inpatients were eligible for enrollment in our study, some populations were excluded, and our results may not generalize to these populations. Non‐English speaking patients were excluded to preserve the validity of our study instruments. In addition, patients with profound aphasia or an admission diagnosis of alcohol withdrawal were excluded. Patients discharged on the first day of their hospitalization were excluded either because they were discharged prior to screening or prior to their first follow‐up visit. Therefore, our results may only be valid in patients who remained in the hospital for over 24 hours. In addition, because we only included medical patients, our results cannot necessarily be generalized to the surgical population.

Finally, parts of the prediction rule (orientation and spelling world backward) are also components of the CAM and were used in the assessment of the outcome, and this may introduce a potential tautology: if patients are disoriented or have poor attention because they cannot spell world backward at admission, they already have fulfilled part of the criteria for delirium. However, a diagnosis of delirium using the CAM involves a comprehensive patient and caregiver interview, and in addition to poor attention, requires the presence of an acute change in mental status and disorganized thinking or altered level of consciousness. Therefore, it is possible, and common, for patients to be disoriented to place and/or unable to spell world backward, yet not be delirious, and predicting a subsequent change in cognition during the hospitalization is still clinically important. It is possible the AWOL tool works by identifying patients with impaired attention and subclinical delirium, but one could argue this makes a strong case for its validity because these patients especially should be triaged to an inpatient unit that specializes in delirium prevention. It is also possible the cognitive tasks that are part of the AWOL tool detect preexisting cognitive impairment, which is in turn a major risk factor for delirium.

Recognizing and classifying the risk of delirium during hospitalization is imperative, considering the illness' significant contribution to healthcare costs, morbidity, and mortality. The cost‐effectiveness of proven interventions to detect and prevent delirium could be magnified with focused implementation in those patients at highest risk.[39, 40, 41] Further research is required to determine whether the combination of delirium prediction rules such as those developed here and prevention strategies will result in decreased rates of delirium and economic savings for the healthcare system.

Disclosures

Drs. Douglas, Hessler, Dhaliwal, Betjemann, Lucatorto, Johnston, Josephson, and Ms. Fukuda and Ms. Alameddine have no conflicts of interest or financial disclosures. This research was made possible by the Ruth E. Raskin Fund and a UCSF Dean's Research Scholarship. These funding agencies had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.

The appropriate role of specialists in hospital management of common medical conditions has been vigorously debated.13 Few argue that specialists serve an important role as consultants, but whether patients with specific conditions should be admitted to the care of specialists or generalists is unresolved. This is demonstrated by the large degree of hospital‐to‐hospital variability in the proportion of patients with myocardial infarction admitted to cardiologists,4 patients with asthma exacerbations admitted to pulmonologists,5 and patients with renal failure admitted to nephrologists.6

Stroke is another common diagnosis, with variable rates of admission to specialists and generalists. Several prior studies have suggested that outcomes after ischemic stroke are better if a neurologist is the attending physician.710 However, these observational studies could not rule out the possibility that differences in outcome were a result of prognosis at the time of admission rather than improvements in medical care. Although these studies have controlled for known prognostic variables, it is possible that unknown, unmeasured, or inadequately measured variables were different in the groups admitted to neurologists and the groups admitted to generalists. These differences, in turn, might account for outcome differences rather than specialist care.

This form of selection bias, a type of confounding by indication, is a constant threat to validity in observational studies. Randomized trials avoid it because the randomization process balances all prognostic variables, both known and unknown, in the treatment groups.11 Observational studies cannot guarantee the same balance of unmeasured risk factors.12 Multivariate modeling is meant to account for prognostic differences between groups in observational studies, but confounding by indication may remain if all the factors that determine prognosis are not accurately measured. We developed a method to avoid confounding by indication by evaluating individual outcome differences associated with practice variability.13 This technique, termed grouped‐treatment (GT) analysis, is related to the instrumental variable approach developed by economists and occasionally applied to health services research.14

In multivariate GT analyses, the institutional proportion of cases admitted to the care of a neurologist is used as a predictor of outcomes rather than whether an individual patient was admitted to neurology. For example, at a hospital where three‐fourths of acute stroke patients are admitted to neurology, all patients are treated as having a 75% chance of admission to neurology. Rather than denoting whether each patient's specialist attending was a neurologist or a generalist, the 0.75 probability of admission to neurology is used for analysis. If admission to an attending neurologist improves ischemic stroke care, then GT analysis should demonstrate that hospitals admitting higher proportions of stroke patients to neurologists have improved outcomes regardless of whether there is selection bias at the individual patient level. In this way, the method bypasses unmeasured confounders at the individual level in its estimates of treatment effects. The method is susceptible to confounding at the group level; that is, unmeasured prognostic differences in patients admitted to hospitals that rely more heavily on neurologists could bias the GT estimate of treatment effect. The GT estimates are accurate if it can be assumed that a hospital's rate of treatment is not associated (in an unmeasured way) with its patient population's intrinsic, pretreatment prognosis. However, practice variability is very common between hospitals and is generally poorly associated with systematic differences in prognosis of treated patients,15, 16 and in this setting GT provides an independent assessment of treatment effect that may either confirm or refute an association found at the individual level, where confounding is nearly always an important issue.

In this study, we evaluated the impact of admission to a neurologist or generalist on outcomes of ischemic stroke patients treated at academic medical centers throughout the United States. We also compared traditional analysis to GT analysis. In doing so, we demonstrate the influence of unmeasured confounders on observational assessments of specialist care and may provide a more accurate measure of the impact of care by a neurologist on outcomes after ischemic stroke.

MATERIALS AND METHODS

We used the University HealthSystem Consortium (UHC) administrative database, which contained patient information from 84 large academic health centers and their 39 associate hospitals, with more than 2.1 million discharges each year.17 We obtained UHC discharge abstracts for all ischemic stroke patients admitted through emergency departments from 1997 through 1999. Discharge abstracts included patient demographics, urgency status (emergent, urgent, elective), illness severity class, admitting and discharge specialties, discharge diagnoses, procedure codes, in‐hospital mortality, length of stay, and total hospital charges. Patients were identified using International Classification of Diseases, Ninth Revision (ICD‐9) codes that were previously recognized as specific indicators of acute ischemic stroke (ICD‐9 433.01, 433.11, 433.21, 433.31, 433.81, 433.91, 434.01, 434.11, 434.91, 436).1820 We limited the cohort to emergency department admissions in order to reduce the likelihood of referral bias.

Variables in the discharge database were validated by comparison with a detailed medical record review. Between June and December 1999, 42 institutions participating in a quality improvement project identified 30 consecutive ischemic stroke cases. Trained analysts or clinicians abstracted information on demographics, medical history, and treatment. Kappa statistics have been previously reported for all individual characteristics except hospital charges, for which medical record review data were not available.21 Demographic and clinical variables in the administrative database tended to agree well with medical record review, with agreement ranging from 85% to 100% (kappa 0.581.00). Because the admitting attending likely directed acute stroke management, this was used to define a patient's attending physician specialty in all analyses. Administrative coding of tissue plasminogen activator (tPA) use was imperfect, with a sensitivity of 50% but a specificity of 100%.22

Institutional rate of admission to neurologists versus generalists was calculated as the percentage over the entire study duration. Unadjusted logistic regression was used to compare the distribution of patient pretreatment prognostic factors between institutions above and below the 50th percentile to determine a rate of admission to neurology because generalized estimating equations that could account for clustering were unable to support these models as a result of diverging estimates. We calculated the yearly volume of ischemic strokes treated at an institution from discharge abstracts, including admissions from all sources, because all treated cases would be expected to increase physician experience.

In‐hospital mortality was chosen as the primary outcome because of its frequency, importance, and coding reliability. Univariate predictors of in‐hospital mortality were identified using Pearson's chi‐square and the Wilcoxon rank sum tests.23 Length of stay (LOS), total hospital charges, and receipt of tPA were secondary outcomes. LOS and total hospital charges were compared using the Wilcoxon rank sum test. LOS and total charge calculations included only those patients surviving to discharge so that early mortality would not be confused with more efficient care. Similarly, we compared demographics and clinical variables of patients admitted to the care of neurologists with those of patients admitted to the care of generalists. To evaluate variability between institutions, we determined the proportion of patients with specific characteristics and outcomes at each institution and report median values and the 10th‐ to 90th‐percentile range among the institutions. The correlation between institutional rate of admission and institutional rate of mortality was evaluated.

In standard multivariate analysis, we assessed physician specialty as a predictor of in‐hospital mortality of individual patients after adjustment for demographic characteristics, admission status (emergent, urgent, elective), comorbid illness severity score (range 04, from 0 = no substantial comorbid illness to 4 = catastrophic comorbid illness), and annual institutional treatment volume of ischemic stroke. UHC defined severity class to represent an individual's overall calculated risk of illness; its value was dependent on the refinement of the Health Care Facility Administration's diagnosis‐related groups (DRGs) and the Sach's Complication Profiler count of total comorbidities present.24, 25 Effects on LOS and total charges, as well as the ability of physician specialty to predict tPA use in individual patients, were similarly evaluated. Analysis of tPA use was restricted to patients admitted to universities that ever coded tPA use, which increased the sensitivity of the indicator to 57%.22 Residual misclassification error of tPA use would be expected to obscure a true underlying association between its use and physician specialty.

In multivariate GT calculations, we used the institutional proportion of cases admitted to a neurologist as a predictor of outcomes. GT analysis is based on the observation that if a treatment is effective, hospitals that use it more frequently should have better patient outcomes and that this association should persist regardless of whether individual‐level selection bias is present. The method assumes that hospital rates of admission to neurology are independent on the patient population's pretreatment prognosis. Because utilization differences between hospitals likely reflect practice variability rather than differences in patient prognosis,15, 16 the influence of unmeasured confounders at the hospital level is expected to be small. Measured variables that proved significant in univariate analyses or were thought to be responsible for an association between overall patient prognosis and modalities and frequencies of acute stroke treatments used, such as institutional treatment volume, were included in the multivariate GT model in order to isolate the effect of increasing rates of admission to neurologists.

We included both institutional and individual data to more accurately specify individual outcomes and covariates compared with an analysis that simply compared institutions' characteristics and their outcomes.26 Generalized estimating equations (GEE) were used in order to account for institutional clustering of predictor variables and outcomes. GEE is similar to logistic regression but produces broader confidence intervals (CIs) because logistic regression ignores the possibility that individuals at institutions are more similar to each other than would be expected by chance alone. We used a compound symmetry correlation structure, which initiates modeling by assuming a constant correlation between observations within each institution as well as between institutions, and used a logistic link function for binary outcomes in order to mimic logistic regression. The natural log transformations of LOS and hospital charges were modeled to reduce positive skew and approximate a normal distribution, and an identity link function was used in GEE to mimic linear regression for these analyses. To evaluate the impact of adjustment, both unadjusted and adjusted analyses were conducted. Methods to calculate power of GT analysis are not available. The Stata statistical package was used for all analyses (version 8.0; Stata Corporation, College Station, TX).

RESULTS

A total of 26,925 patients with ischemic strokes were admitted to neurologists or generalists through the emergency department at 113 institutions participating in the study. Patients admitted to neurologists rather than generalists (Table 1) were younger and more likely to be male, but less likely to have a serious comorbid illness. Institutions varied widely in the demographics of treated patients as well as in the markers of pretreatment prognosis. Institutional annual case volume of all ischemic strokes ranged from 1 to 741. Mortality rate, mean LOS, and mean hospital charges also varied broadly between institutions (Table 1). Patients treated at institutions whose rate of admissions to a neurologist's care was in the upper 50th percentile were younger and more often male, but did not differ in illness severity class (Table 2).

There were 1760 in‐hospital deaths (7.0%). In univariate analysis, older age (P < .001), white ethnicity (P < .001), emergent stroke (P < .001), and increased illness severity (P < .001) were associated with greater risk of death, whereas African‐American (P < .001) and Hispanic (P = .007) ethnicities were protective. No other patient characteristics were important, and institutional annual case volume showed no association with mortality risk.

Overall, 60% of patients with ischemic stroke were admitted to a neurologist's care. In univariate analysis (Table 3), a lower risk of in‐hospital mortality was observed in cases admitted to neurologists (4.6%) compared with those admitted to generalists (9.5%; P < .001). After adjustment in standard multivariable models, the association between neurologist admission and lower risk of death persisted (OR 0.60; 95% CI, 0.500.72; P < .001).

The institutional rate of admission of ischemic stroke patients to neurologists ranged from 0% to 90%, and higher rates were seen at hospitals with higher institutional case volumes (P < .001). There was no correlation between the institutional rate of admission to neurology and the institutional mortality rate (0.33; P = .73). At the individual‐level, greater rates of admission to neurologists had no significant impact on mortality (OR 1.05; 95% CI, 0.851.31; P = .64; Table 3) in unadjusted analysis. After adjustment for patient demographics, comorbid illness severity score, urgency status, and institutional case volume in GT analysis, there remained no association between death and proportion of ischemic stroke cases admitted to neurologists (OR 1.02; 95% CI, 0.791.30; P = .90), consistent with the absence of an association between neurologist care and in‐hospital mortality.

Patients treated by neurologists were likely to have shorter stays (P < .001) and lower charges (P = .01) in univariate analysis (Table 4). In traditional adjusted multivariable analysis, the same associations were seen for LOS (P < .001) and charges (P = .05). However, in adjusted GT analyses, increased institutional rate of admission to neurologists was not associated with briefer LOS (P = .36) and was associated with greater hospital charges (P = .044).

In 1999, 190 (2.2%) ischemic stroke patients received tPA at the 64 universities that had ever coded tPA use. In univariate analysis, patients admitted to a neurologist were more likely to have received tPA (P = .001; Table 3), and this association persisted after adjustment (P < .001). In adjusted GT analysis, institutions admitting a higher proportion of ischemic stroke patients to neurologists also treated patients with tPA more frequently (P = .033).

DISCUSSION

Several prior studies found that ischemic stroke outcomes were better when an attending neurologist was responsible for patient care.710 Traditional analyses of our data also indicate that care by a neurologist lowers inpatient mortality, LOS, and total charges. By contrast, a GT analysis that bypasses selection bias at the patient level suggests there is no independent benefit of neurologist care on mortality or LOS and actually shows higher associated charges.

The discrepancy between standard and GT analyses suggests that healthier patients may have been preferentially admitted to the care of neurologists. Measured pretreatment prognostic factors in our data present a mixed picture. Patients admitted to a neurologist's care were younger, more often male, more often emergently admitted, and less likely to have serious comorbid illnesses. These patient factors were controlled for in all adjusted analyses. Although traditional multivariate analysis attempts to adjust for variations between the 2 patient populations, it cannot adjust for inaccurately measured or unmeasured differences. Using the institutional proportion of admissions to neurologists as a predictor of patient outcomes, we were better able to control for the selection bias associated with differential distribution of patients to teams led by attending neurologists versus generalists.13, 14

Petty et al.7 studied 299 ischemic stroke patients and showed equivalent survival among stroke patients admitted to neurology inpatient teams versus generalist teams with neurologic consultation. However, patients cared for by generalist teams without neurologic consultation fared worse. Their subjects were treated at both academic and community hospitals. In our study, contributing hospitals were solely academic institutions. Because specialty cross talk may be more frequent at university‐based hospitals, academic‐based generalist physicians may be more familiar with recent stroke literature and guidelines than are their community‐based peers. Further, restricting analysis to academic centers in our study should have reduced the potential confounding influences of differences between other aspects of institutional care. Although no information was available on neurologist consultation in our database, informal consultation is believed to play a large but hidden role at academic medical centers. Thus, the inclusion of a formal consultation variable may be misleading at academic medical centers.

Analyzing claims data on 44,099 Medicare beneficiaries with acute ischemic strokes cared for at both academic and community hospitals, Smith et al.10 also recently reported a 10% lower risk of 30‐day mortality and 12% lower risk of rehospitalization for infections and aspiration pneumonitis among patients admitted to the care of neurologists compared with those admitted to the care of generalists. However, the upper 95% confidence interval limits for these 2 findings nearly crossed 1 (ranging from 0.9980.999). The study also concluded that patients cared for collaboratively by generalists and neurologists had a 16% lower 30‐day mortality risk (hazard ratio 0.84; 95% CI, 0.790.90) than those cared for by generalists alone but simultaneously noted that patients admitted to generalists only had more comorbidities than either the collaborative care or neurologist‐only patient groups. If sicker patients were triaged to generalist admission, as occurs in confounding by indication (also known as channeling bias), then incomplete adjustment for comorbid disease may bias outcomes in favor of neurologist involvement. The GT analysis we employed is specifically designed to overcome this exact type of selection bias.

In our study, patients admitted to neurologists received tPA significantly more often than those admitted to generalists. GT analysis also found that hospitals admitting a higher proportion of strokes to neurologists treated more patients with tPA. This result is consistent with a prior study demonstrating that academic institutions employing a vascular neurologist had significantly higher odds of administering tPA.21 Since tPA must be administered within 3 hours of symptom onset,27 it is commonly delivered in the emergency department prior to admission. Thus, patients may be preferentially selected for admission to neurologic services because of their receipt of tPA, rather than that this association reflects an actual increased use of tPA by neurologists over generalists. Alternatively, institutions with a higher rate of stroke admissions to neurology may simply be more familiar with tPA protocols. Importantly, the poor sensitivity of our data for actual tPA administration may affect the analysis of its use by physician specialty; however, the failure to administratively code tPA use is unlikely to be differentially biased based on physician specialty. Thus, undercoding of tPA use would be expected to bias these analyses toward the null.

The potential advantage and efficacy of stroke centers, stroke units, stroke services, and other institutional processes of care are not addressed by our data. Previously, among academic hospitals, we found that acute ischemic stroke mortality was lower at hospitals employing a vascular neurologist and at those whose guidelines allowed only neurologists to administer tPA.21 A later analysis evaluated the impact of all elements of stroke center care supported by the original Brain Attack Coalition consensus28 and found that no single element improved mortality.29 However, recent studies have found significant mortality benefit associated with stroke units30, 31 and stroke services.32 Clearly, the debate continues over these important questions.

Our study had several limitations. First, generalizability may be lessened because only academic medical centers contributed data and only admissions through the ED were included. However, limiting the study population to academic centers provided a homogenous study population and greatly reduced the potential for confounding at the institutional level. Although the selection of ED cases mitigated the effects of referral bias and the use of only academic hospitals minimized interinstitutional differences, institutions whose rate of admissions to neurology was above the 50th percentile differed from those whose rate admissions to neurology was below the 50th percentile. However, this difference did not consistently result in patients with worse pretreatment prognostic factors being cared for at hospitals with higher rates of admission to neurology. Second, there are important limitations to using administrative data. In our study, patients were selected based on diagnostic coding of records analysts at discharge, and the diagnostic accuracy of such coding for stroke is imperfect.33 Furthermore, missing or incomplete information could have impaired adjustments for patient differences. Third, details of patient treatment were limited. The lack of information about formal and informal consultations may have obscured a true difference in outcomes among specialties.7 Additionally, academic institutions may use systematized care plans more often than do community hospitals, potentially minimizing differences between specialties. Fourth, at the time of our study, tPA had been recently introduced into stroke care. Current rates of tPA use among neurologists and generalists may be more similar. Fifth, the ability of in‐hospital mortality to adequately assess quality of care is limited, and longer‐term and functional outcomes would be better measures and more clinically relevant.

After controlling for selection bias using GT analysis, we found stroke outcomes to be similar regardless of whether a neurologist or a generalist was the admitting physician. This result contrasts with the findings of several previous studies that suggested admitting stroke patients to a neurologist resulted in better clinical outcomes.710 Because only 1 neurologist is employed for approximately every 19.8 generalists in the United States34 and 40% of acute strokes were cared for by generalists, even in this sample entirely restricted to university hospitals, such findings would suggest that many U.S. stroke patients receive inferior care. Because the role of the neurologist as consultant rather than as attending physician is significantly more feasible in most practice settings, the demonstration of equivalent outcomes by both types of physicians is reassuring and certainly reinforces the important role that unmeasured confounders may play in observational studies.

However, these results do imply that it is vital that generalists remain fully trained in the current best practices of acute stroke management in order to maintain the equivalence of care suggested here. Given how common acute stroke is, any proposed future hospitalist training, certification, and recertification programs should include a focus on acute stroke management.

The appropriate role of specialists in hospital management of common medical conditions has been vigorously debated.13 Few argue that specialists serve an important role as consultants, but whether patients with specific conditions should be admitted to the care of specialists or generalists is unresolved. This is demonstrated by the large degree of hospital‐to‐hospital variability in the proportion of patients with myocardial infarction admitted to cardiologists,4 patients with asthma exacerbations admitted to pulmonologists,5 and patients with renal failure admitted to nephrologists.6

Stroke is another common diagnosis, with variable rates of admission to specialists and generalists. Several prior studies have suggested that outcomes after ischemic stroke are better if a neurologist is the attending physician.710 However, these observational studies could not rule out the possibility that differences in outcome were a result of prognosis at the time of admission rather than improvements in medical care. Although these studies have controlled for known prognostic variables, it is possible that unknown, unmeasured, or inadequately measured variables were different in the groups admitted to neurologists and the groups admitted to generalists. These differences, in turn, might account for outcome differences rather than specialist care.

This form of selection bias, a type of confounding by indication, is a constant threat to validity in observational studies. Randomized trials avoid it because the randomization process balances all prognostic variables, both known and unknown, in the treatment groups.11 Observational studies cannot guarantee the same balance of unmeasured risk factors.12 Multivariate modeling is meant to account for prognostic differences between groups in observational studies, but confounding by indication may remain if all the factors that determine prognosis are not accurately measured. We developed a method to avoid confounding by indication by evaluating individual outcome differences associated with practice variability.13 This technique, termed grouped‐treatment (GT) analysis, is related to the instrumental variable approach developed by economists and occasionally applied to health services research.14

In multivariate GT analyses, the institutional proportion of cases admitted to the care of a neurologist is used as a predictor of outcomes rather than whether an individual patient was admitted to neurology. For example, at a hospital where three‐fourths of acute stroke patients are admitted to neurology, all patients are treated as having a 75% chance of admission to neurology. Rather than denoting whether each patient's specialist attending was a neurologist or a generalist, the 0.75 probability of admission to neurology is used for analysis. If admission to an attending neurologist improves ischemic stroke care, then GT analysis should demonstrate that hospitals admitting higher proportions of stroke patients to neurologists have improved outcomes regardless of whether there is selection bias at the individual patient level. In this way, the method bypasses unmeasured confounders at the individual level in its estimates of treatment effects. The method is susceptible to confounding at the group level; that is, unmeasured prognostic differences in patients admitted to hospitals that rely more heavily on neurologists could bias the GT estimate of treatment effect. The GT estimates are accurate if it can be assumed that a hospital's rate of treatment is not associated (in an unmeasured way) with its patient population's intrinsic, pretreatment prognosis. However, practice variability is very common between hospitals and is generally poorly associated with systematic differences in prognosis of treated patients,15, 16 and in this setting GT provides an independent assessment of treatment effect that may either confirm or refute an association found at the individual level, where confounding is nearly always an important issue.

In this study, we evaluated the impact of admission to a neurologist or generalist on outcomes of ischemic stroke patients treated at academic medical centers throughout the United States. We also compared traditional analysis to GT analysis. In doing so, we demonstrate the influence of unmeasured confounders on observational assessments of specialist care and may provide a more accurate measure of the impact of care by a neurologist on outcomes after ischemic stroke.

MATERIALS AND METHODS

We used the University HealthSystem Consortium (UHC) administrative database, which contained patient information from 84 large academic health centers and their 39 associate hospitals, with more than 2.1 million discharges each year.17 We obtained UHC discharge abstracts for all ischemic stroke patients admitted through emergency departments from 1997 through 1999. Discharge abstracts included patient demographics, urgency status (emergent, urgent, elective), illness severity class, admitting and discharge specialties, discharge diagnoses, procedure codes, in‐hospital mortality, length of stay, and total hospital charges. Patients were identified using International Classification of Diseases, Ninth Revision (ICD‐9) codes that were previously recognized as specific indicators of acute ischemic stroke (ICD‐9 433.01, 433.11, 433.21, 433.31, 433.81, 433.91, 434.01, 434.11, 434.91, 436).1820 We limited the cohort to emergency department admissions in order to reduce the likelihood of referral bias.

Variables in the discharge database were validated by comparison with a detailed medical record review. Between June and December 1999, 42 institutions participating in a quality improvement project identified 30 consecutive ischemic stroke cases. Trained analysts or clinicians abstracted information on demographics, medical history, and treatment. Kappa statistics have been previously reported for all individual characteristics except hospital charges, for which medical record review data were not available.21 Demographic and clinical variables in the administrative database tended to agree well with medical record review, with agreement ranging from 85% to 100% (kappa 0.581.00). Because the admitting attending likely directed acute stroke management, this was used to define a patient's attending physician specialty in all analyses. Administrative coding of tissue plasminogen activator (tPA) use was imperfect, with a sensitivity of 50% but a specificity of 100%.22

Institutional rate of admission to neurologists versus generalists was calculated as the percentage over the entire study duration. Unadjusted logistic regression was used to compare the distribution of patient pretreatment prognostic factors between institutions above and below the 50th percentile to determine a rate of admission to neurology because generalized estimating equations that could account for clustering were unable to support these models as a result of diverging estimates. We calculated the yearly volume of ischemic strokes treated at an institution from discharge abstracts, including admissions from all sources, because all treated cases would be expected to increase physician experience.

In‐hospital mortality was chosen as the primary outcome because of its frequency, importance, and coding reliability. Univariate predictors of in‐hospital mortality were identified using Pearson's chi‐square and the Wilcoxon rank sum tests.23 Length of stay (LOS), total hospital charges, and receipt of tPA were secondary outcomes. LOS and total hospital charges were compared using the Wilcoxon rank sum test. LOS and total charge calculations included only those patients surviving to discharge so that early mortality would not be confused with more efficient care. Similarly, we compared demographics and clinical variables of patients admitted to the care of neurologists with those of patients admitted to the care of generalists. To evaluate variability between institutions, we determined the proportion of patients with specific characteristics and outcomes at each institution and report median values and the 10th‐ to 90th‐percentile range among the institutions. The correlation between institutional rate of admission and institutional rate of mortality was evaluated.

In standard multivariate analysis, we assessed physician specialty as a predictor of in‐hospital mortality of individual patients after adjustment for demographic characteristics, admission status (emergent, urgent, elective), comorbid illness severity score (range 04, from 0 = no substantial comorbid illness to 4 = catastrophic comorbid illness), and annual institutional treatment volume of ischemic stroke. UHC defined severity class to represent an individual's overall calculated risk of illness; its value was dependent on the refinement of the Health Care Facility Administration's diagnosis‐related groups (DRGs) and the Sach's Complication Profiler count of total comorbidities present.24, 25 Effects on LOS and total charges, as well as the ability of physician specialty to predict tPA use in individual patients, were similarly evaluated. Analysis of tPA use was restricted to patients admitted to universities that ever coded tPA use, which increased the sensitivity of the indicator to 57%.22 Residual misclassification error of tPA use would be expected to obscure a true underlying association between its use and physician specialty.

In multivariate GT calculations, we used the institutional proportion of cases admitted to a neurologist as a predictor of outcomes. GT analysis is based on the observation that if a treatment is effective, hospitals that use it more frequently should have better patient outcomes and that this association should persist regardless of whether individual‐level selection bias is present. The method assumes that hospital rates of admission to neurology are independent on the patient population's pretreatment prognosis. Because utilization differences between hospitals likely reflect practice variability rather than differences in patient prognosis,15, 16 the influence of unmeasured confounders at the hospital level is expected to be small. Measured variables that proved significant in univariate analyses or were thought to be responsible for an association between overall patient prognosis and modalities and frequencies of acute stroke treatments used, such as institutional treatment volume, were included in the multivariate GT model in order to isolate the effect of increasing rates of admission to neurologists.

We included both institutional and individual data to more accurately specify individual outcomes and covariates compared with an analysis that simply compared institutions' characteristics and their outcomes.26 Generalized estimating equations (GEE) were used in order to account for institutional clustering of predictor variables and outcomes. GEE is similar to logistic regression but produces broader confidence intervals (CIs) because logistic regression ignores the possibility that individuals at institutions are more similar to each other than would be expected by chance alone. We used a compound symmetry correlation structure, which initiates modeling by assuming a constant correlation between observations within each institution as well as between institutions, and used a logistic link function for binary outcomes in order to mimic logistic regression. The natural log transformations of LOS and hospital charges were modeled to reduce positive skew and approximate a normal distribution, and an identity link function was used in GEE to mimic linear regression for these analyses. To evaluate the impact of adjustment, both unadjusted and adjusted analyses were conducted. Methods to calculate power of GT analysis are not available. The Stata statistical package was used for all analyses (version 8.0; Stata Corporation, College Station, TX).

RESULTS

A total of 26,925 patients with ischemic strokes were admitted to neurologists or generalists through the emergency department at 113 institutions participating in the study. Patients admitted to neurologists rather than generalists (Table 1) were younger and more likely to be male, but less likely to have a serious comorbid illness. Institutions varied widely in the demographics of treated patients as well as in the markers of pretreatment prognosis. Institutional annual case volume of all ischemic strokes ranged from 1 to 741. Mortality rate, mean LOS, and mean hospital charges also varied broadly between institutions (Table 1). Patients treated at institutions whose rate of admissions to a neurologist's care was in the upper 50th percentile were younger and more often male, but did not differ in illness severity class (Table 2).

There were 1760 in‐hospital deaths (7.0%). In univariate analysis, older age (P < .001), white ethnicity (P < .001), emergent stroke (P < .001), and increased illness severity (P < .001) were associated with greater risk of death, whereas African‐American (P < .001) and Hispanic (P = .007) ethnicities were protective. No other patient characteristics were important, and institutional annual case volume showed no association with mortality risk.

Overall, 60% of patients with ischemic stroke were admitted to a neurologist's care. In univariate analysis (Table 3), a lower risk of in‐hospital mortality was observed in cases admitted to neurologists (4.6%) compared with those admitted to generalists (9.5%; P < .001). After adjustment in standard multivariable models, the association between neurologist admission and lower risk of death persisted (OR 0.60; 95% CI, 0.500.72; P < .001).

The institutional rate of admission of ischemic stroke patients to neurologists ranged from 0% to 90%, and higher rates were seen at hospitals with higher institutional case volumes (P < .001). There was no correlation between the institutional rate of admission to neurology and the institutional mortality rate (0.33; P = .73). At the individual‐level, greater rates of admission to neurologists had no significant impact on mortality (OR 1.05; 95% CI, 0.851.31; P = .64; Table 3) in unadjusted analysis. After adjustment for patient demographics, comorbid illness severity score, urgency status, and institutional case volume in GT analysis, there remained no association between death and proportion of ischemic stroke cases admitted to neurologists (OR 1.02; 95% CI, 0.791.30; P = .90), consistent with the absence of an association between neurologist care and in‐hospital mortality.

Patients treated by neurologists were likely to have shorter stays (P < .001) and lower charges (P = .01) in univariate analysis (Table 4). In traditional adjusted multivariable analysis, the same associations were seen for LOS (P < .001) and charges (P = .05). However, in adjusted GT analyses, increased institutional rate of admission to neurologists was not associated with briefer LOS (P = .36) and was associated with greater hospital charges (P = .044).

In 1999, 190 (2.2%) ischemic stroke patients received tPA at the 64 universities that had ever coded tPA use. In univariate analysis, patients admitted to a neurologist were more likely to have received tPA (P = .001; Table 3), and this association persisted after adjustment (P < .001). In adjusted GT analysis, institutions admitting a higher proportion of ischemic stroke patients to neurologists also treated patients with tPA more frequently (P = .033).

DISCUSSION

Several prior studies found that ischemic stroke outcomes were better when an attending neurologist was responsible for patient care.710 Traditional analyses of our data also indicate that care by a neurologist lowers inpatient mortality, LOS, and total charges. By contrast, a GT analysis that bypasses selection bias at the patient level suggests there is no independent benefit of neurologist care on mortality or LOS and actually shows higher associated charges.

The discrepancy between standard and GT analyses suggests that healthier patients may have been preferentially admitted to the care of neurologists. Measured pretreatment prognostic factors in our data present a mixed picture. Patients admitted to a neurologist's care were younger, more often male, more often emergently admitted, and less likely to have serious comorbid illnesses. These patient factors were controlled for in all adjusted analyses. Although traditional multivariate analysis attempts to adjust for variations between the 2 patient populations, it cannot adjust for inaccurately measured or unmeasured differences. Using the institutional proportion of admissions to neurologists as a predictor of patient outcomes, we were better able to control for the selection bias associated with differential distribution of patients to teams led by attending neurologists versus generalists.13, 14

Petty et al.7 studied 299 ischemic stroke patients and showed equivalent survival among stroke patients admitted to neurology inpatient teams versus generalist teams with neurologic consultation. However, patients cared for by generalist teams without neurologic consultation fared worse. Their subjects were treated at both academic and community hospitals. In our study, contributing hospitals were solely academic institutions. Because specialty cross talk may be more frequent at university‐based hospitals, academic‐based generalist physicians may be more familiar with recent stroke literature and guidelines than are their community‐based peers. Further, restricting analysis to academic centers in our study should have reduced the potential confounding influences of differences between other aspects of institutional care. Although no information was available on neurologist consultation in our database, informal consultation is believed to play a large but hidden role at academic medical centers. Thus, the inclusion of a formal consultation variable may be misleading at academic medical centers.

Analyzing claims data on 44,099 Medicare beneficiaries with acute ischemic strokes cared for at both academic and community hospitals, Smith et al.10 also recently reported a 10% lower risk of 30‐day mortality and 12% lower risk of rehospitalization for infections and aspiration pneumonitis among patients admitted to the care of neurologists compared with those admitted to the care of generalists. However, the upper 95% confidence interval limits for these 2 findings nearly crossed 1 (ranging from 0.9980.999). The study also concluded that patients cared for collaboratively by generalists and neurologists had a 16% lower 30‐day mortality risk (hazard ratio 0.84; 95% CI, 0.790.90) than those cared for by generalists alone but simultaneously noted that patients admitted to generalists only had more comorbidities than either the collaborative care or neurologist‐only patient groups. If sicker patients were triaged to generalist admission, as occurs in confounding by indication (also known as channeling bias), then incomplete adjustment for comorbid disease may bias outcomes in favor of neurologist involvement. The GT analysis we employed is specifically designed to overcome this exact type of selection bias.

In our study, patients admitted to neurologists received tPA significantly more often than those admitted to generalists. GT analysis also found that hospitals admitting a higher proportion of strokes to neurologists treated more patients with tPA. This result is consistent with a prior study demonstrating that academic institutions employing a vascular neurologist had significantly higher odds of administering tPA.21 Since tPA must be administered within 3 hours of symptom onset,27 it is commonly delivered in the emergency department prior to admission. Thus, patients may be preferentially selected for admission to neurologic services because of their receipt of tPA, rather than that this association reflects an actual increased use of tPA by neurologists over generalists. Alternatively, institutions with a higher rate of stroke admissions to neurology may simply be more familiar with tPA protocols. Importantly, the poor sensitivity of our data for actual tPA administration may affect the analysis of its use by physician specialty; however, the failure to administratively code tPA use is unlikely to be differentially biased based on physician specialty. Thus, undercoding of tPA use would be expected to bias these analyses toward the null.

The potential advantage and efficacy of stroke centers, stroke units, stroke services, and other institutional processes of care are not addressed by our data. Previously, among academic hospitals, we found that acute ischemic stroke mortality was lower at hospitals employing a vascular neurologist and at those whose guidelines allowed only neurologists to administer tPA.21 A later analysis evaluated the impact of all elements of stroke center care supported by the original Brain Attack Coalition consensus28 and found that no single element improved mortality.29 However, recent studies have found significant mortality benefit associated with stroke units30, 31 and stroke services.32 Clearly, the debate continues over these important questions.

Our study had several limitations. First, generalizability may be lessened because only academic medical centers contributed data and only admissions through the ED were included. However, limiting the study population to academic centers provided a homogenous study population and greatly reduced the potential for confounding at the institutional level. Although the selection of ED cases mitigated the effects of referral bias and the use of only academic hospitals minimized interinstitutional differences, institutions whose rate of admissions to neurology was above the 50th percentile differed from those whose rate admissions to neurology was below the 50th percentile. However, this difference did not consistently result in patients with worse pretreatment prognostic factors being cared for at hospitals with higher rates of admission to neurology. Second, there are important limitations to using administrative data. In our study, patients were selected based on diagnostic coding of records analysts at discharge, and the diagnostic accuracy of such coding for stroke is imperfect.33 Furthermore, missing or incomplete information could have impaired adjustments for patient differences. Third, details of patient treatment were limited. The lack of information about formal and informal consultations may have obscured a true difference in outcomes among specialties.7 Additionally, academic institutions may use systematized care plans more often than do community hospitals, potentially minimizing differences between specialties. Fourth, at the time of our study, tPA had been recently introduced into stroke care. Current rates of tPA use among neurologists and generalists may be more similar. Fifth, the ability of in‐hospital mortality to adequately assess quality of care is limited, and longer‐term and functional outcomes would be better measures and more clinically relevant.

After controlling for selection bias using GT analysis, we found stroke outcomes to be similar regardless of whether a neurologist or a generalist was the admitting physician. This result contrasts with the findings of several previous studies that suggested admitting stroke patients to a neurologist resulted in better clinical outcomes.710 Because only 1 neurologist is employed for approximately every 19.8 generalists in the United States34 and 40% of acute strokes were cared for by generalists, even in this sample entirely restricted to university hospitals, such findings would suggest that many U.S. stroke patients receive inferior care. Because the role of the neurologist as consultant rather than as attending physician is significantly more feasible in most practice settings, the demonstration of equivalent outcomes by both types of physicians is reassuring and certainly reinforces the important role that unmeasured confounders may play in observational studies.

However, these results do imply that it is vital that generalists remain fully trained in the current best practices of acute stroke management in order to maintain the equivalence of care suggested here. Given how common acute stroke is, any proposed future hospitalist training, certification, and recertification programs should include a focus on acute stroke management.

The appropriate role of specialists in hospital management of common medical conditions has been vigorously debated.13 Few argue that specialists serve an important role as consultants, but whether patients with specific conditions should be admitted to the care of specialists or generalists is unresolved. This is demonstrated by the large degree of hospital‐to‐hospital variability in the proportion of patients with myocardial infarction admitted to cardiologists,4 patients with asthma exacerbations admitted to pulmonologists,5 and patients with renal failure admitted to nephrologists.6

Stroke is another common diagnosis, with variable rates of admission to specialists and generalists. Several prior studies have suggested that outcomes after ischemic stroke are better if a neurologist is the attending physician.710 However, these observational studies could not rule out the possibility that differences in outcome were a result of prognosis at the time of admission rather than improvements in medical care. Although these studies have controlled for known prognostic variables, it is possible that unknown, unmeasured, or inadequately measured variables were different in the groups admitted to neurologists and the groups admitted to generalists. These differences, in turn, might account for outcome differences rather than specialist care.

This form of selection bias, a type of confounding by indication, is a constant threat to validity in observational studies. Randomized trials avoid it because the randomization process balances all prognostic variables, both known and unknown, in the treatment groups.11 Observational studies cannot guarantee the same balance of unmeasured risk factors.12 Multivariate modeling is meant to account for prognostic differences between groups in observational studies, but confounding by indication may remain if all the factors that determine prognosis are not accurately measured. We developed a method to avoid confounding by indication by evaluating individual outcome differences associated with practice variability.13 This technique, termed grouped‐treatment (GT) analysis, is related to the instrumental variable approach developed by economists and occasionally applied to health services research.14

In multivariate GT analyses, the institutional proportion of cases admitted to the care of a neurologist is used as a predictor of outcomes rather than whether an individual patient was admitted to neurology. For example, at a hospital where three‐fourths of acute stroke patients are admitted to neurology, all patients are treated as having a 75% chance of admission to neurology. Rather than denoting whether each patient's specialist attending was a neurologist or a generalist, the 0.75 probability of admission to neurology is used for analysis. If admission to an attending neurologist improves ischemic stroke care, then GT analysis should demonstrate that hospitals admitting higher proportions of stroke patients to neurologists have improved outcomes regardless of whether there is selection bias at the individual patient level. In this way, the method bypasses unmeasured confounders at the individual level in its estimates of treatment effects. The method is susceptible to confounding at the group level; that is, unmeasured prognostic differences in patients admitted to hospitals that rely more heavily on neurologists could bias the GT estimate of treatment effect. The GT estimates are accurate if it can be assumed that a hospital's rate of treatment is not associated (in an unmeasured way) with its patient population's intrinsic, pretreatment prognosis. However, practice variability is very common between hospitals and is generally poorly associated with systematic differences in prognosis of treated patients,15, 16 and in this setting GT provides an independent assessment of treatment effect that may either confirm or refute an association found at the individual level, where confounding is nearly always an important issue.

In this study, we evaluated the impact of admission to a neurologist or generalist on outcomes of ischemic stroke patients treated at academic medical centers throughout the United States. We also compared traditional analysis to GT analysis. In doing so, we demonstrate the influence of unmeasured confounders on observational assessments of specialist care and may provide a more accurate measure of the impact of care by a neurologist on outcomes after ischemic stroke.

MATERIALS AND METHODS

We used the University HealthSystem Consortium (UHC) administrative database, which contained patient information from 84 large academic health centers and their 39 associate hospitals, with more than 2.1 million discharges each year.17 We obtained UHC discharge abstracts for all ischemic stroke patients admitted through emergency departments from 1997 through 1999. Discharge abstracts included patient demographics, urgency status (emergent, urgent, elective), illness severity class, admitting and discharge specialties, discharge diagnoses, procedure codes, in‐hospital mortality, length of stay, and total hospital charges. Patients were identified using International Classification of Diseases, Ninth Revision (ICD‐9) codes that were previously recognized as specific indicators of acute ischemic stroke (ICD‐9 433.01, 433.11, 433.21, 433.31, 433.81, 433.91, 434.01, 434.11, 434.91, 436).1820 We limited the cohort to emergency department admissions in order to reduce the likelihood of referral bias.

Variables in the discharge database were validated by comparison with a detailed medical record review. Between June and December 1999, 42 institutions participating in a quality improvement project identified 30 consecutive ischemic stroke cases. Trained analysts or clinicians abstracted information on demographics, medical history, and treatment. Kappa statistics have been previously reported for all individual characteristics except hospital charges, for which medical record review data were not available.21 Demographic and clinical variables in the administrative database tended to agree well with medical record review, with agreement ranging from 85% to 100% (kappa 0.581.00). Because the admitting attending likely directed acute stroke management, this was used to define a patient's attending physician specialty in all analyses. Administrative coding of tissue plasminogen activator (tPA) use was imperfect, with a sensitivity of 50% but a specificity of 100%.22

Institutional rate of admission to neurologists versus generalists was calculated as the percentage over the entire study duration. Unadjusted logistic regression was used to compare the distribution of patient pretreatment prognostic factors between institutions above and below the 50th percentile to determine a rate of admission to neurology because generalized estimating equations that could account for clustering were unable to support these models as a result of diverging estimates. We calculated the yearly volume of ischemic strokes treated at an institution from discharge abstracts, including admissions from all sources, because all treated cases would be expected to increase physician experience.

In‐hospital mortality was chosen as the primary outcome because of its frequency, importance, and coding reliability. Univariate predictors of in‐hospital mortality were identified using Pearson's chi‐square and the Wilcoxon rank sum tests.23 Length of stay (LOS), total hospital charges, and receipt of tPA were secondary outcomes. LOS and total hospital charges were compared using the Wilcoxon rank sum test. LOS and total charge calculations included only those patients surviving to discharge so that early mortality would not be confused with more efficient care. Similarly, we compared demographics and clinical variables of patients admitted to the care of neurologists with those of patients admitted to the care of generalists. To evaluate variability between institutions, we determined the proportion of patients with specific characteristics and outcomes at each institution and report median values and the 10th‐ to 90th‐percentile range among the institutions. The correlation between institutional rate of admission and institutional rate of mortality was evaluated.

In standard multivariate analysis, we assessed physician specialty as a predictor of in‐hospital mortality of individual patients after adjustment for demographic characteristics, admission status (emergent, urgent, elective), comorbid illness severity score (range 04, from 0 = no substantial comorbid illness to 4 = catastrophic comorbid illness), and annual institutional treatment volume of ischemic stroke. UHC defined severity class to represent an individual's overall calculated risk of illness; its value was dependent on the refinement of the Health Care Facility Administration's diagnosis‐related groups (DRGs) and the Sach's Complication Profiler count of total comorbidities present.24, 25 Effects on LOS and total charges, as well as the ability of physician specialty to predict tPA use in individual patients, were similarly evaluated. Analysis of tPA use was restricted to patients admitted to universities that ever coded tPA use, which increased the sensitivity of the indicator to 57%.22 Residual misclassification error of tPA use would be expected to obscure a true underlying association between its use and physician specialty.

In multivariate GT calculations, we used the institutional proportion of cases admitted to a neurologist as a predictor of outcomes. GT analysis is based on the observation that if a treatment is effective, hospitals that use it more frequently should have better patient outcomes and that this association should persist regardless of whether individual‐level selection bias is present. The method assumes that hospital rates of admission to neurology are independent on the patient population's pretreatment prognosis. Because utilization differences between hospitals likely reflect practice variability rather than differences in patient prognosis,15, 16 the influence of unmeasured confounders at the hospital level is expected to be small. Measured variables that proved significant in univariate analyses or were thought to be responsible for an association between overall patient prognosis and modalities and frequencies of acute stroke treatments used, such as institutional treatment volume, were included in the multivariate GT model in order to isolate the effect of increasing rates of admission to neurologists.

We included both institutional and individual data to more accurately specify individual outcomes and covariates compared with an analysis that simply compared institutions' characteristics and their outcomes.26 Generalized estimating equations (GEE) were used in order to account for institutional clustering of predictor variables and outcomes. GEE is similar to logistic regression but produces broader confidence intervals (CIs) because logistic regression ignores the possibility that individuals at institutions are more similar to each other than would be expected by chance alone. We used a compound symmetry correlation structure, which initiates modeling by assuming a constant correlation between observations within each institution as well as between institutions, and used a logistic link function for binary outcomes in order to mimic logistic regression. The natural log transformations of LOS and hospital charges were modeled to reduce positive skew and approximate a normal distribution, and an identity link function was used in GEE to mimic linear regression for these analyses. To evaluate the impact of adjustment, both unadjusted and adjusted analyses were conducted. Methods to calculate power of GT analysis are not available. The Stata statistical package was used for all analyses (version 8.0; Stata Corporation, College Station, TX).

RESULTS

A total of 26,925 patients with ischemic strokes were admitted to neurologists or generalists through the emergency department at 113 institutions participating in the study. Patients admitted to neurologists rather than generalists (Table 1) were younger and more likely to be male, but less likely to have a serious comorbid illness. Institutions varied widely in the demographics of treated patients as well as in the markers of pretreatment prognosis. Institutional annual case volume of all ischemic strokes ranged from 1 to 741. Mortality rate, mean LOS, and mean hospital charges also varied broadly between institutions (Table 1). Patients treated at institutions whose rate of admissions to a neurologist's care was in the upper 50th percentile were younger and more often male, but did not differ in illness severity class (Table 2).

There were 1760 in‐hospital deaths (7.0%). In univariate analysis, older age (P < .001), white ethnicity (P < .001), emergent stroke (P < .001), and increased illness severity (P < .001) were associated with greater risk of death, whereas African‐American (P < .001) and Hispanic (P = .007) ethnicities were protective. No other patient characteristics were important, and institutional annual case volume showed no association with mortality risk.

Overall, 60% of patients with ischemic stroke were admitted to a neurologist's care. In univariate analysis (Table 3), a lower risk of in‐hospital mortality was observed in cases admitted to neurologists (4.6%) compared with those admitted to generalists (9.5%; P < .001). After adjustment in standard multivariable models, the association between neurologist admission and lower risk of death persisted (OR 0.60; 95% CI, 0.500.72; P < .001).

The institutional rate of admission of ischemic stroke patients to neurologists ranged from 0% to 90%, and higher rates were seen at hospitals with higher institutional case volumes (P < .001). There was no correlation between the institutional rate of admission to neurology and the institutional mortality rate (0.33; P = .73). At the individual‐level, greater rates of admission to neurologists had no significant impact on mortality (OR 1.05; 95% CI, 0.851.31; P = .64; Table 3) in unadjusted analysis. After adjustment for patient demographics, comorbid illness severity score, urgency status, and institutional case volume in GT analysis, there remained no association between death and proportion of ischemic stroke cases admitted to neurologists (OR 1.02; 95% CI, 0.791.30; P = .90), consistent with the absence of an association between neurologist care and in‐hospital mortality.

Patients treated by neurologists were likely to have shorter stays (P < .001) and lower charges (P = .01) in univariate analysis (Table 4). In traditional adjusted multivariable analysis, the same associations were seen for LOS (P < .001) and charges (P = .05). However, in adjusted GT analyses, increased institutional rate of admission to neurologists was not associated with briefer LOS (P = .36) and was associated with greater hospital charges (P = .044).

In 1999, 190 (2.2%) ischemic stroke patients received tPA at the 64 universities that had ever coded tPA use. In univariate analysis, patients admitted to a neurologist were more likely to have received tPA (P = .001; Table 3), and this association persisted after adjustment (P < .001). In adjusted GT analysis, institutions admitting a higher proportion of ischemic stroke patients to neurologists also treated patients with tPA more frequently (P = .033).

DISCUSSION

Several prior studies found that ischemic stroke outcomes were better when an attending neurologist was responsible for patient care.710 Traditional analyses of our data also indicate that care by a neurologist lowers inpatient mortality, LOS, and total charges. By contrast, a GT analysis that bypasses selection bias at the patient level suggests there is no independent benefit of neurologist care on mortality or LOS and actually shows higher associated charges.

The discrepancy between standard and GT analyses suggests that healthier patients may have been preferentially admitted to the care of neurologists. Measured pretreatment prognostic factors in our data present a mixed picture. Patients admitted to a neurologist's care were younger, more often male, more often emergently admitted, and less likely to have serious comorbid illnesses. These patient factors were controlled for in all adjusted analyses. Although traditional multivariate analysis attempts to adjust for variations between the 2 patient populations, it cannot adjust for inaccurately measured or unmeasured differences. Using the institutional proportion of admissions to neurologists as a predictor of patient outcomes, we were better able to control for the selection bias associated with differential distribution of patients to teams led by attending neurologists versus generalists.13, 14

Petty et al.7 studied 299 ischemic stroke patients and showed equivalent survival among stroke patients admitted to neurology inpatient teams versus generalist teams with neurologic consultation. However, patients cared for by generalist teams without neurologic consultation fared worse. Their subjects were treated at both academic and community hospitals. In our study, contributing hospitals were solely academic institutions. Because specialty cross talk may be more frequent at university‐based hospitals, academic‐based generalist physicians may be more familiar with recent stroke literature and guidelines than are their community‐based peers. Further, restricting analysis to academic centers in our study should have reduced the potential confounding influences of differences between other aspects of institutional care. Although no information was available on neurologist consultation in our database, informal consultation is believed to play a large but hidden role at academic medical centers. Thus, the inclusion of a formal consultation variable may be misleading at academic medical centers.

Analyzing claims data on 44,099 Medicare beneficiaries with acute ischemic strokes cared for at both academic and community hospitals, Smith et al.10 also recently reported a 10% lower risk of 30‐day mortality and 12% lower risk of rehospitalization for infections and aspiration pneumonitis among patients admitted to the care of neurologists compared with those admitted to the care of generalists. However, the upper 95% confidence interval limits for these 2 findings nearly crossed 1 (ranging from 0.9980.999). The study also concluded that patients cared for collaboratively by generalists and neurologists had a 16% lower 30‐day mortality risk (hazard ratio 0.84; 95% CI, 0.790.90) than those cared for by generalists alone but simultaneously noted that patients admitted to generalists only had more comorbidities than either the collaborative care or neurologist‐only patient groups. If sicker patients were triaged to generalist admission, as occurs in confounding by indication (also known as channeling bias), then incomplete adjustment for comorbid disease may bias outcomes in favor of neurologist involvement. The GT analysis we employed is specifically designed to overcome this exact type of selection bias.

In our study, patients admitted to neurologists received tPA significantly more often than those admitted to generalists. GT analysis also found that hospitals admitting a higher proportion of strokes to neurologists treated more patients with tPA. This result is consistent with a prior study demonstrating that academic institutions employing a vascular neurologist had significantly higher odds of administering tPA.21 Since tPA must be administered within 3 hours of symptom onset,27 it is commonly delivered in the emergency department prior to admission. Thus, patients may be preferentially selected for admission to neurologic services because of their receipt of tPA, rather than that this association reflects an actual increased use of tPA by neurologists over generalists. Alternatively, institutions with a higher rate of stroke admissions to neurology may simply be more familiar with tPA protocols. Importantly, the poor sensitivity of our data for actual tPA administration may affect the analysis of its use by physician specialty; however, the failure to administratively code tPA use is unlikely to be differentially biased based on physician specialty. Thus, undercoding of tPA use would be expected to bias these analyses toward the null.

The potential advantage and efficacy of stroke centers, stroke units, stroke services, and other institutional processes of care are not addressed by our data. Previously, among academic hospitals, we found that acute ischemic stroke mortality was lower at hospitals employing a vascular neurologist and at those whose guidelines allowed only neurologists to administer tPA.21 A later analysis evaluated the impact of all elements of stroke center care supported by the original Brain Attack Coalition consensus28 and found that no single element improved mortality.29 However, recent studies have found significant mortality benefit associated with stroke units30, 31 and stroke services.32 Clearly, the debate continues over these important questions.

Our study had several limitations. First, generalizability may be lessened because only academic medical centers contributed data and only admissions through the ED were included. However, limiting the study population to academic centers provided a homogenous study population and greatly reduced the potential for confounding at the institutional level. Although the selection of ED cases mitigated the effects of referral bias and the use of only academic hospitals minimized interinstitutional differences, institutions whose rate of admissions to neurology was above the 50th percentile differed from those whose rate admissions to neurology was below the 50th percentile. However, this difference did not consistently result in patients with worse pretreatment prognostic factors being cared for at hospitals with higher rates of admission to neurology. Second, there are important limitations to using administrative data. In our study, patients were selected based on diagnostic coding of records analysts at discharge, and the diagnostic accuracy of such coding for stroke is imperfect.33 Furthermore, missing or incomplete information could have impaired adjustments for patient differences. Third, details of patient treatment were limited. The lack of information about formal and informal consultations may have obscured a true difference in outcomes among specialties.7 Additionally, academic institutions may use systematized care plans more often than do community hospitals, potentially minimizing differences between specialties. Fourth, at the time of our study, tPA had been recently introduced into stroke care. Current rates of tPA use among neurologists and generalists may be more similar. Fifth, the ability of in‐hospital mortality to adequately assess quality of care is limited, and longer‐term and functional outcomes would be better measures and more clinically relevant.

After controlling for selection bias using GT analysis, we found stroke outcomes to be similar regardless of whether a neurologist or a generalist was the admitting physician. This result contrasts with the findings of several previous studies that suggested admitting stroke patients to a neurologist resulted in better clinical outcomes.710 Because only 1 neurologist is employed for approximately every 19.8 generalists in the United States34 and 40% of acute strokes were cared for by generalists, even in this sample entirely restricted to university hospitals, such findings would suggest that many U.S. stroke patients receive inferior care. Because the role of the neurologist as consultant rather than as attending physician is significantly more feasible in most practice settings, the demonstration of equivalent outcomes by both types of physicians is reassuring and certainly reinforces the important role that unmeasured confounders may play in observational studies.

However, these results do imply that it is vital that generalists remain fully trained in the current best practices of acute stroke management in order to maintain the equivalence of care suggested here. Given how common acute stroke is, any proposed future hospitalist training, certification, and recertification programs should include a focus on acute stroke management.

Aortocervicocephalic atherosclerotic disease and coronary artery disease share common risk factors, and patients with one condition are at high risk of harboring or developing the other.1, 2 Over the past decade, several randomized clinical trials of lipid‐lowering medications designed to reduce low‐density lipoprotein cholesterol (LDL‐C) have shown a significant decrease in the risk of coronary events and ischemic stroke among patients who have a history of or are at risk for coronary artery disease, regardless of whether serum cholesterol is elevated.3, 4 Results from more than 3000 stroke patients enrolled in the Heart Protection Study also provide evidence that aggressive lipid‐lowering therapy may prevent recurrent vascular events in individuals who have a total cholesterol level as low as 135 mg/dL and cerebrovascular disease, with or without known coronary artery disease.5

Guidelines from the National Cholesterol Evaluation Program Adult Treatment Panel (ATP) provide target LDL‐C levels for persons with atherosclerotic disease depending on the extent of their vascular risk.6 However, despite the broad dissemination of these guidelines, several published studies of patients with coronary artery disease or dyslipidemia have shown that a large proportion of patients with high vascular risk continue to be underscreened, underdiagnosed, and undertreated for dyslipidemia.79

Few studies have evaluated the quality of cholesterol management among hospitalized patients who have experienced an acute ischemic cerebrovascular event10, 11 So the data are scarce on the management of patients hospitalized for ischemic stroke or transient ischemic attack (TIA) who are, according to ATP criteria, at high risk for future coronary events and on the factors that may govern that management. Systematic reviews have suggested that incorporating a lipid profile during acute stroke presentation could assure baseline assessment and serve as a potential cue for physicians to change their behavior,12 and an American Stroke Association advisory recommends lipid treatment during hospitalization for most patients with ischemic stroke or TIA as it may increase the rate of long‐term use.13

The objectives of this study were to determine the rates of testing for and treatment of dyslipidemia according to national cholesterol guidelines among individuals hospitalized with acute ischemic stroke or TIA and to identify predictors of performance.

METHODS

The California Acute Stroke Prototype Registry (CASPR) is a Centers for Disease Controlsponsored cohort that captured detailed data on patients admitted to 11 hospitals over a 2‐year period. The methods of study have been described elsewhere.14 In brief, CASPR prospectively collected information on acute stroke care at 11 representative hospitals in 5 major population regions of California. Data were collected on diagnostic evaluation, appropriate use of treatment strategies, and disposition on discharge from the hospital. The main goal of CASPR was to pilot‐test a prototype prospective registry of acute stroke and transient ischemic attack to be used as a quality improvement tool. The study population was patients with an admitting or discharge diagnosis of suspected stroke or TIA from November 1, 2002, through January 31, 2003, and from November 1, 2003, through January 31, 2004. The human subjects review board at each participating center approved the study.

For the present analysis, data on all patients with a discharge diagnosis of ischemic stroke or TIA who were admitted during either period were included. We examined the possible association of several variables with 2 primary outcomes: (1) testing lipid profile during hospitalization (as indicated by a documented LDL‐C level) and (2) prescribing lipid‐lowering medication at discharge. In those analyses in which lipid profile testing was the outcome, no variables were considered acceptable reasons for not performing an LDL‐C assessment.

The distribution of LDL‐C levels in this portion of the cohort was determined. Patients were then categorized according to their risk for future coronary events. Patients were classified as at risk for coronary events (ACE) if they either had a documented history of myocardial infarction, coronary artery disease, or diabetes or had undergone carotid endarterectomy or carotid angioplasty/stenting during hospitalization. Criteria for initiating lipid‐lowering therapy were defined according to the ATP III guidelines,6 which were in effect during both CASPR study periods. Continuing the recommendation in ATP II, the ATP III recommendations emphasized that persons with documented coronary artery disease (CAD) receive the most aggressive lipid‐lowering treatment. But this recommendation was expanded to include patients without established CAD, whose coronary risk is equivalent to that of patients with diagnosed CAD.6

As per the ATP III guidelines, CASPR‐ACE patients were considered optimally treated if they were prescribed a lipid‐lowering agent at discharge or if their documented LDL‐C was less than 130 mg/dL. A concurrent history of liver disease, abnormal prothrombin time, life expectancy of less than 1 year, and terminal illness were each considered a valid contraindication to treatment with lipid‐lowering medication. Optimal treatment for non‐ACE patients was defined as receipt of lipid‐lowering medication at discharge or a documented LDL‐C of 160 mg/dL. The rate of optimal treatment of ACE patients was compared to that of non‐ACE patients. The ACE and non‐ACE patients were then further categorized into 1 of 4 groups according to LDL‐C level<100, 100130, 130160, and >160 mg/dLand an assessment for trend of the rate of treatment in each of the 4 categories in the ACE and non‐ACE groups was performed.

RESULTS

Data were available from the 11 CASPR hospitals for 764 patients diagnosed with either ischemic stroke or TIA. Overall, 53.4% of subjects were women, and the average age at hospitalization of 70.4 ( 15.4) years. In the cohort, 55.3% of the patients were non‐Hispanic white, 9.7% were African American, 13.4% were Hispanic, 13% were Asian, and 8.6% were classified as other. Three hundred and nine individuals (40.5% of the cohort) were classified as at risk for coronary events. Of these, 148 (47.8%) had diabetes only, and 160 (51.8%) had a history of MI, CAD, or both. One patient (0.4%) had undergone angioplasty/stenting during hospitalization but had no history of MI, CAD, or diabetes. Only 4 patients (0.52% of the entire cohort) had undergone a carotid endarterectomy or angioplasty/stenting during hospitalization. Rates of lipid assessment and optimal treatment varied widely between hospitals, but testing and treatment were correlated for each hospital. Overall, however, testing and treatment were correlated (Pearson correlation coefficient = 0.35, P < .0001). On an individual hospital level, the correlation was positive and significant for 6 hospitals, positive but not significant for 2 hospitals, and negative but not significant for 3 hospitals.

Overall, LDL‐C levels were determined in 383 patients (50.1%). The likelihood that a patient would have an LDL‐C test performed during hospitalization varied widely by hospital, ranging from 12% to 88% (P < .0001). Univariate variables significantly associated with documented LDL‐C measurement in the overall cohort at the = .10 level were diagnosis of ischemic stroke (as compared to TIA) and history of dyslipidemia (Table 1). In the CASPR cohort, 53% of the ACE subjects received a lipid profile assessment compared to 48% in the rest of the cohort (P = .14). In multivariate analysis, diagnosis of ischemic stroke and history of dyslipidemia remained significantly associated with documented LDL‐C measurement (Table 1).

Lipid‐lowering drugs were prescribed at discharge to 370 patients (48.4%); however, treatment rate varied among hospitals, from a low of 13% of patients to a high of 84% of patients (P < .0001). Univariate factors associated with a higher treatment rate at the = .10 level were diagnosis of ischemic stroke, history of stroke/TIA, history of diabetes, hypertension, history of dyslipidemia, independent ambulation at discharge, and ACE status (Table 2). Patients were less likely to receive lipid‐lowering medication if they had a history of heart failure. Fifty‐nine percent of the CASPR ACE subjects were discharged on lipid‐modifying agents compared to 42% in the rest of the cohort (P = .0006). Multivariate analyses revealed several independent predictors of treatment with lipid‐lowering medication. Diagnosis of ischemic stroke, ACE status, and history of heart failure were negative predictors (less likely to be treated), and history of dyslipidemia was a positive predictor (Table 2). Status as an academic hospital was a hospital characteristic for which a significant association was found. Academic hospitals were significantly more likely to both perform LDL profiles and administer lipid‐lowering medications at discharge than were nonacademic hospitals. This association was found in a logistic regression analysis that did not account for between‐hospital variance. However, when we used GEE analysis, which adjusted for the variance, the difference between academic and nonacademic hospitals was no longer significant.

Three of the patients with documented LDL‐C levels (0.8%) had documented contraindications to therapy. Among all those who had documented LDL‐C levels, the rate of appropriate treatment with lipid‐lowering medications was high in both the ACE and non‐ACE groups (94.6% and 98.6%, respectively; P = .02). However, because only a small number of patients did not receive optimal treatment, the odds ratio of 0.24 had a fairly wide confidence interval (95% CI = 0.06, 0.91). Although a trend toward a higher rate of treatment with increasing LDL‐C level was seen in both the ACE and non‐ACE groups, this trend was only significant for the group with non‐ACE patients (Figure 1).

Figure 1

DISCUSSION

We found that only half the patients hospitalized for ischemic stroke or TIA had LDL‐C levels tested while in the hospital, even among those identified by the ATP guidelines as at high risk for future coronary events. Our findings are in accord with those of the Coverdell Project, which evaluated key features of acute stroke care from 4 prototype registries, those in Georgia, Massachusetts, Michigan, and Ohio, finding that fewer than 40% of acute stroke patients had had lipid profiles checked during hospitalization.11 Our study also evaluated predictors for in‐hospital lipid testing and lipid‐lowering treatment during hospitalization for an acute ischemic cerebrovascular event. We found that lipid testing was correlated with treatment during stroke or TIA hospitalization, suggesting that in‐hospital lipid management is related to an overall appreciation of the importance of lipids.

Understanding the factors resulting in such underperformance is critical for improving patient care and outcomes. Lipid assessment and treatment rates varied widely between CASPR hospitals, reflecting dramatic differences in hospital practice. This finding is similar to that noted in a recent study performed in Europe10 and underscores the need to promote a more uniform approach to in‐hospital care of patients with ischemic stroke or TIA. Our study also found that ischemic stroke patients were much more likely to have their lipid level measured and to be discharged on a lipid‐lowering agent than were TIA patients. This may be so because many treating health care professionals perceive TIAs as benign events that carry a more favorable prognosis than do strokes, or it could be that the length of stay for a TIA, often shorter than that for a stroke, limited in‐hospital testing or planning for patient follow‐up.

A high proportion of non‐ACE, lipid‐tested stroke/TIA patients received lipid‐lowering drug treatment, even when their lipid levels were within the treatment range categorized as nonpharmacologic by the national guidelines. This finding could be a result of one of the goals of the primary study.15 In the primary study, the effect of standardized orders implemented during the second observational period were analyzed by comparing them to those in place during the first observational period to see if they had improved the in‐hospital stroke care process. One of the study goals was optimal discharge utilization of a lipid‐lowering agent, defined as prescription of a lipid modifier or an LDL < 100 mg/dL. There was a significant increase in the number of prescriptions for lipid modifiers at discharge after implementing the standardized orders.15 However, as this study has shown, when existing national cholesterol guidelines were strictly applied to all the patients,6 overall there was a suboptimal rate of utilization of lipid modifiers at discharge.

Lipid profile assessment during stroke admission is one of the 10 performance measures in the performance measure set of the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) Stroke Disease‐Specific Care.16 Initiating therapy with lipid‐lowering agents before discharge may help to maintain continuity of care and clarify therapeutic intent, especially when a different physician is responsible for care after discharge from the hospital. Recent studies indicated that in‐hospital initiation of medication following admission for a vascular event tends to improve longer‐term patient adherence to treatment,17, 18 as well as vascular outcomes,19, 20 and is a strategy favored by the American Stroke Association.13, 21

This study had several limitations. Our definitions of dyslipidemia and of adherence to ATP III goals were based on single measurements of LDL‐C, rather than multiple determinations of lipoprotein subfractions. However, we believe that this approach parallels actual clinical practice more closely. Although LDL‐C is the most important of all the components of the lipid profile,6 because lipid subfractions other than LDL‐C were not collected in the CASPR registry, we may have misclassified a few patients. For instance, extremely high trigylceride levels can render LDL‐C levels inaccurate, and as such, not having a documented LDL‐C may not have always indicated that a lipid panel was not performed. It is also conceivable that physicians might actually have been more thorough in measuring LDL‐C, identifying contraindications to lipid‐lowering therapy, or instituting lipid‐lowering therapy than were noted in the hospital charts. However, for quality assurance purposes, what is documented is the only traceable record of what was actually asked for or done. As such, health care professionals are frequently encouraged to keep updated chart notes. This study was an assessment of in‐hospital behavior; the low utilization of lipid‐lowering agents observed may underestimate the final treatment rate, as we did not evaluate the postdischarge rate of therapy. However, recent data suggest in‐hospital prescription patterns are a major predictor of longer‐term care in the community.17, 22 Last, the CASPR investigators did not collect data on the rate of utilization of lipid agents prior to hospitalization or on the mechanisms by which the strokes and TIAs had occurred. Prehospital utilization of lipid agents has previously been revealed to influence the prescribing of lipid‐lowering agents at discharge.10 Knowledge of the mechanisms of the stroke and TIA events would have increased the number of those eligible for lipid treatment, particularly those whose events were to the result of an atherosclerotic mechanism per ATP III's more expansive definition of CHD risk equivalents, which includes carotid and other forms of clinical atherosclerotic disease.6 However, because the results of other studies that evaluated lipid management in all hospitalized stroke patients (regardless of mechanism)11, 23 or in all patients with any form of clinical atherosclerotic disease24 were in accord with those of our study, it would appear unlikely that such information would have made an overwhelming difference to our results.

In conclusion, the results of the present study suggest that considerable improvement is needed in identifying appropriate candidates among those who have had stroke or TIA and treating them with lipid‐lowering agents. Performing lipid testing in individuals hospitalized with ischemic stroke or TIA is important because it may inform the identification of persons for whom treatment should be initiated or modified. Lipid assessment during hospitalization for stroke/TIA and initiation of lipid‐lowering therapy when indicated are major management steps that all patients with ischemic cerebrovascular events should receive.