Medications can be considered inappropriate when their risk outweighs their benefit. The Beers list1 identifies medications that should be avoided in persons 65 years or older because they are ineffective or pose an unnecessarily high risk or because a safer alternative is available. Initially developed in 1991, the list has gained wide acceptance and has been updated twice.2, 3 In July 1999 it was adopted by the Centers for Medicare & Medicaid Services (CMS) for nursing home regulation, and in 2006 the National Committee on Quality Assurance adopted a modified version as a Health Plan Employer Data and Information Set (HEDIS) measure of quality of care for older Americans.4

A number of studies have demonstrated that inappropriate prescribing is common in the ambulatory setting,57 in nursing homes,8, 9 and in emergency departments10, 11 and that exposure to inappropriate medications is associated with increased risk of adverse drug reactions12 and hospitalization.13, 14 Initial studies of hospitalized patients1517 suggest that potentially inappropriate prescribing is also common among elderly inpatients and that reducing the misuse of psychotropic medications can prevent falls.18 We report on the incidence of and risk factors associated with potentially inappropriate prescribing in a large sample of hospitalized elders.

METHODS

Potentially Inappropriate Prescribing

Using the 2002 updated Beers criteria3 for potentially inappropriate medication (PIM) use in older adults, we identified the total number of PIMs administered to each patient during his or her hospital stay. We classified each PIM as either high or low severity based on the expert consensus expressed in the 1997 update of the Beers criteria.2 The list contains 48 PIMs and an additional 20 that should be avoided in patients with certain conditions. We did not include the second category of PIMs because we did not necessarily have sufficient patient information to make this determination. In addition, some of the standard PIMs, such as laxatives, although inappropriate for chronic outpatient use, could be appropriate in the hospital setting and were excluded from this analysis. Finally, several medications were considered inappropriate only above a given threshold (eg, lorazepam >3.0 mg/day) or for patients without a specific diagnosis (eg, digoxin >0.125 mg/day for patients without atrial fibrillation). We grouped PIMs that had similar side effects into 4 categories: sedatives, anticholinergics, causing orthostasis, or causing bleeding (Fig. 1).

Figure 1

RESULTS

We identified 519,853 patients at least 65 years old during the study period; 564 were excluded because of missing data for key variables or unclear principal diagnosis. An additional 25,318 were excluded because they were cared for by an attending with a surgical specialty. A total of 493,971 patients were included in the study (Table 1). Mean age was 78 years, and 24% of patients were 85 years or older. Forty‐three percent were male, 71% were white, and 39% were currently married. The most common principal diagnoses were community‐acquired pneumonia, congestive heart failure, and acute myocardial infarction. The most common comorbidities were hypertension, diabetes, and chronic pulmonary disease. Medicare was the primary payer for 91% of subjects, and 13% were in managed care plans. Most patients were cared for by internists (49%), family practitioners (18%), or cardiologists (17%). Only 1% of patients had a geriatrician as attending.

Just under half of all patients (49%) received at least 1 PIM, and 6% received 3 or more (Table 2). Thirty‐eight percent received at least 1 drug with a high severity rating (HS‐PIM). The most common PIMs were promethazine, diphenhydramine, propoxyphene, clonidine, amiodarone, and lorazepam (>3 mg/day).

Patient, Physician, and Hospital Factors Associated with PIMs

Patient, physician, and hospital characteristics were all associated with use of PIMs (Table 3). In univariate analyses, older patients were less likely to receive any class of PIM, and this difference was accentuated for HS‐PIMs. Women, American Indians, married people, and those not in managed care plans were slightly more likely to receive PIMs, whereas patients admitted with acute myocardial infarction or congestive heart failure were even more likely to receive PIMs (P < .0001 for all comparisons).

Table 3.Potentially Inappropriate Medication Use by Patient Characteristic

Patient characteristic	Any PIM n (row %)	Any high‐severity PIM n (row %)	Sedatives n (row %)	Anticholinergic effects n (row %)	Causing orthostasis n (row %)	Causing bleeding n (row %)
* An additional 24,602 patients (5%) seen by 42 additional physician specialties were excluded from this analysis. Chi‐square tests indicated all differences by patient characteristics were significant at P < .0001 except there was no significant difference by managed care status for use rates of drugs potentially causing orthostasis or drugs potentially causing bleeding.
Overall	239,771 (49%)	189,448 (38%)	156,384 (32%)	109,293 (22%)	43,805 (9%)	14,744 (3%)
Age group
6574 years	89,168 (53%)	72,573 (43%)	61,399 (36%)	44,792 (27%)	15,799 (9%)	6655 (4%)
7584 years	100,787 (49%)	79,595 (39%)	65,034 (32%)	45,121 (22%)	18,519 (9%)	5727 (3%)
85+ years	49,816 (42%)	37,280 (31%)	29,951 (25%)	19,380 (16%)	9487 (8%)	2362 (2%)
Sex
Male	100,824 (47%)	79,535 (37%)	63,591 (30%)	42,754 (20%)	17,885 (8%)	5771 (3%)
Female	138,947 (49%)	109,913 (39%)	92,793 (33%)	66,539 (24%)	25,920 (9%)	8973 (3%)
Race
White	173,481 (49%)	139,941 (40%)	112,556 (32%)	81,097 (23%)	27,555 (8%)	10,590 (3%)
Black	26,793 (51%)	18,655 (36%)	18,720 (36%)	11,263 (21%)	8925 (17%)	1536 (3%)
Hispanic	8509 (47%)	6370 (35%)	5549 (31%)	3505 (19%)	2047 (11%)	648 (4%)
American Indian	1091 (58%)	849 (45%)	818 (44%)	563 (30%)	190 (10%)	76 (4%)
Asian/Pacific Islander	2386 (40%)	1896 (32%)	1420 (24%)	1023 (17%)	519 (9%)	127 (2%)
Other	27,511 (43%)	21,737 (34%)	17,321 (27%)	11,842 (18%)	4569 (7%)	1767 (3%)
Marital status
Married/partner	96,874 (50%)	77,803 (40%)	63,303 (33%)	45,042 (23%)	16,765 (9%)	5969 (3%)
Widowed	74,622 (48%)	58,012 (37%)	48,367 (31%)	33,516 (22%)	13,865 (9%)	4354 (3%)
Single/separated/divorced	36,583 (48%)	28,799 (38%)	24,251 (32%)	16,115 (21%)	7229 (10%)	2399 (3%)
Other	31,692 (46%)	24,834 (36%)	20,463 (30%)	14,620 (21%)	5946 (9%)	2022 (3%)
Primary diagnosis
Pneumonia	56,845 (46%)	46,271 (38%)	35,353 (29%)	25,484 (21%)	9184 (7%)	4155 (3%)
Heart failure	56,460 (52%)	42,231 (39%)	34,340 (31%)	22,093 (20%)	10,117 (9%)	1945 (2%)
Acute MI	43,046 (61%)	37,849 (54%)	32,560 (46%)	25,568 (36%)	4738 (7%)	2549 (4%)
Ischemic stroke	25,763 (45%)	17,613 (31%)	18,500 (32%)	8742 (15%)	9644 (17%)	1384 (2%)
Chest pain	20,655 (41%)	16,363 (32%)	13,536 (27%)	10,520 (21%)	3474 (7%)	2027 (4%)
COPD	18,876 (42%)	14,626 (33%)	12,087 (27%)	8096 (18%)	3209 (7%)	1139 (3%)
Urinary tract infection	18,126 (46%)	14,495 (37%)	10,008 (25%)	8790 (22%)	3439 (9%)	1545 (4%)
Payer type
Nonmanaged care	212,150 (49%)	168,013 (39%)	138,679 (32%)	97,776 (23%)	38,341 (9%)	12,868 (3%)
Managed care	27,621 (44%)	21,435 (34%)	17,705 (28%)	11,517 (18%)	5464 (9%)	1876 (3%)
Attending physician specialty*
Internal medicine (internist%)	112,664 (47%)	86,907 (36%)	71,382 (30%)	48,746 (20%)	23,221 (10%)	7086 (3%)
Family/general medicine	41,303 (45%)	32,338 (36%)	25,653 (28%)	18,274 (20%)	7660 (8%)	2852 (3%)
Cardiology	48,485 (58%)	40,752 (49%)	34,859 (42%)	25,792 (31%)	5455 (7%)	2542 (3%)
Pulmonology	10,231 (48%)	8105 (38%)	6746 (32%)	4064 (19%)	1739 (8%)	574 (3%)
Hospitalist	7003 (47%)	5443 (36%)	4447 (30%)	3179 (21%)	1471 (10%)	463 (3%)
Nephrology	4508 (55%)	3388 (41%)	3132 (38%)	2054 (25%)	1326 (16%)	198 (2%)
Neurology	2420 (42%)	1789 (31%)	1625 (28%)	851 (15%)	699 (12%)	174 (3%)
Geriatrics	1020 (33%)	785 (25%)	596 (19%)	404 (13%)	196 (6%)	41 (1%)

The HS‐PIM prescribing varied substantially by attending specialty (Fig. 2). Internists, family practitioners, and hospitalists all had similar median rates (33%), cardiologists had a higher median rate (48%), and geriatricians had a lower rate (24%). The most common PIM also differed by specialty: whereas promethazine was the most commonly prescribed drug across most specialties, nephrologists and neurologists used clonidine, pulmonologists used lorazepam, and cardiologists used diphenhydramine most often. Among the 8% of physicians who saw at least 50 patients, there was also great variation in each specialty (Fig. 2). Among internists and cardiologists who saw at least 50 patients, the high‐severity PIM usage rate ranged from 0% to more than 90%.

Figure 2

There was substantial variation in PIM usage among hospitals, most notably by region. The mean proportion of patients receiving PIMs ranged from 34% at hospitals in the Northeast to 55% at hospitals in the South (Table 4). Smaller hospitals and those in urban settings had slightly lower rates, as did those that had geriatricians on staff. The teaching status of the hospital had little effect. Variation at the individual hospital level was extreme (Fig. 3). Although half of all hospitals had rates between 43% and 58%, in 7 hospitals with more than 300 encounters each, PIMs were never prescribed for geriatric patients.

Figure 3

Table 4.Percentage of Patients Prescribed Potentially Inappropriate Medication (PIM) by Hospital Characteristic

	Hospitals Total = 384 n (%)	Patients N = 49,3971 n (%)	Any PIM Mean = 48.2 Mean (SD)	Any high‐severity PIM Mean = 38.7 Mean (SD)	Sedatives Mean = 30.2 Mean (SD)	Anticholinergic effects Mean = 21.5 Mean (SD)	Causing orthostasis Mean = 8.5 Mean (SD)	Causing bleeding Mean = 3.1 Mean (SD)
Note: P values from analysis of variance of hospital use rates for each hospital characteristic. * P < .05, ** P < .001, *** P < .0001.
Hospital region			***	***	***	***	***	**
Midwest	76 (20%)	95,791 (19%)	38.8 (19.7)	30.0 (16.4)	24.3 (13.8)	15.1 (9.9)	6.9 (6.3)	3.1 (2.3)
Northeast	47 (12%)	79,138 (16%)	34.1 (12.6)	26.2 (11.2)	19.0 (9.2)	13.5 (8.1)	4.9 (2.3)	2.1 (1.6)
South	199 (52%)	260,870 (53%)	54.5 (10.1)	42.7 (9.6)	36.0 (10.8)	26.4 (8.6)	10.4 (4.6)	3.6 (2.5)
West	62 (16%)	58,172 (12%)	45.8 (8.1)	37.4 (7.1)	27.3 (7.7)	19.5 (5.7)	7.4 (4.8)	2.7 (1.3)
Teaching status
Nonteaching	297 (77%)	324,948 (66%)	47.3 (14.6)	36.9 (12.3)	29.8 (12.0)	21.3 (9.9)	8.7 (5.4)	3.3 (2.4)
Teaching	87 (23%)	169,023 (34%)	48.2 (16.0)	38.8 (14.2)	31.6 (14.5)	22.1 (10.2)	7.8 (4.4)	2.7 (1.5)
Staffed beds				*	**
22200	143 (37%)	80,741 (16%)	45.5 (16.9)	35.2 (14.6)	27.5 (14.0)	20.1 (10.3)	8.0 (6.2)	3.5 (3.1)
200400	137 (36%)	177,286 (36%)	47.7 (14.2)	37.8 (12.0)	30.5 (11.6)	22.0 (10.0)	8.4 (4.7)	3.0 (1.6)
400+	104 (27%)	235944 (48%)	50.1 (12.4)	39.6 (10.6)	33.5 (10.9)	22.7 (9.3)	9.3 (4.2)	2.9 (1.4)
Population serviced						*	*	**
Rural	119 (31%)	102,799 (21%)	48.4 (13.0)	38.3 (10.6)	29.2 (11.0)	23.2 (9.3)	7.5 (4.0)	3.7 (3.0)
Urban	265 (69%)	391,172 (79%)	47.1 (15.7)	36.9 (13.7)	30.6 (13.2)	20.7 (10.2)	9.0 (5.6)	2.9 (1.8)
Geriatrician presence
No	340 (89%)	409,281 (83%)	47.7 (15.3)	37.6 (13.0)	30.3 (12.8)	21.7 (10.0)	8.4 (5.3)	3.2 (2.3)
Yes	44 (11%)	84,690 (17%)	45.8 (11.4)	35.5 (10.6)	29.4 (10.8)	19.6 (9.4)	9.3 (4.3)	2.9 (1.6)

DISCUSSION

Although Americans age 65 years and older make up less than 15% of the U.S. population, they consume about one third of all prescription drugs20 and account for one third of all hospital admissions.21 Using the Beers list, numerous studies have documented high rates of potentially inappropriate prescribing for community‐dwelling elderly and nursing home patients and, in some studies, an attendant risk of falling,2224 hip fracture,25, 26 hospitalization,13 or death.14 Applying these same criteria to a large sample of medical inpatients, we found that almost half received a potentially inappropriate drug, most of high severity. Moreover, the PIM prescribing rate varied substantially by region, hospital, and attending physician specialty. Although the use of PIMs was associated with patient age, comorbidities, and primary diagnosis, these patient factors explained only a small portion of the variation in prescribing practices across groups of physicians and hospitals.

Using consensus criteria, Beers originally found that 40% of the residents in 12 nursing homes received at least 1 PIM,8 and studies of community‐dwelling elderly demonstrated rates of 21% to 37%, with little change over time.6, 27, 28 Several small studies have examined inpatient prescribing.16, 17, 29, 30 The largest17 found that only 15% of elderly Italian inpatients received a PIM. Our finding, that 49% of inpatients had received at least 1 PIM, may partially reflect the high prevalence of use among elderly US patients in nursing homes and the community.

Regional variation has been demonstrated for ambulatory patients in the US6 and Europe.31 Zhan et al. found slightly higher rates of PIM use in the Midwest and the South (23%) than in the Northeast and the West (19%). Variation in Europe was greater, with 41% of patients in the Czech Republic versus 5.8% of patients in Denmark receiving at least 1 PIM. We found that region was the strongest predictor of in‐hospital HS‐PIM use, with patients in the South most likely and patients in the Northeast least likely to receive HS‐PIMs. This variation persisted even after adjusting for differences in other patient and hospital factors, suggesting that local custom played a large role in the decision to prescribe HS‐PIMs. Moreover, because outpatient rates are more uniform, these large differences seem limited to inpatient practice.

Patient factors have also been examined. Advanced age was associated with decreased PIM use in some studies17, 28, 31 but not in others.6, 27 We found increasing age to be strongly associated with decreased PIM use, suggesting that in the hospital, at least, doctors take care to avoid prescribing certain drugs to the frail elderly. Women appear to be consistently at higher risk than men,6, 27, 28, 31 and white patients are more at risk than those of other races.6 Our finding that certain diagnoses were associated with higher or lower rates has not been reported previously. The lower rates associated with stroke and COPD suggest that prescribers were aware that these patients were at increased risk of delirium and respiratory depression. The higher rates associated with myocardial infarction may have to do with the use of standardized order sets (eg, cath lab orders) that do not consider the age of the patient going for the procedure.

Admission to a geriatric service32 and intervention by a clinical pharmacist33 have been shown to decrease PIM prescribing at discharge. We noted that patients cared for by a geriatrician had the lowest rates of PIM prescribing during hospitalization as well and that hospitals with geriatricians had lower rates overall, possibly demonstrating that geriatricians had a ripple effect on their colleagues. Hospitalists also had lower rates than internists, supporting the notion that hospitalists provide higher‐quality inpatient care.

Our study had some important limitations. First, we only had access to inpatient administrative records. Thus, we could not identify which medications were continued from home and which were begun in the hospital, nor could we know the indications for which specific drugs were prescribed or who prescribed them. Based on published outpatient rates, however, we could assume that many of the drugs were started in the hospital and that others could have been discontinued but were not. Second, the Beers list was developed by the modified Delphi method; there was little empirical evidence of the danger of specific drugs, although some classes, such as benzodiazepines, opiates and digoxin, have been associated with inpatient falls.18, 3436 Furthermore, our administrative database did not allow us to balance the risks and benefits for particular patients; hence, the medications were only potentially inappropriate, and our study did not address the consequences of such prescribing. Although some of these drugs may be appropriate under certain circumstances, it is unlikely that these circumstances would vary by 60% across geographic regions or that internists would encounter these circumstances more often than do hospitalists. Thus, although we could not identify specific patients who received inappropriate medications, we did identify certain hospitals and even whole regions of the country in which the rate of inappropriate prescribing was high. Third, the Beers list, which was developed for outpatient use, may be less relevant in the inpatient setting. However, given that inpatients have more organ dysfunction and are at higher risk of delirium and falls, it may actually be more applicable to hospitalized patients. We similarly did not distinguish between single and multiple doses because the Beers list does not make such a distinction, and there is no empirical evidence that a single dose is safe. Indeed, patients are often at highest risk of falls immediately after initiation of therapy.3739 We did, however, exclude drugs such as laxatives, which may be appropriate for brief inpatient use but not for chronic use.

Our study also had a number of strengths. The large sample size, representing approximately 5% of annual inpatient admissions in the US over 2 years, offered an instructive look at the recent prescribing patterns of thousands of US physicians. We were able to identify many patient, physician, and hospital factors associated with PIM prescribing that have not previously been reported. Some of these factors, such as advanced age and comorbid diagnoses, suggest that physicians do tailor their treatment to individual patients. Nevertheless, patient factors accounted for only a small portion of the variation in prescribing. The largest variation, associated with regional, hospital, and physician factors, highlights the opportunity for improvement.

At the same time, our findings are encouraging for 2 reasons. First, most inappropriate prescribing involved only a handful of medications, so small changes in prescribing patterns could have a tremendous impact. Second, observing the practice of individual physicians and hospitals reveals what is possible. We found that in most specialties there were physicians who rarely or never used PIMs. We also found 7 hospitals, each with at least 300 cases, where no PIMs were ever prescribed.

Where should hospitals focus their efforts to prevent inappropriate prescribing? Our data highlight the complexity of the problem, which seems daunting. PIM prescribing is spread across all specialties, including geriatrics, and although cardiologists had the highest rate of prescribing, internists, who were more numerous, accounted for a much higher overall number of potentially inappropriate prescriptions. It would be instructive to study the 7 hospitals where PIMs were never prescribed or to interview those physicians who never prescribed PIMs, but the anonymous nature of our data would not allow for this. However, our data do suggest some directions. First, hospitals should become aware of their own rates of PIM use because measurement is the first step in quality improvement. Next, hospitals should focus efforts on reducing the use of the most common drugs. Eliminating just 3 drugs promethazine, diphenhydramine, and propoxyphenewould reduce the use of PIMs in 24% of elderly patients. Enlisting hospital pharmacists and electronic health records and reviewing standard order sets for elderly patients are potentially effective strategies. Finally, increasing the presence of geriatricians and hospitalists would be expected to have a modest impact.

In a representative sample of elderly inpatients, we found that almost half received a potentially inappropriate medication and that the rate of inappropriate prescribing varied widely among doctors and hospitals. Additional research is needed to distinguish which of the Beers drugs are most harmful and which patients are at highest risk. Research should also focus on understanding differences in prescribing patterns, perhaps by studying the outliers at both ends of the quality spectrum, and on techniques to minimize non‐patient‐centered variation.

Medications can be considered inappropriate when their risk outweighs their benefit. The Beers list1 identifies medications that should be avoided in persons 65 years or older because they are ineffective or pose an unnecessarily high risk or because a safer alternative is available. Initially developed in 1991, the list has gained wide acceptance and has been updated twice.2, 3 In July 1999 it was adopted by the Centers for Medicare & Medicaid Services (CMS) for nursing home regulation, and in 2006 the National Committee on Quality Assurance adopted a modified version as a Health Plan Employer Data and Information Set (HEDIS) measure of quality of care for older Americans.4

A number of studies have demonstrated that inappropriate prescribing is common in the ambulatory setting,57 in nursing homes,8, 9 and in emergency departments10, 11 and that exposure to inappropriate medications is associated with increased risk of adverse drug reactions12 and hospitalization.13, 14 Initial studies of hospitalized patients1517 suggest that potentially inappropriate prescribing is also common among elderly inpatients and that reducing the misuse of psychotropic medications can prevent falls.18 We report on the incidence of and risk factors associated with potentially inappropriate prescribing in a large sample of hospitalized elders.

METHODS

Potentially Inappropriate Prescribing

Using the 2002 updated Beers criteria3 for potentially inappropriate medication (PIM) use in older adults, we identified the total number of PIMs administered to each patient during his or her hospital stay. We classified each PIM as either high or low severity based on the expert consensus expressed in the 1997 update of the Beers criteria.2 The list contains 48 PIMs and an additional 20 that should be avoided in patients with certain conditions. We did not include the second category of PIMs because we did not necessarily have sufficient patient information to make this determination. In addition, some of the standard PIMs, such as laxatives, although inappropriate for chronic outpatient use, could be appropriate in the hospital setting and were excluded from this analysis. Finally, several medications were considered inappropriate only above a given threshold (eg, lorazepam >3.0 mg/day) or for patients without a specific diagnosis (eg, digoxin >0.125 mg/day for patients without atrial fibrillation). We grouped PIMs that had similar side effects into 4 categories: sedatives, anticholinergics, causing orthostasis, or causing bleeding (Fig. 1).

Figure 1

RESULTS

We identified 519,853 patients at least 65 years old during the study period; 564 were excluded because of missing data for key variables or unclear principal diagnosis. An additional 25,318 were excluded because they were cared for by an attending with a surgical specialty. A total of 493,971 patients were included in the study (Table 1). Mean age was 78 years, and 24% of patients were 85 years or older. Forty‐three percent were male, 71% were white, and 39% were currently married. The most common principal diagnoses were community‐acquired pneumonia, congestive heart failure, and acute myocardial infarction. The most common comorbidities were hypertension, diabetes, and chronic pulmonary disease. Medicare was the primary payer for 91% of subjects, and 13% were in managed care plans. Most patients were cared for by internists (49%), family practitioners (18%), or cardiologists (17%). Only 1% of patients had a geriatrician as attending.

Just under half of all patients (49%) received at least 1 PIM, and 6% received 3 or more (Table 2). Thirty‐eight percent received at least 1 drug with a high severity rating (HS‐PIM). The most common PIMs were promethazine, diphenhydramine, propoxyphene, clonidine, amiodarone, and lorazepam (>3 mg/day).

Patient, Physician, and Hospital Factors Associated with PIMs

Patient, physician, and hospital characteristics were all associated with use of PIMs (Table 3). In univariate analyses, older patients were less likely to receive any class of PIM, and this difference was accentuated for HS‐PIMs. Women, American Indians, married people, and those not in managed care plans were slightly more likely to receive PIMs, whereas patients admitted with acute myocardial infarction or congestive heart failure were even more likely to receive PIMs (P < .0001 for all comparisons).

Table 3.Potentially Inappropriate Medication Use by Patient Characteristic

Patient characteristic	Any PIM n (row %)	Any high‐severity PIM n (row %)	Sedatives n (row %)	Anticholinergic effects n (row %)	Causing orthostasis n (row %)	Causing bleeding n (row %)
* An additional 24,602 patients (5%) seen by 42 additional physician specialties were excluded from this analysis. Chi‐square tests indicated all differences by patient characteristics were significant at P < .0001 except there was no significant difference by managed care status for use rates of drugs potentially causing orthostasis or drugs potentially causing bleeding.
Overall	239,771 (49%)	189,448 (38%)	156,384 (32%)	109,293 (22%)	43,805 (9%)	14,744 (3%)
Age group
6574 years	89,168 (53%)	72,573 (43%)	61,399 (36%)	44,792 (27%)	15,799 (9%)	6655 (4%)
7584 years	100,787 (49%)	79,595 (39%)	65,034 (32%)	45,121 (22%)	18,519 (9%)	5727 (3%)
85+ years	49,816 (42%)	37,280 (31%)	29,951 (25%)	19,380 (16%)	9487 (8%)	2362 (2%)
Sex
Male	100,824 (47%)	79,535 (37%)	63,591 (30%)	42,754 (20%)	17,885 (8%)	5771 (3%)
Female	138,947 (49%)	109,913 (39%)	92,793 (33%)	66,539 (24%)	25,920 (9%)	8973 (3%)
Race
White	173,481 (49%)	139,941 (40%)	112,556 (32%)	81,097 (23%)	27,555 (8%)	10,590 (3%)
Black	26,793 (51%)	18,655 (36%)	18,720 (36%)	11,263 (21%)	8925 (17%)	1536 (3%)
Hispanic	8509 (47%)	6370 (35%)	5549 (31%)	3505 (19%)	2047 (11%)	648 (4%)
American Indian	1091 (58%)	849 (45%)	818 (44%)	563 (30%)	190 (10%)	76 (4%)
Asian/Pacific Islander	2386 (40%)	1896 (32%)	1420 (24%)	1023 (17%)	519 (9%)	127 (2%)
Other	27,511 (43%)	21,737 (34%)	17,321 (27%)	11,842 (18%)	4569 (7%)	1767 (3%)
Marital status
Married/partner	96,874 (50%)	77,803 (40%)	63,303 (33%)	45,042 (23%)	16,765 (9%)	5969 (3%)
Widowed	74,622 (48%)	58,012 (37%)	48,367 (31%)	33,516 (22%)	13,865 (9%)	4354 (3%)
Single/separated/divorced	36,583 (48%)	28,799 (38%)	24,251 (32%)	16,115 (21%)	7229 (10%)	2399 (3%)
Other	31,692 (46%)	24,834 (36%)	20,463 (30%)	14,620 (21%)	5946 (9%)	2022 (3%)
Primary diagnosis
Pneumonia	56,845 (46%)	46,271 (38%)	35,353 (29%)	25,484 (21%)	9184 (7%)	4155 (3%)
Heart failure	56,460 (52%)	42,231 (39%)	34,340 (31%)	22,093 (20%)	10,117 (9%)	1945 (2%)
Acute MI	43,046 (61%)	37,849 (54%)	32,560 (46%)	25,568 (36%)	4738 (7%)	2549 (4%)
Ischemic stroke	25,763 (45%)	17,613 (31%)	18,500 (32%)	8742 (15%)	9644 (17%)	1384 (2%)
Chest pain	20,655 (41%)	16,363 (32%)	13,536 (27%)	10,520 (21%)	3474 (7%)	2027 (4%)
COPD	18,876 (42%)	14,626 (33%)	12,087 (27%)	8096 (18%)	3209 (7%)	1139 (3%)
Urinary tract infection	18,126 (46%)	14,495 (37%)	10,008 (25%)	8790 (22%)	3439 (9%)	1545 (4%)
Payer type
Nonmanaged care	212,150 (49%)	168,013 (39%)	138,679 (32%)	97,776 (23%)	38,341 (9%)	12,868 (3%)
Managed care	27,621 (44%)	21,435 (34%)	17,705 (28%)	11,517 (18%)	5464 (9%)	1876 (3%)
Attending physician specialty*
Internal medicine (internist%)	112,664 (47%)	86,907 (36%)	71,382 (30%)	48,746 (20%)	23,221 (10%)	7086 (3%)
Family/general medicine	41,303 (45%)	32,338 (36%)	25,653 (28%)	18,274 (20%)	7660 (8%)	2852 (3%)
Cardiology	48,485 (58%)	40,752 (49%)	34,859 (42%)	25,792 (31%)	5455 (7%)	2542 (3%)
Pulmonology	10,231 (48%)	8105 (38%)	6746 (32%)	4064 (19%)	1739 (8%)	574 (3%)
Hospitalist	7003 (47%)	5443 (36%)	4447 (30%)	3179 (21%)	1471 (10%)	463 (3%)
Nephrology	4508 (55%)	3388 (41%)	3132 (38%)	2054 (25%)	1326 (16%)	198 (2%)
Neurology	2420 (42%)	1789 (31%)	1625 (28%)	851 (15%)	699 (12%)	174 (3%)
Geriatrics	1020 (33%)	785 (25%)	596 (19%)	404 (13%)	196 (6%)	41 (1%)

The HS‐PIM prescribing varied substantially by attending specialty (Fig. 2). Internists, family practitioners, and hospitalists all had similar median rates (33%), cardiologists had a higher median rate (48%), and geriatricians had a lower rate (24%). The most common PIM also differed by specialty: whereas promethazine was the most commonly prescribed drug across most specialties, nephrologists and neurologists used clonidine, pulmonologists used lorazepam, and cardiologists used diphenhydramine most often. Among the 8% of physicians who saw at least 50 patients, there was also great variation in each specialty (Fig. 2). Among internists and cardiologists who saw at least 50 patients, the high‐severity PIM usage rate ranged from 0% to more than 90%.

Figure 2

There was substantial variation in PIM usage among hospitals, most notably by region. The mean proportion of patients receiving PIMs ranged from 34% at hospitals in the Northeast to 55% at hospitals in the South (Table 4). Smaller hospitals and those in urban settings had slightly lower rates, as did those that had geriatricians on staff. The teaching status of the hospital had little effect. Variation at the individual hospital level was extreme (Fig. 3). Although half of all hospitals had rates between 43% and 58%, in 7 hospitals with more than 300 encounters each, PIMs were never prescribed for geriatric patients.

Figure 3

Table 4.Percentage of Patients Prescribed Potentially Inappropriate Medication (PIM) by Hospital Characteristic

	Hospitals Total = 384 n (%)	Patients N = 49,3971 n (%)	Any PIM Mean = 48.2 Mean (SD)	Any high‐severity PIM Mean = 38.7 Mean (SD)	Sedatives Mean = 30.2 Mean (SD)	Anticholinergic effects Mean = 21.5 Mean (SD)	Causing orthostasis Mean = 8.5 Mean (SD)	Causing bleeding Mean = 3.1 Mean (SD)
Note: P values from analysis of variance of hospital use rates for each hospital characteristic. * P < .05, ** P < .001, *** P < .0001.
Hospital region			***	***	***	***	***	**
Midwest	76 (20%)	95,791 (19%)	38.8 (19.7)	30.0 (16.4)	24.3 (13.8)	15.1 (9.9)	6.9 (6.3)	3.1 (2.3)
Northeast	47 (12%)	79,138 (16%)	34.1 (12.6)	26.2 (11.2)	19.0 (9.2)	13.5 (8.1)	4.9 (2.3)	2.1 (1.6)
South	199 (52%)	260,870 (53%)	54.5 (10.1)	42.7 (9.6)	36.0 (10.8)	26.4 (8.6)	10.4 (4.6)	3.6 (2.5)
West	62 (16%)	58,172 (12%)	45.8 (8.1)	37.4 (7.1)	27.3 (7.7)	19.5 (5.7)	7.4 (4.8)	2.7 (1.3)
Teaching status
Nonteaching	297 (77%)	324,948 (66%)	47.3 (14.6)	36.9 (12.3)	29.8 (12.0)	21.3 (9.9)	8.7 (5.4)	3.3 (2.4)
Teaching	87 (23%)	169,023 (34%)	48.2 (16.0)	38.8 (14.2)	31.6 (14.5)	22.1 (10.2)	7.8 (4.4)	2.7 (1.5)
Staffed beds				*	**
22200	143 (37%)	80,741 (16%)	45.5 (16.9)	35.2 (14.6)	27.5 (14.0)	20.1 (10.3)	8.0 (6.2)	3.5 (3.1)
200400	137 (36%)	177,286 (36%)	47.7 (14.2)	37.8 (12.0)	30.5 (11.6)	22.0 (10.0)	8.4 (4.7)	3.0 (1.6)
400+	104 (27%)	235944 (48%)	50.1 (12.4)	39.6 (10.6)	33.5 (10.9)	22.7 (9.3)	9.3 (4.2)	2.9 (1.4)
Population serviced						*	*	**
Rural	119 (31%)	102,799 (21%)	48.4 (13.0)	38.3 (10.6)	29.2 (11.0)	23.2 (9.3)	7.5 (4.0)	3.7 (3.0)
Urban	265 (69%)	391,172 (79%)	47.1 (15.7)	36.9 (13.7)	30.6 (13.2)	20.7 (10.2)	9.0 (5.6)	2.9 (1.8)
Geriatrician presence
No	340 (89%)	409,281 (83%)	47.7 (15.3)	37.6 (13.0)	30.3 (12.8)	21.7 (10.0)	8.4 (5.3)	3.2 (2.3)
Yes	44 (11%)	84,690 (17%)	45.8 (11.4)	35.5 (10.6)	29.4 (10.8)	19.6 (9.4)	9.3 (4.3)	2.9 (1.6)

DISCUSSION

Although Americans age 65 years and older make up less than 15% of the U.S. population, they consume about one third of all prescription drugs20 and account for one third of all hospital admissions.21 Using the Beers list, numerous studies have documented high rates of potentially inappropriate prescribing for community‐dwelling elderly and nursing home patients and, in some studies, an attendant risk of falling,2224 hip fracture,25, 26 hospitalization,13 or death.14 Applying these same criteria to a large sample of medical inpatients, we found that almost half received a potentially inappropriate drug, most of high severity. Moreover, the PIM prescribing rate varied substantially by region, hospital, and attending physician specialty. Although the use of PIMs was associated with patient age, comorbidities, and primary diagnosis, these patient factors explained only a small portion of the variation in prescribing practices across groups of physicians and hospitals.

Using consensus criteria, Beers originally found that 40% of the residents in 12 nursing homes received at least 1 PIM,8 and studies of community‐dwelling elderly demonstrated rates of 21% to 37%, with little change over time.6, 27, 28 Several small studies have examined inpatient prescribing.16, 17, 29, 30 The largest17 found that only 15% of elderly Italian inpatients received a PIM. Our finding, that 49% of inpatients had received at least 1 PIM, may partially reflect the high prevalence of use among elderly US patients in nursing homes and the community.

Regional variation has been demonstrated for ambulatory patients in the US6 and Europe.31 Zhan et al. found slightly higher rates of PIM use in the Midwest and the South (23%) than in the Northeast and the West (19%). Variation in Europe was greater, with 41% of patients in the Czech Republic versus 5.8% of patients in Denmark receiving at least 1 PIM. We found that region was the strongest predictor of in‐hospital HS‐PIM use, with patients in the South most likely and patients in the Northeast least likely to receive HS‐PIMs. This variation persisted even after adjusting for differences in other patient and hospital factors, suggesting that local custom played a large role in the decision to prescribe HS‐PIMs. Moreover, because outpatient rates are more uniform, these large differences seem limited to inpatient practice.

Patient factors have also been examined. Advanced age was associated with decreased PIM use in some studies17, 28, 31 but not in others.6, 27 We found increasing age to be strongly associated with decreased PIM use, suggesting that in the hospital, at least, doctors take care to avoid prescribing certain drugs to the frail elderly. Women appear to be consistently at higher risk than men,6, 27, 28, 31 and white patients are more at risk than those of other races.6 Our finding that certain diagnoses were associated with higher or lower rates has not been reported previously. The lower rates associated with stroke and COPD suggest that prescribers were aware that these patients were at increased risk of delirium and respiratory depression. The higher rates associated with myocardial infarction may have to do with the use of standardized order sets (eg, cath lab orders) that do not consider the age of the patient going for the procedure.

Admission to a geriatric service32 and intervention by a clinical pharmacist33 have been shown to decrease PIM prescribing at discharge. We noted that patients cared for by a geriatrician had the lowest rates of PIM prescribing during hospitalization as well and that hospitals with geriatricians had lower rates overall, possibly demonstrating that geriatricians had a ripple effect on their colleagues. Hospitalists also had lower rates than internists, supporting the notion that hospitalists provide higher‐quality inpatient care.

Our study had some important limitations. First, we only had access to inpatient administrative records. Thus, we could not identify which medications were continued from home and which were begun in the hospital, nor could we know the indications for which specific drugs were prescribed or who prescribed them. Based on published outpatient rates, however, we could assume that many of the drugs were started in the hospital and that others could have been discontinued but were not. Second, the Beers list was developed by the modified Delphi method; there was little empirical evidence of the danger of specific drugs, although some classes, such as benzodiazepines, opiates and digoxin, have been associated with inpatient falls.18, 3436 Furthermore, our administrative database did not allow us to balance the risks and benefits for particular patients; hence, the medications were only potentially inappropriate, and our study did not address the consequences of such prescribing. Although some of these drugs may be appropriate under certain circumstances, it is unlikely that these circumstances would vary by 60% across geographic regions or that internists would encounter these circumstances more often than do hospitalists. Thus, although we could not identify specific patients who received inappropriate medications, we did identify certain hospitals and even whole regions of the country in which the rate of inappropriate prescribing was high. Third, the Beers list, which was developed for outpatient use, may be less relevant in the inpatient setting. However, given that inpatients have more organ dysfunction and are at higher risk of delirium and falls, it may actually be more applicable to hospitalized patients. We similarly did not distinguish between single and multiple doses because the Beers list does not make such a distinction, and there is no empirical evidence that a single dose is safe. Indeed, patients are often at highest risk of falls immediately after initiation of therapy.3739 We did, however, exclude drugs such as laxatives, which may be appropriate for brief inpatient use but not for chronic use.

Our study also had a number of strengths. The large sample size, representing approximately 5% of annual inpatient admissions in the US over 2 years, offered an instructive look at the recent prescribing patterns of thousands of US physicians. We were able to identify many patient, physician, and hospital factors associated with PIM prescribing that have not previously been reported. Some of these factors, such as advanced age and comorbid diagnoses, suggest that physicians do tailor their treatment to individual patients. Nevertheless, patient factors accounted for only a small portion of the variation in prescribing. The largest variation, associated with regional, hospital, and physician factors, highlights the opportunity for improvement.

At the same time, our findings are encouraging for 2 reasons. First, most inappropriate prescribing involved only a handful of medications, so small changes in prescribing patterns could have a tremendous impact. Second, observing the practice of individual physicians and hospitals reveals what is possible. We found that in most specialties there were physicians who rarely or never used PIMs. We also found 7 hospitals, each with at least 300 cases, where no PIMs were ever prescribed.

Where should hospitals focus their efforts to prevent inappropriate prescribing? Our data highlight the complexity of the problem, which seems daunting. PIM prescribing is spread across all specialties, including geriatrics, and although cardiologists had the highest rate of prescribing, internists, who were more numerous, accounted for a much higher overall number of potentially inappropriate prescriptions. It would be instructive to study the 7 hospitals where PIMs were never prescribed or to interview those physicians who never prescribed PIMs, but the anonymous nature of our data would not allow for this. However, our data do suggest some directions. First, hospitals should become aware of their own rates of PIM use because measurement is the first step in quality improvement. Next, hospitals should focus efforts on reducing the use of the most common drugs. Eliminating just 3 drugs promethazine, diphenhydramine, and propoxyphenewould reduce the use of PIMs in 24% of elderly patients. Enlisting hospital pharmacists and electronic health records and reviewing standard order sets for elderly patients are potentially effective strategies. Finally, increasing the presence of geriatricians and hospitalists would be expected to have a modest impact.

In a representative sample of elderly inpatients, we found that almost half received a potentially inappropriate medication and that the rate of inappropriate prescribing varied widely among doctors and hospitals. Additional research is needed to distinguish which of the Beers drugs are most harmful and which patients are at highest risk. Research should also focus on understanding differences in prescribing patterns, perhaps by studying the outliers at both ends of the quality spectrum, and on techniques to minimize non‐patient‐centered variation.

Medications can be considered inappropriate when their risk outweighs their benefit. The Beers list1 identifies medications that should be avoided in persons 65 years or older because they are ineffective or pose an unnecessarily high risk or because a safer alternative is available. Initially developed in 1991, the list has gained wide acceptance and has been updated twice.2, 3 In July 1999 it was adopted by the Centers for Medicare & Medicaid Services (CMS) for nursing home regulation, and in 2006 the National Committee on Quality Assurance adopted a modified version as a Health Plan Employer Data and Information Set (HEDIS) measure of quality of care for older Americans.4

A number of studies have demonstrated that inappropriate prescribing is common in the ambulatory setting,57 in nursing homes,8, 9 and in emergency departments10, 11 and that exposure to inappropriate medications is associated with increased risk of adverse drug reactions12 and hospitalization.13, 14 Initial studies of hospitalized patients1517 suggest that potentially inappropriate prescribing is also common among elderly inpatients and that reducing the misuse of psychotropic medications can prevent falls.18 We report on the incidence of and risk factors associated with potentially inappropriate prescribing in a large sample of hospitalized elders.

METHODS

Potentially Inappropriate Prescribing

Using the 2002 updated Beers criteria3 for potentially inappropriate medication (PIM) use in older adults, we identified the total number of PIMs administered to each patient during his or her hospital stay. We classified each PIM as either high or low severity based on the expert consensus expressed in the 1997 update of the Beers criteria.2 The list contains 48 PIMs and an additional 20 that should be avoided in patients with certain conditions. We did not include the second category of PIMs because we did not necessarily have sufficient patient information to make this determination. In addition, some of the standard PIMs, such as laxatives, although inappropriate for chronic outpatient use, could be appropriate in the hospital setting and were excluded from this analysis. Finally, several medications were considered inappropriate only above a given threshold (eg, lorazepam >3.0 mg/day) or for patients without a specific diagnosis (eg, digoxin >0.125 mg/day for patients without atrial fibrillation). We grouped PIMs that had similar side effects into 4 categories: sedatives, anticholinergics, causing orthostasis, or causing bleeding (Fig. 1).

Figure 1

RESULTS

We identified 519,853 patients at least 65 years old during the study period; 564 were excluded because of missing data for key variables or unclear principal diagnosis. An additional 25,318 were excluded because they were cared for by an attending with a surgical specialty. A total of 493,971 patients were included in the study (Table 1). Mean age was 78 years, and 24% of patients were 85 years or older. Forty‐three percent were male, 71% were white, and 39% were currently married. The most common principal diagnoses were community‐acquired pneumonia, congestive heart failure, and acute myocardial infarction. The most common comorbidities were hypertension, diabetes, and chronic pulmonary disease. Medicare was the primary payer for 91% of subjects, and 13% were in managed care plans. Most patients were cared for by internists (49%), family practitioners (18%), or cardiologists (17%). Only 1% of patients had a geriatrician as attending.

Just under half of all patients (49%) received at least 1 PIM, and 6% received 3 or more (Table 2). Thirty‐eight percent received at least 1 drug with a high severity rating (HS‐PIM). The most common PIMs were promethazine, diphenhydramine, propoxyphene, clonidine, amiodarone, and lorazepam (>3 mg/day).

Patient, Physician, and Hospital Factors Associated with PIMs

Patient, physician, and hospital characteristics were all associated with use of PIMs (Table 3). In univariate analyses, older patients were less likely to receive any class of PIM, and this difference was accentuated for HS‐PIMs. Women, American Indians, married people, and those not in managed care plans were slightly more likely to receive PIMs, whereas patients admitted with acute myocardial infarction or congestive heart failure were even more likely to receive PIMs (P < .0001 for all comparisons).

Table 3.Potentially Inappropriate Medication Use by Patient Characteristic

Patient characteristic	Any PIM n (row %)	Any high‐severity PIM n (row %)	Sedatives n (row %)	Anticholinergic effects n (row %)	Causing orthostasis n (row %)	Causing bleeding n (row %)
* An additional 24,602 patients (5%) seen by 42 additional physician specialties were excluded from this analysis. Chi‐square tests indicated all differences by patient characteristics were significant at P < .0001 except there was no significant difference by managed care status for use rates of drugs potentially causing orthostasis or drugs potentially causing bleeding.
Overall	239,771 (49%)	189,448 (38%)	156,384 (32%)	109,293 (22%)	43,805 (9%)	14,744 (3%)
Age group
6574 years	89,168 (53%)	72,573 (43%)	61,399 (36%)	44,792 (27%)	15,799 (9%)	6655 (4%)
7584 years	100,787 (49%)	79,595 (39%)	65,034 (32%)	45,121 (22%)	18,519 (9%)	5727 (3%)
85+ years	49,816 (42%)	37,280 (31%)	29,951 (25%)	19,380 (16%)	9487 (8%)	2362 (2%)
Sex
Male	100,824 (47%)	79,535 (37%)	63,591 (30%)	42,754 (20%)	17,885 (8%)	5771 (3%)
Female	138,947 (49%)	109,913 (39%)	92,793 (33%)	66,539 (24%)	25,920 (9%)	8973 (3%)
Race
White	173,481 (49%)	139,941 (40%)	112,556 (32%)	81,097 (23%)	27,555 (8%)	10,590 (3%)
Black	26,793 (51%)	18,655 (36%)	18,720 (36%)	11,263 (21%)	8925 (17%)	1536 (3%)
Hispanic	8509 (47%)	6370 (35%)	5549 (31%)	3505 (19%)	2047 (11%)	648 (4%)
American Indian	1091 (58%)	849 (45%)	818 (44%)	563 (30%)	190 (10%)	76 (4%)
Asian/Pacific Islander	2386 (40%)	1896 (32%)	1420 (24%)	1023 (17%)	519 (9%)	127 (2%)
Other	27,511 (43%)	21,737 (34%)	17,321 (27%)	11,842 (18%)	4569 (7%)	1767 (3%)
Marital status
Married/partner	96,874 (50%)	77,803 (40%)	63,303 (33%)	45,042 (23%)	16,765 (9%)	5969 (3%)
Widowed	74,622 (48%)	58,012 (37%)	48,367 (31%)	33,516 (22%)	13,865 (9%)	4354 (3%)
Single/separated/divorced	36,583 (48%)	28,799 (38%)	24,251 (32%)	16,115 (21%)	7229 (10%)	2399 (3%)
Other	31,692 (46%)	24,834 (36%)	20,463 (30%)	14,620 (21%)	5946 (9%)	2022 (3%)
Primary diagnosis
Pneumonia	56,845 (46%)	46,271 (38%)	35,353 (29%)	25,484 (21%)	9184 (7%)	4155 (3%)
Heart failure	56,460 (52%)	42,231 (39%)	34,340 (31%)	22,093 (20%)	10,117 (9%)	1945 (2%)
Acute MI	43,046 (61%)	37,849 (54%)	32,560 (46%)	25,568 (36%)	4738 (7%)	2549 (4%)
Ischemic stroke	25,763 (45%)	17,613 (31%)	18,500 (32%)	8742 (15%)	9644 (17%)	1384 (2%)
Chest pain	20,655 (41%)	16,363 (32%)	13,536 (27%)	10,520 (21%)	3474 (7%)	2027 (4%)
COPD	18,876 (42%)	14,626 (33%)	12,087 (27%)	8096 (18%)	3209 (7%)	1139 (3%)
Urinary tract infection	18,126 (46%)	14,495 (37%)	10,008 (25%)	8790 (22%)	3439 (9%)	1545 (4%)
Payer type
Nonmanaged care	212,150 (49%)	168,013 (39%)	138,679 (32%)	97,776 (23%)	38,341 (9%)	12,868 (3%)
Managed care	27,621 (44%)	21,435 (34%)	17,705 (28%)	11,517 (18%)	5464 (9%)	1876 (3%)
Attending physician specialty*
Internal medicine (internist%)	112,664 (47%)	86,907 (36%)	71,382 (30%)	48,746 (20%)	23,221 (10%)	7086 (3%)
Family/general medicine	41,303 (45%)	32,338 (36%)	25,653 (28%)	18,274 (20%)	7660 (8%)	2852 (3%)
Cardiology	48,485 (58%)	40,752 (49%)	34,859 (42%)	25,792 (31%)	5455 (7%)	2542 (3%)
Pulmonology	10,231 (48%)	8105 (38%)	6746 (32%)	4064 (19%)	1739 (8%)	574 (3%)
Hospitalist	7003 (47%)	5443 (36%)	4447 (30%)	3179 (21%)	1471 (10%)	463 (3%)
Nephrology	4508 (55%)	3388 (41%)	3132 (38%)	2054 (25%)	1326 (16%)	198 (2%)
Neurology	2420 (42%)	1789 (31%)	1625 (28%)	851 (15%)	699 (12%)	174 (3%)
Geriatrics	1020 (33%)	785 (25%)	596 (19%)	404 (13%)	196 (6%)	41 (1%)

The HS‐PIM prescribing varied substantially by attending specialty (Fig. 2). Internists, family practitioners, and hospitalists all had similar median rates (33%), cardiologists had a higher median rate (48%), and geriatricians had a lower rate (24%). The most common PIM also differed by specialty: whereas promethazine was the most commonly prescribed drug across most specialties, nephrologists and neurologists used clonidine, pulmonologists used lorazepam, and cardiologists used diphenhydramine most often. Among the 8% of physicians who saw at least 50 patients, there was also great variation in each specialty (Fig. 2). Among internists and cardiologists who saw at least 50 patients, the high‐severity PIM usage rate ranged from 0% to more than 90%.

Figure 2

There was substantial variation in PIM usage among hospitals, most notably by region. The mean proportion of patients receiving PIMs ranged from 34% at hospitals in the Northeast to 55% at hospitals in the South (Table 4). Smaller hospitals and those in urban settings had slightly lower rates, as did those that had geriatricians on staff. The teaching status of the hospital had little effect. Variation at the individual hospital level was extreme (Fig. 3). Although half of all hospitals had rates between 43% and 58%, in 7 hospitals with more than 300 encounters each, PIMs were never prescribed for geriatric patients.

Figure 3

Table 4.Percentage of Patients Prescribed Potentially Inappropriate Medication (PIM) by Hospital Characteristic

	Hospitals Total = 384 n (%)	Patients N = 49,3971 n (%)	Any PIM Mean = 48.2 Mean (SD)	Any high‐severity PIM Mean = 38.7 Mean (SD)	Sedatives Mean = 30.2 Mean (SD)	Anticholinergic effects Mean = 21.5 Mean (SD)	Causing orthostasis Mean = 8.5 Mean (SD)	Causing bleeding Mean = 3.1 Mean (SD)
Note: P values from analysis of variance of hospital use rates for each hospital characteristic. * P < .05, ** P < .001, *** P < .0001.
Hospital region			***	***	***	***	***	**
Midwest	76 (20%)	95,791 (19%)	38.8 (19.7)	30.0 (16.4)	24.3 (13.8)	15.1 (9.9)	6.9 (6.3)	3.1 (2.3)
Northeast	47 (12%)	79,138 (16%)	34.1 (12.6)	26.2 (11.2)	19.0 (9.2)	13.5 (8.1)	4.9 (2.3)	2.1 (1.6)
South	199 (52%)	260,870 (53%)	54.5 (10.1)	42.7 (9.6)	36.0 (10.8)	26.4 (8.6)	10.4 (4.6)	3.6 (2.5)
West	62 (16%)	58,172 (12%)	45.8 (8.1)	37.4 (7.1)	27.3 (7.7)	19.5 (5.7)	7.4 (4.8)	2.7 (1.3)
Teaching status
Nonteaching	297 (77%)	324,948 (66%)	47.3 (14.6)	36.9 (12.3)	29.8 (12.0)	21.3 (9.9)	8.7 (5.4)	3.3 (2.4)
Teaching	87 (23%)	169,023 (34%)	48.2 (16.0)	38.8 (14.2)	31.6 (14.5)	22.1 (10.2)	7.8 (4.4)	2.7 (1.5)
Staffed beds				*	**
22200	143 (37%)	80,741 (16%)	45.5 (16.9)	35.2 (14.6)	27.5 (14.0)	20.1 (10.3)	8.0 (6.2)	3.5 (3.1)
200400	137 (36%)	177,286 (36%)	47.7 (14.2)	37.8 (12.0)	30.5 (11.6)	22.0 (10.0)	8.4 (4.7)	3.0 (1.6)
400+	104 (27%)	235944 (48%)	50.1 (12.4)	39.6 (10.6)	33.5 (10.9)	22.7 (9.3)	9.3 (4.2)	2.9 (1.4)
Population serviced						*	*	**
Rural	119 (31%)	102,799 (21%)	48.4 (13.0)	38.3 (10.6)	29.2 (11.0)	23.2 (9.3)	7.5 (4.0)	3.7 (3.0)
Urban	265 (69%)	391,172 (79%)	47.1 (15.7)	36.9 (13.7)	30.6 (13.2)	20.7 (10.2)	9.0 (5.6)	2.9 (1.8)
Geriatrician presence
No	340 (89%)	409,281 (83%)	47.7 (15.3)	37.6 (13.0)	30.3 (12.8)	21.7 (10.0)	8.4 (5.3)	3.2 (2.3)
Yes	44 (11%)	84,690 (17%)	45.8 (11.4)	35.5 (10.6)	29.4 (10.8)	19.6 (9.4)	9.3 (4.3)	2.9 (1.6)

DISCUSSION

Although Americans age 65 years and older make up less than 15% of the U.S. population, they consume about one third of all prescription drugs20 and account for one third of all hospital admissions.21 Using the Beers list, numerous studies have documented high rates of potentially inappropriate prescribing for community‐dwelling elderly and nursing home patients and, in some studies, an attendant risk of falling,2224 hip fracture,25, 26 hospitalization,13 or death.14 Applying these same criteria to a large sample of medical inpatients, we found that almost half received a potentially inappropriate drug, most of high severity. Moreover, the PIM prescribing rate varied substantially by region, hospital, and attending physician specialty. Although the use of PIMs was associated with patient age, comorbidities, and primary diagnosis, these patient factors explained only a small portion of the variation in prescribing practices across groups of physicians and hospitals.

Using consensus criteria, Beers originally found that 40% of the residents in 12 nursing homes received at least 1 PIM,8 and studies of community‐dwelling elderly demonstrated rates of 21% to 37%, with little change over time.6, 27, 28 Several small studies have examined inpatient prescribing.16, 17, 29, 30 The largest17 found that only 15% of elderly Italian inpatients received a PIM. Our finding, that 49% of inpatients had received at least 1 PIM, may partially reflect the high prevalence of use among elderly US patients in nursing homes and the community.

Regional variation has been demonstrated for ambulatory patients in the US6 and Europe.31 Zhan et al. found slightly higher rates of PIM use in the Midwest and the South (23%) than in the Northeast and the West (19%). Variation in Europe was greater, with 41% of patients in the Czech Republic versus 5.8% of patients in Denmark receiving at least 1 PIM. We found that region was the strongest predictor of in‐hospital HS‐PIM use, with patients in the South most likely and patients in the Northeast least likely to receive HS‐PIMs. This variation persisted even after adjusting for differences in other patient and hospital factors, suggesting that local custom played a large role in the decision to prescribe HS‐PIMs. Moreover, because outpatient rates are more uniform, these large differences seem limited to inpatient practice.

Patient factors have also been examined. Advanced age was associated with decreased PIM use in some studies17, 28, 31 but not in others.6, 27 We found increasing age to be strongly associated with decreased PIM use, suggesting that in the hospital, at least, doctors take care to avoid prescribing certain drugs to the frail elderly. Women appear to be consistently at higher risk than men,6, 27, 28, 31 and white patients are more at risk than those of other races.6 Our finding that certain diagnoses were associated with higher or lower rates has not been reported previously. The lower rates associated with stroke and COPD suggest that prescribers were aware that these patients were at increased risk of delirium and respiratory depression. The higher rates associated with myocardial infarction may have to do with the use of standardized order sets (eg, cath lab orders) that do not consider the age of the patient going for the procedure.

Admission to a geriatric service32 and intervention by a clinical pharmacist33 have been shown to decrease PIM prescribing at discharge. We noted that patients cared for by a geriatrician had the lowest rates of PIM prescribing during hospitalization as well and that hospitals with geriatricians had lower rates overall, possibly demonstrating that geriatricians had a ripple effect on their colleagues. Hospitalists also had lower rates than internists, supporting the notion that hospitalists provide higher‐quality inpatient care.

Our study had some important limitations. First, we only had access to inpatient administrative records. Thus, we could not identify which medications were continued from home and which were begun in the hospital, nor could we know the indications for which specific drugs were prescribed or who prescribed them. Based on published outpatient rates, however, we could assume that many of the drugs were started in the hospital and that others could have been discontinued but were not. Second, the Beers list was developed by the modified Delphi method; there was little empirical evidence of the danger of specific drugs, although some classes, such as benzodiazepines, opiates and digoxin, have been associated with inpatient falls.18, 3436 Furthermore, our administrative database did not allow us to balance the risks and benefits for particular patients; hence, the medications were only potentially inappropriate, and our study did not address the consequences of such prescribing. Although some of these drugs may be appropriate under certain circumstances, it is unlikely that these circumstances would vary by 60% across geographic regions or that internists would encounter these circumstances more often than do hospitalists. Thus, although we could not identify specific patients who received inappropriate medications, we did identify certain hospitals and even whole regions of the country in which the rate of inappropriate prescribing was high. Third, the Beers list, which was developed for outpatient use, may be less relevant in the inpatient setting. However, given that inpatients have more organ dysfunction and are at higher risk of delirium and falls, it may actually be more applicable to hospitalized patients. We similarly did not distinguish between single and multiple doses because the Beers list does not make such a distinction, and there is no empirical evidence that a single dose is safe. Indeed, patients are often at highest risk of falls immediately after initiation of therapy.3739 We did, however, exclude drugs such as laxatives, which may be appropriate for brief inpatient use but not for chronic use.

Our study also had a number of strengths. The large sample size, representing approximately 5% of annual inpatient admissions in the US over 2 years, offered an instructive look at the recent prescribing patterns of thousands of US physicians. We were able to identify many patient, physician, and hospital factors associated with PIM prescribing that have not previously been reported. Some of these factors, such as advanced age and comorbid diagnoses, suggest that physicians do tailor their treatment to individual patients. Nevertheless, patient factors accounted for only a small portion of the variation in prescribing. The largest variation, associated with regional, hospital, and physician factors, highlights the opportunity for improvement.

At the same time, our findings are encouraging for 2 reasons. First, most inappropriate prescribing involved only a handful of medications, so small changes in prescribing patterns could have a tremendous impact. Second, observing the practice of individual physicians and hospitals reveals what is possible. We found that in most specialties there were physicians who rarely or never used PIMs. We also found 7 hospitals, each with at least 300 cases, where no PIMs were ever prescribed.

Where should hospitals focus their efforts to prevent inappropriate prescribing? Our data highlight the complexity of the problem, which seems daunting. PIM prescribing is spread across all specialties, including geriatrics, and although cardiologists had the highest rate of prescribing, internists, who were more numerous, accounted for a much higher overall number of potentially inappropriate prescriptions. It would be instructive to study the 7 hospitals where PIMs were never prescribed or to interview those physicians who never prescribed PIMs, but the anonymous nature of our data would not allow for this. However, our data do suggest some directions. First, hospitals should become aware of their own rates of PIM use because measurement is the first step in quality improvement. Next, hospitals should focus efforts on reducing the use of the most common drugs. Eliminating just 3 drugs promethazine, diphenhydramine, and propoxyphenewould reduce the use of PIMs in 24% of elderly patients. Enlisting hospital pharmacists and electronic health records and reviewing standard order sets for elderly patients are potentially effective strategies. Finally, increasing the presence of geriatricians and hospitalists would be expected to have a modest impact.

In a representative sample of elderly inpatients, we found that almost half received a potentially inappropriate medication and that the rate of inappropriate prescribing varied widely among doctors and hospitals. Additional research is needed to distinguish which of the Beers drugs are most harmful and which patients are at highest risk. Research should also focus on understanding differences in prescribing patterns, perhaps by studying the outliers at both ends of the quality spectrum, and on techniques to minimize non‐patient‐centered variation.

Despite impressive advances in cardiac diagnostic technology, cardiac examination (CE) remains an essential skill for screening for abnormal sounds, for evaluating cardiovascular system function, and for guiding further diagnostic testing.16

In practice, these benefits may be attenuated if CE skills are inadequate. Numerous studies have documented substantial CE deficiencies among physicians at various points in their careers from medical school to practice.79 In 1 study, residents' CE mistakes accounted for one‐third of all physical diagnostic errors.10 When murmurs are detected, physicians will often reflexively order an echocardiogram and refer to a cardiologist, regardless of the cost or indication. As a consequence, echocardiography use is rising faster than the aging population or the incidence of cardiac pathological conditions would explain.11 Because cost‐effective medicine depends on the appropriate application of clinical skills like CE, the loss of these skills is a major shortcoming.1215

The reasons for the decline in physicians' CE skills are numerous. High reliance on ordering diagnostic tests,16 conducting teaching rounds away from the bedside,17, 18 time constraints during residency,16, 19 and declining CE skills of faculty members themselves7 all may contribute to the diminished CE skills of residents. Residents, who themselves identify abnormal heart sounds at alarmingly low rates, play an ever‐increasing role in medical students' instruction,7, 9 exacerbating the problem.

Responding to growing concerns over patient safety and quality of care,16, 20 public and professional organizations have called for renewed emphasis on teaching and evaluating clinical skills.21 For example, the American Board of Internal Medicine has added a physical diagnosis component to its recertification program.22 The Accreditation Council for Graduate Medical Education (ACGME) describes general competencies for residents, including patient care that should include proper physical examination skills.23 Although mandating uniform standards is a welcome change for improving CE competence, the challenge remains for medical school deans and program directors to fit structured physical examination skills training into an already crowded curriculum.16, 24 Moreover, the impact of these efforts to improve CE is uncertain because programs lack an objective measure of CE competence.

The CE training is itself a challenge: sight, sound, and touch all contribute to the clinical impression. For this reason, it is difficult to teach away from the bedside. Unlike pulmonary examination, for which a diagnosis is best made by listening, cardiac auscultation is only one (frequently overemphasized) aspect of CE.25 Medical knowledge of cardiac anatomy and physiology, visualization of cardiovascular findings, and integration of auditory and visual findings are all components of accurate CE.7 Traditionally, CE was taught through direct experience with patients at the bedside under the supervision of seasoned clinicians. However, exposure and learning from good teaching patients has waned. Audiotapes, heart sound simulators, mannequins, and other computer‐based interventions have been used as surrogates, but none has been widely adopted.26, 27 The best practice for teaching CE is not known.

To help to improve CE education during residency, we implemented and evaluated a novel Web‐based CE curriculum that emphasized 4 aspects of CE: cardiovascular anatomy and physiology, auditory skills, visual skills, and integration of auscultatory and visual findings. Our hypothesis was that this new curriculum would improve learning of CE skills, that residents would retain what they learn, and that this curriculum would be better than conventional education in teaching CE skills.

METHODS

Study Participants, Site, and Design

Internal medicine (IM) and family medicine (FM) interns (R1s, n = 59) from university‐ and community‐based residency programs, respectively, participated in this controlled trial of an educational intervention to teach CE.

The intervention group consisted of 26 IM and 8 FM interns at the beginning of the academic year in June 2003. To establish baseline scores, all interns took a 50‐question multimedia test of CE competency described previously.7, 28, 29 Subsequently, all interns completed a required 4‐week cardiology ward rotation. During this rotation, they were instructed to complete a Web‐based CE tutorial with accompanying worksheet and to attend 3 one‐hour sessions with a hospitalist instructor. Their schedules were arranged to allow for this educational time. During the third meeting with the instructor, interns were tested again to establish posttraining scores. Finally, at the end of the academic year, interns were tested once again to establish retention scores.

The control group consisted of 25 first‐year IM residents who were tested at the end of their academic year in June 2003. These test scores served as historical controls for interns who had just completed their first year of residency and who had received standard ward rotation without incorporated Web‐based training in CE. Interns from both groups had many opportunities for one‐on‐one instruction in CE because each intern was assigned for the cardiology rotation to a private practice cardiology attending. Figure 1 outlines the number of IM and FM interns eligible and the number actually tested at each stage of the study.

Figure 1

RESULTS

Research Question 1

Individual baseline and posttraining CE Test scores for the intervention group are plotted in the first 2 columns of Figure 2. The posttraining mean score improved from baseline (66.0 5.2 vs. 54.2 5.4, P = .002). The knowledge, audio, and visual subcategories of CE competence showed similar improvements (P .001; Table 3). The score for the integration of audiovisual skills subcategory was higher at baseline than the other subcategory scores and remained unchanged.

Figure 2

Table 3.Subcategory and Overall CE Competency Scores of Intervention and Control Groups for 4 Research Questions

Question 1. Does the Web‐based curriculum improve CE skills?
Subcategory	Intervention baseline (n = 30)		Intervention end of year (n = 30)
	Mean, % (SD)	95% CI	Mean, % (SD)	95% CI	P value (paired t test)
Question 2. Do interns retain this improvement?
Subcategory	Intervention baseline (n = 18)		Intervention end of year (n = 18)
	Mean, % (SD)	95% CI	Mean, % (SD)	95% CI	P value (paired t test)
Question 3. Does clinical training alone improve CE skills?
Subcategory	Intervention baseline* (n = 34)		Control end of year (n = 25)
	Mean, % (SD)	95% CI	Mean, % (SD)	95% CI	P value (t test)
Question 4. Is the Web‐based curriculum better than clinical training alone?
Subcategory	Intervention end of year (n = 23)		Control end of year (n = 25)
	Mean, % (SD)	95% CI	Mean, % (SD)	95% CI	P value (t test)
* Baseline scores for Web training also served as baseline scores for clinical training alone. CE, cardiac examination; SD, standard deviation; 95% CI, 95% confidence interval for mean. The full text of the questions were: Question 1. Immediately after cardiology rotation, concomitant Web‐based training improved knowledge, audio, and visual skills scores as well as overall CE competency score. Question 2. At the end of the year, interns retained the improvement in knowledge and audio skills and overall CE competency. Question 3. The standard cardiology rotation did not significantly improve either CE subcategory scores or the overall competency score. Question 4. When compared at the end of the year, intervention interns (those who received Web training) showed significantly better knowledge, audio, and overall competency scores.
Knowledge	55.9% (14.7%)	50.4%‐61.4%	68.9% (15.3%)	63.2%‐74.6%	<.001
Audio	69.6% (17.4%)	63.1%‐76.0%	83.6% (10.3%)	79.7%‐87.4 %	<.001
Visual	56.7% (21.4%)	48.7%‐64.7%	71.9% (16.6%)	65.7%‐78.1%	<.001
Integration	77.5% (14.5%)	72.1%‐83.0%	76.8% (12.8%)	72.1%‐81.6%	.85
Overall score	54.2% (14.5%)	48.8%‐59.6%	66.0% (13.6%)	60.9%‐71.2%	.002
Knowledge	58.6% (15.2%)	51.0%‐66.2%	67.6% (16.7%)	60.4%‐74.9%	.05
Audio	67.4% (17.1%)	58.9%‐75.9%	82.3% (10.4%)	77.8%‐86.8%	.004
Visual	51.6% (22.0%)	40.7%‐62.5%	65.2% (20.6%)	56.3%‐74.1%	.13
Integration	76.0 % (15.8%)	68.2%‐83.8%	77.1% (13.2%)	71.4%‐82.8%	.95
Overall score	51.8 % (15.0%)	44.3%‐59.3%	63.5% (14.9%)	56.1%‐70.9%	.02
Knowledge	56.5% (14.6%)	51.4%‐61.6%	57.8% (15.4%)	51.4%‐64.1%	.75
Audio	70.0% (16.9%)	64.1%‐75.9%	73.3% (13.6%)	67.7%‐79.0%	.42
Visual	57.6% (20.9%)	50.3%‐64.9%	66.9% (15.3%)	60.6%‐73.2%	.07
Integration	77.8% (13.8%)	72.9%‐82.5%	76.0% (12.2%)	71.0%‐81.0%	.62
Overall score	54.7% (13.7%)	49.9%‐59.5%	56.8% (12.4%)	51.7%‐61.9%	.54
Knowledge	67.6% (16.7%)	60.4%‐74.8%	57.8% (15.4%)	51.4%‐64.1%	0.04
Audio	82.3% (10.4%)	77.8%‐86.8%	73.3% (13.6%)	67.7%‐79.0%	0.01
Visual	65.2% (20.6%)	56.3%‐74.1%	66.9% (15.3%)	60.6%‐73.2%	0.75
Integration	77.1% (13.2%)	71.4%‐82.8%	76.0% (12.2%)	71.0%‐81.0%	0.76
Overall score	64.7% (14.5%)	58.4%‐71.0%	56.8% (12.4%)	51.7%‐61.9%	0.05

DISCUSSION

To improve CE competence of residents, we designed and implemented a Web‐based CE curriculum using virtual patients (VPs), which standardized the interns' exposure to a spectrum of medical conditions and allowed them flexibility in choosing when and where the training occurred.30 In this controlled educational intervention, we found significant improvements in interns' mean CE competency scores immediately following the Web‐based training; these improvements were retained at the end of the academic year. The Web‐based curriculum also improved scores in all 4 subcategories except integration of audio and visual skills. Although cardiac physiology knowledge and audio skills were retained, visual skills showed a steep decline. This decline suggests that visual skills may be more labile and could benefit from more regular reinforcement. It may also be that old habits die hard: in an earlier study of baseline CE competency, we observed that most participants listened with their eyes closed or averted, actively tuning out the visual timing reference that would help them answer the question.7

Comparing control and intervention retention scores suggests that the Web‐based training is more effective than traditional training in CE. The interns spent a mean of 5 hours learning, including 3 hours with the hospitalist‐instructor. Of those 3 hours, 30 minutes was spent taking the test. Earlier studies32, 33 showed improved CE skills with 12‐20 hours of instructor‐led tutorials. The shorter instruction time for interns in this study translated into about half the magnitude of improvement that was seen in third‐year medical students, whose mean score improved from 58.7 6.7 to 73.4 3.8. Moreover, the students in the earlier study continued to show improvement in their CE competency without further intervention, whereas the test scores of interns in this study appeared to plateau. It is therefore likely that 5 hours per year is a lower bound for successful Web‐based CE training. For this reason, we do not recommend fewer than 5 hours per year of CE training using a Web‐based curriculum (including self‐study). Ideally, sessions should be scheduled each year in residency in order to form a critical mass of learning that has the potential to continue improvements beyond residency training.

When surveyed, interns in both the intervention and control groups were very interested in improving their skills but were not confident in their abilities. In fact, confidence did not correlate with ability, which suggests interns (and possibly others)34 do not have a reliable, intrinsic sense of their CE skills. Curiously, the control group reported spending almost twice the number of hours learning CE that the intervention group did. It is unlikely that the intervention group received less traditional instruction than the control group. Therefore, it is more likely that in estimating the hours spent learning CE, the intervention group reported only the formal instruction plus self‐study hours spent on cardiac examination. One way to interpret the greater number of hours of instruction reported by the controls is to look at their end‐of‐the‐year estimate of how much instruction they had received in the previous 12 months. The controls did not benefit from formal instruction in cardiac examination, and so their estimate included the hours spent with physician attendings receiving traditional bedside instruction. The number of hours reported by the controls could be accurate or could be an overestimate. If accurate, the greater number of hours for the controls did not translate into superior performance on the CE Test. If an overestimate, hindsight bias may have inflated the actual hours spent.

Some may argue that VPs are a poor substitute for actual patients. With few reservations, we are inclined to agree. The best training for CE is at the bedside with a clinical master and the luxury of time. Today, however, teaching has shifted from the bedside to conference rooms and even corridors.17, 18 Consequently, opportunities to practice hands‐on CE have become rare. Even when physical examination is performed at the bedside, the quality of the instruction on it depends on the teacher's interest, abilities, and available patients. Our intent was to implement a standardized curriculum that ensures that CE is practiced and tested regularly in order to prevent the creation of a generation of practicing physicians who never develop these skills.

A hospitalist, not a cardiologist, led the intervention. Because cardiac conditions are involved in more than 40% of admissions to the medical wards, a hospitalist should be able to evaluate a patient through physical examination in order to determine whether further tests are necessary or whether a cardiologist should be consulted. In academic medicine, most teaching of cardiac examination is no longer done by cardiologists, but rather by internists.31 Hospital medicine attendings often play a central role as attending physicians for medical students and house officers, and the hospitalist can and should play a larger role in bedside teaching. A Web‐based curriculum for residents that can be accessed at all hours is also suited to residents who have inpatient rotations.

Our CE curriculum was designed for the typical conditions of today's residency training programs, in which education must be balanced with the demands of patient care and the constraints of the limits on resident duty hours.16, 24 Studentfaculty contact time was limited to three 1‐hour sessions, reflecting the time constraints of most training programs.13, 16, 19 The program was incorporated into the regularly scheduled cardiology block throughout the academic year, ensuring that residents received broad exposure to cardiac clinical presentations at a time when bedside encounters with important cardiac physical findings were most likely. The goal was not to reduce or eliminate bedside teaching but to make the limited time that remained more productive. The fidelity of the training to actual bedside encounters was high: unlike the lone audio recordings of heart sounds, VPs in this study also simultaneously presented visual pulsation in the neck or precordium, training the resident to use these visual timing cues while auscultating patients. Furthermore, the Web‐based multimedia technology employed allowed direct comparison of a patient's bedside findings with case‐matched laboratory studies, which was enhanced with explanatory animations and tutorials. Working within the constraints of residency training programs, we were able to standardize the cardiac clinical exposure of trainees and to test for improvement in and retention of CE skills.

Several limitations should be considered. Although VPs mimic bedside patient encounters, we do not know if improvements from this Web‐based curriculum translate into improvements in patient care. Ideally, the study would have been conducted at more than 1 site in order to test whether the positive results we found are replicated elsewhere. Mitigating these drawbacks are the real‐world conditions of our study: the teaching hospital that conducted the study was not involved in the Web site development, and an internist‐hospitalist, not a cardiologist, led the instruction. In theory, the baseline testing itself may have boosted the posttraining scores of the intervention group simply by motivating the interns to improve their test scores. However, only an overall score was reported to interns; they were not told which questions they missed or in what subcategory they were most deficient. Although simple exposure to the test may be the reason for the interns' improvement, it has been shown that the test has testretest reliability over a 4‐ to 6‐week period28, 29 and that test scores do not improve with prior exposure to the test. Because those in the intervention group knew that they would be retested, we cannot eliminate the possibility that they were somehow more motivated to do well on the test by the end of the year. Finally, there may have been differences in the ward training received by the 2 groups of interns studied. The test scores of interns in the control group may not have been as high because of differences in patients admitted, topics discussed, or teaching attendings assigned to them during the academic year; however, because there were no radical changes from the previous year in either the patient population or the group of attending physicians, the differences in the quality of patient encounters between the control and intervention groups are thought to be minor.

In conclusion, the Web‐based curriculum met all the requirements for being effective education: it was self‐directed, interactive, relevant, and cost effective. This novel curriculum standardized learning experience during the cardiology block and saved time for both teacher and trainees. Baseline assessment helped to focus learning. Involvement of an instructor‐hospitalist added accountability, a resource for answering questions, and encouragement for building self‐study skills. More importantly, a realistic and reproducible test of CE skills documented whether the learning objectives were meta feature that is becoming increasingly desirable to residency program directors.35 Our study showed that training with virtual patients yields superior mean CE competency scores over those with standard cardiology ward rotations. Further studies may confirm whether this improvement in CE translates to improvement in making appropriate observations and diagnoses of actual patients.

Despite impressive advances in cardiac diagnostic technology, cardiac examination (CE) remains an essential skill for screening for abnormal sounds, for evaluating cardiovascular system function, and for guiding further diagnostic testing.16

In practice, these benefits may be attenuated if CE skills are inadequate. Numerous studies have documented substantial CE deficiencies among physicians at various points in their careers from medical school to practice.79 In 1 study, residents' CE mistakes accounted for one‐third of all physical diagnostic errors.10 When murmurs are detected, physicians will often reflexively order an echocardiogram and refer to a cardiologist, regardless of the cost or indication. As a consequence, echocardiography use is rising faster than the aging population or the incidence of cardiac pathological conditions would explain.11 Because cost‐effective medicine depends on the appropriate application of clinical skills like CE, the loss of these skills is a major shortcoming.1215

The reasons for the decline in physicians' CE skills are numerous. High reliance on ordering diagnostic tests,16 conducting teaching rounds away from the bedside,17, 18 time constraints during residency,16, 19 and declining CE skills of faculty members themselves7 all may contribute to the diminished CE skills of residents. Residents, who themselves identify abnormal heart sounds at alarmingly low rates, play an ever‐increasing role in medical students' instruction,7, 9 exacerbating the problem.

Responding to growing concerns over patient safety and quality of care,16, 20 public and professional organizations have called for renewed emphasis on teaching and evaluating clinical skills.21 For example, the American Board of Internal Medicine has added a physical diagnosis component to its recertification program.22 The Accreditation Council for Graduate Medical Education (ACGME) describes general competencies for residents, including patient care that should include proper physical examination skills.23 Although mandating uniform standards is a welcome change for improving CE competence, the challenge remains for medical school deans and program directors to fit structured physical examination skills training into an already crowded curriculum.16, 24 Moreover, the impact of these efforts to improve CE is uncertain because programs lack an objective measure of CE competence.

The CE training is itself a challenge: sight, sound, and touch all contribute to the clinical impression. For this reason, it is difficult to teach away from the bedside. Unlike pulmonary examination, for which a diagnosis is best made by listening, cardiac auscultation is only one (frequently overemphasized) aspect of CE.25 Medical knowledge of cardiac anatomy and physiology, visualization of cardiovascular findings, and integration of auditory and visual findings are all components of accurate CE.7 Traditionally, CE was taught through direct experience with patients at the bedside under the supervision of seasoned clinicians. However, exposure and learning from good teaching patients has waned. Audiotapes, heart sound simulators, mannequins, and other computer‐based interventions have been used as surrogates, but none has been widely adopted.26, 27 The best practice for teaching CE is not known.

To help to improve CE education during residency, we implemented and evaluated a novel Web‐based CE curriculum that emphasized 4 aspects of CE: cardiovascular anatomy and physiology, auditory skills, visual skills, and integration of auscultatory and visual findings. Our hypothesis was that this new curriculum would improve learning of CE skills, that residents would retain what they learn, and that this curriculum would be better than conventional education in teaching CE skills.

METHODS

Study Participants, Site, and Design

Internal medicine (IM) and family medicine (FM) interns (R1s, n = 59) from university‐ and community‐based residency programs, respectively, participated in this controlled trial of an educational intervention to teach CE.

The intervention group consisted of 26 IM and 8 FM interns at the beginning of the academic year in June 2003. To establish baseline scores, all interns took a 50‐question multimedia test of CE competency described previously.7, 28, 29 Subsequently, all interns completed a required 4‐week cardiology ward rotation. During this rotation, they were instructed to complete a Web‐based CE tutorial with accompanying worksheet and to attend 3 one‐hour sessions with a hospitalist instructor. Their schedules were arranged to allow for this educational time. During the third meeting with the instructor, interns were tested again to establish posttraining scores. Finally, at the end of the academic year, interns were tested once again to establish retention scores.

The control group consisted of 25 first‐year IM residents who were tested at the end of their academic year in June 2003. These test scores served as historical controls for interns who had just completed their first year of residency and who had received standard ward rotation without incorporated Web‐based training in CE. Interns from both groups had many opportunities for one‐on‐one instruction in CE because each intern was assigned for the cardiology rotation to a private practice cardiology attending. Figure 1 outlines the number of IM and FM interns eligible and the number actually tested at each stage of the study.

Figure 1

RESULTS

Research Question 1

Individual baseline and posttraining CE Test scores for the intervention group are plotted in the first 2 columns of Figure 2. The posttraining mean score improved from baseline (66.0 5.2 vs. 54.2 5.4, P = .002). The knowledge, audio, and visual subcategories of CE competence showed similar improvements (P .001; Table 3). The score for the integration of audiovisual skills subcategory was higher at baseline than the other subcategory scores and remained unchanged.

Figure 2

Table 3.Subcategory and Overall CE Competency Scores of Intervention and Control Groups for 4 Research Questions

Question 1. Does the Web‐based curriculum improve CE skills?
Subcategory	Intervention baseline (n = 30)		Intervention end of year (n = 30)
	Mean, % (SD)	95% CI	Mean, % (SD)	95% CI	P value (paired t test)
Question 2. Do interns retain this improvement?
Subcategory	Intervention baseline (n = 18)		Intervention end of year (n = 18)
	Mean, % (SD)	95% CI	Mean, % (SD)	95% CI	P value (paired t test)
Question 3. Does clinical training alone improve CE skills?
Subcategory	Intervention baseline* (n = 34)		Control end of year (n = 25)
	Mean, % (SD)	95% CI	Mean, % (SD)	95% CI	P value (t test)
Question 4. Is the Web‐based curriculum better than clinical training alone?
Subcategory	Intervention end of year (n = 23)		Control end of year (n = 25)
	Mean, % (SD)	95% CI	Mean, % (SD)	95% CI	P value (t test)
* Baseline scores for Web training also served as baseline scores for clinical training alone. CE, cardiac examination; SD, standard deviation; 95% CI, 95% confidence interval for mean. The full text of the questions were: Question 1. Immediately after cardiology rotation, concomitant Web‐based training improved knowledge, audio, and visual skills scores as well as overall CE competency score. Question 2. At the end of the year, interns retained the improvement in knowledge and audio skills and overall CE competency. Question 3. The standard cardiology rotation did not significantly improve either CE subcategory scores or the overall competency score. Question 4. When compared at the end of the year, intervention interns (those who received Web training) showed significantly better knowledge, audio, and overall competency scores.
Knowledge	55.9% (14.7%)	50.4%‐61.4%	68.9% (15.3%)	63.2%‐74.6%	<.001
Audio	69.6% (17.4%)	63.1%‐76.0%	83.6% (10.3%)	79.7%‐87.4 %	<.001
Visual	56.7% (21.4%)	48.7%‐64.7%	71.9% (16.6%)	65.7%‐78.1%	<.001
Integration	77.5% (14.5%)	72.1%‐83.0%	76.8% (12.8%)	72.1%‐81.6%	.85
Overall score	54.2% (14.5%)	48.8%‐59.6%	66.0% (13.6%)	60.9%‐71.2%	.002
Knowledge	58.6% (15.2%)	51.0%‐66.2%	67.6% (16.7%)	60.4%‐74.9%	.05
Audio	67.4% (17.1%)	58.9%‐75.9%	82.3% (10.4%)	77.8%‐86.8%	.004
Visual	51.6% (22.0%)	40.7%‐62.5%	65.2% (20.6%)	56.3%‐74.1%	.13
Integration	76.0 % (15.8%)	68.2%‐83.8%	77.1% (13.2%)	71.4%‐82.8%	.95
Overall score	51.8 % (15.0%)	44.3%‐59.3%	63.5% (14.9%)	56.1%‐70.9%	.02
Knowledge	56.5% (14.6%)	51.4%‐61.6%	57.8% (15.4%)	51.4%‐64.1%	.75
Audio	70.0% (16.9%)	64.1%‐75.9%	73.3% (13.6%)	67.7%‐79.0%	.42
Visual	57.6% (20.9%)	50.3%‐64.9%	66.9% (15.3%)	60.6%‐73.2%	.07
Integration	77.8% (13.8%)	72.9%‐82.5%	76.0% (12.2%)	71.0%‐81.0%	.62
Overall score	54.7% (13.7%)	49.9%‐59.5%	56.8% (12.4%)	51.7%‐61.9%	.54
Knowledge	67.6% (16.7%)	60.4%‐74.8%	57.8% (15.4%)	51.4%‐64.1%	0.04
Audio	82.3% (10.4%)	77.8%‐86.8%	73.3% (13.6%)	67.7%‐79.0%	0.01
Visual	65.2% (20.6%)	56.3%‐74.1%	66.9% (15.3%)	60.6%‐73.2%	0.75
Integration	77.1% (13.2%)	71.4%‐82.8%	76.0% (12.2%)	71.0%‐81.0%	0.76
Overall score	64.7% (14.5%)	58.4%‐71.0%	56.8% (12.4%)	51.7%‐61.9%	0.05

DISCUSSION

To improve CE competence of residents, we designed and implemented a Web‐based CE curriculum using virtual patients (VPs), which standardized the interns' exposure to a spectrum of medical conditions and allowed them flexibility in choosing when and where the training occurred.30 In this controlled educational intervention, we found significant improvements in interns' mean CE competency scores immediately following the Web‐based training; these improvements were retained at the end of the academic year. The Web‐based curriculum also improved scores in all 4 subcategories except integration of audio and visual skills. Although cardiac physiology knowledge and audio skills were retained, visual skills showed a steep decline. This decline suggests that visual skills may be more labile and could benefit from more regular reinforcement. It may also be that old habits die hard: in an earlier study of baseline CE competency, we observed that most participants listened with their eyes closed or averted, actively tuning out the visual timing reference that would help them answer the question.7

Comparing control and intervention retention scores suggests that the Web‐based training is more effective than traditional training in CE. The interns spent a mean of 5 hours learning, including 3 hours with the hospitalist‐instructor. Of those 3 hours, 30 minutes was spent taking the test. Earlier studies32, 33 showed improved CE skills with 12‐20 hours of instructor‐led tutorials. The shorter instruction time for interns in this study translated into about half the magnitude of improvement that was seen in third‐year medical students, whose mean score improved from 58.7 6.7 to 73.4 3.8. Moreover, the students in the earlier study continued to show improvement in their CE competency without further intervention, whereas the test scores of interns in this study appeared to plateau. It is therefore likely that 5 hours per year is a lower bound for successful Web‐based CE training. For this reason, we do not recommend fewer than 5 hours per year of CE training using a Web‐based curriculum (including self‐study). Ideally, sessions should be scheduled each year in residency in order to form a critical mass of learning that has the potential to continue improvements beyond residency training.

When surveyed, interns in both the intervention and control groups were very interested in improving their skills but were not confident in their abilities. In fact, confidence did not correlate with ability, which suggests interns (and possibly others)34 do not have a reliable, intrinsic sense of their CE skills. Curiously, the control group reported spending almost twice the number of hours learning CE that the intervention group did. It is unlikely that the intervention group received less traditional instruction than the control group. Therefore, it is more likely that in estimating the hours spent learning CE, the intervention group reported only the formal instruction plus self‐study hours spent on cardiac examination. One way to interpret the greater number of hours of instruction reported by the controls is to look at their end‐of‐the‐year estimate of how much instruction they had received in the previous 12 months. The controls did not benefit from formal instruction in cardiac examination, and so their estimate included the hours spent with physician attendings receiving traditional bedside instruction. The number of hours reported by the controls could be accurate or could be an overestimate. If accurate, the greater number of hours for the controls did not translate into superior performance on the CE Test. If an overestimate, hindsight bias may have inflated the actual hours spent.

Some may argue that VPs are a poor substitute for actual patients. With few reservations, we are inclined to agree. The best training for CE is at the bedside with a clinical master and the luxury of time. Today, however, teaching has shifted from the bedside to conference rooms and even corridors.17, 18 Consequently, opportunities to practice hands‐on CE have become rare. Even when physical examination is performed at the bedside, the quality of the instruction on it depends on the teacher's interest, abilities, and available patients. Our intent was to implement a standardized curriculum that ensures that CE is practiced and tested regularly in order to prevent the creation of a generation of practicing physicians who never develop these skills.

A hospitalist, not a cardiologist, led the intervention. Because cardiac conditions are involved in more than 40% of admissions to the medical wards, a hospitalist should be able to evaluate a patient through physical examination in order to determine whether further tests are necessary or whether a cardiologist should be consulted. In academic medicine, most teaching of cardiac examination is no longer done by cardiologists, but rather by internists.31 Hospital medicine attendings often play a central role as attending physicians for medical students and house officers, and the hospitalist can and should play a larger role in bedside teaching. A Web‐based curriculum for residents that can be accessed at all hours is also suited to residents who have inpatient rotations.

Our CE curriculum was designed for the typical conditions of today's residency training programs, in which education must be balanced with the demands of patient care and the constraints of the limits on resident duty hours.16, 24 Studentfaculty contact time was limited to three 1‐hour sessions, reflecting the time constraints of most training programs.13, 16, 19 The program was incorporated into the regularly scheduled cardiology block throughout the academic year, ensuring that residents received broad exposure to cardiac clinical presentations at a time when bedside encounters with important cardiac physical findings were most likely. The goal was not to reduce or eliminate bedside teaching but to make the limited time that remained more productive. The fidelity of the training to actual bedside encounters was high: unlike the lone audio recordings of heart sounds, VPs in this study also simultaneously presented visual pulsation in the neck or precordium, training the resident to use these visual timing cues while auscultating patients. Furthermore, the Web‐based multimedia technology employed allowed direct comparison of a patient's bedside findings with case‐matched laboratory studies, which was enhanced with explanatory animations and tutorials. Working within the constraints of residency training programs, we were able to standardize the cardiac clinical exposure of trainees and to test for improvement in and retention of CE skills.

Several limitations should be considered. Although VPs mimic bedside patient encounters, we do not know if improvements from this Web‐based curriculum translate into improvements in patient care. Ideally, the study would have been conducted at more than 1 site in order to test whether the positive results we found are replicated elsewhere. Mitigating these drawbacks are the real‐world conditions of our study: the teaching hospital that conducted the study was not involved in the Web site development, and an internist‐hospitalist, not a cardiologist, led the instruction. In theory, the baseline testing itself may have boosted the posttraining scores of the intervention group simply by motivating the interns to improve their test scores. However, only an overall score was reported to interns; they were not told which questions they missed or in what subcategory they were most deficient. Although simple exposure to the test may be the reason for the interns' improvement, it has been shown that the test has testretest reliability over a 4‐ to 6‐week period28, 29 and that test scores do not improve with prior exposure to the test. Because those in the intervention group knew that they would be retested, we cannot eliminate the possibility that they were somehow more motivated to do well on the test by the end of the year. Finally, there may have been differences in the ward training received by the 2 groups of interns studied. The test scores of interns in the control group may not have been as high because of differences in patients admitted, topics discussed, or teaching attendings assigned to them during the academic year; however, because there were no radical changes from the previous year in either the patient population or the group of attending physicians, the differences in the quality of patient encounters between the control and intervention groups are thought to be minor.

In conclusion, the Web‐based curriculum met all the requirements for being effective education: it was self‐directed, interactive, relevant, and cost effective. This novel curriculum standardized learning experience during the cardiology block and saved time for both teacher and trainees. Baseline assessment helped to focus learning. Involvement of an instructor‐hospitalist added accountability, a resource for answering questions, and encouragement for building self‐study skills. More importantly, a realistic and reproducible test of CE skills documented whether the learning objectives were meta feature that is becoming increasingly desirable to residency program directors.35 Our study showed that training with virtual patients yields superior mean CE competency scores over those with standard cardiology ward rotations. Further studies may confirm whether this improvement in CE translates to improvement in making appropriate observations and diagnoses of actual patients.

Despite impressive advances in cardiac diagnostic technology, cardiac examination (CE) remains an essential skill for screening for abnormal sounds, for evaluating cardiovascular system function, and for guiding further diagnostic testing.16

In practice, these benefits may be attenuated if CE skills are inadequate. Numerous studies have documented substantial CE deficiencies among physicians at various points in their careers from medical school to practice.79 In 1 study, residents' CE mistakes accounted for one‐third of all physical diagnostic errors.10 When murmurs are detected, physicians will often reflexively order an echocardiogram and refer to a cardiologist, regardless of the cost or indication. As a consequence, echocardiography use is rising faster than the aging population or the incidence of cardiac pathological conditions would explain.11 Because cost‐effective medicine depends on the appropriate application of clinical skills like CE, the loss of these skills is a major shortcoming.1215

The reasons for the decline in physicians' CE skills are numerous. High reliance on ordering diagnostic tests,16 conducting teaching rounds away from the bedside,17, 18 time constraints during residency,16, 19 and declining CE skills of faculty members themselves7 all may contribute to the diminished CE skills of residents. Residents, who themselves identify abnormal heart sounds at alarmingly low rates, play an ever‐increasing role in medical students' instruction,7, 9 exacerbating the problem.

Responding to growing concerns over patient safety and quality of care,16, 20 public and professional organizations have called for renewed emphasis on teaching and evaluating clinical skills.21 For example, the American Board of Internal Medicine has added a physical diagnosis component to its recertification program.22 The Accreditation Council for Graduate Medical Education (ACGME) describes general competencies for residents, including patient care that should include proper physical examination skills.23 Although mandating uniform standards is a welcome change for improving CE competence, the challenge remains for medical school deans and program directors to fit structured physical examination skills training into an already crowded curriculum.16, 24 Moreover, the impact of these efforts to improve CE is uncertain because programs lack an objective measure of CE competence.

The CE training is itself a challenge: sight, sound, and touch all contribute to the clinical impression. For this reason, it is difficult to teach away from the bedside. Unlike pulmonary examination, for which a diagnosis is best made by listening, cardiac auscultation is only one (frequently overemphasized) aspect of CE.25 Medical knowledge of cardiac anatomy and physiology, visualization of cardiovascular findings, and integration of auditory and visual findings are all components of accurate CE.7 Traditionally, CE was taught through direct experience with patients at the bedside under the supervision of seasoned clinicians. However, exposure and learning from good teaching patients has waned. Audiotapes, heart sound simulators, mannequins, and other computer‐based interventions have been used as surrogates, but none has been widely adopted.26, 27 The best practice for teaching CE is not known.

To help to improve CE education during residency, we implemented and evaluated a novel Web‐based CE curriculum that emphasized 4 aspects of CE: cardiovascular anatomy and physiology, auditory skills, visual skills, and integration of auscultatory and visual findings. Our hypothesis was that this new curriculum would improve learning of CE skills, that residents would retain what they learn, and that this curriculum would be better than conventional education in teaching CE skills.

METHODS

Study Participants, Site, and Design

Internal medicine (IM) and family medicine (FM) interns (R1s, n = 59) from university‐ and community‐based residency programs, respectively, participated in this controlled trial of an educational intervention to teach CE.

The intervention group consisted of 26 IM and 8 FM interns at the beginning of the academic year in June 2003. To establish baseline scores, all interns took a 50‐question multimedia test of CE competency described previously.7, 28, 29 Subsequently, all interns completed a required 4‐week cardiology ward rotation. During this rotation, they were instructed to complete a Web‐based CE tutorial with accompanying worksheet and to attend 3 one‐hour sessions with a hospitalist instructor. Their schedules were arranged to allow for this educational time. During the third meeting with the instructor, interns were tested again to establish posttraining scores. Finally, at the end of the academic year, interns were tested once again to establish retention scores.

The control group consisted of 25 first‐year IM residents who were tested at the end of their academic year in June 2003. These test scores served as historical controls for interns who had just completed their first year of residency and who had received standard ward rotation without incorporated Web‐based training in CE. Interns from both groups had many opportunities for one‐on‐one instruction in CE because each intern was assigned for the cardiology rotation to a private practice cardiology attending. Figure 1 outlines the number of IM and FM interns eligible and the number actually tested at each stage of the study.

Figure 1

RESULTS

Research Question 1

Individual baseline and posttraining CE Test scores for the intervention group are plotted in the first 2 columns of Figure 2. The posttraining mean score improved from baseline (66.0 5.2 vs. 54.2 5.4, P = .002). The knowledge, audio, and visual subcategories of CE competence showed similar improvements (P .001; Table 3). The score for the integration of audiovisual skills subcategory was higher at baseline than the other subcategory scores and remained unchanged.

Figure 2

Table 3.Subcategory and Overall CE Competency Scores of Intervention and Control Groups for 4 Research Questions

Question 1. Does the Web‐based curriculum improve CE skills?
Subcategory	Intervention baseline (n = 30)		Intervention end of year (n = 30)
	Mean, % (SD)	95% CI	Mean, % (SD)	95% CI	P value (paired t test)
Question 2. Do interns retain this improvement?
Subcategory	Intervention baseline (n = 18)		Intervention end of year (n = 18)
	Mean, % (SD)	95% CI	Mean, % (SD)	95% CI	P value (paired t test)
Question 3. Does clinical training alone improve CE skills?
Subcategory	Intervention baseline* (n = 34)		Control end of year (n = 25)
	Mean, % (SD)	95% CI	Mean, % (SD)	95% CI	P value (t test)
Question 4. Is the Web‐based curriculum better than clinical training alone?
Subcategory	Intervention end of year (n = 23)		Control end of year (n = 25)
	Mean, % (SD)	95% CI	Mean, % (SD)	95% CI	P value (t test)
* Baseline scores for Web training also served as baseline scores for clinical training alone. CE, cardiac examination; SD, standard deviation; 95% CI, 95% confidence interval for mean. The full text of the questions were: Question 1. Immediately after cardiology rotation, concomitant Web‐based training improved knowledge, audio, and visual skills scores as well as overall CE competency score. Question 2. At the end of the year, interns retained the improvement in knowledge and audio skills and overall CE competency. Question 3. The standard cardiology rotation did not significantly improve either CE subcategory scores or the overall competency score. Question 4. When compared at the end of the year, intervention interns (those who received Web training) showed significantly better knowledge, audio, and overall competency scores.
Knowledge	55.9% (14.7%)	50.4%‐61.4%	68.9% (15.3%)	63.2%‐74.6%	<.001
Audio	69.6% (17.4%)	63.1%‐76.0%	83.6% (10.3%)	79.7%‐87.4 %	<.001
Visual	56.7% (21.4%)	48.7%‐64.7%	71.9% (16.6%)	65.7%‐78.1%	<.001
Integration	77.5% (14.5%)	72.1%‐83.0%	76.8% (12.8%)	72.1%‐81.6%	.85
Overall score	54.2% (14.5%)	48.8%‐59.6%	66.0% (13.6%)	60.9%‐71.2%	.002
Knowledge	58.6% (15.2%)	51.0%‐66.2%	67.6% (16.7%)	60.4%‐74.9%	.05
Audio	67.4% (17.1%)	58.9%‐75.9%	82.3% (10.4%)	77.8%‐86.8%	.004
Visual	51.6% (22.0%)	40.7%‐62.5%	65.2% (20.6%)	56.3%‐74.1%	.13
Integration	76.0 % (15.8%)	68.2%‐83.8%	77.1% (13.2%)	71.4%‐82.8%	.95
Overall score	51.8 % (15.0%)	44.3%‐59.3%	63.5% (14.9%)	56.1%‐70.9%	.02
Knowledge	56.5% (14.6%)	51.4%‐61.6%	57.8% (15.4%)	51.4%‐64.1%	.75
Audio	70.0% (16.9%)	64.1%‐75.9%	73.3% (13.6%)	67.7%‐79.0%	.42
Visual	57.6% (20.9%)	50.3%‐64.9%	66.9% (15.3%)	60.6%‐73.2%	.07
Integration	77.8% (13.8%)	72.9%‐82.5%	76.0% (12.2%)	71.0%‐81.0%	.62
Overall score	54.7% (13.7%)	49.9%‐59.5%	56.8% (12.4%)	51.7%‐61.9%	.54
Knowledge	67.6% (16.7%)	60.4%‐74.8%	57.8% (15.4%)	51.4%‐64.1%	0.04
Audio	82.3% (10.4%)	77.8%‐86.8%	73.3% (13.6%)	67.7%‐79.0%	0.01
Visual	65.2% (20.6%)	56.3%‐74.1%	66.9% (15.3%)	60.6%‐73.2%	0.75
Integration	77.1% (13.2%)	71.4%‐82.8%	76.0% (12.2%)	71.0%‐81.0%	0.76
Overall score	64.7% (14.5%)	58.4%‐71.0%	56.8% (12.4%)	51.7%‐61.9%	0.05

DISCUSSION

To improve CE competence of residents, we designed and implemented a Web‐based CE curriculum using virtual patients (VPs), which standardized the interns' exposure to a spectrum of medical conditions and allowed them flexibility in choosing when and where the training occurred.30 In this controlled educational intervention, we found significant improvements in interns' mean CE competency scores immediately following the Web‐based training; these improvements were retained at the end of the academic year. The Web‐based curriculum also improved scores in all 4 subcategories except integration of audio and visual skills. Although cardiac physiology knowledge and audio skills were retained, visual skills showed a steep decline. This decline suggests that visual skills may be more labile and could benefit from more regular reinforcement. It may also be that old habits die hard: in an earlier study of baseline CE competency, we observed that most participants listened with their eyes closed or averted, actively tuning out the visual timing reference that would help them answer the question.7

Comparing control and intervention retention scores suggests that the Web‐based training is more effective than traditional training in CE. The interns spent a mean of 5 hours learning, including 3 hours with the hospitalist‐instructor. Of those 3 hours, 30 minutes was spent taking the test. Earlier studies32, 33 showed improved CE skills with 12‐20 hours of instructor‐led tutorials. The shorter instruction time for interns in this study translated into about half the magnitude of improvement that was seen in third‐year medical students, whose mean score improved from 58.7 6.7 to 73.4 3.8. Moreover, the students in the earlier study continued to show improvement in their CE competency without further intervention, whereas the test scores of interns in this study appeared to plateau. It is therefore likely that 5 hours per year is a lower bound for successful Web‐based CE training. For this reason, we do not recommend fewer than 5 hours per year of CE training using a Web‐based curriculum (including self‐study). Ideally, sessions should be scheduled each year in residency in order to form a critical mass of learning that has the potential to continue improvements beyond residency training.

When surveyed, interns in both the intervention and control groups were very interested in improving their skills but were not confident in their abilities. In fact, confidence did not correlate with ability, which suggests interns (and possibly others)34 do not have a reliable, intrinsic sense of their CE skills. Curiously, the control group reported spending almost twice the number of hours learning CE that the intervention group did. It is unlikely that the intervention group received less traditional instruction than the control group. Therefore, it is more likely that in estimating the hours spent learning CE, the intervention group reported only the formal instruction plus self‐study hours spent on cardiac examination. One way to interpret the greater number of hours of instruction reported by the controls is to look at their end‐of‐the‐year estimate of how much instruction they had received in the previous 12 months. The controls did not benefit from formal instruction in cardiac examination, and so their estimate included the hours spent with physician attendings receiving traditional bedside instruction. The number of hours reported by the controls could be accurate or could be an overestimate. If accurate, the greater number of hours for the controls did not translate into superior performance on the CE Test. If an overestimate, hindsight bias may have inflated the actual hours spent.

Some may argue that VPs are a poor substitute for actual patients. With few reservations, we are inclined to agree. The best training for CE is at the bedside with a clinical master and the luxury of time. Today, however, teaching has shifted from the bedside to conference rooms and even corridors.17, 18 Consequently, opportunities to practice hands‐on CE have become rare. Even when physical examination is performed at the bedside, the quality of the instruction on it depends on the teacher's interest, abilities, and available patients. Our intent was to implement a standardized curriculum that ensures that CE is practiced and tested regularly in order to prevent the creation of a generation of practicing physicians who never develop these skills.

A hospitalist, not a cardiologist, led the intervention. Because cardiac conditions are involved in more than 40% of admissions to the medical wards, a hospitalist should be able to evaluate a patient through physical examination in order to determine whether further tests are necessary or whether a cardiologist should be consulted. In academic medicine, most teaching of cardiac examination is no longer done by cardiologists, but rather by internists.31 Hospital medicine attendings often play a central role as attending physicians for medical students and house officers, and the hospitalist can and should play a larger role in bedside teaching. A Web‐based curriculum for residents that can be accessed at all hours is also suited to residents who have inpatient rotations.

Our CE curriculum was designed for the typical conditions of today's residency training programs, in which education must be balanced with the demands of patient care and the constraints of the limits on resident duty hours.16, 24 Studentfaculty contact time was limited to three 1‐hour sessions, reflecting the time constraints of most training programs.13, 16, 19 The program was incorporated into the regularly scheduled cardiology block throughout the academic year, ensuring that residents received broad exposure to cardiac clinical presentations at a time when bedside encounters with important cardiac physical findings were most likely. The goal was not to reduce or eliminate bedside teaching but to make the limited time that remained more productive. The fidelity of the training to actual bedside encounters was high: unlike the lone audio recordings of heart sounds, VPs in this study also simultaneously presented visual pulsation in the neck or precordium, training the resident to use these visual timing cues while auscultating patients. Furthermore, the Web‐based multimedia technology employed allowed direct comparison of a patient's bedside findings with case‐matched laboratory studies, which was enhanced with explanatory animations and tutorials. Working within the constraints of residency training programs, we were able to standardize the cardiac clinical exposure of trainees and to test for improvement in and retention of CE skills.

Several limitations should be considered. Although VPs mimic bedside patient encounters, we do not know if improvements from this Web‐based curriculum translate into improvements in patient care. Ideally, the study would have been conducted at more than 1 site in order to test whether the positive results we found are replicated elsewhere. Mitigating these drawbacks are the real‐world conditions of our study: the teaching hospital that conducted the study was not involved in the Web site development, and an internist‐hospitalist, not a cardiologist, led the instruction. In theory, the baseline testing itself may have boosted the posttraining scores of the intervention group simply by motivating the interns to improve their test scores. However, only an overall score was reported to interns; they were not told which questions they missed or in what subcategory they were most deficient. Although simple exposure to the test may be the reason for the interns' improvement, it has been shown that the test has testretest reliability over a 4‐ to 6‐week period28, 29 and that test scores do not improve with prior exposure to the test. Because those in the intervention group knew that they would be retested, we cannot eliminate the possibility that they were somehow more motivated to do well on the test by the end of the year. Finally, there may have been differences in the ward training received by the 2 groups of interns studied. The test scores of interns in the control group may not have been as high because of differences in patients admitted, topics discussed, or teaching attendings assigned to them during the academic year; however, because there were no radical changes from the previous year in either the patient population or the group of attending physicians, the differences in the quality of patient encounters between the control and intervention groups are thought to be minor.

In conclusion, the Web‐based curriculum met all the requirements for being effective education: it was self‐directed, interactive, relevant, and cost effective. This novel curriculum standardized learning experience during the cardiology block and saved time for both teacher and trainees. Baseline assessment helped to focus learning. Involvement of an instructor‐hospitalist added accountability, a resource for answering questions, and encouragement for building self‐study skills. More importantly, a realistic and reproducible test of CE skills documented whether the learning objectives were meta feature that is becoming increasingly desirable to residency program directors.35 Our study showed that training with virtual patients yields superior mean CE competency scores over those with standard cardiology ward rotations. Further studies may confirm whether this improvement in CE translates to improvement in making appropriate observations and diagnoses of actual patients.