Clinical Findings Associated with Radiographic Pneumonia in Nursing Home Residents

Pneumonia is a leading cause of morbidity, mortality, and hospitalization of nursing home residents.^1-8 Atypical presentations and fewer presenting signs and symptoms in older patients complicate diagnosis.^9,10 Also, clinician (physician, nurse practitioner, and physician assistant) visits to nursing homes are often sporadic, and radiology facilities are rarely on the premises. As a consequence, residents are commonly sent to emergency departments for evaluation,^4,11,12 which undoubtedly contributes to a high hospitalization rate.

Clinicians who periodically see nursing home residents could benefit from a simple clinical tool to identify pneumonia. No large studies of community nursing home residents have systematically studied findings associated with pneumonia. As part of the Missouri LRI Project, we examined how well clinical findings predict radiographic pneumonia.

Methods

The Missouri LRI Project was a prospective observational study in 36 nursing homes in Central Missouri and St. Louis designed to investigate predictors of 2 outcomes of lower respiratory infections (LRIs): mortality and functional decline. Potential cases were identified from August 15, 1995, through September 29, 1998; however, all facilities were not involved until fall 1997. Study facilities were similar in size, ownership, and occupancy to national estimates from the 1995 National Nursing Home Survey (data available on request).¹³

We trained nursing home staff to report ill residents with any of 6 respiratory symptoms (eg, cough, dyspnea, sputum production) or 6 general symptoms (eg, fever, decline in mobility, mental status changes). Project nurses called and visited facilities frequently to reinforce reporting. Under a physician-authorized protocol, ill residents with a possible LRI received a standardized evaluation by a trained project nurse and usually a chest radiograph, complete blood count, and a chemistry panel. Complete criteria for triggering an evaluation are listed in Table 1. For this paper, we were concerned with the 90% of evaluated residents who received a chest radiograph. Criteria for excluding residents from evaluation are summarized in the Figure 1.

The nurse evaluation included an inventory of current symptoms, a review of important chronic conditions (eg, congestive heart failure), and a targeted physical examination. The examination included vital signs and the following body areas or systems: ears, nose, and throat; cardiac; abdominal; neurologic; extremities; skin; and a detailed lung examination. Most project nurses had advanced practice training; the remainder had extensive clinical experience and training in physical assessment. All received an individualized training session with a project geriatrician. Project nurses had substantially more experience than the nursing home staff, who usually report clinical findings to physicians.

Results of the evaluation were reported to the attending physician, who made all treatment decisions. Since the evaluations were clinically appropriate care authorized by individual attending physicians, the institutional review boards that reviewed the project allowed us to substantially simplify the consent process to a simple acceptance or refusal of the evaluation. In 9.2% of evaluations the resident was transferred to the hospital before project nurses could complete a physical assessment. In these instances, we obtained vital sign and clinical examination data from hospital records.

Radiographic Classification

Since all subjects had at least one illness symptom, for this analysis we classified the presence or absence of pneumonia on the basis of reported radiographic findings. Using defined criteria, 2 clinicians independently separated radiology reports into 3 categories: (a) negative, (b) possible, or (c) probable or definite for pneumonia (hereafter, probable pneumonia). For example, a report describing “new left lower lobe infiltrate suggestive of pneumonia” would have been rated as probable, while a report indicating “possible infiltrate” or “infiltrate suggestive of pneumonia or congestive heart failure” would have been rated as possible. As radiologists rarely provide completely unequivocal readings, we did not separate probable and definite pneumonia. In St. Louis 2 clinicians evaluated the reports, and in central Missouri 2 of 4 clinicians considered each report. Where there was disagreement, all 6 raters from the 2 sites independently reviewed the reports and then attempted to reach consensus. For 13% of radiographs, the project radiologist independently interpreted the actual films. This occurred when: (1) consensus could not be achieved; or (2) consensus was possible pneumonia, but probable pneumonia was needed to quality the episode as an LRI under the project definition.

Statistical Analyses

As residents could be included more than once, the unit of analysis throughout is episode of illness. In our major analysis, we developed a multivariable logistic model to estimate the probability of radiographic pneumonia (possible or probable). Before beginning modeling, we imputed mean values for missing continuous data and the largest category for missing dichotomous variables (the number of missing values is noted in Table 2). Data imputation is less biased than dropping cases in developing multivariable models.¹⁴

Illness episodes were then randomly assigned to a two thirds model-development and a one third model-validation sample. On the basis of the literature and clinical experience, we defined categories of variables that might relate to the presence or absence of pneumonia, such as lung findings (eg, crackles, wheezes), respiratory symptoms (eg, cough, sputum production), vital signs, findings of delirium (eg, acute confusion, decreased alertness), and laboratory findings. Restricting our focus to the development sample, we selected the best representatives of these groups on clinical and statistical grounds. For continuous variables, we considered the shape of the relationship to presence of pneumonia. For example, both very high and very low pulse rates predicted increased risk of pneumonia. In such cases, we considered several different ways to represent the variable in the model. We also limited the range of some variables to avoid undue influence of outliers (approximately the 1% most extreme values). For example, pulse rate above 140 was set equal to 140.

We then employed forward and backward stepwise logistic regression with possible or probable pneumonia (also referred to as positive x-ray results) as the dependent variable. For final model inclusion, we required variables to bear a plausible relationship to the diagnosis of pneumonia and meet a statistical significance criterion (a=.05).

To obtain final estimates of the relationship of each model variable to pneumonia probability, we considered adjustments for 2 kinds of correlation within our data: (1) individuals are nested within facilities, and (2) subjects could be represented by more than one episode.¹⁵ Using generalized estimating equations (GEE) in Proc Genmod in SAS software (SAS Institute, Cary, NC),¹⁶ we noted that the effect of facilities was minor, but the effect of repeat episodes by the same subject was more marked. Consequently, we used GEE to account for repeat episodes on subjects. To avoid unstable GEE estimates, we dropped 5 episodes in the development sample and 8 in the overall sample (episodes beyond the 5th and 6th per individual, respectively).

Using parameter estimates from the development sample, we tested the model’s discrimination and calibration in the validation sample.¹⁷ To assess discrimination, we used the c-statistic, which evaluates among all possible pairs of individuals whether those with higher predicted risk are more likely to die. The c-statistic is also equal to the area under the receiver operating characteristic curve. To assess calibration—agreement between observed and predicted mortality over the range of predicted risk—we used the Hosmer-Lemeshow goodness-of-fit statistic.¹⁸ We then used estimates fitted to the overall sample to develop a simple additive score to provide a clinically usable prediction rule. Statistical analyses were performed with SAS statistical software.¹⁶

Results

Project nurses performed 2592 evaluations. In 90% (2337), residents received chest x-rays either in the nursing home or on hospital transfer. In 3 additional cases crucial information was missing from nursing home records. This left for final analysis 2334 episodes in 1474 individuals Figure 1.

Fifty-five percent of radiographs were interpreted as negative, 12% showed possible pneumonia, and 33% showed probable pneumonia. Most nursing home residents with pneumonia had few presenting symptoms; 80% had 3 or fewer respiratory or general symptoms. However, only 7.5% of subjects evaluated had no respiratory symptoms. Table 2 shows the relationship of selected variables to radiographic findings of absent, possible, or probable pneumonia. Though a few signs and symptoms are more common in those with positive (possible or probable pneumonia) than negative chest x-ray results, most did not discriminate at all. Fever (temperature Ž38°C) was present in 44.4% of positives but only 28.5% of negatives (P=.001).

Multivariable Analysis and Prediction Score

Our GEE model to predict radiographic pneumonia includes 3 vital sign abnormalities (fever, rapid pulse, and rapid respiratory rate), 2 lung findings (presence of crackles and absence of wheezes), 2 potential indicators of delirium (somnolence or decreased alertness and acute confusion), and elevated white blood count. Table 3 reports GEE estimates for the entire sample. Though only exhibiting fair overall performance, the model did well at distinguishing subjects with a high probability of pneumonia. In the 20% of subjects with the highest predicted risks, more than two thirds had pneumonia.

For the full range of values, the model derived on the development sample showed a c-statistic of 0.672, which reduced to 0.632 in the validation sample. A value of 1.0 would indicate perfect discrimination between those who did and did not have radiographic pneumonia, while a value of 0.5 would indicate no better than chance discrimination. Model calibration was not acceptable in the validation sample (Hosmer-Lemeshow goodness-of-fit statistic, P=.008). Inspection suggested the disagreement between predicted and observed probability of pneumonia was primarily with lower-risk estimates.

Because the model performed relatively well at distinguishing subjects very likely to have pneumonia, we created a simple point system aimed at identifying such high-risk individuals. Table 4 shows the scoring system. For 33% of subjects (score Ž3), there was a 56% or higher probability of radiographic pneumonia. An additional 24% of subjects (score of 2) had 44% probability of radiographic pneumonia. However, even those with the lowest scores (-1 to 0, 15% of subjects) still had a 24% probability of pneumonia. The relationship between the score and the probability of radiographic evidence of pneumonia is shown in Figure W1.*

Discussion

In a large community-based sample, we considered presenting symptoms, signs, and laboratory findings associated with radiographic pneumonia. Individual findings discriminated poorly, and we could not separate out a very-low-risk group. However, our simple scoring system identified approximately one third to slightly more than one half with high probability of pneumonia—individuals who might be treated without a confirmatory chest x-ray. If our data are confirmed, they suggest a simple clinical strategy in patients with respiratory or general symptoms Table 1 that might suggest pneumonia: (1) if there are no respiratory symptoms, consider other conditions, such as a urinary tract infection, that might fully explain the symptoms; (2) obtain information to apply our symptom score Table 4; (3) for those with scores of 2 or higher (some might choose 3 instead), treat for pneumonia; (4) for those with scores of -1, 0, or 1, obtain a chest radiograph as a guide to treatment.

Considering individual findings, fever was significantly more common in pneumonia, but only 43% of those with possible or probable pneumonia had a temperature of at least 38°C. This reaffirms common wisdom and previous findings that fever is frequently absent in elderly people with pneumonia.^9,19 We also confirmed that few signs or symptoms are the norm for nursing home-acquired pneumonia.

Chest examination findings also do not adequately distinguish patients with and without pneumonia Table 2. Also, even expert physicians frequently differ on lung examination findings.²⁰ Nonetheless, presence of crackles and absence of wheezing contribute to our scoring system. Both findings are seen with multiple conditions, but in our data crackles are slightly more associated with pneumonia, while wheezing is more strongly associated with other diseases.

The other components of our scoring system are clinical factors commonly associated with pneumonia. Though none individually discriminates well between those with and without pneumonia Table 2, several combined serve to identify a high-risk group.

Four previous studies from emergency department or outpatient settings developed clinical prediction rules to identify pneumonia.^21-24 Criteria for identifying subjects varied substantially, and each rule has limited accuracy in predicting radiographic pneumonia.²⁰ We had adequate data to evaluate 3 of the rules.^21-23 As is usually the case when transporting a prediction rule to a new sample, none performed any better than our rule (data not shown). Our sample created the very difficult challenge for any prediction rule of a very high overall prevalence of pneumonia (45%). That made it unlikely that we could identify a low-risk group in whom x-ray studies could be readily forgone, but we were able to identify a highrisk group.

Limitations

Our findings are subject to several limitations. All facilities in our study were located in central or eastern Missouri, and not all physicians or eligible residents in those facilities participated. Compared with national data, we studied an unusually representative sample of nursing home residents from 36 facilities, including rural and urban locations. Also, in episodes excluded because of physician nonparticipation, residents were very similar to included residents in age, vital signs, and presenting symptoms (data available on request). More important, we lack an independent validation sample from a different cohort. Clinical prediction rules usually do not perform as well in independent samples. This is exemplified by the poor performance of the 3 rules we considered from other settings. Overall, our logistic model was only modest in discriminating and was not well calibrated for low-risk episodes in our reserved validation sample. Although we have developed a promising scoring system to identify residents with high probability of radiographic pneumonia, it needs to be validated in other samples of nursing home residents to determine its ultimate usefulness. For all these reasons, our results may not generalize.

Also, although we identified residents prospectively, project nurses were unable to evaluate 9.2% of residents before transfer to a hospital. Clinical findings abstracted from medical records, such as lung findings, may not have been complete. It is also possible that project nurses could have missed some important findings. However, our staff provided a higher level of expertise than is typically available in nursing homes. In fact, this may limit application of our findings. Nursing home staff vary widely in their ability to accurately examine residents or even identify illness. In many instances, facility staff had not obtained vital signs at the point when we identified a resident as ill enough to qualify for an evaluation.²⁵ Therefore, in many nursing homes, physicians may lack confidence to apply our rule without an evaluation by a physician, advanced practice nurse, or physician assistant.

Finally, determining whether subjects had pneumonia primarily depended on our classification of radiographic reports. Though radiographs generally included 2 views, many were portable films of variable quality, and frequently there was no previous radiograph for comparison. In some subjects with pneumonia, radiographic infiltrates might not yet have developed. Also, even under ideal conditions, radiologists commonly disagree on the presence of pneumonia.²⁶ Some subjects may have been misclassified. However, unless radiographic technique or interpretation was specifically related to clinical predictors, misclassification would simply diminish the relationship of predictors to pneumonia rather than creating a bias. We reviewed reports rather than radiographs, because that is the information usually available to clinicians faced with diagnosis and treatment decisions. We also paid special attention to avoiding any bias in the interpretations. All data were recorded before interpreting radiology reports and the interpretations were performed independent of clinical data. We also made special efforts to assure consistency in labeling radiology reports as possible, probable, or negative for pneumonia. When lack of agreement persisted, the study radiologist reinterpreted the actual films.

Conclusions

Most nursing home residents with pneumonia have few symptoms. We created a simple scoring to identify nursing home residents who have a high probability of radiographic pneumonia. If our results are confirmed, physicians might consider initiating treatment without an x-ray in such residents. Low scores do not rule out pneumonia, and most physicians would want to press for further diagnosis or treatment in this group.

Acknowledgments

This study was supported by the Agency for Healthcare Research and Quality (grant HS08551) and Dr Mehr’s Robert Wood Johnson Foundation Generalist Physician Faculty Scholars award. Dr Kruse was partially supported by an Institutional National Research Service Award (PE10038) from the Health Resources and Services Administration. Our project would not have been possible without the support of the many attending physicians, administrators, and staff of the involved nursing homes. Dr Clive Levine re-read more than 200 radiographs; Karen Davenport provided crucial administrative support; and Karen Madrone, MPA, assisted with manuscript preparation. Many other unnamed project staff also contributed.

Pneumonia is a leading cause of morbidity, mortality, and hospitalization of nursing home residents.^1-8 Atypical presentations and fewer presenting signs and symptoms in older patients complicate diagnosis.^9,10 Also, clinician (physician, nurse practitioner, and physician assistant) visits to nursing homes are often sporadic, and radiology facilities are rarely on the premises. As a consequence, residents are commonly sent to emergency departments for evaluation,^4,11,12 which undoubtedly contributes to a high hospitalization rate.

Clinicians who periodically see nursing home residents could benefit from a simple clinical tool to identify pneumonia. No large studies of community nursing home residents have systematically studied findings associated with pneumonia. As part of the Missouri LRI Project, we examined how well clinical findings predict radiographic pneumonia.

Methods

The Missouri LRI Project was a prospective observational study in 36 nursing homes in Central Missouri and St. Louis designed to investigate predictors of 2 outcomes of lower respiratory infections (LRIs): mortality and functional decline. Potential cases were identified from August 15, 1995, through September 29, 1998; however, all facilities were not involved until fall 1997. Study facilities were similar in size, ownership, and occupancy to national estimates from the 1995 National Nursing Home Survey (data available on request).¹³

We trained nursing home staff to report ill residents with any of 6 respiratory symptoms (eg, cough, dyspnea, sputum production) or 6 general symptoms (eg, fever, decline in mobility, mental status changes). Project nurses called and visited facilities frequently to reinforce reporting. Under a physician-authorized protocol, ill residents with a possible LRI received a standardized evaluation by a trained project nurse and usually a chest radiograph, complete blood count, and a chemistry panel. Complete criteria for triggering an evaluation are listed in Table 1. For this paper, we were concerned with the 90% of evaluated residents who received a chest radiograph. Criteria for excluding residents from evaluation are summarized in the Figure 1.

The nurse evaluation included an inventory of current symptoms, a review of important chronic conditions (eg, congestive heart failure), and a targeted physical examination. The examination included vital signs and the following body areas or systems: ears, nose, and throat; cardiac; abdominal; neurologic; extremities; skin; and a detailed lung examination. Most project nurses had advanced practice training; the remainder had extensive clinical experience and training in physical assessment. All received an individualized training session with a project geriatrician. Project nurses had substantially more experience than the nursing home staff, who usually report clinical findings to physicians.

Results of the evaluation were reported to the attending physician, who made all treatment decisions. Since the evaluations were clinically appropriate care authorized by individual attending physicians, the institutional review boards that reviewed the project allowed us to substantially simplify the consent process to a simple acceptance or refusal of the evaluation. In 9.2% of evaluations the resident was transferred to the hospital before project nurses could complete a physical assessment. In these instances, we obtained vital sign and clinical examination data from hospital records.

Radiographic Classification

Since all subjects had at least one illness symptom, for this analysis we classified the presence or absence of pneumonia on the basis of reported radiographic findings. Using defined criteria, 2 clinicians independently separated radiology reports into 3 categories: (a) negative, (b) possible, or (c) probable or definite for pneumonia (hereafter, probable pneumonia). For example, a report describing “new left lower lobe infiltrate suggestive of pneumonia” would have been rated as probable, while a report indicating “possible infiltrate” or “infiltrate suggestive of pneumonia or congestive heart failure” would have been rated as possible. As radiologists rarely provide completely unequivocal readings, we did not separate probable and definite pneumonia. In St. Louis 2 clinicians evaluated the reports, and in central Missouri 2 of 4 clinicians considered each report. Where there was disagreement, all 6 raters from the 2 sites independently reviewed the reports and then attempted to reach consensus. For 13% of radiographs, the project radiologist independently interpreted the actual films. This occurred when: (1) consensus could not be achieved; or (2) consensus was possible pneumonia, but probable pneumonia was needed to quality the episode as an LRI under the project definition.

Statistical Analyses

As residents could be included more than once, the unit of analysis throughout is episode of illness. In our major analysis, we developed a multivariable logistic model to estimate the probability of radiographic pneumonia (possible or probable). Before beginning modeling, we imputed mean values for missing continuous data and the largest category for missing dichotomous variables (the number of missing values is noted in Table 2). Data imputation is less biased than dropping cases in developing multivariable models.¹⁴

Illness episodes were then randomly assigned to a two thirds model-development and a one third model-validation sample. On the basis of the literature and clinical experience, we defined categories of variables that might relate to the presence or absence of pneumonia, such as lung findings (eg, crackles, wheezes), respiratory symptoms (eg, cough, sputum production), vital signs, findings of delirium (eg, acute confusion, decreased alertness), and laboratory findings. Restricting our focus to the development sample, we selected the best representatives of these groups on clinical and statistical grounds. For continuous variables, we considered the shape of the relationship to presence of pneumonia. For example, both very high and very low pulse rates predicted increased risk of pneumonia. In such cases, we considered several different ways to represent the variable in the model. We also limited the range of some variables to avoid undue influence of outliers (approximately the 1% most extreme values). For example, pulse rate above 140 was set equal to 140.

We then employed forward and backward stepwise logistic regression with possible or probable pneumonia (also referred to as positive x-ray results) as the dependent variable. For final model inclusion, we required variables to bear a plausible relationship to the diagnosis of pneumonia and meet a statistical significance criterion (a=.05).

To obtain final estimates of the relationship of each model variable to pneumonia probability, we considered adjustments for 2 kinds of correlation within our data: (1) individuals are nested within facilities, and (2) subjects could be represented by more than one episode.¹⁵ Using generalized estimating equations (GEE) in Proc Genmod in SAS software (SAS Institute, Cary, NC),¹⁶ we noted that the effect of facilities was minor, but the effect of repeat episodes by the same subject was more marked. Consequently, we used GEE to account for repeat episodes on subjects. To avoid unstable GEE estimates, we dropped 5 episodes in the development sample and 8 in the overall sample (episodes beyond the 5th and 6th per individual, respectively).

Using parameter estimates from the development sample, we tested the model’s discrimination and calibration in the validation sample.¹⁷ To assess discrimination, we used the c-statistic, which evaluates among all possible pairs of individuals whether those with higher predicted risk are more likely to die. The c-statistic is also equal to the area under the receiver operating characteristic curve. To assess calibration—agreement between observed and predicted mortality over the range of predicted risk—we used the Hosmer-Lemeshow goodness-of-fit statistic.¹⁸ We then used estimates fitted to the overall sample to develop a simple additive score to provide a clinically usable prediction rule. Statistical analyses were performed with SAS statistical software.¹⁶

Results

Project nurses performed 2592 evaluations. In 90% (2337), residents received chest x-rays either in the nursing home or on hospital transfer. In 3 additional cases crucial information was missing from nursing home records. This left for final analysis 2334 episodes in 1474 individuals Figure 1.

Fifty-five percent of radiographs were interpreted as negative, 12% showed possible pneumonia, and 33% showed probable pneumonia. Most nursing home residents with pneumonia had few presenting symptoms; 80% had 3 or fewer respiratory or general symptoms. However, only 7.5% of subjects evaluated had no respiratory symptoms. Table 2 shows the relationship of selected variables to radiographic findings of absent, possible, or probable pneumonia. Though a few signs and symptoms are more common in those with positive (possible or probable pneumonia) than negative chest x-ray results, most did not discriminate at all. Fever (temperature Ž38°C) was present in 44.4% of positives but only 28.5% of negatives (P=.001).

Multivariable Analysis and Prediction Score

Our GEE model to predict radiographic pneumonia includes 3 vital sign abnormalities (fever, rapid pulse, and rapid respiratory rate), 2 lung findings (presence of crackles and absence of wheezes), 2 potential indicators of delirium (somnolence or decreased alertness and acute confusion), and elevated white blood count. Table 3 reports GEE estimates for the entire sample. Though only exhibiting fair overall performance, the model did well at distinguishing subjects with a high probability of pneumonia. In the 20% of subjects with the highest predicted risks, more than two thirds had pneumonia.

For the full range of values, the model derived on the development sample showed a c-statistic of 0.672, which reduced to 0.632 in the validation sample. A value of 1.0 would indicate perfect discrimination between those who did and did not have radiographic pneumonia, while a value of 0.5 would indicate no better than chance discrimination. Model calibration was not acceptable in the validation sample (Hosmer-Lemeshow goodness-of-fit statistic, P=.008). Inspection suggested the disagreement between predicted and observed probability of pneumonia was primarily with lower-risk estimates.

Because the model performed relatively well at distinguishing subjects very likely to have pneumonia, we created a simple point system aimed at identifying such high-risk individuals. Table 4 shows the scoring system. For 33% of subjects (score Ž3), there was a 56% or higher probability of radiographic pneumonia. An additional 24% of subjects (score of 2) had 44% probability of radiographic pneumonia. However, even those with the lowest scores (-1 to 0, 15% of subjects) still had a 24% probability of pneumonia. The relationship between the score and the probability of radiographic evidence of pneumonia is shown in Figure W1.*

Discussion

In a large community-based sample, we considered presenting symptoms, signs, and laboratory findings associated with radiographic pneumonia. Individual findings discriminated poorly, and we could not separate out a very-low-risk group. However, our simple scoring system identified approximately one third to slightly more than one half with high probability of pneumonia—individuals who might be treated without a confirmatory chest x-ray. If our data are confirmed, they suggest a simple clinical strategy in patients with respiratory or general symptoms Table 1 that might suggest pneumonia: (1) if there are no respiratory symptoms, consider other conditions, such as a urinary tract infection, that might fully explain the symptoms; (2) obtain information to apply our symptom score Table 4; (3) for those with scores of 2 or higher (some might choose 3 instead), treat for pneumonia; (4) for those with scores of -1, 0, or 1, obtain a chest radiograph as a guide to treatment.

Considering individual findings, fever was significantly more common in pneumonia, but only 43% of those with possible or probable pneumonia had a temperature of at least 38°C. This reaffirms common wisdom and previous findings that fever is frequently absent in elderly people with pneumonia.^9,19 We also confirmed that few signs or symptoms are the norm for nursing home-acquired pneumonia.

Chest examination findings also do not adequately distinguish patients with and without pneumonia Table 2. Also, even expert physicians frequently differ on lung examination findings.²⁰ Nonetheless, presence of crackles and absence of wheezing contribute to our scoring system. Both findings are seen with multiple conditions, but in our data crackles are slightly more associated with pneumonia, while wheezing is more strongly associated with other diseases.

The other components of our scoring system are clinical factors commonly associated with pneumonia. Though none individually discriminates well between those with and without pneumonia Table 2, several combined serve to identify a high-risk group.

Four previous studies from emergency department or outpatient settings developed clinical prediction rules to identify pneumonia.^21-24 Criteria for identifying subjects varied substantially, and each rule has limited accuracy in predicting radiographic pneumonia.²⁰ We had adequate data to evaluate 3 of the rules.^21-23 As is usually the case when transporting a prediction rule to a new sample, none performed any better than our rule (data not shown). Our sample created the very difficult challenge for any prediction rule of a very high overall prevalence of pneumonia (45%). That made it unlikely that we could identify a low-risk group in whom x-ray studies could be readily forgone, but we were able to identify a highrisk group.

Limitations

Our findings are subject to several limitations. All facilities in our study were located in central or eastern Missouri, and not all physicians or eligible residents in those facilities participated. Compared with national data, we studied an unusually representative sample of nursing home residents from 36 facilities, including rural and urban locations. Also, in episodes excluded because of physician nonparticipation, residents were very similar to included residents in age, vital signs, and presenting symptoms (data available on request). More important, we lack an independent validation sample from a different cohort. Clinical prediction rules usually do not perform as well in independent samples. This is exemplified by the poor performance of the 3 rules we considered from other settings. Overall, our logistic model was only modest in discriminating and was not well calibrated for low-risk episodes in our reserved validation sample. Although we have developed a promising scoring system to identify residents with high probability of radiographic pneumonia, it needs to be validated in other samples of nursing home residents to determine its ultimate usefulness. For all these reasons, our results may not generalize.

Also, although we identified residents prospectively, project nurses were unable to evaluate 9.2% of residents before transfer to a hospital. Clinical findings abstracted from medical records, such as lung findings, may not have been complete. It is also possible that project nurses could have missed some important findings. However, our staff provided a higher level of expertise than is typically available in nursing homes. In fact, this may limit application of our findings. Nursing home staff vary widely in their ability to accurately examine residents or even identify illness. In many instances, facility staff had not obtained vital signs at the point when we identified a resident as ill enough to qualify for an evaluation.²⁵ Therefore, in many nursing homes, physicians may lack confidence to apply our rule without an evaluation by a physician, advanced practice nurse, or physician assistant.

Finally, determining whether subjects had pneumonia primarily depended on our classification of radiographic reports. Though radiographs generally included 2 views, many were portable films of variable quality, and frequently there was no previous radiograph for comparison. In some subjects with pneumonia, radiographic infiltrates might not yet have developed. Also, even under ideal conditions, radiologists commonly disagree on the presence of pneumonia.²⁶ Some subjects may have been misclassified. However, unless radiographic technique or interpretation was specifically related to clinical predictors, misclassification would simply diminish the relationship of predictors to pneumonia rather than creating a bias. We reviewed reports rather than radiographs, because that is the information usually available to clinicians faced with diagnosis and treatment decisions. We also paid special attention to avoiding any bias in the interpretations. All data were recorded before interpreting radiology reports and the interpretations were performed independent of clinical data. We also made special efforts to assure consistency in labeling radiology reports as possible, probable, or negative for pneumonia. When lack of agreement persisted, the study radiologist reinterpreted the actual films.

Conclusions

Most nursing home residents with pneumonia have few symptoms. We created a simple scoring to identify nursing home residents who have a high probability of radiographic pneumonia. If our results are confirmed, physicians might consider initiating treatment without an x-ray in such residents. Low scores do not rule out pneumonia, and most physicians would want to press for further diagnosis or treatment in this group.

Acknowledgments

This study was supported by the Agency for Healthcare Research and Quality (grant HS08551) and Dr Mehr’s Robert Wood Johnson Foundation Generalist Physician Faculty Scholars award. Dr Kruse was partially supported by an Institutional National Research Service Award (PE10038) from the Health Resources and Services Administration. Our project would not have been possible without the support of the many attending physicians, administrators, and staff of the involved nursing homes. Dr Clive Levine re-read more than 200 radiographs; Karen Davenport provided crucial administrative support; and Karen Madrone, MPA, assisted with manuscript preparation. Many other unnamed project staff also contributed.

Pneumonia is a leading cause of morbidity, mortality, and hospitalization of nursing home residents.^1-8 Atypical presentations and fewer presenting signs and symptoms in older patients complicate diagnosis.^9,10 Also, clinician (physician, nurse practitioner, and physician assistant) visits to nursing homes are often sporadic, and radiology facilities are rarely on the premises. As a consequence, residents are commonly sent to emergency departments for evaluation,^4,11,12 which undoubtedly contributes to a high hospitalization rate.

Clinicians who periodically see nursing home residents could benefit from a simple clinical tool to identify pneumonia. No large studies of community nursing home residents have systematically studied findings associated with pneumonia. As part of the Missouri LRI Project, we examined how well clinical findings predict radiographic pneumonia.

Methods

The Missouri LRI Project was a prospective observational study in 36 nursing homes in Central Missouri and St. Louis designed to investigate predictors of 2 outcomes of lower respiratory infections (LRIs): mortality and functional decline. Potential cases were identified from August 15, 1995, through September 29, 1998; however, all facilities were not involved until fall 1997. Study facilities were similar in size, ownership, and occupancy to national estimates from the 1995 National Nursing Home Survey (data available on request).¹³

We trained nursing home staff to report ill residents with any of 6 respiratory symptoms (eg, cough, dyspnea, sputum production) or 6 general symptoms (eg, fever, decline in mobility, mental status changes). Project nurses called and visited facilities frequently to reinforce reporting. Under a physician-authorized protocol, ill residents with a possible LRI received a standardized evaluation by a trained project nurse and usually a chest radiograph, complete blood count, and a chemistry panel. Complete criteria for triggering an evaluation are listed in Table 1. For this paper, we were concerned with the 90% of evaluated residents who received a chest radiograph. Criteria for excluding residents from evaluation are summarized in the Figure 1.

The nurse evaluation included an inventory of current symptoms, a review of important chronic conditions (eg, congestive heart failure), and a targeted physical examination. The examination included vital signs and the following body areas or systems: ears, nose, and throat; cardiac; abdominal; neurologic; extremities; skin; and a detailed lung examination. Most project nurses had advanced practice training; the remainder had extensive clinical experience and training in physical assessment. All received an individualized training session with a project geriatrician. Project nurses had substantially more experience than the nursing home staff, who usually report clinical findings to physicians.

Results of the evaluation were reported to the attending physician, who made all treatment decisions. Since the evaluations were clinically appropriate care authorized by individual attending physicians, the institutional review boards that reviewed the project allowed us to substantially simplify the consent process to a simple acceptance or refusal of the evaluation. In 9.2% of evaluations the resident was transferred to the hospital before project nurses could complete a physical assessment. In these instances, we obtained vital sign and clinical examination data from hospital records.

Radiographic Classification

Since all subjects had at least one illness symptom, for this analysis we classified the presence or absence of pneumonia on the basis of reported radiographic findings. Using defined criteria, 2 clinicians independently separated radiology reports into 3 categories: (a) negative, (b) possible, or (c) probable or definite for pneumonia (hereafter, probable pneumonia). For example, a report describing “new left lower lobe infiltrate suggestive of pneumonia” would have been rated as probable, while a report indicating “possible infiltrate” or “infiltrate suggestive of pneumonia or congestive heart failure” would have been rated as possible. As radiologists rarely provide completely unequivocal readings, we did not separate probable and definite pneumonia. In St. Louis 2 clinicians evaluated the reports, and in central Missouri 2 of 4 clinicians considered each report. Where there was disagreement, all 6 raters from the 2 sites independently reviewed the reports and then attempted to reach consensus. For 13% of radiographs, the project radiologist independently interpreted the actual films. This occurred when: (1) consensus could not be achieved; or (2) consensus was possible pneumonia, but probable pneumonia was needed to quality the episode as an LRI under the project definition.

Statistical Analyses

As residents could be included more than once, the unit of analysis throughout is episode of illness. In our major analysis, we developed a multivariable logistic model to estimate the probability of radiographic pneumonia (possible or probable). Before beginning modeling, we imputed mean values for missing continuous data and the largest category for missing dichotomous variables (the number of missing values is noted in Table 2). Data imputation is less biased than dropping cases in developing multivariable models.¹⁴

Illness episodes were then randomly assigned to a two thirds model-development and a one third model-validation sample. On the basis of the literature and clinical experience, we defined categories of variables that might relate to the presence or absence of pneumonia, such as lung findings (eg, crackles, wheezes), respiratory symptoms (eg, cough, sputum production), vital signs, findings of delirium (eg, acute confusion, decreased alertness), and laboratory findings. Restricting our focus to the development sample, we selected the best representatives of these groups on clinical and statistical grounds. For continuous variables, we considered the shape of the relationship to presence of pneumonia. For example, both very high and very low pulse rates predicted increased risk of pneumonia. In such cases, we considered several different ways to represent the variable in the model. We also limited the range of some variables to avoid undue influence of outliers (approximately the 1% most extreme values). For example, pulse rate above 140 was set equal to 140.

We then employed forward and backward stepwise logistic regression with possible or probable pneumonia (also referred to as positive x-ray results) as the dependent variable. For final model inclusion, we required variables to bear a plausible relationship to the diagnosis of pneumonia and meet a statistical significance criterion (a=.05).

To obtain final estimates of the relationship of each model variable to pneumonia probability, we considered adjustments for 2 kinds of correlation within our data: (1) individuals are nested within facilities, and (2) subjects could be represented by more than one episode.¹⁵ Using generalized estimating equations (GEE) in Proc Genmod in SAS software (SAS Institute, Cary, NC),¹⁶ we noted that the effect of facilities was minor, but the effect of repeat episodes by the same subject was more marked. Consequently, we used GEE to account for repeat episodes on subjects. To avoid unstable GEE estimates, we dropped 5 episodes in the development sample and 8 in the overall sample (episodes beyond the 5th and 6th per individual, respectively).

Using parameter estimates from the development sample, we tested the model’s discrimination and calibration in the validation sample.¹⁷ To assess discrimination, we used the c-statistic, which evaluates among all possible pairs of individuals whether those with higher predicted risk are more likely to die. The c-statistic is also equal to the area under the receiver operating characteristic curve. To assess calibration—agreement between observed and predicted mortality over the range of predicted risk—we used the Hosmer-Lemeshow goodness-of-fit statistic.¹⁸ We then used estimates fitted to the overall sample to develop a simple additive score to provide a clinically usable prediction rule. Statistical analyses were performed with SAS statistical software.¹⁶

Results

Project nurses performed 2592 evaluations. In 90% (2337), residents received chest x-rays either in the nursing home or on hospital transfer. In 3 additional cases crucial information was missing from nursing home records. This left for final analysis 2334 episodes in 1474 individuals Figure 1.

Fifty-five percent of radiographs were interpreted as negative, 12% showed possible pneumonia, and 33% showed probable pneumonia. Most nursing home residents with pneumonia had few presenting symptoms; 80% had 3 or fewer respiratory or general symptoms. However, only 7.5% of subjects evaluated had no respiratory symptoms. Table 2 shows the relationship of selected variables to radiographic findings of absent, possible, or probable pneumonia. Though a few signs and symptoms are more common in those with positive (possible or probable pneumonia) than negative chest x-ray results, most did not discriminate at all. Fever (temperature Ž38°C) was present in 44.4% of positives but only 28.5% of negatives (P=.001).

Multivariable Analysis and Prediction Score

Our GEE model to predict radiographic pneumonia includes 3 vital sign abnormalities (fever, rapid pulse, and rapid respiratory rate), 2 lung findings (presence of crackles and absence of wheezes), 2 potential indicators of delirium (somnolence or decreased alertness and acute confusion), and elevated white blood count. Table 3 reports GEE estimates for the entire sample. Though only exhibiting fair overall performance, the model did well at distinguishing subjects with a high probability of pneumonia. In the 20% of subjects with the highest predicted risks, more than two thirds had pneumonia.

For the full range of values, the model derived on the development sample showed a c-statistic of 0.672, which reduced to 0.632 in the validation sample. A value of 1.0 would indicate perfect discrimination between those who did and did not have radiographic pneumonia, while a value of 0.5 would indicate no better than chance discrimination. Model calibration was not acceptable in the validation sample (Hosmer-Lemeshow goodness-of-fit statistic, P=.008). Inspection suggested the disagreement between predicted and observed probability of pneumonia was primarily with lower-risk estimates.

Because the model performed relatively well at distinguishing subjects very likely to have pneumonia, we created a simple point system aimed at identifying such high-risk individuals. Table 4 shows the scoring system. For 33% of subjects (score Ž3), there was a 56% or higher probability of radiographic pneumonia. An additional 24% of subjects (score of 2) had 44% probability of radiographic pneumonia. However, even those with the lowest scores (-1 to 0, 15% of subjects) still had a 24% probability of pneumonia. The relationship between the score and the probability of radiographic evidence of pneumonia is shown in Figure W1.*

Discussion

In a large community-based sample, we considered presenting symptoms, signs, and laboratory findings associated with radiographic pneumonia. Individual findings discriminated poorly, and we could not separate out a very-low-risk group. However, our simple scoring system identified approximately one third to slightly more than one half with high probability of pneumonia—individuals who might be treated without a confirmatory chest x-ray. If our data are confirmed, they suggest a simple clinical strategy in patients with respiratory or general symptoms Table 1 that might suggest pneumonia: (1) if there are no respiratory symptoms, consider other conditions, such as a urinary tract infection, that might fully explain the symptoms; (2) obtain information to apply our symptom score Table 4; (3) for those with scores of 2 or higher (some might choose 3 instead), treat for pneumonia; (4) for those with scores of -1, 0, or 1, obtain a chest radiograph as a guide to treatment.

Considering individual findings, fever was significantly more common in pneumonia, but only 43% of those with possible or probable pneumonia had a temperature of at least 38°C. This reaffirms common wisdom and previous findings that fever is frequently absent in elderly people with pneumonia.^9,19 We also confirmed that few signs or symptoms are the norm for nursing home-acquired pneumonia.

Chest examination findings also do not adequately distinguish patients with and without pneumonia Table 2. Also, even expert physicians frequently differ on lung examination findings.²⁰ Nonetheless, presence of crackles and absence of wheezing contribute to our scoring system. Both findings are seen with multiple conditions, but in our data crackles are slightly more associated with pneumonia, while wheezing is more strongly associated with other diseases.

The other components of our scoring system are clinical factors commonly associated with pneumonia. Though none individually discriminates well between those with and without pneumonia Table 2, several combined serve to identify a high-risk group.

Four previous studies from emergency department or outpatient settings developed clinical prediction rules to identify pneumonia.^21-24 Criteria for identifying subjects varied substantially, and each rule has limited accuracy in predicting radiographic pneumonia.²⁰ We had adequate data to evaluate 3 of the rules.^21-23 As is usually the case when transporting a prediction rule to a new sample, none performed any better than our rule (data not shown). Our sample created the very difficult challenge for any prediction rule of a very high overall prevalence of pneumonia (45%). That made it unlikely that we could identify a low-risk group in whom x-ray studies could be readily forgone, but we were able to identify a highrisk group.

Limitations

Our findings are subject to several limitations. All facilities in our study were located in central or eastern Missouri, and not all physicians or eligible residents in those facilities participated. Compared with national data, we studied an unusually representative sample of nursing home residents from 36 facilities, including rural and urban locations. Also, in episodes excluded because of physician nonparticipation, residents were very similar to included residents in age, vital signs, and presenting symptoms (data available on request). More important, we lack an independent validation sample from a different cohort. Clinical prediction rules usually do not perform as well in independent samples. This is exemplified by the poor performance of the 3 rules we considered from other settings. Overall, our logistic model was only modest in discriminating and was not well calibrated for low-risk episodes in our reserved validation sample. Although we have developed a promising scoring system to identify residents with high probability of radiographic pneumonia, it needs to be validated in other samples of nursing home residents to determine its ultimate usefulness. For all these reasons, our results may not generalize.

Also, although we identified residents prospectively, project nurses were unable to evaluate 9.2% of residents before transfer to a hospital. Clinical findings abstracted from medical records, such as lung findings, may not have been complete. It is also possible that project nurses could have missed some important findings. However, our staff provided a higher level of expertise than is typically available in nursing homes. In fact, this may limit application of our findings. Nursing home staff vary widely in their ability to accurately examine residents or even identify illness. In many instances, facility staff had not obtained vital signs at the point when we identified a resident as ill enough to qualify for an evaluation.²⁵ Therefore, in many nursing homes, physicians may lack confidence to apply our rule without an evaluation by a physician, advanced practice nurse, or physician assistant.

Finally, determining whether subjects had pneumonia primarily depended on our classification of radiographic reports. Though radiographs generally included 2 views, many were portable films of variable quality, and frequently there was no previous radiograph for comparison. In some subjects with pneumonia, radiographic infiltrates might not yet have developed. Also, even under ideal conditions, radiologists commonly disagree on the presence of pneumonia.²⁶ Some subjects may have been misclassified. However, unless radiographic technique or interpretation was specifically related to clinical predictors, misclassification would simply diminish the relationship of predictors to pneumonia rather than creating a bias. We reviewed reports rather than radiographs, because that is the information usually available to clinicians faced with diagnosis and treatment decisions. We also paid special attention to avoiding any bias in the interpretations. All data were recorded before interpreting radiology reports and the interpretations were performed independent of clinical data. We also made special efforts to assure consistency in labeling radiology reports as possible, probable, or negative for pneumonia. When lack of agreement persisted, the study radiologist reinterpreted the actual films.

Conclusions

Most nursing home residents with pneumonia have few symptoms. We created a simple scoring to identify nursing home residents who have a high probability of radiographic pneumonia. If our results are confirmed, physicians might consider initiating treatment without an x-ray in such residents. Low scores do not rule out pneumonia, and most physicians would want to press for further diagnosis or treatment in this group.

Acknowledgments

This study was supported by the Agency for Healthcare Research and Quality (grant HS08551) and Dr Mehr’s Robert Wood Johnson Foundation Generalist Physician Faculty Scholars award. Dr Kruse was partially supported by an Institutional National Research Service Award (PE10038) from the Health Resources and Services Administration. Our project would not have been possible without the support of the many attending physicians, administrators, and staff of the involved nursing homes. Dr Clive Levine re-read more than 200 radiographs; Karen Davenport provided crucial administrative support; and Karen Madrone, MPA, assisted with manuscript preparation. Many other unnamed project staff also contributed.