Judging the Evidence for Interventions

Diabetes is a very common disease, and amputations are a significant adverse outcome. Physicians feel compelled to do something to prevent these amputations. Logic dictates that sensory neuropathy predates the amputation and is causally related to the trauma that precedes it. Therefore, early detection of neuropathy may lead to interventions that decrease the likelihood of amputations. This has been the focus of a large number of studies and of current medical practice.

As Mayfield and colleagues point out in this issue of the Journal, many studies have evaluated the effectiveness of routine foot examinations for reducing amputation risk. These evaluations, however, have been part of a whole-patient approach to diabetes care. This is unfortunate, because it is not possible to evaluate the independent protective effect of foot examinations when they are part of a multitiered treatment plan. A patient who participates in such a treatment program may have reduced amputation risk, but it is not possible to tell whether the foot examinations led to this reduction or if it was caused by some other factor (such as better diabetes control).

Evaluating the evidence

There are several types of interventions in medicine. We evaluate treatment interventions by measuring outcomes in a randomized trial in which some patients are treated and some are not. Showing that there is an effective treatment is useful information, but a single trial cannot provide evidence about the intensity or duration of the treatment. For example, 30 years ago we treated uncomplicated urinary tract infections for 10 days because that was what the trials and clinical experience dictated. Later, shorter-course treatments became popular when clinical trials showed that a shorter course provided equal effectiveness and fewer side effects. The original trials showed that the treatment was effective but did not tell us the optimal treatment length.

Screening is another type of intervention. We can also evaluate the efficacy of screening by randomizing patients to screening and nonscreening groups and following the outcomes. If screening is shown to be effective, we have important information but still do not know how often it is needed. We can show that performing a Papanicolau (Pap) test every month reduces the risk of cervical cancer, but this does not prove that monthly is the optimal frequency.

Examining the foot of a person with diabetes is a screening procedure, and it would be nice to have randomized trials to evaluate its effectiveness. However, it is difficult to do these trials once the foot examination is a standard of care. In addition, although not uncommon, most patients with diabetes do not receive an amputation, and there is a long period from the onset of diabetes to an amputation. Both of these factors also argue against a clinical trial. Without a randomized trial, we are forced to approach the clinical question using other methods.

Epidemiologic methods, such as the case control study used by Mayfield and coworkers, can be useful. It is impossible, however, to prove causation through an epidemiologic study. Fortunately, there are several questions we can ask of epidemiologic data to evaluate the likelihood and the strength of a causal relationship. Is the evidence from humans? Obviously, animal studies do not provide as much support for causation as human studies. If there is an association, is the association strong? Strong associations are more likely to be tied to causal relationships than weak associations. Is the association consistent? Causal relationships are more likely if every study looking at the association shows the same findings.

We can also ask: Is there a dose-response gradient? This one is very tricky, because the doses used in the literature must be appropriate to see the gradient. You could show no gradient in the reduction of cervical cancer as you move from weekly to monthly to yearly Pap tests. This lack of gradient argues against Pap tests reducing the risk of cervical cancer. However, if the studies had evaluated Pap tests performed annually, every third year, and every sixth year, the appropriate dose response would have been seen.

Another evaluative question is: Is the temporal relationship correct? Although this is helpful with some topics (eg, low cholesterol measured before myocardial infarction), it is obvious that the foot examination must occur before the amputation.

Foot examinations

In the article by Mayfield and colleagues, we see that people who had an amputation had more foot examinations than people who did not. This suggests another difficulty in evaluating interventions: Often those who are more ill are more likely to be screened. Does the benefit documented in the study extend to all patients, or is it confined to those with advanced disease?

Mayfield and coworkers provide important additional evidence supporting the inclusion of foot examinations as a routine part of the evaluation of patients with diabetes. The article by itself does not prove that these examinations are useful. Even with the whole weight of evidence in the literature, this is still a contentious issue. However, the study by Mayfield and colleagues lends additional support to this practice.

There are several limitations to their study, however. The study used a highly select population. As the authors note, the Gila River Indian community has one of the world’s highest reported rates of diabetes and amputations. It may be that diabetes in this population is a fundamentally different disease, and that amputations are more likely and foot examinations are less effective than in the general population. If this were the case, the findings of this study grossly underestimate the true impact of foot examinations. Or it may be that in this population foot examinations are more likely to be beneficial than in another population. If that were the case, the study grossly overestimates the impact of foot examinations for reducing amputation risk.

Each year our government spends billions of dollars on research, but some basic clinical questions remain unanswered. Should we adopt foot screening as a standard in diabetes care? If not, is there a subset of patients that would benefit from screening? If we do screen, how often should it be done? Each visit? Annually? Even simple questions like "What constitutes a quality foot examination?" go unanswered.

The answers may not be determined soon, but family physicians can play a key role in finding them. Our practice-based research networks contain a large number of patients with diabetes, and carefully designed studies can assess either retrospective or even prospective evaluations of diabetes care. Diabetes care is a major challenge, and family physicians provide the bulk of this care to patients. Perhaps we will also provide the answers to some of the key questions about quality care for this disease.

Acknowledgments

Dr Ganiats is a recipient of the American Academy of Family Physicians Advanced Research Training Grant.

Diabetes is a very common disease, and amputations are a significant adverse outcome. Physicians feel compelled to do something to prevent these amputations. Logic dictates that sensory neuropathy predates the amputation and is causally related to the trauma that precedes it. Therefore, early detection of neuropathy may lead to interventions that decrease the likelihood of amputations. This has been the focus of a large number of studies and of current medical practice.

As Mayfield and colleagues point out in this issue of the Journal, many studies have evaluated the effectiveness of routine foot examinations for reducing amputation risk. These evaluations, however, have been part of a whole-patient approach to diabetes care. This is unfortunate, because it is not possible to evaluate the independent protective effect of foot examinations when they are part of a multitiered treatment plan. A patient who participates in such a treatment program may have reduced amputation risk, but it is not possible to tell whether the foot examinations led to this reduction or if it was caused by some other factor (such as better diabetes control).

Evaluating the evidence

There are several types of interventions in medicine. We evaluate treatment interventions by measuring outcomes in a randomized trial in which some patients are treated and some are not. Showing that there is an effective treatment is useful information, but a single trial cannot provide evidence about the intensity or duration of the treatment. For example, 30 years ago we treated uncomplicated urinary tract infections for 10 days because that was what the trials and clinical experience dictated. Later, shorter-course treatments became popular when clinical trials showed that a shorter course provided equal effectiveness and fewer side effects. The original trials showed that the treatment was effective but did not tell us the optimal treatment length.

Screening is another type of intervention. We can also evaluate the efficacy of screening by randomizing patients to screening and nonscreening groups and following the outcomes. If screening is shown to be effective, we have important information but still do not know how often it is needed. We can show that performing a Papanicolau (Pap) test every month reduces the risk of cervical cancer, but this does not prove that monthly is the optimal frequency.

Examining the foot of a person with diabetes is a screening procedure, and it would be nice to have randomized trials to evaluate its effectiveness. However, it is difficult to do these trials once the foot examination is a standard of care. In addition, although not uncommon, most patients with diabetes do not receive an amputation, and there is a long period from the onset of diabetes to an amputation. Both of these factors also argue against a clinical trial. Without a randomized trial, we are forced to approach the clinical question using other methods.

Epidemiologic methods, such as the case control study used by Mayfield and coworkers, can be useful. It is impossible, however, to prove causation through an epidemiologic study. Fortunately, there are several questions we can ask of epidemiologic data to evaluate the likelihood and the strength of a causal relationship. Is the evidence from humans? Obviously, animal studies do not provide as much support for causation as human studies. If there is an association, is the association strong? Strong associations are more likely to be tied to causal relationships than weak associations. Is the association consistent? Causal relationships are more likely if every study looking at the association shows the same findings.

We can also ask: Is there a dose-response gradient? This one is very tricky, because the doses used in the literature must be appropriate to see the gradient. You could show no gradient in the reduction of cervical cancer as you move from weekly to monthly to yearly Pap tests. This lack of gradient argues against Pap tests reducing the risk of cervical cancer. However, if the studies had evaluated Pap tests performed annually, every third year, and every sixth year, the appropriate dose response would have been seen.

Another evaluative question is: Is the temporal relationship correct? Although this is helpful with some topics (eg, low cholesterol measured before myocardial infarction), it is obvious that the foot examination must occur before the amputation.

Foot examinations

In the article by Mayfield and colleagues, we see that people who had an amputation had more foot examinations than people who did not. This suggests another difficulty in evaluating interventions: Often those who are more ill are more likely to be screened. Does the benefit documented in the study extend to all patients, or is it confined to those with advanced disease?

Mayfield and coworkers provide important additional evidence supporting the inclusion of foot examinations as a routine part of the evaluation of patients with diabetes. The article by itself does not prove that these examinations are useful. Even with the whole weight of evidence in the literature, this is still a contentious issue. However, the study by Mayfield and colleagues lends additional support to this practice.

There are several limitations to their study, however. The study used a highly select population. As the authors note, the Gila River Indian community has one of the world’s highest reported rates of diabetes and amputations. It may be that diabetes in this population is a fundamentally different disease, and that amputations are more likely and foot examinations are less effective than in the general population. If this were the case, the findings of this study grossly underestimate the true impact of foot examinations. Or it may be that in this population foot examinations are more likely to be beneficial than in another population. If that were the case, the study grossly overestimates the impact of foot examinations for reducing amputation risk.

Each year our government spends billions of dollars on research, but some basic clinical questions remain unanswered. Should we adopt foot screening as a standard in diabetes care? If not, is there a subset of patients that would benefit from screening? If we do screen, how often should it be done? Each visit? Annually? Even simple questions like "What constitutes a quality foot examination?" go unanswered.

The answers may not be determined soon, but family physicians can play a key role in finding them. Our practice-based research networks contain a large number of patients with diabetes, and carefully designed studies can assess either retrospective or even prospective evaluations of diabetes care. Diabetes care is a major challenge, and family physicians provide the bulk of this care to patients. Perhaps we will also provide the answers to some of the key questions about quality care for this disease.

Acknowledgments

Dr Ganiats is a recipient of the American Academy of Family Physicians Advanced Research Training Grant.

Diabetes is a very common disease, and amputations are a significant adverse outcome. Physicians feel compelled to do something to prevent these amputations. Logic dictates that sensory neuropathy predates the amputation and is causally related to the trauma that precedes it. Therefore, early detection of neuropathy may lead to interventions that decrease the likelihood of amputations. This has been the focus of a large number of studies and of current medical practice.

As Mayfield and colleagues point out in this issue of the Journal, many studies have evaluated the effectiveness of routine foot examinations for reducing amputation risk. These evaluations, however, have been part of a whole-patient approach to diabetes care. This is unfortunate, because it is not possible to evaluate the independent protective effect of foot examinations when they are part of a multitiered treatment plan. A patient who participates in such a treatment program may have reduced amputation risk, but it is not possible to tell whether the foot examinations led to this reduction or if it was caused by some other factor (such as better diabetes control).

Evaluating the evidence

There are several types of interventions in medicine. We evaluate treatment interventions by measuring outcomes in a randomized trial in which some patients are treated and some are not. Showing that there is an effective treatment is useful information, but a single trial cannot provide evidence about the intensity or duration of the treatment. For example, 30 years ago we treated uncomplicated urinary tract infections for 10 days because that was what the trials and clinical experience dictated. Later, shorter-course treatments became popular when clinical trials showed that a shorter course provided equal effectiveness and fewer side effects. The original trials showed that the treatment was effective but did not tell us the optimal treatment length.

Screening is another type of intervention. We can also evaluate the efficacy of screening by randomizing patients to screening and nonscreening groups and following the outcomes. If screening is shown to be effective, we have important information but still do not know how often it is needed. We can show that performing a Papanicolau (Pap) test every month reduces the risk of cervical cancer, but this does not prove that monthly is the optimal frequency.

Examining the foot of a person with diabetes is a screening procedure, and it would be nice to have randomized trials to evaluate its effectiveness. However, it is difficult to do these trials once the foot examination is a standard of care. In addition, although not uncommon, most patients with diabetes do not receive an amputation, and there is a long period from the onset of diabetes to an amputation. Both of these factors also argue against a clinical trial. Without a randomized trial, we are forced to approach the clinical question using other methods.

Epidemiologic methods, such as the case control study used by Mayfield and coworkers, can be useful. It is impossible, however, to prove causation through an epidemiologic study. Fortunately, there are several questions we can ask of epidemiologic data to evaluate the likelihood and the strength of a causal relationship. Is the evidence from humans? Obviously, animal studies do not provide as much support for causation as human studies. If there is an association, is the association strong? Strong associations are more likely to be tied to causal relationships than weak associations. Is the association consistent? Causal relationships are more likely if every study looking at the association shows the same findings.

We can also ask: Is there a dose-response gradient? This one is very tricky, because the doses used in the literature must be appropriate to see the gradient. You could show no gradient in the reduction of cervical cancer as you move from weekly to monthly to yearly Pap tests. This lack of gradient argues against Pap tests reducing the risk of cervical cancer. However, if the studies had evaluated Pap tests performed annually, every third year, and every sixth year, the appropriate dose response would have been seen.

Another evaluative question is: Is the temporal relationship correct? Although this is helpful with some topics (eg, low cholesterol measured before myocardial infarction), it is obvious that the foot examination must occur before the amputation.

Foot examinations

In the article by Mayfield and colleagues, we see that people who had an amputation had more foot examinations than people who did not. This suggests another difficulty in evaluating interventions: Often those who are more ill are more likely to be screened. Does the benefit documented in the study extend to all patients, or is it confined to those with advanced disease?

Mayfield and coworkers provide important additional evidence supporting the inclusion of foot examinations as a routine part of the evaluation of patients with diabetes. The article by itself does not prove that these examinations are useful. Even with the whole weight of evidence in the literature, this is still a contentious issue. However, the study by Mayfield and colleagues lends additional support to this practice.

There are several limitations to their study, however. The study used a highly select population. As the authors note, the Gila River Indian community has one of the world’s highest reported rates of diabetes and amputations. It may be that diabetes in this population is a fundamentally different disease, and that amputations are more likely and foot examinations are less effective than in the general population. If this were the case, the findings of this study grossly underestimate the true impact of foot examinations. Or it may be that in this population foot examinations are more likely to be beneficial than in another population. If that were the case, the study grossly overestimates the impact of foot examinations for reducing amputation risk.

Each year our government spends billions of dollars on research, but some basic clinical questions remain unanswered. Should we adopt foot screening as a standard in diabetes care? If not, is there a subset of patients that would benefit from screening? If we do screen, how often should it be done? Each visit? Annually? Even simple questions like "What constitutes a quality foot examination?" go unanswered.

The answers may not be determined soon, but family physicians can play a key role in finding them. Our practice-based research networks contain a large number of patients with diabetes, and carefully designed studies can assess either retrospective or even prospective evaluations of diabetes care. Diabetes care is a major challenge, and family physicians provide the bulk of this care to patients. Perhaps we will also provide the answers to some of the key questions about quality care for this disease.

Acknowledgments

Dr Ganiats is a recipient of the American Academy of Family Physicians Advanced Research Training Grant.