Interpretation of confidence intervals

The question “How statistically significant are the results of a study?” can be answered in several ways. For example, the difference in mean blood-pressure lowering effect between a new drug and placebo from a specific study can be expressed as:

Difference=8.5 mm Hg; P=.03

However, the true effect of the drug is unlikely to be exactly 8.5 mm Hg—if the study were repeated, the results would be somewhat different. The variation is not explained by the P value. The P value is prone to misinterpretation, as will be explained in a future Language of Evidence. It tells us only that a difference is significant; it says nothing about its magnitude or precision.

Confidence intervals are a better alternative. Consider the difference above expressed as a confidence interval:

Difference=8.5 mm Hg; 95% CI, 6.3–10.8 or, graphically:

The probability of obtaining a result of 8.5 mm Hg, assuming the true value is not within the interval of 6.3–10.8 mm Hg, is 5% or less. In other words, one wouldn’t get the result above very often if the true value is outside the interval. This interpretation seems convoluted. It is easier to think of it this way: “The true value for the difference is most likely between 6.3 and 10.8. It is unlikely to be outside of this interval.”

3 Important principles

1. The wider a confidence interval, the less precise the estimate. Precision depends upon sample size. Therefore, the larger the sample size of a study, the narrower the confidence interval and the better the estimate. In the example above, a similar study with a much larger sample size may yield a narrower difference: 8.5 mm Hg; 95% CI, 7.4–8.9.
2. The 90% (or lower) confidence interval for an estimate is narrower than the 95% confidence interval; a 99% confidence interval is wider. This makes sense, since we are surer that a true value lies between 2 widely separated numbers than 2 more narrowly separated numbers.
3. If the confidence interval includes “no difference”(in this case, zero change in blood pressure), the corresponding Pvalue must be >.05. “No difference” for a result expressed as a subtraction is zero; for a result expressed as a ratio (such as an absolute difference or a relative risk), it would be 1. It is important to keep this difference in mind when interpreting confidence intervals.

Correspondence
Goutham Rao, MD, 3518 Fifth Avenue, Pittsburgh, PA 15261. E-mail: raog@msx.upmc.edu.

The question “How statistically significant are the results of a study?” can be answered in several ways. For example, the difference in mean blood-pressure lowering effect between a new drug and placebo from a specific study can be expressed as:

Difference=8.5 mm Hg; P=.03

However, the true effect of the drug is unlikely to be exactly 8.5 mm Hg—if the study were repeated, the results would be somewhat different. The variation is not explained by the P value. The P value is prone to misinterpretation, as will be explained in a future Language of Evidence. It tells us only that a difference is significant; it says nothing about its magnitude or precision.

Confidence intervals are a better alternative. Consider the difference above expressed as a confidence interval:

Difference=8.5 mm Hg; 95% CI, 6.3–10.8 or, graphically:

The probability of obtaining a result of 8.5 mm Hg, assuming the true value is not within the interval of 6.3–10.8 mm Hg, is 5% or less. In other words, one wouldn’t get the result above very often if the true value is outside the interval. This interpretation seems convoluted. It is easier to think of it this way: “The true value for the difference is most likely between 6.3 and 10.8. It is unlikely to be outside of this interval.”

3 Important principles

1. The wider a confidence interval, the less precise the estimate. Precision depends upon sample size. Therefore, the larger the sample size of a study, the narrower the confidence interval and the better the estimate. In the example above, a similar study with a much larger sample size may yield a narrower difference: 8.5 mm Hg; 95% CI, 7.4–8.9.
2. The 90% (or lower) confidence interval for an estimate is narrower than the 95% confidence interval; a 99% confidence interval is wider. This makes sense, since we are surer that a true value lies between 2 widely separated numbers than 2 more narrowly separated numbers.
3. If the confidence interval includes “no difference”(in this case, zero change in blood pressure), the corresponding Pvalue must be >.05. “No difference” for a result expressed as a subtraction is zero; for a result expressed as a ratio (such as an absolute difference or a relative risk), it would be 1. It is important to keep this difference in mind when interpreting confidence intervals.

Correspondence
Goutham Rao, MD, 3518 Fifth Avenue, Pittsburgh, PA 15261. E-mail: raog@msx.upmc.edu.

The question “How statistically significant are the results of a study?” can be answered in several ways. For example, the difference in mean blood-pressure lowering effect between a new drug and placebo from a specific study can be expressed as:

Difference=8.5 mm Hg; P=.03

However, the true effect of the drug is unlikely to be exactly 8.5 mm Hg—if the study were repeated, the results would be somewhat different. The variation is not explained by the P value. The P value is prone to misinterpretation, as will be explained in a future Language of Evidence. It tells us only that a difference is significant; it says nothing about its magnitude or precision.

Confidence intervals are a better alternative. Consider the difference above expressed as a confidence interval:

Difference=8.5 mm Hg; 95% CI, 6.3–10.8 or, graphically:

The probability of obtaining a result of 8.5 mm Hg, assuming the true value is not within the interval of 6.3–10.8 mm Hg, is 5% or less. In other words, one wouldn’t get the result above very often if the true value is outside the interval. This interpretation seems convoluted. It is easier to think of it this way: “The true value for the difference is most likely between 6.3 and 10.8. It is unlikely to be outside of this interval.”

3 Important principles

1. The wider a confidence interval, the less precise the estimate. Precision depends upon sample size. Therefore, the larger the sample size of a study, the narrower the confidence interval and the better the estimate. In the example above, a similar study with a much larger sample size may yield a narrower difference: 8.5 mm Hg; 95% CI, 7.4–8.9.
2. The 90% (or lower) confidence interval for an estimate is narrower than the 95% confidence interval; a 99% confidence interval is wider. This makes sense, since we are surer that a true value lies between 2 widely separated numbers than 2 more narrowly separated numbers.
3. If the confidence interval includes “no difference”(in this case, zero change in blood pressure), the corresponding Pvalue must be >.05. “No difference” for a result expressed as a subtraction is zero; for a result expressed as a ratio (such as an absolute difference or a relative risk), it would be 1. It is important to keep this difference in mind when interpreting confidence intervals.

Correspondence
Goutham Rao, MD, 3518 Fifth Avenue, Pittsburgh, PA 15261. E-mail: raog@msx.upmc.edu.