Vertebroplasty, cognitive dissonance, and evidence-based medicine: What do we do when the ‘evidence’ says we are wrong?

Cognitive dissonance describes how we respond to conflicting information that challenges our existing belief, the uncomfortable feeling we get when new evidence calls into question things that we “know” are true.

See related commentary

To the point: two recent clinical trials^1,2 have called into question the efficacy of vertebroplasty for treating osteoporotic vertebral compression fractures and have led many of us to question many of our assumptions, not only about vertebroplasty but also about evidencebased medicine.

Osteoporotic vertebral compression fractures are very common: more than 700,000 are estimated to occur in the United States annually. ³ They are costly and are associated with a risk of death.⁴ Fortunately, most heal without problems over 4 to 6 weeks with conventional treatment, ie, activity modification, analgesics, and bracing.

However, some patients do not seem to do so well and are debilitated by the pain of the fracture. Conventional fracture surgery carries very high risk and poor outcomes,⁵ and so has been reserved mostly for patients with neurologic deficits.

VERTEBROPLASTY GOES MAINSTREAM

Given these facts, investigators began looking for alternative treatments. One that rose to the fore was polymethylmethacrylate cement to stabilize the fracture. This technique, called vertebroplasty, involves injecting liquid cement through a needle into the vertebral body, where it hardens and is thought to restore stability.

Since the first description of vertebroplasty for treating symptomatic hemangiomas,⁶ many papers have been published about the procedure and about similar ones, now grouped under the general heading of vertebral augmentation. This includes kyphoplasty and other newer proprietary techniques. These procedures have been widely accepted, and their use is growing. They have shown good results in several prospective case series, and nonrandomized and randomized controlled studies have shown them to be more effective than conventional medical treatment.^7–25 For example, VERTOS, a small prospective randomized trial, showed that vertebroplasty was superior to conventional medical treatment.²⁵ When Wardlaw et al²⁴ showed that shortterm outcomes were better with kyphoplasty than with conventional medical therapy in a prospective randomized trial, many of us had moved past questioning whether vertebral augmentation is effective and were debating the relative merits of different methods and materials.

On a personal level, most of us became proponents of these procedures because we saw dramatic results—usually unequivocal. Most patients report significant improvement in pain immediately after the procedure, and many bedridden patients are able to leave the hospital within hours. In spine surgery, few procedures give such dramatic results with so few complications.

TWO NEW STUDIES UPSET ESTABLISHED BELIEF

This is why I am having such a hard time digesting the results of the trials by Buchbinder et al² and Kallmes et al,¹ published in the August 9, 2009, issue of the New England Journal of Medicine. Both were randomized controlled trials that used sham surgery rather than conventional medical treatment as the control. The sham procedure in each trial was the same as the intervention, with local anesthetic infiltration of the periosteum and mixing of the cement (so that the patients smelled its distinctive odor), but without placing the needle into the vertebra and injecting the cement.

In the study by Buchbinder et al,² the real treatment had no benefit in any primary or secondary end point. This study did not allow crossovers.

In the study by Kallmes et al,¹ more patients who received the real treatment reported clinically meaningful improvement in pain (a secondary end point), but the difference was not quite statistically significant (64% vs 48%, P = .06). In this trial, patients were allowed to cross over to the other study group after 1 month, and significantly more patients crossed over from the sham surgery group to the active treatment group than the other way around (43% vs 12%, P < .001).

My first instinct was to pick through the papers for flaws that would invalidate the results— and there were some problems. Both studies were initially planned to include more patients and therefore to have greater statistical power, but they were reassessed because of slow enrollment. In the study by Kallmes et al,¹ the difference in clinically meaningful improvement might have reached statistical significance if the trial had been larger. The study by Buchbinder et al² was a multicenter trial, but one center accounted for 53 (69%) of the 78 patients. Could this have biased the results?

The surgeon in me also seized for a while on the idea that since all of the interventions in both studies were done by interventional radiologists, the problem may have been in patient selection and that radiologists are not as astute as we are. However, even a surgeon’s ego cannot support this interpretation.

As I looked in more detail at the response I had written to these trials, I realized these criticisms were hardly fatal flaws, and the fact that two separate well-designed studies reached the same conclusion enhances their validity.

One concern that does bear some scrutiny is that the trials were too small to identify subgroups that may benefit from the procedure. In my experience, vertebral augmentation seems to have better results with certain types of fractures. Patients with a mobile pseudarthrotic cleft pattern of fracture seem to do much better than those with the more common nonmobile fracture.

THE POWERFUL PLACEBO EFFECT

Many commentaries on these two trials have discussed a famous study of a different procedure for a different condition. In this study, Moseley et al²⁶ evaluated the use of arthroscopy to treat osteoarthritis of the knee and found that sham arthroscopy was as effective as real arthroscopy and that both were better than conventional treatment.

I was not long out of my orthopedic residency when this trial was published and was very aware of the debate that preceded it, as I once had to prepare a talk about it for resident rounds. I remember that there was a lively debate in the orthopedic community over the efficacy of the procedure before the results of this trial were released.

In contrast, the vertebral augmentation controversy had become a debate about the relative efficacy and the economics of specific techniques, not about the effectiveness of the entire concept. The mainstream had accepted the validity of the procedure, which was not the case in the knee arthroscopy trial.

In both vertebroplasty studies, the activetreatment groups and the sham-treatment groups all showed significant and rapid improvement in pain and disability, and these results were maintained over the study period. Though most vertebral compression fractures do heal, the clinical improvement is usually gradual over a period of weeks. This raises the possibility that the sham treatment was actualy an active placebo.

There is some evidence to support this possibility. In a randomized trial of the efficacy of selective nerve root blocks for lumbar radiculopathy, Riew et al²⁷ showed that injection with a local anesthetic alone, although not as efficacious as a local anesthetic plus a corticosteroid at allowing patients to avoid surgery, showed an effect long after the expected duration of the anesthetic. The effect persisted even at 5 years of follow-up.²⁸

Is it possible that the local anesthetic in this trial and the vertebroplasty trials acted as some sort of “reset button” for pain sensation? This is an area that may bear further investigation.

WHERE DOES THIS LEAVE US?

So where does this leave us? On one hand, randomized controlled trials comparing vertebral augmentation with conventional medical therapy^24,25 showed augmentation to be beneficial. On the other hand, the studies by Kallmes et al¹ and Buchbinder et al² indicate vertebroplasty is no more effective than sham surgery.

It is very difficult for me to look at my own experience with vertebral augmentation and say that, on the basis of these trials, I am no longer going to offer it to my patients. I understand on an intellectual level that these trials call the efficacy of the procedure into question, but on a visceral level I cannot rationalize it. When faced with a patient who is barely ambulatory or in fact bed-bound due to pain, my experience tells me that vertebral augmentation has a very high chance of getting them ambulatory within hours. The trials of vertebroplasty would indicate this is a placebo effect or that local anesthetic alone is as effective, but I am not yet ready to make that leap.

Cognitive dissonance seems to rule.

Cognitive dissonance describes how we respond to conflicting information that challenges our existing belief, the uncomfortable feeling we get when new evidence calls into question things that we “know” are true.

See related commentary

To the point: two recent clinical trials^1,2 have called into question the efficacy of vertebroplasty for treating osteoporotic vertebral compression fractures and have led many of us to question many of our assumptions, not only about vertebroplasty but also about evidencebased medicine.

Osteoporotic vertebral compression fractures are very common: more than 700,000 are estimated to occur in the United States annually. ³ They are costly and are associated with a risk of death.⁴ Fortunately, most heal without problems over 4 to 6 weeks with conventional treatment, ie, activity modification, analgesics, and bracing.

However, some patients do not seem to do so well and are debilitated by the pain of the fracture. Conventional fracture surgery carries very high risk and poor outcomes,⁵ and so has been reserved mostly for patients with neurologic deficits.

VERTEBROPLASTY GOES MAINSTREAM

Given these facts, investigators began looking for alternative treatments. One that rose to the fore was polymethylmethacrylate cement to stabilize the fracture. This technique, called vertebroplasty, involves injecting liquid cement through a needle into the vertebral body, where it hardens and is thought to restore stability.

Since the first description of vertebroplasty for treating symptomatic hemangiomas,⁶ many papers have been published about the procedure and about similar ones, now grouped under the general heading of vertebral augmentation. This includes kyphoplasty and other newer proprietary techniques. These procedures have been widely accepted, and their use is growing. They have shown good results in several prospective case series, and nonrandomized and randomized controlled studies have shown them to be more effective than conventional medical treatment.^7–25 For example, VERTOS, a small prospective randomized trial, showed that vertebroplasty was superior to conventional medical treatment.²⁵ When Wardlaw et al²⁴ showed that shortterm outcomes were better with kyphoplasty than with conventional medical therapy in a prospective randomized trial, many of us had moved past questioning whether vertebral augmentation is effective and were debating the relative merits of different methods and materials.

On a personal level, most of us became proponents of these procedures because we saw dramatic results—usually unequivocal. Most patients report significant improvement in pain immediately after the procedure, and many bedridden patients are able to leave the hospital within hours. In spine surgery, few procedures give such dramatic results with so few complications.

TWO NEW STUDIES UPSET ESTABLISHED BELIEF

This is why I am having such a hard time digesting the results of the trials by Buchbinder et al² and Kallmes et al,¹ published in the August 9, 2009, issue of the New England Journal of Medicine. Both were randomized controlled trials that used sham surgery rather than conventional medical treatment as the control. The sham procedure in each trial was the same as the intervention, with local anesthetic infiltration of the periosteum and mixing of the cement (so that the patients smelled its distinctive odor), but without placing the needle into the vertebra and injecting the cement.

In the study by Buchbinder et al,² the real treatment had no benefit in any primary or secondary end point. This study did not allow crossovers.

In the study by Kallmes et al,¹ more patients who received the real treatment reported clinically meaningful improvement in pain (a secondary end point), but the difference was not quite statistically significant (64% vs 48%, P = .06). In this trial, patients were allowed to cross over to the other study group after 1 month, and significantly more patients crossed over from the sham surgery group to the active treatment group than the other way around (43% vs 12%, P < .001).

My first instinct was to pick through the papers for flaws that would invalidate the results— and there were some problems. Both studies were initially planned to include more patients and therefore to have greater statistical power, but they were reassessed because of slow enrollment. In the study by Kallmes et al,¹ the difference in clinically meaningful improvement might have reached statistical significance if the trial had been larger. The study by Buchbinder et al² was a multicenter trial, but one center accounted for 53 (69%) of the 78 patients. Could this have biased the results?

The surgeon in me also seized for a while on the idea that since all of the interventions in both studies were done by interventional radiologists, the problem may have been in patient selection and that radiologists are not as astute as we are. However, even a surgeon’s ego cannot support this interpretation.

As I looked in more detail at the response I had written to these trials, I realized these criticisms were hardly fatal flaws, and the fact that two separate well-designed studies reached the same conclusion enhances their validity.

One concern that does bear some scrutiny is that the trials were too small to identify subgroups that may benefit from the procedure. In my experience, vertebral augmentation seems to have better results with certain types of fractures. Patients with a mobile pseudarthrotic cleft pattern of fracture seem to do much better than those with the more common nonmobile fracture.

THE POWERFUL PLACEBO EFFECT

Many commentaries on these two trials have discussed a famous study of a different procedure for a different condition. In this study, Moseley et al²⁶ evaluated the use of arthroscopy to treat osteoarthritis of the knee and found that sham arthroscopy was as effective as real arthroscopy and that both were better than conventional treatment.

I was not long out of my orthopedic residency when this trial was published and was very aware of the debate that preceded it, as I once had to prepare a talk about it for resident rounds. I remember that there was a lively debate in the orthopedic community over the efficacy of the procedure before the results of this trial were released.

In contrast, the vertebral augmentation controversy had become a debate about the relative efficacy and the economics of specific techniques, not about the effectiveness of the entire concept. The mainstream had accepted the validity of the procedure, which was not the case in the knee arthroscopy trial.

In both vertebroplasty studies, the activetreatment groups and the sham-treatment groups all showed significant and rapid improvement in pain and disability, and these results were maintained over the study period. Though most vertebral compression fractures do heal, the clinical improvement is usually gradual over a period of weeks. This raises the possibility that the sham treatment was actualy an active placebo.

There is some evidence to support this possibility. In a randomized trial of the efficacy of selective nerve root blocks for lumbar radiculopathy, Riew et al²⁷ showed that injection with a local anesthetic alone, although not as efficacious as a local anesthetic plus a corticosteroid at allowing patients to avoid surgery, showed an effect long after the expected duration of the anesthetic. The effect persisted even at 5 years of follow-up.²⁸

Is it possible that the local anesthetic in this trial and the vertebroplasty trials acted as some sort of “reset button” for pain sensation? This is an area that may bear further investigation.

WHERE DOES THIS LEAVE US?

So where does this leave us? On one hand, randomized controlled trials comparing vertebral augmentation with conventional medical therapy^24,25 showed augmentation to be beneficial. On the other hand, the studies by Kallmes et al¹ and Buchbinder et al² indicate vertebroplasty is no more effective than sham surgery.

It is very difficult for me to look at my own experience with vertebral augmentation and say that, on the basis of these trials, I am no longer going to offer it to my patients. I understand on an intellectual level that these trials call the efficacy of the procedure into question, but on a visceral level I cannot rationalize it. When faced with a patient who is barely ambulatory or in fact bed-bound due to pain, my experience tells me that vertebral augmentation has a very high chance of getting them ambulatory within hours. The trials of vertebroplasty would indicate this is a placebo effect or that local anesthetic alone is as effective, but I am not yet ready to make that leap.

Cognitive dissonance seems to rule.

Cognitive dissonance describes how we respond to conflicting information that challenges our existing belief, the uncomfortable feeling we get when new evidence calls into question things that we “know” are true.

See related commentary

To the point: two recent clinical trials^1,2 have called into question the efficacy of vertebroplasty for treating osteoporotic vertebral compression fractures and have led many of us to question many of our assumptions, not only about vertebroplasty but also about evidencebased medicine.

Osteoporotic vertebral compression fractures are very common: more than 700,000 are estimated to occur in the United States annually. ³ They are costly and are associated with a risk of death.⁴ Fortunately, most heal without problems over 4 to 6 weeks with conventional treatment, ie, activity modification, analgesics, and bracing.

However, some patients do not seem to do so well and are debilitated by the pain of the fracture. Conventional fracture surgery carries very high risk and poor outcomes,⁵ and so has been reserved mostly for patients with neurologic deficits.

VERTEBROPLASTY GOES MAINSTREAM

Given these facts, investigators began looking for alternative treatments. One that rose to the fore was polymethylmethacrylate cement to stabilize the fracture. This technique, called vertebroplasty, involves injecting liquid cement through a needle into the vertebral body, where it hardens and is thought to restore stability.

Since the first description of vertebroplasty for treating symptomatic hemangiomas,⁶ many papers have been published about the procedure and about similar ones, now grouped under the general heading of vertebral augmentation. This includes kyphoplasty and other newer proprietary techniques. These procedures have been widely accepted, and their use is growing. They have shown good results in several prospective case series, and nonrandomized and randomized controlled studies have shown them to be more effective than conventional medical treatment.^7–25 For example, VERTOS, a small prospective randomized trial, showed that vertebroplasty was superior to conventional medical treatment.²⁵ When Wardlaw et al²⁴ showed that shortterm outcomes were better with kyphoplasty than with conventional medical therapy in a prospective randomized trial, many of us had moved past questioning whether vertebral augmentation is effective and were debating the relative merits of different methods and materials.

On a personal level, most of us became proponents of these procedures because we saw dramatic results—usually unequivocal. Most patients report significant improvement in pain immediately after the procedure, and many bedridden patients are able to leave the hospital within hours. In spine surgery, few procedures give such dramatic results with so few complications.

TWO NEW STUDIES UPSET ESTABLISHED BELIEF

This is why I am having such a hard time digesting the results of the trials by Buchbinder et al² and Kallmes et al,¹ published in the August 9, 2009, issue of the New England Journal of Medicine. Both were randomized controlled trials that used sham surgery rather than conventional medical treatment as the control. The sham procedure in each trial was the same as the intervention, with local anesthetic infiltration of the periosteum and mixing of the cement (so that the patients smelled its distinctive odor), but without placing the needle into the vertebra and injecting the cement.

In the study by Buchbinder et al,² the real treatment had no benefit in any primary or secondary end point. This study did not allow crossovers.

In the study by Kallmes et al,¹ more patients who received the real treatment reported clinically meaningful improvement in pain (a secondary end point), but the difference was not quite statistically significant (64% vs 48%, P = .06). In this trial, patients were allowed to cross over to the other study group after 1 month, and significantly more patients crossed over from the sham surgery group to the active treatment group than the other way around (43% vs 12%, P < .001).

My first instinct was to pick through the papers for flaws that would invalidate the results— and there were some problems. Both studies were initially planned to include more patients and therefore to have greater statistical power, but they were reassessed because of slow enrollment. In the study by Kallmes et al,¹ the difference in clinically meaningful improvement might have reached statistical significance if the trial had been larger. The study by Buchbinder et al² was a multicenter trial, but one center accounted for 53 (69%) of the 78 patients. Could this have biased the results?

The surgeon in me also seized for a while on the idea that since all of the interventions in both studies were done by interventional radiologists, the problem may have been in patient selection and that radiologists are not as astute as we are. However, even a surgeon’s ego cannot support this interpretation.

As I looked in more detail at the response I had written to these trials, I realized these criticisms were hardly fatal flaws, and the fact that two separate well-designed studies reached the same conclusion enhances their validity.

One concern that does bear some scrutiny is that the trials were too small to identify subgroups that may benefit from the procedure. In my experience, vertebral augmentation seems to have better results with certain types of fractures. Patients with a mobile pseudarthrotic cleft pattern of fracture seem to do much better than those with the more common nonmobile fracture.

THE POWERFUL PLACEBO EFFECT

Many commentaries on these two trials have discussed a famous study of a different procedure for a different condition. In this study, Moseley et al²⁶ evaluated the use of arthroscopy to treat osteoarthritis of the knee and found that sham arthroscopy was as effective as real arthroscopy and that both were better than conventional treatment.

I was not long out of my orthopedic residency when this trial was published and was very aware of the debate that preceded it, as I once had to prepare a talk about it for resident rounds. I remember that there was a lively debate in the orthopedic community over the efficacy of the procedure before the results of this trial were released.

In contrast, the vertebral augmentation controversy had become a debate about the relative efficacy and the economics of specific techniques, not about the effectiveness of the entire concept. The mainstream had accepted the validity of the procedure, which was not the case in the knee arthroscopy trial.

In both vertebroplasty studies, the activetreatment groups and the sham-treatment groups all showed significant and rapid improvement in pain and disability, and these results were maintained over the study period. Though most vertebral compression fractures do heal, the clinical improvement is usually gradual over a period of weeks. This raises the possibility that the sham treatment was actualy an active placebo.

There is some evidence to support this possibility. In a randomized trial of the efficacy of selective nerve root blocks for lumbar radiculopathy, Riew et al²⁷ showed that injection with a local anesthetic alone, although not as efficacious as a local anesthetic plus a corticosteroid at allowing patients to avoid surgery, showed an effect long after the expected duration of the anesthetic. The effect persisted even at 5 years of follow-up.²⁸

Is it possible that the local anesthetic in this trial and the vertebroplasty trials acted as some sort of “reset button” for pain sensation? This is an area that may bear further investigation.

WHERE DOES THIS LEAVE US?

So where does this leave us? On one hand, randomized controlled trials comparing vertebral augmentation with conventional medical therapy^24,25 showed augmentation to be beneficial. On the other hand, the studies by Kallmes et al¹ and Buchbinder et al² indicate vertebroplasty is no more effective than sham surgery.

It is very difficult for me to look at my own experience with vertebral augmentation and say that, on the basis of these trials, I am no longer going to offer it to my patients. I understand on an intellectual level that these trials call the efficacy of the procedure into question, but on a visceral level I cannot rationalize it. When faced with a patient who is barely ambulatory or in fact bed-bound due to pain, my experience tells me that vertebral augmentation has a very high chance of getting them ambulatory within hours. The trials of vertebroplasty would indicate this is a placebo effect or that local anesthetic alone is as effective, but I am not yet ready to make that leap.

Cognitive dissonance seems to rule.