Shoulder arthroplasty template data require careful interpretation

Proprietary templating software to guide the positioning of total shoulder arthroplasty (TSA) generate very different measures for inclination and version, according to a study that compared four programs and reported in an abstract scheduled for release at the annual meeting of the American Academy of Orthopaedic Surgeons. The meeting was canceled due to COVID-19.

“It is not a question of one software being better than another. They are just different, and they are device specific,” reported Brent B. Wiesel, MD, chief of the shoulder service at the MedStar Georgetown Orthopaedic Institute, Washington.

The variations were substantial and clinically relevant, suggesting that surgeons need to be aware of these differences when switching between the devices, according to Dr. Wiesel. He said that there is no gold standard for positioning total shoulder arthroplasty, which prevents any conclusion about the superiority of one over the other.

In this study, 76 CT scans obtained from shoulders of patients with glenohumeral arthritis were analyzed for native glenoid version and inclination by the ArthrexVIP, Tornier BluePrint, Stryker TrueSight, and ExactechGPS software programs. Dr. Wiesel explained that these are among the most commonly used programs, but there are others.

After extracting the recommended version and inclination measures from each software program, agreement between measures was calculated with an analysis of variance (ANOVA) test. The variance across programs was highly significant for both native glenoid version and inclination (P < .001).

Inter-rater reliability of the software outputs analyzed with Krippendorff’s alpha, for which a value of 1.0 signals perfect agreement and a value of 0 signals complete disagreement, reinforced the discord. For the 76 scans, the values for version and inclination were 0.272 and 0.303, respectively. Both are extremely low.

“The suggested threshold for high reliability is a value of 0.8 or greater,” said Dr. Wiesel, who was contacted about these data after the AAOS annual meeting was canceled. “The lowest acceptable limit for reliability is 0.667 or greater.”

There was disagreement across all programs. The only agreement to reach an acceptable Krippendorff’s alpha was generated by the Tornier BluePrint and Stryker TrueSight programs. These programs modestly agreed on version (0.706 on the Krippendorff’s alpha), but agreement on inclination was below the acceptable threshold.

“In other words, if you take the same scan from the same patient, you will get different angles from these different templating software programs,” Dr. Wiesel said.

There are several messages from these data, according to Dr. Wiesel. In addition to demonstrating the programs generate outputs that do not agree, he suggested that the values provided by the programs should not be considered absolute. Rather, the software values should be interpreted in the context of the individual patient.

“It is easy to get lazy, but it is important to remember that the software is a tool rather than something that will do the procedure for you,” Dr. Wiesel said. He reported that when the software guidance is not consistent with his own experience, he proceeds cautiously.

“On several occasions when the software has provided measures that are not consistent with my own perception, I have not been happy when I went with the software,” he said. “So typically I go with my gut when there is a discrepancy, and the data from this study supports that.”

Because of the difficulty in creating a gold standard for templating when there are multiple variables that influence optimal positioning of components, Dr. Wiesel suggested that “crowd thinking” might eventually determine the values that produce the best results. By crowd thinking, he was referring to Big Data analysis, collating data from a large number of cases performed by a large number of surgeons.

“All of these software programs provide reasonable guidance, but each has different advantages and disadvantages, and it is important to be aware that they are different,” Dr. Wiesel reported.

There are differences in the templating software, and they should be taken into consideration, according to another expert who has looked at this issue. Senior author of a randomized trial evaluating planning strategies for total shoulder arthroplasty ( J Bone Joint Surg AM. 2019:101;446-57), Eric T. Ricchetti, MD, an orthopedic surgeon and director of the shoulder center at the Cleveland Clinic, offered a similar perspective on templating.

“I agree that surgeons should be familiar with the differences that exist in templating software,” Dr. Ricchetti said. Basing his remarks on his own experience and reiterating the conclusion of the AAOS study, he added, “the methods that are used to identify the bone anatomy of the shoulder can vary across software programs, potentially resulting in differences in subsequent measures of glenoid pathology, such as version and inclination, that may impact surgical decision making.”

Dr. Wiesel reports no potential conflicts of interest.

SOURCE: Wiesel B et al. AAOS 2020. Abstract 212.

Proprietary templating software to guide the positioning of total shoulder arthroplasty (TSA) generate very different measures for inclination and version, according to a study that compared four programs and reported in an abstract scheduled for release at the annual meeting of the American Academy of Orthopaedic Surgeons. The meeting was canceled due to COVID-19.

“It is not a question of one software being better than another. They are just different, and they are device specific,” reported Brent B. Wiesel, MD, chief of the shoulder service at the MedStar Georgetown Orthopaedic Institute, Washington.

The variations were substantial and clinically relevant, suggesting that surgeons need to be aware of these differences when switching between the devices, according to Dr. Wiesel. He said that there is no gold standard for positioning total shoulder arthroplasty, which prevents any conclusion about the superiority of one over the other.

In this study, 76 CT scans obtained from shoulders of patients with glenohumeral arthritis were analyzed for native glenoid version and inclination by the ArthrexVIP, Tornier BluePrint, Stryker TrueSight, and ExactechGPS software programs. Dr. Wiesel explained that these are among the most commonly used programs, but there are others.

After extracting the recommended version and inclination measures from each software program, agreement between measures was calculated with an analysis of variance (ANOVA) test. The variance across programs was highly significant for both native glenoid version and inclination (P < .001).

Inter-rater reliability of the software outputs analyzed with Krippendorff’s alpha, for which a value of 1.0 signals perfect agreement and a value of 0 signals complete disagreement, reinforced the discord. For the 76 scans, the values for version and inclination were 0.272 and 0.303, respectively. Both are extremely low.

“The suggested threshold for high reliability is a value of 0.8 or greater,” said Dr. Wiesel, who was contacted about these data after the AAOS annual meeting was canceled. “The lowest acceptable limit for reliability is 0.667 or greater.”

There was disagreement across all programs. The only agreement to reach an acceptable Krippendorff’s alpha was generated by the Tornier BluePrint and Stryker TrueSight programs. These programs modestly agreed on version (0.706 on the Krippendorff’s alpha), but agreement on inclination was below the acceptable threshold.

“In other words, if you take the same scan from the same patient, you will get different angles from these different templating software programs,” Dr. Wiesel said.

There are several messages from these data, according to Dr. Wiesel. In addition to demonstrating the programs generate outputs that do not agree, he suggested that the values provided by the programs should not be considered absolute. Rather, the software values should be interpreted in the context of the individual patient.

“It is easy to get lazy, but it is important to remember that the software is a tool rather than something that will do the procedure for you,” Dr. Wiesel said. He reported that when the software guidance is not consistent with his own experience, he proceeds cautiously.

“On several occasions when the software has provided measures that are not consistent with my own perception, I have not been happy when I went with the software,” he said. “So typically I go with my gut when there is a discrepancy, and the data from this study supports that.”

Because of the difficulty in creating a gold standard for templating when there are multiple variables that influence optimal positioning of components, Dr. Wiesel suggested that “crowd thinking” might eventually determine the values that produce the best results. By crowd thinking, he was referring to Big Data analysis, collating data from a large number of cases performed by a large number of surgeons.

“All of these software programs provide reasonable guidance, but each has different advantages and disadvantages, and it is important to be aware that they are different,” Dr. Wiesel reported.

There are differences in the templating software, and they should be taken into consideration, according to another expert who has looked at this issue. Senior author of a randomized trial evaluating planning strategies for total shoulder arthroplasty ( J Bone Joint Surg AM. 2019:101;446-57), Eric T. Ricchetti, MD, an orthopedic surgeon and director of the shoulder center at the Cleveland Clinic, offered a similar perspective on templating.

“I agree that surgeons should be familiar with the differences that exist in templating software,” Dr. Ricchetti said. Basing his remarks on his own experience and reiterating the conclusion of the AAOS study, he added, “the methods that are used to identify the bone anatomy of the shoulder can vary across software programs, potentially resulting in differences in subsequent measures of glenoid pathology, such as version and inclination, that may impact surgical decision making.”

Dr. Wiesel reports no potential conflicts of interest.

SOURCE: Wiesel B et al. AAOS 2020. Abstract 212.

Proprietary templating software to guide the positioning of total shoulder arthroplasty (TSA) generate very different measures for inclination and version, according to a study that compared four programs and reported in an abstract scheduled for release at the annual meeting of the American Academy of Orthopaedic Surgeons. The meeting was canceled due to COVID-19.

“It is not a question of one software being better than another. They are just different, and they are device specific,” reported Brent B. Wiesel, MD, chief of the shoulder service at the MedStar Georgetown Orthopaedic Institute, Washington.

The variations were substantial and clinically relevant, suggesting that surgeons need to be aware of these differences when switching between the devices, according to Dr. Wiesel. He said that there is no gold standard for positioning total shoulder arthroplasty, which prevents any conclusion about the superiority of one over the other.

In this study, 76 CT scans obtained from shoulders of patients with glenohumeral arthritis were analyzed for native glenoid version and inclination by the ArthrexVIP, Tornier BluePrint, Stryker TrueSight, and ExactechGPS software programs. Dr. Wiesel explained that these are among the most commonly used programs, but there are others.

After extracting the recommended version and inclination measures from each software program, agreement between measures was calculated with an analysis of variance (ANOVA) test. The variance across programs was highly significant for both native glenoid version and inclination (P < .001).

Inter-rater reliability of the software outputs analyzed with Krippendorff’s alpha, for which a value of 1.0 signals perfect agreement and a value of 0 signals complete disagreement, reinforced the discord. For the 76 scans, the values for version and inclination were 0.272 and 0.303, respectively. Both are extremely low.

“The suggested threshold for high reliability is a value of 0.8 or greater,” said Dr. Wiesel, who was contacted about these data after the AAOS annual meeting was canceled. “The lowest acceptable limit for reliability is 0.667 or greater.”

There was disagreement across all programs. The only agreement to reach an acceptable Krippendorff’s alpha was generated by the Tornier BluePrint and Stryker TrueSight programs. These programs modestly agreed on version (0.706 on the Krippendorff’s alpha), but agreement on inclination was below the acceptable threshold.

“In other words, if you take the same scan from the same patient, you will get different angles from these different templating software programs,” Dr. Wiesel said.

There are several messages from these data, according to Dr. Wiesel. In addition to demonstrating the programs generate outputs that do not agree, he suggested that the values provided by the programs should not be considered absolute. Rather, the software values should be interpreted in the context of the individual patient.

“It is easy to get lazy, but it is important to remember that the software is a tool rather than something that will do the procedure for you,” Dr. Wiesel said. He reported that when the software guidance is not consistent with his own experience, he proceeds cautiously.

“On several occasions when the software has provided measures that are not consistent with my own perception, I have not been happy when I went with the software,” he said. “So typically I go with my gut when there is a discrepancy, and the data from this study supports that.”

Because of the difficulty in creating a gold standard for templating when there are multiple variables that influence optimal positioning of components, Dr. Wiesel suggested that “crowd thinking” might eventually determine the values that produce the best results. By crowd thinking, he was referring to Big Data analysis, collating data from a large number of cases performed by a large number of surgeons.

“All of these software programs provide reasonable guidance, but each has different advantages and disadvantages, and it is important to be aware that they are different,” Dr. Wiesel reported.

There are differences in the templating software, and they should be taken into consideration, according to another expert who has looked at this issue. Senior author of a randomized trial evaluating planning strategies for total shoulder arthroplasty ( J Bone Joint Surg AM. 2019:101;446-57), Eric T. Ricchetti, MD, an orthopedic surgeon and director of the shoulder center at the Cleveland Clinic, offered a similar perspective on templating.

“I agree that surgeons should be familiar with the differences that exist in templating software,” Dr. Ricchetti said. Basing his remarks on his own experience and reiterating the conclusion of the AAOS study, he added, “the methods that are used to identify the bone anatomy of the shoulder can vary across software programs, potentially resulting in differences in subsequent measures of glenoid pathology, such as version and inclination, that may impact surgical decision making.”

Dr. Wiesel reports no potential conflicts of interest.

SOURCE: Wiesel B et al. AAOS 2020. Abstract 212.