Atilio Barbeito, MD, MPH

Days or weeks before a scheduled surgical or invasive procedure involving anesthesia, evaluations are conducted to assess a patient’s condition and risk, optimize their status, and prepare them for their procedure. A comprehensive pre-anesthesia evaluation visit includes a history of present illness, the evaluation of comorbidities and medication use, the assessment of health habits such as alcohol and tobacco use, functional capacity and nutritional assessments, and the identification of social support deficiencies that may influence recovery. It also includes a focused physical examination and laboratory and other ancillary testing as needed and may include optimization interventions such as anemia management or prehabilitation. Conducting pre-anesthesia evaluations before surgery has been shown to reduce delays and cancellations, unnecessary preprocedure testing, hospital length of stay, and in-hospital mortality.^1-4

The pre-anesthesia evaluation is usually conducted in person, although other modalities have been in use for several years and have accelerated since the advent of the COVID-19 pandemic. Specifically, audio-only telephone visits are used in many settings to conduct abbreviated forms of a pre-anesthesia evaluation, typically for less-invasive procedures. When patients are evaluated over the telephone, the physical examination and testing are deferred until the day of the procedure. Another modality is the use of synchronous video telehealth. Emerging evidence for the use of video-based care in anesthesiology provides encouraging results. Several institutions have proven the technological feasibility of performing preoperative evaluations via video.^5,6 Compared with in-person evaluations, these visits seem to have similar surgery cancellation rates, improved patient satisfaction, and reduced wait times and costs.^7-9

As part of a quality improvement project, we studied the use of telehealth for pre-anesthesia evaluations within the US Department of Veterans Affairs (VA). An internal review found overall low utilization of these modalities before the COVID-19 pandemic that accelerated toward telehealth during the pandemic: The largest uptake was with telephone visits. Given the increasing adoption of telehealth for pre-anesthesia evaluations and the marked preference for telephone over video modalities among VA practitioners during the COVID-19 pandemic, we sought to understand the barriers and facilitators to the adoption of telephone- and video-based pre-anesthesia evaluation visits within the VA.

Methods

Our objective was to assess health care practitioners’ (HCPs) preferences regarding pre-anesthesia evaluation modalities (in-person, telephone, or video), and the perceived advantages and barriers to adoption for each modality. We followed the Strengthening the Reporting of Observational studies in Epidemiology (STROBE) guideline and Checklist for statistical Assessment of Medical Papers (CHAMP) statement.^10,11 The survey was deemed a quality improvement activity that was exempt from institutional review board oversight by the VA National Anesthesia Program Office and the VA Office of Connected Care.

A survey was distributed to all VA anesthesiology service chiefs via email between April 27, 2022, and May 3, 2022. Three emails were sent to each participant (initial invitation and 2 reminders). The respondents were asked to identify themselves by facility and role and to indicate whether their anesthesiology service performed any pre-anesthesia evaluations, including any telephone- or video-based evaluations; and whether their service has a dedicated pre-anesthesia evaluation clinic.

A second set of questions referred to the use of telephone- and video-based preprocedure evaluations. The questions were based on branch logic and depended on the respondent’s answers concerning their use of telephone- and video-based evaluations. Questions included statements about perceived barriers to the adoption of these pre-anesthesia evaluation modalities. Each item was rated on a 5-point Likert scale, (completely disagree [1] to completely agree [5]). A third section measured acceptability and feasibility of video using the validated Acceptability of Intervention Measure (AIM) and Feasibility of Intervention Measure (FIM)questionnaires.¹² These instruments are 4-item measures of implementation outcomes that are often considered indicators of implementation success.¹³Acceptability is the perception among implementation stakeholders that a given treatment, service, practice, or innovation is agreeable, palatable, or satisfactory. Feasibility is defined as the extent to which a new treatment or an innovation can be successfully used or carried out within a given agency or setting.¹³ The criterion for acceptability is personal, meaning that different HCPs may have differing needs, preferences, and expectations regarding the same intervention. The criterion for feasibility is practical. An intervention may be considered to be feasible if the required tasks can be performed easily or conveniently. Finally, 2 open-ended questions allowed respondents to identify the most important factor that allowed the implementation of telehealth for pre-anesthesia evaluations in their service, and provide comments about the use of telehealth for pre-anesthesia evaluations at the VA. All questions were developed by the authors except for the 2 implementation measure instruments.

The survey was administered using an electronic survey platform (Qualtrics, version April 2022) and sent by email alongside a brief introductory video. Participation was voluntary and anonymous, as no personal information was collected. Responses were attributed to each facility, using the self-declared affiliation. When an affiliation was not provided, we deduced it using the latitude/longitude of the respondent, a feature included in the survey software. No incentives were provided. Data were stored and maintained in a secure VA server. All completed surveys were included. Some facilities had > 1 complete response, and all were included. Facilities that provided > 1 response and where responses were discordant, we clarified with the facility service chief. Incomplete responses were excluded from the analysis.

 

Statistics

For this analysis, the 2 positive sentiment responses (agree and completely agree) and the 2 negative sentiment responses (disagree and completely disagree) in the Likert scale were collapsed into single categories (good and poor, respectively). The neither agree nor disagree responses were coded as neutral. Our analysis began with a visual exploration of all variables to evaluate the frequency, percentage, and near-zero variance for categorical variables.¹⁴ Near-zero variance occurs when a categorical variable has a low frequency of unique values over the sample size (ie, the variable is almost constant), and we addressed it by combining different variable categorizations. We handled missing values through imputation algorithms followed by sensitivity analyses to verify whether our results were stable with and without imputation. We performed comparisons for the exploratory analysis using P values for one-way analysis of variance tests for numeric variables and χ²tests for categorical variables. We considered P values < .05 to be statistically significant. We also used correlation matrices and plots as exploratory analysis tools to better understand all items’ correlations. We used Pearson, polychoric, and polyserial correlation tests as appropriate for numeric, ordinal, and logical items.

Our modeling strategy involved a series of generalized linear models (GLMs) with a Gaussian family, ie, multiple linear regression models, to assess the association between (1) facilities’ preferences regarding pre-anesthesia evaluation modalities; (2) advantages between modalities; and (3) barriers to the adoption of telehealth and the ability to perform different pre-anesthesia evaluation-related tasks. In addition, we used backward deletion to reach the most parsimonious model based on a series of likelihood-ratio tests comparing nested models. Results are reported as predicted means with 95% confidence intervals, with results being interpreted as significant when any 2 predicted means do not overlap between different estimates along with P for trends < .001. We performed all analyses using the R language.¹⁵

Results

Of 109 surveyed facilities, 50 (46%) responded to the survey. The final study sample included 67 responses, and 55 were included in the analysis. Twelve responses were excluded from the analysis as they were either incomplete or test responses. Three facilities had > 1 complete response (2 facilities had 2 responses and 1 facility had 4 responses), and these were all included in the analysis.

Thirty-six locations were complex inpatient facilities, and 32 (89%) had pre-anesthesia evaluation clinics (Table 1).

Twenty-two facilities reported using both telephone and video, 11 telephone only, 5 video only, and 12 neither. Considering the 55 individual responses, 25 respondents reported using both telephone and video, 12 reported using telephone only, 5 using video only, and 13 reported using neither telephone nor video for pre-anesthesia evaluations.

The ability to obtain a history of present illness was rated good/very good via telephone for 34 respondents (92%) and 25 for video (86%). Assessing comorbidities and health habits was rated good/very good via telephone for 32 respondents (89%) and 31 respondents (86%), respectively, and via video for 24 respondents (83%) and 23 respondents (79%), respectively (Figure 1).

Fewer respondents rated the ability to estimate exercise capacity or mental health pathology good/very good: 26 respondents (70%) and 23 respondents (62%) for telephone, respectively, and 18 (62%) and 17 (59%) for video, respectively. The ability to assess nutritional status was rated lowest with 9 respondents (24%) rating it positively for telephone and 15 (52%) for video.

To compare differences between the 2 remote pre-anesthesia evaluation modalities, we created GLMs evaluating the association between each modality and the perceived ability to perform the tasks. For GLMs, we transformed the values of the categories into numerical (ie, 1, poor; 2, neutral; 3, good). Compared with telephone, video was rated more favorably regarding the assessment of nutritional status (mean, 2.1; 95% CI, 1.8-2.3 vs mean, 2.4; 95% CI, 2.2-2.7; P = .04) (eAppendix 1, available at doi:10.12788/fp.0387). No other significant differences in ratings existed between the 2 remote pre-anesthesia evaluation modalities.

The most significant barriers (cited as significant or very significant in the survey) included the inability to perform a physical examination, which was noted by 13 respondents (72%) and 15 respondents (60%) for telephone and video, respectively. The inability to obtain vital signs was rated as a significant barrier for telephone by 12 respondents (67%) and for video by 15 respondents (60%)(Figure 2).

Other less-cited barriers included concerns about patient safety and risk; patient preference; cultural barriers; lack of support from staff; and lack of evidence for its effectiveness. Specific to video care, patients’ lack of access to a computer was cited as a barrier by 12 respondents (48%), whereas only 3 (17%) cited lack of telephone as a barrier. Lastly, lack of information technology support was cited as a barrier for video visits by 8 respondents (32%). To determine differences in perceived barriers to the implementation of phone vs video pre-anesthesia evaluations, we created GLM evaluating the association between these 2 modalities and the perceived ability to perform commonly performed pre-anesthesia evaluation visit tasks. For GLM, again we transformed the values of the categories into numeric (ie, not a significant barrier, 1; somewhat a barrier, 2; a significant barrier, 3). There were no significant differences in ratings between the 2 remote pre-anesthesia evaluation modalities (eAppendix 2, available at doi:10.12788/fp.0387).

The average FIM score was 3.7, with the highest score among respondents who used both phone and video (Table 2). The average AIM score was 3.4, with the highest score among respondents who used both telehealth modalities. The internal consistency of the implementation measures was excellent (Cronbach’s α 0.95 and 0.975 for FIM and AIM, respectively).

 

Discussion

We surveyed 109 anesthesiology services across the VA regarding barriers to implementing telephone- and video-based pre-anesthesia evaluation visits. We found that 12 (23%) of the 50 anesthesiology services responding to this survey still conduct the totality of their pre-anesthesia evaluations in person. This represents an opportunity to further disseminate the appropriate use of telehealth and potentially reduce travel time, costs, and low-value testing, as it is well established that remote pre-anesthesia evaluations for low-risk procedures are safe and effective.⁶

We also found no difference between telephone and video regarding users’ perceived ability to perform any of the basic pre-anesthesia evaluation tasks except for assessing patients’ nutritional status, which was rated as easier using video than telephone. According to those not using telephone and/or video, the biggest barriers to implementation of telehealth visits were the inability to obtain vital signs and to perform a physical examination. This finding was unexpected, as facilities that conduct remote evaluations typically defer these tasks to the day of surgery, a practice that has been well established and shown to be safe and efficient. Respondents also identified patient-level factors (eg, patient preference, lack of telephone or computer) as significant barriers. Finally, feasibility ratings were higher than acceptability ratings with regards to the implementation of telehealth.

In 2004, the first use of telehealth for pre-anesthesia evaluations was reported by Wong and colleagues.¹⁶ Since then, several case series and a literature review have documented the efficacy, safety, and patient and HCP satisfaction with the use of telehealth for pre-anesthesia evaluations. A study by Mullen-Fortino and colleagues showed reduced visit times when telehealth was used for pre-anesthesia evaluation.⁸ Another study at VA hospitals showed that 88% of veterans reported that telemedicine saved them time and money.¹⁷ A report of 35 patients in rural Australia reported 98% satisfaction with the video quality of the visit, 95% perceived efficacy, and 87% preference for telehealth compared with driving to be seen in person.¹⁸ These reports conflict with the perceptions of the respondents of our survey, who identified patient preference as an important barrier to adoption of telehealth. Given these findings, research is needed on veterans’ perceptions on the use of telehealth modalities for pre-anesthesia evaluations; if their perceptions are similarly favorable, it will be important to communicate this information to HCPs and leadership, which may help increase subsequent telehealth adoption.

Despite the reported safety, efficacy, and high satisfaction of video visits among anesthesiology teams conducting pre-anesthesia evaluations, its use remains low at VA. We have found that most facilities in the VA system chose telephone platforms during the COVID-19 pandemic. One possibility is that the adoption of video modalities among pre-anesthesia evaluation clinics in the VA system is resource intensive or difficult from the HCP’s perspective. When combined with the lack of perceived advantages over telephone as we found in our survey, most practitioners resort to the technologically less demanding and more familiar telephone platform. The results from FIM and AIM support this. While both telephone and video have high feasibility scores, acceptability scores are lower for video, even among those currently using this technology. Our findings do not rule out the utility of video-based care in perioperative medicine. Rather than a yes/no proposition, future studies need to establish the precise indications for video for pre-anesthesia evaluations; that is, situations where video visits offer an advantage over telephone. For example, video could be used to deliver preoperative optimization therapies, such as supervised exercise or mental health interventions or to guide the achievement of certain milestones before surgery in patients with chronic conditions, such as target glucose values or the treatment of anemia. Future studies should explore the perceived benefits of video over telephone among centers offering these more advanced optimization interventions.

Limitations

We received responses from a subset of VA anesthesiology services; therefore, they may not be representative of the entire VA system. Facilities designated by the VA as inpatient complex were overrepresented (72% of our sample vs 50% of the total facilities nationally), and ambulatory centers (those designed by the VA as ambulatory procedural center with basic or advanced capabilities) were underrepresented (2% of our sample vs 22% nationally). Despite this, the response rate was high, and no geographic area appeared to be underrepresented. In addition, we surveyed pre-anesthesia evaluation facilities led by anesthesiologists, and the results may not be representative of the preferences of HCPs working in nonanesthesiology led pre-anesthesia evaluation clinics. Finally, just 11 facilities used both telephone and video; therefore, a true direct comparison between these 2 platforms was limited. The VA serves a unique patient population, and the findings may not be completely applicable to the non-VA population.

Conclusions

We found no significant perceived advantages of video over telephone in the ability to conduct routine pre-anesthesia evaluations among a sample of anesthesiology HCPs in the VA except for the perceived ability to assess nutritional status. HCPs with no telehealth experience cited the inability to perform a physical examination and obtain vital signs as the most significant barriers to implementation. Respondents not using telephone cited concerns about safety. Video visits in this clinical setting had additional perceived barriers to implementation, such as lack of information technology and staff support and patient-level barriers. Video had lower acceptability by HCPs. Given findings that pre-anesthesia evaluations can be conducted effectively via telehealth and have high levels of patient satisfaction, future work should focus on increasing uptake of these remote modalities. Additionally, research on the most appropriate uses of video visits within perioperative care is also needed.

Days or weeks before a scheduled surgical or invasive procedure involving anesthesia, evaluations are conducted to assess a patient’s condition and risk, optimize their status, and prepare them for their procedure. A comprehensive pre-anesthesia evaluation visit includes a history of present illness, the evaluation of comorbidities and medication use, the assessment of health habits such as alcohol and tobacco use, functional capacity and nutritional assessments, and the identification of social support deficiencies that may influence recovery. It also includes a focused physical examination and laboratory and other ancillary testing as needed and may include optimization interventions such as anemia management or prehabilitation. Conducting pre-anesthesia evaluations before surgery has been shown to reduce delays and cancellations, unnecessary preprocedure testing, hospital length of stay, and in-hospital mortality.^1-4

The pre-anesthesia evaluation is usually conducted in person, although other modalities have been in use for several years and have accelerated since the advent of the COVID-19 pandemic. Specifically, audio-only telephone visits are used in many settings to conduct abbreviated forms of a pre-anesthesia evaluation, typically for less-invasive procedures. When patients are evaluated over the telephone, the physical examination and testing are deferred until the day of the procedure. Another modality is the use of synchronous video telehealth. Emerging evidence for the use of video-based care in anesthesiology provides encouraging results. Several institutions have proven the technological feasibility of performing preoperative evaluations via video.^5,6 Compared with in-person evaluations, these visits seem to have similar surgery cancellation rates, improved patient satisfaction, and reduced wait times and costs.^7-9

As part of a quality improvement project, we studied the use of telehealth for pre-anesthesia evaluations within the US Department of Veterans Affairs (VA). An internal review found overall low utilization of these modalities before the COVID-19 pandemic that accelerated toward telehealth during the pandemic: The largest uptake was with telephone visits. Given the increasing adoption of telehealth for pre-anesthesia evaluations and the marked preference for telephone over video modalities among VA practitioners during the COVID-19 pandemic, we sought to understand the barriers and facilitators to the adoption of telephone- and video-based pre-anesthesia evaluation visits within the VA.

Methods

Our objective was to assess health care practitioners’ (HCPs) preferences regarding pre-anesthesia evaluation modalities (in-person, telephone, or video), and the perceived advantages and barriers to adoption for each modality. We followed the Strengthening the Reporting of Observational studies in Epidemiology (STROBE) guideline and Checklist for statistical Assessment of Medical Papers (CHAMP) statement.^10,11 The survey was deemed a quality improvement activity that was exempt from institutional review board oversight by the VA National Anesthesia Program Office and the VA Office of Connected Care.

A survey was distributed to all VA anesthesiology service chiefs via email between April 27, 2022, and May 3, 2022. Three emails were sent to each participant (initial invitation and 2 reminders). The respondents were asked to identify themselves by facility and role and to indicate whether their anesthesiology service performed any pre-anesthesia evaluations, including any telephone- or video-based evaluations; and whether their service has a dedicated pre-anesthesia evaluation clinic.

A second set of questions referred to the use of telephone- and video-based preprocedure evaluations. The questions were based on branch logic and depended on the respondent’s answers concerning their use of telephone- and video-based evaluations. Questions included statements about perceived barriers to the adoption of these pre-anesthesia evaluation modalities. Each item was rated on a 5-point Likert scale, (completely disagree [1] to completely agree [5]). A third section measured acceptability and feasibility of video using the validated Acceptability of Intervention Measure (AIM) and Feasibility of Intervention Measure (FIM)questionnaires.¹² These instruments are 4-item measures of implementation outcomes that are often considered indicators of implementation success.¹³Acceptability is the perception among implementation stakeholders that a given treatment, service, practice, or innovation is agreeable, palatable, or satisfactory. Feasibility is defined as the extent to which a new treatment or an innovation can be successfully used or carried out within a given agency or setting.¹³ The criterion for acceptability is personal, meaning that different HCPs may have differing needs, preferences, and expectations regarding the same intervention. The criterion for feasibility is practical. An intervention may be considered to be feasible if the required tasks can be performed easily or conveniently. Finally, 2 open-ended questions allowed respondents to identify the most important factor that allowed the implementation of telehealth for pre-anesthesia evaluations in their service, and provide comments about the use of telehealth for pre-anesthesia evaluations at the VA. All questions were developed by the authors except for the 2 implementation measure instruments.

The survey was administered using an electronic survey platform (Qualtrics, version April 2022) and sent by email alongside a brief introductory video. Participation was voluntary and anonymous, as no personal information was collected. Responses were attributed to each facility, using the self-declared affiliation. When an affiliation was not provided, we deduced it using the latitude/longitude of the respondent, a feature included in the survey software. No incentives were provided. Data were stored and maintained in a secure VA server. All completed surveys were included. Some facilities had > 1 complete response, and all were included. Facilities that provided > 1 response and where responses were discordant, we clarified with the facility service chief. Incomplete responses were excluded from the analysis.

 

Statistics

For this analysis, the 2 positive sentiment responses (agree and completely agree) and the 2 negative sentiment responses (disagree and completely disagree) in the Likert scale were collapsed into single categories (good and poor, respectively). The neither agree nor disagree responses were coded as neutral. Our analysis began with a visual exploration of all variables to evaluate the frequency, percentage, and near-zero variance for categorical variables.¹⁴ Near-zero variance occurs when a categorical variable has a low frequency of unique values over the sample size (ie, the variable is almost constant), and we addressed it by combining different variable categorizations. We handled missing values through imputation algorithms followed by sensitivity analyses to verify whether our results were stable with and without imputation. We performed comparisons for the exploratory analysis using P values for one-way analysis of variance tests for numeric variables and χ²tests for categorical variables. We considered P values < .05 to be statistically significant. We also used correlation matrices and plots as exploratory analysis tools to better understand all items’ correlations. We used Pearson, polychoric, and polyserial correlation tests as appropriate for numeric, ordinal, and logical items.

Our modeling strategy involved a series of generalized linear models (GLMs) with a Gaussian family, ie, multiple linear regression models, to assess the association between (1) facilities’ preferences regarding pre-anesthesia evaluation modalities; (2) advantages between modalities; and (3) barriers to the adoption of telehealth and the ability to perform different pre-anesthesia evaluation-related tasks. In addition, we used backward deletion to reach the most parsimonious model based on a series of likelihood-ratio tests comparing nested models. Results are reported as predicted means with 95% confidence intervals, with results being interpreted as significant when any 2 predicted means do not overlap between different estimates along with P for trends < .001. We performed all analyses using the R language.¹⁵

Results

Of 109 surveyed facilities, 50 (46%) responded to the survey. The final study sample included 67 responses, and 55 were included in the analysis. Twelve responses were excluded from the analysis as they were either incomplete or test responses. Three facilities had > 1 complete response (2 facilities had 2 responses and 1 facility had 4 responses), and these were all included in the analysis.

Thirty-six locations were complex inpatient facilities, and 32 (89%) had pre-anesthesia evaluation clinics (Table 1).

Twenty-two facilities reported using both telephone and video, 11 telephone only, 5 video only, and 12 neither. Considering the 55 individual responses, 25 respondents reported using both telephone and video, 12 reported using telephone only, 5 using video only, and 13 reported using neither telephone nor video for pre-anesthesia evaluations.

The ability to obtain a history of present illness was rated good/very good via telephone for 34 respondents (92%) and 25 for video (86%). Assessing comorbidities and health habits was rated good/very good via telephone for 32 respondents (89%) and 31 respondents (86%), respectively, and via video for 24 respondents (83%) and 23 respondents (79%), respectively (Figure 1).

Fewer respondents rated the ability to estimate exercise capacity or mental health pathology good/very good: 26 respondents (70%) and 23 respondents (62%) for telephone, respectively, and 18 (62%) and 17 (59%) for video, respectively. The ability to assess nutritional status was rated lowest with 9 respondents (24%) rating it positively for telephone and 15 (52%) for video.

To compare differences between the 2 remote pre-anesthesia evaluation modalities, we created GLMs evaluating the association between each modality and the perceived ability to perform the tasks. For GLMs, we transformed the values of the categories into numerical (ie, 1, poor; 2, neutral; 3, good). Compared with telephone, video was rated more favorably regarding the assessment of nutritional status (mean, 2.1; 95% CI, 1.8-2.3 vs mean, 2.4; 95% CI, 2.2-2.7; P = .04) (eAppendix 1, available at doi:10.12788/fp.0387). No other significant differences in ratings existed between the 2 remote pre-anesthesia evaluation modalities.

The most significant barriers (cited as significant or very significant in the survey) included the inability to perform a physical examination, which was noted by 13 respondents (72%) and 15 respondents (60%) for telephone and video, respectively. The inability to obtain vital signs was rated as a significant barrier for telephone by 12 respondents (67%) and for video by 15 respondents (60%)(Figure 2).

Other less-cited barriers included concerns about patient safety and risk; patient preference; cultural barriers; lack of support from staff; and lack of evidence for its effectiveness. Specific to video care, patients’ lack of access to a computer was cited as a barrier by 12 respondents (48%), whereas only 3 (17%) cited lack of telephone as a barrier. Lastly, lack of information technology support was cited as a barrier for video visits by 8 respondents (32%). To determine differences in perceived barriers to the implementation of phone vs video pre-anesthesia evaluations, we created GLM evaluating the association between these 2 modalities and the perceived ability to perform commonly performed pre-anesthesia evaluation visit tasks. For GLM, again we transformed the values of the categories into numeric (ie, not a significant barrier, 1; somewhat a barrier, 2; a significant barrier, 3). There were no significant differences in ratings between the 2 remote pre-anesthesia evaluation modalities (eAppendix 2, available at doi:10.12788/fp.0387).

The average FIM score was 3.7, with the highest score among respondents who used both phone and video (Table 2). The average AIM score was 3.4, with the highest score among respondents who used both telehealth modalities. The internal consistency of the implementation measures was excellent (Cronbach’s α 0.95 and 0.975 for FIM and AIM, respectively).

 

Discussion

We surveyed 109 anesthesiology services across the VA regarding barriers to implementing telephone- and video-based pre-anesthesia evaluation visits. We found that 12 (23%) of the 50 anesthesiology services responding to this survey still conduct the totality of their pre-anesthesia evaluations in person. This represents an opportunity to further disseminate the appropriate use of telehealth and potentially reduce travel time, costs, and low-value testing, as it is well established that remote pre-anesthesia evaluations for low-risk procedures are safe and effective.⁶

We also found no difference between telephone and video regarding users’ perceived ability to perform any of the basic pre-anesthesia evaluation tasks except for assessing patients’ nutritional status, which was rated as easier using video than telephone. According to those not using telephone and/or video, the biggest barriers to implementation of telehealth visits were the inability to obtain vital signs and to perform a physical examination. This finding was unexpected, as facilities that conduct remote evaluations typically defer these tasks to the day of surgery, a practice that has been well established and shown to be safe and efficient. Respondents also identified patient-level factors (eg, patient preference, lack of telephone or computer) as significant barriers. Finally, feasibility ratings were higher than acceptability ratings with regards to the implementation of telehealth.

In 2004, the first use of telehealth for pre-anesthesia evaluations was reported by Wong and colleagues.¹⁶ Since then, several case series and a literature review have documented the efficacy, safety, and patient and HCP satisfaction with the use of telehealth for pre-anesthesia evaluations. A study by Mullen-Fortino and colleagues showed reduced visit times when telehealth was used for pre-anesthesia evaluation.⁸ Another study at VA hospitals showed that 88% of veterans reported that telemedicine saved them time and money.¹⁷ A report of 35 patients in rural Australia reported 98% satisfaction with the video quality of the visit, 95% perceived efficacy, and 87% preference for telehealth compared with driving to be seen in person.¹⁸ These reports conflict with the perceptions of the respondents of our survey, who identified patient preference as an important barrier to adoption of telehealth. Given these findings, research is needed on veterans’ perceptions on the use of telehealth modalities for pre-anesthesia evaluations; if their perceptions are similarly favorable, it will be important to communicate this information to HCPs and leadership, which may help increase subsequent telehealth adoption.

Despite the reported safety, efficacy, and high satisfaction of video visits among anesthesiology teams conducting pre-anesthesia evaluations, its use remains low at VA. We have found that most facilities in the VA system chose telephone platforms during the COVID-19 pandemic. One possibility is that the adoption of video modalities among pre-anesthesia evaluation clinics in the VA system is resource intensive or difficult from the HCP’s perspective. When combined with the lack of perceived advantages over telephone as we found in our survey, most practitioners resort to the technologically less demanding and more familiar telephone platform. The results from FIM and AIM support this. While both telephone and video have high feasibility scores, acceptability scores are lower for video, even among those currently using this technology. Our findings do not rule out the utility of video-based care in perioperative medicine. Rather than a yes/no proposition, future studies need to establish the precise indications for video for pre-anesthesia evaluations; that is, situations where video visits offer an advantage over telephone. For example, video could be used to deliver preoperative optimization therapies, such as supervised exercise or mental health interventions or to guide the achievement of certain milestones before surgery in patients with chronic conditions, such as target glucose values or the treatment of anemia. Future studies should explore the perceived benefits of video over telephone among centers offering these more advanced optimization interventions.

Limitations

We received responses from a subset of VA anesthesiology services; therefore, they may not be representative of the entire VA system. Facilities designated by the VA as inpatient complex were overrepresented (72% of our sample vs 50% of the total facilities nationally), and ambulatory centers (those designed by the VA as ambulatory procedural center with basic or advanced capabilities) were underrepresented (2% of our sample vs 22% nationally). Despite this, the response rate was high, and no geographic area appeared to be underrepresented. In addition, we surveyed pre-anesthesia evaluation facilities led by anesthesiologists, and the results may not be representative of the preferences of HCPs working in nonanesthesiology led pre-anesthesia evaluation clinics. Finally, just 11 facilities used both telephone and video; therefore, a true direct comparison between these 2 platforms was limited. The VA serves a unique patient population, and the findings may not be completely applicable to the non-VA population.

Conclusions

We found no significant perceived advantages of video over telephone in the ability to conduct routine pre-anesthesia evaluations among a sample of anesthesiology HCPs in the VA except for the perceived ability to assess nutritional status. HCPs with no telehealth experience cited the inability to perform a physical examination and obtain vital signs as the most significant barriers to implementation. Respondents not using telephone cited concerns about safety. Video visits in this clinical setting had additional perceived barriers to implementation, such as lack of information technology and staff support and patient-level barriers. Video had lower acceptability by HCPs. Given findings that pre-anesthesia evaluations can be conducted effectively via telehealth and have high levels of patient satisfaction, future work should focus on increasing uptake of these remote modalities. Additionally, research on the most appropriate uses of video visits within perioperative care is also needed.

Days or weeks before a scheduled surgical or invasive procedure involving anesthesia, evaluations are conducted to assess a patient’s condition and risk, optimize their status, and prepare them for their procedure. A comprehensive pre-anesthesia evaluation visit includes a history of present illness, the evaluation of comorbidities and medication use, the assessment of health habits such as alcohol and tobacco use, functional capacity and nutritional assessments, and the identification of social support deficiencies that may influence recovery. It also includes a focused physical examination and laboratory and other ancillary testing as needed and may include optimization interventions such as anemia management or prehabilitation. Conducting pre-anesthesia evaluations before surgery has been shown to reduce delays and cancellations, unnecessary preprocedure testing, hospital length of stay, and in-hospital mortality.^1-4

The pre-anesthesia evaluation is usually conducted in person, although other modalities have been in use for several years and have accelerated since the advent of the COVID-19 pandemic. Specifically, audio-only telephone visits are used in many settings to conduct abbreviated forms of a pre-anesthesia evaluation, typically for less-invasive procedures. When patients are evaluated over the telephone, the physical examination and testing are deferred until the day of the procedure. Another modality is the use of synchronous video telehealth. Emerging evidence for the use of video-based care in anesthesiology provides encouraging results. Several institutions have proven the technological feasibility of performing preoperative evaluations via video.^5,6 Compared with in-person evaluations, these visits seem to have similar surgery cancellation rates, improved patient satisfaction, and reduced wait times and costs.^7-9

As part of a quality improvement project, we studied the use of telehealth for pre-anesthesia evaluations within the US Department of Veterans Affairs (VA). An internal review found overall low utilization of these modalities before the COVID-19 pandemic that accelerated toward telehealth during the pandemic: The largest uptake was with telephone visits. Given the increasing adoption of telehealth for pre-anesthesia evaluations and the marked preference for telephone over video modalities among VA practitioners during the COVID-19 pandemic, we sought to understand the barriers and facilitators to the adoption of telephone- and video-based pre-anesthesia evaluation visits within the VA.

Methods

Our objective was to assess health care practitioners’ (HCPs) preferences regarding pre-anesthesia evaluation modalities (in-person, telephone, or video), and the perceived advantages and barriers to adoption for each modality. We followed the Strengthening the Reporting of Observational studies in Epidemiology (STROBE) guideline and Checklist for statistical Assessment of Medical Papers (CHAMP) statement.^10,11 The survey was deemed a quality improvement activity that was exempt from institutional review board oversight by the VA National Anesthesia Program Office and the VA Office of Connected Care.

A survey was distributed to all VA anesthesiology service chiefs via email between April 27, 2022, and May 3, 2022. Three emails were sent to each participant (initial invitation and 2 reminders). The respondents were asked to identify themselves by facility and role and to indicate whether their anesthesiology service performed any pre-anesthesia evaluations, including any telephone- or video-based evaluations; and whether their service has a dedicated pre-anesthesia evaluation clinic.

A second set of questions referred to the use of telephone- and video-based preprocedure evaluations. The questions were based on branch logic and depended on the respondent’s answers concerning their use of telephone- and video-based evaluations. Questions included statements about perceived barriers to the adoption of these pre-anesthesia evaluation modalities. Each item was rated on a 5-point Likert scale, (completely disagree [1] to completely agree [5]). A third section measured acceptability and feasibility of video using the validated Acceptability of Intervention Measure (AIM) and Feasibility of Intervention Measure (FIM)questionnaires.¹² These instruments are 4-item measures of implementation outcomes that are often considered indicators of implementation success.¹³Acceptability is the perception among implementation stakeholders that a given treatment, service, practice, or innovation is agreeable, palatable, or satisfactory. Feasibility is defined as the extent to which a new treatment or an innovation can be successfully used or carried out within a given agency or setting.¹³ The criterion for acceptability is personal, meaning that different HCPs may have differing needs, preferences, and expectations regarding the same intervention. The criterion for feasibility is practical. An intervention may be considered to be feasible if the required tasks can be performed easily or conveniently. Finally, 2 open-ended questions allowed respondents to identify the most important factor that allowed the implementation of telehealth for pre-anesthesia evaluations in their service, and provide comments about the use of telehealth for pre-anesthesia evaluations at the VA. All questions were developed by the authors except for the 2 implementation measure instruments.

The survey was administered using an electronic survey platform (Qualtrics, version April 2022) and sent by email alongside a brief introductory video. Participation was voluntary and anonymous, as no personal information was collected. Responses were attributed to each facility, using the self-declared affiliation. When an affiliation was not provided, we deduced it using the latitude/longitude of the respondent, a feature included in the survey software. No incentives were provided. Data were stored and maintained in a secure VA server. All completed surveys were included. Some facilities had > 1 complete response, and all were included. Facilities that provided > 1 response and where responses were discordant, we clarified with the facility service chief. Incomplete responses were excluded from the analysis.

 

Statistics

For this analysis, the 2 positive sentiment responses (agree and completely agree) and the 2 negative sentiment responses (disagree and completely disagree) in the Likert scale were collapsed into single categories (good and poor, respectively). The neither agree nor disagree responses were coded as neutral. Our analysis began with a visual exploration of all variables to evaluate the frequency, percentage, and near-zero variance for categorical variables.¹⁴ Near-zero variance occurs when a categorical variable has a low frequency of unique values over the sample size (ie, the variable is almost constant), and we addressed it by combining different variable categorizations. We handled missing values through imputation algorithms followed by sensitivity analyses to verify whether our results were stable with and without imputation. We performed comparisons for the exploratory analysis using P values for one-way analysis of variance tests for numeric variables and χ²tests for categorical variables. We considered P values < .05 to be statistically significant. We also used correlation matrices and plots as exploratory analysis tools to better understand all items’ correlations. We used Pearson, polychoric, and polyserial correlation tests as appropriate for numeric, ordinal, and logical items.

Our modeling strategy involved a series of generalized linear models (GLMs) with a Gaussian family, ie, multiple linear regression models, to assess the association between (1) facilities’ preferences regarding pre-anesthesia evaluation modalities; (2) advantages between modalities; and (3) barriers to the adoption of telehealth and the ability to perform different pre-anesthesia evaluation-related tasks. In addition, we used backward deletion to reach the most parsimonious model based on a series of likelihood-ratio tests comparing nested models. Results are reported as predicted means with 95% confidence intervals, with results being interpreted as significant when any 2 predicted means do not overlap between different estimates along with P for trends < .001. We performed all analyses using the R language.¹⁵

Results

Of 109 surveyed facilities, 50 (46%) responded to the survey. The final study sample included 67 responses, and 55 were included in the analysis. Twelve responses were excluded from the analysis as they were either incomplete or test responses. Three facilities had > 1 complete response (2 facilities had 2 responses and 1 facility had 4 responses), and these were all included in the analysis.

Thirty-six locations were complex inpatient facilities, and 32 (89%) had pre-anesthesia evaluation clinics (Table 1).

Twenty-two facilities reported using both telephone and video, 11 telephone only, 5 video only, and 12 neither. Considering the 55 individual responses, 25 respondents reported using both telephone and video, 12 reported using telephone only, 5 using video only, and 13 reported using neither telephone nor video for pre-anesthesia evaluations.

The ability to obtain a history of present illness was rated good/very good via telephone for 34 respondents (92%) and 25 for video (86%). Assessing comorbidities and health habits was rated good/very good via telephone for 32 respondents (89%) and 31 respondents (86%), respectively, and via video for 24 respondents (83%) and 23 respondents (79%), respectively (Figure 1).

Fewer respondents rated the ability to estimate exercise capacity or mental health pathology good/very good: 26 respondents (70%) and 23 respondents (62%) for telephone, respectively, and 18 (62%) and 17 (59%) for video, respectively. The ability to assess nutritional status was rated lowest with 9 respondents (24%) rating it positively for telephone and 15 (52%) for video.

To compare differences between the 2 remote pre-anesthesia evaluation modalities, we created GLMs evaluating the association between each modality and the perceived ability to perform the tasks. For GLMs, we transformed the values of the categories into numerical (ie, 1, poor; 2, neutral; 3, good). Compared with telephone, video was rated more favorably regarding the assessment of nutritional status (mean, 2.1; 95% CI, 1.8-2.3 vs mean, 2.4; 95% CI, 2.2-2.7; P = .04) (eAppendix 1, available at doi:10.12788/fp.0387). No other significant differences in ratings existed between the 2 remote pre-anesthesia evaluation modalities.

The most significant barriers (cited as significant or very significant in the survey) included the inability to perform a physical examination, which was noted by 13 respondents (72%) and 15 respondents (60%) for telephone and video, respectively. The inability to obtain vital signs was rated as a significant barrier for telephone by 12 respondents (67%) and for video by 15 respondents (60%)(Figure 2).

Other less-cited barriers included concerns about patient safety and risk; patient preference; cultural barriers; lack of support from staff; and lack of evidence for its effectiveness. Specific to video care, patients’ lack of access to a computer was cited as a barrier by 12 respondents (48%), whereas only 3 (17%) cited lack of telephone as a barrier. Lastly, lack of information technology support was cited as a barrier for video visits by 8 respondents (32%). To determine differences in perceived barriers to the implementation of phone vs video pre-anesthesia evaluations, we created GLM evaluating the association between these 2 modalities and the perceived ability to perform commonly performed pre-anesthesia evaluation visit tasks. For GLM, again we transformed the values of the categories into numeric (ie, not a significant barrier, 1; somewhat a barrier, 2; a significant barrier, 3). There were no significant differences in ratings between the 2 remote pre-anesthesia evaluation modalities (eAppendix 2, available at doi:10.12788/fp.0387).

The average FIM score was 3.7, with the highest score among respondents who used both phone and video (Table 2). The average AIM score was 3.4, with the highest score among respondents who used both telehealth modalities. The internal consistency of the implementation measures was excellent (Cronbach’s α 0.95 and 0.975 for FIM and AIM, respectively).

 

Discussion

We surveyed 109 anesthesiology services across the VA regarding barriers to implementing telephone- and video-based pre-anesthesia evaluation visits. We found that 12 (23%) of the 50 anesthesiology services responding to this survey still conduct the totality of their pre-anesthesia evaluations in person. This represents an opportunity to further disseminate the appropriate use of telehealth and potentially reduce travel time, costs, and low-value testing, as it is well established that remote pre-anesthesia evaluations for low-risk procedures are safe and effective.⁶

We also found no difference between telephone and video regarding users’ perceived ability to perform any of the basic pre-anesthesia evaluation tasks except for assessing patients’ nutritional status, which was rated as easier using video than telephone. According to those not using telephone and/or video, the biggest barriers to implementation of telehealth visits were the inability to obtain vital signs and to perform a physical examination. This finding was unexpected, as facilities that conduct remote evaluations typically defer these tasks to the day of surgery, a practice that has been well established and shown to be safe and efficient. Respondents also identified patient-level factors (eg, patient preference, lack of telephone or computer) as significant barriers. Finally, feasibility ratings were higher than acceptability ratings with regards to the implementation of telehealth.

In 2004, the first use of telehealth for pre-anesthesia evaluations was reported by Wong and colleagues.¹⁶ Since then, several case series and a literature review have documented the efficacy, safety, and patient and HCP satisfaction with the use of telehealth for pre-anesthesia evaluations. A study by Mullen-Fortino and colleagues showed reduced visit times when telehealth was used for pre-anesthesia evaluation.⁸ Another study at VA hospitals showed that 88% of veterans reported that telemedicine saved them time and money.¹⁷ A report of 35 patients in rural Australia reported 98% satisfaction with the video quality of the visit, 95% perceived efficacy, and 87% preference for telehealth compared with driving to be seen in person.¹⁸ These reports conflict with the perceptions of the respondents of our survey, who identified patient preference as an important barrier to adoption of telehealth. Given these findings, research is needed on veterans’ perceptions on the use of telehealth modalities for pre-anesthesia evaluations; if their perceptions are similarly favorable, it will be important to communicate this information to HCPs and leadership, which may help increase subsequent telehealth adoption.

Despite the reported safety, efficacy, and high satisfaction of video visits among anesthesiology teams conducting pre-anesthesia evaluations, its use remains low at VA. We have found that most facilities in the VA system chose telephone platforms during the COVID-19 pandemic. One possibility is that the adoption of video modalities among pre-anesthesia evaluation clinics in the VA system is resource intensive or difficult from the HCP’s perspective. When combined with the lack of perceived advantages over telephone as we found in our survey, most practitioners resort to the technologically less demanding and more familiar telephone platform. The results from FIM and AIM support this. While both telephone and video have high feasibility scores, acceptability scores are lower for video, even among those currently using this technology. Our findings do not rule out the utility of video-based care in perioperative medicine. Rather than a yes/no proposition, future studies need to establish the precise indications for video for pre-anesthesia evaluations; that is, situations where video visits offer an advantage over telephone. For example, video could be used to deliver preoperative optimization therapies, such as supervised exercise or mental health interventions or to guide the achievement of certain milestones before surgery in patients with chronic conditions, such as target glucose values or the treatment of anemia. Future studies should explore the perceived benefits of video over telephone among centers offering these more advanced optimization interventions.

Limitations

We received responses from a subset of VA anesthesiology services; therefore, they may not be representative of the entire VA system. Facilities designated by the VA as inpatient complex were overrepresented (72% of our sample vs 50% of the total facilities nationally), and ambulatory centers (those designed by the VA as ambulatory procedural center with basic or advanced capabilities) were underrepresented (2% of our sample vs 22% nationally). Despite this, the response rate was high, and no geographic area appeared to be underrepresented. In addition, we surveyed pre-anesthesia evaluation facilities led by anesthesiologists, and the results may not be representative of the preferences of HCPs working in nonanesthesiology led pre-anesthesia evaluation clinics. Finally, just 11 facilities used both telephone and video; therefore, a true direct comparison between these 2 platforms was limited. The VA serves a unique patient population, and the findings may not be completely applicable to the non-VA population.

Conclusions

We found no significant perceived advantages of video over telephone in the ability to conduct routine pre-anesthesia evaluations among a sample of anesthesiology HCPs in the VA except for the perceived ability to assess nutritional status. HCPs with no telehealth experience cited the inability to perform a physical examination and obtain vital signs as the most significant barriers to implementation. Respondents not using telephone cited concerns about safety. Video visits in this clinical setting had additional perceived barriers to implementation, such as lack of information technology and staff support and patient-level barriers. Video had lower acceptability by HCPs. Given findings that pre-anesthesia evaluations can be conducted effectively via telehealth and have high levels of patient satisfaction, future work should focus on increasing uptake of these remote modalities. Additionally, research on the most appropriate uses of video visits within perioperative care is also needed.

Risk calculators can be of great value in guiding clinical decision making, patient-centered precision medicine, and resource allocation.¹ Several perioperative risk prediction models have emerged in recent decades that estimate specific hazards (eg, cardiovascular complications after noncardiac surgery) with varying accuracy and utility. In the perioperative sphere, the time windows are often limited to an index hospitalization or 30 days following surgery or discharge.^2-9 Although longer periods are of interest to patients, families, and health systems, few widely used or validated models are designed to look beyond this very narrow window.^10,11 In addition, perioperative risk prediction models do not routinely incorporate parameters of a wide variety of health or demographic domains, such as patterns of health care, health care utilization, or medication use.

In 2013, in response to the need for near real-time information to guide delivery of enhanced care management services, the Veterans Health Administration (VHA) Office of Informatics and Analytics developed automated risk prediction models that used detailed electronic health record (EHR) data. These models were used to report Care Assessment Need (CAN) scores each week for all VHA enrollees and include data from a wide array of health domains. These CAN scores predict the risk for hospitalization, death, or either event within 90 days and 1 year.^12,13 Each score is reported as both a predicted probability (0-1) and as a percentile in relation to all other VHA enrollees (a value between 1 and 99).¹³ The data used to calculate CAN scores are listed in Table 1.¹²

Surgical procedures or admissions would not be differentiated from nonsurgical admissions or other procedural clinic visits, and as such, it is not possible to isolate the effect of undergoing a surgical procedure from another health-related event on the CAN score. At the same time though, a short-term increase in system utilization caused by an elective surgical procedure such as a total knee replacement (TKR) would presumably be reflected in a change in CAN score, but this has not been studied.

Since their introduction, CAN scores have been routinely accessed by primary care teams and used to facilitate care coordination for thousands of VHA patients. However, these CAN scores are currently not available to VHA surgeons, anesthesiologists, or other perioperative clinicians. In this study, we examine the distributions of preoperative CAN scores and explore the relationships of preoperative CAN 1-year mortality scores with 1-year survival following discharge and length of stay (LOS) during index hospitalization in a cohort of US veterans who underwent TKR, the most common elective operation performed within the VHA system.

Methods

Following approval of the Durham Veterans Affairs Medical Center Institutional Review Board, all necessary data were extracted from the VHA Corporate Data Warehouse (CDW) repository.¹⁴ Informed consent was waived due to the minimal risk nature of the study.

We used Current Procedural Terminology codes (27438, 27446, 27447, 27486, 27487, 27488) and International Classification of Diseases, 9th edition clinical modification procedure codes (81.54, 81.55, 81.59, 00.80-00.84) to identify all veterans who had undergone primary or revision TKR between July 2014 and December 2015 in VHA Veterans Integrated Service Network 1 (Maine, Vermont, New Hampshire, Massachusetts, Connecticut, Rhode Island, New York, Pennsylvania, West Virginia, Virginia, North Carolina). Because we focused on outcomes following hospital discharge, patients who died before discharge were excluded from the analysis. Preoperative CAN 1-year mortality score was chosen as the measure under the assumption that long-term survival may be the most meaningful of the 4 possible CAN score measures.

Our primary objective was to determine distribution of preoperative CAN scores in the study population. Our secondary was to study relationships among the preoperative CAN 1-year mortality scores and 1-year mortality and hospital LOS.

Study Variables

For each patient, we extracted the date of index surgery. The primary exposure or independent variable was the CAN score in the week prior to this date. Because prior study has shown that CAN scores trajectories do not significantly change over time, the date-stamped CAN scores in the week before surgery represent what would have been available to clinicians in a preoperative setting.¹⁵ Since CAN scores are refreshed and overwritten every week, we extracted archived scores from the CDW.

 

For the 1-year survival outcome, the primary dependent variable, we queried the vital status files in the CDW for the date of death if applicable. We confirmed survival beyond 1 year by examining vital signs in the CDW for a minimum of 2 independent encounters beyond 1 year after the date of discharge. To compute the index LOS, the secondary outcome, we computed the difference between the date of admission and date of hospital discharge.

Statistical Methods

The parameters and performance of the multivariable logistic regression models developed to compute the various CAN mortality and hospitalization risk scores have been previously described.¹² Briefly, Wang and colleagues created parsimonious regression models using backward selection. Model discrimination was evaluated using C (concordance)-statistic. Model calibration was assessed by comparing predicted vs observed event rates by risk deciles and performing Cox proportional hazards regression.

We plotted histograms to display preoperative CAN scores as a simple measure of distribution (Figure 1). We also examined the cumulative proportion of patients at each preoperative CAN 1-year mortality score.

Using a conventional t test, we compared means of preoperative CAN 1-year mortality scores in patients who survived vs those who died within 1 year. We also constructed a plot of the proportion of patients who had died within 1 year vs preoperative CAN 1-year mortality scores. Kaplan-Meier curves were then constructed examining 1-year survival by CAN 1-year mortality score by terciles.

Finally, we examined the relationship between preoperative CAN 1-year mortality scores and index LOS in 2 ways: We plotted LOS across CAN scores, and we constructed a locally weighted regression and smoothing scatterplot. LOESS (locally weighted polynomial regression) models, involve fitting the polynomial using weighted least squares, giving more weight to points near the point whose response is being estimated and less weight to points further away. LOESS models help fit a smooth curve to data for which linear and nonlinear least squares regression models do not perform well and in situations where no theoretical model exists. LOESS are ideal for modeling complex processes. All analyses were performed using SAS, version 9.3.

Results

We identified 8206 patients who had undergone a TKR over the 18-month study period. The overall mean (SD) for age was 65 (8.41) years; 93% were male, and 78% were White veterans. Patient demographics are well described in a previous publication.^16,17

In terms of model parameters for the CAN score models, C-statistics for the 90-day outcome models were as follows: 0.833 for the model predicting hospitalization (95% CI, 0.832-0.834); 0.865 for the model predicting death (95% CI, 0.863-0.876); and 0.811 for the model predicting either event (95% CI, 0.810-0.812). C-statistics for the 1-year outcome models were 0.809 for the model predicting hospitalization (95% CI, 0.808-0.810); 0.851 for the model predicting death (95% CI, 0.849-0.852); and 0.787 for the model predicting either event (95% CI, 0.786-0.787). Models were well calibrated with α = 0 and β = 1, demonstrating strong agreement between observed and predicted event rates.

The distribution of preoperative CAN 1-year mortality scores was close to normal (median, 50; interquartile range, 40; mean [SD] 48 [25.6]) (eTable). The original CAN score models were developed having an equal number of patients in each strata and as such, are normally distributed.¹² Our cohort was similar in pattern of distribution. Distributions of the remaining preoperative CAN scores (90-day mortality, 1-year hospitalization, 90-day hospitalization) are shown in Figures 2, 3, and 4. Not surprisingly, histograms for both 90-day and 1-year hospitalization were skewed toward higher scores, indicating that these patients were expected to be hospitalized in the near future.

Overall, 1.4% (110/8096) of patients died within 1 year of surgery. Comparing 1-year mortality CAN scores in survivors vs nonsurvivors, we found statistically significant differences in means (47 vs 66 respectively, P < .001) and medians (45 vs 75 respectively, P < .001) (Table 2). In the plot examining the relationship between preoperative 1-year mortality CAN scores and 1-year mortality, the percentage who died within 1 year increased initially for patients with CAN scores > 60 and again exponentially for patients with CAN scores > 80. Examining Kaplan-Meier curves, we found that survivors and nonsurvivors separated early after surgery, and the differences between the top tercile and the middle/lower terciles were statistically significant (P < .001). Mortality rates were about 0.5% in the lower and middle terciles but about 2% in the upper tercile (Figure 5).

In the plot examining the relationship between CAN scores and index LOS, the LOS rose significantly beyond a CAN score of 60 and dramatically beyond a CAN score of 80 (Figure 6). LOESS curves also showed 2 inflection points suggesting an incremental and sequential rise in the LOS with increasing CAN scores (Figure 7). Mean (SD) LOS in days for the lowest to highest terciles was 2.6 (1.7), 2.8 (2.1), and 3.6 (2.2), respectively.

 

Discussion

CAN scores are automatically generated each week by EHR-based multivariable risk models. These scores have excellent predictive accuracy for 90-day and 1-year mortality and hospitalization and are routinely used by VHA primary care teams to assist with clinical operations.¹³ We studied the distribution of CAN 1-year mortality scores in a preoperative context and examined relationships of the preoperative CAN 1-year mortality scores with postoperative mortality and LOS in 8206 veterans who underwent TKR.

There are several noteworthy findings. First, the overall 1-year mortality rate observed following TKR (1.4%) was similar to other published reports.^18,19 Not surprisingly, preoperative CAN 1-year mortality scores were significantly higher in veterans who died compared with those of survivors. The majority of patients who died had a preoperative CAN 1-year mortality score > 75 while most who survived had a preoperative CAN 1-year mortality score < 45 (P < .001). Interestingly, the same scores showed a nonlinear correlation with LOS. Index LOS was about 4 days in patients in the highest tercile of CAN scores vs 2.5 days in the lowest tercile, but the initial increase in LOS was detected at a CAN score of about 55 to 60.

In addition, mortality rate varied widely in different segments of the population when grouped according to preoperative CAN scores. One-year mortality rates in the highest tercile reached 2%, about 4-fold higher than that of lower terciles (0.5%). Examination of the Kaplan-Meier curves showed that this difference in mortality between the highest tercile and the lower 2 groups appears soon after discharge and continues to increase over time, suggesting that the factors contributing to the increased mortality are present at the time of discharge and persist beyond the postoperative period. In summary, although CAN scores were not designed for use in the perioperative context, we found that preoperative CAN 1-year mortality scores are broadly predictive of mortality, but especially for increases in LOS following elective TKA, both increases in hospital LOS following elective TKA and mortality over the year after TKA.

Our findings raise several important questions. The decision to undergo elective surgery is complex. Arguably, individuals who undergo elective knee replacement should be healthy enough to undergo, recover, and reap the benefits from a procedure that does not extend life. The distribution of preoperative CAN 1-year mortality scores for our study population was similar to that of the general VHA enrollee population with similar measured mortality rates (≤ 0.5% vs ≥ 1.7% in the low and high terciles, respectively).¹ Further study comparing outcomes in matched cohorts who did and did not undergo joint replacement would be of interest. In lieu of this, though, the association of high but not extreme CAN scores with increased hospital LOS may potentially be used to guide allocation of resources to this group, obviating the increased cost and risk to which this group is exposed. And the additional insight afforded by CAN scores may enhance shared decision-making models by identifying patients at the very highest risk (eg, 1-year mortality CAN score ≥ 90), patients who conceivably might not survive long enough to recover from and enjoy their reconstructed knee, who might in the long run be harmed by undergoing the procedure.

Many total joint arthroplasties are performed in older patients, a population in which frailty is increasingly recognized as a significant risk factor for poor outcomes.^20,21 CAN scores reliably identify high-risk patients and have been shown to correlate with frailty in this group.²² Multiple authors have reported improved outcomes with cost reductions after implementation of programs targeting modifiable risk factors in high-risk surgical candidates.^23-25 A preoperative assessment that includes the CAN score may be valuable in identifying patients who would benefit most from prehabilitation programs or other interventions designed to blunt the impact of frailty. It is true that many elements used to calculate the CAN score would not be considered modifiable, especially in the short term. However, specific contributors to frailty, such as nutritional status and polypharmacy might be potential candidates. As with all multivariable risk prediction models, there are multiple paths to a high CAN score, and further research to identify clinically relevant subgroups may help inform efforts to improve perioperative care within this population.

Hospital LOS is of intense interest for many reasons, not least its utility as a surrogate for cost and increased risk for immediate perioperative adverse events, such as multidrug-resistant hospital acquired infections, need for postacute facility-based rehabilitation, and deconditioning that increase risks of falls and fractures in the older population.^26-29 In addition, its importance is magnified due to the COVID-19 pandemic context in which restarting elective surgery programs has changed traditional criteria by which patients are scheduled for surgery.

We have shown that elevated CAN scores are able to identify patients at risk for extended hospital stays and, as such, may be useful additional data in allocating scarce operating room time and other resources for optimal patient and health care provider safety.^30,31 Individual surgeons and hospital systems would, of course, decide which patients should be triaged to go first, based on local priorities; however, choosing lower risk patients with minimal risk of morbidity and mortality while pursuing prehabilitation for higher risk patients is a reasonable approach.

 

Limitations

Our study has several limitations. Only a single surgical procedure was included, albeit the most common one performed in the VHA. In addition, no information was available concerning the precise clinical course for these patients, such as the duration of surgery, anesthetic technique, and management of acute, perioperative course. Although the assumption was made that patients received standard care in a manner such that these factors would not significantly affect either their mortality or their LOS out of proportion to their preoperative clinical status, confounding cannot be excluded. Therefore, further study is necessary to determine whether CAN scores can accurately predict mortality and/or LOS for patients undergoing other procedures. Further, a clinical trial is required to assess whether systematic provision of the CAN score at the point of surgery would impact care and, more important, impact outcomes. In addition, multivariable analyses were not performed, including and excluding various components of the CAN score models. Currently, CAN scores could be made available to the surgical/anesthesia communities at minimal or no cost and are updated automatically. Model calibration and discrimination in this particular setting were not validated.

Because our interest is in leveraging an existing resource to a current clinical and operational problem rather than in creating or validating a new tool, we chose to test the simple bivariate relationship between preoperative CAN scores and outcomes. We chose the preoperative 1-year mortality CAN score from among the 4 options under the assumption that long-term survival is the most meaningful of the 4 candidate outcomes. Finally, while the CAN scores are currently only calculated and generated for patients cared for within the VHA, few data elements are unavailable to civilian health systems. The most problematic would be documentation of actual prescription filling, but this is a topic of increasing interest to the medical and academic communities and access to such information we hope will improve.^32-34

Conclusions

Although designed for use by VHA primary care teams, CAN scores also may have value for perioperative clinicians, predicting mortality and prolonged hospital LOS in those with elevated 1-year mortality scores. Advantages of CAN scores relative to other perioperative risk calculators lies in their ability to predict long-term rather than 30-day survival and that they are automatically generated on a near-real-time basis for all patients who receive care in VHA ambulatory clinics. Further study is needed to determine practical utility in shared decision making, preoperative evaluation and optimization, and perioperative resource allocation.

Acknowledgments

This work was supported by the US Department of Veterans Affairs (VA) National Center for Patient Safety, Field Office 10A4E, through the Patient Safety Center of Inquiry at the Durham VA Medical Center in North Carolina. The study also received support from the Center of Innovation to Accelerate Discovery and Practice Transformation (CIN 13-410) at the Durham VA Health Care System.

Risk calculators can be of great value in guiding clinical decision making, patient-centered precision medicine, and resource allocation.¹ Several perioperative risk prediction models have emerged in recent decades that estimate specific hazards (eg, cardiovascular complications after noncardiac surgery) with varying accuracy and utility. In the perioperative sphere, the time windows are often limited to an index hospitalization or 30 days following surgery or discharge.^2-9 Although longer periods are of interest to patients, families, and health systems, few widely used or validated models are designed to look beyond this very narrow window.^10,11 In addition, perioperative risk prediction models do not routinely incorporate parameters of a wide variety of health or demographic domains, such as patterns of health care, health care utilization, or medication use.

In 2013, in response to the need for near real-time information to guide delivery of enhanced care management services, the Veterans Health Administration (VHA) Office of Informatics and Analytics developed automated risk prediction models that used detailed electronic health record (EHR) data. These models were used to report Care Assessment Need (CAN) scores each week for all VHA enrollees and include data from a wide array of health domains. These CAN scores predict the risk for hospitalization, death, or either event within 90 days and 1 year.^12,13 Each score is reported as both a predicted probability (0-1) and as a percentile in relation to all other VHA enrollees (a value between 1 and 99).¹³ The data used to calculate CAN scores are listed in Table 1.¹²

Surgical procedures or admissions would not be differentiated from nonsurgical admissions or other procedural clinic visits, and as such, it is not possible to isolate the effect of undergoing a surgical procedure from another health-related event on the CAN score. At the same time though, a short-term increase in system utilization caused by an elective surgical procedure such as a total knee replacement (TKR) would presumably be reflected in a change in CAN score, but this has not been studied.

Since their introduction, CAN scores have been routinely accessed by primary care teams and used to facilitate care coordination for thousands of VHA patients. However, these CAN scores are currently not available to VHA surgeons, anesthesiologists, or other perioperative clinicians. In this study, we examine the distributions of preoperative CAN scores and explore the relationships of preoperative CAN 1-year mortality scores with 1-year survival following discharge and length of stay (LOS) during index hospitalization in a cohort of US veterans who underwent TKR, the most common elective operation performed within the VHA system.

Methods

Following approval of the Durham Veterans Affairs Medical Center Institutional Review Board, all necessary data were extracted from the VHA Corporate Data Warehouse (CDW) repository.¹⁴ Informed consent was waived due to the minimal risk nature of the study.

We used Current Procedural Terminology codes (27438, 27446, 27447, 27486, 27487, 27488) and International Classification of Diseases, 9th edition clinical modification procedure codes (81.54, 81.55, 81.59, 00.80-00.84) to identify all veterans who had undergone primary or revision TKR between July 2014 and December 2015 in VHA Veterans Integrated Service Network 1 (Maine, Vermont, New Hampshire, Massachusetts, Connecticut, Rhode Island, New York, Pennsylvania, West Virginia, Virginia, North Carolina). Because we focused on outcomes following hospital discharge, patients who died before discharge were excluded from the analysis. Preoperative CAN 1-year mortality score was chosen as the measure under the assumption that long-term survival may be the most meaningful of the 4 possible CAN score measures.

Our primary objective was to determine distribution of preoperative CAN scores in the study population. Our secondary was to study relationships among the preoperative CAN 1-year mortality scores and 1-year mortality and hospital LOS.

Study Variables

For each patient, we extracted the date of index surgery. The primary exposure or independent variable was the CAN score in the week prior to this date. Because prior study has shown that CAN scores trajectories do not significantly change over time, the date-stamped CAN scores in the week before surgery represent what would have been available to clinicians in a preoperative setting.¹⁵ Since CAN scores are refreshed and overwritten every week, we extracted archived scores from the CDW.

 

For the 1-year survival outcome, the primary dependent variable, we queried the vital status files in the CDW for the date of death if applicable. We confirmed survival beyond 1 year by examining vital signs in the CDW for a minimum of 2 independent encounters beyond 1 year after the date of discharge. To compute the index LOS, the secondary outcome, we computed the difference between the date of admission and date of hospital discharge.

Statistical Methods

The parameters and performance of the multivariable logistic regression models developed to compute the various CAN mortality and hospitalization risk scores have been previously described.¹² Briefly, Wang and colleagues created parsimonious regression models using backward selection. Model discrimination was evaluated using C (concordance)-statistic. Model calibration was assessed by comparing predicted vs observed event rates by risk deciles and performing Cox proportional hazards regression.

We plotted histograms to display preoperative CAN scores as a simple measure of distribution (Figure 1). We also examined the cumulative proportion of patients at each preoperative CAN 1-year mortality score.

Using a conventional t test, we compared means of preoperative CAN 1-year mortality scores in patients who survived vs those who died within 1 year. We also constructed a plot of the proportion of patients who had died within 1 year vs preoperative CAN 1-year mortality scores. Kaplan-Meier curves were then constructed examining 1-year survival by CAN 1-year mortality score by terciles.

Finally, we examined the relationship between preoperative CAN 1-year mortality scores and index LOS in 2 ways: We plotted LOS across CAN scores, and we constructed a locally weighted regression and smoothing scatterplot. LOESS (locally weighted polynomial regression) models, involve fitting the polynomial using weighted least squares, giving more weight to points near the point whose response is being estimated and less weight to points further away. LOESS models help fit a smooth curve to data for which linear and nonlinear least squares regression models do not perform well and in situations where no theoretical model exists. LOESS are ideal for modeling complex processes. All analyses were performed using SAS, version 9.3.

Results

We identified 8206 patients who had undergone a TKR over the 18-month study period. The overall mean (SD) for age was 65 (8.41) years; 93% were male, and 78% were White veterans. Patient demographics are well described in a previous publication.^16,17

In terms of model parameters for the CAN score models, C-statistics for the 90-day outcome models were as follows: 0.833 for the model predicting hospitalization (95% CI, 0.832-0.834); 0.865 for the model predicting death (95% CI, 0.863-0.876); and 0.811 for the model predicting either event (95% CI, 0.810-0.812). C-statistics for the 1-year outcome models were 0.809 for the model predicting hospitalization (95% CI, 0.808-0.810); 0.851 for the model predicting death (95% CI, 0.849-0.852); and 0.787 for the model predicting either event (95% CI, 0.786-0.787). Models were well calibrated with α = 0 and β = 1, demonstrating strong agreement between observed and predicted event rates.

The distribution of preoperative CAN 1-year mortality scores was close to normal (median, 50; interquartile range, 40; mean [SD] 48 [25.6]) (eTable). The original CAN score models were developed having an equal number of patients in each strata and as such, are normally distributed.¹² Our cohort was similar in pattern of distribution. Distributions of the remaining preoperative CAN scores (90-day mortality, 1-year hospitalization, 90-day hospitalization) are shown in Figures 2, 3, and 4. Not surprisingly, histograms for both 90-day and 1-year hospitalization were skewed toward higher scores, indicating that these patients were expected to be hospitalized in the near future.

Overall, 1.4% (110/8096) of patients died within 1 year of surgery. Comparing 1-year mortality CAN scores in survivors vs nonsurvivors, we found statistically significant differences in means (47 vs 66 respectively, P < .001) and medians (45 vs 75 respectively, P < .001) (Table 2). In the plot examining the relationship between preoperative 1-year mortality CAN scores and 1-year mortality, the percentage who died within 1 year increased initially for patients with CAN scores > 60 and again exponentially for patients with CAN scores > 80. Examining Kaplan-Meier curves, we found that survivors and nonsurvivors separated early after surgery, and the differences between the top tercile and the middle/lower terciles were statistically significant (P < .001). Mortality rates were about 0.5% in the lower and middle terciles but about 2% in the upper tercile (Figure 5).

In the plot examining the relationship between CAN scores and index LOS, the LOS rose significantly beyond a CAN score of 60 and dramatically beyond a CAN score of 80 (Figure 6). LOESS curves also showed 2 inflection points suggesting an incremental and sequential rise in the LOS with increasing CAN scores (Figure 7). Mean (SD) LOS in days for the lowest to highest terciles was 2.6 (1.7), 2.8 (2.1), and 3.6 (2.2), respectively.

 

Discussion

CAN scores are automatically generated each week by EHR-based multivariable risk models. These scores have excellent predictive accuracy for 90-day and 1-year mortality and hospitalization and are routinely used by VHA primary care teams to assist with clinical operations.¹³ We studied the distribution of CAN 1-year mortality scores in a preoperative context and examined relationships of the preoperative CAN 1-year mortality scores with postoperative mortality and LOS in 8206 veterans who underwent TKR.

There are several noteworthy findings. First, the overall 1-year mortality rate observed following TKR (1.4%) was similar to other published reports.^18,19 Not surprisingly, preoperative CAN 1-year mortality scores were significantly higher in veterans who died compared with those of survivors. The majority of patients who died had a preoperative CAN 1-year mortality score > 75 while most who survived had a preoperative CAN 1-year mortality score < 45 (P < .001). Interestingly, the same scores showed a nonlinear correlation with LOS. Index LOS was about 4 days in patients in the highest tercile of CAN scores vs 2.5 days in the lowest tercile, but the initial increase in LOS was detected at a CAN score of about 55 to 60.

In addition, mortality rate varied widely in different segments of the population when grouped according to preoperative CAN scores. One-year mortality rates in the highest tercile reached 2%, about 4-fold higher than that of lower terciles (0.5%). Examination of the Kaplan-Meier curves showed that this difference in mortality between the highest tercile and the lower 2 groups appears soon after discharge and continues to increase over time, suggesting that the factors contributing to the increased mortality are present at the time of discharge and persist beyond the postoperative period. In summary, although CAN scores were not designed for use in the perioperative context, we found that preoperative CAN 1-year mortality scores are broadly predictive of mortality, but especially for increases in LOS following elective TKA, both increases in hospital LOS following elective TKA and mortality over the year after TKA.

Our findings raise several important questions. The decision to undergo elective surgery is complex. Arguably, individuals who undergo elective knee replacement should be healthy enough to undergo, recover, and reap the benefits from a procedure that does not extend life. The distribution of preoperative CAN 1-year mortality scores for our study population was similar to that of the general VHA enrollee population with similar measured mortality rates (≤ 0.5% vs ≥ 1.7% in the low and high terciles, respectively).¹ Further study comparing outcomes in matched cohorts who did and did not undergo joint replacement would be of interest. In lieu of this, though, the association of high but not extreme CAN scores with increased hospital LOS may potentially be used to guide allocation of resources to this group, obviating the increased cost and risk to which this group is exposed. And the additional insight afforded by CAN scores may enhance shared decision-making models by identifying patients at the very highest risk (eg, 1-year mortality CAN score ≥ 90), patients who conceivably might not survive long enough to recover from and enjoy their reconstructed knee, who might in the long run be harmed by undergoing the procedure.

Many total joint arthroplasties are performed in older patients, a population in which frailty is increasingly recognized as a significant risk factor for poor outcomes.^20,21 CAN scores reliably identify high-risk patients and have been shown to correlate with frailty in this group.²² Multiple authors have reported improved outcomes with cost reductions after implementation of programs targeting modifiable risk factors in high-risk surgical candidates.^23-25 A preoperative assessment that includes the CAN score may be valuable in identifying patients who would benefit most from prehabilitation programs or other interventions designed to blunt the impact of frailty. It is true that many elements used to calculate the CAN score would not be considered modifiable, especially in the short term. However, specific contributors to frailty, such as nutritional status and polypharmacy might be potential candidates. As with all multivariable risk prediction models, there are multiple paths to a high CAN score, and further research to identify clinically relevant subgroups may help inform efforts to improve perioperative care within this population.

Hospital LOS is of intense interest for many reasons, not least its utility as a surrogate for cost and increased risk for immediate perioperative adverse events, such as multidrug-resistant hospital acquired infections, need for postacute facility-based rehabilitation, and deconditioning that increase risks of falls and fractures in the older population.^26-29 In addition, its importance is magnified due to the COVID-19 pandemic context in which restarting elective surgery programs has changed traditional criteria by which patients are scheduled for surgery.

We have shown that elevated CAN scores are able to identify patients at risk for extended hospital stays and, as such, may be useful additional data in allocating scarce operating room time and other resources for optimal patient and health care provider safety.^30,31 Individual surgeons and hospital systems would, of course, decide which patients should be triaged to go first, based on local priorities; however, choosing lower risk patients with minimal risk of morbidity and mortality while pursuing prehabilitation for higher risk patients is a reasonable approach.

 

Limitations

Our study has several limitations. Only a single surgical procedure was included, albeit the most common one performed in the VHA. In addition, no information was available concerning the precise clinical course for these patients, such as the duration of surgery, anesthetic technique, and management of acute, perioperative course. Although the assumption was made that patients received standard care in a manner such that these factors would not significantly affect either their mortality or their LOS out of proportion to their preoperative clinical status, confounding cannot be excluded. Therefore, further study is necessary to determine whether CAN scores can accurately predict mortality and/or LOS for patients undergoing other procedures. Further, a clinical trial is required to assess whether systematic provision of the CAN score at the point of surgery would impact care and, more important, impact outcomes. In addition, multivariable analyses were not performed, including and excluding various components of the CAN score models. Currently, CAN scores could be made available to the surgical/anesthesia communities at minimal or no cost and are updated automatically. Model calibration and discrimination in this particular setting were not validated.

Because our interest is in leveraging an existing resource to a current clinical and operational problem rather than in creating or validating a new tool, we chose to test the simple bivariate relationship between preoperative CAN scores and outcomes. We chose the preoperative 1-year mortality CAN score from among the 4 options under the assumption that long-term survival is the most meaningful of the 4 candidate outcomes. Finally, while the CAN scores are currently only calculated and generated for patients cared for within the VHA, few data elements are unavailable to civilian health systems. The most problematic would be documentation of actual prescription filling, but this is a topic of increasing interest to the medical and academic communities and access to such information we hope will improve.^32-34

Conclusions

Although designed for use by VHA primary care teams, CAN scores also may have value for perioperative clinicians, predicting mortality and prolonged hospital LOS in those with elevated 1-year mortality scores. Advantages of CAN scores relative to other perioperative risk calculators lies in their ability to predict long-term rather than 30-day survival and that they are automatically generated on a near-real-time basis for all patients who receive care in VHA ambulatory clinics. Further study is needed to determine practical utility in shared decision making, preoperative evaluation and optimization, and perioperative resource allocation.

Acknowledgments

This work was supported by the US Department of Veterans Affairs (VA) National Center for Patient Safety, Field Office 10A4E, through the Patient Safety Center of Inquiry at the Durham VA Medical Center in North Carolina. The study also received support from the Center of Innovation to Accelerate Discovery and Practice Transformation (CIN 13-410) at the Durham VA Health Care System.

Risk calculators can be of great value in guiding clinical decision making, patient-centered precision medicine, and resource allocation.¹ Several perioperative risk prediction models have emerged in recent decades that estimate specific hazards (eg, cardiovascular complications after noncardiac surgery) with varying accuracy and utility. In the perioperative sphere, the time windows are often limited to an index hospitalization or 30 days following surgery or discharge.^2-9 Although longer periods are of interest to patients, families, and health systems, few widely used or validated models are designed to look beyond this very narrow window.^10,11 In addition, perioperative risk prediction models do not routinely incorporate parameters of a wide variety of health or demographic domains, such as patterns of health care, health care utilization, or medication use.

In 2013, in response to the need for near real-time information to guide delivery of enhanced care management services, the Veterans Health Administration (VHA) Office of Informatics and Analytics developed automated risk prediction models that used detailed electronic health record (EHR) data. These models were used to report Care Assessment Need (CAN) scores each week for all VHA enrollees and include data from a wide array of health domains. These CAN scores predict the risk for hospitalization, death, or either event within 90 days and 1 year.^12,13 Each score is reported as both a predicted probability (0-1) and as a percentile in relation to all other VHA enrollees (a value between 1 and 99).¹³ The data used to calculate CAN scores are listed in Table 1.¹²

Surgical procedures or admissions would not be differentiated from nonsurgical admissions or other procedural clinic visits, and as such, it is not possible to isolate the effect of undergoing a surgical procedure from another health-related event on the CAN score. At the same time though, a short-term increase in system utilization caused by an elective surgical procedure such as a total knee replacement (TKR) would presumably be reflected in a change in CAN score, but this has not been studied.

Since their introduction, CAN scores have been routinely accessed by primary care teams and used to facilitate care coordination for thousands of VHA patients. However, these CAN scores are currently not available to VHA surgeons, anesthesiologists, or other perioperative clinicians. In this study, we examine the distributions of preoperative CAN scores and explore the relationships of preoperative CAN 1-year mortality scores with 1-year survival following discharge and length of stay (LOS) during index hospitalization in a cohort of US veterans who underwent TKR, the most common elective operation performed within the VHA system.

Methods

Following approval of the Durham Veterans Affairs Medical Center Institutional Review Board, all necessary data were extracted from the VHA Corporate Data Warehouse (CDW) repository.¹⁴ Informed consent was waived due to the minimal risk nature of the study.

We used Current Procedural Terminology codes (27438, 27446, 27447, 27486, 27487, 27488) and International Classification of Diseases, 9th edition clinical modification procedure codes (81.54, 81.55, 81.59, 00.80-00.84) to identify all veterans who had undergone primary or revision TKR between July 2014 and December 2015 in VHA Veterans Integrated Service Network 1 (Maine, Vermont, New Hampshire, Massachusetts, Connecticut, Rhode Island, New York, Pennsylvania, West Virginia, Virginia, North Carolina). Because we focused on outcomes following hospital discharge, patients who died before discharge were excluded from the analysis. Preoperative CAN 1-year mortality score was chosen as the measure under the assumption that long-term survival may be the most meaningful of the 4 possible CAN score measures.

Our primary objective was to determine distribution of preoperative CAN scores in the study population. Our secondary was to study relationships among the preoperative CAN 1-year mortality scores and 1-year mortality and hospital LOS.

Study Variables

For each patient, we extracted the date of index surgery. The primary exposure or independent variable was the CAN score in the week prior to this date. Because prior study has shown that CAN scores trajectories do not significantly change over time, the date-stamped CAN scores in the week before surgery represent what would have been available to clinicians in a preoperative setting.¹⁵ Since CAN scores are refreshed and overwritten every week, we extracted archived scores from the CDW.

 

For the 1-year survival outcome, the primary dependent variable, we queried the vital status files in the CDW for the date of death if applicable. We confirmed survival beyond 1 year by examining vital signs in the CDW for a minimum of 2 independent encounters beyond 1 year after the date of discharge. To compute the index LOS, the secondary outcome, we computed the difference between the date of admission and date of hospital discharge.

Statistical Methods

The parameters and performance of the multivariable logistic regression models developed to compute the various CAN mortality and hospitalization risk scores have been previously described.¹² Briefly, Wang and colleagues created parsimonious regression models using backward selection. Model discrimination was evaluated using C (concordance)-statistic. Model calibration was assessed by comparing predicted vs observed event rates by risk deciles and performing Cox proportional hazards regression.

We plotted histograms to display preoperative CAN scores as a simple measure of distribution (Figure 1). We also examined the cumulative proportion of patients at each preoperative CAN 1-year mortality score.

Using a conventional t test, we compared means of preoperative CAN 1-year mortality scores in patients who survived vs those who died within 1 year. We also constructed a plot of the proportion of patients who had died within 1 year vs preoperative CAN 1-year mortality scores. Kaplan-Meier curves were then constructed examining 1-year survival by CAN 1-year mortality score by terciles.

Finally, we examined the relationship between preoperative CAN 1-year mortality scores and index LOS in 2 ways: We plotted LOS across CAN scores, and we constructed a locally weighted regression and smoothing scatterplot. LOESS (locally weighted polynomial regression) models, involve fitting the polynomial using weighted least squares, giving more weight to points near the point whose response is being estimated and less weight to points further away. LOESS models help fit a smooth curve to data for which linear and nonlinear least squares regression models do not perform well and in situations where no theoretical model exists. LOESS are ideal for modeling complex processes. All analyses were performed using SAS, version 9.3.

Results

We identified 8206 patients who had undergone a TKR over the 18-month study period. The overall mean (SD) for age was 65 (8.41) years; 93% were male, and 78% were White veterans. Patient demographics are well described in a previous publication.^16,17

In terms of model parameters for the CAN score models, C-statistics for the 90-day outcome models were as follows: 0.833 for the model predicting hospitalization (95% CI, 0.832-0.834); 0.865 for the model predicting death (95% CI, 0.863-0.876); and 0.811 for the model predicting either event (95% CI, 0.810-0.812). C-statistics for the 1-year outcome models were 0.809 for the model predicting hospitalization (95% CI, 0.808-0.810); 0.851 for the model predicting death (95% CI, 0.849-0.852); and 0.787 for the model predicting either event (95% CI, 0.786-0.787). Models were well calibrated with α = 0 and β = 1, demonstrating strong agreement between observed and predicted event rates.

The distribution of preoperative CAN 1-year mortality scores was close to normal (median, 50; interquartile range, 40; mean [SD] 48 [25.6]) (eTable). The original CAN score models were developed having an equal number of patients in each strata and as such, are normally distributed.¹² Our cohort was similar in pattern of distribution. Distributions of the remaining preoperative CAN scores (90-day mortality, 1-year hospitalization, 90-day hospitalization) are shown in Figures 2, 3, and 4. Not surprisingly, histograms for both 90-day and 1-year hospitalization were skewed toward higher scores, indicating that these patients were expected to be hospitalized in the near future.

Overall, 1.4% (110/8096) of patients died within 1 year of surgery. Comparing 1-year mortality CAN scores in survivors vs nonsurvivors, we found statistically significant differences in means (47 vs 66 respectively, P < .001) and medians (45 vs 75 respectively, P < .001) (Table 2). In the plot examining the relationship between preoperative 1-year mortality CAN scores and 1-year mortality, the percentage who died within 1 year increased initially for patients with CAN scores > 60 and again exponentially for patients with CAN scores > 80. Examining Kaplan-Meier curves, we found that survivors and nonsurvivors separated early after surgery, and the differences between the top tercile and the middle/lower terciles were statistically significant (P < .001). Mortality rates were about 0.5% in the lower and middle terciles but about 2% in the upper tercile (Figure 5).

In the plot examining the relationship between CAN scores and index LOS, the LOS rose significantly beyond a CAN score of 60 and dramatically beyond a CAN score of 80 (Figure 6). LOESS curves also showed 2 inflection points suggesting an incremental and sequential rise in the LOS with increasing CAN scores (Figure 7). Mean (SD) LOS in days for the lowest to highest terciles was 2.6 (1.7), 2.8 (2.1), and 3.6 (2.2), respectively.

 

Discussion

CAN scores are automatically generated each week by EHR-based multivariable risk models. These scores have excellent predictive accuracy for 90-day and 1-year mortality and hospitalization and are routinely used by VHA primary care teams to assist with clinical operations.¹³ We studied the distribution of CAN 1-year mortality scores in a preoperative context and examined relationships of the preoperative CAN 1-year mortality scores with postoperative mortality and LOS in 8206 veterans who underwent TKR.

There are several noteworthy findings. First, the overall 1-year mortality rate observed following TKR (1.4%) was similar to other published reports.^18,19 Not surprisingly, preoperative CAN 1-year mortality scores were significantly higher in veterans who died compared with those of survivors. The majority of patients who died had a preoperative CAN 1-year mortality score > 75 while most who survived had a preoperative CAN 1-year mortality score < 45 (P < .001). Interestingly, the same scores showed a nonlinear correlation with LOS. Index LOS was about 4 days in patients in the highest tercile of CAN scores vs 2.5 days in the lowest tercile, but the initial increase in LOS was detected at a CAN score of about 55 to 60.

In addition, mortality rate varied widely in different segments of the population when grouped according to preoperative CAN scores. One-year mortality rates in the highest tercile reached 2%, about 4-fold higher than that of lower terciles (0.5%). Examination of the Kaplan-Meier curves showed that this difference in mortality between the highest tercile and the lower 2 groups appears soon after discharge and continues to increase over time, suggesting that the factors contributing to the increased mortality are present at the time of discharge and persist beyond the postoperative period. In summary, although CAN scores were not designed for use in the perioperative context, we found that preoperative CAN 1-year mortality scores are broadly predictive of mortality, but especially for increases in LOS following elective TKA, both increases in hospital LOS following elective TKA and mortality over the year after TKA.

Our findings raise several important questions. The decision to undergo elective surgery is complex. Arguably, individuals who undergo elective knee replacement should be healthy enough to undergo, recover, and reap the benefits from a procedure that does not extend life. The distribution of preoperative CAN 1-year mortality scores for our study population was similar to that of the general VHA enrollee population with similar measured mortality rates (≤ 0.5% vs ≥ 1.7% in the low and high terciles, respectively).¹ Further study comparing outcomes in matched cohorts who did and did not undergo joint replacement would be of interest. In lieu of this, though, the association of high but not extreme CAN scores with increased hospital LOS may potentially be used to guide allocation of resources to this group, obviating the increased cost and risk to which this group is exposed. And the additional insight afforded by CAN scores may enhance shared decision-making models by identifying patients at the very highest risk (eg, 1-year mortality CAN score ≥ 90), patients who conceivably might not survive long enough to recover from and enjoy their reconstructed knee, who might in the long run be harmed by undergoing the procedure.

Many total joint arthroplasties are performed in older patients, a population in which frailty is increasingly recognized as a significant risk factor for poor outcomes.^20,21 CAN scores reliably identify high-risk patients and have been shown to correlate with frailty in this group.²² Multiple authors have reported improved outcomes with cost reductions after implementation of programs targeting modifiable risk factors in high-risk surgical candidates.^23-25 A preoperative assessment that includes the CAN score may be valuable in identifying patients who would benefit most from prehabilitation programs or other interventions designed to blunt the impact of frailty. It is true that many elements used to calculate the CAN score would not be considered modifiable, especially in the short term. However, specific contributors to frailty, such as nutritional status and polypharmacy might be potential candidates. As with all multivariable risk prediction models, there are multiple paths to a high CAN score, and further research to identify clinically relevant subgroups may help inform efforts to improve perioperative care within this population.

Hospital LOS is of intense interest for many reasons, not least its utility as a surrogate for cost and increased risk for immediate perioperative adverse events, such as multidrug-resistant hospital acquired infections, need for postacute facility-based rehabilitation, and deconditioning that increase risks of falls and fractures in the older population.^26-29 In addition, its importance is magnified due to the COVID-19 pandemic context in which restarting elective surgery programs has changed traditional criteria by which patients are scheduled for surgery.

We have shown that elevated CAN scores are able to identify patients at risk for extended hospital stays and, as such, may be useful additional data in allocating scarce operating room time and other resources for optimal patient and health care provider safety.^30,31 Individual surgeons and hospital systems would, of course, decide which patients should be triaged to go first, based on local priorities; however, choosing lower risk patients with minimal risk of morbidity and mortality while pursuing prehabilitation for higher risk patients is a reasonable approach.

 

Limitations

Our study has several limitations. Only a single surgical procedure was included, albeit the most common one performed in the VHA. In addition, no information was available concerning the precise clinical course for these patients, such as the duration of surgery, anesthetic technique, and management of acute, perioperative course. Although the assumption was made that patients received standard care in a manner such that these factors would not significantly affect either their mortality or their LOS out of proportion to their preoperative clinical status, confounding cannot be excluded. Therefore, further study is necessary to determine whether CAN scores can accurately predict mortality and/or LOS for patients undergoing other procedures. Further, a clinical trial is required to assess whether systematic provision of the CAN score at the point of surgery would impact care and, more important, impact outcomes. In addition, multivariable analyses were not performed, including and excluding various components of the CAN score models. Currently, CAN scores could be made available to the surgical/anesthesia communities at minimal or no cost and are updated automatically. Model calibration and discrimination in this particular setting were not validated.

Because our interest is in leveraging an existing resource to a current clinical and operational problem rather than in creating or validating a new tool, we chose to test the simple bivariate relationship between preoperative CAN scores and outcomes. We chose the preoperative 1-year mortality CAN score from among the 4 options under the assumption that long-term survival is the most meaningful of the 4 candidate outcomes. Finally, while the CAN scores are currently only calculated and generated for patients cared for within the VHA, few data elements are unavailable to civilian health systems. The most problematic would be documentation of actual prescription filling, but this is a topic of increasing interest to the medical and academic communities and access to such information we hope will improve.^32-34

Conclusions

Although designed for use by VHA primary care teams, CAN scores also may have value for perioperative clinicians, predicting mortality and prolonged hospital LOS in those with elevated 1-year mortality scores. Advantages of CAN scores relative to other perioperative risk calculators lies in their ability to predict long-term rather than 30-day survival and that they are automatically generated on a near-real-time basis for all patients who receive care in VHA ambulatory clinics. Further study is needed to determine practical utility in shared decision making, preoperative evaluation and optimization, and perioperative resource allocation.

Acknowledgments

This work was supported by the US Department of Veterans Affairs (VA) National Center for Patient Safety, Field Office 10A4E, through the Patient Safety Center of Inquiry at the Durham VA Medical Center in North Carolina. The study also received support from the Center of Innovation to Accelerate Discovery and Practice Transformation (CIN 13-410) at the Durham VA Health Care System.