User login
ABSTRACT
Purpose To derive a predictive model for obstructive sleep apnea (OSA) in primary care practice, using home-based overnight oximetry results to refine posttest probability (PTP) of disease after initial risk stratification with the Sleep Apnea Clinical Score (SACS).
Methods We performed secondary analyses on data from a SACS validation cohort, to compare the diagnostic accuracy of 3 overnight oximetry measurements (oxygen desaturation index [ODI], mean saturation, and minimum saturation) in predicting OSA. Receiver operator characteristics (ROC) were computed for each measurement independently and sequentially after risk stratifying with SACS. We examined the implications of oximetry results for OSA PTP for participants categorized as intermediate risk (SACS 6-14; 66/191 participants [35%]; OSA probability 41%). We calculated positive likelihood ratios (LR) for multiple ODI results and determined which ones allowed recalibration to high- or low-risk PTP.
Results Among the 3 oximetry findings, ODI best predicted OSA (area under the curve [AUC], 0.88; 95% confidence interval [CI], 0.83-0.93). An ODI ≥8.4 (likelihood ratio [LR], 4.19; 95% CI, 2.87-6.10) created a PTP of 77%, while an ODI of 0 to <8.4 (LR, 0.19, 95% CI, 0.12-0.33) created a 14% PTP. Sequential application of SACS and ODI results yielded an AUC result of 0.90 (95% CI, 0.85-0.95).
Conclusions SACS risk stratification provides an advantage over clinical gestalt. In those at intermediate risk, ODI results provide a simple and clinically useful way to further refine diagnostic prediction. Sequential use of SACS and selectively employed overnight oximetry may limit unnecessary polysomnography. Oximetry testing should be avoided in patients deemed low or high risk by SACS, as positive results do not substantially recalibrate risk.
Obstructive sleep apnea (OSA) is a prevalent and underdiagnosed condition. The National Sleep Foundation estimates that 18 million Americans have OSA.1 Primary care practice may be the best setting in which to identify OSA, as many of our patients have conditions frequently associated with apnea (eg, hypertension, obesity, diabetes, arrhythmia, and neurologic illness). Up to a third of patients in primary care practice may be at increased risk.2,3
Clinical guidelines of the American Academy of Sleep Medicine (AASM) recommend obtaining a sleep history to evaluate for possible OSA in 3 instances: as part of a routine health maintenance examination, during evaluation of specific complaints associated with OSA (eg, snoring, apnea, daytime sleepiness), and during comprehensive evaluations for individuals with high-risk conditions (ie, obesity, congestive heart failure, refractory hypertension, diabetes, stroke history).4
The American College of Physicians (ACP) Clinical Practice Guideline suggests assessing individuals who have unexplained daytime sleepiness.5 The ACP considers this assessment “High-Value Care,” as “evidence shows that before diagnosis, patients with OSA have higher rates of health care use, more frequent and longer hospital stays, and higher health care costs than after diagnosis.”5
Continue to: We recently validated the diagnostic accuracy...
We recently validated the diagnostic accuracy of the Sleep Apnea Clinical Score (SACS) for use in a primary care patient population suspected of having OSA.6 SACS uses historical and clinical data to derive a score that identifies a patient’s risk level.7 However, as an alternative to the 2 levels described in Flemons’ SACS,7 we propose creating 3 risk strata (FIGURE 17,8). We believe that patients at high risk (SACS ≥15) should be encouraged to undergo sleep evaluations as their posttest probability (PTP) of OSA is 75% to 80%. Individuals at low risk (SACS ≤5; PTP <20%) could receive lifestyle advice and simple clinical interventions that decrease symptoms (eg, weight loss, increased physical activity, sleeping on one’s side). For low-risk patients, clinical observation and reevaluation could take place over time with their primary care provider, without additional testing or referral to specialists.
What about patients at intermediate risk? Many patients suspected of having OSA will be assigned to intermediate risk (SACS 6-14), and their PTP of OSA remains at 40% to 45%, the pre-test level most commonly encountered in suspected OSA. As polysomnography is a limited and expensive clinical resource, intermediate-risk patients would benefit from recalibration of their SACS-based risk assessment using an additional surrogate test such as home-based overnight oximetry. Our internal OSA practice guidelines recommend referral for sleep medicine consultation when oximetry results are abnormal—specifically, an oxygen desaturation index (ODI) of ≥5, a mean saturation less than 89%, and a minimum saturation of 75% or less.
Our objectives in this study were to compare the diagnostic implications of these 3 measurements from home-based overnight oximetry reports and use the most relevant result to derive a predictive model further refining PTP of OSA in a primary care patient population first stratified to intermediate risk by SACS.
METHODS
Subjects
We performed secondary analyses on data obtained from our SACS validation cohort.6 In brief, these were patients suspected of having OSA based on the presence of signs, symptoms, or associated risk factors. One hundred ninety-one patients completed all assessments. Sixty-six of 191 patients (35%) were categorized as intermediate risk (SACS 6-14; OSA probability 41% [27/66]).
Data collection and analyses
Participants completed home-based overnight oximetry using Nonin Model 2500 oximeters (Nonin Medical Inc., Plymouth, Minn). We transferred oximetry results from the sleep lab database to a statistical program for analyses of ODI, mean saturation, and minimal saturation. ODI was defined as the number of 4% drops in saturation from baseline divided by the number of hours of recording time. Although the AASM states that a diagnosis of OSA is confirmed if the number of obstructive events is more than 15 per hour or more than 5 per hour in a patient who reports related symptoms,4 we defined OSA as an apnea-hypopnea index (AHI) of >10 based on polysomnography (as this was the threshold used in the derivation cohort for SACS).7 We demonstrated the predictive ability of SACS at various AHI definitions of OSA in our validation cohort.6 The use of SACS in our validation cohort showed a statistically similar ability to predict OSA at both an AHI of 10 and 20, compared with the derivation cohort.
Continue to: We entered additional information...
We entered additional information reported directly by patients and obtained from their sleep studies into a REDCap database and transferred that to our statistical program. We used descriptive statistics to determine ranges and central tendencies of oximetry results. Receiver operator characteristic (ROC) analyses described the predictive abilities for each oximetry result individually and in serial application with prior SACS determinations. For comparison, we used the area under the ROC curve (AUC) from logistic regression to model the probability of OSA.
We calculated positive likelihood ratios (LR) and 95% confidence intervals (CI) to determine the degree of oximetry abnormality that would recalibrate risk either to a high PTP of OSA (>75%) or a low PTP (<25%). We sorted intermediate-risk SACS scores into quintiles based on ODI results to compare the resulting PTPs of OSA. We applied the PTP of OSA from our previous work (using the SACS score to compute the LR) as the new PTP, estimated the LR based on ODI, and computed an updated PTP of OSA. We also used ROC analysis to determine the optimal cutoff value of the ODI.
Finally, in accordance with our internal clinical practice recommendations, we examined the predictive ability of a “positive” ODI result of ≥5 to recalibrate risk prediction for OSA for patients in the low-risk group. We performed analyses using SAS 9.4 (SAS Institute, Cary, NC).
RESULTS
One hundred ninety-one subjects completed assessments. The median and quartile results for ODI, mean saturation, and minimum saturation are found in TABLE 1. TABLE 2 shows the distribution of patients with positive oximetry results. An ODI of 5 or greater was the most frequent abnormal result (135/191; 70.7%).
We used the AUC to measure the comparative abilities of SACS and the 3 overnight oximetry results in predicting OSA (TABLE 3). ODI results demonstrated the best ability to predict OSA, compared with polysomnography as the relative gold standard (AUC, 0.88; 95% confidence interval [CI], 0.83-0.93). Serial application of SACS and ODI yielded even better diagnostic results (AUC, 0.90; 95% CI, 0.85-0.95).
Continue to: As ODI was found to be the strongest predictor of OSA...
As ODI was found to be the strongest predictor of OSA, we grouped these results in quintiles and calculated positive LRs. TABLE 4 shows their effect on PTP of disease among patients with intermediate risk. An ODI result >10 effected an upward recalibration of disease probability (LR, 2.33; 95% CI, 1.27-4.26). The optimal cutoff of ODI to discriminate between those with and without OSA was determined by ROC analysis. An ODI greater than 8.4 created a PTP of disease of approximately 73% to 77%.
Our internal clinical guidelines recommend referring patients with an ODI of 5 or greater for sleep medicine consultation. We examined the ability of this ODI result to recalibrate disease suspicion for a patient at low risk (SACS ≤5). The LR for ODI of 5 or greater is 2.1, but this only results in a recalibration of risk from 24% pretest probability in our validation cohort to 41% PTP (95% CI, 33-49). This low cutoff for a positive test creates false-positive results more than 40% of the time due to low specificity (0.58). This is insufficient to change the suspicion of disease, resulting only in a shift to intermediate OSA risk.
DISCUSSION
Among 3 different oximetry measurements, an ODI ≥10 best predicts OSA, both independently and when used sequentially after the SACS. ODI was by far the most frequent abnormality on oximetry in our cohort, thereby increasing its utility in clinical decision making. For those subjects at intermediate risk, a cutoff of 10 for the ODI result may be a simple and clinically effective way to recalibrate risk and aid in making referral decisions. (This may also be simpler and more easily remembered by clinicians than the 8.4 ODI results from the ROC analyses.)
Assessment is inadequate without a clinical prediction rule. Unfortunately, providers cannot simply rely on clinical gestalt in diagnosing OSA. In their derivation cohort, Flemens et al examined the LRs created by SACS and by clinician prediction based on history and physical exam.7 The SACS LRs ranged from 5.17 to 0.25, a 20-fold range. This reflected superior diagnostic information compared with subjective physician impression, where LRs ranged from 3.7 to 0.52, a seven-fold range. Myers et al prepared a meta-analysis of 4 different trials that examined physicians’ ability to predict OSA.9 Despite the researchers’ use of experienced sleep medicine doctors, the overall diagnostic accuracy of clinical impression was modest (summary positive LR, 1.7; 95% CI, 1.5-2; I2 = 0%; summary negative LR, 0.67; 95% CI, 0.60-0.74; I2 = 10%; sensitivity, 58%; specificity, 67%). This is similar to reliance on a single clinical sign or symptom to predict OSA.
Wise use of oximetry augments SACS calculation. To limit unnecessary oximetry testing in low- and high-risk groups and to avoid polysomnography in cases of a low PTP of disease, we advocate limiting oximetry testing to individuals in the SACS intermediate-risk group (FIGURE 2) wherein ODI results can potentially recalibrate risk assessment up or down. (Those in the high- risk group should be referred to a sleep medicine specialist.) Our institutional recommendation of using an ODI result of ≥5 as a threshold to increase suspicion of disease requires a caveat for the low-risk group. “Positive” results at that low diagnostic threshold are frequently false.
Continue to: Multiple benefits of SACS
Multiple benefits of SACS. We believe using the SACS calculation during clinical encounters with patients potentially at risk for OSA would increase diagnostic accuracy. Performing risk stratification with SACS should not be an undue burden on providers, and the increased time spent with patients has its own benefits, including helping them better understand their risk. Using this standardized process—augmented, as needed, with overnight ODI assessment—might also encourage more patients to follow through on subsequent recommendations, as their risk is further quantified objectively. Lastly, unnecessary testing with polysomnography could be avoided.
Limitations of our study. This study’s findings were derived from a patient population in a single institution. Replication of the findings from other settings would be helpful.
Looking forward. It is yet unclear if clinicians will embrace these strategies in real-world primary care practice. We have designed an implementation-and-dissemination trial to assess whether family physicians will use the SACS clinical predication rule in everyday practice and whether our evidence-based recommendations about overnight oximetry will be followed. Underlying our suggested clinical evaluation pathway (FIGURE 2) is the belief that there is value gained from sharing the decision-making process with patients. Although we provide new evidence that informs these conversations, the patient’s values and preferences are important when determining the best direction to proceed in the evaluation for suspected OSA. These recommendations are intended to aid, not replace, good clinical judgment.
Home-based sleep testing has become more widely available, is convenient for patients, and is less expensive than lab-based polysomnography. Our study did not directly address the appropriate circumstances for home studies in clinical evaluation. We rely on the expertise of our sleep medicine colleagues to determine which patients are appropriate candidates for home-based studies.
The AASM states that “portable monitors (PM) for the diagnosis of OSA should be [used] only in conjunction with a comprehensive sleep evaluation. Clinical sleep evaluations using PM must be supervised by a practitioner with board certification in sleep medicine or an individual who fulfills the eligibility criteria for the sleep medicine certification examination.”4 Additionally, the group recommends that PM “may be used in the unattended setting as an alternative to polysomnography for the diagnosis of OSA in patients with a high pretest probability of moderate to severe OSA and no comorbid sleep disorder or major comorbid medical disorders.”4
Continue to: GRANT SUPPORT
GRANT SUPPORT
The use of the REDCap database is supported by grant UL1 TR000135. This work was supported by a Mayo Foundation CR-20 grant awarded to Dr. Mookadam as Principal investigator and Dr. Grover as Coinvestigator.
Statistical analyses were supported, in part, by the Department of Family Medicine, Mayo Clinic, Scottsdale, Ariz.
CORRESPONDENCE
Michael Grover, DO, Mayo Clinic Thunderbird Primary Care Center-Family Medicine, 13737 N 92nd Street, Scottsdale, AZ 85260; grover.michael@mayo.edu
1. National Sleep Foundation. Sleep apnea. https://sleepfoundation.org/sleep-disorders-problems/sleep-apnea. Accessed September 14, 2018.
2. Grover M, Mookadam M, Armas D, et al. Identifying patients at risk for obstructive sleep apnea in a primary care practice. J Am Board Fam Med. 2011;24:152-160.
3. Mold JW, Quattlebaum C, Schinnerer E, et al. Identification by primary care clinicians of patients with obstructive sleep apnea: a practice-based research network (PBRN) study. J Am Board Fam Med. 2011;24:138-145.
4. Epstein LJ, Kristo D, Strollo PJ, Jr., et al; Adult Obstructive Sleep Apnea Task Force of the American Academy of Sleep Medicine. Clinical guideline for the evaluation, management and long-term care of obstructive sleep apnea in adults. J Clin Sleep Med. 2009;5:263-276.
5. Qaseem A, Dallas P, Owens DK, et al. Diagnosis of obstructive sleep apnea in adults: a clinical practice guideline from the American College of Physicians. Ann Intern Med. 2014;161:210-220.
6. Grover M, Mookadam M, Chang Y-H, et al. Validating the Sleep Apnea Clinical Score for use in primary care populations. Mayo Clin Proc. 2016;91:469-476.
7. Flemons WW, Whitelaw WA, Brant R, et al. Likelihood ratios for a sleep apnea clinical prediction rule. Am J Respir Crit Care Med. 1994;150:1279-1285.
8. Gali B, Whalen FX, Gay PC, et al. Management plan to reduce risks in perioperative care of patients with presumed obstructive sleep apnea syndrome. J Clin Sleep Med. 2007;3:582-588.
9. Myers KA, Mrkobrada M, Simel DL. Does this patient have obstructive sleep apnea?: The rational clinical examination systematic review. JAMA. 2013;310(7):731-741.
ABSTRACT
Purpose To derive a predictive model for obstructive sleep apnea (OSA) in primary care practice, using home-based overnight oximetry results to refine posttest probability (PTP) of disease after initial risk stratification with the Sleep Apnea Clinical Score (SACS).
Methods We performed secondary analyses on data from a SACS validation cohort, to compare the diagnostic accuracy of 3 overnight oximetry measurements (oxygen desaturation index [ODI], mean saturation, and minimum saturation) in predicting OSA. Receiver operator characteristics (ROC) were computed for each measurement independently and sequentially after risk stratifying with SACS. We examined the implications of oximetry results for OSA PTP for participants categorized as intermediate risk (SACS 6-14; 66/191 participants [35%]; OSA probability 41%). We calculated positive likelihood ratios (LR) for multiple ODI results and determined which ones allowed recalibration to high- or low-risk PTP.
Results Among the 3 oximetry findings, ODI best predicted OSA (area under the curve [AUC], 0.88; 95% confidence interval [CI], 0.83-0.93). An ODI ≥8.4 (likelihood ratio [LR], 4.19; 95% CI, 2.87-6.10) created a PTP of 77%, while an ODI of 0 to <8.4 (LR, 0.19, 95% CI, 0.12-0.33) created a 14% PTP. Sequential application of SACS and ODI results yielded an AUC result of 0.90 (95% CI, 0.85-0.95).
Conclusions SACS risk stratification provides an advantage over clinical gestalt. In those at intermediate risk, ODI results provide a simple and clinically useful way to further refine diagnostic prediction. Sequential use of SACS and selectively employed overnight oximetry may limit unnecessary polysomnography. Oximetry testing should be avoided in patients deemed low or high risk by SACS, as positive results do not substantially recalibrate risk.
Obstructive sleep apnea (OSA) is a prevalent and underdiagnosed condition. The National Sleep Foundation estimates that 18 million Americans have OSA.1 Primary care practice may be the best setting in which to identify OSA, as many of our patients have conditions frequently associated with apnea (eg, hypertension, obesity, diabetes, arrhythmia, and neurologic illness). Up to a third of patients in primary care practice may be at increased risk.2,3
Clinical guidelines of the American Academy of Sleep Medicine (AASM) recommend obtaining a sleep history to evaluate for possible OSA in 3 instances: as part of a routine health maintenance examination, during evaluation of specific complaints associated with OSA (eg, snoring, apnea, daytime sleepiness), and during comprehensive evaluations for individuals with high-risk conditions (ie, obesity, congestive heart failure, refractory hypertension, diabetes, stroke history).4
The American College of Physicians (ACP) Clinical Practice Guideline suggests assessing individuals who have unexplained daytime sleepiness.5 The ACP considers this assessment “High-Value Care,” as “evidence shows that before diagnosis, patients with OSA have higher rates of health care use, more frequent and longer hospital stays, and higher health care costs than after diagnosis.”5
Continue to: We recently validated the diagnostic accuracy...
We recently validated the diagnostic accuracy of the Sleep Apnea Clinical Score (SACS) for use in a primary care patient population suspected of having OSA.6 SACS uses historical and clinical data to derive a score that identifies a patient’s risk level.7 However, as an alternative to the 2 levels described in Flemons’ SACS,7 we propose creating 3 risk strata (FIGURE 17,8). We believe that patients at high risk (SACS ≥15) should be encouraged to undergo sleep evaluations as their posttest probability (PTP) of OSA is 75% to 80%. Individuals at low risk (SACS ≤5; PTP <20%) could receive lifestyle advice and simple clinical interventions that decrease symptoms (eg, weight loss, increased physical activity, sleeping on one’s side). For low-risk patients, clinical observation and reevaluation could take place over time with their primary care provider, without additional testing or referral to specialists.
What about patients at intermediate risk? Many patients suspected of having OSA will be assigned to intermediate risk (SACS 6-14), and their PTP of OSA remains at 40% to 45%, the pre-test level most commonly encountered in suspected OSA. As polysomnography is a limited and expensive clinical resource, intermediate-risk patients would benefit from recalibration of their SACS-based risk assessment using an additional surrogate test such as home-based overnight oximetry. Our internal OSA practice guidelines recommend referral for sleep medicine consultation when oximetry results are abnormal—specifically, an oxygen desaturation index (ODI) of ≥5, a mean saturation less than 89%, and a minimum saturation of 75% or less.
Our objectives in this study were to compare the diagnostic implications of these 3 measurements from home-based overnight oximetry reports and use the most relevant result to derive a predictive model further refining PTP of OSA in a primary care patient population first stratified to intermediate risk by SACS.
METHODS
Subjects
We performed secondary analyses on data obtained from our SACS validation cohort.6 In brief, these were patients suspected of having OSA based on the presence of signs, symptoms, or associated risk factors. One hundred ninety-one patients completed all assessments. Sixty-six of 191 patients (35%) were categorized as intermediate risk (SACS 6-14; OSA probability 41% [27/66]).
Data collection and analyses
Participants completed home-based overnight oximetry using Nonin Model 2500 oximeters (Nonin Medical Inc., Plymouth, Minn). We transferred oximetry results from the sleep lab database to a statistical program for analyses of ODI, mean saturation, and minimal saturation. ODI was defined as the number of 4% drops in saturation from baseline divided by the number of hours of recording time. Although the AASM states that a diagnosis of OSA is confirmed if the number of obstructive events is more than 15 per hour or more than 5 per hour in a patient who reports related symptoms,4 we defined OSA as an apnea-hypopnea index (AHI) of >10 based on polysomnography (as this was the threshold used in the derivation cohort for SACS).7 We demonstrated the predictive ability of SACS at various AHI definitions of OSA in our validation cohort.6 The use of SACS in our validation cohort showed a statistically similar ability to predict OSA at both an AHI of 10 and 20, compared with the derivation cohort.
Continue to: We entered additional information...
We entered additional information reported directly by patients and obtained from their sleep studies into a REDCap database and transferred that to our statistical program. We used descriptive statistics to determine ranges and central tendencies of oximetry results. Receiver operator characteristic (ROC) analyses described the predictive abilities for each oximetry result individually and in serial application with prior SACS determinations. For comparison, we used the area under the ROC curve (AUC) from logistic regression to model the probability of OSA.
We calculated positive likelihood ratios (LR) and 95% confidence intervals (CI) to determine the degree of oximetry abnormality that would recalibrate risk either to a high PTP of OSA (>75%) or a low PTP (<25%). We sorted intermediate-risk SACS scores into quintiles based on ODI results to compare the resulting PTPs of OSA. We applied the PTP of OSA from our previous work (using the SACS score to compute the LR) as the new PTP, estimated the LR based on ODI, and computed an updated PTP of OSA. We also used ROC analysis to determine the optimal cutoff value of the ODI.
Finally, in accordance with our internal clinical practice recommendations, we examined the predictive ability of a “positive” ODI result of ≥5 to recalibrate risk prediction for OSA for patients in the low-risk group. We performed analyses using SAS 9.4 (SAS Institute, Cary, NC).
RESULTS
One hundred ninety-one subjects completed assessments. The median and quartile results for ODI, mean saturation, and minimum saturation are found in TABLE 1. TABLE 2 shows the distribution of patients with positive oximetry results. An ODI of 5 or greater was the most frequent abnormal result (135/191; 70.7%).
We used the AUC to measure the comparative abilities of SACS and the 3 overnight oximetry results in predicting OSA (TABLE 3). ODI results demonstrated the best ability to predict OSA, compared with polysomnography as the relative gold standard (AUC, 0.88; 95% confidence interval [CI], 0.83-0.93). Serial application of SACS and ODI yielded even better diagnostic results (AUC, 0.90; 95% CI, 0.85-0.95).
Continue to: As ODI was found to be the strongest predictor of OSA...
As ODI was found to be the strongest predictor of OSA, we grouped these results in quintiles and calculated positive LRs. TABLE 4 shows their effect on PTP of disease among patients with intermediate risk. An ODI result >10 effected an upward recalibration of disease probability (LR, 2.33; 95% CI, 1.27-4.26). The optimal cutoff of ODI to discriminate between those with and without OSA was determined by ROC analysis. An ODI greater than 8.4 created a PTP of disease of approximately 73% to 77%.
Our internal clinical guidelines recommend referring patients with an ODI of 5 or greater for sleep medicine consultation. We examined the ability of this ODI result to recalibrate disease suspicion for a patient at low risk (SACS ≤5). The LR for ODI of 5 or greater is 2.1, but this only results in a recalibration of risk from 24% pretest probability in our validation cohort to 41% PTP (95% CI, 33-49). This low cutoff for a positive test creates false-positive results more than 40% of the time due to low specificity (0.58). This is insufficient to change the suspicion of disease, resulting only in a shift to intermediate OSA risk.
DISCUSSION
Among 3 different oximetry measurements, an ODI ≥10 best predicts OSA, both independently and when used sequentially after the SACS. ODI was by far the most frequent abnormality on oximetry in our cohort, thereby increasing its utility in clinical decision making. For those subjects at intermediate risk, a cutoff of 10 for the ODI result may be a simple and clinically effective way to recalibrate risk and aid in making referral decisions. (This may also be simpler and more easily remembered by clinicians than the 8.4 ODI results from the ROC analyses.)
Assessment is inadequate without a clinical prediction rule. Unfortunately, providers cannot simply rely on clinical gestalt in diagnosing OSA. In their derivation cohort, Flemens et al examined the LRs created by SACS and by clinician prediction based on history and physical exam.7 The SACS LRs ranged from 5.17 to 0.25, a 20-fold range. This reflected superior diagnostic information compared with subjective physician impression, where LRs ranged from 3.7 to 0.52, a seven-fold range. Myers et al prepared a meta-analysis of 4 different trials that examined physicians’ ability to predict OSA.9 Despite the researchers’ use of experienced sleep medicine doctors, the overall diagnostic accuracy of clinical impression was modest (summary positive LR, 1.7; 95% CI, 1.5-2; I2 = 0%; summary negative LR, 0.67; 95% CI, 0.60-0.74; I2 = 10%; sensitivity, 58%; specificity, 67%). This is similar to reliance on a single clinical sign or symptom to predict OSA.
Wise use of oximetry augments SACS calculation. To limit unnecessary oximetry testing in low- and high-risk groups and to avoid polysomnography in cases of a low PTP of disease, we advocate limiting oximetry testing to individuals in the SACS intermediate-risk group (FIGURE 2) wherein ODI results can potentially recalibrate risk assessment up or down. (Those in the high- risk group should be referred to a sleep medicine specialist.) Our institutional recommendation of using an ODI result of ≥5 as a threshold to increase suspicion of disease requires a caveat for the low-risk group. “Positive” results at that low diagnostic threshold are frequently false.
Continue to: Multiple benefits of SACS
Multiple benefits of SACS. We believe using the SACS calculation during clinical encounters with patients potentially at risk for OSA would increase diagnostic accuracy. Performing risk stratification with SACS should not be an undue burden on providers, and the increased time spent with patients has its own benefits, including helping them better understand their risk. Using this standardized process—augmented, as needed, with overnight ODI assessment—might also encourage more patients to follow through on subsequent recommendations, as their risk is further quantified objectively. Lastly, unnecessary testing with polysomnography could be avoided.
Limitations of our study. This study’s findings were derived from a patient population in a single institution. Replication of the findings from other settings would be helpful.
Looking forward. It is yet unclear if clinicians will embrace these strategies in real-world primary care practice. We have designed an implementation-and-dissemination trial to assess whether family physicians will use the SACS clinical predication rule in everyday practice and whether our evidence-based recommendations about overnight oximetry will be followed. Underlying our suggested clinical evaluation pathway (FIGURE 2) is the belief that there is value gained from sharing the decision-making process with patients. Although we provide new evidence that informs these conversations, the patient’s values and preferences are important when determining the best direction to proceed in the evaluation for suspected OSA. These recommendations are intended to aid, not replace, good clinical judgment.
Home-based sleep testing has become more widely available, is convenient for patients, and is less expensive than lab-based polysomnography. Our study did not directly address the appropriate circumstances for home studies in clinical evaluation. We rely on the expertise of our sleep medicine colleagues to determine which patients are appropriate candidates for home-based studies.
The AASM states that “portable monitors (PM) for the diagnosis of OSA should be [used] only in conjunction with a comprehensive sleep evaluation. Clinical sleep evaluations using PM must be supervised by a practitioner with board certification in sleep medicine or an individual who fulfills the eligibility criteria for the sleep medicine certification examination.”4 Additionally, the group recommends that PM “may be used in the unattended setting as an alternative to polysomnography for the diagnosis of OSA in patients with a high pretest probability of moderate to severe OSA and no comorbid sleep disorder or major comorbid medical disorders.”4
Continue to: GRANT SUPPORT
GRANT SUPPORT
The use of the REDCap database is supported by grant UL1 TR000135. This work was supported by a Mayo Foundation CR-20 grant awarded to Dr. Mookadam as Principal investigator and Dr. Grover as Coinvestigator.
Statistical analyses were supported, in part, by the Department of Family Medicine, Mayo Clinic, Scottsdale, Ariz.
CORRESPONDENCE
Michael Grover, DO, Mayo Clinic Thunderbird Primary Care Center-Family Medicine, 13737 N 92nd Street, Scottsdale, AZ 85260; grover.michael@mayo.edu
ABSTRACT
Purpose To derive a predictive model for obstructive sleep apnea (OSA) in primary care practice, using home-based overnight oximetry results to refine posttest probability (PTP) of disease after initial risk stratification with the Sleep Apnea Clinical Score (SACS).
Methods We performed secondary analyses on data from a SACS validation cohort, to compare the diagnostic accuracy of 3 overnight oximetry measurements (oxygen desaturation index [ODI], mean saturation, and minimum saturation) in predicting OSA. Receiver operator characteristics (ROC) were computed for each measurement independently and sequentially after risk stratifying with SACS. We examined the implications of oximetry results for OSA PTP for participants categorized as intermediate risk (SACS 6-14; 66/191 participants [35%]; OSA probability 41%). We calculated positive likelihood ratios (LR) for multiple ODI results and determined which ones allowed recalibration to high- or low-risk PTP.
Results Among the 3 oximetry findings, ODI best predicted OSA (area under the curve [AUC], 0.88; 95% confidence interval [CI], 0.83-0.93). An ODI ≥8.4 (likelihood ratio [LR], 4.19; 95% CI, 2.87-6.10) created a PTP of 77%, while an ODI of 0 to <8.4 (LR, 0.19, 95% CI, 0.12-0.33) created a 14% PTP. Sequential application of SACS and ODI results yielded an AUC result of 0.90 (95% CI, 0.85-0.95).
Conclusions SACS risk stratification provides an advantage over clinical gestalt. In those at intermediate risk, ODI results provide a simple and clinically useful way to further refine diagnostic prediction. Sequential use of SACS and selectively employed overnight oximetry may limit unnecessary polysomnography. Oximetry testing should be avoided in patients deemed low or high risk by SACS, as positive results do not substantially recalibrate risk.
Obstructive sleep apnea (OSA) is a prevalent and underdiagnosed condition. The National Sleep Foundation estimates that 18 million Americans have OSA.1 Primary care practice may be the best setting in which to identify OSA, as many of our patients have conditions frequently associated with apnea (eg, hypertension, obesity, diabetes, arrhythmia, and neurologic illness). Up to a third of patients in primary care practice may be at increased risk.2,3
Clinical guidelines of the American Academy of Sleep Medicine (AASM) recommend obtaining a sleep history to evaluate for possible OSA in 3 instances: as part of a routine health maintenance examination, during evaluation of specific complaints associated with OSA (eg, snoring, apnea, daytime sleepiness), and during comprehensive evaluations for individuals with high-risk conditions (ie, obesity, congestive heart failure, refractory hypertension, diabetes, stroke history).4
The American College of Physicians (ACP) Clinical Practice Guideline suggests assessing individuals who have unexplained daytime sleepiness.5 The ACP considers this assessment “High-Value Care,” as “evidence shows that before diagnosis, patients with OSA have higher rates of health care use, more frequent and longer hospital stays, and higher health care costs than after diagnosis.”5
Continue to: We recently validated the diagnostic accuracy...
We recently validated the diagnostic accuracy of the Sleep Apnea Clinical Score (SACS) for use in a primary care patient population suspected of having OSA.6 SACS uses historical and clinical data to derive a score that identifies a patient’s risk level.7 However, as an alternative to the 2 levels described in Flemons’ SACS,7 we propose creating 3 risk strata (FIGURE 17,8). We believe that patients at high risk (SACS ≥15) should be encouraged to undergo sleep evaluations as their posttest probability (PTP) of OSA is 75% to 80%. Individuals at low risk (SACS ≤5; PTP <20%) could receive lifestyle advice and simple clinical interventions that decrease symptoms (eg, weight loss, increased physical activity, sleeping on one’s side). For low-risk patients, clinical observation and reevaluation could take place over time with their primary care provider, without additional testing or referral to specialists.
What about patients at intermediate risk? Many patients suspected of having OSA will be assigned to intermediate risk (SACS 6-14), and their PTP of OSA remains at 40% to 45%, the pre-test level most commonly encountered in suspected OSA. As polysomnography is a limited and expensive clinical resource, intermediate-risk patients would benefit from recalibration of their SACS-based risk assessment using an additional surrogate test such as home-based overnight oximetry. Our internal OSA practice guidelines recommend referral for sleep medicine consultation when oximetry results are abnormal—specifically, an oxygen desaturation index (ODI) of ≥5, a mean saturation less than 89%, and a minimum saturation of 75% or less.
Our objectives in this study were to compare the diagnostic implications of these 3 measurements from home-based overnight oximetry reports and use the most relevant result to derive a predictive model further refining PTP of OSA in a primary care patient population first stratified to intermediate risk by SACS.
METHODS
Subjects
We performed secondary analyses on data obtained from our SACS validation cohort.6 In brief, these were patients suspected of having OSA based on the presence of signs, symptoms, or associated risk factors. One hundred ninety-one patients completed all assessments. Sixty-six of 191 patients (35%) were categorized as intermediate risk (SACS 6-14; OSA probability 41% [27/66]).
Data collection and analyses
Participants completed home-based overnight oximetry using Nonin Model 2500 oximeters (Nonin Medical Inc., Plymouth, Minn). We transferred oximetry results from the sleep lab database to a statistical program for analyses of ODI, mean saturation, and minimal saturation. ODI was defined as the number of 4% drops in saturation from baseline divided by the number of hours of recording time. Although the AASM states that a diagnosis of OSA is confirmed if the number of obstructive events is more than 15 per hour or more than 5 per hour in a patient who reports related symptoms,4 we defined OSA as an apnea-hypopnea index (AHI) of >10 based on polysomnography (as this was the threshold used in the derivation cohort for SACS).7 We demonstrated the predictive ability of SACS at various AHI definitions of OSA in our validation cohort.6 The use of SACS in our validation cohort showed a statistically similar ability to predict OSA at both an AHI of 10 and 20, compared with the derivation cohort.
Continue to: We entered additional information...
We entered additional information reported directly by patients and obtained from their sleep studies into a REDCap database and transferred that to our statistical program. We used descriptive statistics to determine ranges and central tendencies of oximetry results. Receiver operator characteristic (ROC) analyses described the predictive abilities for each oximetry result individually and in serial application with prior SACS determinations. For comparison, we used the area under the ROC curve (AUC) from logistic regression to model the probability of OSA.
We calculated positive likelihood ratios (LR) and 95% confidence intervals (CI) to determine the degree of oximetry abnormality that would recalibrate risk either to a high PTP of OSA (>75%) or a low PTP (<25%). We sorted intermediate-risk SACS scores into quintiles based on ODI results to compare the resulting PTPs of OSA. We applied the PTP of OSA from our previous work (using the SACS score to compute the LR) as the new PTP, estimated the LR based on ODI, and computed an updated PTP of OSA. We also used ROC analysis to determine the optimal cutoff value of the ODI.
Finally, in accordance with our internal clinical practice recommendations, we examined the predictive ability of a “positive” ODI result of ≥5 to recalibrate risk prediction for OSA for patients in the low-risk group. We performed analyses using SAS 9.4 (SAS Institute, Cary, NC).
RESULTS
One hundred ninety-one subjects completed assessments. The median and quartile results for ODI, mean saturation, and minimum saturation are found in TABLE 1. TABLE 2 shows the distribution of patients with positive oximetry results. An ODI of 5 or greater was the most frequent abnormal result (135/191; 70.7%).
We used the AUC to measure the comparative abilities of SACS and the 3 overnight oximetry results in predicting OSA (TABLE 3). ODI results demonstrated the best ability to predict OSA, compared with polysomnography as the relative gold standard (AUC, 0.88; 95% confidence interval [CI], 0.83-0.93). Serial application of SACS and ODI yielded even better diagnostic results (AUC, 0.90; 95% CI, 0.85-0.95).
Continue to: As ODI was found to be the strongest predictor of OSA...
As ODI was found to be the strongest predictor of OSA, we grouped these results in quintiles and calculated positive LRs. TABLE 4 shows their effect on PTP of disease among patients with intermediate risk. An ODI result >10 effected an upward recalibration of disease probability (LR, 2.33; 95% CI, 1.27-4.26). The optimal cutoff of ODI to discriminate between those with and without OSA was determined by ROC analysis. An ODI greater than 8.4 created a PTP of disease of approximately 73% to 77%.
Our internal clinical guidelines recommend referring patients with an ODI of 5 or greater for sleep medicine consultation. We examined the ability of this ODI result to recalibrate disease suspicion for a patient at low risk (SACS ≤5). The LR for ODI of 5 or greater is 2.1, but this only results in a recalibration of risk from 24% pretest probability in our validation cohort to 41% PTP (95% CI, 33-49). This low cutoff for a positive test creates false-positive results more than 40% of the time due to low specificity (0.58). This is insufficient to change the suspicion of disease, resulting only in a shift to intermediate OSA risk.
DISCUSSION
Among 3 different oximetry measurements, an ODI ≥10 best predicts OSA, both independently and when used sequentially after the SACS. ODI was by far the most frequent abnormality on oximetry in our cohort, thereby increasing its utility in clinical decision making. For those subjects at intermediate risk, a cutoff of 10 for the ODI result may be a simple and clinically effective way to recalibrate risk and aid in making referral decisions. (This may also be simpler and more easily remembered by clinicians than the 8.4 ODI results from the ROC analyses.)
Assessment is inadequate without a clinical prediction rule. Unfortunately, providers cannot simply rely on clinical gestalt in diagnosing OSA. In their derivation cohort, Flemens et al examined the LRs created by SACS and by clinician prediction based on history and physical exam.7 The SACS LRs ranged from 5.17 to 0.25, a 20-fold range. This reflected superior diagnostic information compared with subjective physician impression, where LRs ranged from 3.7 to 0.52, a seven-fold range. Myers et al prepared a meta-analysis of 4 different trials that examined physicians’ ability to predict OSA.9 Despite the researchers’ use of experienced sleep medicine doctors, the overall diagnostic accuracy of clinical impression was modest (summary positive LR, 1.7; 95% CI, 1.5-2; I2 = 0%; summary negative LR, 0.67; 95% CI, 0.60-0.74; I2 = 10%; sensitivity, 58%; specificity, 67%). This is similar to reliance on a single clinical sign or symptom to predict OSA.
Wise use of oximetry augments SACS calculation. To limit unnecessary oximetry testing in low- and high-risk groups and to avoid polysomnography in cases of a low PTP of disease, we advocate limiting oximetry testing to individuals in the SACS intermediate-risk group (FIGURE 2) wherein ODI results can potentially recalibrate risk assessment up or down. (Those in the high- risk group should be referred to a sleep medicine specialist.) Our institutional recommendation of using an ODI result of ≥5 as a threshold to increase suspicion of disease requires a caveat for the low-risk group. “Positive” results at that low diagnostic threshold are frequently false.
Continue to: Multiple benefits of SACS
Multiple benefits of SACS. We believe using the SACS calculation during clinical encounters with patients potentially at risk for OSA would increase diagnostic accuracy. Performing risk stratification with SACS should not be an undue burden on providers, and the increased time spent with patients has its own benefits, including helping them better understand their risk. Using this standardized process—augmented, as needed, with overnight ODI assessment—might also encourage more patients to follow through on subsequent recommendations, as their risk is further quantified objectively. Lastly, unnecessary testing with polysomnography could be avoided.
Limitations of our study. This study’s findings were derived from a patient population in a single institution. Replication of the findings from other settings would be helpful.
Looking forward. It is yet unclear if clinicians will embrace these strategies in real-world primary care practice. We have designed an implementation-and-dissemination trial to assess whether family physicians will use the SACS clinical predication rule in everyday practice and whether our evidence-based recommendations about overnight oximetry will be followed. Underlying our suggested clinical evaluation pathway (FIGURE 2) is the belief that there is value gained from sharing the decision-making process with patients. Although we provide new evidence that informs these conversations, the patient’s values and preferences are important when determining the best direction to proceed in the evaluation for suspected OSA. These recommendations are intended to aid, not replace, good clinical judgment.
Home-based sleep testing has become more widely available, is convenient for patients, and is less expensive than lab-based polysomnography. Our study did not directly address the appropriate circumstances for home studies in clinical evaluation. We rely on the expertise of our sleep medicine colleagues to determine which patients are appropriate candidates for home-based studies.
The AASM states that “portable monitors (PM) for the diagnosis of OSA should be [used] only in conjunction with a comprehensive sleep evaluation. Clinical sleep evaluations using PM must be supervised by a practitioner with board certification in sleep medicine or an individual who fulfills the eligibility criteria for the sleep medicine certification examination.”4 Additionally, the group recommends that PM “may be used in the unattended setting as an alternative to polysomnography for the diagnosis of OSA in patients with a high pretest probability of moderate to severe OSA and no comorbid sleep disorder or major comorbid medical disorders.”4
Continue to: GRANT SUPPORT
GRANT SUPPORT
The use of the REDCap database is supported by grant UL1 TR000135. This work was supported by a Mayo Foundation CR-20 grant awarded to Dr. Mookadam as Principal investigator and Dr. Grover as Coinvestigator.
Statistical analyses were supported, in part, by the Department of Family Medicine, Mayo Clinic, Scottsdale, Ariz.
CORRESPONDENCE
Michael Grover, DO, Mayo Clinic Thunderbird Primary Care Center-Family Medicine, 13737 N 92nd Street, Scottsdale, AZ 85260; grover.michael@mayo.edu
1. National Sleep Foundation. Sleep apnea. https://sleepfoundation.org/sleep-disorders-problems/sleep-apnea. Accessed September 14, 2018.
2. Grover M, Mookadam M, Armas D, et al. Identifying patients at risk for obstructive sleep apnea in a primary care practice. J Am Board Fam Med. 2011;24:152-160.
3. Mold JW, Quattlebaum C, Schinnerer E, et al. Identification by primary care clinicians of patients with obstructive sleep apnea: a practice-based research network (PBRN) study. J Am Board Fam Med. 2011;24:138-145.
4. Epstein LJ, Kristo D, Strollo PJ, Jr., et al; Adult Obstructive Sleep Apnea Task Force of the American Academy of Sleep Medicine. Clinical guideline for the evaluation, management and long-term care of obstructive sleep apnea in adults. J Clin Sleep Med. 2009;5:263-276.
5. Qaseem A, Dallas P, Owens DK, et al. Diagnosis of obstructive sleep apnea in adults: a clinical practice guideline from the American College of Physicians. Ann Intern Med. 2014;161:210-220.
6. Grover M, Mookadam M, Chang Y-H, et al. Validating the Sleep Apnea Clinical Score for use in primary care populations. Mayo Clin Proc. 2016;91:469-476.
7. Flemons WW, Whitelaw WA, Brant R, et al. Likelihood ratios for a sleep apnea clinical prediction rule. Am J Respir Crit Care Med. 1994;150:1279-1285.
8. Gali B, Whalen FX, Gay PC, et al. Management plan to reduce risks in perioperative care of patients with presumed obstructive sleep apnea syndrome. J Clin Sleep Med. 2007;3:582-588.
9. Myers KA, Mrkobrada M, Simel DL. Does this patient have obstructive sleep apnea?: The rational clinical examination systematic review. JAMA. 2013;310(7):731-741.
1. National Sleep Foundation. Sleep apnea. https://sleepfoundation.org/sleep-disorders-problems/sleep-apnea. Accessed September 14, 2018.
2. Grover M, Mookadam M, Armas D, et al. Identifying patients at risk for obstructive sleep apnea in a primary care practice. J Am Board Fam Med. 2011;24:152-160.
3. Mold JW, Quattlebaum C, Schinnerer E, et al. Identification by primary care clinicians of patients with obstructive sleep apnea: a practice-based research network (PBRN) study. J Am Board Fam Med. 2011;24:138-145.
4. Epstein LJ, Kristo D, Strollo PJ, Jr., et al; Adult Obstructive Sleep Apnea Task Force of the American Academy of Sleep Medicine. Clinical guideline for the evaluation, management and long-term care of obstructive sleep apnea in adults. J Clin Sleep Med. 2009;5:263-276.
5. Qaseem A, Dallas P, Owens DK, et al. Diagnosis of obstructive sleep apnea in adults: a clinical practice guideline from the American College of Physicians. Ann Intern Med. 2014;161:210-220.
6. Grover M, Mookadam M, Chang Y-H, et al. Validating the Sleep Apnea Clinical Score for use in primary care populations. Mayo Clin Proc. 2016;91:469-476.
7. Flemons WW, Whitelaw WA, Brant R, et al. Likelihood ratios for a sleep apnea clinical prediction rule. Am J Respir Crit Care Med. 1994;150:1279-1285.
8. Gali B, Whalen FX, Gay PC, et al. Management plan to reduce risks in perioperative care of patients with presumed obstructive sleep apnea syndrome. J Clin Sleep Med. 2007;3:582-588.
9. Myers KA, Mrkobrada M, Simel DL. Does this patient have obstructive sleep apnea?: The rational clinical examination systematic review. JAMA. 2013;310(7):731-741.
