Article Type
Changed
Mon, 01/02/2017 - 19:34
Display Headline
Testing the functional assessment of mentation: A mobile application based assessment of mental status

Altered mental status (AMS) is a complex spectrum of cognitive deficits that includes orientation, memory, language, visuospatial ability, and perception.[1] The clinical definitions of both delirium and dementia include AMS as a hallmark clinical prerequisite. Regardless of etiology, this broader AMS definition is particularly salient in the hospital setting, where AMS is present in up to 60% of inpatients and is associated with longer hospital stay as well as increased morbidity and mortality.[2, 3] Not surprisingly, due to the complexity of identifying and assessing changes in mental status, clinically relevant AMS is often undetected among inpatients.[2] However, when detected, the most common causes of AMS (infection, polypharmacy, and pain) are treatable, suggesting that early AMS identification could alert clinicians to early signs of clinical decompensation, potentially improving clinical outcomes.[4]

Because rapid and systemic clinical detection of AMS is limited by the complexity of mental status, a number of assessments have been created, each with their own advantages, limitations, and target populations. These assessments are often limited by time‐intensive administration, subjectivity of mental status assessment, and lack of sensitivity in general medicine patients. Time‐intensive measures, such as the Short Portable Mental Status Questionnaire (SPMSQ) have utility in the research setting, whereas current common clinical risk stratification tools (eg, National Early Warning Score) utilize simpler measures such as the Alert, Voice, Pain, Unresponsive (AVPU) and Glasgow Coma Scale (GCS) as measures of mental status.[2, 5, 6, 7, 8, 9]

To address the need for a brief, clinically feasible, accurate tool in clinical detection of AMS, our group developed a mobile application for working memory testing, the Functional Assessment of Mentation (FAMTM). In this study, we aimed to identify baseline scoring distributions of the FAMTM in a nonhospitalized subgroup, as well as assess the correlation of the FAMTM to discharge disposition and compare it to the SPMSQ in inpatients.

METHODS

Study Design

We conducted a prospective observational study. Data were collected from both hospitalized and nonhospitalized adult participants as 2 distinct subgroups. Nonhospitalized adult subjects were recruited from a university medical campus (June 2013July 2013; IRB‐12‐0175). Hospitalized participants were recruited from the general medicine service as part of an ongoing study measuring quality of care and resource allocation at the same academic medical center (June 2014August 2014; IRB‐9967).[10]

FAMTM Application

The FAMTM application is a bedside tool for working memory assessment developed for the iPhone mobile operating system (Apple Inc., Cupertino, CA) and presented on an iPad mini (Apple). The application interface displays 4 colored rectangles individually labeled with a number (see Supporting Figure 1 in the online version of this article). The testing portion of the application presents a sequence of numbered rectangles, illuminated 1 at a time in random order. Subjects are prompted first to watch and remember the sequence and then repeat the sequence by touching the screen within each numbered rectangle. Successful reproduction of the sequence is followed by a distinct and longer sequence, whereas unsuccessful attempts are followed by a shorter sequence. The final FAMTM score corresponds to the longest sequence of rectangles successfully repeated by the subject.

Figure 1
FAMTM vs GCS by SPMSQ score. Box plot of FAMTM (light grey boxes) and GCS scores (dark grey boxes). Median, and 25th and 75th quartile results are demonstrated by vertical lines within in the box, and range is shown by the whiskers. One subject was excluded from analysis due to lack of a GCS score in the medical record. Abbreviations: FAMTM, Functional Assessment of Mentation; GCS, Glasgow Coma Scale; SPMSQ, Short Portable Mental Status Questionnaire.

Data Collection

In the nonhospitalized subject population, research assistants collected demographic data immediately prior to FAMTM administration. Among hospitalized subjects, GCS information was collected by nursing staff as part of standard clinical care. One research assistant administered the SPMSQ while a second assistant, blinded to the SPMSQ and GCS scores, administered the FAMTM. Clinical data were obtained from medical records (EPIC Systems Corp., Verona, WI). Discharge disposition was dichotomized as discharged home or not.

Statistical Analyses

Demographic characteristics of the 2 subject populations were compared using Student t tests (continuous variables) and 2 tests (categorical variables). Score distribution and discharge disposition comparison was conducted with the Mann‐Whitney U test and area under receiver operating characteristic curve (AUC) analysis, using the trapezoidal rule.[11] Multivariable linear regression was used to investigate the impact of age, race, education, discharge disposition, and hospitalization status on patient scores and times. Correlations between the FAMTM and SPMSQ scores and between the GCS and SPMSQ scores were calculated using the Spearman rank test. Significance was set at a 2‐sided P value of <0.05. Analyses were conducted using Stata version 13.1 (StataCorp, College Station, TX).

RESULTS

A total of 931 subjects were enrolled in the study. In the nonhospitalized subgroup, 651 consented to study participation and 612 were included in final analysis. Subjects were excluded if they started but did not complete the application (n = 36) or were under the age of 18 years (n = 3). Of the 363 hospitalized subjects approached for enrollment, 319 were included in the final analysis. Subjects were excluded if they refused to participate (n = 23), were under the age of 18 (n = 2), had technical failures (n = 5), or had physical or visual limitations that precluded them from participation (n = 14). Within the hospitalized subgroup, 268 subjects were discharged home (85%). The table displays demographics and score distributions by subgroup.1

Demographics and FAMTM Score Distribution by Hospitalization Category
Nonhospitalized Subjects, n = 612 Hospitalized Subjects Discharged Home, n = 268 Hospitalized Subjects Discharged Elsewhere, n = 48 P Value
  • NOTE: Abbreviations: FAMTM, Functional Assessment of Mentation; IQR, interquartile range. Three hospitalized subjects were excluded from analysis due to being discharged against medical advice. FAMTM score comparisons were significant for all stepwise comparisons using P < 0.05.

Age, y 52 18 52 19 62 17 <0.001
Female sex 343 (56%) 158 (59%) 26 (54%) 0.63
Education <0.001
Less than high school graduate 31 (5%) 32 (12%) 7 (15%)
High school graduate 312 (51%) 153 (57%) 26 (54%)
College graduate 263 (43%) 43 (16%) 8 (17%)
Missing 6 (1%) 40 (15%) 7 (15%)
Race <0.001
Black 196 (32%) 185 (69%) 34 (71%)
White 324 (53%) 75 (28%) 13 (27%)
Other 86 (14%) 4 (1%) 4 (1%)
Missing 6 (1%) 4 (1%) 0 (0%)
FAMTM score, median (IQR) 5 (47) 5 (36) 3 (15) <0.001

The median FAMTM score for the combined study population was 5 (interquartile range [IQR] 36), and median time to completion was 55 seconds (IQR 4567 seconds). A graded reduction was found in the FAMTM score for all stepwise comparisons between nonhospitalized subjects, hospitalized subjects discharged home, and hospitalized subjects not discharged home (median 5 [IQR 47] vs 5 [IQR 36] vs 3 [IQR 15]; P < 0.001 for all pairwise comparisons). The AUC for the FAMTM predicting discharge disposition (home vs not) was 0.66 (95% confidence interval [CI]: 0.58‐0.74]. After adjusting for confounders, higher FAMTM scores were independently associated with not being hospitalized, being discharged home, higher levels of education, younger age, and white race (see Supporting Table 1 in the online version of this article). Additionally, in the hospitalized subgroup, decreasing FAMTM score was significantly correlated with increasing errors on the SPMSQ (Spearman = 0.27, P < 0.001), whereas the GCS score was not correlated with the SPMSQ (Spearman = 0.05, P = 0.40) (Figure 1).

DISCUSSION

We demonstrated the utility of a rapid and accurate mobile application for assessment of mental status. The FAMTM was able to be quickly administered with a median time to completion of approximately 1 minute. The ability to detect mild alterations in mental status was shown through concurrent validity by FAMTM correlation with the SPMSQ and predictive validity with the association between the FAMTM and discharge disposition. Our study highlights the potential for the FAMTM to be used as a sensitive marker of AMS.

The novel design of the FAMTM presents unique advantages compared to current mental status testing. First, the FAMTM could allow patients with hearing impairment or language barriers to complete a mental status assessment. Additionally, the approximately 1‐minute median time to completion is much faster than other established mental status assessments including the SPMSQ (510 minutes). Compared to the SPMSQ taking 5 minutes, in a 400‐bed hospital, taken once per nursing shift, the FAMTM would save approximately 20,000 hours and 10 nursing full‐time equivalents per year.[5] Finally, many current mental status tests such as the Confusion Assessment Model utilize subjective mental status assessments.[2] However, the FAMTM is designed to be conducted through self‐assessment and, thus, could theoretically be free of observer bias. This potential for self‐administration expands beyond other proposed alternative testing mechanisms of the AMS such as ultrabrief assessments that include items such as asking subjects the months of the year backwards, and what is the day of the week?, and assessing arousal.[12, 13, 14]

In research settings and commonly in hospitals, the GCS and AVPU are used clinically for mental status assessment of hospitalized patients.[6, 15] However, similar to previous literature, our study found that the vast majority of hospitalized patients were defined as neurologically intact by the GCS, which is the more accurate predictor of the 2.[7] One major strength of the FAMTM was that it identified an extensive gradation of scores for patients previously labeled as merely alert, providing greater resolution than the GCS in quantifying mental status.

One of the key benefits of the FAMTM is that it can be measured longitudinally over the course of a patient's hospital stay. Therefore, once a baseline FAMTM score is established, variation from the patient's personal baseline could indicate mental status deterioration, which would not be affected by the patient's demographics, health status, or underlying neurocognitive deficits.

There were important limitations to this study. First, limited generalizability of these data may exist due to the single‐center setting and patient population. However, this initial study provides pilot data for further expansion into the potential broad applicability of the FAMTM to other patient populations and settings. Additionally, the cost of large‐scale implementation of the FAMTM is unknown and was beyond the scope of this pilot study. However, to reduce costs, the FAMTM technology could be integrated into existing hospital technology infrastructure. Finally, the scope of this study prevented a complete assessment of all validity measures or comparison to other mental status assessments such as the digit span or serial sevens tests. However, predictive and concurrent validity were assessed with comparison by discharge disposition, SPMSQ, and GCS scores.

In conclusion, this pilot study identifies the FAMTM application as a potentially clinically useful, novel, rapid, and feasible assessment tool of mental status in a general medicine inpatient setting.

Acknowledgements

The authors thank Frank Zadravecz, MPH, for his support with this project.

Disclosures: This research was supported in part by a grant from the National Institutes of Health (NIA 2T35AG029795‐07) and in part by career development awards granted to Dr. Churpek, Dr. Edelson, and Dr. Press by the National Heart, Lung, and Blood Institute (K08 HL121080, K23 HL097157, and K23 HL118151, respectively). Dr. Churpek has received honoraria from Chest for invited speaking engagements. Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. In addition, Dr. Edelson has received research support from Philips Healthcare (Andover, MA), the American Heart Association (Dallas, TX), and Laerdal Medical (Stavanger, Norway). She has ownership interest in Quant HC (Chicago, IL), which is developing products for risk stratification of hospitalized patients. All other authors report no potential conflicts of interest.

Files
References
  1. Han JH, Wilber ST. Altered mental status in older patients in the emergency department. Clin Geriatr Med. 2013;29(1):101136.
  2. Inouye SK, Dyck CH, Alessi CA, Balkin S, Siegal AP, Horwitz RI. Clarifying confusion: the confusion assessment method. A new method for detection of delirium. Ann Intern Med. 1990;113(12):941948.
  3. Buist M, Bernard S, Nguyen TV, Moore G, Anderson J. Association between clinically abnormal observations and subsequent in‐hospital mortality: a prospective study. Resuscitation. 2004;62(2):137141.
  4. Schuurmans MJ, Duursma SA, Shortridge‐Baggett LM. Early recognition of delirium: review of the literature. J Clin Nurs. 2001;10(6):721729.
  5. Pfeiffer E. A short portable mental status questionnaire for the assessment of organic brain deficit in elderly patients. J Am Geriatr Soc. 1975;23(10):433441.
  6. Smith GB, Prytherch DR, Meredith P, Schmidt PE, Featherstone PI. The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation. 2013;84(4):465470.
  7. Zadravecz FJ, Tien L, Robertson‐Dick BJ, et al. Comparison of mental‐status scales for predicting mortality on the general wards. J Hosp Med. 2015;10(10):658663.
  8. Teasdale G, Jennett B. Assessment of coma and impaired consciousness: a practical scale. Lancet. 1974;304(7872):8184.
  9. Erkinjuntti T, Sulkava R, Wikström J, Autio L. Short Portable Mental Status Questionnaire as a Screening Test for Dementia and Delirium Among the Elderly. J Am Geriatr Soc. 1987;35(5):412416.
  10. Meltzer D, Manning W, Morrison J, et al. Effects of physician experience on costs and outcomes on an academic general medicine service: results of a trial of hospitalists. Ann Intern Med. 2002;137(11):866874.
  11. DeLong ER, DeLong DM, Clarke‐Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837845.
  12. Fick DM, Inouye SK, Guess J, et al. Preliminary development of an ultrabrief two‐item bedside test for delirium. J Hosp Med. 2015;10(10):645650.
  13. Yevchak AM, Doherty K, Archambault EG, Kelly B, Fonda JR, Rudolph JL. The association between an ultrabrief cognitive screening in older adults and hospital outcomes. J Hosp Med. 2015;10(10):651657.
  14. Yang FM, Jones RN, Inouye SK, et al. Selecting optimal screening items for delirium: an application of item response theory. BMC Med Res Methodol. 2013;13:8.
  15. Holdgate A, Ching N, Angonese L. Variability in agreement between physicians and nurses when measuring the Glasgow Coma Scale in the emergency department limits its clinical usefulness. Emerg Med Australas. 2006;18(4):379384.
Article PDF
Issue
Journal of Hospital Medicine - 11(7)
Publications
Page Number
463-466
Sections
Files
Files
Article PDF
Article PDF

Altered mental status (AMS) is a complex spectrum of cognitive deficits that includes orientation, memory, language, visuospatial ability, and perception.[1] The clinical definitions of both delirium and dementia include AMS as a hallmark clinical prerequisite. Regardless of etiology, this broader AMS definition is particularly salient in the hospital setting, where AMS is present in up to 60% of inpatients and is associated with longer hospital stay as well as increased morbidity and mortality.[2, 3] Not surprisingly, due to the complexity of identifying and assessing changes in mental status, clinically relevant AMS is often undetected among inpatients.[2] However, when detected, the most common causes of AMS (infection, polypharmacy, and pain) are treatable, suggesting that early AMS identification could alert clinicians to early signs of clinical decompensation, potentially improving clinical outcomes.[4]

Because rapid and systemic clinical detection of AMS is limited by the complexity of mental status, a number of assessments have been created, each with their own advantages, limitations, and target populations. These assessments are often limited by time‐intensive administration, subjectivity of mental status assessment, and lack of sensitivity in general medicine patients. Time‐intensive measures, such as the Short Portable Mental Status Questionnaire (SPMSQ) have utility in the research setting, whereas current common clinical risk stratification tools (eg, National Early Warning Score) utilize simpler measures such as the Alert, Voice, Pain, Unresponsive (AVPU) and Glasgow Coma Scale (GCS) as measures of mental status.[2, 5, 6, 7, 8, 9]

To address the need for a brief, clinically feasible, accurate tool in clinical detection of AMS, our group developed a mobile application for working memory testing, the Functional Assessment of Mentation (FAMTM). In this study, we aimed to identify baseline scoring distributions of the FAMTM in a nonhospitalized subgroup, as well as assess the correlation of the FAMTM to discharge disposition and compare it to the SPMSQ in inpatients.

METHODS

Study Design

We conducted a prospective observational study. Data were collected from both hospitalized and nonhospitalized adult participants as 2 distinct subgroups. Nonhospitalized adult subjects were recruited from a university medical campus (June 2013July 2013; IRB‐12‐0175). Hospitalized participants were recruited from the general medicine service as part of an ongoing study measuring quality of care and resource allocation at the same academic medical center (June 2014August 2014; IRB‐9967).[10]

FAMTM Application

The FAMTM application is a bedside tool for working memory assessment developed for the iPhone mobile operating system (Apple Inc., Cupertino, CA) and presented on an iPad mini (Apple). The application interface displays 4 colored rectangles individually labeled with a number (see Supporting Figure 1 in the online version of this article). The testing portion of the application presents a sequence of numbered rectangles, illuminated 1 at a time in random order. Subjects are prompted first to watch and remember the sequence and then repeat the sequence by touching the screen within each numbered rectangle. Successful reproduction of the sequence is followed by a distinct and longer sequence, whereas unsuccessful attempts are followed by a shorter sequence. The final FAMTM score corresponds to the longest sequence of rectangles successfully repeated by the subject.

Figure 1
FAMTM vs GCS by SPMSQ score. Box plot of FAMTM (light grey boxes) and GCS scores (dark grey boxes). Median, and 25th and 75th quartile results are demonstrated by vertical lines within in the box, and range is shown by the whiskers. One subject was excluded from analysis due to lack of a GCS score in the medical record. Abbreviations: FAMTM, Functional Assessment of Mentation; GCS, Glasgow Coma Scale; SPMSQ, Short Portable Mental Status Questionnaire.

Data Collection

In the nonhospitalized subject population, research assistants collected demographic data immediately prior to FAMTM administration. Among hospitalized subjects, GCS information was collected by nursing staff as part of standard clinical care. One research assistant administered the SPMSQ while a second assistant, blinded to the SPMSQ and GCS scores, administered the FAMTM. Clinical data were obtained from medical records (EPIC Systems Corp., Verona, WI). Discharge disposition was dichotomized as discharged home or not.

Statistical Analyses

Demographic characteristics of the 2 subject populations were compared using Student t tests (continuous variables) and 2 tests (categorical variables). Score distribution and discharge disposition comparison was conducted with the Mann‐Whitney U test and area under receiver operating characteristic curve (AUC) analysis, using the trapezoidal rule.[11] Multivariable linear regression was used to investigate the impact of age, race, education, discharge disposition, and hospitalization status on patient scores and times. Correlations between the FAMTM and SPMSQ scores and between the GCS and SPMSQ scores were calculated using the Spearman rank test. Significance was set at a 2‐sided P value of <0.05. Analyses were conducted using Stata version 13.1 (StataCorp, College Station, TX).

RESULTS

A total of 931 subjects were enrolled in the study. In the nonhospitalized subgroup, 651 consented to study participation and 612 were included in final analysis. Subjects were excluded if they started but did not complete the application (n = 36) or were under the age of 18 years (n = 3). Of the 363 hospitalized subjects approached for enrollment, 319 were included in the final analysis. Subjects were excluded if they refused to participate (n = 23), were under the age of 18 (n = 2), had technical failures (n = 5), or had physical or visual limitations that precluded them from participation (n = 14). Within the hospitalized subgroup, 268 subjects were discharged home (85%). The table displays demographics and score distributions by subgroup.1

Demographics and FAMTM Score Distribution by Hospitalization Category
Nonhospitalized Subjects, n = 612 Hospitalized Subjects Discharged Home, n = 268 Hospitalized Subjects Discharged Elsewhere, n = 48 P Value
  • NOTE: Abbreviations: FAMTM, Functional Assessment of Mentation; IQR, interquartile range. Three hospitalized subjects were excluded from analysis due to being discharged against medical advice. FAMTM score comparisons were significant for all stepwise comparisons using P < 0.05.

Age, y 52 18 52 19 62 17 <0.001
Female sex 343 (56%) 158 (59%) 26 (54%) 0.63
Education <0.001
Less than high school graduate 31 (5%) 32 (12%) 7 (15%)
High school graduate 312 (51%) 153 (57%) 26 (54%)
College graduate 263 (43%) 43 (16%) 8 (17%)
Missing 6 (1%) 40 (15%) 7 (15%)
Race <0.001
Black 196 (32%) 185 (69%) 34 (71%)
White 324 (53%) 75 (28%) 13 (27%)
Other 86 (14%) 4 (1%) 4 (1%)
Missing 6 (1%) 4 (1%) 0 (0%)
FAMTM score, median (IQR) 5 (47) 5 (36) 3 (15) <0.001

The median FAMTM score for the combined study population was 5 (interquartile range [IQR] 36), and median time to completion was 55 seconds (IQR 4567 seconds). A graded reduction was found in the FAMTM score for all stepwise comparisons between nonhospitalized subjects, hospitalized subjects discharged home, and hospitalized subjects not discharged home (median 5 [IQR 47] vs 5 [IQR 36] vs 3 [IQR 15]; P < 0.001 for all pairwise comparisons). The AUC for the FAMTM predicting discharge disposition (home vs not) was 0.66 (95% confidence interval [CI]: 0.58‐0.74]. After adjusting for confounders, higher FAMTM scores were independently associated with not being hospitalized, being discharged home, higher levels of education, younger age, and white race (see Supporting Table 1 in the online version of this article). Additionally, in the hospitalized subgroup, decreasing FAMTM score was significantly correlated with increasing errors on the SPMSQ (Spearman = 0.27, P < 0.001), whereas the GCS score was not correlated with the SPMSQ (Spearman = 0.05, P = 0.40) (Figure 1).

DISCUSSION

We demonstrated the utility of a rapid and accurate mobile application for assessment of mental status. The FAMTM was able to be quickly administered with a median time to completion of approximately 1 minute. The ability to detect mild alterations in mental status was shown through concurrent validity by FAMTM correlation with the SPMSQ and predictive validity with the association between the FAMTM and discharge disposition. Our study highlights the potential for the FAMTM to be used as a sensitive marker of AMS.

The novel design of the FAMTM presents unique advantages compared to current mental status testing. First, the FAMTM could allow patients with hearing impairment or language barriers to complete a mental status assessment. Additionally, the approximately 1‐minute median time to completion is much faster than other established mental status assessments including the SPMSQ (510 minutes). Compared to the SPMSQ taking 5 minutes, in a 400‐bed hospital, taken once per nursing shift, the FAMTM would save approximately 20,000 hours and 10 nursing full‐time equivalents per year.[5] Finally, many current mental status tests such as the Confusion Assessment Model utilize subjective mental status assessments.[2] However, the FAMTM is designed to be conducted through self‐assessment and, thus, could theoretically be free of observer bias. This potential for self‐administration expands beyond other proposed alternative testing mechanisms of the AMS such as ultrabrief assessments that include items such as asking subjects the months of the year backwards, and what is the day of the week?, and assessing arousal.[12, 13, 14]

In research settings and commonly in hospitals, the GCS and AVPU are used clinically for mental status assessment of hospitalized patients.[6, 15] However, similar to previous literature, our study found that the vast majority of hospitalized patients were defined as neurologically intact by the GCS, which is the more accurate predictor of the 2.[7] One major strength of the FAMTM was that it identified an extensive gradation of scores for patients previously labeled as merely alert, providing greater resolution than the GCS in quantifying mental status.

One of the key benefits of the FAMTM is that it can be measured longitudinally over the course of a patient's hospital stay. Therefore, once a baseline FAMTM score is established, variation from the patient's personal baseline could indicate mental status deterioration, which would not be affected by the patient's demographics, health status, or underlying neurocognitive deficits.

There were important limitations to this study. First, limited generalizability of these data may exist due to the single‐center setting and patient population. However, this initial study provides pilot data for further expansion into the potential broad applicability of the FAMTM to other patient populations and settings. Additionally, the cost of large‐scale implementation of the FAMTM is unknown and was beyond the scope of this pilot study. However, to reduce costs, the FAMTM technology could be integrated into existing hospital technology infrastructure. Finally, the scope of this study prevented a complete assessment of all validity measures or comparison to other mental status assessments such as the digit span or serial sevens tests. However, predictive and concurrent validity were assessed with comparison by discharge disposition, SPMSQ, and GCS scores.

In conclusion, this pilot study identifies the FAMTM application as a potentially clinically useful, novel, rapid, and feasible assessment tool of mental status in a general medicine inpatient setting.

Acknowledgements

The authors thank Frank Zadravecz, MPH, for his support with this project.

Disclosures: This research was supported in part by a grant from the National Institutes of Health (NIA 2T35AG029795‐07) and in part by career development awards granted to Dr. Churpek, Dr. Edelson, and Dr. Press by the National Heart, Lung, and Blood Institute (K08 HL121080, K23 HL097157, and K23 HL118151, respectively). Dr. Churpek has received honoraria from Chest for invited speaking engagements. Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. In addition, Dr. Edelson has received research support from Philips Healthcare (Andover, MA), the American Heart Association (Dallas, TX), and Laerdal Medical (Stavanger, Norway). She has ownership interest in Quant HC (Chicago, IL), which is developing products for risk stratification of hospitalized patients. All other authors report no potential conflicts of interest.

Altered mental status (AMS) is a complex spectrum of cognitive deficits that includes orientation, memory, language, visuospatial ability, and perception.[1] The clinical definitions of both delirium and dementia include AMS as a hallmark clinical prerequisite. Regardless of etiology, this broader AMS definition is particularly salient in the hospital setting, where AMS is present in up to 60% of inpatients and is associated with longer hospital stay as well as increased morbidity and mortality.[2, 3] Not surprisingly, due to the complexity of identifying and assessing changes in mental status, clinically relevant AMS is often undetected among inpatients.[2] However, when detected, the most common causes of AMS (infection, polypharmacy, and pain) are treatable, suggesting that early AMS identification could alert clinicians to early signs of clinical decompensation, potentially improving clinical outcomes.[4]

Because rapid and systemic clinical detection of AMS is limited by the complexity of mental status, a number of assessments have been created, each with their own advantages, limitations, and target populations. These assessments are often limited by time‐intensive administration, subjectivity of mental status assessment, and lack of sensitivity in general medicine patients. Time‐intensive measures, such as the Short Portable Mental Status Questionnaire (SPMSQ) have utility in the research setting, whereas current common clinical risk stratification tools (eg, National Early Warning Score) utilize simpler measures such as the Alert, Voice, Pain, Unresponsive (AVPU) and Glasgow Coma Scale (GCS) as measures of mental status.[2, 5, 6, 7, 8, 9]

To address the need for a brief, clinically feasible, accurate tool in clinical detection of AMS, our group developed a mobile application for working memory testing, the Functional Assessment of Mentation (FAMTM). In this study, we aimed to identify baseline scoring distributions of the FAMTM in a nonhospitalized subgroup, as well as assess the correlation of the FAMTM to discharge disposition and compare it to the SPMSQ in inpatients.

METHODS

Study Design

We conducted a prospective observational study. Data were collected from both hospitalized and nonhospitalized adult participants as 2 distinct subgroups. Nonhospitalized adult subjects were recruited from a university medical campus (June 2013July 2013; IRB‐12‐0175). Hospitalized participants were recruited from the general medicine service as part of an ongoing study measuring quality of care and resource allocation at the same academic medical center (June 2014August 2014; IRB‐9967).[10]

FAMTM Application

The FAMTM application is a bedside tool for working memory assessment developed for the iPhone mobile operating system (Apple Inc., Cupertino, CA) and presented on an iPad mini (Apple). The application interface displays 4 colored rectangles individually labeled with a number (see Supporting Figure 1 in the online version of this article). The testing portion of the application presents a sequence of numbered rectangles, illuminated 1 at a time in random order. Subjects are prompted first to watch and remember the sequence and then repeat the sequence by touching the screen within each numbered rectangle. Successful reproduction of the sequence is followed by a distinct and longer sequence, whereas unsuccessful attempts are followed by a shorter sequence. The final FAMTM score corresponds to the longest sequence of rectangles successfully repeated by the subject.

Figure 1
FAMTM vs GCS by SPMSQ score. Box plot of FAMTM (light grey boxes) and GCS scores (dark grey boxes). Median, and 25th and 75th quartile results are demonstrated by vertical lines within in the box, and range is shown by the whiskers. One subject was excluded from analysis due to lack of a GCS score in the medical record. Abbreviations: FAMTM, Functional Assessment of Mentation; GCS, Glasgow Coma Scale; SPMSQ, Short Portable Mental Status Questionnaire.

Data Collection

In the nonhospitalized subject population, research assistants collected demographic data immediately prior to FAMTM administration. Among hospitalized subjects, GCS information was collected by nursing staff as part of standard clinical care. One research assistant administered the SPMSQ while a second assistant, blinded to the SPMSQ and GCS scores, administered the FAMTM. Clinical data were obtained from medical records (EPIC Systems Corp., Verona, WI). Discharge disposition was dichotomized as discharged home or not.

Statistical Analyses

Demographic characteristics of the 2 subject populations were compared using Student t tests (continuous variables) and 2 tests (categorical variables). Score distribution and discharge disposition comparison was conducted with the Mann‐Whitney U test and area under receiver operating characteristic curve (AUC) analysis, using the trapezoidal rule.[11] Multivariable linear regression was used to investigate the impact of age, race, education, discharge disposition, and hospitalization status on patient scores and times. Correlations between the FAMTM and SPMSQ scores and between the GCS and SPMSQ scores were calculated using the Spearman rank test. Significance was set at a 2‐sided P value of <0.05. Analyses were conducted using Stata version 13.1 (StataCorp, College Station, TX).

RESULTS

A total of 931 subjects were enrolled in the study. In the nonhospitalized subgroup, 651 consented to study participation and 612 were included in final analysis. Subjects were excluded if they started but did not complete the application (n = 36) or were under the age of 18 years (n = 3). Of the 363 hospitalized subjects approached for enrollment, 319 were included in the final analysis. Subjects were excluded if they refused to participate (n = 23), were under the age of 18 (n = 2), had technical failures (n = 5), or had physical or visual limitations that precluded them from participation (n = 14). Within the hospitalized subgroup, 268 subjects were discharged home (85%). The table displays demographics and score distributions by subgroup.1

Demographics and FAMTM Score Distribution by Hospitalization Category
Nonhospitalized Subjects, n = 612 Hospitalized Subjects Discharged Home, n = 268 Hospitalized Subjects Discharged Elsewhere, n = 48 P Value
  • NOTE: Abbreviations: FAMTM, Functional Assessment of Mentation; IQR, interquartile range. Three hospitalized subjects were excluded from analysis due to being discharged against medical advice. FAMTM score comparisons were significant for all stepwise comparisons using P < 0.05.

Age, y 52 18 52 19 62 17 <0.001
Female sex 343 (56%) 158 (59%) 26 (54%) 0.63
Education <0.001
Less than high school graduate 31 (5%) 32 (12%) 7 (15%)
High school graduate 312 (51%) 153 (57%) 26 (54%)
College graduate 263 (43%) 43 (16%) 8 (17%)
Missing 6 (1%) 40 (15%) 7 (15%)
Race <0.001
Black 196 (32%) 185 (69%) 34 (71%)
White 324 (53%) 75 (28%) 13 (27%)
Other 86 (14%) 4 (1%) 4 (1%)
Missing 6 (1%) 4 (1%) 0 (0%)
FAMTM score, median (IQR) 5 (47) 5 (36) 3 (15) <0.001

The median FAMTM score for the combined study population was 5 (interquartile range [IQR] 36), and median time to completion was 55 seconds (IQR 4567 seconds). A graded reduction was found in the FAMTM score for all stepwise comparisons between nonhospitalized subjects, hospitalized subjects discharged home, and hospitalized subjects not discharged home (median 5 [IQR 47] vs 5 [IQR 36] vs 3 [IQR 15]; P < 0.001 for all pairwise comparisons). The AUC for the FAMTM predicting discharge disposition (home vs not) was 0.66 (95% confidence interval [CI]: 0.58‐0.74]. After adjusting for confounders, higher FAMTM scores were independently associated with not being hospitalized, being discharged home, higher levels of education, younger age, and white race (see Supporting Table 1 in the online version of this article). Additionally, in the hospitalized subgroup, decreasing FAMTM score was significantly correlated with increasing errors on the SPMSQ (Spearman = 0.27, P < 0.001), whereas the GCS score was not correlated with the SPMSQ (Spearman = 0.05, P = 0.40) (Figure 1).

DISCUSSION

We demonstrated the utility of a rapid and accurate mobile application for assessment of mental status. The FAMTM was able to be quickly administered with a median time to completion of approximately 1 minute. The ability to detect mild alterations in mental status was shown through concurrent validity by FAMTM correlation with the SPMSQ and predictive validity with the association between the FAMTM and discharge disposition. Our study highlights the potential for the FAMTM to be used as a sensitive marker of AMS.

The novel design of the FAMTM presents unique advantages compared to current mental status testing. First, the FAMTM could allow patients with hearing impairment or language barriers to complete a mental status assessment. Additionally, the approximately 1‐minute median time to completion is much faster than other established mental status assessments including the SPMSQ (510 minutes). Compared to the SPMSQ taking 5 minutes, in a 400‐bed hospital, taken once per nursing shift, the FAMTM would save approximately 20,000 hours and 10 nursing full‐time equivalents per year.[5] Finally, many current mental status tests such as the Confusion Assessment Model utilize subjective mental status assessments.[2] However, the FAMTM is designed to be conducted through self‐assessment and, thus, could theoretically be free of observer bias. This potential for self‐administration expands beyond other proposed alternative testing mechanisms of the AMS such as ultrabrief assessments that include items such as asking subjects the months of the year backwards, and what is the day of the week?, and assessing arousal.[12, 13, 14]

In research settings and commonly in hospitals, the GCS and AVPU are used clinically for mental status assessment of hospitalized patients.[6, 15] However, similar to previous literature, our study found that the vast majority of hospitalized patients were defined as neurologically intact by the GCS, which is the more accurate predictor of the 2.[7] One major strength of the FAMTM was that it identified an extensive gradation of scores for patients previously labeled as merely alert, providing greater resolution than the GCS in quantifying mental status.

One of the key benefits of the FAMTM is that it can be measured longitudinally over the course of a patient's hospital stay. Therefore, once a baseline FAMTM score is established, variation from the patient's personal baseline could indicate mental status deterioration, which would not be affected by the patient's demographics, health status, or underlying neurocognitive deficits.

There were important limitations to this study. First, limited generalizability of these data may exist due to the single‐center setting and patient population. However, this initial study provides pilot data for further expansion into the potential broad applicability of the FAMTM to other patient populations and settings. Additionally, the cost of large‐scale implementation of the FAMTM is unknown and was beyond the scope of this pilot study. However, to reduce costs, the FAMTM technology could be integrated into existing hospital technology infrastructure. Finally, the scope of this study prevented a complete assessment of all validity measures or comparison to other mental status assessments such as the digit span or serial sevens tests. However, predictive and concurrent validity were assessed with comparison by discharge disposition, SPMSQ, and GCS scores.

In conclusion, this pilot study identifies the FAMTM application as a potentially clinically useful, novel, rapid, and feasible assessment tool of mental status in a general medicine inpatient setting.

Acknowledgements

The authors thank Frank Zadravecz, MPH, for his support with this project.

Disclosures: This research was supported in part by a grant from the National Institutes of Health (NIA 2T35AG029795‐07) and in part by career development awards granted to Dr. Churpek, Dr. Edelson, and Dr. Press by the National Heart, Lung, and Blood Institute (K08 HL121080, K23 HL097157, and K23 HL118151, respectively). Dr. Churpek has received honoraria from Chest for invited speaking engagements. Drs. Churpek and Edelson have a patent pending (ARCD. P0535US.P2) for risk stratification algorithms for hospitalized patients. In addition, Dr. Edelson has received research support from Philips Healthcare (Andover, MA), the American Heart Association (Dallas, TX), and Laerdal Medical (Stavanger, Norway). She has ownership interest in Quant HC (Chicago, IL), which is developing products for risk stratification of hospitalized patients. All other authors report no potential conflicts of interest.

References
  1. Han JH, Wilber ST. Altered mental status in older patients in the emergency department. Clin Geriatr Med. 2013;29(1):101136.
  2. Inouye SK, Dyck CH, Alessi CA, Balkin S, Siegal AP, Horwitz RI. Clarifying confusion: the confusion assessment method. A new method for detection of delirium. Ann Intern Med. 1990;113(12):941948.
  3. Buist M, Bernard S, Nguyen TV, Moore G, Anderson J. Association between clinically abnormal observations and subsequent in‐hospital mortality: a prospective study. Resuscitation. 2004;62(2):137141.
  4. Schuurmans MJ, Duursma SA, Shortridge‐Baggett LM. Early recognition of delirium: review of the literature. J Clin Nurs. 2001;10(6):721729.
  5. Pfeiffer E. A short portable mental status questionnaire for the assessment of organic brain deficit in elderly patients. J Am Geriatr Soc. 1975;23(10):433441.
  6. Smith GB, Prytherch DR, Meredith P, Schmidt PE, Featherstone PI. The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation. 2013;84(4):465470.
  7. Zadravecz FJ, Tien L, Robertson‐Dick BJ, et al. Comparison of mental‐status scales for predicting mortality on the general wards. J Hosp Med. 2015;10(10):658663.
  8. Teasdale G, Jennett B. Assessment of coma and impaired consciousness: a practical scale. Lancet. 1974;304(7872):8184.
  9. Erkinjuntti T, Sulkava R, Wikström J, Autio L. Short Portable Mental Status Questionnaire as a Screening Test for Dementia and Delirium Among the Elderly. J Am Geriatr Soc. 1987;35(5):412416.
  10. Meltzer D, Manning W, Morrison J, et al. Effects of physician experience on costs and outcomes on an academic general medicine service: results of a trial of hospitalists. Ann Intern Med. 2002;137(11):866874.
  11. DeLong ER, DeLong DM, Clarke‐Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837845.
  12. Fick DM, Inouye SK, Guess J, et al. Preliminary development of an ultrabrief two‐item bedside test for delirium. J Hosp Med. 2015;10(10):645650.
  13. Yevchak AM, Doherty K, Archambault EG, Kelly B, Fonda JR, Rudolph JL. The association between an ultrabrief cognitive screening in older adults and hospital outcomes. J Hosp Med. 2015;10(10):651657.
  14. Yang FM, Jones RN, Inouye SK, et al. Selecting optimal screening items for delirium: an application of item response theory. BMC Med Res Methodol. 2013;13:8.
  15. Holdgate A, Ching N, Angonese L. Variability in agreement between physicians and nurses when measuring the Glasgow Coma Scale in the emergency department limits its clinical usefulness. Emerg Med Australas. 2006;18(4):379384.
References
  1. Han JH, Wilber ST. Altered mental status in older patients in the emergency department. Clin Geriatr Med. 2013;29(1):101136.
  2. Inouye SK, Dyck CH, Alessi CA, Balkin S, Siegal AP, Horwitz RI. Clarifying confusion: the confusion assessment method. A new method for detection of delirium. Ann Intern Med. 1990;113(12):941948.
  3. Buist M, Bernard S, Nguyen TV, Moore G, Anderson J. Association between clinically abnormal observations and subsequent in‐hospital mortality: a prospective study. Resuscitation. 2004;62(2):137141.
  4. Schuurmans MJ, Duursma SA, Shortridge‐Baggett LM. Early recognition of delirium: review of the literature. J Clin Nurs. 2001;10(6):721729.
  5. Pfeiffer E. A short portable mental status questionnaire for the assessment of organic brain deficit in elderly patients. J Am Geriatr Soc. 1975;23(10):433441.
  6. Smith GB, Prytherch DR, Meredith P, Schmidt PE, Featherstone PI. The ability of the National Early Warning Score (NEWS) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation. 2013;84(4):465470.
  7. Zadravecz FJ, Tien L, Robertson‐Dick BJ, et al. Comparison of mental‐status scales for predicting mortality on the general wards. J Hosp Med. 2015;10(10):658663.
  8. Teasdale G, Jennett B. Assessment of coma and impaired consciousness: a practical scale. Lancet. 1974;304(7872):8184.
  9. Erkinjuntti T, Sulkava R, Wikström J, Autio L. Short Portable Mental Status Questionnaire as a Screening Test for Dementia and Delirium Among the Elderly. J Am Geriatr Soc. 1987;35(5):412416.
  10. Meltzer D, Manning W, Morrison J, et al. Effects of physician experience on costs and outcomes on an academic general medicine service: results of a trial of hospitalists. Ann Intern Med. 2002;137(11):866874.
  11. DeLong ER, DeLong DM, Clarke‐Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837845.
  12. Fick DM, Inouye SK, Guess J, et al. Preliminary development of an ultrabrief two‐item bedside test for delirium. J Hosp Med. 2015;10(10):645650.
  13. Yevchak AM, Doherty K, Archambault EG, Kelly B, Fonda JR, Rudolph JL. The association between an ultrabrief cognitive screening in older adults and hospital outcomes. J Hosp Med. 2015;10(10):651657.
  14. Yang FM, Jones RN, Inouye SK, et al. Selecting optimal screening items for delirium: an application of item response theory. BMC Med Res Methodol. 2013;13:8.
  15. Holdgate A, Ching N, Angonese L. Variability in agreement between physicians and nurses when measuring the Glasgow Coma Scale in the emergency department limits its clinical usefulness. Emerg Med Australas. 2006;18(4):379384.
Issue
Journal of Hospital Medicine - 11(7)
Issue
Journal of Hospital Medicine - 11(7)
Page Number
463-466
Page Number
463-466
Publications
Publications
Article Type
Display Headline
Testing the functional assessment of mentation: A mobile application based assessment of mental status
Display Headline
Testing the functional assessment of mentation: A mobile application based assessment of mental status
Sections
Article Source
© 2016 Society of Hospital Medicine
Disallow All Ads
Correspondence Location
Address for correspondence and reprint requests: Dana P. Edelson, MD, University of Chicago Medicine, Section of Hospital Medicine, 5841 S. Maryland Avenue, MC 5000, Chicago, IL 60637; Telephone: 773‐834‐2191; Fax: 773‐795‐7398; E‐mail: dperes@medicine.bsd.uchicago.edu
Content Gating
Gated (full article locked unless allowed per User)
Gating Strategy
First Peek Free
Article PDF Media
Media Files