User login
That is the message behind numerous publications in recent years, and the COVID-19 pandemic has accelerated that trend.
“The publications have just skyrocketed since 2018, but I think there are still some additional tests that we need to validate using this medium of assessment. Also, I think we need to kind of put on our thinking caps as a field and think outside the box. What novel tests can we develop that will capitalize upon the telehealth environment – interactive tests that are monitoring [the individuals’] performance in real time and giving the examiner feedback, things like that,” said Munro Cullum, PhD, in an interview. Dr. Cullum spoke on the topic at the 2021 Alzheimer’s Association International Conference.
Still, challenges remain, especially factors in the home environment that can adversely affect testing. “Some of our tests are a question-answer, pencil-paper sort of tests that can be well suited to a telemedicine environment, [but] other tests don’t translate as well. So we still have a ways to go to kind of get our test to the next generation when being administered during this type of assessment. But a lot of the verbal tests work extremely well,” said Dr. Cullum, who is a professor of psychiatry at the University of Texas Southwestern Medical Center, Dallas.
Preliminary evidence of equivalence
Some years ago, Dr. Cullum was interested in getting a better understanding of what existing tests could best be performed remotely, and what populations could most benefit from remote assessments. Existing studies were generally supportive of remote testing, but varied significantly in their methodology and design. He went on to publish a study in 2014 showing equivalency of existing tests in the in-person and remote environment, and that helped pave the way for a wave of more recent studies that seem to confirm equivalence of in-person methods.
“If you look at the literature overall, there is a nice, growing body of evidence suggesting support for a host of neuropsychological test instruments. For the most part, almost all have shown good reliability across test conditions,” Dr. Cullum said during the talk.
He said that he is often asked if different test norms will be required for remote tests, but that doesn’t seem to be a concern. “It looks like the regular old neuropsych test norms should serve as well in this remote assessment environment. Although as within hospital testing of patients, conservative use of norms is always an order. They are interpretive guidelines,” he added.
One concern is potential threats to validity within the home environment. He posted an image of a woman at home, taking a remote cognitive test. The desk she sat at overlooked a wooded scene, and had a sewing machine on it. A small dog lay in her lap. “So assessing the home environment, ensuring that it is as close to a clinical standard setting as possible, is certainly advised,” said Dr. Cullum.
Although much progress has been made in studying existing tests in a telemedicine environment, many commonly used tests still haven’t been studied. The risk of intrusions and distractions, and even connectivity issues, can be limiting factors. Some tests may be ineligible for remote use due to copyright issues that might prevent required materials from being displayed online. For those reasons and others, not all individuals are suited for a remote test.
Finally, remote tests should be viewed with healthy skepticism. “In doing clinical evaluations this way, we have to be extra careful to not mis- or overinterpret the findings in case there were any distractions or glitches in the examination that came up during the test,” said Dr. Cullum.
Looking toward the future
Moving forward, Dr. Cullum called for more research to design new tests to exploit the telehealth format. “I think this is a really important opportunity for new test development in neuropsychology with increasing incorporation of computerized measures and integration with more cognitive neuroscience and clinical neuropsychology principles.”
He also suggested that remote testing could be combined with neuroimaging, neuromodulation, and even portable magnetoencephalography. “These opportunities for research can enhance compliance, enhance large-scale studies to allow for the inclusion of brief cognitive outcome metrics that might not have other otherwise been [possible],” said Dr. Cullum.
During the question-and-answer session, someone asked if the momentum towards telehealth will continue once the COVID-19 pandemic recedes. “We believe telehealth is here to stay, or at least I do,” said session moderator Allison Lindauer, PhD, who was asked to comment. Dr. Lindauer is an associate professor at the Layton Aging and Alzheimer’s Disease Center in Portland, Ore.
Dr. Lindauer has also conducted studies on telehealth-delivered assessments and also found encouraging results. “Work like this says, we have confidence in our work, we can believe that what we’re assessing and what we’re doing – if we did it face to face, we would get similar results,” Dr. Lindauer said in an interview.
Plenty of challenges remain, and the most important is widely available broadband internet, said Dr. Lindauer. “We need a huge push to get broadband everywhere. Granted, you’re going to have people that don’t want to use the computer, or they’re nervous about doing it online. But in my experience, most people with enough coaching can do it and are fine with it.”
Dr. Cullum and Dr. Lindauer have no relevant financial disclosures.
That is the message behind numerous publications in recent years, and the COVID-19 pandemic has accelerated that trend.
“The publications have just skyrocketed since 2018, but I think there are still some additional tests that we need to validate using this medium of assessment. Also, I think we need to kind of put on our thinking caps as a field and think outside the box. What novel tests can we develop that will capitalize upon the telehealth environment – interactive tests that are monitoring [the individuals’] performance in real time and giving the examiner feedback, things like that,” said Munro Cullum, PhD, in an interview. Dr. Cullum spoke on the topic at the 2021 Alzheimer’s Association International Conference.
Still, challenges remain, especially factors in the home environment that can adversely affect testing. “Some of our tests are a question-answer, pencil-paper sort of tests that can be well suited to a telemedicine environment, [but] other tests don’t translate as well. So we still have a ways to go to kind of get our test to the next generation when being administered during this type of assessment. But a lot of the verbal tests work extremely well,” said Dr. Cullum, who is a professor of psychiatry at the University of Texas Southwestern Medical Center, Dallas.
Preliminary evidence of equivalence
Some years ago, Dr. Cullum was interested in getting a better understanding of what existing tests could best be performed remotely, and what populations could most benefit from remote assessments. Existing studies were generally supportive of remote testing, but varied significantly in their methodology and design. He went on to publish a study in 2014 showing equivalency of existing tests in the in-person and remote environment, and that helped pave the way for a wave of more recent studies that seem to confirm equivalence of in-person methods.
“If you look at the literature overall, there is a nice, growing body of evidence suggesting support for a host of neuropsychological test instruments. For the most part, almost all have shown good reliability across test conditions,” Dr. Cullum said during the talk.
He said that he is often asked if different test norms will be required for remote tests, but that doesn’t seem to be a concern. “It looks like the regular old neuropsych test norms should serve as well in this remote assessment environment. Although as within hospital testing of patients, conservative use of norms is always an order. They are interpretive guidelines,” he added.
One concern is potential threats to validity within the home environment. He posted an image of a woman at home, taking a remote cognitive test. The desk she sat at overlooked a wooded scene, and had a sewing machine on it. A small dog lay in her lap. “So assessing the home environment, ensuring that it is as close to a clinical standard setting as possible, is certainly advised,” said Dr. Cullum.
Although much progress has been made in studying existing tests in a telemedicine environment, many commonly used tests still haven’t been studied. The risk of intrusions and distractions, and even connectivity issues, can be limiting factors. Some tests may be ineligible for remote use due to copyright issues that might prevent required materials from being displayed online. For those reasons and others, not all individuals are suited for a remote test.
Finally, remote tests should be viewed with healthy skepticism. “In doing clinical evaluations this way, we have to be extra careful to not mis- or overinterpret the findings in case there were any distractions or glitches in the examination that came up during the test,” said Dr. Cullum.
Looking toward the future
Moving forward, Dr. Cullum called for more research to design new tests to exploit the telehealth format. “I think this is a really important opportunity for new test development in neuropsychology with increasing incorporation of computerized measures and integration with more cognitive neuroscience and clinical neuropsychology principles.”
He also suggested that remote testing could be combined with neuroimaging, neuromodulation, and even portable magnetoencephalography. “These opportunities for research can enhance compliance, enhance large-scale studies to allow for the inclusion of brief cognitive outcome metrics that might not have other otherwise been [possible],” said Dr. Cullum.
During the question-and-answer session, someone asked if the momentum towards telehealth will continue once the COVID-19 pandemic recedes. “We believe telehealth is here to stay, or at least I do,” said session moderator Allison Lindauer, PhD, who was asked to comment. Dr. Lindauer is an associate professor at the Layton Aging and Alzheimer’s Disease Center in Portland, Ore.
Dr. Lindauer has also conducted studies on telehealth-delivered assessments and also found encouraging results. “Work like this says, we have confidence in our work, we can believe that what we’re assessing and what we’re doing – if we did it face to face, we would get similar results,” Dr. Lindauer said in an interview.
Plenty of challenges remain, and the most important is widely available broadband internet, said Dr. Lindauer. “We need a huge push to get broadband everywhere. Granted, you’re going to have people that don’t want to use the computer, or they’re nervous about doing it online. But in my experience, most people with enough coaching can do it and are fine with it.”
Dr. Cullum and Dr. Lindauer have no relevant financial disclosures.
That is the message behind numerous publications in recent years, and the COVID-19 pandemic has accelerated that trend.
“The publications have just skyrocketed since 2018, but I think there are still some additional tests that we need to validate using this medium of assessment. Also, I think we need to kind of put on our thinking caps as a field and think outside the box. What novel tests can we develop that will capitalize upon the telehealth environment – interactive tests that are monitoring [the individuals’] performance in real time and giving the examiner feedback, things like that,” said Munro Cullum, PhD, in an interview. Dr. Cullum spoke on the topic at the 2021 Alzheimer’s Association International Conference.
Still, challenges remain, especially factors in the home environment that can adversely affect testing. “Some of our tests are a question-answer, pencil-paper sort of tests that can be well suited to a telemedicine environment, [but] other tests don’t translate as well. So we still have a ways to go to kind of get our test to the next generation when being administered during this type of assessment. But a lot of the verbal tests work extremely well,” said Dr. Cullum, who is a professor of psychiatry at the University of Texas Southwestern Medical Center, Dallas.
Preliminary evidence of equivalence
Some years ago, Dr. Cullum was interested in getting a better understanding of what existing tests could best be performed remotely, and what populations could most benefit from remote assessments. Existing studies were generally supportive of remote testing, but varied significantly in their methodology and design. He went on to publish a study in 2014 showing equivalency of existing tests in the in-person and remote environment, and that helped pave the way for a wave of more recent studies that seem to confirm equivalence of in-person methods.
“If you look at the literature overall, there is a nice, growing body of evidence suggesting support for a host of neuropsychological test instruments. For the most part, almost all have shown good reliability across test conditions,” Dr. Cullum said during the talk.
He said that he is often asked if different test norms will be required for remote tests, but that doesn’t seem to be a concern. “It looks like the regular old neuropsych test norms should serve as well in this remote assessment environment. Although as within hospital testing of patients, conservative use of norms is always an order. They are interpretive guidelines,” he added.
One concern is potential threats to validity within the home environment. He posted an image of a woman at home, taking a remote cognitive test. The desk she sat at overlooked a wooded scene, and had a sewing machine on it. A small dog lay in her lap. “So assessing the home environment, ensuring that it is as close to a clinical standard setting as possible, is certainly advised,” said Dr. Cullum.
Although much progress has been made in studying existing tests in a telemedicine environment, many commonly used tests still haven’t been studied. The risk of intrusions and distractions, and even connectivity issues, can be limiting factors. Some tests may be ineligible for remote use due to copyright issues that might prevent required materials from being displayed online. For those reasons and others, not all individuals are suited for a remote test.
Finally, remote tests should be viewed with healthy skepticism. “In doing clinical evaluations this way, we have to be extra careful to not mis- or overinterpret the findings in case there were any distractions or glitches in the examination that came up during the test,” said Dr. Cullum.
Looking toward the future
Moving forward, Dr. Cullum called for more research to design new tests to exploit the telehealth format. “I think this is a really important opportunity for new test development in neuropsychology with increasing incorporation of computerized measures and integration with more cognitive neuroscience and clinical neuropsychology principles.”
He also suggested that remote testing could be combined with neuroimaging, neuromodulation, and even portable magnetoencephalography. “These opportunities for research can enhance compliance, enhance large-scale studies to allow for the inclusion of brief cognitive outcome metrics that might not have other otherwise been [possible],” said Dr. Cullum.
During the question-and-answer session, someone asked if the momentum towards telehealth will continue once the COVID-19 pandemic recedes. “We believe telehealth is here to stay, or at least I do,” said session moderator Allison Lindauer, PhD, who was asked to comment. Dr. Lindauer is an associate professor at the Layton Aging and Alzheimer’s Disease Center in Portland, Ore.
Dr. Lindauer has also conducted studies on telehealth-delivered assessments and also found encouraging results. “Work like this says, we have confidence in our work, we can believe that what we’re assessing and what we’re doing – if we did it face to face, we would get similar results,” Dr. Lindauer said in an interview.
Plenty of challenges remain, and the most important is widely available broadband internet, said Dr. Lindauer. “We need a huge push to get broadband everywhere. Granted, you’re going to have people that don’t want to use the computer, or they’re nervous about doing it online. But in my experience, most people with enough coaching can do it and are fine with it.”
Dr. Cullum and Dr. Lindauer have no relevant financial disclosures.
FROM AAIC 2021