Two-minute Alzheimer’s test? Not so fast!

BOSTON – The quality of websites that purport to help visitors or other people acting on their behalf to diagnose Alzheimer’s disease are very low and should not be used for such a purpose, cautioned an investigator at the Alzheimer’s Association International Conference 2013.

Independent panelists who reviewed 16 online tests for Alzheimer’s disease (AD) found that none provided validated results, and all were sorely lacking in ethical standards, reported Julie Robillard, Ph.D., a postdoctoral fellow at the National Core for Neuroethics at the University of British Columbia in Vancouver, Canada.

More than half of all Americans aged 65 years and older use the Internet, and approximately 80% of all adults find health information online, Dr. Robillard noted.

"There’s an increasingly popular behavior online, which is self-diagnosis, and the truth is we have almost no information about the quality of information about Alzheimer’s online," she said.

To at least partially fill in this knowledge gap, she and her colleagues conducted a keyword search to identify 16 freely available online tests purporting to be for Alzheimer’s disease and asked two independent panels of expert reviewers to rate each test on a scale of 1 (very poor) to 10 (excellent) for content, breadth, peer review, and reliability.

The tests were found on news, not-for-profit, academic, commercial, and entertainment sites, with monthly traffic ranging from 200 to 8.8 million unique visits. At least one test could be completed in about 2 minutes; others took as long as 1 hour to complete, Dr. Robillard estimated.

"So, in 2 minutes, you can go online and find out if you have Alzheimer’s disease," she joked.

Thirteen were designed to be self tests, and three were designed to be filled out by a proxy such as a relative or friend. Six sites relied primarily on questionnaires asking about risk factors and symptoms, and 10 used performance measures.

The tests gave users a variety of possible outcomes. For example, nine tests classified users as not at risk, at risk, or as already having AD. Another three rated responses on a continuum based on test answers. Of the four remaining tests, two were pass/fail, one was fail only, and one offered no outcomes.

The scientific validity and reliability of the tests were rated as mediocre at best by the panelists, indicating that the tests were generally poor at achieving their stated goals. The tests were generally poor at including current peer-reviewed evidence, and the experts also said that test/retest reliability was also quite poor.

Experts in human-computer interaction gave decent marks (scores of about 7) for their ease of use. But clinicians, ethicists, and neuropsychologists were less forgiving on whether the tests could detect if the user was performing properly or merely playing around and how easy the tests were to use for those with lower levels of computer literacy. None rated the usability of any test much higher than 5.

None of the tests rated better than poor for meeting various ethical standards, such as providing upfront information about the nature and scope of the test, informed consent, and privacy discussions. The panelists also gave poor ratings to the tests’ wording of outcomes, the clarity of their interpretation, and their provision of appropriate advice for follow-up. Conflict-of-interest disclosures were not present, either.

"Is self-diagnosis appropriate at all? I think that professional consensus is that it’s not, so we have to wonder how we handle these tests being online and people doing them," Dr. Robillard said.

Her next step will be to explore in depth the potential harms or benefits of online dementia screening and to evaluate how such testing may affect clinician/patient relationships.

The study was supported by the Canadian Institute for Health Research and the Canadian Dementia Knowledge Translation Network. Dr. Robillard disclosed that two of her coauthors are involved in developing a computerized testing tool for mild cognitive impairment and dementia for use in specialty-care settings. Dr. Robillard reported having no personal disclosures.

BOSTON – The quality of websites that purport to help visitors or other people acting on their behalf to diagnose Alzheimer’s disease are very low and should not be used for such a purpose, cautioned an investigator at the Alzheimer’s Association International Conference 2013.

Independent panelists who reviewed 16 online tests for Alzheimer’s disease (AD) found that none provided validated results, and all were sorely lacking in ethical standards, reported Julie Robillard, Ph.D., a postdoctoral fellow at the National Core for Neuroethics at the University of British Columbia in Vancouver, Canada.

More than half of all Americans aged 65 years and older use the Internet, and approximately 80% of all adults find health information online, Dr. Robillard noted.

"There’s an increasingly popular behavior online, which is self-diagnosis, and the truth is we have almost no information about the quality of information about Alzheimer’s online," she said.

To at least partially fill in this knowledge gap, she and her colleagues conducted a keyword search to identify 16 freely available online tests purporting to be for Alzheimer’s disease and asked two independent panels of expert reviewers to rate each test on a scale of 1 (very poor) to 10 (excellent) for content, breadth, peer review, and reliability.

The tests were found on news, not-for-profit, academic, commercial, and entertainment sites, with monthly traffic ranging from 200 to 8.8 million unique visits. At least one test could be completed in about 2 minutes; others took as long as 1 hour to complete, Dr. Robillard estimated.

"So, in 2 minutes, you can go online and find out if you have Alzheimer’s disease," she joked.

Thirteen were designed to be self tests, and three were designed to be filled out by a proxy such as a relative or friend. Six sites relied primarily on questionnaires asking about risk factors and symptoms, and 10 used performance measures.

The tests gave users a variety of possible outcomes. For example, nine tests classified users as not at risk, at risk, or as already having AD. Another three rated responses on a continuum based on test answers. Of the four remaining tests, two were pass/fail, one was fail only, and one offered no outcomes.

The scientific validity and reliability of the tests were rated as mediocre at best by the panelists, indicating that the tests were generally poor at achieving their stated goals. The tests were generally poor at including current peer-reviewed evidence, and the experts also said that test/retest reliability was also quite poor.

Experts in human-computer interaction gave decent marks (scores of about 7) for their ease of use. But clinicians, ethicists, and neuropsychologists were less forgiving on whether the tests could detect if the user was performing properly or merely playing around and how easy the tests were to use for those with lower levels of computer literacy. None rated the usability of any test much higher than 5.

None of the tests rated better than poor for meeting various ethical standards, such as providing upfront information about the nature and scope of the test, informed consent, and privacy discussions. The panelists also gave poor ratings to the tests’ wording of outcomes, the clarity of their interpretation, and their provision of appropriate advice for follow-up. Conflict-of-interest disclosures were not present, either.

"Is self-diagnosis appropriate at all? I think that professional consensus is that it’s not, so we have to wonder how we handle these tests being online and people doing them," Dr. Robillard said.

Her next step will be to explore in depth the potential harms or benefits of online dementia screening and to evaluate how such testing may affect clinician/patient relationships.

The study was supported by the Canadian Institute for Health Research and the Canadian Dementia Knowledge Translation Network. Dr. Robillard disclosed that two of her coauthors are involved in developing a computerized testing tool for mild cognitive impairment and dementia for use in specialty-care settings. Dr. Robillard reported having no personal disclosures.

BOSTON – The quality of websites that purport to help visitors or other people acting on their behalf to diagnose Alzheimer’s disease are very low and should not be used for such a purpose, cautioned an investigator at the Alzheimer’s Association International Conference 2013.

Independent panelists who reviewed 16 online tests for Alzheimer’s disease (AD) found that none provided validated results, and all were sorely lacking in ethical standards, reported Julie Robillard, Ph.D., a postdoctoral fellow at the National Core for Neuroethics at the University of British Columbia in Vancouver, Canada.

More than half of all Americans aged 65 years and older use the Internet, and approximately 80% of all adults find health information online, Dr. Robillard noted.

"There’s an increasingly popular behavior online, which is self-diagnosis, and the truth is we have almost no information about the quality of information about Alzheimer’s online," she said.

To at least partially fill in this knowledge gap, she and her colleagues conducted a keyword search to identify 16 freely available online tests purporting to be for Alzheimer’s disease and asked two independent panels of expert reviewers to rate each test on a scale of 1 (very poor) to 10 (excellent) for content, breadth, peer review, and reliability.

The tests were found on news, not-for-profit, academic, commercial, and entertainment sites, with monthly traffic ranging from 200 to 8.8 million unique visits. At least one test could be completed in about 2 minutes; others took as long as 1 hour to complete, Dr. Robillard estimated.

"So, in 2 minutes, you can go online and find out if you have Alzheimer’s disease," she joked.

Thirteen were designed to be self tests, and three were designed to be filled out by a proxy such as a relative or friend. Six sites relied primarily on questionnaires asking about risk factors and symptoms, and 10 used performance measures.

The tests gave users a variety of possible outcomes. For example, nine tests classified users as not at risk, at risk, or as already having AD. Another three rated responses on a continuum based on test answers. Of the four remaining tests, two were pass/fail, one was fail only, and one offered no outcomes.

The scientific validity and reliability of the tests were rated as mediocre at best by the panelists, indicating that the tests were generally poor at achieving their stated goals. The tests were generally poor at including current peer-reviewed evidence, and the experts also said that test/retest reliability was also quite poor.

Experts in human-computer interaction gave decent marks (scores of about 7) for their ease of use. But clinicians, ethicists, and neuropsychologists were less forgiving on whether the tests could detect if the user was performing properly or merely playing around and how easy the tests were to use for those with lower levels of computer literacy. None rated the usability of any test much higher than 5.

None of the tests rated better than poor for meeting various ethical standards, such as providing upfront information about the nature and scope of the test, informed consent, and privacy discussions. The panelists also gave poor ratings to the tests’ wording of outcomes, the clarity of their interpretation, and their provision of appropriate advice for follow-up. Conflict-of-interest disclosures were not present, either.

"Is self-diagnosis appropriate at all? I think that professional consensus is that it’s not, so we have to wonder how we handle these tests being online and people doing them," Dr. Robillard said.

Her next step will be to explore in depth the potential harms or benefits of online dementia screening and to evaluate how such testing may affect clinician/patient relationships.

The study was supported by the Canadian Institute for Health Research and the Canadian Dementia Knowledge Translation Network. Dr. Robillard disclosed that two of her coauthors are involved in developing a computerized testing tool for mild cognitive impairment and dementia for use in specialty-care settings. Dr. Robillard reported having no personal disclosures.