Comparing the efficacy of virtually administered and in person cognitive screening tools during the COVID-19 pandemic

Author: Shreyas Bhalle
Program: Biomedical Sciences (MS)
Mentor(s): Kanishk Sharma, MD
Poster #: 100
Session/Time: B/3:40 p.m.

Abstract

Introduction:

Because of the COVID-19 pandemic, there were many restrictions placed on in-person elective testing for cognitive assessments. Testing was adapted to be virtually administered. Given the rapid onset of the pandemic, there was a significant increase in telehealth services across healthcare settings. There is a growing practice of delivering clinical neuropsychological services using teleconferencing video technology and is shown to be well accepted by patients. Although virtual administration of testing has improved the feasibility of delivering these tests, there has been expressed concern regarding best practice guidelines for implementation of virtual neuropsychological testing (NPT) in clinical practice. Administration of several different neuropsychological testing modalities will be analyzed to better understand test accuracy using remote cognitive testing. More than 600 patients were tested virtually from March of 2020 to August of 2021 as a direct result of the COVID-19 pandemic and its social distancing and self-isolation protocols.

Methods:

The goal of this project is to compare the diagnostic accuracy of virtual administration of the eleven components of the NPT to in-person administration of NPT in the diagnosis of cognitive impairment. It is also the objective of the study to identify differences in results specific to each component of testing as well as determining the influence of social demographics on virtual versus in-person assessment. Several data points will be collected including scores of the components of the cognitive assessment, social demographics, as well as degree of cognitive impairment as determined by fellowship trained Geriatrician with expertise in neurocognitive disorders: No cognitive impairment, MCI, dementia, Alzheimer's disease, other. Data collection for the 600 patients will be conducted from patients who were seen between March 2020 and August 2021. This virtual NPT was administered by training staff at the memory clinic. Patients received this testing while at home, at a home of a close friend of family member, or at a nursing facility. Before testing was started, participants were asked to eliminate any distractions and following the testing, the results were reviewed by the clinician. My role in this project is to conduct retrospective medical record review for those patients who were referred for evaluation at the Glennan Center Memory Consultation Clinic. Those with a complete NPT will be identified from the clinic through AllScripts and from the registry through RedCap. Participants will be identified and those with at least one virtual NPT data will be assigned to a group titled "Virtual NPT" while those with only in-person NPT data will be assigned to "in-person NPT" group. There are currently an estimated 600 patients predicated to have both virtual and in-person NPT data available. Those with either virtual or in-person NPT data from 06/01/2017 to 12/21/2022 will be included in the study. After data collection, data will be extracted and stored in RedCap as deidentified data which will then be used to compare in-person and virtual psychometric assessment. Statistical analysis for this retrospective, cross-sectional study will be conducted using many different modalities. For the demographic compositions of the two samples, a t-test for the continuous variables or a chi-square test for categorical variables will be implemented. Given the modelling or multiple outcomes, a multiple-comparison correction will also be implemented by adjusting the alpha-level in determining statistical significance. A multiple regression model will be used to investigate the relation between test type and test score. Data will be reported in aggregate trends.

Results:

You will notice that the methods defined in the previous section is written for what I will be doing rather than what I have done so far. This is because of the limitations associated with the Institutional Review Board (IRB) as well as attaining access for AllScripts. While I had submitted all the necessary paperwork in late May/early June, I only received IRB approval for this study on 07/26/2023 allowing me to start work on this project a month ago. However, despite IRB approval being so late, I am still currently waiting on approval for AllScripts EHR access to begin my retrospective data collection. Once this is completed, I will be able to begin data collection and follow the aforementioned protocol for in-person versus virtual NPT analysis.

Conclusion:

There are currently no conclusions from the study given the ongoing nature of approval for AllScripts access as previously mentioned. However, as I will be continuing this research into the academic year, I hope to gain approval, complete data collection, and analyze the results within the next few months with an ideal completion date of 11/2023.