Skip to main content
Download PDF

Use of the Clock Drawing Test and the Rey–Osterrieth Complex Figure Test-copy with convolutional neural networks to predict cognitive impairment

Alzheimer's Research & Therapy volume 13, Article number: 85 (2021) Cite this article

Abstract

Background

The Clock Drawing Test (CDT) and Rey–Osterrieth Complex Figure Test (RCFT) are widely used as a part of neuropsychological test batteries to assess cognitive function. Our objective was to confirm the prediction accuracies of the RCFT-copy and CDT for cognitive impairment (CI) using convolutional neural network algorithms as a screening tool.

Methods

The CDT and RCFT-copy data were obtained from patients aged 60–80 years who had more than 6 years of education. In total, 747 CDT and 980 RCFT-copy figures were utilized. Convolutional neural network algorithms using TensorFlow (ver. 2.3.0) on the Colab cloud platform (www.colab.research.google.com) were used for preprocessing and modeling. We measured the prediction accuracy of each drawing test 10 times using this dataset with the following classes: normal cognition (NC) vs. mildly impaired cognition (MI), NC vs. severely impaired cognition (SI), and NC vs. CI (MI + SI).

Results

The accuracy of the CDT was better for differentiating MI (CDT, 78.04 ± 2.75; RCFT-copy, not being trained) and SI from NC (CDT, 91.45 ± 0.83; RCFT-copy, 90.27 ± 1.52); however, the RCFT-copy was better at predicting CI (CDT, 77.37 ± 1.77; RCFT, 83.52 ± 1.41). The accuracy for a 3-way classification (NC vs. MI vs. SI) was approximately 71% for both tests; no significant difference was found between them.

Conclusions

The two drawing tests showed good performance for predicting severe impairment of cognition; however, a drawing test alone is not enough to predict overall CI. There are some limitations to our study: the sample size was small, all the participants did not perform both the CDT and RCFT-copy, and only the copy condition of the RCFT was used. Algorithms involving memory performance and longitudinal changes are worth future exploration. These results may contribute to improved home-based healthcare delivery.

Background

There is a growing interest in the use of artificial intelligence in clinical practice [1,2,3]. Efforts are underway for the prediction and diagnosis of prodromal or early-stage dementia [4,5,6,7] at home and in clinical settings [8, 9].

Performing a cognitive assessment is essential for establishing an objective diagnosis in patients with cognitive complaints [10]. Most of the currently used screening tools have been constructed based on neuropsychological tests. The Rey–Osterrieth Complex Figure Test (RCFT) is widely used by neuropsychologists to assess cognitive function. The test was first developed by Rey in 1941 [11] and has proved to be a useful tool for analyzing visuospatial construction, perceptual organization, and visual memory in clinical evaluations and research studies [12]. Patients with parieto-occipital lesions, especially on the right side, have difficulties in spatial organization while drawing, probably because of visual disorientation [13]. Patients with frontal lobe damage show impairment in programming abilities with respect to figure reproduction [14, 15]. Patients with early-stage Alzheimer’s disease (AD) perform poorly on this test [16]. Seo et al. showed that that the copy condition of the test was associated with spatial organization and planning, and it significantly predicted the conversion to pre-MCI or MCI [17]. The salience of visuospatial and organizational skills as evaluated by the copy condition of the RCFT differs according to the level of intelligence [18]. To obtain a more quantitative value for the accuracy of a participant’s drawing, many researchers use the RCFT based on the Osterrieth scoring criteria to diagnose cognitive impairment (CI) [19].

The Clock Drawing Test (CDT) is also widely used as a screening test for patients with dementia because it is easy to use and reflects a variety of cognitive functions, including visuospatial function, frontal lobe execution, and memory (of clock-related concepts). The CDT requires a participant to draw the hour and minute hands of the clock to show the time “11:10”. In patients with frontal lobe dysfunction, abstract thinking is compromised, which makes them prone to stimulus-bound errors wherein they process information at a more perceptual level than at a semantic level. Thus, they have difficulty recording “10” as “2”, and since “10” is adjacent to “11”, their attention is pulled toward the “10”, and they set the minute hand to “10” instead of “2” [20, 21]. Studies related to dementia have reported that the CDT is useful in the screening of cognitive impairment [22, 23] and that it can be used for screening MCI [11]. The CDT has a variety of scoring systems [24]. Among them, the Consortium to Establish a Registry for Alzheimer’s Disease CDT [25] is known to be the simplest method with a high diagnostic efficiency [26].

Detecting the severity of dementia is important for clinical and research purposes, and the Clinical Dementia Rating Scale (CDR) is one of the most commonly used tools for this assessment. The CDR comprises the global and sum of boxes (SOB) scores. The CDR-SOB score is considered a more detailed quantitative index than the global score and provides more information regarding patients with mild dementia. Previous studies have shown that the CDR-SOB scores may have the potential for discriminating between patients with MCI and those with very early stage AD dementia who are assigned a global CDR score of 0.5. Patients with MCI were assigned a CDR-SOB score of 1.8 ± 0.8, and those with very mild AD were assigned a CDR-SOB score of 3.0 ± 0.8 [27]. O’Bryant et al. classified the severity of dementia (normal to severe) based on SOB scores (0–18). In their system, a CDR-SOB score of 0 indicated normal cognition, 0.5–2.5 indicated suspicious damage, and 3.0–4.0, indicated very mild dementia [28]. Our database did not contain clinical information such as that regarding MCI or dementia, which is why the participants in this study were classified as having normal, mild, and severe CI based on the CDR-SOB score. However, unlike O’Bryant et al., we arbitrarily classified the degree of cognitive impairment: normal cognition (NC), 0–1.5; mild impairment of cognition (MI), 2.0–3.5; and severe impairment of cognition (SI), 4-.

Several studies have demonstrated that a digital CDT of a limited number of participants was able to differentiate patients with AD and other dementia syndromes from healthy controls using machine learning [29, 30]. However, digital CDT requires special equipment, and in deep learning, a greater amount of data with a good quality yields better result. Therefore, we predicted CI with deep learning based on a greater amount of drawing test data than that analyzed in previous studies. We investigated whether the CDT and RCFT can be used as screening tests to predict CI using convolutional neural network (CNN) algorithms. We also investigated whether the CDT, which measures various cognitive functions, was better than the RCFT in predicting CI. Our objective was to evaluate the prediction accuracies of these two tests for CI and compare them.

Methods

Dataset

Anonymous neuropsychological data from Jan. 2018 to Sep. 2020 at the Memory Clinics at Seoul National University Bundang Hospital and Chung-Ang University Hospital were retrospectively collected. The RCFT-copy and CDT figures that were drawn by patients aged 60–80 years with more than 6 years of education were selected. There were a total of 747 CDT and 980 RCFT-copy figures.

The original RCFT [11] and CDT were conducted by trained psychologists in the neurology outpatient testing room. The participants were given an A4 size paper and a pencil and instructed to copy the “Rey complex figure” and/or draw a “clock” indicating the time “11:10.” During the CDT, the following instructions were given: “You have to draw a clock. Draw a circle first and write all the numbers in it.” After the patients wrote the numbers, they were instructed as follows: “Now draw hands on the clock to indicate the time 11:10.” Test participants were clinically classified by dementia-specialized clinicians based on the CDR-SOB score into the following groups: normal cognition (NC), 0–1.5; mild impairment of cognition (MI), 2.0–3.5; and severe impairment of cognition (SI), 4- [28].

Model training and statistical analyses

The datasets of the CDT and RCFT-copy figures were organized into four classes: NC vs. MI, NC vs. SI, NC vs. CI (MI+SI), and NC vs. MI vs. SI. The datasets were prepared for three 2-way evaluations and one 3-way evaluation with respect to each CDT and RCFT-copy figure. The 2-way classifications for differentiating MI, SI, or CI from NC were performed in the CDT and RCFT datasets. The 3-way classification differentiated NC, MI, and SI in both the datasets.

All the algorithms were performed on the Colab cloud platform (www.colab.research.google.com). To model each algorithm, the dataset was subjected to the following preprocessing steps. As the dataset was relatively small number for machine learning, we augmented the image data. We made a replica image with a 10% height reduction and another one with a 10% width reduction, compared to each original drawing. All the images including the original drawing and two replicas were placed in a 600-dpi template.

TensorFlow (ver. 2.3.0) on Colab, which is a commonly used open-source, Python-based software library for machine learning developed by Google was used for preprocessing and modeling [31]. As an example, the code that predicted CI in the RCFT-copy dataset is given in the Supplementary Table. We imported the data in the “.png” format and used the “validation_split” function from “tf.keras.preprocessing.image_dataset_from_directory” to randomly split the data into training and test datasets. The training data size was 70%, which indicated the percentage of the data to be withheld for training; the validation dataset was thus composed of the remaining 30% of the data. The features were normalized with “tensorflow.keras.layers.experimental.preprocession.Rescaling(1./255).” We implemented data augmentation with “RandomZoom” and “RandomRotation” using “experimental Keras Preprocessing Layers.” This artificial neural network consists of five convolutional and maxpooling layers, and a dropout layer was inserted before connecting it to a fully connected neural network. The dropout rate was 0.2–0.3; therefore, 2–3 out of 10 weights were connected to the next layer to prevent overfitting. The cost was calculated using “Sparse_Categorical_Crossentropy” and minimized by means of the “adam” optimizer method. Model training was performed with a batch size of 20 and 40–72 epochs, depending on the dataset. During model training, optimal dropout rates and epochs were found and adjusted. After the dropout rate and epochs were defined, we obtained the average prediction accuracy based on 10 trials.

Results

Demographic and clinical characteristics including age, education level, and Mini-Mental Status Examination scores of the CDT and RCFT-copy datasets are given in Table 1. No differences in age and education levels were found among the NC, MI, and SI groups in the CDT and RCFT-copy datasets (p > 0.05).

Table 1 Demographic and clinical characteristics according to cognition status based on the CDT and RCFT-copy
Full size table

The CDT was more accurate in differentiating MI (CDT, 78.04 ± 2.75%; RCFT, not being trained) and SI from NC (CDT, 91.45 ± 0.83%; RCFT, 90.27 ± 1.52%) (Table 2); however, the RCFT-copy was better at predicting CI (CDT, 77.37 ± 1.77%; RCFT, 83.52 ± 1.41%). The accuracy of the 3-way classification (NC vs. MI vs. SI) was approximately 71% (Fig. 1).

Table 2 Mean accuracies of the CDT and RCFT-copy for the prediction of cognitive impairment
Full size table
Fig. 1
figure 1

Accuracies of algorithms for the prediction of cognitive impairment in the validation datasets of CDT and RCFT-copy figures. CDT, clock drawing test; RCFT-copy, Rey–Osterrieth Complex Figure Test-copy

Full size image

Discussion

The algorithm for predicting CI was more accurate in the RCFT-copy dataset than in the CDT dataset, but the algorithm for predicting MI in the RCFT-copy dataset had not being trained. However, at other levels of cognitive impairment, namely the prediction of SI in a 2-way classification and MI and SI in a 3-way classification, the two tests were nearly equal in their predictive accuracy in both the CDT and RCFT-copy datasets. We had expected the CDT to be superior to the RCFT-copy in predicting CI; this is because the RCFT-copy simply evaluates visual constructional function based on the copying of the figure, whereas the CDT not only evaluates visual constructional function and clock semantics but also the inhibitory function [20]. The CDT was thought to be more advantageous for evaluating various aspects of cognitive function. However, there was no difference between the two tests in distinguishing MI and SI from NC in the 3-way classification. Rather, the RCFT was better at distinguishing CI (including MI and SI) from NC in the 2-way classification. A voxel-based morphometric study that evaluated the relationship between the RCFT and brain volume showed that the RCFT score and the right caudate nucleus volume were positively correlated [32]. Therefore, in addition to visuospatial function evaluation, the RCFT may be used to evaluate frontal executive function. Another study found that poor copy scores in the RCFT were associated with greater beta amyloid burden in the frontal area on C-Pittsburgh B positron emission tomography/computed tomography and F-FC119S positron emission tomography/computed tomography [33]. These studies have shown that the RCFT is associated with cognitive functions other than the visual constructional function.

In our study, the prediction of SI in the validation dataset had an accuracy greater than 90%; however, the accuracy of differentiating between MI and NC was less than 80%. A slight cognitive decline was difficult to detect using the CDT. Moreover, the algorithm predicting MI in the RCFT-copy dataset was not well-trained. For differentiating MI from NC in a 2-way classification of the RCFT-copy dataset, the algorithms’ accuracy was only approximately ~ 55% and too variable; we did not consider this to be meaningful training, because a probability of approximately 50% exists even at random. However, it was better at predicting CI than the CDT algorithm was. Since prediction with RCFT-copy using the MI dataset did not show a good accuracy, it can be expected that the prediction of CI in a CI dataset that includes this MI dataset will not be accurate. Even if the MI prediction did not have a good accuracy, the algorithm using the CI dataset would produce different features with different weights, leading to better results than expected. However, we were unable to figure out what features the machine extracts and how much weight it gives to them.

There is a limitation in directly comparing the RCFT-copy and CDT machine learning algorithms as not all the participants performed both tests; only 91.7% (685/747) of the participants in the CDT dataset and 94.2% (923/980) of those in the RFCT-copy dataset had performed both tests. Overall, the two algorithms that predict CI seemed to have worked well. However, as the RCFT-copy algorithm could not be trained to select a patient with MI, the CDT algorithm seemed relatively advantageous.

Age and education have a strong effect on the performance in these tests [34, 35]. Most of the available norms provide either percentile scores/means and standard deviations for age-defined classes. However, in this study, age and educational levels were not included in the algorithm. We selected participants according to age and education levels; their age ranged from 60 to 80 years, as changes in cognitive function are expected in this age range, and they had more than 6 years of formal education (to minimize the impact of a low level of education on the performance in the drawing tests). If more substantial data could have been obtained, it would have been possible to predict CI based on the variables of age and education level.

We found that the machine learning algorithms based on the RCFT-copy and CDT datasets worked well in terms of predicting CI. Although the two drawing tests alone cannot sufficiently predict CI cross-sectionally, detecting changes in cognition using a longitudinal dataset is worth future exploration. It should be noted that the drawing test alone does not substitute for formal neuropsychological tests to predict overall CI. This study suggests the potential for home-based care services using drawing test algorithms to monitor or screen for CIs.

Limitations

There are several limitations to our study. The small sample size may limit the generalizability of our findings. All the participants did not perform both the CDT and RCFT-copy, which can limit the direct comparison of the two tests. In this study, only the copy condition of the RCFT was used. The RCFT consists of the copy and visual memory recall (immediate and delayed recall) conditions; memory performance is important for the screening of CI. Therefore, including the delayed recall condition of the RCFT in future studies may help in better prediction of CI. The memory recall condition of the RCFT is thought to require a more complex machine learning model; this approach will be attempted in our future research.

Conclusions

The CDT and RCFT-copy showed good performance for predicting SI; however, drawing tests alone are not enough to predict overall CI. Results from drawing tests and CNN algorithms can help improve home-based healthcare delivery. Algorithms involving memory performance and longitudinal changes are worth exploring in future studies.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

AD:

Alzheimer’s disease

CDT:

Clock drawing test

RCFT:

Rey–Osterrieth Complex Figure Test

RCFT-copy:

Rey–Osterrieth Complex Figure Test-copy

CNN:

Convolutional neural network

MCI:

Mild cognitive impairment

NC:

Normal cognition

MI:

Mild impairment of cognition

SI:

Severe impairment of cognition

CI:

Cognitive impairment

CDR:

Clinical Dementia Rating Scale

SOB:

Sum of Boxes

References

  1. Benjamens S, Dhunnoo P, Mesko B. The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database. NPJ Digit Med. 2020;3(1):118. https://doi.org/10.1038/s41746-020-00324-0.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Ramkumar PN, Kunze KN, Haeberle HS, Karnuta JM, Luu BC, Nwachukwu BU, et al. Clinical and research medical applications of artificial intelligence. Arthroscopy. 2020:S0749-8063(20)30687-3. https://doi.org/10.1016/j.arthro.2020.08.009.

  3. Albahri OS, Zaidan AA, Albahri AS, Zaidan BB, Abdulkareem KH, Al-Qaysi ZT, et al. Systematic review of artificial intelligence techniques in the detection and classification of COVID-19 medical images in terms of evaluation and benchmarking: taxonomy analysis, challenges, future solutions and methodological aspects. J Infect Public Health. 2020;13(10):1381–96. https://doi.org/10.1016/j.jiph.2020.06.028.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Aschwanden D, Aichele S, Ghisletta P, Terracciano A, Kliegel M, Sutin AR, Brown J, Allemand M. Predicting cognitive impairment and dementia: a machine learning approach. J Alzheimers Dis. 2020;75(3):717–28. https://doi.org/10.3233/JAD-190967.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Chen R, Herskovits EH. Machine-learning techniques for building a diagnostic model for very mild dementia. Neuroimage. 2010;52(1):234–44. https://doi.org/10.1016/j.neuroimage.2010.03.084.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Dallora AL, Eivazzadeh S, Mendes E, Berglund J, Anderberg P. Machine learning and microsimulation techniques on the prognosis of dementia: a systematic literature review. PLoS One. 2017;12(6):e0179804. https://doi.org/10.1371/journal.pone.0179804.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Nori VS, Hane CA, Crown WH, Au R, Burke WJ, Sanghavi DM, Bleicher P. Machine learning models to predict onset of dementia: a label learning approach. Alzheimers Dement (N Y). 2019;5(1):918–25. https://doi.org/10.1016/j.trci.2019.10.006.

    Article  Google Scholar 

  8. Ahamed F, Shahrestani S, Cheung H. Internet of things and machine learning for healthy ageing: identifying the early signs of dementia. Sensors (Basel). 2020;20(21):6031.

  9. Enshaeifar S, Zoha A, Markides A, Skillman S, Acton ST, Elsaleh T, Hassanpour M, Ahrabian A, Kenny M, Klein S, Rostill H, Nilforooshan R, Barnaghi P. Health management and pattern analysis of daily living activities of people with dementia using in-home sensors and machine learning techniques. PLoS One. 2018;13(5):e0195605. https://doi.org/10.1371/journal.pone.0195605.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Ahn HJ, Chin J, Park A, Lee BH, Suh MK, Seo SW, Na DL. Seoul Neuropsychological Screening Battery-dementia version (SNSB-D): a useful tool for assessing and monitoring cognitive impairments in dementia patients. J Korean Med Sci. 2010;25(7):1071–6. https://doi.org/10.3346/jkms.2010.25.7.1071.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Rey A. L'examen psychologique dans les cas d'encéphalopathie traumatique. (Les problems.). [The psychological examination in cases of traumatic encepholopathy. Problems.]. Archives de Psychologie. 1941;28:215–85.

    Google Scholar 

  12. Cherrier MM, Mendez MF, Dave M, Perryman KM. Performance on the Rey-Osterrieth complex figure test in Alzheimer disease and vascular dementia. Neuropsychiatry Neuropsychol Behav Neurol. 1999;12(2):95–101.

    CAS  PubMed  Google Scholar 

  13. Binder LM. Constructional strategies on complex figure drawings after unilateral brain damage. J Clin Neuropsychol. 1982;4(1):51–8. https://doi.org/10.1080/01688638208401116.

    Article  CAS  PubMed  Google Scholar 

  14. Pillon B. Troubles visuo-constructifs et méthodes de compensation: Résultats de 85 patients atteints de lésions cérébrales. Neuropsychologia. 1981;19(3):375–83. https://doi.org/10.1016/0028-3932(81)90067-1.

    Article  CAS  PubMed  Google Scholar 

  15. Lezak MD. Neuropsychological assessment. 3rd. ed. New York: Oxford University Press; 1995.

  16. Cummings JL, Benson DF. Dementia of the Alzheimer type. An inventory of diagnostic clinical features. J Am Geriatr Soc. 1986;34(1):12–9. https://doi.org/10.1111/j.1532-5415.1986.tb06334.x.

    Article  CAS  PubMed  Google Scholar 

  17. Seo EH, Kim H, Choi KY, Lee KH, Choo IH. Pre-mild cognitive impairment: can visual memory predict who rapidly convert to mild cognitive impairment? Psychiatry Investig. 2018;15(9):869–75. https://doi.org/10.30773/pi.2018.07.29.1.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Fujii DE, Lloyd HA, Miyamoto K. The salience of visuospatial and organizational skills in reproducing the Rey-Osterreith Complex Figure in subjects with high and low IQs. Clin Neuropsychol. 2000;14(4):551–4. https://doi.org/10.1076/clin.14.4.551.7206.

    Article  CAS  PubMed  Google Scholar 

  19. Rey A, Ferrara-Mori G. appliquée Cdp. Reattivo della figura complessa: Manuale: Organizzazi*oni speciali; 1967.

    Google Scholar 

  20. Shulman KI. Clock-drawing: is it the ideal cognitive screening test? Int J Geriatr Psychiatry. 2000;15(6):548–61. https://doi.org/10.1002/1099-1166(200006)15:6<548::AID-GPS242>3.0.CO;2-U.

    Article  CAS  PubMed  Google Scholar 

  21. Freedman M, Leach L, Kaplan E, Winocur G, Shulman KI, Delis DC. Clock drawing: A neuropsychological analysis. New York: Oxford University Press; 1994. p. vi. 182-vi

    Google Scholar 

  22. Kim S, Jahng S, Yu K-H, Lee B-C, Kang Y. Usefulness of the clock drawing test as a cognitive screening instrument for mild cognitive impairment and mild dementia: an evaluation using three scoring systems. Dement Neurocogn Disord. 2018;17(3):100–9. https://doi.org/10.12779/dnd.2018.17.3.100.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Allone C, Lo Buono V, Corallo F, Bonanno L, Palmeri R, Di Lorenzo G, et al. Cognitive impairment in Parkinson’s disease, Alzheimer’s dementia, and vascular dementia: the role of the clock-drawing test. Psychogeriatrics. 2018;18(2):123–31. https://doi.org/10.1111/psyg.12294.

    Article  PubMed  Google Scholar 

  24. Lee KS, Kim EA, Hong CH, Lee DW, Oh BH, Cheong HK. Clock drawing test in mild cognitive impairment: quantitative analysis of four scoring methods and qualitative analysis. Dement Geriatr Cogn Disord. 2008;26(6):483–9. https://doi.org/10.1159/000167879.

    Article  PubMed  Google Scholar 

  25. Muller S, Herde L, Preische O, Zeller A, Heymann P, Robens S, et al. Diagnostic value of digital clock drawing test in comparison with CERAD neuropsychological battery total score for discrimination of patients in the early course of Alzheimer’s disease from healthy individuals. Sci Rep. 2019;9(1):3543. https://doi.org/10.1038/s41598-019-40010-0.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Lee KS, Cheong H-K, Oh BH, Hong CH, Lee D-W. Reliability and validity of four scoring methods of clock drawing test for screening dementia and mild cognitive impairment. Dement Neurocogn Disord. 2009;53.

  27. Grundman M, Petersen RC, Ferris SH, Thomas RG, Aisen PS, Bennett DA, Foster NL, Jack CR Jr, Galasko DR, Doody R, Kaye J, Sano M, Mohs R, Gauthier S, Kim HT, Jin S, Schultz AN, Schafer K, Mulnard R, van Dyck C, Mintzer J, Zamrini EY, Cahn-Weiner D, Thal LJ, Alzheimer's Disease Cooperative Study. Mild cognitive impairment can be distinguished from Alzheimer disease and normal aging for clinical trials. Arch Neurol. 2004;61(1):59–66. https://doi.org/10.1001/archneur.61.1.59.

    Article  PubMed  Google Scholar 

  28. O’Bryant SE, Waring SC, Cullum CM, Hall J, Lacritz L, Massman PJ, et al. Staging dementia using clinical dementia rating scale sum of boxes scores: a Texas Alzheimer’s research consortium study. Arch Neurol. 2008;65(8):1091–5. https://doi.org/10.1001/archneur.65.8.1091.

  29. Davis R, Libon DJ, Au R, Pitman D, Penney DL. THink: inferring cognitive status from subtle behaviors. Proc Conf AAAI Artif Intell. 2014;2014:2898–905.

    PubMed  PubMed Central  Google Scholar 

  30. Souillard-Mandar W, Davis R, Rudin C, Au R, Libon DJ, Swenson R, Price CC, Lamar M, Penney DL. Learning classification models of cognitive conditions from subtle behaviors in the digital clock drawing test. Mach Learn. 2016;102(3):393–441. https://doi.org/10.1007/s10994-015-5529-5.

  31. Rampasek L, Goldenberg A. TensorFlow: biology’s gateway to deep learning? Cell Syst. 2016;2(1):12–4. https://doi.org/10.1016/j.cels.2016.01.009.

    Article  CAS  PubMed  Google Scholar 

  32. Lee S-Y, Yoon S-Y, Kim M-J, Rhee HY, Ryu C-W, Jahng G-H. Investigation of the correlation between Seoul neuropsychological screening battery scores and the gray matter volume after correction of covariates of the age, gender, and genotypes in patients with AD and MCI. J Korean Soc Magn Reson Med. 2013;17(4):294–307. https://doi.org/10.13104/jksmrm.2013.17.4.294.

    Article  Google Scholar 

  33. Park SY, Byun BH, Kim BI, Lim SM, Ko IO, Lee KC, Kim KM, Kim YK, Lee JY, Bu SH, Kim JH, Chi DY, Ha JH. The correlation of neuropsychological evaluation with 11C-PiB and 18F-FC119S amyloid PET in mild cognitive impairment and Alzheimer disease. Medicine (Baltimore). 2020;99(16):e19620. https://doi.org/10.1097/MD.0000000000019620.

    Article  CAS  Google Scholar 

  34. Rosselli M, Ardila A. Effects of age, education, and gender on the Rey-osterrieth complex figure. Clin Neuropsychol. 1991;5(4):370–6. https://doi.org/10.1080/13854049108404104.

    Article  Google Scholar 

  35. Boone KB, Lesser IM, Hill-gutierrez E, Berman NG, D'Elia LF. Rey-osterrieth complex figure performance in healthy, older adults: relationship to age, education, sex, and IQ. Clin Neuropsychol. 1993;7(1):22–8. https://doi.org/10.1080/13854049308401884.

    Article  Google Scholar 

Download references

Acknowledgements

The authors wish to thank Jaeyoung Youn (Department of Korean Medicine, Sangji University) and Sujin An (Department of Psychology, Chung-Ang University), who contributed to the preprocessing of the data presented here.

Funding

The costs for manuscript publication, design of the study, data management, and writing of the manuscript were supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2017S1A6A3A01078538).

Author information

Authors and Affiliations

  1. Department of Neurology, College of Medicine, Chung-Ang University, Seoul, Republic of Korea

    Young Chul Youn, Suk-Won Ahn, Hae-Won Shin & Kwang-Yeol Park

  2. Department of Medical Informatics, Chung-Ang University College of Medicine, Seoul, Republic of Korea

    Young Chul Youn, Kwang-Yeol Park & Sang Yun Kim

  3. Department of Neurology, Seoul National University College of Medicine & Seoul National University Bundang Hospital, Seoul, Republic of Korea

    Jung-Min Pyun, Nayoung Ryu, Min Jae Baek & Young Ho Park

  4. Department of Neurology, Kangwon National University Hospital, Kangwon National University College of Medicine, Chuncheon, Republic of Korea

    Jae-Won Jang

  5. Department of Neurology, Seoul National University College of Medicine & Neurocognitive Behavior Center, Seoul National University Bundang Hospital, 300 Gumi-dong, Bundang-gu, Seongnam-si, Gyeonggi-do, 463-707, Republic of Korea

    Sang Yun Kim

Authors
  1. Young Chul Youn
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jung-Min Pyun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nayoung Ryu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Min Jae Baek
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jae-Won Jang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Young Ho Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Suk-Won Ahn
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Hae-Won Shin
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Kwang-Yeol Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Sang Yun Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

YCY planned, organized, and designed all the experiments and results, collected the data, and wrote the manuscript. JP, NR, and MJB collected the data and reviewed and revised the manuscript. JJ, YHP, SA, HS, and KP reviewed and revised the manuscript. SYK reviewed all the experiments and results, collected clinical data, and supervised the study. All the authors have read and approved the final version of the manuscript.

Corresponding author

Correspondence to Sang Yun Kim.

Ethics declarations

Ethics approval and consent to participate

Anonymous neuropsychological test datasets were used retrospectively from the Seoul National University Bundang Hospital and Chung-Ang University Hospital. The study was approved by the institutional review boards of the Seoul National University Bundang Hospital and Chung-Ang University Hospital (approval numbers, 1792-012-300). The need for written informed consent was waived due to the retrospective nature of the study.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Supplementary Table

. The codes for prediction of the cognitive impairment with the two drawing test.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Youn, Y.C., Pyun, JM., Ryu, N. et al. Use of the Clock Drawing Test and the Rey–Osterrieth Complex Figure Test-copy with convolutional neural networks to predict cognitive impairment. Alz Res Therapy 13, 85 (2021). https://doi.org/10.1186/s13195-021-00821-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13195-021-00821-8

Keywords

Alzheimer's Research & Therapy

ISSN: 1758-9193