Publications

International journal articles

C. de Vente, K.A. Vermeer, N. Jaccard, H. Wang, H. Sun, F. Khader, D. Truhn, T. Aimyshev, Y. Zhanibekuly, T. Le, A. Galdran, M.Á. González Ballester, G. Carneiro, D.R. G, H.P. S, D. Puthussery, H. Liu, Z. Yang, S. Kondo, S. Kasai, E. Wang, A. Durvasula, J. Heras, M.Á. Zapata, T. Araújo, G. Aresta, H. Bogunović, M. Arikan, Y.C. Lee, H.B. Cho, Y.H. Choi, A. Qayyum, I. Razzak, B. van Ginneken, H.G. Lemij and C.I. Sánchez. "AIROGS: Artificial Intelligence for RObust Glaucoma Screening Challenge", IEEE transactions on medical imaging, 2023;43(1):542-557.

Cited by 20 DOI PMID

H.G. Lemij, C. de Vente, C.I. Sánchez and K.A. Vermeer. "Characteristics of a large, labeled dataset for the training of artificial intelligence for glaucoma screening with fundus photographs", Ophthalmology Science, 2023;100300.

C. de Vente, L.H. Boulogne, K. Vaidhya Venkadesh, C. Sital, N. Lessmann, C. Jacobs, C.I. Sánchez and B. van Ginneken. "Automated COVID-19 Grading with Convolutional Neural Networks in Computed Tomography Scans: A Systematic Comparison", IEEE Transactions on Artificial Intelligence, 2022;3(2):129-138.

Cited by 18 DOI

R. Schwartz, H. Khalid, S. Liakopoulos, Y. Ouyang, C. de Vente, C. González-Gonzalo, A.Y. Lee, R. Guymer, E.Y. Chew, C. Egan and others. "A Deep Learning Framework for the Detection and Quantification of Reticular Pseudodrusen and Drusen on Optical Coherence Tomography", Translational Vision Science & Technology, 2022;11(12):3-3.

N. Lessmann, C.I. Sánchez, L. Beenen, L.H. Boulogne, M. Brink, E. Calli, J. Charbonnier, T. Dofferhoff, W.M. van Everdingen, P.K. Gerke, B. Geurts, H.A. Gietema, M. Groeneveld, L. van Harten, N. Hendrix, W. Hendrix, H.J. Huisman, I. Isgum, C. Jacobs, R. Kluge, M. Kok, J. Krdzalic, B. Lassen-Schmidt, K. van Leeuwen, J. Meakin, M. Overkamp, T. van Rees Vellinga, E.M. van Rikxoort, R. Samperna, C. Schaefer-Prokop, S. Schalekamp, E.T. Scholten, C. Sital, L. Stöger, J. Teuwen, K. Vaidhya Venkadesh, C. de Vente, M. Vermaat, W. Xie, B. de Wilde, M. Prokop and B. van Ginneken. "Automated Assessment of COVID-19 Reporting and Data System and Chest CT Severity Scores in Patients Suspected of Having COVID-19 Using Artificial Intelligence", Radiology, 2021;298(1):E18-E28.

Cited by 160 DOI PMID

Z. Xiong, Q. Xia, Z. Hu, N. Huang, C. Bian, Y. Zheng, S. Vesal, N. Ravikumar, A. Maier, X. Yang, P. Heng, D. Ni, C. Li, Q. Tong, W. Si, E. Puybareau, Y. Khoudli, T. Graud, C. Chen, W. Bai, D. Rueckert, L. Xu, X. Zhuang, X. Luo, S. Jia, M. Sermesant, Y. Liu, K. Wang, D. Borra, A. Masci, C. Corsi, C. de Vente, M. Veta, R. Karim, C. Jayachandran Preetha, S. Engelhardt, M. Qiao, Y. Wang, Q. Tao, M. Nuñez-Garcia, O. Camara, N. Savioli, P. Lamata and J. Zhao. "A global benchmark of algorithms for segmenting the left atrium from late gadolinium-enhanced cardiac magnetic resonance imaging", Medical Image Analysis, 2021;67:101832.

Cited by 185 PMID

C. de Vente, P. Vos, M. Hosseinzadeh, J. Pluim and M. Veta. "Deep learning regression for prostate cancer detection and grading in bi-parametric MRI", IEEE Transactions on Biomedical Engineering, 2020;68(2):374-383.

Cited by 103 DOI PMID

Preprints

C. de Vente, B. van Ginneken, C.B. Hoyng, C.C. Klaver and C.I. Sánchez. "Uncertainty-Aware Multiple-Instance Learning for Reliable Classification: Application to Optical Coherence Tomography", arXiv:2302.03116, 2023.

Abstracts

C. de Vente, A. Tufail, S. Schmitz-Valckenberg, M. Saßmannshausen, C. Hoyng and C.I. Sánchez on behalf of the MACUSTAR consortium. "OCT Super-Resolution for Data Standardization using AI: A MACUSTAR report", in: Association for Research in Vision and Ophthalmology, 2023.

H.G. Lemij, C. de Vente, C.I. Sánchez, J. Cuadros, N. Jaccard and K. Vermeer. "Glaucomatous features in fundus photographs of eyes with 'Referable glaucoma' of a large population based labeled data set for training an Artificial Intelligence (AI) algorithm for glaucoma screening", in: Association for Research in Vision and Ophthalmology, 2022.

R. Schwartz, H. Khalid, S. Liakopoulos, Y. Ouyang, C. de Vente, C.G. Gonzalo, A.Y. Lee, C.A. Egan, C.I. Sánchez and A. Tufail. "A deep learning pipeline for the detection and quantification of drusen and reticular pseudodrusen on optical coherence tomography", in: Association for Research in Vision and Ophthalmology, 2022.

C. de Vente, K. Vermeer, N. Jaccard, H.G. Lemij and C.I. Sánchez. "AIROGS: Artificial Intelligence for RObust Glaucoma Screening Challenge", in: Imaging and Morphometry Association for Glaucoma in Europe, 2022.

C. de Vente, C. González-Gonzalo, E.F. Thee, M. van Grinsven, C.C. Klaver and C.I. Sánchez. "Making AI Transferable Across OCT Scanners from Different Vendors", in: Association for Research in Vision and Ophthalmology, 2021.

C. González-Gonzalo, E.F. Thee, B. Liefers, C. de Vente, C.C. Klaver and C.I. Sánchez. "Hierarchical curriculum learning for robust automated detection of low-prevalence retinal disease features: application to reticular pseudodrusen", in: Association for Research in Vision and Ophthalmology, 2021.

C. de Vente, M. van Grinsven, S. De Zanet, A. Mosinska, R. Sznitman, C. Klaver and C.I. Sánchez. "Estimating Uncertainty of Deep Neural Networks for Age-related Macular Degeneration Grading using Optical Coherence Tomography", in: Association for Research in Vision and Ophthalmology, 2020.

A. Ardu, B. Liefers, C. de Vente, C. González-Gonzalo, C. Klaver and C.I. Sánchez. "Artificial Intelligence for the Classification and Quantification of Reticular Pseudodrusen in Multimodal Retinal Images", in: European Society of Retina Specialists, 2020.

Papers in conference proceedings

C. de Vente, M. Veta, O. Razeghi, S. Niederer, J. Pluim, K. Rhode and R. Karim. "Convolutional neural networks for segmentation of the left atrium from gadolinium-enhancement MRI images", in: International Workshop on Statistical Atlases and Computational Models of the Heart, 2018, pages 348-356.

C. de Vente, K. Vermeer, N. Jaccard, H.G. Lemij and C.I. Sánchez. "AIROGS: Artificial Intelligence for RObust Glaucoma Screening Challenge", in: Imaging and Morphometry Association for Glaucoma in Europe, 2022.

@conference{Vent22,
author = {de Vente, Coen and Vermeer, Koen and Jaccard, Nicolas and Lemij, Hans G and S{\'a}nchez, Clara I},
booktitle ={Imaging and Morphometry Association for Glaucoma in Europe}
url = {https://drive.google.com/file/d/11DfeyNA8I4UVZGkhn_NVIOIGbcEesfCw/view?usp=sharing},
title = {AIROGS: Artificial Intelligence for RObust Glaucoma Screening Challenge},

Methods: 112,732 CFPs from 60,071 subjects from a population screening program for diabetic retinopathy, obtained from EyePACS, California, USA, were manually labeled by 20 carefully selected and continuously monitored ophthalmologists and optometrists. They each labeled a portion of the full set of images as “referable glaucoma” (RG), “no referable glaucoma” (NRG) or “ungradable” (U). Each CFP was graded by 2 randomly selected graders; if their labels matched, it was considered the final label. In case of disagreement, the CFP was graded by a glaucoma specialist; his label was the final label. We split the data into a development set of 101,442 CFPs and a test set of 11,290 CFPs. The challenge task was to classify CFPs as RG or NRG, while additionally providing a decision on whether images were ungradable. To encourage the development of methodologies with inherent robustness mechanisms, we only included CFPs labeled as U in the test set and not in the development set. Challenge participants submitted their solutions as Docker1 containers to our online evaluation platform2. We ran their submitted algorithms on our test set, which is not publicly available. Subsequently, we assessed glaucoma screening performance using the partial area under the receiver operator characteristic curve (pAUC) (90-100% specificity) and sensitivity at 95% specificity (S). To measure robustness, we calculated Cohen's kappa score (κU) and the area under the receiver operator characteristic curve (AUCU), using the decisions generated by the algorithms on image ungradability.

Results: The challenge is currently running and we are still accepting submissions at the time of writing. Up to now, 289 users have joined the challenge, 208 persons have requested access to the dataset, 26 teams have been formed on the challenge platform, and 13 submissions from 7 unique participants have been successfully submitted to the last preliminary test set, which contains 10% of the test data. The best pAUC, S, κU and AUCU on this preliminary test set were 89.1%, 83.8%, 44.5% and 91.5%, respectively. The means and standard deviations for these metrics over all submissions were 82.2% ± 9.6%, 72.0% ± 19.4%, 20.6% ± 14.9%, 76.8% ± 11.6%, respectively.

Conclusions: We present a challenge based on real-world data for glaucoma screening by CFP. The initial results are promising, as the performances are high and the preliminary sensitivity at 95% specificity exceeds our target of 80%. The final winners and their solutions will be presented at the 19th IMAGE meeting.
},
year = {2022},
}

C. de Vente, C. González-Gonzalo, E.F. Thee, M. van Grinsven, C.C. Klaver and C.I. Sánchez. "Making AI Transferable Across OCT Scanners from Different Vendors", in: Association for Research in Vision and Ophthalmology, 2021.

@conference{Vent21a,
  author = {de Vente, Coen and Gonz\'{a}lez-Gonzalo, Cristina and Thee, Eric F. and van Grinsven, Mark and Klaver, Caroline C.W. and S\'{a}nchez, Clara I.},
  booktitle ={Association for Research in Vision and Ophthalmology}
  url = {https://iovs.arvojournals.org/article.aspx?articleid=2775505},
  title = {Making AI Transferable Across OCT Scanners from Different Vendors},

  Methods: 2,598 and 680 Heidelberg Spectralis OCT scans from the European Genetic Database were used for development and testing, respectively. We tested transferability with 339 AMD-enriched Topcon OCTs from the Rotterdam Study. AMD severity classification was determined manually in accordance with the Cologne Image Reading Center and Laboratory and Rotterdam Classification, respectively. Classifications were harmonized for the evaluation of the DNNs. The proposed DNN considers each B-scan separately using a 2D ResNet-18, and internally combines the intermediate outputs related to each B-scan using a multiple instance learning approach. Even though the proposed DNN provides both B-scan level and OCT-volume level decisions, the architecture is trained end-to-end using only full volume gradings. This specific architecture makes our method robust to the variability of scanning protocols across vendors, as it is invariant to B-scan spacing. We compare this approach to a baseline that classifies the full OCT scan directly using a 3D ResNet-18.

  Results: The quadratic weighted kappa (QWK) for the baseline method dropped from 0.852 on the Heidelberg Spectralis dataset to 0.523 on the Topcon dataset. This QWK drop was smaller (p = 0.001) for our approach, which dropped from 0.849 to 0.717. The difference in area under the Receiver Operating Characteristic (AUC) drop was also smaller (p < 0.001) for our approach (0.969 to 0.906, -6.5%) than for the baseline method (0.971 to 0.806, -17.0%).

  Conclusions: We present a DNN for AMD classification on OCT scans that transfers well to scans from vendors that were not used for development. This alleviates the need for retraining on data from these scanner types, which is an expensive process in terms of data acquisition, model development, and human annotation time. Furthermore, this increases the applicability of AI for OCT classification in broader scopes than the settings in which they were developed.},
  year = {2021},
  scholar_id = {504386056410585983},
}

C. González-Gonzalo, E.F. Thee, B. Liefers, C. de Vente, C.C. Klaver and C.I. Sánchez. "Hierarchical curriculum learning for robust automated detection of low-prevalence retinal disease features: application to reticular pseudodrusen", in: Association for Research in Vision and Ophthalmology, 2021.

@conference{Gonz21,
  author = {Gonz\'{a}lez-Gonzalo, Cristina and Thee, Eric F. and Liefers, Bart and de Vente, Coen and Klaver, Caroline C.W. and S\'{a}nchez, Clara I.},
  booktitle ={Association for Research in Vision and Ophthalmology}
  url = {https://iovs.arvojournals.org/article.aspx?articleid=2773295},
  title = {Hierarchical curriculum learning for robust automated detection of low-prevalence retinal disease features: application to reticular pseudodrusen},

  Methods: Color fundus images (CFI) from the AREDS dataset were used for DNN development (106,994 CFI) and testing (27,066 CFI). An external test set (RS1-6) was generated with 2,790 CFI from the Rotterdam Study. In both datasets CFI were graded from generic to specific features. This allows to establish a hierarchy of binary classification tasks with decreasing prevalence: presence of AMD findings (AREDS prevalence: 88%; RS1-6: 77%), drusen (85%; 73%), large drusen (40%; 24%), RPD (1%; 4%). We created a hierarchical curriculum and developed a DNN (HC-DNN) that learned each task sequentially. We computed its performance for RPD detection in both test sets and compared it to a baseline DNN (B-DNN) that learned to detect RPD from scratch disregarding hierarchical information. We studied their robustness across datasets, while reducing the size of data available for development (same prevalences)

  Results: Area under the receiver operating characteristic curve (AUC) was used to measure RPD detection performance. When large development data were available, there was no significant difference between DNNs (100% data, HC-DNN: 0.96 (95% CI, 0.94-0.97) in AREDS, 0.82 (0.78-0.86) in RS1-6; B-DNN: 0.95 (0.94-0.96) in AREDS, 0.83 (0.79-0.87) in RS1-6). However, HC-DNN achieved better performance and robustness across datasets when development data were highly reduced (<50% data, p-values<0.05) (1% data, HC-DNN: 0.63 (0.60-0.66) in AREDS, 0.76 (0.72-0.80) in RS1-6; B-DNN: 0.53 (0.49-0.56) in AREDS, 0.48 (0.42-0.53) in RS1-6).

  Conclusions: Hierarchical curriculum learning allows for knowledge transfer from general, higher-prevalence features and becomes beneficial for the detection of low-prevalence retinal features, such as RPD, in scarce data settings. Moreover, exploiting hierarchical information improves DNN robustness across datasets.},
  year = {2021},
  scholar_id = {8274882984481634472},
}

C. de Vente, M. van Grinsven, S. De Zanet, A. Mosinska, R. Sznitman, C. Klaver and C.I. Sánchez. "Estimating Uncertainty of Deep Neural Networks for Age-related Macular Degeneration Grading using Optical Coherence Tomography", in: Association for Research in Vision and Ophthalmology, 2020.

@conference{Vent20,
author = {de Vente, Coen and van Grinsven, Mark and De Zanet, Sandro and Mosinska, Agata and Sznitman, Raphael and Klaver, Caroline and S\'{a}nchez, Clara I.},
booktitle ={Association for Research in Vision and Ophthalmology}
title = {Estimating Uncertainty of Deep Neural Networks for Age-related Macular Degeneration Grading using Optical Coherence Tomography},
url={https://iovs.arvojournals.org/article.aspx?articleid=2769262},

Methods: 1,264 OCT volumes from 633 patients from the European Genetic Database (EUGENDA) were graded as one of five stages of AMD (No AMD, Early AMD, Intermediate AMD, Advanced AMD: GA, and Advanced AMD: CNV). Ten different 3D DenseNet-121 models that take a full OCT volume as input were used to predict the corresponding AMD stage. These networks were all trained on the same dataset. However, each of these networks were initialized differently. The class with the maximum average softmax output of these models was used as the final prediction. The confidence measure was the normalized average softmax output for that class.

Results: The algorithm achieved an area under the Receiver Operating Characteristic of 0.9785 and a quadratic-weighted kappa score of 0.8935. The mean uncertainty, calculated as 1 - the mean confidence score, for incorrect predictions was 1.9 times as high as the mean uncertainty for correct predictions. When only using the probability output of a single network, this ratio was 1.4. Another measure for uncertainty estimation performance is the Expected Calibration Error (ECE), where a lower value is better. When comparing the method to the probability output of a single network, the ECE improved from 0.0971 to 0.0324. Figure 1 shows examples of both confident and unconfident predictions.

Conclusions: We present a method for improving uncertainty estimation for AMD grading in OCT, by combining the output of multiple individually trained CNNs. This increased reliability of system confidences can contribute to building trust in CNNs for retinal disease screening. Furthermore, this technique is a first step towards selective prediction in retinal disease screening, where only cases with high uncertainty predictions need to be referred for expert evaluation.},
year = {2020},
month = {6},
scholar_id = {13938730969248371423},
}

A. Ardu, B. Liefers, C. de Vente, C. González-Gonzalo, C. Klaver and C.I. Sánchez. "Artificial Intelligence for the Classification and Quantification of Reticular Pseudodrusen in Multimodal Retinal Images", in: European Society of Retina Specialists, 2020.

@Conference{Ardu20,
author = {Ardu, Alessandro and Liefers, Bart and de Vente, Coen and Gonz\'{a}lez-Gonzalo, Cristina and Klaver, Caroline and S\'{a}nchez, Clara I.},
booktitle ={European Society of Retina Specialists}
title = {Artificial Intelligence for the Classification and Quantification of Reticular Pseudodrusen in Multimodal Retinal Images},
Reticular pseudodrusen (RPD) are retinal lesions highly correlated with the risk of developing end-stage age-related macular degeneration (AMD) and, therefore, relevant biomarkers for understanding the progression of AMD. Due to the subtle features characterizing RPD, multiple imaging modalities are often necessary to confirm the presence and extension of RPD, considerably increasing the workload of the expert graders. We propose a deep neural network (DNN) architecture that classifies and quantifies RPD using multimodal retinal images.
Setting:
A cross-sectional study that compares the performance of three expert graders with a DNN trained for identifying and quantifying RPD. Conducted on retinal images drawn from the Rotterdam Study, a population-based cohort, in three modalities: color fundus photographs (CFP), fundus autofluorescence images (FAF) and near-infrared reflectance images (NIR).
Methods:
Multimodal images of 278 eyes of 230 patients were retrieved from the Rotterdam Study database. Of those, 72 eyes showed presence of RPD, 108 had soft distinct/indistinct drusen, and 98 had no signs of drusen as confirmed by the Rotterdam Study graders. Delineations of the areas affected with RPD were made in consensus by two human experts using CFP and NIR images simultaneously and were used as reference standard (RS) for RPD area quantification. The data was randomly divided, patient-wise, in training (243) and test (35) sets for model development and evaluation. A DNN was developed for RPD classification and quantification. The proposed DNN is based on an encoder-decoder architecture. The model jointly inputs a set of co-registered retinal image modalities (CFP, NIR, FAF) and outputs a heatmap image containing, per pixel, the likelihood of RPD presence. The 99th percentile of the values contained in this heatmap measures the likelihood of RPD presence. Three independent graders manually delineated RPD in all eyes of the test set based on the CFP and NIR and their performance was compared with the DNN in the tasks of RPD classification and quantification.
Results:
The proposed DNN obtained an area under the receiver operating characteristic curve (AUROC) with 95% confidence interval (CI) of 0.939[0.818-1.0], a sensitivity (SE) of 0.928 and specificity (SP) of 0.809 for the detection of RPD in multimodal imaging. For RPD quantification, the DNN achieved a mean Dice coefficient (DSC) of 0.632+-0.261 and an intra-class correlation (ICC) of 0.676[0.294-0.999]. Comparably, for RPD classification, grader 1 obtained SE/SP pairs of 1.0/0.785, grader 2 of 1.0/0.5 and grader 3 of 1.0/0.785. For RPD quantification, the graders obtained mean DSC of 0.619+-0.196, 0.573+-0.170 and 0.697+-0.157, respectively, and an ICC of 0.721[0.340-0.999], 0.597[0.288-0.999], 0.751[0.294-0.999], respectively. Of the DNN's three false negatives, none of them was correctly classified by the three graders. The model correctly classified RPD in three of the six eyes where graders disagreed and in the only eye where none of the graders found RPD. Overall, 65.1% of the area indicated as RPD by the reference was delineated by at least one grader and only 26.5% of the total was graded as RPD by all experts. The DNN only missed 23.2% of the areas that all three graders identified correctly.
Conclusions:
The proposed DNN showed promising capacities in the tasks of classifying and quantifying RPD lesions on multimodal retinal images. The results show that the model is able to correctly classify and quantify RPD on eyes where lesions are difficult to spot. The probabilistic output of the model allows for the classification of RPD at different levels of confidence and indicates what retinal areas are most likely affected. This is in line with the manual assessment done by the graders. To this point, the model is developed to classify and quantify RPD only on CFP, FAF and NIR. However, introducing other imaging modalities, such as OCT, might help diminish ambiguities in the classification and quantification of this abnormality. Therefore, a future direction for improving the proposed method is to include OCT scans as an additional input to the model. Automatic classification and quantification of RPD using deep learning on multimodal images will enable the automatic and accurate analysis of increasingly large amounts of data for clinical studies and will facilitate AMD screening in the elderly by decreasing the workload of the expert graders.
Financial Disclosure:
None},
month = {9},
year = {2020},
}

Want to get in contact?

sitemap

contact

hello@coendevente.com

schedule a meeting with me

links