Coronary artery segmentation in non-contrast calcium scoring CT images using deep learning

Abstract

Precise localization of coronary arteries in Computed Tomography (CT) scans is critical from the perspective of medical assessment of coronary artery disease. Although various methods exist that offer high-quality segmentation of coronary arteries in cardiac contrast-enhanced CT scans, the potential of less invasive, non-contrast CT in this area is still not fully exploited. Since such fine anatomical structures are hardly visible in this type of medical images, the existing methods are characterized by high recall and low precision, and are used mainly for filtering of atherosclerotic plaques in the context of calcium scoring. In this paper, we address this research gap and introduce a deep learning algorithm for segmenting coronary arteries in multi-vendor ECG-gated non-contrast cardiac CT images which benefits from a novel framework for semi-automatic generation of Ground Truth (GT) via image registration. We hypothesize that the proposed GT generation process is much more efficient in this case than manual segmentation, since it allows for a fast generation of large volumes of diverse data, which leads to well-generalizing models. To investigate and thoroughly evaluate the segmentation quality based on such an approach, we propose a novel method for manual mesh-to-image registration, which is used to create our test-GT. The experimental study shows that the trained model has significantly higher accuracy than the GT used for training, and leads to the Dice and clDice metrics close to the interrater variability.

[GLI]organization=Graylight Imaging,addressline=ul. Bojkowska 37a, city=Gliwice, postcode=44-100, state=Silesian Voivodeship, country=Poland

[US]organization=University of Silesia,addressline=ul. Bankowa 12, city=Katowice, postcode=40-007, state=Silesian Voivodeship, country=Poland

[SUT]organization=Silesian University of Technology,addressline=ul. Akademicka 16, city=Gliwice, postcode=44-100, state=Silesian Voivodeship, country=Poland

[SCCS]organization=Silesian Center for Heart Diseases,addressline=ul. Marii Skłodowskiej-Curie 9, city=Zabrze, postcode=41-800, state=Silesian Voivodeship, country=Poland

1 Introduction

Segmentation of coronary arteries in Computed Tomography (CT) scans plays a critical role in the assessment and treatment of coronary artery disease, which is one of the leading causes of death worldwide go_executive_2013 ; sun_computational_2014 . The availability of coronary artery models reconstructed based on CT with contrast enhancement, frequently referred to as Coronary Computed Tomography Angiography (CCTA), allows for utilization of computational techniques in a variety of problems, including blood flow simulation and biomechanical analysis sun_computational_2014 ; malawski_deep_2022 , virtual stenting and stent testing beier_hemodynamics_2016 ; sun_personalized_2019 , and it enables building virtual reality as well as physical models for educational purposes silva_emerging_2018 ; yoo_hands-surgical_2017 . The effort required for traditional, manual segmentation of coronary arteries makes it impractical in clinical applications due to the unacceptable amount of analysis time it demands. With the advent of deep learning in medical imaging in recent years isensee_nnu-net_2021 ; wasserthal_totalsegmentator_2022 , fast, robust, and fully automatic segmentation of coronary arteries became possible gharleghi_automated_2022 , which opens a wide range of new research pathways for applying various computational techniques in clinical practice.

Figure 1: The challenge of segmenting the coronary vessel from non-contrast CT is demonstrated by an example of the Left Anterior Descending (LAD) artery visualized in Stretched Curvilinear Reformulation (CPR)—the same vessel is shown in non-contrast and contrast CT. In each case, we show the CT image alone (left) and an overlaid ML segmentation mask (right). The red arrows point to the areas where Hounsfield values of the vessel and surrounding tissue are practically the same, which makes applying classic segmentation techniques, like region growth methods, impossible. Note that the Hounsfield range for non-contrast and contrast CT is different, and is set to $(-120,200)$ and $(-120,800)$, respectively.

The state-of-the-art techniques for segmenting coronary arteries in CCTA commonly exploit 2D and 3D U-Net architectures gharleghi_automated_2022 , which have been successfully used to solve a variety of problems in medical image segmentation siddique_u-net_2021 . The best-performing methods to date isensee_nnu-net_2021 yield mean Dice scores up to 0.88 over the unseen test scans gharleghi_automated_2022 . Nevertheless, a challenge persists to accurately segment distal parts of coronary arteries and maintain topological correctness of the resulting coronary vessel tree models, which is neither sufficiently reflected in the most common loss functions used in the network training nor in the standard segmentation quality metrics. In order to thoroughly quantify the capabilities of emerging algorithms, an array of metrics have been suggested. Apart from standard segmentation metrics such as Dice score, Recall, and Precision, specialized metrics for vessel segmentation have been proposed, including centerline Dice (clDice) Shit_2021_CVPR as well as the corresponding clPrecision and clRecall.

One of the most important challenges in the field is accurate segmentation of coronary arteries in non-contrast CT scans, used so far mainly for calcium scoring agatston_quantification_1990 , but, if done more precisely, having a great potential in terms of improving downstream tasks such as assignment of calcifications to proper branches of the coronary artery tree. Moreover, by solving such a problem, one could significantly improve explainability of the automatic calcium scoring methods via utilization of visualization techniques taking advantage of additional anatomical information. Potentially, Machine Learning (ML) models based on non-contrast CT could offer diagnostic value for patients with health contradictions to contrast agent administration, as well. Nevertheless, such a segmentation task is particularly demanding due to limited information about coronary arteries, being very fine-grained structures, in an image without contrast enhancement isgum_automatic_2012 ; wolterink_automatic_2014 ; yang_automatic_2016 ; lessmann_automatic_2018 . Even for experienced medical imaging specialists, it is challenging to model the main branches of the vessel tree based on such images using the commonly available segmentation tools. In Fig. 1, we demonstrate this difficulty in an example of a well-visible Left Coronary Artery (LCA) vessel fragment in non-contrast CT containing no imaging artifacts. Even in this case, the coronary vessel merges with the surrounding tissue in certain regions (indicated by the arrows in Fig. 1), making local signal thresholding insufficient for accurate segmentation. Thus, the segmentation algorithm must incorporate data extrapolation and leverage the atlas’ knowledge of the vascular tree anatomy to ensure precise vessel geometry segmentation. We address this research gap of automated coronary artery segmentation in non-contrast CT by proposing a novel framework for automatic generation of GT data and training ML models based on weak supervision by a human expert who performs only fast, binary decisions to assess the quality of training data, which is the main contribution of this work. We demonstrate the value of this process by introducing a new, manual mask-to-image registration method and applying it to a set of cases, including open-source data, to generate unique GT samples for testing, which we make publicly available. In Section 1.2, the most important contributions of our work are are discussed in more detail, whereas the current state of the art in the field is summarized in Section 1.1.

1.1 Coronary artery segmentation in non-contrast CT

The approaches toward automatic segmentation of coronary arteries in non-contrast CT can be split into two groups, (i) using image registration, including selected atlas-based approaches, and (ii) those which exploit ML, where deep neural networks play an essential role. The first group of methods takes advantage of the fact that the coronary arteries are relatively well visible in contrast-enhanced CT. Since in the standard protocol for atherosclerosis assessment both contrast and non-contrast series are acquired shahzad_patient-specific_2010 , it is possible to use the methods for coronary artery segmentation in contrast CT, and align the resulting model with the non-contrast image using registration techniques klein_elastix_2010 ; shamonin_fast_2013 ; joshi_unbiased_2004 . When only a non-contrast CT scan is available, a generic vessel tree model can be used to segment the coronary arteries in the non-contrast image directly. In one of the first approaches of this type, Shahzad et al. shahzad_patient-specific_2010 ; shahzad_vessel_2013 used 85 contrast CT scans with manual annotations of the main coronary arteries’ centerlines to construct a set of 10 atlases via nonrigid transformation of images. As a result, each of the atlases contained voxel density information indicating the location of coronary vessels. Finally, each of the atlas images was aligned with the non-contrast scan based on automatic registration using Elastix klein_elastix_2010 , and the resulting density information was averaged to give a rough estimation of the location of the main coronary arteries. A similar, multi-atlas approach was used by Wolterink et al. wolterink_automatic_2014 . Here, a small set of contrast CT images with manual annotations of LAD, Left Circumflex (LCX), and Right Coronary Artery (RCA), were aligned with the test image by means of affine transformation, which was followed by elastic registration using Elastix as well klein_elastix_2010 . The transformations were used to propagate the coronary vessel tree models from the atlases, and consensus centerlines were estimated using a Selective and Iterative Method for Performance Level Estimation (SIMPLE) langerak_label_2010 . In an alternative approach, Kelm et al. kelm_automatic_2014 used a robust centerline tracing algorithm zheng_robust_2013 to extract patient-specific coronary centerline vessel trees from contrast CT scans. At the same time, in both contrast and non-contrast series, pericardium and the aortic root were segmented using the marginal space learning algorithm zheng_four-chamber_2008 . The masks were used to fit the coronary centerline tree to the non-contrast image via registration using the thin-plate-spline model zheng_pericardium_2014 . Then, Kondo et al. kondo_semi-automatic_2015 performed a patient-specific registration of the main coronary arteries, segmented in contrast CT, to the non-contrast CT. However, the authors used a semi-automatic process utilizing region growing techniques to obtain the segmentations in the contrast CT scans and relied on a different registration framework johnson_brainsfit_2007 . Finally, Yang et al. yang_automatic_2016 also used a patient-specific image registration consisting of consecutive affine and nonrigid transformations. The coronary arteries were automatically segmented based on the original contrast CT scan using Frangi’s vesselness filter frangi_multiscale_1998 . To increase the accuracy of the coronary artery segmentation, after the standard image-to-image registration, nonrigid registration was performed exclusively within the estimated heart region. All in all, despite the high practical value of the registration-based coronary vessel segmentation techniques, which helped to significantly improve the calcium scoring process, the segmentation accuracy is relatively low lee_fully_2021 , especially in the context of the recent developments in deep learning for medical image analysis suganyadevi_review_2022 ; chen_recent_2022 ; isensee_nnu-net_2021 ; malawski_deep_2022 ; kotowski_detecting_2023 . Moreover, compared to deep learning solutions, atlas-based methods are considerably slower and the computation time increases substantially for a high number of atlases lee_fully_2021 . Additionally, methods using patient-specific registration require availability of both contrast and non-contrast CT series. The above mentioned drawbacks of the registration-based methods motivate the development of deep learning approaches for segmenting coronary arteries in non-contrast CT.

The second group of methods consists of approaches which benefit from classic ML relying on handcrafted feature extractors as well as deep neural networks that exploit automated representation learning isensee_nnu-net_2021 ; wasserthal_totalsegmentator_2022 . Brunner et al. brunner_toward_2010 proposed to use affine transformations of the scan coordinate system to obtain a common representation of the heart region across different patients. The reference frame was transformed to match predefined anatomical landmarks, being the aortic root and the apex of the heart. Then, using transformed coordinates as the input features, a support vector machine was trained on manually labeled trajectory points of the main coronary artery zones, i.e., LAD with Left Main (LM) coronary artery, LCX, and RCA, where each of the zones was divided into three equidistant subsections. The model was not used directly to segment the coronary arteries, but was utilized for labeling of coronary vessel calcifications. The method, however, is not fully automatic, since it requires localization of the anatomical landmarks by a human expert. Gonzalez et al. gonzalez_automated_2016 proposed an automated Agatston score computation methodology based on a ML method for identifying the beginnings of the coronary arteries. The great vessels of the heart, e.g., aorta, pulmonary artery, and coronaries, were detected utilizing 2D object detectors, being adaptive boosting classifiers using Haar wavelets as the input features. The segmented organs were used to enhance the detection of coronary artery calcifications based on standard thresholding agatston_quantification_1990 by defining a region of interest corresponding to the location of the heart. However, it is difficult to judge the accuracy of detecting the coronary arteries. Also, the method uses exclusively 2D images, which is a severe limitation in medical image analysis. Finally, Lee et al. lee_fully_2021 proposed a promising approach using both image registration and 3D deep learning techniques for image segmentation. For each of 100 patients, contrast CT images with manual segmentations of coronary arteries and other surrounding structures including, i.a., aorta, left and right ventricle, and mitral annulus, were aligned with the corresponding non-contrast scans using image registration based on Elastix klein_elastix_2010 ; shamonin_fast_2013 . The deformed masks of organs in the reference frame of non-contrast CT scans, together with the images, were used as GT for training 3D-patch U-Nets simonyan_very_2015 . For detection of coronary arteries, a patch of $64\times 64\times 64$ voxels of size of 1 [mm] in each direction was used, and the masks were dilated two times with $2\times 2\times 1$ morphological structuring elements to account for registration errors. The information about the location of the main branches of the coronary vessel tree, i.e., RCA, LAD, LCX, LM, was included in GT as well. The resulting deep network was able to learn an anatomical atlas model of coronary arteries, which was used for detecting coronary artery calcifications. Although the accuracy metrics for coronary artery segmentation were not presented, the results for calcium scoring were better than in the other works kurkure_supervised_2010 ; shahzad_vessel_2013 ; wolterink_automatic_2015 , which suggests good segmentation accuracy. Furthermore, the method allows to reduce the computational time compared to the multi-atlas approaches shahzad_patient-specific_2010 ; shahzad_vessel_2013 ; wolterink_automatic_2014 . The segmentation model, however, was tailored for detecting coronary calcium and used dilated, low-resolution masks covering larger regions around the coronary arteries, and it is not suitable for tasks requiring higher level of precision, such as lumen segmentation. Moreover, it seems that the outputs of the U-Net model were post-processed to include only the two largest components, which might result in an increase of false negative classifications, especially for the cases including severe calcifications and artifacts.

1.2 Motivation and contribution of this work

The available approaches based on ECG-gated non-contrast CT mainly focus on calcium scoring rather than coronary artery segmentation itself, and therefore, usually provide only low-resolution segmentation masks. Here, precise coronary tree models obtained from non-contrast CT images could be directly utilized to develop improved algorithms that automatically segment calcification within coronary arteries, thereby enhancing their effectiveness not only in calculating the total coronary artery calcium score but also in improving the assignment of calcification to the appropriate segment of the coronary artery tree. Currently available software on commercial diagnostic consoles still does not provide automated solutions to those tasks. Additionally, visualization of coronary arteries in non-contrast CT images can also be employed for the detection of potential anomalies in the origin and course of the coronary arteries. Furthermore, promising efforts are currently underway to develop advanced algorithms utilizing radiomics for the assessment of coronary artery plaques and pericoronary adipose tissue, which are based on manual segmentations of coronary arteries on ECG-gated non-contrast CT scans Jiang2022 . Therefore, full automation of the segmentation process of non-contrast CT images could enhance the performance of these algorithms and pave the way for broader utilization of novel markers for cardiovascular risk assessment.

Figure 2: Framework for semi-automatic GT generation and training of non-contrast coronary artery segmentation models.

The objectives of this work are multi-fold. The primary objective is thus to propose a scalable end-to-end framework (Fig. 2) for automated, high-quality segmentation of coronary arteries in multi-vendor ECG-gated non-contrast cardiac CT scans, and to thoroughly evaluate its accuracy. To ensure full reproducibility of experiments, we build our implementation on publicly available state-of-the-art image registration yushkevich_fast_2016 and deep learning isensee_nnu-net_2021 techniques. Moreover, we propose a novel methodology for the generation of GT for testing the resulting ML model based on manual registration of high-resolution coronary artery masks segmented in contrast CT to the corresponding non-contrast CT images. This allows for computation of both standard and specialized segmentation quality metrics, which we confront with the metrics from an interrater evaluation. The results of our experimental study performed over a stratified test set consisting of 17 multi-vendor ECG-gated non-contrast cardiac CT scans with the manually registered coronary models confirm high potential of the proposed deep learning approach, which effectively learns an atlas able to interpolate the segmentation in the areas without a clear signal in the image. Based on qualitative analysis, we conclude that the obtained segmentations are significantly more accurate than the ones presented in the literature, opening up possibilities for their use in other application areas.

The remainder of this paper is structured as follows. Section 2 presents our coronary artery segmentation framework. In Section 3, the manual mesh-to-image registration method for generation of test-GT is proposed. Section 4 summarizes the dataset used in this study. In Section 5, a description of the training process and the qualitative and quantitative evaluation of the segmentation quality is presented. Section 6 concludes the work and gives indications for the future research.

2 Registration-based deep learning framework for coronary artery segmentation

In this section, a semi-automatic framework for GT generation and deep learning model training is described. Our approach is inspired by Lee et al. lee_fully_2021 , however, we exploit the state-of-the-art open-source algorithms yushkevich_fast_2016 ; isensee_nnu-net_2021 with default parameters to allow for an easy reimplementation of the proposed method and replication of results. In contrast to Lee et al. lee_fully_2021 , our research prioritizes high Precision in segmentation models over Recall, as our aim is to reconstruct the anatomy of coronary vessels, rather than to simply facilitate calcium scoring. Therefore, we suggest to use upsampled images and apply additional verification steps by medical imaging experts to generate GT for training of the neural network model. Fig. 2 presents an overview of the proposed framework, whose main components are discussed in detail in the sections below.

2.1 Generation of GT for training deep learning models

Due to the challenging character of manual segmentation of coronary arteries in non-contrast CT, which prohibits the creation of large datasets required by deep learning techniques, we propose to semi-automatically generate the training GT based on a three-step approach. For each patient, we perform an image-to-image registration by moving the contrast scan to the reference frame of the non-contrast CT. This first step is realized within the Greedy framework yushkevich_fast_2016 ; joshi_unbiased_2004 with default parameters. For the sake of generality, we rely in the further processing exclusively on the registered images. Hence, the registration process can be treated as a black box, and other algorithms keszei_survey_2017 , including commercial solutions, can be easily used here as well. Then, we propose to use a model of coronary arteries trained on contrast CT to segment coronary vessels in the reference frame of the non-contrast image based on the registered contrast scans. We exploit our 3D U-Net model malawski_deep_2022 built upon the nnU-Net framework isensee_nnu-net_2021 . However, thanks to the generality of the proposed approach, any other model wasserthal_totalsegmentator_2022 can be easily applied to automatically generate GT candidates.

Finally, the candidate GT samples consisting of non-contrast images and the corresponding coronary artery segmentation models are reviewed by a human expert. We suggest selecting a subset of samples that satisfy specific quality criteria, evaluated based on visual inspection of a registered coronary vessel mask overlaid on a non-contrast image. Here, as the evaluation criterion, we took the segmentation visual accuracy of the proximal parts of coronary arteries. Those parts are usually clearly visible in non-contrast CT images, as they do not “touch” the myocardium (see the upper part of Fig. 1).

2.2 Deep learning segmentation model

We utilize the state-of-the-art nnU-Net framework isensee_nnu-net_2021 for training of a coronary artery segmentation model based on the semi-automatically generated GT. The nnU-Net automatically configures its hyperparameters and network structure during the training process, and delivers an optimized 3D U-Net model. Both non-contrast CT scans and segmentation masks are in the upsampled isotropic spacing of 0.35 mm. Due to the demanding hardware requirements of the high-resolution coronary vessel segmentation, the final segmentation is generated based on network inferences in multiple patches of the $160\times 160\times 96$ size (in voxels). The output models of coronary arteries are post-processed to eliminate the artifacts generated by the neural network. This is accomplished by filtering out all the groups of connected voxels of volume less than 50 mm³ and removing all the voxel groups which are positioned entirely outside of pericardium, which we segment in non-contrast CT using an openly available SegTHOR model lambert_segthor_2020 .

3 Manual mesh-to-image registration method for generation of test-GT

Since the deep learning models presented in this paper are trained based exclusively on the semi-automatically generated GT, the evaluation of the segmentation quality plays a critical role, and a separate GT for testing is needed to thoroughly understand the generalization capabilities of the elaborated models. However, using standard tools based on region-growing techniques to generate test-GT is not feasible due to low image intensity gradients in the CT scans without contrast enhancement (Fig. 1). Hence, we propose a novel method for generating test-GT via manual registration of geometric models of coronary arteries to non-contrast CT scans. Here, we describe a process based on the tools available in Blender, an open-source 3D computer graphics software widely used in various domains. However, the presented concepts can be easily transferred to other graphics software packages as well.

We introduce a manual mesh-to-image registration method which operates as follows. First, given a 3D surface mesh of the coronary vessel tree being registered, i.e., LCA or RCA, a centerline representation is automatically extracted using skeletonization techniques saha_survey_2016 or similar approaches izzo_vascular_2018 . Here, we used the Vascular Modeling Toolkit (VMTK), which is an open-source software package specialized for the geometric processing of blood vessels. The resulting representation can contain artifacts, e.g., extra centerlines, hence, the models should be manually post-processed in the 3D computer graphics software to remove the additional objects.

Figure 3: High-resolution LCA tree model with its skeletal representation in Blender. The tips and roots of individual Bones coincide with the vertices of the centerlines.

(a) Initial mesh.

(b) Axial view (initial LCA contours).

(c) Step 1.

(d) Axial view, LCA contours (Step 1).

(e) Step 2.

(f) Axial view, LCA contours (Step 2).

Figure 4: The alignment of a 3D coronary vessel tree model (panels (a), (c), and (e)) with the non-contrast CT scan (panels (b), (d), and (f)) based on rotations of subsequent Bones. The black solid arrows and circles in the 3D views (a), (c), and (e) indicate the center of rotation at a given alignment step. The rotation directions are shown using dashed curves. The corresponding contours of the LCA tree at a fixed slice of the CT scan are depicted in the axial views (b), (d), and (f).

Subsequently, based on the extracted centerlines, we create a skeletal representation consisting of the Blender’s Bone objects, which will serve as a geometric parameterization of the original, high-resolution 3D mesh model. To achieve this, Bones are created along consecutive sections of each centerline, by aligning their roots and tips with the curve vertices using the Snap functionality, as depicted in Fig. 3. We recommend using Bones of approximately 3 mm length or shorter, if a refinement of the representation is needed due to high curvature of a centerline. In Blender, we propose to use the Extrude tool to quickly generate a single Chain of Bones for the entire coronary vessel tree. At $n$-furcation points, the Extrusion operation should be used more than once to create separate branches belonging to the same Chain of Bones. As a result, the downstream Bones will move in a coordinated manner with the Bones located in the upstream branches when the latter ones are displaced. It is of note that for the sections where multiple centerlines overlap, the Bone objects should be created only once. Finally, a relationship between the generated skeletal representation and the high-resolution vessel tree model has to be established. In Blender, this can be done using Armature Deform Parenting, e.g., With Automatic Weights. As a result, the influence of each Bone on the mesh to deform is calculated automatically according to the distance from the Bone to the vertices of the mesh.

Based on the newly created skeletal parameterization of the coronary vessel tree, the actual manual registration process can be carried out. This is done by iteratively rotating tips of the Bones to match the contours of the tree model with the vessels visible in the non-contrast CT scan based on an axial view, as shown in Fig. 4. The process starts by applying a rigid transformation of the initial vessel tree model to align its root with the ostium, which is usually clearly visible in a non-contrast CT scan. Subsequently, starting from the root of the tree, individual Bones are rotated as illustrated in Fig. 4. The deformations of the vessel tree are very intuitive since the child branches of the Bone being modified are rotated by the same angle, which is only a rigid transformation of the downstream part of the model. Once a part of the branch is no longer visible in the non-contrast scan, it is cut at that point, and all of its child branches are removed from the model.

The manual mesh-to-image process is finalized by applying local corrections of the shape of the tree, e.g., with use of Grab and Thumb brushes in Blender. Finally, the resulting, post-processed coronary vessel tree models are stored and used as test-GT in the evaluation presented in Section 5.2.

4 Materials

To train a model capable of segmenting coronary arteries in ECG-gated non-contrast CT scans according to the procedure presented in Section 2.2, a set of contrast-enhanced and the corresponding non-contrast CT scans is required to generate GT semi-automatically, as described in Section 2.1. Hence, in the section below, both types of CT scans, as well as the geometric models of coronary vessels resulting from the semi-automatic segmentation process are described. Moreover, to evaluate the capabilities of the trained model, a dataset consisting of patient-specific segmentations of coronary arteries based on contrast CT and the corresponding non-contrast CT scans is required. Following the mesh-to-image registration method presented in Sect. 3, we use this data to create a test-GT dataset, which is described in the following section.

4.1 Selection of CT scans

The selection of samples comprising the datasets used in this study for training of the models was based on the qualitative assessment of segmentation quality (as described in Section 2.1) by a medical imaging specialist with 4 years of clinical experience. Additionally, scans with anatomical abnormalities or high noise levels were excluded according to the specialist’s visual inspection. The final result of the selection procedure was a dataset of 98 multi-vendor ECG-gated cardiac non-contrast CT scans with corresponding masks of coronary vessels. We refer to the data corresponding to each patient as a case. The manually prepared test-GT (Section 3) was based on a randomly chosen set of 17 multi-vendor scans from a separate pool of cardiac CT scans, each containing both contrast and non-contrast series (Table 1). A subset of 9 cases, 3 in the training and 6 in test-GT sets, were derived from the OrCaScore dataset wolterink_evaluation_2016 , being a public set of calcified plaque scores from the MICCAI 2014 Challenge on Automatic Coronary Calcium Scoring.

Table 1: Characteristics of the CT scans used in this study.

black!7! Characteristic	Training dataset	test-GT dataset
LightCyan Population characteristics
Age, years	$59\pm 11$	$64\pm 9$
Male	35%	18%
LightCyan CT scans
Scanner model
Siemens SOMATOM      Definition AS+	3	2
Siemens SOMATOM      Definition Flash	92	10
Siemens SOMATOM      Force	0	2
GE Lightspeed VCT	1	3
Philips Brilliance iCT	2	1
Toshiba Aquilion ONE	0	2
red!20!green!15!    Non-contrast CT scan
Tube voltage, [kVp]
$120$	98	17
In-plane voxel size, [mm]	$0.41\pm 0.04$	$0.44\pm 0.08$
red!20!green!15!    CCTA scan
Tube voltage, [kVp]
$80$	0	3
$100$	91	13
$120$	5	1
$140$	2	0
In-plane voxel size, [mm]	$0.41\pm 0.04$	$0.40\pm 0.04$

4.2 GT segmentations

In this study, we exploit two separate GT sets: a semi-automatically generated GT (Section 2.1) for training deep learning models (98 segmentations of coronary arteries), alongside the test-GT set for evaluating their generalization capabilities. To generate the test-GT, a group of experienced medical imaging specialists manually segmented coronary arteries in the contrast CT scans. Then, a set of 17 coronary tree segmentations was created by a medical imaging expert (4 years of experience), by aligning the contrast-based segmentations with the non-contrast CT scans according to the procedure described in Section 3. As a result, a test-GT set consisting of pairs of non-contrast CT scans and the aligned coronary vessel segmentations for 17 patients was created. To allow for an easy inspection of the resulting segmentations, we publish bujny_dataset_2023 a subset of 6 test-GT samples generated based on the scans belonging to the OrCaScore dataset wolterink_evaluation_2016 . Additionally, to assess the quality of the manual alignment procedure, a second rater (medical expert, 2 years of clinical experience) performed additional alignments on a randomly selected set of 10 vessel trees. These segmentations were used in the quantitative evaluation of our model (Section 5.2).

Table 2: Coronary artery segmentation metrics obtained by the deep model over the validation and test-GT sets, including the impact of post-processing (vol50: filtered out all groups of connected voxels of volume less than 50 mm³, pericardium: removed all voxel groups which lay entirely outside of pericardium)—the results obtained using the final deep learning processing chain are boldfaced. The metrics for interrater test (test-GT vs. second rater) are shown as well.

LightCyan				Dice			Recall			Precision
Compared segmentations	Post-processing	mean	median	std. dev.	mean	median	std. dev.	mean	median	std. dev.
blue!5!black!7!green!20 Interrater test	None	0.67	0.67	0.09	0.67	0.65	0.11	0.68	0.68	0.07
red!5!black!7!orange!20 ML model vs. validation set	None	0.57	0.55	0.10	0.55	0.53	0.11	0.61	0.60	0.10
ML model vs. test-GT	vol50+pericardium	0.65	0.65	0.08	0.61	0.59	0.10	0.73	0.76	0.11
	vol50	0.64	0.64	0.08	0.61	0.59	0.10	0.71	0.74	0.13
	pericardium	0.65	0.64	0.07	0.63	0.62	0.10	0.68	0.69	0.10
	None	0.62	0.63	0.09	0.63	0.62	0.09	0.63	0.65	0.13
LightCyan				clDice			clRecall			clPrecision
Compared segmentations	Post-processing	mean	median	std. dev.	mean	median	std. dev.	mean	median	std. dev.
blue!5!black!7!green!20 Interrater test	None	0.64	0.63	0.06	0.64	0.57	0.1	0.65	0.66	0.06
red!5!black!7!orange!20 ML model vs. validation set	None	0.43	0.41	0.11	0.34	0.31	0.10	0.59	0.58	0.15
ML model vs. test-GT	vol50+pericardium	0.65	0.63	0.08	0.52	0.49	0.11	0.91	0.95	0.08
	vol50	0.64	0.63	0.06	0.52	0.48	0.11	0.87	0.93	0.13
	pericardium	0.65	0.65	0.07	0.56	0.52	0.09	0.79	0.81	0.07
	None	0.62	0.62	0.10	0.56	0.53	0.10	0.71	0.74	0.14

5 Results

In this section, we report and discuss the results of our experimental study. Section 5.1 describes the training process of the network for segmentation of coronary arteries in non-contrast CT scans. In Section 5.2, we present a critical, qualitative as well as quantitative analysis of the deep model.

5.1 Training a deep learning model

The dataset of 98 cases (Section 4) was used for training a deep neural network within the nnU-Net framework, where 20 CT scans were used exclusively for validation of the model during the training procedure. The training process was carried out until reaching nnU-Net’s default limit of 1000 epochs. The quality metrics for the inferences of the resulting model on the validation set (20 cases) are presented in Table 4.2.

With a mean validation set’s Dice at the level of $0.57\pm 0.10$, the segmentation accuracy is relatively low compared to the best methods for segmentation of coronary arteries based on contrast CT, reaching Dice scores up to 0.88 gharleghi_automated_2022 . However, there are two important aspects which should be noted here. First, for coronary arteries, being very thin structures, the Dice metric decreases substantially even in case of small errors on the voxel level, which also results in low interrater Dice for this type of segmentation problems. The information contained in the non-contrast CT scan is far more scarce than in contrast CT, which naturally leads to higher segmentation errors as well. Secondly, and more importantly, the metrics presented here are calculated on a validation set consisting of semi-automatically generated GT. The sole criterion used in the selection of CT scans accepted in our study for the training set and the validation set was the alignment of the proximal regions of the coronary arteries. Therefore, the lower Dice scores observed in this case may be attributed to the lower quality of the GT data in the distal regions. This issue is illustrated in Fig. 5, which shows that segmentation of the distal part of RCA based on the trained model is more consistent with the CT scan than the GT used for training. Indeed, in some cases, the inference of the model is better than the underlying GT used for both training and validation, which is counter-intuitive, but can be justified by the strong regularization imposed on the U-Net model. The network learns robust filters to detect vessels from those parts of the dataset where registration is correct and is not degraded by misinformation caused by massive but relatively rare misalignments. Fig. 5 is a prominent example that the model is more accurate than the GT used for its training.

Figure 5: Example of a mismatch between the GT created based on registration ($\mdblksquare$), the inference of the neural network ($\mdblksquare$), and the manually aligned contrast GT ($\mdblksquare$) in the distal part of RCA (a), juxtaposed with the corresponding non-contrast CT scan. Panel (b) is a CPR projection along the vessel centerline indicated in (a) by a solid black rectangle. Panels (c) and (d) show cuts in the vessel’s short axis at two distinct points. Based on the visual comparison, we can note that the inference of the deep model and manually aligned vessel are consistent with the image, unlike the GT used in the training process.

Even though a deep learning model trained on imperfect data may produce visually appealing segmentation results, it is important to accurately evaluate its performance using reliable GT data. To address this issue, we manually created the test-GT dataset, as detailed in Section 3. The evaluation results based on test-GT are presented in the following section.

5.2 Quantitative evaluation based on test-GT

We utilize a manually aligned test-GT which contains information regarding the actual anatomy of the patient’s coronary vessels to assess the model quality. Classic segmentation measures, including Dice, Precision, and Recall, along with their centerline counterparts, clDice, clPrecision, and clRecall, are computed. These metrics are presented in Fig. 6 and gathered in Table 4.2. A subset of 6 inferences based on the scans from the OrCaScore dataset wolterink_evaluation_2016 is available online bujny_dataset_2023 .

Figure 6: Distribution of the metrics obtained by the deep learning (ML) model over the test-GT set ($\mdblksquare$) and the ML model tested on the validation set ($\mdblksquare$). The non-transparent areas indicate the sample range. Additionally, the $25\%$ and 75% percentiles ($\cdot$$\cdot$) and median (--) are marked in the plot.

The primary outcome of this work is an ML model for the segmentation of coronary arteries in non-contrast CT scans, yielding Dice coefficient of $0.65\pm 0.08$ on the manually registered test-GT. Fig. 6 summarizes these results and compares the distribution of the metrics obtained on the validation set as well as test-GT. The evaluation reveals that the performance of the model computed on the test-GT is consistently better than when computed over the validation set, which is a consequence of the significant but rare misalignment during image-to-image registration. This observation is confirmed by Mann-Whitney tests verifying if the differences in distributions of the metrics are statistically significant. For Dice, Precision, clDice, clRecall and clPrecision they are indeed statistically significant with the $p$-values of 0.0305, 0.0008, $<0.0001$, $<0.0001$, and $<0.0001$, respectively. For Recall, the $p$-value amounted to 0.1582, showing that this metric remained consistent for both datasets, with significantly better Precision for test-GT.

While the overlap metrics (Dice, Precision, and Recall) are close to each other on both datasets, the metrics describing the coronary tree, i.e., clDice, clPrecision, and clRecall, are significantly higher when computed over the test-GT set. In case of the segmentations based on our ML model, the Dice coefficient is a geometric average of a low Recall and a high Precision. To correctly understand why Recall is low, let us note that it is intrinsically limited by the low visibility of coronary vessels in non-contrast CT data. At the same time, we compare the model-based segmentations with the tree geometries derived from high-quality contrast CT scans. Conversely, high Precision indicates that if the model detects a vessel, it will likely be the actual vessel of a patient. The degree of overlap between coronary trees is quantified by the clPrecision metric, which achieves values exceeding 0.9.

Examining the model’s inferences on a per-test-case basis can provide valuable insights. Here, we present only three extreme cases: the worst, medium, and the best inferences of the deep learning model on test-GT, as quantified by Dice. In Fig. 7 we can appreciate that the basic coronary tree structure is recovered in all the cases. The worst case, with Dice equal 0.53, has a significant discontinuity in the RCA tree, caused by a motion artifact in the CT acquisition. In all other cases, small and easily repairable discontinuities are present. One should also note that the source of the low Recall and clRecall values lies primarily in missing, hardly visible vessel endings.

Figure 7: Inferences obtained using the ML model for three selected test-GT (green) samples and the corresponding, manually registered coronary vessel trees (light gray). The worst (Dice below average), average, and the best (Dice above average) cases are presented. The ML model correctly predicts vessel geometry in all cases, and the results coincide with the manually aligned GT. However, in all cases, many small branches are missing. Red arrows indicate model’s imperfections such as small discontinuities and missing branches.

The nnU-Net uses a standard sliding window for CT image processing, which results in some false positives as patches often do not have information regarding the global heart position. Thus, it is reasonable to use post-processing techniques to eliminate small disconnected vessel fragments (the vol50 post-processing routine) and structures outside the pericardium (pericardium post-processing), which, in this study, was segmented using another nnU-Net model, SegTHOR lambert_segthor_2020 . Our study reveals that the post-processing (vol50+pericardium) does significantly affect Dice, Recall, Precision, clRecall and clPrecision metrics (the $p$-values obtained using two-side Wilcoxon tests amounted to 0.0002 for Dice, and $<0.0001$ for all other metrics)—this systematic improvement can be also appreciated in Table 4.2. It is of note that the combined post-processing (vol50+pericardium) offered statistically significant improvements (at $p<0.05$) when compared with other post-processing variants involving either vol50 or pericardium post-processing, as well as the lack of post-processing, in the majority ($11/18$) of pairwise comparisons and metrics. Therefore, all of the results presented in the analysis above included the (vol50+pericardium) post-processing step.

Finally, we approached the problem of assessing the quality of the manual alignment procedure. The results including a comparison with a second rater are shown in Fig. 8, demonstrating that the procedure includes a certain degree of randomness. It can be noted that the second rater scores Dice and clDice slightly but systematically higher than the ML model compared to the first rater. However, in these cases, Precision and Recall are similar to each other. Based on these findings, we conclude that the model’s performance is on par with the interrater agreement, indicating that it likely extracts all the information that can be extracted from non-contrast CT scans, considering the image resolution and signal level.

Figure 8: Distribution of the interrater metrics ($\mdblksquare$) and those obtained by the ML model tested on the subset of test-GT dataset containing the same five cases ($\mdblksquare$). The non-transparent areas indicate the sample range. Additionally, the $25\%$ and $75\%$ percentiles ($\cdot$$\cdot$), and median (--) are marked.

6 Conclusion and Outlook

Segmentation of coronary arteries based on non-contrast cardiac computed tomography scans is a challenging medical image analysis task. However, using CT without contrast enhancement can be highly beneficial for patient diagnosis, as it is less invasive than contrast-based examinations. This paper proposed a method for achieving high-precision segmentation of coronary arteries based on multi-vendor ECG-gated non-contrast cardiac CT images, using a framework for semi-automatic Ground Truth (GT) generation (via image registration) and a deep neural network model. The effectiveness of the proposed solution was confirmed through a quantitative analysis over the test set generated using a novel manual mesh-to-image registration method, which was ultimately exploited to evaluate the quality of our approach.

The major finding of this work is that our deep neural network model is able to effectively generalize over the unseen multi-vendor ECG-gated non-contrast cardiac CT scans, and it allows for achieving high-quality results despite the relatively low-quality GT used for training. It is of clinical importance, as the segmentation of coronary arteries in low-dose non-contrast calcium scoring CT images can be utilized to generate more advanced algorithms for automatic coronary artery calcium scoring, leading to improved accuracy of coronary artery calcification detection and their assignment to the appropriate coronary tree segment. More precise segmentation of coronary arteries on non-contrast cardiac CT scans can also aid in identifying coronary anomalies and potentially enhance the performance of radiomics algorithms in evaluating coronary artery plaques and pericoronary adipose tissue Jiang2022 . Moreover, in future studies utilizing high-resolution ECG-gated non-contrast cardiac CT images, it could be tested whether deep learning coronary artery segmentation allows for the extraction of additional clinically relevant information from these images. Such an approach could be implemented initially in selected patient subgroups, such as those with contraindications to intravenous iodine-based contrast agent administration.

To further increase the segmentation accuracy, it is necessary to improve the quality of GT. Manual generation of GT seems unreasonable due to the time-consuming and user-dependent nature of this process. Improving the registration method is the key to increasing the quality of the semi-automatically generated GT and, consequently, creating better segmentation models. Another promising approach might be to eliminate the post-processing step and incorporate it into the machine learning algorithm via utilization of specialized techniques, e.g., graph neural networks. Modifying the model’s architecture may also lead to an increase in the connectivity of the obtained segmentations. A greater challenge, however, is how to improve the model’s ability to segment the vessel endings, which are practically invisible in the non-contrast CT scans. Finally, we are aware of the limitation of this study concerning the size of the test sets—our current efforts are focused on validating the proposed framework over the scans acquired in other clinical sites (and using different scanners as well). We do, however, believe that the current validation procedure indeed provides solid insights that reflect the operational capabilities of the proposed technique.

Declaration of competing interest

All authors declare that they have no competing interests.

Acknowledgments

This work was supported by the National Centre for Research and Development POIR.01.01.01-00-0092/20. Jakub Nalepa was supported by the Silesian University of Technology grant for maintaining and developing research potential.

Supplementary material

A subset of 6 manually-registered test-GT samples generated based on the scans belonging to the OrCaScore dataset wolterink_evaluation_2016 and the corresponding segmentations using the non-contrast coronary artery segmentation model presented in the paper is available under: https://doi.org/10.5281/zenodo.7808199.