Accuracy and feasibility of using artificial intelligence software to segment the organs at risk for patients treated with radiotherapy for head and neck cancer

Abstract

Purpose: This study aimed to evaluate the accuracy of an in-house developed artificial intelligence (AI) based software in contouring organs at risk (OARs) namely the parotids, submandibular glands, mandible, brainstem, optic nerves, and optic chiasm. Methodology: Ten computed tomography images (CT) were identified from an online open-source dataset developed by the University of Ljubljana. The accuracy of the AI software in relation to an expert gold standard contour was assessed by calculating the volume ratio, Clinical Index (CI), and 95% Hausdorff distance (95%HD). A high DSC (closer to 1) and low 95%HD (closer to 0mm) indicate higher accuracy. Two experienced radiographers were asked to rate the AI-generated contours using a Likert scale ranging from 1 (unusable) to 5 (clinically acceptable). In addition, the radiographers were also asked to provide feedback on the use of the system through the use of a questionnaire. Results: The mean DSC score for all structures was 0.56±0.26 and was highest for the brainstem (0.81±0.08) and lowest for the right optic nerve (0.18±0.14). The mean 95%HD was 12.07mm±16.68 and was highest for the mandible (57.24mm ± 3.15) and lowest for the optic chiasm (4.54mm±3.39). Compared to the gold standard segmentations, the mean AI segmented volume was smaller for all OARs except for the left parotid and brainstem. However, the difference was only significant for the left parotid and optic nerves (P<0.05). The mean rating provided by the radiographers was 2.47±0.68. The highest rating was provided for the right parotid (3.10±0.21) and the lowest for the optic chiasm (1.36±0.48). Under-contouring for some OARs, particularly the mandible and unusable optic chiasm contours were key issues highlighted by the radiographers. Conclusion: The AI software has potential for clinical use however further refinement is required for some OARs, particularly the mandible, optic nerves, and optic chiasm.