A generative adversarial network for delineation of retinal interfaces in OCT B-scans with age-related macular degeneration

Abstract

Age-related macular degeneration (AMD) is a retinal disease that can impair the central vision permanently. Accurate delineation of the retinal pigment epithelium (RPE) and Bruch’s membrane (BM) in optical coherence tomography (OCT) B-scans is crucial for diagnosing and monitoring AMD. While automated segmentation methods exist for early AMD stages, late-stage AMD remains a challenging area due to the pronounced disruption of the RPE and BM. To ensure spatial contiguity in the boundary delineation of RPE and BM, both the global and local contextual information must be learned. In this context, we propose a generative adversarial network (GAN) to segment these significant retinal interfaces in OCT B-scans from AMD patients. A UNet++ model with its deep supervision is trained using a hybrid loss function combining adversarial loss and multi-class cross-entropy (CE) segmentation loss. The CE loss learns the local features by optimizing the per-pixel accuracy, while the adversarial loss captures a broader context by learning overall layer label statistics. This loss combination allows the model to capture fine details in the ordered retinal layer structure and guide layer boundaries along discontinuities in the RPE and BM in severe AMD cases. Additionally, a graph search algorithm refines boundary delineations from predicted segmentation maps. The model’s effectiveness is validated on the DUEIA and AROI datasets, which include OCT B-scans from both AMD-affected and healthy individuals. The proposed approach achieves Mean Absolute Errors (MAE) of 0.45 and 1.19 on the respective datasets, demonstrating its capability to handle boundary segmentation in severe AMD cases.