Journal of Pathology Informatics最新文献

A vast multitude of tasks in histopathology could potentially benefit from the support of artificial intelligence (AI). Many examples have been shown in the literature and first commercial products with FDA or CE-IVDR clearance are available. However, two key challenges remain: (1) a scarcity of thoroughly annotated images, respectively the laboriousness of this task, and (2) the creation of robust models that can cope with the data heterogeneity in the field (domain generalization). In this work, we investigate how the combination of prototypical few-shot classification models and data augmentation can address both of these challenges. Based on annotated data sets that include multiple centers, multiple scanners, and two tumor entities, we examine the robustness and the adaptability of few-shot classifiers in multiple scenarios. We demonstrate that data from one scanner and one site are sufficient to train robust few-shot classification models by applying domain-specific data augmentation. The models achieved classification performance of around 90% on a multiscanner and multicenter database, which is on par with the accuracy achieved on the primary single-center single-scanner data. Various convolutional neural network (CNN) architectures can be used for feature extraction in the few-shot model. A comparison of nine state-of-the-art architectures yielded that EfficientNet B0 provides the best trade-off between accuracy and inference time. The classification of prototypical few-shot models directly relies on class prototypes derived from example images of each class. Therefore, we investigated the influence of prototypes originating from images from different scanners and evaluated their performance also on the multiscanner database. Again, our few-shot model showed a stable performance with an average absolute deviation in accuracy compared to the primary prototypes of 1.8% points. Finally, we examined the adaptability to a new tumor entity: classification of tissue sections containing urothelial carcinoma into normal, tumor, and necrotic regions. Only three annotations per subclass (e.g., muscle and adipose tissue are subclasses of normal tissue) were provided to adapt the few-shot model, which obtained an overall accuracy of 93.6%. These results demonstrate that prototypical few-shot classification is an ideal technology for realizing an interactive AI authoring system as it only requires few annotations and can be adapted to new tasks without involving retraining of the underlying feature extraction CNN, which would in turn require a selection of hyper-parameters based on data science expert knowledge. Similarly, it can be regarded as a guided annotation system. To this end, we realized a workflow and user interface that targets non-technical users.

In digital pathology, whole-slide images (WSIs) are widely used for applications such as cancer diagnosis and prognosis prediction. Vision transformer (ViT) models have recently emerged as a promising method for encoding large regions of WSIs while preserving spatial relationships among patches. However, due to the large number of model parameters and limited labeled data, applying transformer models to WSIs remains challenging. In this study, we propose a pretext task to train the transformer model in a self-supervised manner. Our model, MaskHIT, uses the transformer output to reconstruct masked patches, measured by contrastive loss. We pre-trained MaskHIT model using over 7000 WSIs from TCGA and extensively evaluated its performance in multiple experiments, covering survival prediction, cancer subtype classification, and grade prediction tasks. Our experiments demonstrate that the pre-training procedure enables context-aware understanding of WSIs, facilitates the learning of representative histological features based on patch positions and visual patterns, and is essential for the ViT model to achieve optimal results on WSI-level tasks. The pre-trained MaskHIT surpasses various multiple instance learning approaches by 3% and 2% on survival prediction and cancer subtype classification tasks, and also outperforms recent state-of-the-art transformer-based methods. Finally, a comparison between the attention maps generated by the MaskHIT model with pathologist's annotations indicates that the model can accurately identify clinically relevant histological structures on the whole slide for each task.

Background

Kidney biopsy is the gold-standard for diagnosing medical renal diseases, but the accuracy of the diagnosis greatly depends on the quality of the biopsy specimen, particularly the amount of renal cortex obtained. Inadequate biopsies, characterized by insufficient cortex or predominant medulla, can lead to inconclusive or incorrect diagnoses, and repeat biopsy. Unfortunately, there has been a concerning increase in the rate of inadequate kidney biopsies, and not all medical centers have access to trained professionals who can assess biopsy adequacy in real time. In response to this challenge, we aimed to develop a machine learning model capable of assessing the percentage cortex of each biopsy pass using smartphone images of the kidney biopsy tissue at the time of biopsy.

Methods

747 kidney biopsy cores and corresponding smartphone macro images were collected from five unused deceased donor kidneys. Each core was imaged, formalin-fixed, sectioned, and stained with Periodic acid–Schiff (PAS) to determine cortex percentage. The fresh unfixed core images were captured using the macro camera on an iPhone 13 Pro. Two experienced renal pathologists independently reviewed the PAS-stained sections to determine the cortex percentage. For the purpose of this study, the biopsies with less than 30% cortex were labeled as inadequate, while those with 30% or more cortex were classified as adequate. The dataset was divided into training (n=643), validation (n=30), and test (n=74) sets. Preprocessing steps involved converting High-Efficiency Image Container iPhone format images to JPEG, normalization, and renal tissue segmentation using a U-Net deep learning model. Subsequently, a classification deep learning model was trained on the renal tissue region of interest and corresponding class label.

Results

The deep learning model achieved an accuracy of 85% on the training data. On the independent test dataset, the model exhibited an accuracy of 81%. For inadequate samples in the test dataset, the model showed a sensitivity of 71%, suggesting its capability to identify cases with inadequate cortical representation. The area under the receiver-operating curve (AUC-ROC) on the test dataset was 0.80.

Conclusion

We successfully developed and tested a machine learning model for classifying smartphone images of kidney biopsies as either adequate or inadequate, based on the amount of cortex determined by expert renal pathologists. The model's promising results suggest its potential as a smartphone application to assist real-time assessment of kidney biopsy tissue, particularly in settings with limited access to trained personnel. Further refinements and validations are warranted to optimize the model's performance.

Over the past decade, artificial intelligence (AI) methods in pathology have advanced substantially. However, integration into routine clinical practice has been slow due to numerous challenges, including technical and regulatory hurdles in translating research results into clinical diagnostic products and the lack of standardized interfaces.

The open and vendor-neutral EMPAIA initiative addresses these challenges. Here, we provide an overview of EMPAIA's achievements and lessons learned. EMPAIA integrates various stakeholders of the pathology AI ecosystem, i.e., pathologists, computer scientists, and industry. In close collaboration, we developed technical interoperability standards, recommendations for AI testing and product development, and explainability methods. We implemented the modular and open-source EMPAIA Platform and successfully integrated 14 AI-based image analysis apps from eight different vendors, demonstrating how different apps can use a single standardized interface. We prioritized requirements and evaluated the use of AI in real clinical settings with 14 different pathology laboratories in Europe and Asia. In addition to technical developments, we created a forum for all stakeholders to share information and experiences on digital pathology and AI. Commercial, clinical, and academic stakeholders can now adopt EMPAIA's common open-source interfaces, providing a unique opportunity for large-scale standardization and streamlining of processes.

Further efforts are needed to effectively and broadly establish AI assistance in routine laboratory use. To this end, a sustainable infrastructure, the non-profit association EMPAIA International, has been established to continue standardization and support broad implementation and advocacy for an AI-assisted digital pathology future.

Analysis of gene expression at the single-cell level could help predict the effectiveness of therapies in the field of chronic inflammatory diseases such as arthritis. Here, we demonstrate an adopted approach for processing images from the Slide-seq method. Using a puck, which consists of about 50,000 DNA barcode beads, an RNA sequence of a cell is to be read. The pucks are repeatedly brought into contact with liquids and then recorded with a conventional epifluorescence microscope. The image analysis initially consists of stitching the partial images of a sequence recording, registering images from different sequences, and finally reading out the bases. The new method enables the use of an inexpensive epifluorescence microscope instead of a confocal microscope.

Eye tracking has been used for decades in attempt to understand the cognitive processes of individuals. From memory access to problem-solving to decision-making, such insight has the potential to improve workflows and the education of students to become experts in relevant fields. Until recently, the traditional use of microscopes in pathology made eye tracking exceptionally difficult. However, the digital revolution of pathology from conventional microscopes to digital whole slide images allows for new research to be conducted and information to be learned with regards to pathologist visual search patterns and learning experiences. This has the promise to make pathology education more efficient and engaging, ultimately creating stronger and more proficient generations of pathologists to come. The goal of this review on eye tracking in pathology is to characterize and compare the visual search patterns of pathologists. The PubMed and Web of Science databases were searched using ‘pathology’ AND ‘eye tracking’ synonyms. A total of 22 relevant full-text articles published up to and including 2023 were identified and included in this review. Thematic analysis was conducted to organize each study into one or more of the 10 themes identified to characterize the visual search patterns of pathologists: (1) effect of experience, (2) fixations, (3) zooming, (4) panning, (5) saccades, (6) pupil diameter, (7) interpretation time, (8) strategies, (9) machine learning, and (10) education. Expert pathologists were found to have higher diagnostic accuracy, fewer fixations, and shorter interpretation times than pathologists with less experience. Further, literature on eye tracking in pathology indicates that there are several visual strategies for diagnostic interpretation of digital pathology images, but no evidence of a superior strategy exists. The educational implications of eye tracking in pathology have also been explored but the effect of teaching novices how to search as an expert remains unclear. In this article, the main challenges and prospects of eye tracking in pathology are briefly discussed along with their implications to the field.

Knee osteoarthritis (OA) is a prevalent condition causing significant disability, particularly among the elderly, necessitating advancements in diagnostic methodologies to facilitate early detection and treatment. Traditional OA diagnosis, relying on radiography and physical exams, faces limitations in accuracy and objectivity. This underscores the need for more advanced diagnostic methods, such as machine learning (ML) and deep learning (DL), to improve OA detection and classification. This research introduces a novel ensemble learning approach for image data feature extraction which ingeniously combines the strengths of 2 advanced (ML) models with a (DL) method to substantially improve the accuracy of OA detection from radiographic images. This innovative strategy aims to address the limitations of traditional diagnostic tools by leveraging the enhanced sensitivity and specificity of combined ML and DL models. The methodology deployed in this study encompasses the application of 10 ML models to a comprehensive publicly available Kaggle dataset with a total of 3615 samples of knee X-ray images. Through rigorous k-fold cross-validation and meticulous hyperparameter optimization, we also included evaluation metrics like accuracy, receiver operating characteristic, precision, recall, and F1-score to assess our models' performance effectively. The proposed novel CDK (convolutional neural network, decision tree, K-nearest classifier) ensemble approach for feature extraction is designed to synergize the predictive capabilities of individual models, thereby significantly improving the detection accuracy of OA indicators within radiographic images. We applied several ML and DL approaches to the newly created feature set to evaluate performance. The CDK ensemble model outperformed state-of-the-art studies with a high-performance score of 99.72% accuracy. This remarkable achievement underscores the model's exceptional capability in the early detection of OA, highlighting its superiority in comparison to existing methods.

The Gleason score is an important predictor of prognosis in prostate cancer. However, its subjective nature can result in over- or under-grading. Our objective was to train an artificial intelligence (AI)-based algorithm to grade prostate cancer in specimens from patients who underwent radical prostatectomy (RP) and to assess the correlation of AI-estimated proportions of different Gleason patterns with biochemical recurrence-free survival (RFS), metastasis-free survival (MFS), and overall survival (OS). Training and validation of algorithms for cancer detection and grading were completed with three large datasets containing a total of 580 whole-mount prostate slides from 191 RP patients at two centers and 6218 annotated needle biopsy slides from the publicly available Prostate Cancer Grading Assessment dataset. A cancer detection model was trained using MobileNetV3 on 0.5 mm × 0.5 mm cancer areas (tiles) captured at 10× magnification. For cancer grading, a Gleason pattern detector was trained on tiles using a ResNet50 convolutional neural network and a selective CutMix training strategy involving a mixture of real and artificial examples. This strategy resulted in improved model generalizability in the test set compared with three different control experiments when evaluated on both needle biopsy slides and whole-mount prostate slides from different centers. In an additional test cohort of RP patients who were clinically followed over 30 years, quantitative Gleason pattern AI estimates achieved concordance indexes of 0.69, 0.72, and 0.64 for predicting RFS, MFS, and OS times, outperforming the control experiments and International Society of Urological Pathology system (ISUP) grading by pathologists. Finally, unsupervised clustering of test RP patient specimens into low-, medium-, and high-risk groups based on AI-estimated proportions of each Gleason pattern resulted in significantly improved RFS and MFS stratification compared with ISUP grading. In summary, deep learning-based quantitative Gleason scoring using a selective CutMix training strategy may improve prognostication after prostate cancer surgery.

Background

Prostate cancer ranks as the most frequently diagnosed cancer in men in the USA, with significant mortality rates. Early detection is pivotal for optimal patient outcomes, providing increased treatment options and potentially less invasive interventions. There remain significant challenges in prostate cancer histopathology, including the potential for missed diagnoses due to pathologist variability and subjective interpretations.

Methods

To address these challenges, this study investigates the ability of artificial intelligence (AI) to enhance diagnostic accuracy. The Galen™ Prostate AI algorithm was validated on a cohort of Puerto Rican men to demonstrate its efficacy in cancer detection and Gleason grading. Subsequently, the AI algorithm was integrated into routine clinical practice during a 3-year period at a CLIA certified precision pathology laboratory.

Results

The Galen™ Prostate AI algorithm showed a 96.7% (95% CI 95.6–97.8) specificity and a 96.6% (95% CI 93.3–98.8) sensitivity for prostate cancer detection and 82.1% specificity (95% CI 73.9–88.5) and 81.1% sensitivity (95% CI 73.7–87.2) for distinction of Gleason Grade Group 1 from Grade Group 2+. The subsequent AI integration into routine clinical use examined prostate cancer diagnoses on >122,000 slides and 9200 cases over 3 years and had an overall AI Impact ™ factor of 1.8%.

Conclusions

The potential of AI to be a powerful, reliable, and effective diagnostic tool for pathologists is highlighted, while the AI Impact™ in a real-world setting demonstrates the ability of AI to standardize prostate cancer diagnosis at a high level of performance across pathologists.

Background

Currently, there is a paucity of guidelines relating to displays used for digital pathology making procurement decisions, and optimal display configuration, challenging.

Experience suggests pathologists have personal preferences for brightness when using a conventional microscope which we hypothesized could be used as a predictor for display setup.

Methods

We conducted an online survey across six NHS hospitals, totalling 108 practicing pathologists, to capture brightness adjustment habits on both microscopes and displays.

A convenience subsample of respondents was then invited to take part in a practical task to determine microscope brightness and display luminance preferences in the normal working environment. A novel adaptation for a lightmeter was developed to directly measure the light output from the microscope eyepiece.

Results

The survey (response rate 59% n=64) indicates 81% of respondents adjust the brightness on their microscope. In comparison, only 11% report adjusting their digital display. Display adjustments were more likely to be for visual comfort and ambient light compensation rather than for tissue factors, common for microscope adjustments. Part of this discrepancy relates to lack of knowledge of how to adjust displays and lack of guidance on whether this is safe; But, 66% felt that the ability to adjust the light on the display was important.

Twenty consultants took part in the practical brightness assessment. Light preferences on the microscope showed no correlation with display preferences, except where a pathologist has a markedly brighter microscope light preference. All of the preferences in this cohort were for a display luminance of <500 cd/m², with 90% preferring 350 cd/m² or less. There was no correlation between these preferences and the ambient lighting in the room.

Conclusions

We conclude that microscope preferences can only be used to predict display luminance requirements where the microscope is being used at very high brightness levels. A display capable of a brightness of 500 cd/m² should be suitable for almost all pathologists with 300 cd/m² suitable for the majority. Although display luminance is not frequently changed by users, the ability to do so was felt to be important by the majority of respondents.

Further work needs to be undertaken to establish the relationship between diagnostic performance, luminance preferences, and ambient lighting levels.