Purpose: Recent developments in computational pathology have been driven by advances in vision foundation models (VFMs), particularly the Segment Anything Model (SAM). This model facilitates nuclei segmentation through two primary methods: prompt-based zero-shot segmentation and the use of cell-specific SAM models for direct segmentation. These approaches enable effective segmentation across a range of nuclei and cells. However, general VFMs often face challenges with fine-grained semantic segmentation, such as identifying specific nuclei subtypes or particular cells.
Approach: In this paper, we propose the molecular empowered all-in-SAM model to advance computational pathology by leveraging the capabilities of VFMs. This model incorporates a full-stack approach, focusing on (1) annotation-engaging lay annotators through molecular empowered learning to reduce the need for detailed pixel-level annotations, (2) learning-adapting the SAM model to emphasize specific semantics, which utilizes its strong generalizability with SAM adapter, and (3) refinement-enhancing segmentation accuracy by integrating molecular oriented corrective learning.
Results: Experimental results from both in-house and public datasets show that the all-in-SAM model significantly improves cell classification performance, even when faced with varying annotation quality.
Conclusions: Our approach not only reduces the workload for annotators but also extends the accessibility of precise biomedical image analysis to resource-limited settings, thereby advancing medical diagnostics and automating pathology image analysis.
Purpose: Intraoperative liver deformation and the need to glance repeatedly between the operative field and a remote monitor undermine the precision and workflow of image-guided liver surgery. Existing mixed reality (MR) prototypes address only isolated aspects of this challenge and lack quantitative validation in deformable anatomy.
Approach: We introduce a fully self-contained MR navigation system for liver surgery that runs on a MR headset and bridges this clinical gap by (1) stabilizing holographic content with an external retro-reflective reference tool that defines a fixed world origin, (2) tracking instruments and surface points in real time with the headset's depth camera, and (3) compensating soft-tissue deformation through a weighted ICP + linearized iterative boundary reconstruction pipeline. A lightweight server-client architecture streams deformation-corrected 3D models to the headset and enables hands-free control via voice commands.
Results: Validation on a multistate liver-phantom protocol demonstrated that the reference tool reduced mean hologram drift from to and improved tracking accuracy from to . Across five simulated deformation states, nonrigid registration lowered surface target registration error from to -an average 57% error reduction-yielding sub-4 mm guidance accuracy.
Conclusions: By unifying stable MR visualization, tool tracking, and biomechanical deformation correction in a single headset, the proposed platform eliminates monitor-related context switching and restores spatial fidelity lost to liver motion. The device-agnostic framework is extendable to open approaches and potentially laparoscopic workflows and other soft-tissue interventions, marking a significant step toward MR-enabled surgical navigation.
Purpose: We aim to propose a reliable registration pipeline tailored for multimodal mouse bone imaging using X-ray microscopy (XRM) and light-sheet fluorescence microscopy (LSFM). These imaging modalities have emerged as pivotal tools in preclinical research, particularly for studying bone remodeling diseases such as osteoporosis. Although multimodal registration enables micrometer-level structural correspondence and facilitates functional analysis, conventional landmark-, feature-, or intensity-based approaches are often infeasible due to inconsistent signal characteristics and significant misalignment resulting from independent scanning, especially in real-world and reference-free scenarios.
Approach: To address these challenges, we introduce BigReg, an automatic, two-stage registration pipeline optimized for high-resolution XRM and LSFM volumes. The first stage involves extracting surface features and applying two successive global-to-local point-cloud-based methods for coarse alignment. The subsequent stage refines this alignment in the 3D Fourier domain using a modified cross-correlation technique, achieving precise volumetric registration.
Results: Evaluations using expert-annotated landmarks and augmented test data demonstrate that BigReg approaches the accuracy of landmark-based registration with a landmark distance (LMD) of and a landmark fitness (LM fitness) of . Moreover, BigReg can provide an optimal initialization for mutual information-based methods that otherwise fail independently, further reducing LMD to and increasing LM fitness to .
Conclusions: To the best of our knowledge, BigReg is the first automated method to successfully register XRM and LSFM volumes without requiring manual intervention or prior alignment cues. Its ability to accurately align fine-scale structures, such as lacunae in XRM and osteocytes in LSFM, opens up new avenues for quantitative, multimodal analysis of bone microarchitecture and disease pathology, particularly in studies of osteoporosis.
Purpose: The Bayesian ideal observer (IO) is a special model observer that achieves the best possible performance on tasks that involve signal detection or discrimination. Although IOs are desired for optimizing and assessing imaging technologies, they remain difficult to compute. Previously, a hybrid method that combines deep learning (DL) with a Markov-Chain Monte Carlo (MCMC) method was proposed for estimating the IO test statistic for joint signal detection-estimation tasks. That method will be referred to as the hybrid MCMC method. However, the hybrid MCMC method was restricted to use cases that involved relatively simple stochastic background and signal models.
Approach: The previously developed hybrid MCMC method is generalized by utilizing a framework that integrates deep generative modeling into the MCMC sampling process. This method employs a generative adversarial network (GAN) that is trained on object or signal ensembles to establish data-driven stochastic object and signal models, respectively, and will be referred to as the hybrid MCMC-GAN method. This circumvents the limitation of traditional MCMC methods and enables the estimation of the IO test statistic with consideration of broader classes of clinically relevant object and signal models.
Results: The hybrid MCMC-GAN method was evaluated on two binary detection-estimation tasks in which the observer must detect a signal and estimate its amplitude if the signal is detected. First, a stylized signal-known-statistically (SKS) and background-known-exactly task was considered. A GAN was employed to establish a stochastic signal model, enabling direct comparison of our GAN-based IO approximation with a closed-form expression for the IO decision strategy. The results confirmed that the proposed method could accurately approximate the performance of the true IO. Next, an SKS and background-known-statistically (BKS) task was considered. Here, a GAN was employed to establish a stochastic object model that described anatomical variability in an ensemble of magnetic resonance (MR) brain images. This represented a setting where traditional MCMC methods are inapplicable. In this study, although a reference estimate of the true IO performance was unavailable, the hybrid MCMC-GAN produced area under the estimation receiver operating characteristic curve (AEROC) estimates that exceeded those of a sub-ideal observer that represented a lower bound for the IO performance.
Conclusion: By combining GAN-based generative modeling with MCMC, the hybrid MCMC-GAN method extends a previously proposed IO approximation method to more general detection-estimation tasks. This provides a new capability to benchmark and optimize imaging-system performance through virtual imaging studies.
The editorial introduces JMI Volume 12 Issue 5, as it marks the beginning of a new academic year with a celebration of innovation, mentorship, and the evolving role of scholarly publishing in medical imaging. It emphasizes that impactful research is not just about results, but about teaching, sharing insights, and advancing the field through curiosity, collaboration, and community-driven resources.
Purpose: The accurate assessment of computed tomography (CT) image quality is crucial for ensuring diagnostic reliability while minimizing radiation dose. Radiologists' evaluations are time-consuming and labor-intensive. Existing automated approaches often require large CT datasets with predefined image quality assessment (IQA) scores, which often do not align well with clinical evaluations. We aim to develop a reference-free, automated method for CT IQA that closely reflects radiologists' evaluations, reducing the dependency on large annotated datasets.
Approach: We propose Task-Focused Knowledge Transfer (TFKT), a deep learning-based IQA method leveraging knowledge transfer from task-similar natural image datasets. TFKT incorporates a hybrid convolutional neural network-transformer model, enabling accurate quality predictions by learning from natural image distortions with human-annotated mean opinion scores. The model is pre-trained on natural image datasets and fine-tuned on low-dose computed tomography perceptual image quality assessment data to ensure task-specific adaptability.
Results: Extensive evaluations demonstrate that the proposed TFKT method effectively predicts IQA scores aligned with radiologists' assessments on in-domain datasets and generalizes well to out-of-domain clinical pediatric CT exams. The model achieves robust performance without requiring high-dose reference images. Our model is capable of assessing the quality of CT image slices in a second.
Conclusions: The proposed TFKT approach provides a scalable, accurate, and reference-free solution for CT IQA. The model bridges the gap between traditional and deep learning-based IQA, offering clinically relevant and computationally efficient assessments applicable to real-world clinical settings.
Purpose: Our goal was to develop a deep network for whole-head segmentation, including clinical magnetic resonance imaging (MRI) with abnormal anatomy, and compile the first public benchmark dataset for this purpose. We collected 98 MRIs with volumetric segmentation labels for a diverse set of human subjects, including normal and abnormal anatomy in clinical cases of stroke and disorders of consciousness.
Approach: Training labels were generated by manually correcting initial automated segmentations for skin/scalp, skull, cerebro-spinal fluid, gray matter, white matter, air cavity, and extracephalic air. We developed a "MultiAxial" network consisting of three 2D U-Net that operate independently in sagittal, axial, and coronal planes, which are then combined to produce a single 3D segmentation.
Results: The MultiAxial network achieved a test-set Dice scores of (median ± interquartile range) on whole-head segmentation, including gray and white matter. This was compared with for Multipriors and for SPM12, two standard tools currently available for this task. The MultiAxial network gains in robustness by avoiding the need for coregistration with an atlas. It performed well in regions with abnormal anatomy and on images that have been de-identified. It enables more accurate and robust current flow modeling when incorporated into ROAST, a widely used modeling toolbox for transcranial electric stimulation.
Conclusions: We are releasing a new state-of-the-art tool for whole-head MRI segmentation in abnormal anatomy, along with the largest volume of labeled clinical head MRIs, including labels for nonbrain structures. Together, the model and data may serve as a benchmark for future efforts.
Purpose: Medical implants, often made of dense materials, pose significant challenges to accurate computed tomography (CT) reconstruction, especially near implants due to beam hardening and partial-volume artifacts. Moreover, diagnostics involving implants often require separate visualization for implants and anatomy. In this work, we propose a approach for joint estimation of anatomy and implants as separate volumes using a mixed prior model.
Approach: We leverage a learning-based prior for anatomy and a sparsity prior for implants to decouple the two volumes. In addition, a hybrid mono-polyenergetic forward model is employed to accommodate the spectral effects of implants, and a multiresolution object model is used to achieve high-resolution implant reconstruction. The reconstruction process alternates between diffusion posterior sampling for anatomy updates and classic optimization for implants and spectral coefficients.
Results: Evaluations were performed on emulated cardiac imaging with stent and spine imaging with pedicle screws. The structures of the cardiac stent with 0.25 mm wires were clearly visualized in the implant images, whereas the blooming artifacts around the stent were effectively suppressed in the anatomical reconstruction. For pedicle screws, the proposed algorithm mitigated streaking and beam-hardening artifacts in the anatomy volume, demonstrating significant improvements in SSIM and PSNR compared with frequency-splitting metal artifact reduction and model-based reconstruction on slices containing implants.
Conclusion: The proposed mixed prior model coupled with a hybrid spectral and multiresolution model can help to separate spatially and spectrally distinct objects that differ from anatomical features in single-energy CT, improving both image quality and separate visualization of implants and anatomy.
Purpose: Many studies caution against using radiomic features that are sensitive to contouring variability in predictive models for disease stratification. Consequently, metrics such as the intraclass correlation coefficient (ICC) are recommended to guide feature selection based on stability. However, the direct impact of segmentation variability on the performance of predictive models remains underexplored. We examine how segmentation variability affects both feature stability and predictive performance in the radiomics-based classification of triple-negative breast cancer (TNBC) using breast magnetic resonance imaging.
Approach: We analyzed 244 images from the Duke dataset, introducing segmentation variability through controlled modifications of manual segmentations. For each segmentation mask, explainable radiomic features were selected using Shapley Additive exPlanations and used to train logistic regression models. Feature stability across segmentations was assessed via ICC, Pearson's correlation, and reliability scores quantifying the relationship between segmentation variability and feature robustness.
Results: Model performances in predicting TNBC do not exhibit a significant difference across varying segmentations. The most explicative and predictive features exhibit decreasing ICC as segmentation accuracy decreases. However, their predictive power remains intact due to low ICC combined with high Pearson's correlation. No shared numerical relationship is found between feature stability and segmentation variability among the most predictive features.
Conclusions: Moderate segmentation variability has a limited impact on model performance. Although incorporating peritumoral information may reduce feature reproducibility, it does not compromise predictive utility. Notably, feature stability is not a strict prerequisite for predictive relevance, highlighting that exclusive reliance on ICC or stability metrics for feature selection may inadvertently discard informative features.
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：