International Journal of Computer Assisted Radiology and Surgery最新文献

Purpose: Estimating the 6 degrees of freedom (DoF) pose of an endoscope is crucial for various applications in minimally invasive computer-assisted surgery. Image-based approaches are some of the most practical solutions for pose estimation in surgical environments, due to a limited workspace and sensor constraints. However, these methods often struggle or fail in dynamic scenes, such as those involving tissue deformation, surgical tool movement, and tool-tissue interaction.

Methods: We propose DyEndoVO, an end-to-end visual odometry method in dynamic endoscopic scenes. Our method consists of a transformer-based motion detection network and a weighted pose-optimization module. The motion detection network infers scene dynamics and guides the pose estimation. Furthermore, we introduce a semi-synthetic dataset featuring tissue and tool movement categories. It serves as training data, improving pose estimation accuracy, and also includes motion masks to enable a fine-grained inspection and evaluation.

Results: DyEndoVO significantly outperforms state-of-the-art methods in pose estimation for dynamic surgical scenes. Despite being trained solely on a synthetic dataset, our method generalizes well to real-world data without fine-tuning. Further analysis attributes this success to the effective detection of scene dynamics and the adaptation in the learned weight toward pose estimation; moreover, the semi-synthetic dataset also plays a key role in bridging the sim-to-real gap.

Conclusions: In this work, we aim to improve the accuracy and robustness of pose estimation in challenging dynamic surgical scenes, by effectively handling scene dynamics. Our method, combined with the proposed synthetic dataset, demonstrates improved performance in pose estimation and generalizes well to real-world data, showing its potential in advancing related works such as SLAM and 3D reconstruction in complex surgical environments.

Purpose: Deep learning models for endoscopic assistance applications predominantly focus on image-based tasks, such as tool detection, anatomical classification, and workflow segmentation. However, these approaches often neglect the integration of natural language, limiting their assistance capabilities. This work adopts a proven architecture for vision-language models (VLM) to perform multi-task learning for image classification, text prediction, and surgical report generation, specifically for endoscopic ENT surgeries.

Methods: We adopted a VLM architecture utilizing encoders biased for the endoscopy domain for image and text embedding and combine them via cross-attention. The model was trained on a newly created multi-task dataset derived from 30 annotated endoscopic procedures, comprising 130,000 multi-label images, anatomical descriptions, and synchronized surgical reports. Two variations of the model, a lightweight 61M parameter and a 176M parameter model, were evaluated both against an existing baseline from previous mono-task studies as well as the EndoVit and SurgicalGPT models as external references. Ablation studies investigate the influence of removing image or text embeddings, and cross-attention on the task performance. Performance was measured for landmark classification, structured text prediction, and report generation using precision, recall, F1-score, BLEU-2, ROUGE-L, and cosine similarity metrics.

Results: The VLM base model improves the baseline f1 score for image classification by up to 12% and natural language text generation by up to 14% across image classification and report generation tasks. The text generation of structured language tasks, however, showed minimal gains, indicating limitations in structured sentence learning from combined image-text embeddings. EndoViT and SurgicalGPT slightly trail our domain-specific VLM. The image-only and text-only ablations confirm that the vision component benefits language tasks, whereas text has limited impact on landmark detection.

Conclusion: We developed a vision-language model capable of integrating image and text data for endoscopic ENT assistance tasks, that is able to replace three isolated models, delivering multi-task assistance while outperforming prior and general‑purpose baselines. Remaining challenges include the handling of imbalanced class distributions and limited gains on templated structured text.

Purpose: Achieving a prosthetic femoral version (PFV) within the target range of 10-20° is crucial for optimal biomechanics in total hip arthroplasty (THA). Predicting the PFV preoperatively is challenging due to the limited understanding of the relationship between native femoral version (NFV) and the morphology of the intramedullary canal. This study aims to quantify the 3D morphological variability and identify the most variable anatomical features of the proximal femur pre- and post-operatively.

Methods: Pre- and post-operative CT scans from 62 patients (31 males, 31 females) who underwent THA and received a single stem design (straight, triple-tapered) were analysed. Four femoral models were generated per patient: 1. Native proximal femur, 2. Native femur after neck osteotomy, 3. Internal femoral canal after neck osteotomy, and 4. Reconstructed femur. Statistical Shape Models (SSMs) were developed separately by sex, and principal component analysis (PCA) was used to identify dominant modes of anatomical variation.

Results: The first three principal components (PCs) accounted for over 60% of shape variability across all models. PFV showed weak correlation with NFV as variability existed between the SSM of the internal femoral canal and SSM of the native proximal femur. Sex-specific differences in the measured NFV and PFV were found, with females exhibiting a greater range and a more anteverted femur/femoral stem. The female canal model showed intramedullary version variability; however, this variability was not present in the first three PCs in the corresponding male model.

Conclusions: This study demonstrates that PFV cannot be reliably predicted from NFV alone. These findings underscore the need for advanced, 3D preoperative planning tools to better predict stem version and accommodate patient-specific anatomy. Additionally, the increased variability observed in females may warrant sex-specific consideration in implant design choice and surgical technique.

Purpose: Oropharyngeal dysphagia affects up to half of head and neck cancer (HNC) patients. Multi-swallow video-fluoroscopic swallow studies (VFSS) combined with high-resolution impedance manometry (HRIM) offer a comprehensive assessment of swallowing function. However, their use in HNC populations is limited by high clinical workload and complexity of data collection and analysis with existing software.

Methods: To address the data collection challenge, we propose a framework for automatic swallow detection in simultaneous VFSS-HRIM examinations. The framework identifies candidate swallow intervals in continuous VFSS videos using an optimized double-sweep optical flow algorithm. Each candidate interval is then classified using a pressure-based swallow template derived from three annotated samples, leveraging features such as normalized peak-to-peak amplitude, mean, and standard deviation from upper esophageal sphincter sensors.

Results: The methodology was evaluated on 97 swallows from twelve post-head and neck cancer patients. The detection pipeline achieved 95% Recall and 92% F1-score. Importantly, the number of required HRIM annotations was reduced by 63%, substantially decreasing clinician workload while maintaining high accuracy.

Conclusion: This framework overcomes limitations of current software for simultaneous VFSS-HRIM collection by enabling high-accuracy, low-input swallow detection in HNC patients. Validated on a heterogeneous patient cohort, it initiates the groundwork for scalable, objective, and multimodal swallowing assessment.

Purpose: Domain generalization plays a crucial role in analyzing medical images from diverse clinics, scanner vendors, and imaging modalities. Existing methods often require substantial computational resources to train a highly generalized segmentation network, presenting challenges in terms of both availability and cost. The goal of this work is to evaluate a novel, yet simple and effective method for enhancing the generalization of deep learning models in segmentation across varying modalities.

Methods: Eight augmentation methods will be applied individually to a source domain dataset in order to generalize deep learning models. These models will then be tested on completely unseen target domain datasets from a different imaging modality and compared against a lower baseline model. By leveraging standard augmentation techniques, extensive intensity augmentations, and carefully chosen color transformations, we aim to address the domain shift problem, particularly in the cross-modality setting.

Results: Our novel CmapAug method, when combined with standard augmentation techniques, resulted in a substantial improvement in the Dice Score, outperforming the baseline. While the baseline struggled to segment the liver structure in some test cases, our selective combination of augmentation methods achieved Dice scores as high as 83.2%.

Conclusion: Our results highlight the general effectiveness of the tested augmentation methods in addressing domain generalization and mitigating the domain shift problem caused by differences in imaging modalities between the source and target domains. The proposed augmentation strategy offers a simple yet powerful solution to this challenge, with significant potential in clinical scenarios where annotated data from the target domain are limited or unavailable.

Purpose: Robotic systems for catheter ablation have been in clinical use for many years. While their impact on the clinical outcome and procedure times is well studied, aspects like usability and operator workload have received limited attention in the literature. Reduced workload and stress levels benefit the operator's mental and physical health, and can also lower the risk of errors and ultimately improve patient safety. The aim of this study is to investigate the workload and usability of remote magnetic navigation compared to conventional manual navigation.

Methods: We performed a user study with eight electrophysiologists. Each participant performed identical in-vitro navigation tasks replicating those found in pulmonary vein isolation using both manual and magnetic navigation. Magnetic navigation experiments were performed using the Navion, a mobile electromagnetic navigation system.

Results: Magnetic navigation significantly improved usability (p < 0.02) and workload (p < 0.01) compared to manual navigation, measured using the System Usability Scale (magnetic: 85.6 ± 9.3 vs. manual: 75.0 ± 17.8) and NASA Task Load Index (magnetic: 72.4 ± 13.5 vs. manual: 45.8 ± 16.7). Additionally, task completion times were shorter (p < 0.01) with magnetic navigation (284.6 ± 80.7 s) compared to manual navigation (411.0 ± 123.7 s).

Conclusion: The findings of this study suggest that remote magnetic navigation using the Navion significantly improves operator experiences in terms of workload and usability, reinforcing the case for wider adoption of well-designed robotic systems in cardiac electrophysiology labs.

Purpose: Shape modeling of volumetric medical images plays a crucial role in quantitative analysis and surgical planning for computer-aided diagnosis. However, automatic shape reconstruction from deep learning models often suffers from limited image resolution and the lack of shape prior constraints. This study aims to address these challenges by developing a method that enables reliable and accurate anatomical shape modeling in the continuous space.

Methods: We present the Reliable Shape Interaction with Implicit Template (ReShapeIT) network, which represents anatomical structures using continuous implicit fields rather than discrete voxel grids. The approach combines a category-specific implicit template field with a deformation network to encode anatomical shapes from training shapes. In addition, a Template Interaction Module (TIM) is designed to refine test cases by aligning learned template shapes with instance-specific latent codes.

Results: We evaluated ReShapeIT on three anatomical datasets-Liver, Pancreas, and Lung Lobe. The proposed method outperforms state-of-the-art approaches in 3D shape reconstruction, achieving Chamfer Distance/Earth Mover's Distance scores of 0.225/0.318 for Liver, 0.125/0.067 for Pancreas, and 0.414/0.098 for Lung Lobe.

Conclusion: ReShapeIT provides a reliable and generalizable solution for implicit anatomical shape modeling by leveraging shared template priors and instance-level deformations. The implementation is publicly available at: https://github.com/EndoluminalSurgicalVision-IMR/ReShapeIT .

Purpose: Accurate prediction of treatment outcomes is crucial for personalized treatment in head and neck squamous cell carcinoma (HNSCC). Beyond one-year survival, assessing long-term enteral nutrition dependence is essential for optimizing patient counseling and resource allocation. This preliminary study aimed to predict one-year survival and feeding tube dependence in surgically treated HNSCC patients using classical machine learning.

Methods: This proof-of-principle retrospective study included 558 surgically treated HNSCC patients. Baseline clinical data, routine blood markers, and MRI-based radiomic features were collected before treatment. Additional postsurgical treatments within one year were also recorded. Random forest classifiers were trained to predict one-year survival and feeding tube dependence. Model explainability was assessed using Shapley Additive exPlanation (SHAP) values.

Results: Using tenfold stratified cross-validation, clinical data showed the highest predictive performance for survival (AUC = 0.75 ± 0.10; p < 0.001). Blood (AUC = 0.67 ± 0.17; p = 0.001) and imaging (AUC = 0.68 ± 0.16; p = 0.26) showed moderate performance, and multimodal integration did not improve predictions (AUC = 0.68 ± 0.16; p = 0.38). For feeding tube dependence, all modalities had low predictive power (AUC ≤ 0.66; p > 0.05). However, postsurgical treatment information outperformed all other modalities (AUC = 0.67 ± 0.07; p = 0.002), but had the lowest predictive value for survival (AUC = 0.57 ± 0.11; p = 0.08).

Conclusion: Clinical data appeared to be the strongest predictor of one-year survival in surgically treated HNSCC, although overall predictive performance was moderate. Postsurgical treatment information played a key role in predicting tube feeding dependence. While multimodal integration did not enhance overall model performance, it showed modest gains for weaker individual modalities, suggesting potential complementarity that warrants further investigation.

Purpose: Accurate prediction of overall survival (OS) in glioblastoma patients is critical for advancing personalized treatments and improving clinical trial design. Conventional radiomics approaches rely on manually engineered features, which limit their ability to capture complex, high-dimensional imaging patterns. This study employs a deep learning architecture to process MRI data for automated glioma segmentation and feature extraction, leveraging high-level representations from the encoder's latent space.

Methods: Multimodal MRI data from the BraTS2020 dataset and a proprietary dataset from Fondazione IRCCS Istituto Neurologico Carlo Besta (Milan, Italy) were processed independently using a U-Net-like model pre-trained on BraTS2018 and fine-tuned on BraTS2020. Features extracted from the encoder's latent space represented hierarchical imaging patterns. These features were combined with clinical variable (patient's age) and reduced via principal component analysis (PCA) to enhance computational efficiency. Machine learning classifiers-including random forest, XGBoost, and a fully connected neural network-were trained on the reduced feature vectors for OS classification.

Results: In the four-modality BraTS4CH setting, the multi-layer perceptron achieved the best performance (F1 = 0.71, AUC = 0.74, accuracy = 0.71). When limited to two modalities on BraTS2020 (BraTS2CH), MLP again led (F1 = 0.67, AUC = 0.70, accuracy = 0.67). On the IRCCS Besta two-modality cohort (Besta2CH), XGBoost produced the highest F1-score and accuracy (F1 = 0.65, accuracy = 0.66), while MLP obtained the top AUC (0.70). These results are competitive with-and in some metrics exceed-state-of-the-art reports, demonstrating the robustness and scalability of our automated framework relative to traditional radiomics and AI-driven approaches.

Conclusion: Integrating encoder-derived features from multimodal MRI data with clinical variables offers a scalable and effective approach for OS prediction in glioblastoma patients. This study demonstrates the potential of deep learning to address traditional radiomics limitations, paving the way for more precise and personalized prognostic tools.

Purpose: Minimally invasive surgical approaches are currently the standard of care for men with prostate cancer, presenting higher rates of erectile function preservation. With these laparoscopic techniques, there is an increasing amount of data and information available. Adaptive systems can play an important role, acting as an intelligent information filter, assuring that all the available information can become useful for the procedure and not overwhelming for the surgeon. Standardizing and structuring the surgical workflow are key requirements for such smart assistants to recognize the different surgical steps through context information about the environment. This work aims to do a detailed characterization of a laparoscopic radical prostatectomy procedure, focusing on the formalization of medical expert knowledge, via surgical process modeling.

Methods: Data were acquired manually, via online and offline observation, and discussion with medical experts. A total of 14 procedures were observed. Both manual laparoscopic radical prostatectomy and robot-assisted laparoscopic prostatectomy were studied. The derived SPM focuses only on the intraoperatory part of the procedure, with constant feedback from the endoscopic camera. For surgery observation, a dedicated Excel template was developed.

Results: The final model is represented in a descriptive and numerical format, combining task description with a workflow diagram arrangement for ease of interpretation. Practical applications of the generated surgical process model are exemplified with the creation of activation trees for surgical phase identification. Anatomical structures are reported for each phase, distinguishing between visible and inferable ones. Additionally, the surgeons involved are identified, surgical instruments, and actions performed in each phase. A total of 11 phases were identified and characterized. Average surgery duration is 87 min.

Conclusion: The generated surgical process model is a first step toward the development of a context-aware surgical assistant and can potentially be used as a roadmap by other research teams, operating room managers and surgical teams.