International Journal of Computer Assisted Radiology and Surgery最新文献

Purpose Surgical gestures are fundamental components of surgical procedures, encompassing actions such as cutting, suturing, and knot-tying. Gesture recognition plays a pivotal role in the automatic analysis of surgical data. Although recent advancements have improved surgical gesture recognition, much of the existing research relies on simulations or minimally invasive surgery data, failing to capture the complexities of open surgery. In this study, we introduce and employ a new open surgery dataset focused on closing incisions after saphenous vein harvesting. Methods Our goal is to improve gesture recognition accuracy by incorporating tool pose estimation and 3D hand pose predictions of surgeons. We employ MS-TCN++  and LTContext  for gesture recognition, and further enhance performance through an ensemble of models using different modalities-video, tool pose, and hand pose data.Results The results reveal that using an ensemble model combining all three modalities provides a substantial improvement over video-only approaches, leading to statistically significant gains across multiple evaluation metrics. We further demonstrate that the model can rely solely on hand and tool poses, completely discarding the video input, while still achieving comparable performance. Additionally, we show that an ensemble model using only hand and tool poses produces results that are either: statistically significantly better than using video alone, or not statistically significantly different.Conclusion This study highlights the effectiveness of integrating multimodal data for surgical gesture recognition. By combining video, hand pose, and tool pose information, our approach achieves higher accuracy and robustness compared to video-only methods. Moreover, the comparable performance of pose-only models suggests a promising, privacy-preserving alternative for surgical data analysis.

Purpose: To discuss the economic and environmental implications of implementing large language models (LLMs) in radiology, highlighting both their transformative potential and the challenges they pose for equitable and sustainable adoption.

Methods: Current trends in AI investment, infrastructure requirements, operational costs, and environmental impact associated with LLMs are analyzed, highlighting the specific challenges of integrating LLMs into radiological workflows, including data privacy, regulatory compliance, and cost barriers for healthcare institutions. The analysis also considers the costs of model validation, maintenance, and updates, as well as investments in system integration, staff training, and cybersecurity for clinical implementation.

Results: LLMs have revolutionized natural language processing and offer promising applications in radiology, such as improved diagnostic support and workflow optimization. However, their deployment involves substantial financial and environmental costs. Training and operating these models require high-performance computing infrastructure, significant energy consumption, and large volumes of annotated data. Water usage and CO₂ emissions from data centers further raise sustainability concerns, while ongoing operational costs add to the financial burden. Subscription fees and per-query pricing may restrict access for smaller institutions, widening existing inequalities.

Conclusion: While LLMs offer significant benefits for radiology, their high economic and environmental costs present challenges to widespread and equitable adoption. Responsible use, sustainable practices, and policy frameworks are essential to ensure that AI-driven innovations do not exacerbate existing disparities in healthcare access and quality.

Purpose: Anatomical identification during abdominal surgery is subjective given unclear boundaries of anatomical structures. Semantic segmentation of these structures relies on an accurate identification of the boundaries which carries an unknown uncertainty. Given its inherent subjectivity, it is important to assess annotation adequacy. This study aims to evaluate variability in anatomical structure identification and segmentation using MedSAM by surgical residents.

Methods: Images from the Dresden Surgical Anatomy Dataset and the Endoscapes2023 Dataset were semantically annotated by a group of surgery residents using MedSAM in the following classes: abdominal wall, colon, liver, small bowel, spleen, stomach and gallbladder. Each class had 3 to 4 sets of annotations. Inter-annotator variability was assessed through DSC, ICC, BIoU and using the Simultaneous Truth and Performance Level Estimation algorithm to obtain a consensus mask and by calculating Fleiss' kappa agreement between all annotations and reference.

Results: The study showed strong inter-annotator agreement among surgical residents, with DSC values of 0.84-0.95 and Fleiss' kappa between 0.85 and 0.91. Surface area reliability was good to excellent (ICC = 0.62-0.91), while boundary delineation showed lower reproducibility (BIoU = 0.092-0.157). STAPLE consensus masks confirmed consistent overall shape annotations despite variability in boundary precision.

Conclusion: The study demonstrated low variability in the semantic segmentation of intraperitoneal organs in minimally invasive abdominal surgery, performed by surgical residents using MedSAM. While DSC and Fleiss' kappa values confirm strong inter-annotator agreement, the relatively low BIoU values point to challenges in boundary precision, especially for anatomically complex or variable structures. These results establish a benchmark for expanding annotation efforts to larger datasets and more detailed anatomical features.

Purpose: Computer-assisted surgical navigation systems have been developed to improve the precision of total knee arthroplasty (TKA) by providing real-time guidance on implant alignment relative to patient anatomy. However, surface registration remains a key source of error that can propagate through the surgical workflow. This study investigates how patient-specific femoral bone geometry influences registration accuracy, aiming to enhance the reliability and consistency of computer-assisted orthopedic procedures.

Methods: Eighteen high-fidelity 3D-printed femur models were used to simulate intraoperative digitization. Surface points collected from the distal femur were registered to preoperative CT-derived models using a rigid iterative closest point (ICP) algorithm. Registration accuracy was quantified across six degrees of freedom. An in-house statistical shape model (SSM), built from 114 CT femurs, was employed to extract shape coefficients and correlate them with the measured registration errors. To verify robustness, additional analyses were conducted using synthetic and in silico CT-based femur datasets.

Results: Significant correlations (p-values < 0.05) were observed between specific shape coefficients and registration errors. The third and fourth principal shape modes showed the strongest associations with rotational misalignments, particularly flexion-extension and varus-valgus components. These findings demonstrate that geometric variability in the distal femur, especially condylar morphology, plays a major role in determining the stability and accuracy of surface-based registration.

Conclusions: Registration errors in TKA are strongly influenced by patient-specific bone geometry. Shape features derived from statistical shape models can serve as reliable predictors of registration performance, providing quantitative insight into how anatomical variability impacts surgical precision and alignment accuracy in computer-assisted total knee arthroplasty.

Purpose: Scan path planning is an essential technology for fully automated robotic ultrasound (US). During US screening in head and neck lesions, it is necessary to observe the entire neck, including the thyroid, parotid, submandibular glands, and cervical lymph nodes efficiently. This study developed a personalized scan path planning for robotic US in head and neck lesions.

Methods: We proposed the personalized scan path planning that can estimate the scan position by applying the non-rigid registration between the patient neck image and the similar neck image among the previous patients. To select the most similar neck image in the database, the unique similarity index with the combination of multiple similarity metrics (neck area, position of the centroid, SSIM, Dice coefficient, and contours of the neck area) was developed. Fifty neck images were used to determine the optimal combination experimentally. The non-rigid registration from each of the 50 images to the other 49 images, resulting in a total of 2,450 registration pairs, was conducted, and the correlations between each similarity metric and registration error were analyzed.

Results: The best-performing similarity index achieved a correlation coefficient of 0.730 between the similarity score and registration error, which shows that the proposed index can select the similar image with a high accuracy of scan path registration. With the proposed scan path planning, the feasibility study is performed on six healthy subjects. The results showed a success rate of 94.4% for obtaining the specific target images required for the neck and head US examinations.

Conclusion: The proposed scan path planning allows the scan to adapt to individual differences in the neck, which can contribute to the automated robotic US in head and neck lesions.

Purpose: Estimating the 6 degrees of freedom (DoF) pose of an endoscope is crucial for various applications in minimally invasive computer-assisted surgery. Image-based approaches are some of the most practical solutions for pose estimation in surgical environments, due to a limited workspace and sensor constraints. However, these methods often struggle or fail in dynamic scenes, such as those involving tissue deformation, surgical tool movement, and tool-tissue interaction.

Methods: We propose DyEndoVO, an end-to-end visual odometry method in dynamic endoscopic scenes. Our method consists of a transformer-based motion detection network and a weighted pose-optimization module. The motion detection network infers scene dynamics and guides the pose estimation. Furthermore, we introduce a semi-synthetic dataset featuring tissue and tool movement categories. It serves as training data, improving pose estimation accuracy, and also includes motion masks to enable a fine-grained inspection and evaluation.

Results: DyEndoVO significantly outperforms state-of-the-art methods in pose estimation for dynamic surgical scenes. Despite being trained solely on a synthetic dataset, our method generalizes well to real-world data without fine-tuning. Further analysis attributes this success to the effective detection of scene dynamics and the adaptation in the learned weight toward pose estimation; moreover, the semi-synthetic dataset also plays a key role in bridging the sim-to-real gap.

Conclusions: In this work, we aim to improve the accuracy and robustness of pose estimation in challenging dynamic surgical scenes, by effectively handling scene dynamics. Our method, combined with the proposed synthetic dataset, demonstrates improved performance in pose estimation and generalizes well to real-world data, showing its potential in advancing related works such as SLAM and 3D reconstruction in complex surgical environments.

Purpose: Deep learning models for endoscopic assistance applications predominantly focus on image-based tasks, such as tool detection, anatomical classification, and workflow segmentation. However, these approaches often neglect the integration of natural language, limiting their assistance capabilities. This work adopts a proven architecture for vision-language models (VLM) to perform multi-task learning for image classification, text prediction, and surgical report generation, specifically for endoscopic ENT surgeries.

Methods: We adopted a VLM architecture utilizing encoders biased for the endoscopy domain for image and text embedding and combine them via cross-attention. The model was trained on a newly created multi-task dataset derived from 30 annotated endoscopic procedures, comprising 130,000 multi-label images, anatomical descriptions, and synchronized surgical reports. Two variations of the model, a lightweight 61M parameter and a 176M parameter model, were evaluated both against an existing baseline from previous mono-task studies as well as the EndoVit and SurgicalGPT models as external references. Ablation studies investigate the influence of removing image or text embeddings, and cross-attention on the task performance. Performance was measured for landmark classification, structured text prediction, and report generation using precision, recall, F1-score, BLEU-2, ROUGE-L, and cosine similarity metrics.

Results: The VLM base model improves the baseline f1 score for image classification by up to 12% and natural language text generation by up to 14% across image classification and report generation tasks. The text generation of structured language tasks, however, showed minimal gains, indicating limitations in structured sentence learning from combined image-text embeddings. EndoViT and SurgicalGPT slightly trail our domain-specific VLM. The image-only and text-only ablations confirm that the vision component benefits language tasks, whereas text has limited impact on landmark detection.

Conclusion: We developed a vision-language model capable of integrating image and text data for endoscopic ENT assistance tasks, that is able to replace three isolated models, delivering multi-task assistance while outperforming prior and general‑purpose baselines. Remaining challenges include the handling of imbalanced class distributions and limited gains on templated structured text.

Purpose: Achieving a prosthetic femoral version (PFV) within the target range of 10-20° is crucial for optimal biomechanics in total hip arthroplasty (THA). Predicting the PFV preoperatively is challenging due to the limited understanding of the relationship between native femoral version (NFV) and the morphology of the intramedullary canal. This study aims to quantify the 3D morphological variability and identify the most variable anatomical features of the proximal femur pre- and post-operatively.

Methods: Pre- and post-operative CT scans from 62 patients (31 males, 31 females) who underwent THA and received a single stem design (straight, triple-tapered) were analysed. Four femoral models were generated per patient: 1. Native proximal femur, 2. Native femur after neck osteotomy, 3. Internal femoral canal after neck osteotomy, and 4. Reconstructed femur. Statistical Shape Models (SSMs) were developed separately by sex, and principal component analysis (PCA) was used to identify dominant modes of anatomical variation.

Results: The first three principal components (PCs) accounted for over 60% of shape variability across all models. PFV showed weak correlation with NFV as variability existed between the SSM of the internal femoral canal and SSM of the native proximal femur. Sex-specific differences in the measured NFV and PFV were found, with females exhibiting a greater range and a more anteverted femur/femoral stem. The female canal model showed intramedullary version variability; however, this variability was not present in the first three PCs in the corresponding male model.

Conclusions: This study demonstrates that PFV cannot be reliably predicted from NFV alone. These findings underscore the need for advanced, 3D preoperative planning tools to better predict stem version and accommodate patient-specific anatomy. Additionally, the increased variability observed in females may warrant sex-specific consideration in implant design choice and surgical technique.

Purpose: Oropharyngeal dysphagia affects up to half of head and neck cancer (HNC) patients. Multi-swallow video-fluoroscopic swallow studies (VFSS) combined with high-resolution impedance manometry (HRIM) offer a comprehensive assessment of swallowing function. However, their use in HNC populations is limited by high clinical workload and complexity of data collection and analysis with existing software.

Methods: To address the data collection challenge, we propose a framework for automatic swallow detection in simultaneous VFSS-HRIM examinations. The framework identifies candidate swallow intervals in continuous VFSS videos using an optimized double-sweep optical flow algorithm. Each candidate interval is then classified using a pressure-based swallow template derived from three annotated samples, leveraging features such as normalized peak-to-peak amplitude, mean, and standard deviation from upper esophageal sphincter sensors.

Results: The methodology was evaluated on 97 swallows from twelve post-head and neck cancer patients. The detection pipeline achieved 95% Recall and 92% F1-score. Importantly, the number of required HRIM annotations was reduced by 63%, substantially decreasing clinician workload while maintaining high accuracy.

Conclusion: This framework overcomes limitations of current software for simultaneous VFSS-HRIM collection by enabling high-accuracy, low-input swallow detection in HNC patients. Validated on a heterogeneous patient cohort, it initiates the groundwork for scalable, objective, and multimodal swallowing assessment.

Purpose: Domain generalization plays a crucial role in analyzing medical images from diverse clinics, scanner vendors, and imaging modalities. Existing methods often require substantial computational resources to train a highly generalized segmentation network, presenting challenges in terms of both availability and cost. The goal of this work is to evaluate a novel, yet simple and effective method for enhancing the generalization of deep learning models in segmentation across varying modalities.

Methods: Eight augmentation methods will be applied individually to a source domain dataset in order to generalize deep learning models. These models will then be tested on completely unseen target domain datasets from a different imaging modality and compared against a lower baseline model. By leveraging standard augmentation techniques, extensive intensity augmentations, and carefully chosen color transformations, we aim to address the domain shift problem, particularly in the cross-modality setting.

Results: Our novel CmapAug method, when combined with standard augmentation techniques, resulted in a substantial improvement in the Dice Score, outperforming the baseline. While the baseline struggled to segment the liver structure in some test cases, our selective combination of augmentation methods achieved Dice scores as high as 83.2%.

Conclusion: Our results highlight the general effectiveness of the tested augmentation methods in addressing domain generalization and mitigating the domain shift problem caused by differences in imaging modalities between the source and target domains. The proposed augmentation strategy offers a simple yet powerful solution to this challenge, with significant potential in clinical scenarios where annotated data from the target domain are limited or unavailable.