International Journal of Computer Assisted Radiology and Surgery最新文献

Purpose: In surgical skill development, a trainee's goal is to move through the learning curve and achieve expert performance. Goal directed training, with expert performance as the goal, can facilitate skill development; however, there are currently few methods available to encode expert-like simulation performance into learning strategies that can be practiced independently.

Methods: We propose a novel method of surgical simulation skill analysis through segmenting and evaluating kinematic data with instantaneous screw axes (ISA) theory and K-means clustering. In ISA, single degree of freedom (DOF) tasks can be represented as displacements about a single screw axis; however, we propose extending this method to more complex tasks defining them with clusters of similar ISAs in the surgical environment, decomposing them into a sequence of 1DOF movements. In this paper, we present an ISA algorithm and apply it to surgeon manipulator poses across fourteen suturing and knot-tying gestures obtained from the JIGSAWS surgical dataset. We also apply this method to entire simulated suturing demonstrations across a 6-month training period from the BGU-SKILLS dataset. We implemented K-means clustering to segment these movements into sub-gestures. We hypothesize that individuals with greater levels of expertise should exhibit more concise actions with minimal extraneous movement; therefore, fewer clusters should be required to decompose their simulation performance.

Results: Our ISA algorithm was applied to 1136 gestures from ten surgeons across three skill levels and 324 unsegmented demonstrations collected from 18 surgical residents over a training period of 6 months. We performed a Kruskal-Wallis analysis with a Dunn-Sidak post-hoc test on the number of ISA clusters required to decompose each gesture. We found that highly task-constrained gestures required significantly fewer numbers of clusters for expert and/or intermediate groups when compared to novices on suturing tasks only.

Conclusion: Our results suggest that this method can be used to identify task-constrained gestures within independently performed suturing surgical simulations and classify them into higher skill and lower skill sets. This analysis can also provide geometric feedback on performed gestures vs expert gestures, providing personalized automated performance analysis for surgical trainees leading to personalized educational training.

Purpose: Depth estimation from stereoscopic laparoscopic videos is of vital importance in computer-assisted intervention due to its potential for downstream tasks in laparoscopic surgical navigation. Previous works mostly focus on depth estimation from static frames, while temporal information in stereoscopic laparoscopic videos is largely ignored.

Methods: A spatiotemporal swin (ST-Swin) transformer-based network, referred to as ST-Swin TransNet, is proposed for depth estimation in stereoscopic surgical videos. Built upon a symmetric encoder-decoder architecture consisting of 12 ST-Swin blocks, ST-Swin TransNet extracts spatiotemporal features for efficient and accurate depth estimation, where the ST-Swin blocks are designed to capture spatiotemporal information from stereo video sequences via self-attention mechanism. Given binocular laparoscopic videos, ST-Swin TransNet exploits hierarchical spatiotemporal features to predict disparity maps.

Results: Comprehensive experiments are conducted on two typical yet challenging public datasets to evaluate the performance of the proposed method. We additionally demonstrate the feasibility of applying ST-Swin TransNet to video see-through augmented reality (VST-AR) navigation in laparoscopic surgery. Our method achieved a mean absolute depth error (mADE) of 3.33 mm in depth estimation and a mean absolute distance (mAD) of 1.07 mm in VST-AR navigation.

Conclusion: A spatiotemporal swin transformer-based network for self-supervised depth estimation in binocular laparoscopic surgical videos was developed. Results from the comprehensive experiments demonstrate the superior performance of the proposed method over the state-of-the-art methods.

Purpose Surgical gestures are fundamental components of surgical procedures, encompassing actions such as cutting, suturing, and knot-tying. Gesture recognition plays a pivotal role in the automatic analysis of surgical data. Although recent advancements have improved surgical gesture recognition, much of the existing research relies on simulations or minimally invasive surgery data, failing to capture the complexities of open surgery. In this study, we introduce and employ a new open surgery dataset focused on closing incisions after saphenous vein harvesting. Methods Our goal is to improve gesture recognition accuracy by incorporating tool pose estimation and 3D hand pose predictions of surgeons. We employ MS-TCN++  and LTContext  for gesture recognition, and further enhance performance through an ensemble of models using different modalities-video, tool pose, and hand pose data.Results The results reveal that using an ensemble model combining all three modalities provides a substantial improvement over video-only approaches, leading to statistically significant gains across multiple evaluation metrics. We further demonstrate that the model can rely solely on hand and tool poses, completely discarding the video input, while still achieving comparable performance. Additionally, we show that an ensemble model using only hand and tool poses produces results that are either: statistically significantly better than using video alone, or not statistically significantly different.Conclusion This study highlights the effectiveness of integrating multimodal data for surgical gesture recognition. By combining video, hand pose, and tool pose information, our approach achieves higher accuracy and robustness compared to video-only methods. Moreover, the comparable performance of pose-only models suggests a promising, privacy-preserving alternative for surgical data analysis.

Purpose: To discuss the economic and environmental implications of implementing large language models (LLMs) in radiology, highlighting both their transformative potential and the challenges they pose for equitable and sustainable adoption.

Methods: Current trends in AI investment, infrastructure requirements, operational costs, and environmental impact associated with LLMs are analyzed, highlighting the specific challenges of integrating LLMs into radiological workflows, including data privacy, regulatory compliance, and cost barriers for healthcare institutions. The analysis also considers the costs of model validation, maintenance, and updates, as well as investments in system integration, staff training, and cybersecurity for clinical implementation.

Results: LLMs have revolutionized natural language processing and offer promising applications in radiology, such as improved diagnostic support and workflow optimization. However, their deployment involves substantial financial and environmental costs. Training and operating these models require high-performance computing infrastructure, significant energy consumption, and large volumes of annotated data. Water usage and CO₂ emissions from data centers further raise sustainability concerns, while ongoing operational costs add to the financial burden. Subscription fees and per-query pricing may restrict access for smaller institutions, widening existing inequalities.

Conclusion: While LLMs offer significant benefits for radiology, their high economic and environmental costs present challenges to widespread and equitable adoption. Responsible use, sustainable practices, and policy frameworks are essential to ensure that AI-driven innovations do not exacerbate existing disparities in healthcare access and quality.

Purpose: Anatomical identification during abdominal surgery is subjective given unclear boundaries of anatomical structures. Semantic segmentation of these structures relies on an accurate identification of the boundaries which carries an unknown uncertainty. Given its inherent subjectivity, it is important to assess annotation adequacy. This study aims to evaluate variability in anatomical structure identification and segmentation using MedSAM by surgical residents.

Methods: Images from the Dresden Surgical Anatomy Dataset and the Endoscapes2023 Dataset were semantically annotated by a group of surgery residents using MedSAM in the following classes: abdominal wall, colon, liver, small bowel, spleen, stomach and gallbladder. Each class had 3 to 4 sets of annotations. Inter-annotator variability was assessed through DSC, ICC, BIoU and using the Simultaneous Truth and Performance Level Estimation algorithm to obtain a consensus mask and by calculating Fleiss' kappa agreement between all annotations and reference.

Results: The study showed strong inter-annotator agreement among surgical residents, with DSC values of 0.84-0.95 and Fleiss' kappa between 0.85 and 0.91. Surface area reliability was good to excellent (ICC = 0.62-0.91), while boundary delineation showed lower reproducibility (BIoU = 0.092-0.157). STAPLE consensus masks confirmed consistent overall shape annotations despite variability in boundary precision.

Conclusion: The study demonstrated low variability in the semantic segmentation of intraperitoneal organs in minimally invasive abdominal surgery, performed by surgical residents using MedSAM. While DSC and Fleiss' kappa values confirm strong inter-annotator agreement, the relatively low BIoU values point to challenges in boundary precision, especially for anatomically complex or variable structures. These results establish a benchmark for expanding annotation efforts to larger datasets and more detailed anatomical features.

Purpose: Computer-assisted surgical navigation systems have been developed to improve the precision of total knee arthroplasty (TKA) by providing real-time guidance on implant alignment relative to patient anatomy. However, surface registration remains a key source of error that can propagate through the surgical workflow. This study investigates how patient-specific femoral bone geometry influences registration accuracy, aiming to enhance the reliability and consistency of computer-assisted orthopedic procedures.

Methods: Eighteen high-fidelity 3D-printed femur models were used to simulate intraoperative digitization. Surface points collected from the distal femur were registered to preoperative CT-derived models using a rigid iterative closest point (ICP) algorithm. Registration accuracy was quantified across six degrees of freedom. An in-house statistical shape model (SSM), built from 114 CT femurs, was employed to extract shape coefficients and correlate them with the measured registration errors. To verify robustness, additional analyses were conducted using synthetic and in silico CT-based femur datasets.

Results: Significant correlations (p-values < 0.05) were observed between specific shape coefficients and registration errors. The third and fourth principal shape modes showed the strongest associations with rotational misalignments, particularly flexion-extension and varus-valgus components. These findings demonstrate that geometric variability in the distal femur, especially condylar morphology, plays a major role in determining the stability and accuracy of surface-based registration.

Conclusions: Registration errors in TKA are strongly influenced by patient-specific bone geometry. Shape features derived from statistical shape models can serve as reliable predictors of registration performance, providing quantitative insight into how anatomical variability impacts surgical precision and alignment accuracy in computer-assisted total knee arthroplasty.

Purpose: Scan path planning is an essential technology for fully automated robotic ultrasound (US). During US screening in head and neck lesions, it is necessary to observe the entire neck, including the thyroid, parotid, submandibular glands, and cervical lymph nodes efficiently. This study developed a personalized scan path planning for robotic US in head and neck lesions.

Methods: We proposed the personalized scan path planning that can estimate the scan position by applying the non-rigid registration between the patient neck image and the similar neck image among the previous patients. To select the most similar neck image in the database, the unique similarity index with the combination of multiple similarity metrics (neck area, position of the centroid, SSIM, Dice coefficient, and contours of the neck area) was developed. Fifty neck images were used to determine the optimal combination experimentally. The non-rigid registration from each of the 50 images to the other 49 images, resulting in a total of 2,450 registration pairs, was conducted, and the correlations between each similarity metric and registration error were analyzed.

Results: The best-performing similarity index achieved a correlation coefficient of 0.730 between the similarity score and registration error, which shows that the proposed index can select the similar image with a high accuracy of scan path registration. With the proposed scan path planning, the feasibility study is performed on six healthy subjects. The results showed a success rate of 94.4% for obtaining the specific target images required for the neck and head US examinations.

Conclusion: The proposed scan path planning allows the scan to adapt to individual differences in the neck, which can contribute to the automated robotic US in head and neck lesions.

Purpose: Estimating the 6 degrees of freedom (DoF) pose of an endoscope is crucial for various applications in minimally invasive computer-assisted surgery. Image-based approaches are some of the most practical solutions for pose estimation in surgical environments, due to a limited workspace and sensor constraints. However, these methods often struggle or fail in dynamic scenes, such as those involving tissue deformation, surgical tool movement, and tool-tissue interaction.

Methods: We propose DyEndoVO, an end-to-end visual odometry method in dynamic endoscopic scenes. Our method consists of a transformer-based motion detection network and a weighted pose-optimization module. The motion detection network infers scene dynamics and guides the pose estimation. Furthermore, we introduce a semi-synthetic dataset featuring tissue and tool movement categories. It serves as training data, improving pose estimation accuracy, and also includes motion masks to enable a fine-grained inspection and evaluation.

Results: DyEndoVO significantly outperforms state-of-the-art methods in pose estimation for dynamic surgical scenes. Despite being trained solely on a synthetic dataset, our method generalizes well to real-world data without fine-tuning. Further analysis attributes this success to the effective detection of scene dynamics and the adaptation in the learned weight toward pose estimation; moreover, the semi-synthetic dataset also plays a key role in bridging the sim-to-real gap.

Conclusions: In this work, we aim to improve the accuracy and robustness of pose estimation in challenging dynamic surgical scenes, by effectively handling scene dynamics. Our method, combined with the proposed synthetic dataset, demonstrates improved performance in pose estimation and generalizes well to real-world data, showing its potential in advancing related works such as SLAM and 3D reconstruction in complex surgical environments.

Purpose: Deep learning models for endoscopic assistance applications predominantly focus on image-based tasks, such as tool detection, anatomical classification, and workflow segmentation. However, these approaches often neglect the integration of natural language, limiting their assistance capabilities. This work adopts a proven architecture for vision-language models (VLM) to perform multi-task learning for image classification, text prediction, and surgical report generation, specifically for endoscopic ENT surgeries.

Methods: We adopted a VLM architecture utilizing encoders biased for the endoscopy domain for image and text embedding and combine them via cross-attention. The model was trained on a newly created multi-task dataset derived from 30 annotated endoscopic procedures, comprising 130,000 multi-label images, anatomical descriptions, and synchronized surgical reports. Two variations of the model, a lightweight 61M parameter and a 176M parameter model, were evaluated both against an existing baseline from previous mono-task studies as well as the EndoVit and SurgicalGPT models as external references. Ablation studies investigate the influence of removing image or text embeddings, and cross-attention on the task performance. Performance was measured for landmark classification, structured text prediction, and report generation using precision, recall, F1-score, BLEU-2, ROUGE-L, and cosine similarity metrics.

Results: The VLM base model improves the baseline f1 score for image classification by up to 12% and natural language text generation by up to 14% across image classification and report generation tasks. The text generation of structured language tasks, however, showed minimal gains, indicating limitations in structured sentence learning from combined image-text embeddings. EndoViT and SurgicalGPT slightly trail our domain-specific VLM. The image-only and text-only ablations confirm that the vision component benefits language tasks, whereas text has limited impact on landmark detection.

Conclusion: We developed a vision-language model capable of integrating image and text data for endoscopic ENT assistance tasks, that is able to replace three isolated models, delivering multi-task assistance while outperforming prior and general‑purpose baselines. Remaining challenges include the handling of imbalanced class distributions and limited gains on templated structured text.

Purpose: Achieving a prosthetic femoral version (PFV) within the target range of 10-20° is crucial for optimal biomechanics in total hip arthroplasty (THA). Predicting the PFV preoperatively is challenging due to the limited understanding of the relationship between native femoral version (NFV) and the morphology of the intramedullary canal. This study aims to quantify the 3D morphological variability and identify the most variable anatomical features of the proximal femur pre- and post-operatively.

Methods: Pre- and post-operative CT scans from 62 patients (31 males, 31 females) who underwent THA and received a single stem design (straight, triple-tapered) were analysed. Four femoral models were generated per patient: 1. Native proximal femur, 2. Native femur after neck osteotomy, 3. Internal femoral canal after neck osteotomy, and 4. Reconstructed femur. Statistical Shape Models (SSMs) were developed separately by sex, and principal component analysis (PCA) was used to identify dominant modes of anatomical variation.

Results: The first three principal components (PCs) accounted for over 60% of shape variability across all models. PFV showed weak correlation with NFV as variability existed between the SSM of the internal femoral canal and SSM of the native proximal femur. Sex-specific differences in the measured NFV and PFV were found, with females exhibiting a greater range and a more anteverted femur/femoral stem. The female canal model showed intramedullary version variability; however, this variability was not present in the first three PCs in the corresponding male model.

Conclusions: This study demonstrates that PFV cannot be reliably predicted from NFV alone. These findings underscore the need for advanced, 3D preoperative planning tools to better predict stem version and accommodate patient-specific anatomy. Additionally, the increased variability observed in females may warrant sex-specific consideration in implant design choice and surgical technique.