International Journal of Computer Assisted Radiology and Surgery最新文献

Purpose: Teamwork is fundamental to medical practice and relies on seamless collaboration among professionals with different tasks. Integrating robotic systems into this environment demands smooth interactions. Human action recognition, which infers a person's state without explicit input, can support this. We focus on handovers between medical staff, using the actions as implicit cues for robotic assistance to replace the giving party in such scenarios.

Methods: Skeletal information processed with differing machine learning algorithms makes it possible to derive actions out of sequential image data. Transferred to the medical context, we aim to infer actions defined for each situation in two datasets, a surgery in the operating room and a care intervention in the patient ward, depicting a handover between staff. We aim to abstract movement patterns across individuals through skeletal representation, leveraging the spatiotemporal information of medical handovers to enable future robotic systems to interact based on implicit cues.

Results: We report an F1 score of $0.736\pm 0.045$ for the OR dataset with ST-GCN and an F1 score of $0.941\pm 0.009$ for the Ward dataset with the SkateFormer human action recognition. The defined actions showed distinction in the confusion matrix with limitations on actions with a rapid transition like approach and reach as well as the handover actions in the OR.

Conclusion: The handover phases in two medical contexts, a minimally invasive surgery and a wound dressing on the patient station, are recognized with the proposed framework. This lays a first step for the integration of robotic assistance in the handover of medical material or instruments.

Purpose: Temporomandibular joint (TMJ) prosthesis implantation is an effective procedure for treating temporomandibular joint disorders. Traditionally, preoperative planning for TMJ surgery has been conducted manually by experienced surgeons, which often results in longer operating time and less reliable prosthesis placement. This study proposes an automated surgical planning algorithm for TMJ prosthesis implantation that calculates the optimal position for prosthesis placement.

Methods: Firstly, the STL model of the patient's craniomaxillofacial structure is populated with a point cloud, and the oriented bounding box is calculated. Next, the point cloud that meets specific constraints is filtered based on the characteristics of the anatomical structures. Subsequently, the contact surfaces of the prosthesis with the zygomatic arch and the mandible are generated according to the distribution of the point cloud. Finally, a system of linear equations is established based on the geometric constraints and solved to determine the precise placement position and orientation of the prosthesis.

Results: A group of 12 patients with 24 clinical cases was utilized for automatic planning to evaluate the efficiency and quality of the system. The results demonstrated that the average time required for the automatic planning algorithm was under 30 s for the entire procedure. Furthermore, the calculation of the bone-prosthesis contact area indicated that the quality of the automatic planning is comparable to that of plans created by professional surgeons.

Conclusions: Compared to the cumbersome manual planning methods currently used for TMJ prosthesis implantation, the approach proposed in this paper facilitates efficient and accurate automated preoperative planning. This method simulates the optimal placement of TMJ prosthesis to ensure long-term stability, demonstrating significant clinical potential for future applications in TMJ prosthesis implantation surgery.

Purpose: Artificial intelligence (AI) is rapidly transforming diagnostic and interventional radiology, supported by accelerating regulatory approvals and clinical adoption. Despite progress, integration varies across modalities and procedures. This study is a structured narrative review of four representative workflows-MRI and CT screening, coronary stenting, and liver cryoablation-to quantify automation readiness, accuracy gains, and efficiency improvements. The novelty lies in comparing diagnostic and interventional domains to highlight distinct maturity levels and future opportunities for AI-driven workflow optimization and clinical value creation.

Methods: A structured analysis was performed identifying 43 workflow steps across the four selected procedures. Each step was evaluated for potential automation, accuracy improvement, and ability to provide new clinical insights, considering current availability and projected 2030 maturity. The assessment drew on peer-reviewed literature, FDA approvals, and industry data (2015-2025). A structured taxonomy distinguished between full automation, human-augmented improvements, and novel AI-enabled guidance functions.

Results: Diagnostic imaging showed higher maturity than interventional workflows. Currently, 70% of MRI and 64% of CT steps have available AI solutions, compared to 55% in coronary stenting and 36% in liver cryoablation. By 2030, nearly all steps are expected to be AI-supported. AI achieved up to 94% segmentation accuracy, 95% nodule detection sensitivity, 30-75% scan time reductions, and 30-50% faster reporting. Interventional applications improved catheter navigation, probe placement, and ablation success but still required significant human oversight.

Conclusions: AI has already demonstrated measurable gains in diagnostic accuracy, efficiency, and workflow standardization. Interventional applications are emerging, with future growth expected in guidance, robotics, and real-time optimization. Despite progress, key limitations include algorithm generalizability, clinical interpretability, organizational readiness, and regulatory uncertainty. AI will augment rather than replace human expertise, with collaborative human-AI workflows being essential. Future integration efforts must address interoperability, workforce adaptation, and ethical considerations to ensure safe, equitable, and clinically impactful deployment.

Purpose: System calibration, including hand-robot and robot-world calibration, is an essential step that directly influences the location accuracy of surgical robots. Conventional calibration methods for orthognathic robot systems (ORSs) face significant challenges in handling irregularly shaped end tools, leading to manual intervention and compromised accuracy. Therefore, an automatic method has been proposed to improve the calibration efficiency and accuracy of ORSs.

Methods: The core innovation of the proposed method lies in enabling automation of both pre-intraoperative image registration and robotic hand-eye calibration, via aligning the 3D model of irregularly shaped tools to the preoperative CT space. It can effectively minimize errors caused by manual intervention. firstly, the equations of hand-eye-tool calibration were reconstructed using the preoperative graphic information to define tool endpoints (TEPs). Then, the transformation matrices were solved via a robust optimization method based on least squares. finally, the whole calibration process was completed automatically with robot path planning without human involvement.

Results: A group of simulated robot-assisted orthognathic surgery experiments was performed. The proposed method achieved a calibration error of 1.04 ± 0.54 mm, and the total execution error were reduced to 1.56 ± 0.61 mm.

Conclusion: The experimental results proved that the proposed calibration method could not only automate the calibration process, but also effectively improve the accuracy and stability of the system. It is expected to pave the way for more autonomous and efficient surgical procedures. Also, there are some limitations need to be overcome, including dependency on marker-based tracking and small sample size. Future work will integrate markerless tracking and machine learning for further optimization.

Purpose: The recognition of surgical instrument-tissue interactions can enhance the surgical workflow analysis, improve automated safety systems and enable skill assessment in minimally invasive surgery. However, current deep learning methods for surgical instrument-tissue interaction recognition often rely on static images or coarse temporal sampling, limiting their ability to capture rapid surgical dynamics. Therefore, this study systematically investigates the impact of incorporating fine-grained temporal context into deep learning models for interaction recognition.

Methods: We conduct extensive experiments with multiple curated video-based datasets to investigate the influence of fine-grained temporal context for the task of instrument-tissue interaction recognition using video transformer with spatio-temporal feature extraction capabilities. Additionally, we propose a multi-task-attention module that utilizes cross-attention and a gating mechanism to improve communication between the subtasks of identifying the surgical instrument, atomic action, and anatomical target.

Results: Our study demonstrates the benefit of utilizing the fine-grained temporal context for recognition of instrument-tissue interactions, with an optimal sampling rate of 6-8 Hz identified for the examined datasets. Furthermore, our proposed MTAM significantly outperforms state-of-the-art multi-task video transformer on the CholecT45-Vid and GraSP-Vid datasets, achieving relative increases of $4.8\%$ and $5.9\%$ in surgical instrument-tissue interaction recognition, respectively.

Conclusions: In this work, we demonstrate the benefits of using a fine-grained temporal context rather than static images or coarse temporal context for the task of surgical instrument-tissue interaction recognition. We also show that leveraging cross-attention with spatio-temporal features from various subtasks leads to improved surgical instrument-tissue interaction recognition performance. The project is available at: https://lennart-maack.github.io/InstrTissRec-MTAM .

Purpose: This article reports on the development and feasibility testing of an MR-safe robotic needle driver. The needle driver is pneumatically actuated and designed for automatic insertion and extraction of needles along a straight trajectory within the MRI scanner.

Method: All parts use plastic resins and composite materials to ensure MR-safe operation. A needle could be clamped in the needle carriage using a pneumatically operated clamp. The clamp is designed to be easily attached and detached from the needle driver. Clamps with different opening sizes could accommodate a range of needles from 18 to 22 gauge. To mimic the manual procedure of needle insertion, a pneumatically operated rack-and-pinion mechanism simultaneously translates and rotates the needle carriage along a helical slot. Signal-to-noise ratio (SNR) and 2-D geometric distortion were measured to evaluate the MRI compatibility. Targeting was measured with an electromagnetic tracker. We also evaluated the maximum force that could be generated at the tip of the needle with different clamping pressures using a force sensor.

Results: We recorded the maximum percentage change in SNR for multiple configurations of needle drivers as 6.6% and the maximum geometric distortion at 0.24%. The needle driver's mean positioning accuracy for 105 targets at 50 mm depth was 2.38 ± 1.00 mm in a composite tissue phantom. The angulation error for the straight trajectory was 0.51°, and the mean linear trajectory deviation was statistically negligible. The measured force at the needle tip was 1.17N, 1.6N, and 2.12N at 30, 40, and 50 psi, respectively.

Conclusion: This preliminary study showed that the prototype of our robotic needle driver works as intended for the insertion and extraction of the needle. The driver is MR-safe and serves as a suitable platform for MRI-guided interventions.

Purpose: Automatic recognition of surgical workflows is a complex yet essential task of context-aware systems in the operating room. However, achieving high accuracy in phase recognition remains a challenge due to the complexity of surgical procedures. While recent deep learning models have made significant progress, individual models often exhibit limitations-some may excel at capturing spatial features, while others are better at modeling temporal dependencies or handling class imbalance.

Methods: This study investigates the use of ensemble learning to combine the complementary strengths of diverse architectures, aiming to mitigate individual model weaknesses and improve performance in surgical phase recognition using the Cholec80 dataset. A variety of advanced deep learning architectures was integrated into a single ensemble. Models were carefully selected and tuned to ensure diversity, resulting in a final set of 15 unique ensembles. Ensemble strategies were explored to determine the most effective method for combining the distinct models.

Results: The results demonstrated that ensemble learning significantly improved performance. Among the ensemble strategies tested, majority voting achieved the highest F1-score, followed by the proposed artificial neural network StackingNet. Ensembles with high model diversity showed superior performance compared to those with lower diversity. The optimal ensemble configuration integrated top-performing models from different architectures, leading to improvements in accuracy, F1-score, and Jaccard Index by 1.48 %, 3.68 %, and 5.43 %, respectively, compared to the best individual models.

Conclusion: This study demonstrates that ensemble learning can substantially enhance surgical phase recognition by leveraging the complementary strengths of diverse deep learning models. Ensemble size, diversity, and meta-model selection were identified as key factors influencing performance. The resulting improvements translate into clinically meaningful benefits by enabling more reliable context-aware guidance, reducing misclassifications during critical phases, and improving surgeons' trust in artificial intelligence (AI) systems.

Purpose: Prostate cancer is a prevalent malignant tumor in men, and accurate diagnosis and personalized treatment rely on multimodal imaging, such as MRI and TRUS. However, differences in imaging mechanisms and prostate deformation due to ultrasound probe compression pose significant challenges for high-quality registration between the two modalities.

Methods: In this study, we propose a label-aware weakly supervised diffusion model for MRI-TRUS multimodal image registration. First, we align label centroid positions by maximizing the Dice coefficient to correct initial biases. Second, we combine label supervision with a diffusion model to generate high-quality deformation fields. Finally, we incorporate a feature-guided module to better preserve edge structures and improve registration smoothness.

Results: Experiments conducted on the µ-RegPro dataset demonstrate that our method outperforms current state-of-the-art (SOTA) approaches across multiple evaluation metrics. Specifically, it achieves a Dice coefficient of 0.880 and reduces the target registration error (TRE) to 0.940, significantly surpassing unsupervised methods such as VoxelMorph, FSDiffReg, and supervised methods like LocalNet and AutoFuse. The results show that preliminary label centroid alignment effectively enhances the performance of the diffusion-based deformation registration model, reducing the TRE from 3.084 to 0.940. The ablation study demonstrates that the feature-guided diffusion module effectively suppresses deformation field folding, while the label-aware module enhances label alignment. When combined, the proposed framework achieves a favorable balance, substantially improving registration accuracy (Dice = 0.880, TRE = 0.940) with reduced folding (|J|_≤0 = 0.134). This method exhibits strong robustness and generalizability in handling large deformations in target regions while preserving details in nontarget regions.

Conclusion: The proposed label-aware weakly supervised diffusion model enables accurate and efficient MRI-TRUS multimodal image registration, offering strong potential for clinical applications such as prostate cancer diagnosis, targeted biopsy, and image-guided navigation.

Purpose: Virtual reality (VR) has attracted attention in healthcare for many promising applications including pre-surgical planning. Currently, there exists a critical gap in comprehension of the impact of VR on physicians' thinking. Self-reported data from surveys and metrics based on confidence and task completion may not yield sufficiently detailed understanding of the complex decision making and cognitive load experienced by surgeons during VR-based pre-surgical planning.

Methods: Our research aims to address the gap in understanding the impact of VR on physicians' mental models through a novel methodology of self-directed think-aloud protocols, offering deeper perspectives into physicians' thought processes within the virtual 3D environment. We performed qualitative analysis of recorded verbalizations and actions in VR in addition to quantitative measures from the NASA task load index (NASA-TLX). Analysis was conducted to identify thematic sequences in VR which influenced clinical decision making when reviewing patient anatomy.

Results: We find a significant increase in reported physician confidence in understanding of the patient anatomy from before VR to after (p = 0.012) and identified several common patterns of 3D exploration of the anatomy in VR. Physicians also reported low cognitive stress on the NASA-TLX.

Conclusion: Our findings indicate VR has value beyond simulating surgery, helping physicians to confirm findings from conventional medical imaging, visualize approaches with detail, and help make complex decisions while mentally preparing for surgery. These findings provide evidence that VR and related 3D visualization are helpful for pre-surgical planning of complex cases.

Purpose: Bayesian networks (BNs) are valuable for clinical decision support due to their transparency and interpretability. However, BN modelling requires considerable manual effort. This study explores how integrating large language models (LLMs) with retrieval-augmented generation (RAG) can improve BN modelling by increasing efficiency, reducing cognitive workload, and ensuring accuracy.

Methods: We developed a web-based BN modelling service that integrates an LLM-RAG pipeline. A fine-tuned GTE-Large embedding model was employed for knowledge retrieval, optimised through recursive chunking and query expansion. To ensure accurate BN suggestions, we defined a causal structure for medical idioms by unifying existing BN frameworks. GPT-4 and Mixtral 8x7B were used to handle complex data interpretation and to generate modelling suggestions, respectively. A user study with four clinicians assessed usability, retrieval accuracy, and cognitive workload using NASA-TLX. The study demonstrated the system's potential for efficient and clinically relevant BN modelling.

Results: The RAG pipeline improved retrieval accuracy and answer relevance. Recursive chunking with the fine-tuned embedding model GTE-Large achieved the highest retrieval accuracy score (0.9). Query expansion and Hyde optimisation enhanced retrieval accuracy for semantic chunking (0.75 to 0.85). Responses maintained high faithfulness ( $\ge$ 0.9). However, the LLM occasionally failed to adhere to predefined causal structures and medical idioms. All clinicians, regardless of BN experience, created comprehensive models within one hour. Experienced clinicians produced more complex models, but occasionally introduced causality errors, while less experienced users adhered more accurately to predefined structures. The tool reduced cognitive workload (2/7 NASA-TLX) and was described as intuitive, although workflow interruptions and minor technical issues highlighted areas for improvement.

Conclusion: Integrating LLM-RAG into BN modelling enhances efficiency and accuracy. Future work may focus on automated preprocessing, refinements of the user interface, and extending the RAG pipeline with validation steps and external biomedical sources. Generative AI holds promise for expert-driven knowledge modelling.