International Journal of Computer Assisted Radiology and Surgery最新文献

Purpose: Artificial intelligence (AI) is rapidly transforming diagnostic and interventional radiology, supported by accelerating regulatory approvals and clinical adoption. Despite progress, integration varies across modalities and procedures. This study is a structured narrative review of four representative workflows-MRI and CT screening, coronary stenting, and liver cryoablation-to quantify automation readiness, accuracy gains, and efficiency improvements. The novelty lies in comparing diagnostic and interventional domains to highlight distinct maturity levels and future opportunities for AI-driven workflow optimization and clinical value creation.

Methods: A structured analysis was performed identifying 43 workflow steps across the four selected procedures. Each step was evaluated for potential automation, accuracy improvement, and ability to provide new clinical insights, considering current availability and projected 2030 maturity. The assessment drew on peer-reviewed literature, FDA approvals, and industry data (2015-2025). A structured taxonomy distinguished between full automation, human-augmented improvements, and novel AI-enabled guidance functions.

Results: Diagnostic imaging showed higher maturity than interventional workflows. Currently, 70% of MRI and 64% of CT steps have available AI solutions, compared to 55% in coronary stenting and 36% in liver cryoablation. By 2030, nearly all steps are expected to be AI-supported. AI achieved up to 94% segmentation accuracy, 95% nodule detection sensitivity, 30-75% scan time reductions, and 30-50% faster reporting. Interventional applications improved catheter navigation, probe placement, and ablation success but still required significant human oversight.

Conclusions: AI has already demonstrated measurable gains in diagnostic accuracy, efficiency, and workflow standardization. Interventional applications are emerging, with future growth expected in guidance, robotics, and real-time optimization. Despite progress, key limitations include algorithm generalizability, clinical interpretability, organizational readiness, and regulatory uncertainty. AI will augment rather than replace human expertise, with collaborative human-AI workflows being essential. Future integration efforts must address interoperability, workforce adaptation, and ethical considerations to ensure safe, equitable, and clinically impactful deployment.

Purpose: System calibration, including hand-robot and robot-world calibration, is an essential step that directly influences the location accuracy of surgical robots. Conventional calibration methods for orthognathic robot systems (ORSs) face significant challenges in handling irregularly shaped end tools, leading to manual intervention and compromised accuracy. Therefore, an automatic method has been proposed to improve the calibration efficiency and accuracy of ORSs.

Methods: The core innovation of the proposed method lies in enabling automation of both pre-intraoperative image registration and robotic hand-eye calibration, via aligning the 3D model of irregularly shaped tools to the preoperative CT space. It can effectively minimize errors caused by manual intervention. firstly, the equations of hand-eye-tool calibration were reconstructed using the preoperative graphic information to define tool endpoints (TEPs). Then, the transformation matrices were solved via a robust optimization method based on least squares. finally, the whole calibration process was completed automatically with robot path planning without human involvement.

Results: A group of simulated robot-assisted orthognathic surgery experiments was performed. The proposed method achieved a calibration error of 1.04 ± 0.54 mm, and the total execution error were reduced to 1.56 ± 0.61 mm.

Conclusion: The experimental results proved that the proposed calibration method could not only automate the calibration process, but also effectively improve the accuracy and stability of the system. It is expected to pave the way for more autonomous and efficient surgical procedures. Also, there are some limitations need to be overcome, including dependency on marker-based tracking and small sample size. Future work will integrate markerless tracking and machine learning for further optimization.

Purpose: The recognition of surgical instrument-tissue interactions can enhance the surgical workflow analysis, improve automated safety systems and enable skill assessment in minimally invasive surgery. However, current deep learning methods for surgical instrument-tissue interaction recognition often rely on static images or coarse temporal sampling, limiting their ability to capture rapid surgical dynamics. Therefore, this study systematically investigates the impact of incorporating fine-grained temporal context into deep learning models for interaction recognition.

Methods: We conduct extensive experiments with multiple curated video-based datasets to investigate the influence of fine-grained temporal context for the task of instrument-tissue interaction recognition using video transformer with spatio-temporal feature extraction capabilities. Additionally, we propose a multi-task-attention module that utilizes cross-attention and a gating mechanism to improve communication between the subtasks of identifying the surgical instrument, atomic action, and anatomical target.

Results: Our study demonstrates the benefit of utilizing the fine-grained temporal context for recognition of instrument-tissue interactions, with an optimal sampling rate of 6-8 Hz identified for the examined datasets. Furthermore, our proposed MTAM significantly outperforms state-of-the-art multi-task video transformer on the CholecT45-Vid and GraSP-Vid datasets, achieving relative increases of $4.8\%$ and $5.9\%$ in surgical instrument-tissue interaction recognition, respectively.

Conclusions: In this work, we demonstrate the benefits of using a fine-grained temporal context rather than static images or coarse temporal context for the task of surgical instrument-tissue interaction recognition. We also show that leveraging cross-attention with spatio-temporal features from various subtasks leads to improved surgical instrument-tissue interaction recognition performance. The project is available at: https://lennart-maack.github.io/InstrTissRec-MTAM .

Purpose: This article reports on the development and feasibility testing of an MR-safe robotic needle driver. The needle driver is pneumatically actuated and designed for automatic insertion and extraction of needles along a straight trajectory within the MRI scanner.

Method: All parts use plastic resins and composite materials to ensure MR-safe operation. A needle could be clamped in the needle carriage using a pneumatically operated clamp. The clamp is designed to be easily attached and detached from the needle driver. Clamps with different opening sizes could accommodate a range of needles from 18 to 22 gauge. To mimic the manual procedure of needle insertion, a pneumatically operated rack-and-pinion mechanism simultaneously translates and rotates the needle carriage along a helical slot. Signal-to-noise ratio (SNR) and 2-D geometric distortion were measured to evaluate the MRI compatibility. Targeting was measured with an electromagnetic tracker. We also evaluated the maximum force that could be generated at the tip of the needle with different clamping pressures using a force sensor.

Results: We recorded the maximum percentage change in SNR for multiple configurations of needle drivers as 6.6% and the maximum geometric distortion at 0.24%. The needle driver's mean positioning accuracy for 105 targets at 50 mm depth was 2.38 ± 1.00 mm in a composite tissue phantom. The angulation error for the straight trajectory was 0.51°, and the mean linear trajectory deviation was statistically negligible. The measured force at the needle tip was 1.17N, 1.6N, and 2.12N at 30, 40, and 50 psi, respectively.

Conclusion: This preliminary study showed that the prototype of our robotic needle driver works as intended for the insertion and extraction of the needle. The driver is MR-safe and serves as a suitable platform for MRI-guided interventions.

Purpose: Automatic recognition of surgical workflows is a complex yet essential task of context-aware systems in the operating room. However, achieving high accuracy in phase recognition remains a challenge due to the complexity of surgical procedures. While recent deep learning models have made significant progress, individual models often exhibit limitations-some may excel at capturing spatial features, while others are better at modeling temporal dependencies or handling class imbalance.

Methods: This study investigates the use of ensemble learning to combine the complementary strengths of diverse architectures, aiming to mitigate individual model weaknesses and improve performance in surgical phase recognition using the Cholec80 dataset. A variety of advanced deep learning architectures was integrated into a single ensemble. Models were carefully selected and tuned to ensure diversity, resulting in a final set of 15 unique ensembles. Ensemble strategies were explored to determine the most effective method for combining the distinct models.

Results: The results demonstrated that ensemble learning significantly improved performance. Among the ensemble strategies tested, majority voting achieved the highest F1-score, followed by the proposed artificial neural network StackingNet. Ensembles with high model diversity showed superior performance compared to those with lower diversity. The optimal ensemble configuration integrated top-performing models from different architectures, leading to improvements in accuracy, F1-score, and Jaccard Index by 1.48 %, 3.68 %, and 5.43 %, respectively, compared to the best individual models.

Conclusion: This study demonstrates that ensemble learning can substantially enhance surgical phase recognition by leveraging the complementary strengths of diverse deep learning models. Ensemble size, diversity, and meta-model selection were identified as key factors influencing performance. The resulting improvements translate into clinically meaningful benefits by enabling more reliable context-aware guidance, reducing misclassifications during critical phases, and improving surgeons' trust in artificial intelligence (AI) systems.

Purpose: Prostate cancer is a prevalent malignant tumor in men, and accurate diagnosis and personalized treatment rely on multimodal imaging, such as MRI and TRUS. However, differences in imaging mechanisms and prostate deformation due to ultrasound probe compression pose significant challenges for high-quality registration between the two modalities.

Methods: In this study, we propose a label-aware weakly supervised diffusion model for MRI-TRUS multimodal image registration. First, we align label centroid positions by maximizing the Dice coefficient to correct initial biases. Second, we combine label supervision with a diffusion model to generate high-quality deformation fields. Finally, we incorporate a feature-guided module to better preserve edge structures and improve registration smoothness.

Results: Experiments conducted on the µ-RegPro dataset demonstrate that our method outperforms current state-of-the-art (SOTA) approaches across multiple evaluation metrics. Specifically, it achieves a Dice coefficient of 0.880 and reduces the target registration error (TRE) to 0.940, significantly surpassing unsupervised methods such as VoxelMorph, FSDiffReg, and supervised methods like LocalNet and AutoFuse. The results show that preliminary label centroid alignment effectively enhances the performance of the diffusion-based deformation registration model, reducing the TRE from 3.084 to 0.940. The ablation study demonstrates that the feature-guided diffusion module effectively suppresses deformation field folding, while the label-aware module enhances label alignment. When combined, the proposed framework achieves a favorable balance, substantially improving registration accuracy (Dice = 0.880, TRE = 0.940) with reduced folding (|J|_≤0 = 0.134). This method exhibits strong robustness and generalizability in handling large deformations in target regions while preserving details in nontarget regions.

Conclusion: The proposed label-aware weakly supervised diffusion model enables accurate and efficient MRI-TRUS multimodal image registration, offering strong potential for clinical applications such as prostate cancer diagnosis, targeted biopsy, and image-guided navigation.

Purpose: Virtual reality (VR) has attracted attention in healthcare for many promising applications including pre-surgical planning. Currently, there exists a critical gap in comprehension of the impact of VR on physicians' thinking. Self-reported data from surveys and metrics based on confidence and task completion may not yield sufficiently detailed understanding of the complex decision making and cognitive load experienced by surgeons during VR-based pre-surgical planning.

Methods: Our research aims to address the gap in understanding the impact of VR on physicians' mental models through a novel methodology of self-directed think-aloud protocols, offering deeper perspectives into physicians' thought processes within the virtual 3D environment. We performed qualitative analysis of recorded verbalizations and actions in VR in addition to quantitative measures from the NASA task load index (NASA-TLX). Analysis was conducted to identify thematic sequences in VR which influenced clinical decision making when reviewing patient anatomy.

Results: We find a significant increase in reported physician confidence in understanding of the patient anatomy from before VR to after (p = 0.012) and identified several common patterns of 3D exploration of the anatomy in VR. Physicians also reported low cognitive stress on the NASA-TLX.

Conclusion: Our findings indicate VR has value beyond simulating surgery, helping physicians to confirm findings from conventional medical imaging, visualize approaches with detail, and help make complex decisions while mentally preparing for surgery. These findings provide evidence that VR and related 3D visualization are helpful for pre-surgical planning of complex cases.

Purpose: Bayesian networks (BNs) are valuable for clinical decision support due to their transparency and interpretability. However, BN modelling requires considerable manual effort. This study explores how integrating large language models (LLMs) with retrieval-augmented generation (RAG) can improve BN modelling by increasing efficiency, reducing cognitive workload, and ensuring accuracy.

Methods: We developed a web-based BN modelling service that integrates an LLM-RAG pipeline. A fine-tuned GTE-Large embedding model was employed for knowledge retrieval, optimised through recursive chunking and query expansion. To ensure accurate BN suggestions, we defined a causal structure for medical idioms by unifying existing BN frameworks. GPT-4 and Mixtral 8x7B were used to handle complex data interpretation and to generate modelling suggestions, respectively. A user study with four clinicians assessed usability, retrieval accuracy, and cognitive workload using NASA-TLX. The study demonstrated the system's potential for efficient and clinically relevant BN modelling.

Results: The RAG pipeline improved retrieval accuracy and answer relevance. Recursive chunking with the fine-tuned embedding model GTE-Large achieved the highest retrieval accuracy score (0.9). Query expansion and Hyde optimisation enhanced retrieval accuracy for semantic chunking (0.75 to 0.85). Responses maintained high faithfulness ( $\ge$ 0.9). However, the LLM occasionally failed to adhere to predefined causal structures and medical idioms. All clinicians, regardless of BN experience, created comprehensive models within one hour. Experienced clinicians produced more complex models, but occasionally introduced causality errors, while less experienced users adhered more accurately to predefined structures. The tool reduced cognitive workload (2/7 NASA-TLX) and was described as intuitive, although workflow interruptions and minor technical issues highlighted areas for improvement.

Conclusion: Integrating LLM-RAG into BN modelling enhances efficiency and accuracy. Future work may focus on automated preprocessing, refinements of the user interface, and extending the RAG pipeline with validation steps and external biomedical sources. Generative AI holds promise for expert-driven knowledge modelling.

Purpose: Epilepsy surgery is a potential curative treatment for people with focal epilepsy. Intraoperative electrocorticogram (ioECoG) recordings from the brain guide neurosurgeons during resection. Accurate localization of epileptic activity and thus the ioECoG grids is critical for successful outcomes. We aim to develop and evaluate the feasibility of a novel method for localizing small, deformable objects using augmented reality (AR) head-mounted displays (HMDs) and artificial intelligence (AI). AR HMDs combine cameras and patient overlay visualization in a compact design.

Methods: We developed an image processing method for the HoloLens 2 to localize a 64-electrode ioECoG grid even when individual electrodes are indistinguishable due to low resolution. The method combines object detection, super-resolution, and pose estimation AI models with stereo triangulation. A synthetic dataset of 90,000 images trained the super-resolution and pose estimation models. The system was tested in a controlled environment against an optical tracker as ground truth. Accuracy was evaluated at distances between 40 and 90 cm.

Results: The system achieved sub-5 mm accuracy in localizing the ioECoG grid at distances shorter than 60 cm. At 40 cm, the accuracy remained below 2 mm, with an average standard deviation of less than 0.5 mm. At 60 cm the method processed on average 24 stereo frames per second.

Conclusion: This study demonstrates the feasibility of localizing small, deformable objects like ioECoG grids using AR HMDs. While results indicate clinically acceptable accuracy, further research is needed to validate the method in clinical environments and assess its impact on surgical precision and outcomes.