Castellanos Robles, D. | Search bioRxiv

bioRxiv2025

Photoacoustics-guided Real-Time Closed-loop Control of Magnetic Microrobots through Deep Learning

Medical microrobots promise to increase the efficacy and reduce the invasiveness of certain medical procedures in the future. Real-time tracking of the microrobot, actuation, and closed-loop control of its position under in vivo conditions is crucial to fulfill the task at hand. We present a system for closed-loop control of magnetic microrobots using dual-mode ultrasound and photoacoustic imaging. It employs GPU-accelerated beamforming and tracking to achieve real-time operation with a closed-loop cycle time of 100 ms. Artifacts from simultaneous imaging and magnetic actuation are suppressed through time-multiplexing. To address the challenge of detecting microrobots in low-contrast, strong-background images, we implemented real-time Deep Learning-based tracking. A custom dataset of various types of microrobots is curated from long-duration closed-loop control measurements and employed to fine-tune a pre-trained detection model. We introduce a platform for real-time closed-loop control of microrobots and demonstrate its performance with a 300 m spiral-shaped microrobot following a figure-of-8 shape under photoacoustic imaging guidance. The localization error is evaluated against an optical reference measurement. Our results show that photoacoustic-based tracking significantly outperforms ultrasound tracking, with the deep learning approach further reducing missed detections. This demonstrates the algorithms ability to generalize to a previously unseen type of microrobot. We envision this platform to advance medical microrobotics research by providing real-time closed-loop control of untethered microrobots under deep tissue.

bioengineering↗

bioRxiv2025

Influence of hyperparameters on the performance of deep learning-based microrobotic localization under phantom tissue

For the effective operation of medical microrobots within living organisms and precise targeting, it is imperative to employ imaging techniques closely integrated with real-time deep-tissue tracking methods. However, due to a typically low Signal-to-Noise ratio images with strong background, it is hard for traditional tracking methods to achieve sufficient accuracy. This challenge can be addressed by deep learning-based tracking with a real-time detection model. However, a multitude of design choices and Hyperparameters influence the performance. In this study we compared the influence of the hyperparameters and model architecture versions of the "you only look once" (YOLO) network. We use experimental data from a magnetic microrobot imaged with Photoacoustics through 5 mm phantom tissue to evaluate the tracking in comparison with an optical reference. The deep-learning based methods consistently achieved lower missing-detection ratios. Regarding the Root Mean Square localization error, we observed that increasing the weight of the box loss function and utilizing the distribution focal loss can enhance the performance by 10%. Furthermore, it can be seen that YOLOv9 consistently outperformed its predecessor YOLOv8. This study quantifies the robustness of deep-learning based tracking of medical microrobots under tissues.

bioengineering↗